Separating namespaces from files in Python

Question

I'm writing a Python library in which various objects are represented as Python classes. However, the user is not supposed to create instances of those classes directly. Instead, there are a set of convenience functions to create them. So my module source currently looks like this:

class Foo:
 pass
class Bar:
 pass
def create_object(x):
 # ... either return a Foo or a Bar,
 # depending on the value of x.

The problem is, this doesn't separate the interface from the implementation very nicely. What I want is to hide the classes away a bit. Not hide them completely, but just make it a bit more obvious that their __init__ functions aren't part of the public interface, even though their methods are.

From the user perspective, it seems it would be logical for the classes to be in a separate module. So you call myModule.make_object(10) and get an object of type myModule.objects.Foo.

However, from a readability / maintainability perspective this would cause me a headache, because module namespaces are tied to the files they're in. I'd have to keep cross-referencing between two files, one containing the classes and the other containing the functions that create them. This wouldn't be a logical structure for my project, and I want to avoid it.

Edit: to be clear, I'm not talking about cyclic dependencies - the classes Foo and Bar don't call create_object. I just mean that if the factory functions are in one file and the classes are in another, then in order to understand how my code works, future-me will have to first look up create_object in one file, and then find the constructors it calls in a different file. There are many classes and many factory functions, so this would have to be done a lot. I find this sort of jumping back and forth taxes my short-term memory unnecessarily, and generally makes things harder to understand. I'd prefer to have a single file, containing square and rectangle classes, followed by functions that create squares and rectangles, followed by circle and ellipse classes, followed by functions that create those, etc.

So I guess my questions are:

(1) Is there a way to create the objects module without having a separate file for it? I know I can create a module by instantiating types.ModuleType, but I can't work out how to declare the classes to be in the new module.

(2) Assuming I can solve (1), would that actually be a sensible thing to do?

(3) Or am I thinking about this the wrong way somehow? What would be the standard way to solve this issue in Python?

• Benjamin: Have you considered using the __all__atrribute?
• N. Virgo: @Benjamin as I understand it, that has an effect if the user does from myModule import *, but not if they do import myModule. Because of this, it doesn't seem ideal to me, for the purpose of making clear what's part of the API.

Answer 1

I don't think it's sensible to separate the classes from builder if this causes cross-references (and probably cyclic imports to solve). If your goal is simply to expose what is meant to be public or private there are better ways to explicit that.

The easiest I can think of is to have your module structured as follow :

mylib
|-- __init__.py
|-- objects.py

Where objects.py is exactly what you already have and __init__.py simply does:

from .objects import create_object

So if a user imports mylib the only available symbols in it will be the public API.

If there are many functions like create_object following naming convention, you can automate their import into __init__.py by including something like the following in __init__.py:

import objects
import inspect
import sys
current_module = sys.modules[__name__]
create_functions = [(name, func) for (name, func) in inspect.getmembers(objects, inspect.isfunction)
 if name.startswith('create_')]
for name, func in create_functions:
 setattr(current_module, name, func)

• N. Virgo: That makes sense. The violation of DRY is slightly annoying (Every time I add a function I have to make sure it's also added in __init__.py) but apart from that it achieves exactly what I was trying to achieve.

Answer 2

A straightforward way would be to declare _Foo and _Bar classes.

You mark them as private, the intent is for them to be used inside the module but not outside it.

• N. Virgo: I considered this, but the issue is that the user does use the classes, they just don't create them. So if they call create_object they will get an object of type _Foo rather than Foo, which looks a little untidy, in my opinion.