Python factory class with dynamic imports

Question

I'm trying to write a factory class that essentially provides a user-friendly frontend to creating objects of different kinds depending on keywords. So the user will just have to import this module, and then use module.factory.make_new with the relevant keywords.

The directory structure will be pretty flat: this file will sit in a directory along with SubclassA.py which defines the class SubclassA likewise for SubclassB etc. That is, each subclass is in its own module of the same name.

If the user wants to dig a bit deeper, they can provide their extension to the code by writing SubclassC.py which provides class SubclassC and then add the line factory.register_format("subclassc", "SubclassC") to the bottom of this file (or, eventually, this will be handled by a config file). Here's what I have so far:

import importlib
class TrackerFactory:
 def __init__(self):
 self.subclass_dict = {}
 def register_format(self, subclass_name, subclass_package):
 fmt = importlib.import_module(subclass_package)
 self.subclass_dict[subclass_name] = getattr(fmt, subclass_package)
 def make_new(self, subclass, **kwargs):
 return self.subclass_dict[subclass](subclass, **kwargs)
factory = TrackerFactory()
factory.register_format("subclassA", "SubclassA")
factory.register_format("subclassB", "SubclassB")

I'm just wondering what the possible downsides of this sort of approach are, and what kind of gotchas I might have to look out for. Is there a better way to do this? I get the feeling I can achieve what I want with class methods and a class variable dictionary, and thus avoid hardcoding an instance of the factory, but I couldn't get it to work...

Some things I'm not bothered by:

• sneaky hidden imports not at the top of the file (the advantages of dynamically loading the dependencies of only the registered formats outweigh the poor style, for me). And in any case, this is only used to load a very specific type to module. The modules themselves being loaded in this way (SubclassA etc) could have lots of unusual dependencies.

Some things I've already thought of:

• extend the register_format method to allow the module name and class name to differ
• use some try/except logic
• since, in my use case, there will actually be only one or a few objects actually instantiated, I could juggle things around so that the import is actually in make_new (and subclass_dict just holds a string rather than the function), and then registering a format you won't use that you don't have the dependencies for wouldn't choke.

• FMc: (1) What project directory structure does this code expect? (2) The import_module() function takes a module name as its first argument, but you are calling it subtype_name. Why are you describing a module as though it were a type? (3) All of which makes me think I don't understand your purpose, and I suspect others could be in the same boat. Perhaps you can edit your question to explain both how to set up an intended use case (directory structure and file names) and what the benefits are.
• Seamus: Thanks for those comments. "type" was just an unfortunate word choice. The directory structure is pretty flat and is explained in the second para now. I am currently relying on the fact that the modules I'm importing this way all contain a class of the same name as the module name, which is maybe a confusing way to do things...
• RootTwo: It sounds a bit like you are trying to make a plug-in system.
• Seamus: Yeah. Or rather, I'm writing this in such a way that it could easily support plugins.
• Peilonrayz: "I also want to allow the user to provide their extension by making SubclassC.py which provides class SubclassC. then add the line factory.register_format("subclassc", "SubclassC") to the bottom of this file." Are you users editing the file? To double check, users will be editing your library's code? If not can you add the feature to your code. Afterwards I think I could give you a helpful review.

Answer 1

If a factory pattern is indeed called for - much of the time it is not - there is an easier way that needs neither importlib nor explicit registration calls.

If you follow something like this layout:

mypackage/
mypackage/__init__.py
mypackage/tracker_factory.py
mypackage/trackers/__init__.py
mypackage/trackers/type_a.py
mypackage/trackers/type_b.py
...

then in mypackage/trackers/__init__.py:

from .type_a import TypeA
from .type_b import TypeB
...

then your factory can import mypackage.trackers as all_trackers, do a dir(all_trackers), and every subtype that you care about will be listed. Instead of a dynamic import plus runtime registration calls, this uses traditional imports and will look up the individual type using getattr() on the module object. The class and module names are automatically able to differ though it would not be a good idea to do so. The subtype modules would still be able to import, as you put it, "unusual dependencies".

If you are concerned about the performance impact of initializing modules that you may not end up using, I think this concern is unwarranted - though of course you haven't shown any code as evidence one way or the other. A properly-written Python module should be fast and safe to load, and should only incur execution expenses when something is called.

• Seamus: Reading through what I wrote, I realise that what I have above doesn't do one of the things I originally wanted to do, which is have different keywords effectively call the same subclass with different options. That's why your approach isn't ideal. I guess I could make each user keyword into a class that is just a subclass with some parameters fixed...
• Reinderien: This approach will not impede the passage of kwargs from your factory method to the constructor.
• RootTwo: How does this accommodate a user adding their own type_c.py? Would they also need to edit the __init__.py to add a from .type_c import TypeC? Perhaps __init__.py could loop over the files in the directory and use importlib.import_module()?
• Seamus: In thinking about your answer, I realised that my code isn't working as intended. I've edited my question to make clear that this sort of solution isn't suitable: 1, it doesn't give me an abstraction of user-friendly keywords for object creation, and 2, all dependencies of all modules need to be satisfied for this code to work.