How to choose an imported python file by user's command line argument

Question

In a main python file, I import another python files, say their names are file1, file2, file3 and all of them have a function inside them named scrape(). I am trying to choose which file's scrape() will run according to user input, like the following:

python main.py file1

Here is the relevant part of my code:

import file1
import file2
import file3
fileName = sys.argv[1]
for func in ['%s.scrape' % fileName]:
 meta, infos = func()

However, I get this error message:

Traceback (most recent call last):
File "main.py", line 50, in <module>
meta, infos = func()
TypeError: 'str' object is not callable

Note that it works when I use for func in [file1.scrape]: I just can't use user input as the imported file name. Can someone tell me how to do it?

• sahasrara62: Possible duplicate of Import python script with arguments

Answer 1

You are trying to call func as a function, when it's really a string you built from the command-line argument.

For your purposes, as also mentioned in prashant's linked post, you might want to use something like the imp module.

Here's a quick example

import sys
import imp
# `imp.load_source` requires the full path to the module
# This will load the module provided as `user_selection`
# You can then either `import user_selection`, or use the `mod` to access the package internals directly
mod = imp.load_source("user_selection", "/<mypath>/site-packages/pytz/__init__.py")
# I'm using `user_selection` and `mod` instead of `pytz`
import user_selection
print(user_selection.all_timezones)
print(mod.all_timezones)

In your case, you might have to use imp.find_module to get the full path from just the name, or provide the full paths directly in the command line.

This should be a starting point

import sys
import imp
file_name = sys.argv[1]
f, filename, desc = imp.find_module(file_name, ['/path/where/modules/live'])
mod = imp.load_module("selected_module", f, filename, desc)
mod.scrape()