homepage

The need to resolve a dotted name to a Python object is spreading in the stdlib: pydoc has locate and resolve, packaging has util.resolve_name, unittest has something else, etc. For the benefit of stdlib maintainers as well as the community, I think such functionality should be exposed publicly by the inspect module.

To be a little clearer, this is about dotted import names, not regular dotted names.
pydoc.locate(path, forceload=0):
 """Locate an object by name or dotted path, importing as necessary"""
pydoc.resolve(thing, forceload=0):
 """Given an object or a path to an object, get the object and its name.""" 
It is not completely clear to me how this is different from using __import__ but I will believe it is. It something like this is in at least 3 places, +1 on factoring it out to one place.

Sorry if I was unclear; the functions work with a dotted name as a string and resolve it to an object. __import__ works with module, whereas this kind of functions work with mod.name, pkg.mod.name.attr, etc., that is, they combine import and getattr.

In addition, error handling/reporting is not trivial to get right. We’ve had to fix the code in distutils2 and it’s still not quite right (#12703).
I opened this report because I’d like to see all stdlib modules use the same functions and I’d prefer people to copy-paste the same robust code for backports.

New changeset 1405df4a1535 by Éric Araujo in branch 'default':
Expand tests and fix bugs in packaging.util.resolve_name.
http://hg.python.org/cpython/rev/1405df4a1535 

The version in logging.config appears to be doing the same job, but is shorter:
def _resolve(name):
 """Resolve a dotted name to a global object."""
 name = name.split('.')
 used = name.pop(0)
 found = __import__(used)
 for n in name:
 used = used + '.' + n
 try:
 found = getattr(found, n)
 except AttributeError:
 __import__(used)
 found = getattr(found, n)
 return found
The line "used = used + '.' + n" could of course be improved.

New changeset 5df1065ddb8b by Éric Araujo in branch 'default':
Expand tests and fix bugs in util.resolve_name.
http://hg.python.org/distutils2/rev/5df1065ddb8b 

Attaching new patch to address Vinay Sajip's suggestions on Rietveld. Thank you, Vinay.

Something to consider with these functions: it is probably desirable to also support the popular alternate notation which uses an explicit ":" to separate the module name from the reference within the module.
For example, setuptools entry points and nose test references both use that format.
The alternate notation should be fairly easy to both detect and handle through "name.partition(':')"

Can you provide a couple of examples of that notation?

> Can you provide a couple of examples of that notation?
entry_points = """
[console_scripts]
pybabel = babel.messages.frontend:main
[distutils.commands]
compile_catalog = babel.messages.frontend:compile_catalog
extract_messages = babel.messages.frontend:extract_messages
init_catalog = babel.messages.frontend:init_catalog
update_catalog = babel.messages.frontend:update_catalog
[distutils.setup_keywords]
message_extractors = babel.messages.frontend:check_message_extractors
[babel.checkers]
num_plurals = babel.messages.checkers:num_plurals
python_format = babel.messages.checkers:python_format
[babel.extractors]
ignore = babel.messages.extract:extract_nothing
python = babel.messages.extract:extract_python
javascript = babel.messages.extract:extract_javascript
"""
Source:
http://babel.edgewall.org/browser/trunk/setup.py#L38 

Another question to consider: is inspect the best place for this? I don't think it is, because
(a) It's not really an inspection facility
(b) Importing inspect to get this functionality would pull in lots
 of stuff which wouldn't be used in the typical use case.
I think it makes more sense for it to be in importlib. Accordingly adding Brett to nosy, for his thoughts.

importlib.util.resolve_name() already exists for resolving an explicit relative import name to an absolute one, so closing this as out of date.

> importlib.util.resolve_name() already exists
But that's not what the proposed functionality is for, is it? This covers finding values inside an imported module which can be accessed via a dotted path from the module globals.

Importlib already has importlib.import_module() (since Python 2.7) and that's as far as I'm willing to go for finding a module by name. Anything past that is a getarr() call on the resulting module and thus not worth adding to importlib.

> and thus not worth adding to importlib.
Okay, fair enough. It's not purely an import function, though partly related to imports.

With the lack of support, I suggest closing this.

> With the lack of support, I suggest closing this.
Well, the lack of support is just for adding to importlib. It still seems useful to refactor it out ... just not clear where the best place is. Perhaps pkgutil? It's definitely useful to have in the stdlib in one place. I'll come up with a PR soon.

FTR, Django has a similar helper:
 from django.utils.module_loading import import_string
 ValidationError = import_string('django.core.exceptions.ValidationError')
https://docs.djangoproject.com/en/3.0/ref/utils/#django.utils.module_loading.import_string 

New changeset 1ed61617a4a6632905ad6a0b440cd2cafb8b6414 by Vinay Sajip in branch 'master':
bpo-12915: Add pkgutil.resolve_name (GH-18310)
https://github.com/python/cpython/commit/1ed61617a4a6632905ad6a0b440cd2cafb8b6414

The regex looks too strict. Python 3 accepts non-ASCII module names.
Should existing functions be modified to reuse the new function?

> The regex looks too strict. Python 3 accepts non-ASCII module names.
Would adding the re.U flag be enough, or are you thinking of other possibilities?

To be more precise, one could use
^(?!\d)(\w+)(\.(?!\d)(\w+))* for _DOTTED_WORDS and use the re.U flag when compiling the pattern. That should match any packages where segments start with a letter and continue with alphanumerics. Do you agree?

I reopen the issue to discuss the non-ASCII identifiers.
Currently, the code uses [a-z_]\w*, but "é" is a valid Python module name for example:
$ echo 'print("here")' > é.py
$ python3 -c 'import é'
here
I'm not sure how to design the regex. I just reported a potential issue ;-) See the PEP 3131 and str.isidentifier() method:
https://docs.python.org/dev/library/stdtypes.html#str.isidentifier 

> I'm not sure how to design the regex.
Did you not look at the PR I added to address this (Unicode chars), with a regex change? I added you as a reviewer.

New changeset 4f17c5cd9a1ec50fe8de7ef68c39220a01a862cb by Vinay Sajip in branch 'master':
bpo-12915: Improve Unicode support for package names and attributes. (GH-18517)
https://github.com/python/cpython/commit/4f17c5cd9a1ec50fe8de7ef68c39220a01a862cb

PR18517 has, likely, a utf-8 dependency. AIX, default latin-1 does not accept the new test.
Starting with this merge AIX bot fails with:
======================================================================
ERROR: test_name_resolution (test.test_pkgutil.PkgutilTests)
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/test/test_pkgutil.py", line 249, in test_name_resolution
 os.makedirs(d, exist_ok=True)
 File "/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/os.py", line 223, in makedirs
 mkdir(name, mode)
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 17-19: ordinal not in range(256)
Not completely resolved.

> File "/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/test/test_pkgutil.py", line 249, in test_name_resolution
> os.makedirs(d, exist_ok=True)
(...)
> UnicodeEncodeError: 'latin-1' codec can't encode characters in position 17-19: ordinal not in range(256)
The test should be modified to skip the current uw value in the loop ("continue") if the filesystem encoding cannot encode the Unicode string (catch UnicodeEncodeError on the makedirs() call).

I am very busy with normal work this week. However I’ll attempt to add a pr with your (Victor"s) suggestion. 
Sent from my iPhone
> On 29 Feb 2020, at 23:36, STINNER Victor <report@bugs.python.org> wrote:
> 
> 
> STINNER Victor <vstinner@python.org> added the comment:
> 
>> File "/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/test/test_pkgutil.py", line 249, in test_name_resolution
>> os.makedirs(d, exist_ok=True)
> (...)
>> UnicodeEncodeError: 'latin-1' codec can't encode characters in position 17-19: ordinal not in range(256)
> 
> The test should be modified to skip the current uw value in the loop ("continue") if the filesystem encoding cannot encode the Unicode string (catch UnicodeEncodeError on the makedirs() call).
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue12915>
> _______________________________________
>

New changeset e0acec15412203359d59db33e447698ee51e07fc by Michael Felt in branch 'master':
bpo-12915: Skip test_pkgutil.test_name_resolution() non-encodable filenames (GH-18720)
https://github.com/python/cpython/commit/e0acec15412203359d59db33e447698ee51e07fc

I understand that all known issues are now fixed, so I close the issue.

See bpo-41154 for possible regression introduced by these changes.