homepage

With the following package directory structure--
foo/
 __init__.py
 __main__.py
 from foo import bar
 bar.py
 print('***')
 raise ImportError('test...')
Running--
$ ./python.exe -m foo
Yields--
***
Traceback (most recent call last):
 File ".../python/cpython/cpython/Lib/runpy.py", line 162, in _run_module_as_main
 "__main__", fname, loader, pkg_name)
 File ".../python/cpython/cpython/Lib/runpy.py", line 75, in _run_code
 exec(code, run_globals)
 File "./foo/__main__.py", line 1, in <module>
 from foo import bar
ImportError: cannot import name bar
$
The exception text gets swallowed.
Changing the relative import "from foo import bar" to "import foo.bar" does show the exception text.

Looks very similar to issue15111.

I randomly ran into this issue again. I'm not sure this was ever resolved (i.e. I think it may always have been different from issue 15111).
I still get the above behavior in the default branch.
And here is what I get in the 3.2 branch (the error information is not swallowed):
 
$ ./python.exe -m foo
***
Traceback (most recent call last):
 File ".../Lib/runpy.py", line 161, in _run_module_as_main
 "__main__", fname, loader, pkg_name)
 File ".../Lib/runpy.py", line 74, in _run_code
 exec(code, run_globals)
 File ".../foo/__main__.py", line 1, in <module>
 from foo import bar
 File "foo/bar.py", line 2, in <module>
 raise ImportError('test...')
ImportError: test...

I ran into this again because an error while running `./python.exe -m test` was getting masked. The use of __main__.py in the package may be the distinguishing characteristic.

Should this issue be fixed before the release? If it is not fixed, certain problems found after the release may become harder to report and diagnose (because the true source of error will be masked).
Two months ago issue 15111 which was thought to be the same was given a priority of "high".

It has nothing to do with runpy and __main__.py and everything to do with rev 78619:0d52f125dd32 (which fixed issue #15715) which was done to ignore bogus names in fromlist. If you simply change the import line to "import foo.bar" then you get the expected result.
The options I see to solve this without invalidating issue #15715 is:
1) Parse the ImportError message to notice that it was a lookup failure and not some other ImportError (ewww)
2) Importlib has to tag every exception is raises because of a finder failure to know when an ImportError was triggered because the module wasn't found (less eww, but still brittle as it means only importlib or people aware of the flag will have the expected semantics)
3) Create a ModuleNotFoundError exception that subclasses ImportError and catch that (breaks doctests and introduces a new exception that people will need to be aware of, plus the question of whether it should just exist in importlib or be a builtin)
4) change importlib._bootstrap._handle_fromlist (and various other bits of importlib code) to call _find_module() and silently ignore when no finder is found as desired (probably need to add a flag to _find_and_load() to signal whether finder failure just returns None or raises ImportError)
While I prefer 3, I think it's a bit late in the release to try to introduce a new exception to begin separating the meaning of 16 different ``raise ImportError`` cases in importlib._bootstrap based on inheritance. My gut says 4 is the best solution given the timeframe. I should be able to get a patch in on Friday if that works for people.

Here is a formal unit test case that passes in 3.2 but not in 3.3 (a "simpler" case not using __main__.py).
(script_helper.create_empty_file() doesn't seem to be available in 3.2.)

Thanks for the test, Chris. It can probably be simplified using the utilities in test_importlib (e.g. managing the cleanup of sys.path, using mocked loaders to raise the exception instead of having to write to the file system, etc.), but otherwise the idea of the test is accurate.

You're welcome, Brett. I'll let you or someone else recast the test using the latest preferred techniques. I was just using the style of the immediately surrounding tests.

> While I prefer 3, I think it's a bit late in the release to try to
> introduce a new exception to begin separating the meaning of 16
> different ``raise ImportError`` cases in importlib._bootstrap based on
> inheritance. My gut says 4 is the best solution given the timeframe.
Agreed. I still think option 3 would be suitable and have created issue 15767 to track it.

I don't agree that this is a blocker; would be nice to fix it, of course.

It will get fixed today.

I am running the test suite now using the "secret" attribute on ImportError. I tried to pass a flag, but locking became a bit messy/complicated. And I also realized that if I didn't do this then using different implementation of import_ in importlib wouldn't work because I would be hard-coding in what import implementation was used which would bypass the accelerated code in import.c. Plus with the compatibility issues I have had in the passed, I didn't want to skip a step of import that someone was probably relying on.
Once Python 3.4 comes out I will create a new exception that is raised when no module is found and catch that class specifically to avoid this hack.

New changeset ca4bf8e10bc0 by Brett Cannon in branch 'default':
Issue #15316: Let exceptions raised during imports triggered by the
http://hg.python.org/cpython/rev/ca4bf8e10bc0