homepage

With attached teestmodule.zip and svn revision 67006,
$ ./python.exe testmodule.zip 
Traceback (most recent call last):
 File "/Users/sasha/Work/python-svn/trunk/Lib/runpy.py", line 121, in 
_run_module_as_main
 "__main__", fname, loader, pkg_name)
 File "/Users/sasha/Work/python-svn/trunk/Lib/runpy.py", line 34, in 
_run_code
 exec code in run_globals
 File "testmodule.zip/__main__.py", line 7, in <module>
 File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 1817, 
in testmod
 for test in finder.find(m, name, globs=globs, 
extraglobs=extraglobs):
 File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 842, in 
find
 self._find(tests, obj, name, module, source_lines, globs, {})
 File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 884, in 
_find
 test = self._get_test(obj, name, module, globs, source_lines)
 File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 965, in 
_get_test
 if filename[-4:] in (".pyc", ".pyo"):
TypeError: 'NoneType' object is unsubscriptable

Attached patch addresses the crash, but two issues remain:
$ ./python.exe testmodule.zip 
**********************************************************************
File "__main__", line ?, in __main__.c
Failed example:
 'line 2'
Expected nothing
Got:
 'line 2'
**********************************************************************
1 items had failures:
 1 of 1 in __main__.c
***Test Failed*** 1 failures.
The line number for the failed example cannot be found and file name is 
reported as __main__.c instead of __main__.py

The next patch, doctest-1.patch addresses the line number issue, but the 
file name is still wrong.
$ ./python.exe testmodule.zip 
**********************************************************************
File "__main__", line 2, in __main__.c
Failed example:
 'line 2'
Expected nothing
Got:
 'line 2'
**********************************************************************
1 items had failures:
 1 of 1 in __main__.c
***Test Failed*** 1 failures.

I don't think there is an easy way to fix source file name reporting 
without injecting __file__ = '<path to archive>/<filename>' into the 
module namespace is zipimporter, but I don't know if any code relies on 
__file__ set to None.
Setting __file__ to (in the test case) testmodule.zip/__main__.py will 
eliminate the need for the first patch and the second patch will only need 
to pass globs to linecache.getlines() to fix the line number issue.

runpy needs a non-standard PEP 302 extension to set __file__ correctly
in the modules it runs. The pkgutil stuff it uses to find pure Python
modules in the filesystem supports that extension, but zipimport doesn't
(yet).

I created issue4512 to implement get_filename extension in zipimport. 
Will post a patch shortly.

While the patch at issue4512 fixes the crash, it does not fix the line 
number issue. I would prefer to start with doctest-1.patch like solution 
because changing the PEP 302 loader protocol by adding get_filename needs 
additional discussion and certainly is not a candidate for 2.5.x or 2.6.x 
bug fix releases.

The precise solution still seems to be debated, so this is out of scope
for 2.5.3.

Fixed for 2.7 in r67750.
Will be ported to 2.6, 3.0 and 3.1.

Final revisions for fix:
2.7 = r67751 (there was a new test file missing from the initial checkin)
2.6 = r67752
3.1 = r67753
3.0 = r67754 

Nick,
Your commit does not fix the line number reporting from doctest failures:
$ ./python.exe testmodule.zip
**********************************************************************
File "testmodule.zip/__main__.py", line ?, in __main__.c
...
Note the '?' in the output. Did you miss the doctest.py changes in
doctest-1.patch, or should I open a separate issue for the line number
bug?
There is also a problem of __main__.c which my patch does not address,
but it is less important than the line number.
On Sun, Dec 14, 2008 at 7:11 AM, Nick Coghlan <report@bugs.python.org> wrote:
>
> Nick Coghlan <ncoghlan@gmail.com> added the comment:
>
> Final revisions for fix:
> 2.7 = r67751 (there was a new test file missing from the initial checkin)
> 2.6 = r67752
> 3.1 = r67753
> 3.0 = r67754
>
> ----------
> resolution: -> fixed
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue4197>
> _______________________________________
>

Re-opening, pending development of a fully passing test specifically for
the __main__.py in zipfile case.

I have a test case locally that demonstrates both the missing line
number in the __main__.py from a zipfile case, as well as the display of
main.c as the filename (the latter is not specific to the zipfile case -
it also happens for a normal main module).
It still seems to me that there is something going wrong at a deeper
level for doctest to be getting the "in main.c" part wrong. I want to
get to the bottom of that before working around the problem at the line
number lookup stage.

It turns out you were pretty close to pinpointing the problem in
doctest, but didn't quite manage to identify which step was going wrong.
The problem was actually that even after __file__ was being set
correctly, the call to linecache.getlines wasn't being given the
__loader__ access it needed in order to retrieve the source code from
inside the zipfile.
The __main__.c thing turned out just to be a red herring caused by the
choice of class name in the test code (I changed it to be __main__.Test
for the checked in unit test).
Checkin versions:
2.7 = r67790
2.6 = r67791
3.1 = r67792
3.0 = r67793