homepage

Created to record the egregious hack of relying on the test_runpy infrastructure in order to test pkgutil.walk_packages.
It gets the job done, but is a really messy way of going about it. Worth cleaning up by factoring the support code out to a helper module when there isn't a release deadline looming.

Also, a second test case should be added to cover the zipimporter.zipimporter component.

Such helper functionality could also be used in the tests of unittest.TestLoader.loadTestsFromName(). See, for example, the tests proposed for issue 7559.

The walk_packages tests should also be enhanced to ensure correctly handling of namespace packages

Note that the problem described in #14787 will limit the scope of what can be tested.

I'd like to work on this if that's okay. I've already written test cases for issue 7559 and issue 15299 that required creating temporary packages, so I understand the need. And I'd also like to create test cases for issue 14787, which should really be done after this.
I think this should be split into two parts though: (1) refactor the create-package code in test_runpy into a separate helper module, and (2) move the refactored walk_package tests from test_runpy into test_pkgutil (so it's easier to see what has changed).
I can create a new issue for (1).

That sounds good, and thanks for working on this - if we can get the tests cleaned up for the 3.3 release, that will make ongoing maintenance a lot easier.

Has anyone made any progress with this issue or others referenced like #7559 or #14787 ? Regardless I'd like to help out directly if possible as I'm suffering from an acute case of triagitis :-)

Dropped down my priority list (it's a nice to have to make the code base a
bit easier to work on, rather than something essential).

Hardly a good start :-( From the IDLE shell.
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:38:22) [MSC v.1600 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> from pkgutil import walk_packages
>>> packages = list(walk_packages())
Traceback (most recent call last):
 File "<pyshell#1>", line 1, in <module>
 packages = list(walk_packages())
 File "C:\Python34\lib\pkgutil.py", line 86, in walk_packages
 __import__(name)
 File "C:\Python34\lib\site-packages\pip\__init__.py", line 9, in <module>
 from pip.log import logger
 File "C:\Python34\lib\site-packages\pip\log.py", line 8, in <module>
 from pip import backwardcompat
 File "C:\Python34\lib\site-packages\pip\backwardcompat\__init__.py", line 27, in <module>
 console_encoding = sys.__stdout__.encoding
AttributeError: 'NoneType' object has no attribute 'encoding'
Then from the command line.
C:\Users\Mark\MyPython>type wptest.py
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from pkgutil import walk_packages
packages = list(walk_packages())
C:\Users\Mark\MyPython>wptest.py
c:\python34\lib\site-packages\wx\core.py:22: UserWarning: wxPython/wxWidgets release number mismatch
 warnings.warn("wxPython/wxWidgets release number mismatch")
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
C:\Users\Mark\MyPython\wptest.py in <module>()
 4 from pkgutil import walk_packages
 5
----> 6 packages = list(walk_packages())
c:\python34\lib\pkgutil.py in walk_packages(path, prefix, onerror)
 99 path = [p for p in path if not seen(p)]
 100
--> 101 yield from walk_packages(path, name+'.', onerror)
 102
 103
c:\python34\lib\pkgutil.py in walk_packages(path, prefix, onerror)
 99 path = [p for p in path if not seen(p)]
 100
--> 101 yield from walk_packages(path, name+'.', onerror)
 102
 103
c:\python34\lib\pkgutil.py in walk_packages(path, prefix, onerror)
 99 path = [p for p in path if not seen(p)]
 100
--> 101 yield from walk_packages(path, name+'.', onerror)
 102
 103
c:\python34\lib\pkgutil.py in walk_packages(path, prefix, onerror)
 99 path = [p for p in path if not seen(p)]
 100
--> 101 yield from walk_packages(path, name+'.', onerror)
 102
 103
c:\python34\lib\pkgutil.py in walk_packages(path, prefix, onerror)
 84 if ispkg:
 85 try:
---> 86 __import__(name)
 87 except ImportError:
 88 if onerror is not None:
c:\python34\lib\site-packages\wx\lib\pubsub\core\arg1\__init__.py in <module>()
 14
 15 msg = 'Should not import this directly, used by pubsub.core if applicable'
---> 16 raise RuntimeError(msg)
RuntimeError: Should not import this directly, used by pubsub.core if applicable
But then.
C:\Users\Mark\MyPython>cd c:\cpython\pcbuild
c:\cpython\PCbuild>python_d.exe C:\Users\Mark\MyPython\wptest.py
c:\cpython\PCbuild>
No problems above!!!
Given Brett's comment in msg165456 "But if we do this then I want to deprecate pkgutil in Python 3.4" I don't believe that this is worth pursuing, or have I missed something?

I got overruled on the deprecation idea due to utility code like walk_packages that pkgutil contains.