homepage

There is a problem with unittest discovering with namespace packages.
Given the following directory structure, the following command fails with the errors.
# Directory Structure
```
/home/hagihara/test.cybozu/infra/forest/lib/python3/
 cybozu/ # <- namespace package
 cmdb/
 __init__.py
 test.py
```
# Commands
```
# setup module path
echo /home/hagihara/test.cybozu/infra/forest/lib/python3 > sudo dd of=/usr/lib/python3/dist-packages/cybozu.pth
cd /home/hagihara/test.cybozu/infra
python3 -m unittest discover -s cybozu
```
# Errors
```
$ python3 -m unittest discover -s cybozu
Traceback (most recent call last):
 File "/usr/lib/python3.4/unittest/loader.py", line 221, in discover
 os.path.dirname((the_module.__file__)))
AttributeError: 'module' object has no attribute '__file__'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
 File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
 "__main__", mod_spec)
 File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
 exec(code, run_globals)
 File "/usr/lib/python3.4/unittest/__main__.py", line 18, in <module>
 main(module=None)
 File "/usr/lib/python3.4/unittest/main.py", line 92, in __init__
 self.parseArgs(argv)
 File "/usr/lib/python3.4/unittest/main.py", line 116, in parseArgs
 self._do_discovery(argv[2:])
 File "/usr/lib/python3.4/unittest/main.py", line 225, in _do_discovery
 self.test = loader.discover(self.start, self.pattern, self.top)
 File "/usr/lib/python3.4/unittest/loader.py", line 242, in discover
 namespace=True))
 File "/usr/lib/python3.4/unittest/loader.py", line 349, in _find_tests
 namespace=namespace)
 File "/usr/lib/python3.4/unittest/loader.py", line 310, in _find_tests
 name = self._get_name_from_path(full_path)
 File "/usr/lib/python3.4/unittest/loader.py", line 284, in _get_name_from_path
 assert not _relpath.startswith('..'), "Path must be within the project"
AssertionError: Path must be within the project
```
# Cause of the error
This error happens because TestLoader.discover does not set `self._top_level_dir` properly.
`/usr/lib/python3.4/unittest/loader.py`
```
class TestLoader(object):
 ...
 def discover(self, start_dir, pattern='test*.py', top_level_dir=None):
 ...
 if os.path.isdir(os.path.abspath(start_dir)):
 ...
 else:
 # support for discovery from dotted module names
 try:
 __import__(start_dir)
 except ImportError:
 is_not_importable = True
 else:
 try:
 start_dir = os.path.abspath(
 os.path.dirname((the_module.__file__)))
 except AttributeError:
 # look for namespace packages
 ...
 if spec and spec.loader is None:
 if spec.submodule_search_locations is not None:
 is_namespace = True
 for path in the_module.__path__:
 if (not set_implicit_top and
 not path.startswith(top_level_dir)):
 continue
 self._top_level_dir = \ <--- cause of the error.
 (path.split(the_module.__name__
 .replace(".", os.path.sep))[0])
```
If path of the module contains the string same as the module name,
the path is split at incorrect position and invalid value is set to `self._top_level_dir`.
```
path.split(the_module.__name__.replace(".", os.path.sep))[0]
```
the_module -> cybozu
cybozu.__path__ -> ['/home/hagihara/test.cybozu/infra/forest/lib/python3/cybozu']
path.split('cybozu')[0] -> '/home/hagihara/test.'

Just in case, please take a look at issues #17457 and #23882 to see if they already cover the bug.

Thanks for suggestion. I checked both issues and found that this bug is created in #17457 and #23882 does not fix it.

If you need a test case, try https://gitlab.com/warsaw/flufl.lock 

I think this bug actually is fixed now. I've tried my previously failing case in Python 3.5 and 3.6 head and I don't get the error.