homepage

I noticed this while checking issue35961. test_venv is always failing on GCC which is marked as optional in Travis.
Log : https://travis-ci.org/python/cpython/jobs/492123436#L1909
0:39:35 load avg: 1.00 [390/416] test_venv
test test_venv failed -- Traceback (most recent call last):
 File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 309, in test_multiprocessing
 out, err = check_output([envpy, '-c',
 File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 37, in check_output
 raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['/tmp/tmpg8ubeyfn/bin/python', '-c', 'from multiprocessing import Pool; print(Pool(1).apply_async("Python".lower).get(3))']' died with <Signals.SIGABRT: 6>.
Also GCC test have been timing out for at least past 4 months : https://python.zulipchat.com/#narrow/stream/116742-core.2Fhelp/topic/GCC.20build.20in.20Travis.20always.20times.20out 

Pablo: Can this issue be related bpo-35378?
commit 3766f18c524c57784eea7c0001602017d2122156
Author: Pablo Galindo <Pablogsal@gmail.com>
Date: Mon Feb 11 17:29:00 2019 +0000
 bpo-35378: Fix multiprocessing.Pool references (GH-11627)

That looks like a similar example of the problem that bpo35378 solves. But the PR is merged and the test keeps failing:
test test_venv failed -- Traceback (most recent call last):
 File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 309, in test_multiprocessing
 out, err = check_output([envpy, '-c',
 File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 37, in check_output
 raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['/tmp/tmplplx90pf/bin/python', '-c', 'from multiprocessing import Pool; print(Pool(1).apply_async("Python".lower).get(3))']' died with <Signals.SIGABRT: 6>.
https://travis-ci.org/python/cpython/jobs/492305033
Running that command locally on master does not die with SIGABRT:
❯ ./python -c 'from multiprocessing import Pool; print(Pool(1).apply_async("Python".lower).get(3))'; echo $?
python
/home/pablogsal/github/cpython/Lib/multiprocessing/pool.py:234: ResourceWarning: unclosed running multiprocessing pool <multiprocessing.pool.Pool state=RUN pool_size=1>
 _warn(f"unclosed running multiprocessing pool {self!r}",
ResourceWarning: Enable tracemalloc to get the object allocation traceback
0

I can reproduce the crash using the same command than the job.
Simplified commands to reproduce the bug:
./configure && make
./python -m venv venv
./venv/bin/python -m test test_venv -m test.test_venv.BasicTest.test_multiprocessing

The Python of the virtual environment created by the test doesn't work:
$ /tmp/tmp0vybyos2/bin/python -c pass
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: initfsencoding: failed to get the Python codec of the filesystem encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f2956ab8740 (most recent call first):
Aborted

Hum, test_executable() of test_venv is skipped if run in a venv, but not test_multiprocessing():
 # If a venv is created from a source build and that venv is used to
 # run the test, the pyvenv.cfg in the venv created in the test will
 # point to the venv being used to run the test, and we lose the link
 # to the source build - so Python can't initialise properly.
 @skipInVenv
 ...
 self.run_with_capture(venv.create, self.env_dir)
 envpy = os.path.join(os.path.realpath(self.env_dir),
 self.bindir, self.exe)
 ...
 def test_multiprocessing(self):
 ...
 self.run_with_capture(venv.create, self.env_dir)
 envpy = os.path.join(os.path.realpath(self.env_dir),
 self.bindir, self.exe)
 ...
The test has been added by:
commit 4e02f8f8b4baab63f927cfd87b401200ba2969e9
Author: Steve Dower <steve.dower@microsoft.com>
Date: Fri Jan 25 14:59:12 2019 -0800

I guess we need a "skipInLinuxVenv", since it works fine on Windows :)

I'm on this. I'll make a new (or possibly complete replacement) for @skipInVenv that is @requireVenvCreate, since that's the common piece that doesn't work in a venv (except on Windows).

New changeset 8bba81fd55873148c65b7d0e6a6effbd63048c76 by Steve Dower in branch 'master':
bpo-35978: Correctly skips venv tests in venvs (GH-12220)
https://github.com/python/cpython/commit/8bba81fd55873148c65b7d0e6a6effbd63048c76

New changeset b0967fe4ed2e0e15f14ea574f82970a3fd4a5556 by Miss Islington (bot) in branch '3.7':
bpo-35978: Correctly skips venv tests in venvs (GH-12220)
https://github.com/python/cpython/commit/b0967fe4ed2e0e15f14ea574f82970a3fd4a5556