homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_multiprocessing failure (test_pool_worker_lifetime)
Type: behavior Stage: resolved
Components: Library (Lib), Tests Versions: Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: asksol, jnoller, neologix, pitrou, python-dev, vstinner
Priority: normal Keywords:

Created on 2011年04月04日 21:31 by pitrou, last changed 2022年04月11日 14:57 by admin. This issue is now closed.

Messages (6)
msg132974 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011年04月04日 21:31
This sometimes happens on the buildbots:
test test_multiprocessing failed -- Traceback (most recent call last):
 File "D:\cygwin\home\db3l\buildarea3円.x.bolen-windows\build\lib\test\test_multiprocessing.py", line 1191, in test_pool_worker_lifetime
 self.assertNotIn(None, finalworkerpids)
AssertionError: None unexpectedly found in [1788, 3984, None]
e.g. http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/4334/steps/test/logs/stdio 
msg133161 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2011年04月06日 20:11
Does this only happen on Cygwin buildbots ?
If yes, then it might simply be an issue with Cygwin's fork implementation, which is much slower than natively.
Right now, the test waits 0.5s before checking that the processes are started, after repopulating the pool. While 0.5s is normally way enough for forking a couple processes, it seems that under Cygwin this can take a surprising amount of time, see http://old.nabble.com/Slow-fork-issue---Win-x64-td19538601.html and also http://superuser.com/questions/133313/can-i-speed-up-cygwins-fork
Unless I misunderstand their benchmarks, the fork (+exec) rate of "date" from a shell can be as low as 5/sec, so I can only guess what forking cpython would take.
Maybe we could try to increase the timeout before checking the PIDs:
+ countdown = 10
- countdown = 10
 while countdown and not all(w.is_alive() for w in p._pool):
 countdown -= 1
 time.sleep(DELTA)
msg133164 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011年04月06日 20:32
There is no Cygwin buildbot, this is a regular Windows buildbot (despite the path in the traceback).
That said, spawning processes is quite slow under Windows anyway (much slower than under Linux). So we could up the countdown.
msg133166 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011年04月06日 20:54
New changeset c4a514199dba by Antoine Pitrou in branch '3.2':
Issue #11766: increase countdown waiting for a pool of processes to start
http://hg.python.org/cpython/rev/c4a514199dba
New changeset 3eac8302a448 by Antoine Pitrou in branch 'default':
Issue #11766: increase countdown waiting for a pool of processes to start
http://hg.python.org/cpython/rev/3eac8302a448 
msg133167 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011年04月06日 21:01
New changeset 2e4cdaffe493 by Antoine Pitrou in branch '2.7':
Issue #11766: increase countdown waiting for a pool of processes to start
http://hg.python.org/cpython/rev/2e4cdaffe493 
msg133168 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011年04月06日 21:03
Hopefully fixed now, thanks for your diagnosis!
History
Date User Action Args
2022年04月11日 14:57:15adminsetgithub: 55975
2011年04月06日 21:03:06pitrousetstatus: open -> closed
versions: - Python 3.1
messages: + msg133168

resolution: fixed
stage: resolved
2011年04月06日 21:01:40python-devsetmessages: + msg133167
2011年04月06日 20:54:55python-devsetnosy: + python-dev
messages: + msg133166
2011年04月06日 20:32:31pitrousetmessages: + msg133164
2011年04月06日 20:11:35neologixsetnosy: + neologix
messages: + msg133161
2011年04月05日 10:48:58vstinnersetnosy: + vstinner
2011年04月04日 21:31:27pitroucreate

AltStyle によって変換されたページ (->オリジナル) /