homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing: freezing apps on Windows
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: davin, jnoller, sbt, sgm, spongebob
Priority: normal Keywords:

Created on 2009年07月11日 06:08 by sgm, last changed 2022年04月11日 14:56 by admin. This issue is now closed.

Messages (10)
msg90405 - (view) Author: Stuart Mentzer (sgm) Date: 2009年07月11日 06:08
Freezing apps with multiprocessing on Windows seems to be broken.
First, in get_command_line in multiprocessing/forking.py I find that
this code:
 if getattr(sys, 'frozen', False):
 return [sys.executable, '--multiprocessing-fork']
 else:
 prog = 'from multiprocessing.forking import main; main()'
 return [_python_exe, '-c', prog, '--multiprocessing-fork']
should be:
 elif getattr(sys, 'frozen', False) and not WINEXE:
 return [sys.executable, '--multiprocessing-fork']
 else:
 prog = 'from multiprocessing.forking import main; main()'
 return [_python_exe, '-c', prog, '--multiprocessing-fork']
in order for the _python_exe set with multiprocessing.set_executable to
be used rather than your app's exe.
Second, I can then get a working "frozen" package if I include
pythonw.exe (and use set_executable to point to it) and a subset of
Python's Lib directory that my process needs to call. If this is as
intended then it needs to be documented. This may just be a flaw in py2exe.
Third, the multiprocessing documentation page description for
set_executable has example code with the older setExecutable call.
msg90646 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009年07月17日 22:49
Thanks for noticing this. Unfortunately, I don't know enough about 
py2exe/windows to dig into that part. I can fix the low hanging fruit 
though
msg90690 - (view) Author: Stuart Mentzer (sgm) Date: 2009年07月18日 21:13
Thanks Jesse. If you make broader changes that my patch I am happy to
test on Windows and with py2exe if that is helpful.
I will try to get the py2exe folks to look at the broader
multiprocessing issues I describe as well. If multiprocessing supports
freezing on Windows then it would be nice if its docs had a short primer
on what you'll actually need to do to get that working with freeze tools.
Stuart
msg90691 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009年07月18日 21:16
Sounds good - I've personally never used freeze on windows, so having some 
docs and a patch to help make it better for those that do is a definite 
plus
msg91416 - (view) Author: Stuart Mentzer (sgm) Date: 2009年08月08日 05:15
Further experimentation revealed that freeze_support() works properly
and the proposed patch is not necessary or desirable for the general
freeze case. In the case of an embedded app that calls set_executable()
it seems that the "else" block calling _python_exe should be taken but
won't be, so I still think there may be a problem here.
In our situation it turned out that an imported module was grabbing the
command line arguments before freeze_support() was called and when that
was addressed the need for a separate python executable and calling
set_executable() was eliminated.
I suggest that the documentation for freeze_support() make it clear that
it must get called before an application processes the arguments or that
the application must allow a --multiprocessing-fork option with a string
value so that the initialization continues until freeze_support() is called.
msg91418 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009年08月08日 13:46
Thanks for following up on this Stuart - I'll update the docs accordingly
msg98154 - (view) Author: spongebob (spongebob) Date: 2010年01月22日 17:38
I can't seem to freeze an app on Windows using multiprocessing.Manager()
Are apps that use Managers freezable??
Example:
from multiprocessing import freeze_support,Manager
if __name__ == '__main__':
 freeze_support()
 m=Manager()
will throw errors, when run:
Process SyncManager-1:
Traceback (most recent call last):
 File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.process"
, line 232, in _bootstrap
 File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.process"
, line 88, in run
 File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.managers
", line 520, in _run_server
IOError: [Errno 6] The handle is invalid
Traceback (most recent call last):
 File "<string>", line 5, in <module>
 File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing", line 9
8, in Manager
 File "multi_exe\build\pyi.win32\multi_exe\outPYZ1.pyz/multiprocessing.managers
", line 499, in start
EOFError
msg189783 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2013年05月21日 20:40
Could someone answer the question posed in msg98154 please.
msg191520 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2013年06月20日 14:40
I just tried freezing the program
 from multiprocessing import freeze_support,Manager
 if __name__ == '__main__':
 freeze_support()
 m=Manager()
 l = m.list([1,2,3])
 l.append(4)
 print(l)
 print(repr(l))
using cx_Freeze with Python 2.7, and it worked fine:
 PS> cxfreeze.bat foo.py
 copying C:\Python27\lib\site-packages\cx_Freeze\bases\Console.exe -> C:\Tmp\dir\dist\foo.exe
 copying C:\Windows\system32\python27.dll -> C:\Tmp\dir\dist\python27.dll
 writing zip file C:\Tmp\dir\dist\foo.exe
 ...
 PS> dist\foo
 [1, 2, 3, 4]
 <ListProxy object, typeid 'list' at 0x206f410>
msg237281 - (view) Author: Davin Potts (davin) * (Python committer) Date: 2015年03月05日 18:03
The original issue now appears addressed in the docs (thanks goes to Stuart and Jesse) though it was not explicitly tracked here as a patch file.
The follow-on secondary issue from spongebob (if that is your real name) could not be reproduced by Richard.
This issue should have been marked closed->resolved some time ago.
History
Date User Action Args
2022年04月11日 14:56:50adminsetgithub: 50710
2015年03月05日 18:03:43davinsetstatus: open -> closed

type: behavior

nosy: + davin
messages: + msg237281
resolution: fixed
stage: resolved
2014年02月03日 17:07:29BreamoreBoysetnosy: - BreamoreBoy
2013年06月20日 14:40:59sbtsetmessages: + msg191520
2013年05月21日 22:22:33sbtsetnosy: + sbt
2013年05月21日 20:40:00BreamoreBoysetnosy: + BreamoreBoy
messages: + msg189783
2010年01月22日 17:38:29spongebobsetnosy: + spongebob
messages: + msg98154
2009年08月08日 13:46:26jnollersetmessages: + msg91418
2009年08月08日 05:15:25sgmsetmessages: + msg91416
2009年07月18日 21:16:14jnollersetmessages: + msg90691
2009年07月18日 21:13:45sgmsetmessages: + msg90690
2009年07月17日 22:49:35jnollersetmessages: + msg90646
2009年07月17日 22:31:54ncoghlansetassignee: jnoller

nosy: + jnoller
2009年07月11日 06:08:26sgmcreate

AltStyle によって変換されたページ (->オリジナル) /