homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: multiprocessing adds built-in types to the global copyreg.dispatch_table
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.0, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: jnoller Nosy List: alexandre.vassalotti, jnoller, roudkerk
Priority: normal Keywords:

Created on 2008年07月13日 18:20 by alexandre.vassalotti, last changed 2022年04月11日 14:56 by admin. This issue is now closed.

Messages (3)
msg69615 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008年07月13日 18:20
The multiprocessing module modifies the global copyreg.dispatch_table to
extend pickling to several built-in types (mostly callable types). In my
humble opinion, this is unacceptable for a standard library module.
Here is an example of a behaviour change made by multiprocessing:
 >>> import pickle
 >>> pickle.dumps(str.isalpha)
 Traceback (most recent call last):
 ...
 _pickle.PicklingError: Can't pickle <class 'method_descriptor'>:
attribute lookup builtins.method_descriptor failed
 >>> import multiprocessing.util
 >>> pickle.dumps(str.isalpha)
 
b'\x80\x03cbuiltins\ngetattr\nq\x00cbuiltins\nstr\nq\x01X\x07\x00\x00\x00isalphaq\x02\x86q\x03Rq\x04.'
There was some discussion in issue 558238 about allowing methods to be
pickled. However, no consensus was reached.
In addition, there is a bug in the method pickling support added by
multiprocessing in Python 3.0:
 Python 2.6b1+ (trunk:64899:64900, Jul 13 2008, 13:33:02) 
 [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import multiprocessing.util, pickle
 >>> class A(object):
 ... def foo(self): pass
 ... 
 >>> pickle.dumps(A().foo, 2)
 
'\x80\x02c__builtin__\ngetattr\nq\x00c__main__\nA\nq\x01)\x81q\x02}q\x03bU\x03fooq\x04\x86q\x05Rq\x06.'
 >>> pickle.dumps(A.foo, 2)
 
'\x80\x02c__builtin__\ngetattr\nq\x00c__main__\nA\nq\x01U\x03fooq\x02\x86q\x03Rq\x04.'
 
 Python 3.0b1+ (py3k:64876M, Jul 11 2008, 12:20:51) 
 [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import multiprocessing.util, pickle
 >>> class A(object):
 ... def foo(self): pass
 ... 
 >>> pickle.dumps(A().foo, 2)
 Traceback (most recent call last):
 ...
 pickle.PicklingError: Can't pickle <class 'method'>: it's not found as
builtins.method
 >>> pickle.dumps(A.foo, 2)
 Traceback (most recent call last):
 ...
 pickle.PicklingError: Can't pickle <function foo at 0xb7b1392c>: it's
not found as __main__.foo
A better solution for the interprocess communication needs of
multiprocessing would be to define custom subclass of Pickler with
extended support for built-in types. If needed, I am willing to extend
the _pickle module in Python 3.0 to support modification to its
"dispatch table", like the undocumented dispatch_table attribute in
pickle.Pickler.
msg70423 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2008年07月30日 11:50
Alexandre - can you take a look at the solution for issue3125 and tell me 
if this satisfies your concerns? Note that the merge-forward is blocked in 
py3k by issue3385 (which is assigned to you)
msg80520 - (view) Author: Jesse Noller (jnoller) * (Python committer) Date: 2009年01月25日 18:34
Alexandre, this behavior no longer occurs in trunk, and a custom 
ForkingPickler was added as part of issue 3125
I am going to close this as fixed, but please reopen if there is continued 
undesired behavior.
History
Date User Action Args
2022年04月11日 14:56:36adminsetgithub: 47600
2009年01月25日 18:34:49jnollersetstatus: open -> closed
resolution: fixed
messages: + msg80520
2008年07月30日 11:50:21jnollersetmessages: + msg70423
2008年07月15日 13:38:11jnollersetnosy: + roudkerk
2008年07月13日 18:23:46benjamin.petersonsetassignee: jnoller
nosy: + jnoller
2008年07月13日 18:20:41alexandre.vassalotticreate

AltStyle によって変換されたページ (->オリジナル) /