homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: private dispatch table for picklers
Type: enhancement Stage: resolved
Components: Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: alexandre.vassalotti, pitrou, python-dev, sbt
Priority: normal Keywords: patch

Created on 2012年03月01日 14:34 by sbt, last changed 2022年04月11日 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
pickle_dispatch.patch sbt, 2012年03月01日 14:34
pickle_dispatch.patch sbt, 2012年03月04日 14:51 review
Messages (6)
msg154695 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012年03月01日 14:34
Currently the only documented way to have customised pickling for a type is to register a reduction function with the global dispatch table managed by the copyreg module. But such global changes are liable to disrupt other code which uses pickling.
Multiprocessing deals with this by defining a ForkingPickler class which subclasses the pure python _Pickler class (using undocumented features), and supports registering reduction functions specifically for that class.
I would like to see some documented alternative which works with both C and Python implementations. At least then multiprocessing can avoid using slow pure python pickling. 
The attached patch allows a pickler object to have a private dispatch table which it uses *instead* of the global one. It lets one write code like
 p = pickle.Pickler(...)
 p.dispatch_table = copyreg.dispatch_table.copy()
 p.dispatch_table[SomeClass] = reduce_SomeClass
or
 class MyPickler(pickle.Pickler):
 dispatch_table = copyreg.dispatch_table.copy()
 MyPickler.dispatch_table[SomeClass] = reduce_SomeClass
 p = MyPickler(...)
The equivalent using copyreg would be
 copyreg.pickle(SomeClass, reduce_SomeClass)
 p = pickle.Pickler(...)
msg154847 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012年03月03日 19:56
That looks like a good idea.
I don't understand the following code:
+ try:
+ self._dispatch_table = self.dispatch_table
+ except AttributeError:
+ self._dispatch_table = dispatch_table
... since self.dispatch_table is a property returning self._dispatch_table. Did you mean type(self).dispatch_table?
Also, you need to update the docs (which will also make the intended semantics of the patch clearer :-)).
msg154893 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012年03月04日 13:49
> I don't understand the following code:
> ... 
> since self.dispatch_table is a property returning
> self._dispatch_table. Did you mean type(self).dispatch_table?
More or less. That code was a botched attempt to match the behaviour of the C implementation.
The C implementation does not expose the dispatch table unless it has been explicitly set (on the pickler or the pickler class), and it ignores any "dispatch_table" (or "persistent_id") attribute on the metaclass.
I will do a fixed patch with docs.
msg154897 - (view) Author: Richard Oudkerk (sbt) * (Python committer) Date: 2012年03月04日 14:51
Updated patch with docs.
msg154901 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2012年03月04日 17:35
New changeset f7a9a10ae0c0 by Antoine Pitrou in branch 'default':
Issue #14166: Pickler objects now have an optional `dispatch_table` attribute which allows to set custom per-pickler reduction functions.
http://hg.python.org/cpython/rev/f7a9a10ae0c0 
msg154902 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012年03月04日 17:39
I've replaced occurrences of "pickle.Pickler" with "self.pickler_class" in the tests. Otherwise, the patch looks perfect and I've now committed it. Thank you!
History
Date User Action Args
2022年04月11日 14:57:27adminsetgithub: 58374
2012年03月04日 17:39:05pitrousetstatus: open -> closed
resolution: fixed
messages: + msg154902

stage: patch review -> resolved
2012年03月04日 17:35:49python-devsetnosy: + python-dev
messages: + msg154901
2012年03月04日 14:51:04sbtsetfiles: + pickle_dispatch.patch

messages: + msg154897
2012年03月04日 13:49:26sbtsetmessages: + msg154893
2012年03月03日 19:56:47pitrousetmessages: + msg154847
stage: patch review
2012年03月02日 04:36:55eric.araujosetnosy: + pitrou, alexandre.vassalotti
2012年03月01日 14:34:09sbtcreate

AltStyle によって変換されたページ (->オリジナル) /