This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2014年11月02日 15:28 by serhiy.storchaka, last changed 2022年04月11日 14:58 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| pickle_newobj.patch | serhiy.storchaka, 2014年11月02日 15:28 | review | ||
| pickle_newobj_2.patch | serhiy.storchaka, 2014年12月16日 12:53 | review | ||
| Messages (5) | |||
|---|---|---|---|
| msg230490 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年11月02日 15:28 | |
Currently in pickle with protocol 4 the NEWOBJ_EX opcode is used to reconstruct objects with __getnewargs__() or __getnewargs_ex__() even if there no keyword arguments.
>>> import pickle, pickletools, collections
>>> P = collections.namedtuple('P', 'x y')
>>> pickletools.dis(pickle.dumps(P(12, 34), 3))
0: \x80 PROTO 3
2: c GLOBAL '__main__ P'
14: q BINPUT 0
16: K BININT1 12
18: K BININT1 34
20: \x86 TUPLE2
21: q BINPUT 1
23: \x81 NEWOBJ
24: q BINPUT 2
26: . STOP
highest protocol among opcodes = 2
>>> pickletools.dis(pickle.dumps(P(12, 34), 4))
0: \x80 PROTO 4
2: \x95 FRAME 28
11: \x8c SHORT_BINUNICODE '__main__'
21: \x94 MEMOIZE
22: \x8c SHORT_BINUNICODE 'P'
25: \x94 MEMOIZE
26: \x93 STACK_GLOBAL
27: \x94 MEMOIZE
28: K BININT1 12
30: K BININT1 34
32: \x86 TUPLE2
33: \x94 MEMOIZE
34: } EMPTY_DICT
35: \x94 MEMOIZE
36: \x92 NEWOBJ_EX
37: \x94 MEMOIZE
38: . STOP
highest protocol among opcodes = 4
These EMPTY_DICT//MEMOIZE//NEWOBJ_EX can be replaced by single NEWOBJ. Actually there is a regression in protocol 4 against protocol 3, so may be following patch should be applied on 3.4 too.
Here is a patch which uses __newobj__ instead of __newobj_ex__ if there are no keywords arguments. Also it reduce the number of lines of C code and adds tests for classes with __getnewargs__() and __getnewargs_ex__().
|
|||
| msg232660 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年12月15日 12:35 | |
Ping. |
|||
| msg232686 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年12月15日 22:06 | |
Some examples (with issue19858 for protocol 4 optimization). Unpatched: >>> len(pickle.dumps([P(12, 34) for i in range(1000)], 3)) 17258 >>> len(pickletools.optimize(pickle.dumps([P(12, 34) for i in range(1000)], 3))) 8018 >>> len(pickle.dumps([P(i, -i) for i in range(1000)], 3)) 20999 >>> len(pickletools.optimize(pickle.dumps([P(i, -i) for i in range(1000)], 3))) 11759 >>> len(pickle.dumps([P(12, 34) for i in range(1000)], 4)) 12031 >>> len(pickletools.optimize(pickle.dumps([P(12, 34) for i in range(1000)], 4))) 9028 >>> len(pickle.dumps([P(i, -i) for i in range(1000)], 4)) 15772 >>> len(pickletools.optimize(pickle.dumps([P(i, -i) for i in range(1000)], 4))) 12769 Patched: >>> len(pickle.dumps([P(12, 34) for i in range(1000)], 4)) 10031 >>> len(pickletools.optimize(pickle.dumps([P(12, 34) for i in range(1000)], 4))) 8028 >>> len(pickle.dumps([P(i, -i) for i in range(1000)], 4)) 13772 >>> len(pickletools.optimize(pickle.dumps([P(i, -i) for i in range(1000)], 4))) 11769 |
|||
| msg232739 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2014年12月16日 12:53 | |
Updated patch addresses Antoine's comments. Also added few comments explaining binary data. |
|||
| msg232751 - (view) | Author: Roundup Robot (python-dev) (Python triager) | Date: 2014年12月16日 18:03 | |
New changeset 2ffaac4c8e53 by Serhiy Storchaka in branch 'default': Issue #22783: Pickling now uses the NEWOBJ opcode instead of the NEWOBJ_EX https://hg.python.org/cpython/rev/2ffaac4c8e53 |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:58:09 | admin | set | github: 66972 |
| 2014年12月16日 18:09:27 | serhiy.storchaka | set | status: open -> closed assignee: serhiy.storchaka resolution: fixed stage: patch review -> resolved |
| 2014年12月16日 18:03:09 | python-dev | set | nosy:
+ python-dev messages: + msg232751 |
| 2014年12月16日 12:53:32 | serhiy.storchaka | set | files:
+ pickle_newobj_2.patch messages: + msg232739 |
| 2014年12月15日 22:06:54 | serhiy.storchaka | set | messages: + msg232686 |
| 2014年12月15日 12:35:55 | serhiy.storchaka | set | keywords:
+ needs review messages: + msg232660 |
| 2014年11月02日 15:28:42 | serhiy.storchaka | create | |