homepage

Thanks for the nod to the pickletools module. The structure of the streams do change between python 2.6 and 2.7, at least for files created with cPickle. You can see this from the file sizes that I quoted in my previous post. Below are some snippets of the disassemblies of pickle files generated using make_file.py:
pickle, python 2.6 and 2.7:
 0: \x80 PROTO 2
 2: } EMPTY_DICT
 3: q BINPUT 0
 5: ( MARK
 6: U SHORT_BINSTRING 'ArLLX'
 13: q BINPUT 1
 15: \x85 TUPLE1
 16: q BINPUT 2
 18: K BININT1 0
 ...
cPickle, python 2.6:
 0: \x80 PROTO 2
 2: } EMPTY_DICT
 3: q BINPUT 1
 5: ( MARK
 6: U SHORT_BINSTRING 'XYoGB'
 13: \x85 TUPLE1
 14: q BINPUT 2
 16: K BININT1 0
 ...
cPickle, python 2.7:
 0: \x80 PROTO 2
 2: } EMPTY_DICT
 3: q BINPUT 1
 5: ( MARK
 6: U SHORT_BINSTRING 'QGgRa'
 13: \x85 TUPLE1
 14: K BININT1 1
 ...
As you can see, pickle memoizes both the string and the tuple. cPickle2.6 just memoizes the tuple, while cPickle2.7 doesn't memoize either. It therefore seems that the problem is somehow related to the memo...