homepage

Fix problems related to reduce cycles during pickling. A
"reduce cycle" is what happens when a __reduce__
implementation returns an args that cycles back through the
object it tried to reduce. This can't work because the
unpickler has to call the constructor with those args, but
it doesn't yet have that object to be able to place inside
the args. There are two problems related to this:
 1. The standard reduce implementation for proto < 2 in
 copy_reg (_reduce_ex) doesn't correctly deal with
 recursive structures. reduce_2 in typeobject.c does the
 right thing by using listitems and dictitems. Fix
 _reduce_ex by making it do the same thing. This is okay
 for proto < 2 because those arguments are a pickler-
 side feature. Tested in test_stdreducecycle.
 2. Our pickle implementations don't check for reduce
 cycles. This is somewhat cosmetic except that stack
 overflow protection is imperfect (for cPickle), causing
 crashes, and some kinds of cycles trigger asserts (in
 pickle). Fixed in pickle and cPickle by introducing a
 reducing_now set; on entering save_reduce, the object
 id being saved must not be in that set or we've been
 called recursively while saving the callable or
 arguments. Tested in test_reduce_cycle.
This shouldn't change any semantics. That reduce shouldn't
introduce cycles through args isn't documented, but it can't
work any other way.
Possible improvement: If we want to support reducing of real
immutable containers we might have to relax the reduction
cycle test to give the cycle a chance of resolving itself
normally (by constructing a partial object to pass to the
constructor and filling it in later). I'm not sure if this
trouble is worth it just to avoid writing a
frozenset.__setstate__.
See also: http://mail.python.
org/pipermail/python-dev/2004-October/049714.html
It would be very good if someone familiar with pickle
reviewed this; I am still not very confident that I
completely understand all the issues.