homepage

Using:
ActivePython 2.4.2 Build 10 (ActiveState Corp.) based 
on
Python 2.4.2 (#67, Jan 17 2006, 15:36:03) [MSC v.1310 
32 bit (Intel)] on win32
For reasons I do not understand, the following class 
leaks itself continuously:
class AttrDict(dict):
 def __init__(self, *args, **kw):
 dict.__init__(self, *args, **kw)
 self.__dict__ = self
Whereas this version does not:
class AttrDict(dict):
 def __init__(self, *args, **kw):
 dict.__init__(self, *args, **kw)
 
 def __getattr__(self, key):
 return self[key]
 
 def __setattr__(self, key, value):
 self[key] = value
My test looks like this:
for n in xrange(1000000):
 import gc
 gc.collect()
 ad = AttrDict()
 ad['x'] = n
 ad.y = ad.x
 print n, ad.x, ad.y
And I sit and watch in the windows task manager while 
the process grows and grows. With the __getattr__ 
version, it doesn't grow.

Logged In: YES 
user_id=4771
This is caused by the tp_clear not doing its job -- 
in this case, tp_clear is subtype_clear(), which does
not reset the __dict__ slot to NULL because it assumes
that the __dict__ slot's content itself will be cleared,
which is perfectly true but doesn't help if self.__dict__
is self.
Attached a patch to fix this. It's kind of hard to
test for this bug because all instances of AttrDict
are really cleared, weakrefs to them are removed,
etc.
Also attached is an example showing a similar bug: a
cycle through the ob_type field, with a object U
whose class is itself. It is harder to clear this
link because we cannot just set ob_type to NULL in
subtype_clear. Ideas welcome...

Logged In: YES 
user_id=33168
Armin, why not just check any leaking test cases into
Lib/test/leakers? I have no idea if your patch is correct
or not. I wouldn't be surprised if there are a bunch more
issues similar to this. What if you stick self in
self.__dict__ (I'm guessing this is ok, but there are a
bunch of variations) or start playing with weakrefs?

Attached test case demonstrates the leak.

I've reproduced this problem with 2.7, 3.1 and 3.2.

Added the two tests in Lib/test/leakers as r45389 (in 2006) and r84296 (now).

Could this patch please be committed to Python? We have just run into this problem in production, where our own variant of AttrDict was shown to be leaking.
It is possible to work around the problem by implementing explicit __getattr__ and __setattr__, but that is both slower and trickier to get right.

New changeset 3787e896dbe9 by Benjamin Peterson in branch '3.2':
allow cycles throught the __dict__ slot to be cleared (closes #1469629)
http://hg.python.org/cpython/rev/3787e896dbe9 

New changeset c7623da4e2af by Benjamin Peterson in branch '2.7':
allow cycles throught the __dict__ slot to be cleared (closes #1469629)
http://hg.python.org/cpython/rev/c7623da4e2af