homepage

On Tue, Mar 6, 2012 at 9:43 AM, Jim Jewett <report@bugs.python.org> wrote:
>
> Jim Jewett <jimjjewett@gmail.com> added the comment:
>
> On Tue, Mar 6, 2012 at 11:56 AM, Mark Shannon wrote:
>
>> Jim Jewett:
>>> Can't this be triggered by non-malicious code that just happened
>>> to have a python comparison and get hit with a thread switch?
>
>> So, they are writing to a dict in one thread while reading from the
>> same dict in another thread, without any external locks and with
>> keys written in Python.
>
> Correct. For example, it could be a configuration manager, or a
> cache, or even a worklist. If they're just adding new keys, or even
> deleting other (==> NOT the one being looked up) keys, why should that
> keep them from finding the existing, unchanged keys?
Use a lock or a built-in key class. I realize that neither is ideal,
but then, neither was the old situation.
>>> I'm not sure how often it happens, but today it would not be visible
>>> to the user; after the patch, users will see a sporadic failure that
>>> they can't easily defend against.
>
>> I suspect, they are already seeing sporadic failures.
>
> How?
>
> The chain terminates as soon as the dict doesn't resize; without
> malicious intent, the odds of hitting several resizes in a row are so
> miniscule that it probably hasn't even slowed them down.
Now I'm torn. If we'd have this RuntimeError from the start, would we
consider it a flaw in the dict implementation or a feature? The
RuntimeError when changing a dict's size while iterating over it is
definitely a feature (so as to allow the implementation to rehash
everything upon insert/delete). But this is not quite the same. Or is
it? On the one hand I think the scenario is pretty unlikely (mostly
because it involves a user-defined comparison); OTOH it would be quite
nasty to debug. Or would it? You do get a decent error message...
Note that Victor's alternate fix (nomodify.diff) has the same problem
-- it just raises RuntimeError in the mutating thread rather than in
the lookup thread.