homepage

This patch changes os.environ to inherit builtin dict rather than UserDict.

What's the rationale for this change?

PEP 390. It's cleaner and faster.

I still must be missing something. There is no PEP 390, AFAICT.

Forgive me. I meant 290.

Sorry, it still makes no sense. Up to r56113, PEP 290 doesn't seem to
talk about UserDict, os.environ, or inheritance from new-style classes.
What section are you specifically referring to?

I know that it doesn't mention inheriting dict specifically, but you
asked why, so I was referring to the Rationale: "Modernization options
arise when new versions of Python add features that allow improved
clarity or higher performance than previously available." Sorry for the
confusion.

The performance gain is rather good:
hsoft-dev:python hsoft$ ./python.exe -m "timeit" -s "import os" 
"os.environ['HOME']"
1000000 loops, best of 3: 1.16 usec per loop
hsoft-dev:python hsoft$ patch -p0 < environ-modern.diff 
patching file Lib/os.py
hsoft-dev:python hsoft$ ./python.exe -m "timeit" -s "import os" 
"os.environ['HOME']"
1000000 loops, best of 3: 0.428 usec per loop
The regression tests still pass, and modernization of the code is a nice 
thing.
+1

Small comment on the patch:
 def clear(self):
- for key in self.data.keys():
+ for key in self.keys():
 unsetenv(key)
- del self.data[key]
+ del self[key]
It looks like the patched version will unsetenv twice, Change
del self[key]
to
dict.__delitem__(self, key)
+1

Here's a corrected version.

See #1367711 for a discussion whether this is really the way to go.

And another.

> dict doesn't dynamically bind any calls; to
> properly replace the semantics of dict, you
> have to implement *all* API.
What was the rationale for this decision? To me it looks like a hold-
over from the time when dicts were not subclassable.
I agree, this is a topic for python-ideas rather than bug-track, but if 
you could point me to any prior discussions on this issue, it would help 
me to either formulate a specific proposal or drop the issue before more 
electrons are sacrificed.

> What was the rationale for this decision? To me it looks like a hold-
> over from the time when dicts were not subclassable.
I reckon it's faster this way, and you want basic datatypes like dict to
be fast.

> What was the rationale for this decision? To me it looks like a hold-
> over from the time when dicts were not subclassable.
> 
> I agree, this is a topic for python-ideas rather than bug-track, but if 
> you could point me to any prior discussions on this issue, it would help 
> me to either formulate a specific proposal or drop the issue before more 
> electrons are sacrificed.
Can't find the original discussion right now, but one statement is at
http://www.mailinglistarchive.com/python-dev@python.org/msg01728.html
In essence, inheriting from dict is intentionally unspecified: if you
only change some methods, but not all, the behavior of the
non-overridden methods is unspecified. This is necessary because
a) specifying it would be a lot of work, and
b) specifying it might unreasonable restrict future implementations, and
c) specifying late binding for some of the methods would unreasonably
 slow down their implementation, even if the common case that dict is
 not subclassed.

The builtins make direct calls to their own internal methods. This has 
existed for a long time. It would be hard to change without crushing 
performance. Changing it violates Meyer's open-closed principle. And 
changing it also introduces weird, unexpected behaviors (i.e. a 
seemingly innocent method override can break pickling for an object -- 
we had a bug report on this once for dicts and sets). Also, making 
these kind of changes makes it *much* harder for a builtin to guarantee 
that it invariants are maintained and opening the door for segfaults.

First, if the new thread on dict behavior does not make sense, please 
see discussion at issue1367711 where it started. (My mistake following 
up here.)
Second, the ability to subclass built-in type is such a great feature, 
that it is a shame that doing so comes with so many caveats. The way I 
understand objections to issue1367711 patch is that (1) it is hard to 
derive from dict properly and (2) derived classes may be broken by 
future changes to dict. The second objection applies to UserDict 
derivation as much as dict derivation, only UserDict is unlikely to be 
touched ever in the future. As to the first objection, if the burden is 
that all methods have to be overriden, let os.environ lead by example 
and do it. UserDict is a kludge from the XX century, stdlib should stop 
promoting its use.
Since there is so much resistance to making dict more subclass-friendly, 
what would you say to reimplementing DictMixin in C as say 'dict.mixin'?

> The builtins make direct calls to their own internal methods.
Raymond, I guess issue2067 escaped your review. :-)

Py3.0 updte: UserDict is going to survive into Py3.0 and will be moved 
into the collections module. DictMixin is replaced by the abstract 
base classes for mappings.
I think the discussion here has grown far beyond the original bug 
report. I recommend this one be closed and that you start a thread on 
python-3000 or python-dev if you want to propose altering basic APIs or 
moving the ABC code into C.
I can't say that I support your "UserDict is a kludge ..." comment. 
The class has its uses and the standard library will continue to use it 
where appropriate. 
FWIW, at one time I also thought all uses of UserDict would ultimately 
be supplanted by dict subclassing, but I found that it was somewhat 
difficult to remove in some circumstances and that its API had some 
advantages (i.e. it's easier to write subclass code like self.data
[computedkey]=computedvalue than to dispatch to dict.__setitem__(self, 
computedkey, computedvalue) -- the latter form is somewhat errorprone 
when the subclass methods become more complex).
Will look at 1367711 as you suggest.

Let's get back on-topic.
I assume you are recommending to close this issue by rejecting the 
patch. I disagree. The patch can be fixed to properly override all 
methods and a unit test can be added to guarantee that all dict methods 
are overridden.
The patch has two distinct benefits: (1) UserDict is not loaded on 
startup; and (2) os.environ is faster. IMHO, these two reasons make the 
patch worth pursuing.
BTW, one of the objections based on exec .. in os.environ behavior (see http://bugs.python.org/msg49131) has become invalid since then.
Specific issues should of course be addressed.

I also disagree that UserDict is a clutch, and that inheriting from dict
is in any way more modern or cleaner than inheriting from UserDict.
Hence I'm rejecting this patch. To "appeal", please discuss on python-dev.

Did anyone mention "clutch"? :-)
Oh, well, please close issue1367711 as a duplicate.