[Python-Dev] namespace for generator expressions

Mon Jan 26 17:36:19 EST 2004

At 09:02 AM 1/27/04 +1100, Delaney, Timothy C (Timothy) wrote:
> > From: Guido van Rossum
> >
> > > I'm happy to see progress on the generator expressions
> > implementation,
> > > but I think the specification of namespaces, which is just a sketch,
> > > might be simplified.
> >
> > Ouch! Where were you when this PEP was discussed on python-dev? I
> > was originally strongly in your camp, but Tim and several others
> > convinced me that in every single case where a generator expression
> > has a free variable, you want early binding, not late.
>>I think I agree with Jeremy. I was originally in the early-binding camp, 
>but I think we're better off trusting the programmer.

The intent is that listcomps should be "safely" replaceable with genexprs 
anywhere that an iterator is acceptable in place of a list.
>How often is a generator expression not going to be evaluated almost 
>immediately? I guess, when they're passed to a function. But even in that 
>case, how often are the bindings going to change? Except in pathological 
>cases, they won't.

A trivial example is:
iterators = []
for i in range(5):
 iterators.append(x*2 for x in range(i))
print map(list,iterators)
If a listcomp is used, you get:
[[],[0,],[0,2],[0,2,4],[0,2,4,6],[0,2,4,6,8]]
If genexprs do late binding, you get:
[[0,2,4,6,8],[0,2,4,6,8],[0,2,4,6,8],[0,2,4,6,8],[0,2,4,6,8]]
which is a significant difference in semantics. To make this code work 
with a late binding genexpr, it becomes necessary to create a function 
definition, thus negating the benefit of using the genexpr in the first place.
As to how common this is, ask how often you use a list comprehension inside 
of another loop, and then ask how often you'd use a genexpr instead if it 
was available. Finally, ask yourself whether you're likely to remember 
that you need to totally rewrite the structure if you decide to use a 
genexpr instead. :)
(Of course, I also often forget that free variables don't bind early in 
nested functions! It always seems strange to me that I have to write a 
function that returns a function in order to just define a function with 
bound variables. Indeed, it's hard to think of a time where I've *ever* 
wanted late binding of variables in a nested function.)