[Python-checkins] python/nondist/peps pep-0289.txt,1.4,1.5

Thu Oct 23 02:36:39 EDT 2003

Update of /cvsroot/python/python/nondist/peps
In directory sc8-pr-cvs1:/tmp/cvs-serv24085
Modified Files:
	pep-0289.txt 
Log Message:
Add lots of detail and examples to the Details section. Free
variables are now captured at definition time.
Index: pep-0289.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0289.txt,v
retrieving revision 1.4
retrieving revision 1.5
diff -C2 -d -r1.4 -r1.5
*** pep-0289.txt	22 Oct 2003 22:08:06 -0000	1.4
--- pep-0289.txt	23 Oct 2003 06:36:37 -0000	1.5
***************
*** 66,70 ****

 dotproduct = sum(x*y for x,y in itertools.izip(x_vector, y_vector))
! 
 Having a syntax similar to list comprehensions also makes it easy to
 convert existing code into an generator expression when scaling up
--- 66,70 ----

 dotproduct = sum(x*y for x,y in itertools.izip(x_vector, y_vector))
! 
 Having a syntax similar to list comprehensions also makes it easy to
 convert existing code into an generator expression when scaling up
***************
*** 86,99 ****
 ===========

! 1. The semantics of a generator expression are equivalent to creating
! an anonymous generator function and calling it. There's still discussion
! about whether that generator function should copy the current value of
! all free variables into default arguments.

! 2. The syntax requires that a generator expression always needs to be inside
! a set of parentheses and cannot have a comma on either side. Unfortunately,
! this is different from list comprehensions. While [1, x for x in R] is
! illegal, [x for x in 1, 2, 3] is legal, meaning [x for x in (1,2,3)].
! With reference to the file Grammar/Grammar in CVS, two rules change:

 a) The rule::
--- 86,112 ----
 ===========

! (None of this is exact enough in the eye of a reader from Mars, but I
! hope the examples convey the intention well enough for a discussion in
! c.l.py. The Python Reference Manual should contain a 100% exact
! semantic and syntactic specification.)

! 1. The semantics of a generator expression are equivalent to creating
! an anonymous generator function and calling it. For example::
! 
! g = (x**2 for x in range(10))
! print g.next()
! 
! is equivalent to::
! 
! def __gen():
! for x in range(10):
! yield x**2
! g = __gen()
! print g.next()
! 
! 2. The syntax requires that a generator expression always needs to be
! directly inside a set of parentheses and cannot have a comma on either
! side. With reference to the file Grammar/Grammar in CVS, two rules
! change:

 a) The rule::
***************
*** 110,140 ****
 b) The rule for arglist needs similar changes.

! 2. The loop variable is not exposed to the surrounding function. This
! facilates the implementation and makes typical use cases more reliable.
! In some future version of Python, list comprehensions will also hide the
 induction variable from the surrounding code (and, in Py2.4, warnings
 will be issued for code accessing the induction variable).
- 
- 3. There is still discussion about whether variable referenced in generator
- expressions will exhibit late binding just like other Python code. In the
- following example, the iterator runs *after* the value of y is set to one::

! def h():
! y = 0
! l = [1,2]
! def gen(S):
! for x in S:
! yield x+y
! it = gen(l)
! y = 1
! for v in it:
! print v

! 4. List comprehensions will remain unchanged::

 [x for x in S] # This is a list comprehension.
 [(x for x in S)] # This is a list containing one generator expression.

 Reduction Functions
--- 123,212 ----
 b) The rule for arglist needs similar changes.

+ This means that you can write::

! sum(x**2 for x in range(10))
! 
! but you would have to write::
! 
! reduce(operator.add, (x**2 for x in range(10)))
! 
! and also::
! 
! g = (x**2 for i in range(10))
! 
! i.e. if a function call has a single positional argument, it can be a
! generator expression without extra parentheses, but in all other cases
! you have to parenthesize it.
! 
! 3. The loop variable (if it is a simple variable or a tuple of simple
! variables) is not exposed to the surrounding function. This facilates
! the implementation and makes typical use cases more reliable. In some
! future version of Python, list comprehensions will also hide the
 induction variable from the surrounding code (and, in Py2.4, warnings
 will be issued for code accessing the induction variable).

! For example::

! x = "hello"
! y = list(x for x in "abc")
! print x # prints "hello", not "c"
! 
! (Loop variables may also use constructs like x[i] or x.a; this form
! may be deprecated.)
! 
! 4. All free variable bindings are captured at the time this function
! is defined, and passed into it using default argument values. For
! example::
! 
! x = 0
! g = (x for c in "abc") # x is not the loop variable!
! x = 1
! print g.next() # prints 0 (captured x), not 1 (current x)
! 
! This behavior of free variables is almost always what you want when
! the generator expression is evaluated at a later point than its
! definition. In fact, to date, no examples have been found of code
! where it would be better to use the execution-time instead of the
! definition-time value of a free variable.
! 
! Note that free variables aren't copied, only their binding is
! captured. They may still change if they are mutable, for example::
! 
! x = []
! g = (x for c in "abc")
! x.append(1)
! print g.next() # prints [1], not []
! 
! 5. List comprehensions will remain unchanged. For example::

 [x for x in S] # This is a list comprehension.
 [(x for x in S)] # This is a list containing one generator expression.

+ Unfortunately, there is currently a slight syntactic difference. The
+ expression::
+ 
+ [x for x in 1, 2, 3]
+ 
+ is legal, meaning::
+ 
+ [x for x in (1, 2, 3)]
+ 
+ But generator expressions will not allow the former version::
+ 
+ (x for x in 1, 2, 3)
+ 
+ is illegal.
+ 
+ The former list comprehension syntax will become illegal in Python
+ 3.0, and should be deprecated in Python 2.4 and beyond.
+ 
+ List comprehensions also "leak" their loop variable into the
+ surrounding scope. This will also change in Python 3.0, so that the
+ semantic definition of a list comprehension in Python 3.0 will be
+ equivalent to list(<generator expression>). Python 2.4 and beyond
+ should issue a deprecation warning if a list comprehension's loop
+ variable has the same name as a variable used in the immediately
+ surrounding scope.
+ 

 Reduction Functions
***************
*** 153,159 ****
 * Raymond Hettinger first proposed the idea of "generator comprehensions"
 in January 2002.
! 
 * Peter Norvig resurrected the discussion in his proposal for
! Accumulation Displays [3]_.

 * Alex Martelli provided critical measurements that proved the performance
--- 225,231 ----
 * Raymond Hettinger first proposed the idea of "generator comprehensions"
 in January 2002.
! 
 * Peter Norvig resurrected the discussion in his proposal for
! Accumulation Displays.

 * Alex Martelli provided critical measurements that proved the performance
***************
*** 180,184 ****
 .. [3] Peter Norvig's Accumulation Display Proposal
 http:///www.norvig.com/pyacc.html
! 
 .. [4] Jeff Epler had worked up a patch demonstrating
 the previously proposed bracket and yield syntax
--- 252,256 ----
 .. [3] Peter Norvig's Accumulation Display Proposal
 http:///www.norvig.com/pyacc.html
! 
 .. [4] Jeff Epler had worked up a patch demonstrating
 the previously proposed bracket and yield syntax