[Python-Dev] PEP-xxx: Unification of for statement and list-comp syntax

Sun May 21 17:10:51 CEST 2006

Hi all!
The following PEP tries to make the case for a slight unification of for 
statement and list comprehension syntax.
Comments appreciated, including on the sample implementation.
===
PEP: xxx
Title: Unification of for-statement and list-comprehension syntax
Version: $Revision$
Last-Modified: $Date$
Author: Heiko Wundram <me at modelnine.org>
Status: Active
Type: Standards Track
Content-Type: text/plain
Created: 21-May-2006
Post-History: 21-May-2006 17:00 GMT+0200
Abstract
 When list comprehensions were introduced, they added the ability
 to add conditions which are tested before the expression which is
 associated with the list comprehension is evaluated. This is
 often used to create new lists which consist only of those items
 of the original list which match the specified condition(s). For
 example:
 [node for node in tree if node.haschildren()]
 will create a new list which only contains those items of the
 original list (tree) whose items match the havechildren()
 condition. Generator expressions work similarily.
 With a standard for-loop, this corresponds to adding a continue
 statement testing for the negated expression at the beginning of
 the loop body.
 As I've noticed that I find myself typing the latter quite often
 in code I write, it would only be sensible to add the corresponding
 syntax for the for statement:
 for node in tree if node.haschildren():
 <do something with node>
 as syntactic sugar for:
 for node in tree:
 if not node.haschildren():
 continue
 <do something with node>
 There are several other methods (including generator-expressions or
 list-comprehensions, the itertools module, or the builtin filter
 function) to achieve this same goal, but all of them make the code
 longer and harder to understand and/or require more memory, because
 of the generation of an intermediate list.
Implementation details
 The implementation of this feature requires changes to the Python
 grammar, to allow for a variable number of 'if'-expressions before
 the colon of a 'for'-statement:
 for_stmt: 'for' exprlist 'in' testlist_safe ('if' old_test)* ':'
 suite ['else' ':' suite]
 This change would replace testlist with testlist_safe as the
 'in'-expression of a for statement, in line with the definition of
 list comprehensions in the Python grammar.
 Each of the 'if'-expressions is evaluated in turn (if present), until
 one is found False, in which case the 'for'-statement restarts at the
 next item from the generator of the 'in'-expression immediately
 (the tests are thus short-circuting), or until all are found to be
 True (or there are no tests), in which case the suite body is executed.
 The behaviour of the 'else'-suite is unchanged.
 The intermediate code that is generated is modelled after the
 byte-code that is generated for list comprehensions:
 def f():
 for x in range(10) if x == 1:
 print x
 would generate:
 2 0 SETUP_LOOP 42 (to 45)
 3 LOAD_GLOBAL 0 (range)
 6 LOAD_CONST 1 (10)
 9 CALL_FUNCTION 1
 12 GET_ITER
 >> 13 FOR_ITER 28 (to 44)
 16 STORE_FAST 0 (x)
 19 LOAD_FAST 0 (x)
 22 LOAD_CONST 2 (1)
 25 COMPARE_OP 2 (==)
 28 JUMP_IF_FALSE 9 (to 40)
 31 POP_TOP
 3 32 LOAD_FAST 0 (x)
 35 PRINT_ITEM
 36 PRINT_NEWLINE
 37 JUMP_ABSOLUTE 13
 >> 40 POP_TOP
 41 JUMP_ABSOLUTE 13
 >> 44 POP_BLOCK
 >> 45 LOAD_CONST 0 (None)
 48 RETURN_VALUE
 where all tests are inserted immediately at the beginning of the
 loop body, and jump to a new block if found to be false which pops
 the comparision from the stack and jumps back to the beginning of
 the loop to fetch the next item.
Implementation issues
 The changes are backwards-compatible, as they don't change the
 default behaviour of the 'for'-loop. Also, as the changes that
 this PEP proposes don't change the byte-code structure of the
 interpreter, old byte-code continues to run on Python with this
 addition unchanged.
Implementation
 A sample implementation (with updates to the grammar documentation
 and a small test case) is available at:
 
http://sourceforge.net/tracker/index.php?func=detail&aid=1492509&group_id=5470&atid=305470
Copyright
 This document has been placed in the public domain.
===
--- Heiko.