homepage

In Python 3.2, a tuple like (1,-2,3) will not be optimized into a constants at compile time. The tuple is built at run-time. Earlier versions of Python optimized these tuples at compile time.
Here's an example program.
# test.py
from dis import dis
def x(): return (1,2,3)
def y(): return (1,-2,3)
print ("dis x:")
dis(x)
print()
print("dis y:")
dis(y)
The compiler in 3.2rc3 produces code for function y() that builds the tuple at run-time while the tuple in x() is optimized at compile time.
C:\tmp>c:\python32\python --version
Python 3.2rc3
C:\tmp>c:\python32\python test.py
dis x:
 3 0 LOAD_CONST 4 ((1, 2, 3))
 3 RETURN_VALUE
dis y:
 4 0 LOAD_CONST 1 (1)
 3 LOAD_CONST 4 (-2)
 6 LOAD_CONST 3 (3)
 9 BUILD_TUPLE 3
 12 RETURN_VALUE
However, under 3.1.3, the tuples in both functions are optimized at compile time.
C:\tmp>c:\python31\python test.py
dis x:
 3 0 LOAD_CONST 4 ((1, 2, 3))
 3 RETURN_VALUE
dis y:
 4 0 LOAD_CONST 4 ((1, -2, 3))
 3 RETURN_VALUE
Although the compiled code produced 3.2rc3 is correct, it is much slower than the code generated by 3.1.3.

I have also determined that negative elements interfere with the frozenset optimization described in issue6690. http://bugs.python.org/issue6690.
Here's an example program:
# test.py
from dis import dis
def x(var): return var in {1,2,3} # Note curly braces. These are sets.
def y(var): return var in {1,-2,3}
print ("dis x:")
dis(x)
print()
print("dis y:")
dis(y)
Running this produces:
C:\tmp>c:\Python32\python.exe test.py
dis x:
 3 0 LOAD_FAST 0 (var)
 3 LOAD_CONST 4 (frozenset({1, 2, 3}))
 6 COMPARE_OP 6 (in)
 9 RETURN_VALUE
dis y:
 4 0 LOAD_FAST 0 (var)
 3 LOAD_CONST 1 (1)
 6 LOAD_CONST 4 (-2)
 9 LOAD_CONST 3 (3)
 12 BUILD_SET 3
 15 COMPARE_OP 6 (in)
 18 RETURN_VALUE

I wonder if this has anything to do with issue 9011?

The culprit is r82043: "Issue #9011: Remove buggy and unnecessary ST->AST compilation code".

The problem is that the "UNARY_NEGATIVE + LOAD_CONST" optimization, which occurs first in bytecode order, inserts a NOP and prevents the tuple optimization to work.

I think r82043 may also explain why 3.1.3 can fold the expression 2 * -3 into -6 while 3.2rc3 cannot.
# test.py
from dis import dis
def y(): 2 * -3
print("dis y:")
dis(y)
C:\tmp>c:\Python32\python.exe --version
Python 3.2rc3
C:\tmp>c:\Python32\python.exe test.py
dis y:
 3 0 LOAD_CONST 1 (2)
 3 LOAD_CONST 3 (-3)
 6 BINARY_MULTIPLY
 7 POP_TOP
 8 LOAD_CONST 0 (None)
 11 RETURN_VALUE
C:\tmp>c:\Python31\python.exe --version
Python 3.1.3
C:\tmp>c:\Python31\python.exe test.py
dis y:
 3 0 LOAD_CONST 3 (-6)
 3 POP_TOP
 4 LOAD_CONST 0 (None)
 7 RETURN_VALUE

Ouch. Obviously test_peephole doesn't have enough tests...

Here's a patch against the version of test_peepholer.py in 3.2rc3. It verifies that expressions like the following are optimized:
3*-4
(1,-2,3)
a in {1,-2,3)

Here is a patch that enables advanced (recursive) constant folding.
Example:
python -c "import dis; f=lambda x: x in {(3*-5)+(-1-6),(1,-2,3)*2,None};dis.dis(f)"
With 3.1:
 1 0 LOAD_FAST 0 (x) 
 3 LOAD_CONST 7 (-15) 
 6 LOAD_CONST 8 (-7) 
 9 BINARY_ADD 
 10 LOAD_CONST 10 ((1, -2, 3, 1, -2, 3)) 
 13 LOAD_CONST 11 (None) 
 16 BUILD_SET 3 
 19 COMPARE_OP 6 (in) 
 22 RETURN_VALUE 
With 3.2+patch:
 1 0 LOAD_FAST 0 (x) 
 3 LOAD_CONST 14 (frozenset({None, -22, (1, -2, 3, 1, -2, 3)})) 
 6 COMPARE_OP 6 (in) 
 9 RETURN_VALUE

Unassigning. I don't think that r82043 is the *real* culprit here; that bugfix just happened to expose a deficiency in the peepholer; one that's already present in other situations:
>>> dis.dis(lambda: 2*(3*4))
 1 0 LOAD_CONST 1 (2)
 3 LOAD_CONST 4 (12)
 6 BINARY_MULTIPLY 
 7 RETURN_VALUE 
>>> dis.dis(lambda: (2*3)*4)
 1 0 LOAD_CONST 5 (24)
 3 RETURN_VALUE 
Antoine, does your patch take care of this case too?

Le samedi 19 février 2011 à 10:45 +0000, Mark Dickinson a écrit :
> Mark Dickinson <dickinsm@gmail.com> added the comment:
> 
> Unassigning. I don't think that r82043 is the *real* culprit here; that bugfix just happened to expose a deficiency in the peepholer; one that's already present in other situations:
> 
> >>> dis.dis(lambda: 2*(3*4))
> 1 0 LOAD_CONST 1 (2)
> 3 LOAD_CONST 4 (12)
> 6 BINARY_MULTIPLY 
> 7 RETURN_VALUE 
> >>> dis.dis(lambda: (2*3)*4)
> 1 0 LOAD_CONST 5 (24)
> 3 RETURN_VALUE 
> 
> Antoine, does your patch take care of this case too?
It should. Can you test?

> It should. Can you test?
Ah, you're asking me to actually do some (minimal) work instead of just complaining?
Yep, the patch tests fine over here (OS X 10.6), and fixes the 2*(3*4) case.

This patch has tests and is also able to constant-fold tuples larger than 256 elements.

Forgot to attach new patch, sorry.

As discussed on the list, peephole refuses to fold -0. The reasons for this are unclear. Folding was disabled with this commit:
http://hg.python.org/cpython/diff/660419bdb4ae/Python/compile.c
Here's a trivial patch to enable the folding again, along with a test case. make test passes with the patch.
The change is independent from Antoine's patches.

fold-0.patch looks good to me, but why do you include tests only for the float case (-0.0) and not the integer case (-0)?
Style nitpick: "def negzero(): return -(1.0-1.0)" should be on two source lines, not one.

Mark, looks better now?

(forgot parens around 0)

New changeset 14205d0fee45 by Antoine Pitrou in branch 'default':
Issue #11244: The peephole optimizer is now able to constant-fold
http://hg.python.org/cpython/rev/14205d0fee45 

Ok, Eugene's patch is left to apply now. Can I assign this to you, Mark?

Eugene's patch looks like a correct fix to the regression. I'll review further in the next couple of days.
Antoine, we need to further discuss your checkin to the peephole optimizer. I believe it was inappropriate.

> Eugene's patch looks like a correct fix to the regression.
No, it's orthogonal.

Yes, my patch doesn't fix the regression, only a special case of -0.

Eugene's new patch looks good to me; +1 on applying it.
Raymond, do you happen to remember why it was necessary to add the zero-check in the first place?

Is anyone reviewing my patch? It's just 1 line long. Should it be moved to another issue? Though, technically, it's needed to address the regression in question: Python 3.1 folds -0, the current code still does not.

Eugene, according to Mark your patch is fine. It just needs someone to commit it. I would personally prefer to let Mark handle it, since he's our specialist in negative zeroes (and other numerical subtleties) ;)

New changeset ead9c1b9f547 by Mark Dickinson in branch 'default':
Issue #11244: Remove outdated peepholer check that was preventing the peepholer from folding -0 and -0.0. Thanks Eugene Toder for the patch.
http://hg.python.org/cpython/rev/ead9c1b9f547 

Fixed in 'default' branch. Note that the regression still exists in 3.2; I'm not sure that it's worth backporting the two fixes.