homepage

Issue #22091 points out a quirk in the compile function and use of the __debug__ "constant" causing inconsistent behavior when the optimize level of the compile call differs from that of the main interpreter; __debug__ in an `if` or `while` statement is compiled out, but all other uses load it dynamically (at runtime), so in a mixed environment (interpreter at optimization=0, compile at optimize=2), you get non-obvious behavior.
This behavior appears to be a consequence of __debug__ being handled by a special case for the `if` and `while` statements in compile.c that statements of that form to be compiled out, but *not* for similar constructs, e.g. `a if __debug__ else b` and `__debug__ or a` are always evaluated at runtime, whether or not `compile` is involved. The `expr_constant` function here https://hg.python.org/cpython/file/fd0ac7ba091e/Python/compile.c#l3542 is responsible for this (and only called in the same file, for `if` and `while` statements).
I'm not sure I understand the peephole optimizer, but if it can operate recursively (that is, an initial replacement of A->B where B could be optimized from B->C is optimized in a subsequent pass, turning all uses of A to C eventually), it seems like the "correct" solution would be to piggyback on optimizations for True and False, by having the peephole optimizer replace LOAD_NAME/LOAD_GLOBAL for __debug__ with an appropriate LOAD_CONST, True or False, based on the compile environment. This would fix this bug (making __debug__ evaluate in the `compile` call, so the environment when the compiled code is executed doesn't matter), and it would optimize all the other cases that the current special cases for `if` and `while` don't cover by letting the recursive pass optimize them out the same way uses of literal True and False is optimized, so uses of ternary expressions, non-`if`/`while` boolean operations, and all other operations using __debug__ are optimized using a constant (and possibly optimized out of existence) without needing to independently maintain separate optimizations all over the codebase for __debug__. Might also allow the removal of the explicit special case for __debug__ in compile.c for `if` and `while`, simplifying that code a bit.
This would fix the problem from #22091 and also make __debug__ reliably useful for "optimization", since all uses of it would "compile out" to optimized runtime code, where right now only two cases do so.
Does this seem reasonable?
Note: I added the nosy list from #22091 here, since that bug is really just a special case of this one for `compile` only.

I think that the complete solution of this problem is making __debug__ a named constant like None, True, False.
This needs changing the grammar of Python.

That would also work. The argument I'd give in favor of performing a pass that replaces it with a literal True or False is that you don't have update as many places, don't have to worry about missing a place, and you don't have to decide if __debug__ is a reference to True or False, or a new object entirely.
It's just too easy to miss a case where __debug__ should be special and not notice (because optimizations aren't heavily tested for specific byte code outputs or anything), where a missed optimization for the True or False constant is much less likely to go unnoticed.

New changeset 3325a6780c81f1ea51190370b5454879c4862a37 by Serhiy Storchaka in branch 'master':
bpo-27169: The __debug__ constant is now optimized out at compile time. (#4880)
https://github.com/python/cpython/commit/3325a6780c81f1ea51190370b5454879c4862a37

New changeset b82da9ebb20053637e731fd40589e1e52f9f8f6e by Serhiy Storchaka in branch '3.6':
[3.6] bpo-27169: The __debug__ constant is now optimized out at compile time. (GH-4880) (#4882)
https://github.com/python/cpython/commit/b82da9ebb20053637e731fd40589e1e52f9f8f6e

New changeset bd6ec4d79e8575df3d08f8a89ba721930032714c by Serhiy Storchaka in branch 'master':
bpo-32365: Fix a reference leak when compile __debug__. (#4916)
https://github.com/python/cpython/commit/bd6ec4d79e8575df3d08f8a89ba721930032714c

New changeset 5659743b5693c9e23313a74117948294e35013f4 by Serhiy Storchaka (Miss Islington (bot)) in branch '3.6':
bpo-32365: Fix a reference leak when compile __debug__. (GH-4916) (#4918)
https://github.com/python/cpython/commit/5659743b5693c9e23313a74117948294e35013f4