homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: execfile/exec execution in other than global scope uses locals(), leading to undefined behavior
Type: behavior Stage: needs patch
Components: Documentation Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Arfrever, benjamin.peterson, docs@python, miss-islington, r.david.murray, techtonik, terry.reedy
Priority: normal Keywords: patch

Created on 2012年12月26日 00:15 by techtonik, last changed 2022年04月11日 14:57 by admin.

Files
File name Uploaded Description Edit
b3.py techtonik, 2012年12月26日 00:15
a.py techtonik, 2012年12月26日 00:22
b2.py techtonik, 2012年12月26日 00:22
Pull Requests
URL Status Linked Edit
PR 24469 merged terry.reedy, 2021年02月07日 04:07
PR 24470 merged miss-islington, 2021年02月07日 05:29
PR 24471 merged miss-islington, 2021年02月07日 05:29
Messages (14)
msg178177 - (view) Author: anatoly techtonik (techtonik) Date: 2012年12月26日 00:15
When a Python file is exec()uted, it magically fails to find names in imported modules. The most magical thing in the examples below (b3.py in attach for Python 3) is that first reference to wintypes.LONG in print statement is actually successfull.
--- a.py
from ctypes import wintypes
print(wintypes.LONG)
class LOGFONT(object):
 field = wintypes.LONG
--- b2.py (Python 2 version)
def main():
 execfile('a.py')
main()
--- Output
<class 'ctypes.c_long'>
Traceback (most recent call last):
 File "b2.py", line 4, in <module>
 main()
 File "b2.py", line 2, in main
 execfile('a.py')
 File "a.py", line 5, in <module>
 class LOGFONT(object):
 File "a.py", line 6, in LOGFONT
 field = wintypes.LONG
NameError: name 'wintypes' is not defined
msg178180 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012年12月26日 01:25
The fact that the print works should be a clue that Python is in fact finding the module and importing it. So your problem actually has to do with namespaces, which is something you have to think about when using exec or execfile. You can see this by replacing your import with any variable setting (say, a=1) and referencing that in the class body.
The problem here is that execfile is operating inside a function, therefore the local and global namespaces are different. wintypes gets imported into the *local* namespace.
Now, if you inline this type of code by hand, wintypes (or a) is found in the local namespace when the class statement is executed. But when it is done via execfile, it is not. 
I'm not clear on whether or not this is a bug, but if it isn't there is certainly missing documentation in the description of execfile.
msg178184 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012年12月26日 03:25
The best solution is to just always pass an explicit namespace to exec. That should be documented.
msg178190 - (view) Author: anatoly techtonik (techtonik) Date: 2012年12月26日 07:08
The workaround with the best case is a magical knowledge, which many don't possess and don't understand (I still don't get it). It's very tempting to ask why passing explicit namespace is the best solution, but instead I'd like to concentrate on this case where execfile() is not given any arguments. The documentation says:
 If both dictionaries are omitted, the expression is executed in the environment where execfile() is called.
From this description I understand that the code should be executed just like inline code. Why it can not? What limitation of Python doesn't make this possible? Is there any secret reason under cover?
msg178232 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012年12月26日 16:00
Basically, it calls locals() in the function scope, which is undefined behavior.
msg178253 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012年12月26日 20:55
Do you mean that *modifying* locals() in the function scope is undefined behavior? That makes sense, and is a documented limitation.
So we need to clarify the execfile/exec documentation to note that if called in anything other than the global scope, locals() will get used as the locals dictionary, and this will lead to undefined behavior if any operation is performed that updates the local namespace...and thus you are best recommend to always pass one or two dictionaries to execfile/exec, so that you *know* what is getting updated.
Although I have to say that the exec/execfile doing something that is specified to lead to undefined behavior smells like a bug. (Not that we could fix it even if we agreed that it was, for backward compatibility reasons.)
msg178255 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2012年12月26日 20:56
The best thing would be if we could kill the default use of locals() and globals() in execfile, but that's probably Py4 material.
msg178442 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012年12月29日 00:00
This is at most a further doc clarification issue as the code is working as documented. In a previous issue, I added the following sentence to all current versions to try to clarify this type of behavior: " If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition." (This follows " Remember that at module level, globals and locals are the same dictionary") Let's try it:
class Dummy:
 from ctypes import wintypes
 print(wintypes.LONG)
 class LOGFONT(object):
 field = wintypes.LONG
>>>
<class 'ctypes.c_long'>
Traceback (most recent call last):
...
 File "F:\Python\mypy\tem.py", line 6, in LOGFONT
 field = wintypes.LONG
NameError: name 'wintypes' is not defined
Bingo! Anatoly's result is just as documented.
The doc later says "modifications to the default locals dictionary should not be attempted." Anatoly's result is an example of why not.
Lets distill the situation:
1. The globals dict and locals mapping passed to exec are either the same object or different objects. This relation determines the execution behavior.
2. If they are the same object, the code is executed as if at module scope. They are the same if exec is called with both defaults at module scope, where globals() is locals(), or if they are explicitly made the same ("globals = locals()", "locals = globals()", or "globals=dic, locals=dic").
3. If they are different objects, the code is executed as if embedded in a (dummy) class definition. They are different if exec is called with both defaults within a class or function scope*, where globals() is not locals(), or if explicit settings leave them different ("globals = dic" where dic is not locals(), "locals=map", where map is not globals, or "globals=dic, locals=map", where dic is not map).
I believe this nails the situation#.
* In 2.x, comprehensions do not create a function scope, but in 3.x, they do. Lambda expressions always do. This is why I did not write 'within a class or function definition', as some might not see that as including comprehensions.
# The new last sentence of the second paragraph, quoted above, contradicts the older first sentence: "In all cases, if the optional parts are omitted, the code is executed in the current scope." Before 2.2, when the 'current scope' of a function was limited to global and local namespaces, that sentence was true. Indeed, it summarized points 1,2,3 above. I believe that it is not true now, and should be revised, as nonlocal namespaces cannot be seen by exec'ed code. I believe I checked that before adding the new sentence, but I would recheck before revising. I am thinking about how to perhaps rewrite the paragraph.
msg178456 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012年12月29日 01:26
It looks like you are correct Terry. The problem, I think, is that the behavior of name spaces inside a class definition is probably the least intuitive aspect of python scoping, so that sentence, while technically complete, doesn't provide enough guidance.
msg178472 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012年12月29日 04:45
I suppose you could say that I kicked that particular can over to the class doc ;-).
The fundamental problem with exec is that it is at least as complicated as Python, since it executes any legal python code, and in fact is even more complicated* because there are various possible relationships with the calling context. Moreover, it always returns None, so that *any* effect is a side-effect, which tends to be 'magical'.
* For one thing, people can write and run normal python code without knowing that a = b (and other binding statements, like import) at module scope means locals()['a'] rather than globals()['a']. At module scope, there are the same because globals() is locals(). Within exec'ed code, they may not be the same thing even for 'top level' code. This is exactly what tripped up Anatoly in his example with the import.
I am thinking that a short How To Exec() might be a good idea, since a real explanation is too much for even a half-page entry in the built-ins chapter.
Note: the following doc statement "Be aware that the return and yield statements may not be used outside of function definitions" needs to have nonlocal added.
>>> nonlocal a
SyntaxError: nonlocal declaration not allowed at module level
>>> exec('nonlocal a')
Traceback (most recent call last):
 File "<pyshell#19>", line 1, in <module>
 exec('nonlocal a')
 File "<string>", line None
SyntaxError: nonlocal declaration not allowed at module level
>>> def f(): exec('nonlocal a') 
f()
...
SyntaxError: nonlocal declaration not allowed at module level
This again points to why exec can be confusing. compile() considers the string it compiles to be top-level code without any surrounding context. However, exec() enables one to run 'top level' code with different globals and locals. There is no exact precedent for this in normal operation. The closest is execution of code within a class statement (before the type(name, dic, bases) part). But even that is not absolutely the same for nonlocal (though this could be the only exception ;-).
>>> >>> class C: nonlocal a
SyntaxError: no binding for nonlocal 'a' found
A different error here (arguably not the best) -- the same as
>>> def f(): nonlocal a
SyntaxError: no binding for nonlocal 'a' found
msg386576 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021年02月07日 05:29
New changeset 0ec57e25c918b859b9f8d464e34e0ac859c2f8b3 by Terry Jan Reedy in branch 'master':
bpo-16781: In 'exec' doc, add 'nonlocal' to 'yield' and 'return' (GH-2446)
https://github.com/python/cpython/commit/0ec57e25c918b859b9f8d464e34e0ac859c2f8b3
msg386577 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021年02月07日 05:30
PR is based on 'Note:...' directly above. I am still thinking about a patch for the namespace paragraph.
msg386578 - (view) Author: miss-islington (miss-islington) Date: 2021年02月07日 05:38
New changeset 920bf6a3a656e329c2bcbb761eb8c13c46c8cd05 by Miss Islington (bot) in branch '3.8':
bpo-16781: In 'exec' doc, add 'nonlocal' to 'yield' and 'return' (GH-2446)
https://github.com/python/cpython/commit/920bf6a3a656e329c2bcbb761eb8c13c46c8cd05
msg386591 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2021年02月07日 14:14
New changeset 863eb7170b3017399fb2b786a1e3feb6457e54c2 by Miss Islington (bot) in branch '3.9':
bpo-16781: In 'exec' doc, add 'nonlocal' to 'yield' and 'return' (GH-2446)
https://github.com/python/cpython/commit/863eb7170b3017399fb2b786a1e3feb6457e54c2
History
Date User Action Args
2022年04月11日 14:57:39adminsetgithub: 60985
2021年02月07日 14:14:23terry.reedysetmessages: + msg386591
2021年02月07日 05:38:57miss-islingtonsetnosy: + miss-islington
messages: + msg386578
2021年02月07日 05:30:38terry.reedysetnosy: - miss-islington

messages: + msg386577
stage: patch review -> needs patch
2021年02月07日 05:29:20miss-islingtonsetpull_requests: + pull_request23265
2021年02月07日 05:29:13miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request23264
2021年02月07日 05:29:06terry.reedysetmessages: + msg386576
2021年02月07日 04:07:32terry.reedysetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request23263
2020年11月10日 20:46:55iritkatrielsetversions: + Python 3.8, Python 3.9, Python 3.10, - Python 2.6, Python 2.7, Python 3.2, Python 3.3, Python 3.4
2012年12月29日 05:58:19Arfreversetnosy: + Arfrever
2012年12月29日 04:45:39terry.reedysetmessages: + msg178472
2012年12月29日 01:26:19r.david.murraysetmessages: + msg178456
2012年12月29日 00:00:20terry.reedysetnosy: + terry.reedy, docs@python
messages: + msg178442

assignee: docs@python
components: + Documentation, - Interpreter Core
stage: needs patch
2012年12月26日 20:56:40benjamin.petersonsetmessages: + msg178255
2012年12月26日 20:55:09r.david.murraysetmessages: + msg178253
title: execfile/exec execution of class statement does not access locals() -> execfile/exec execution in other than global scope uses locals(), leading to undefined behavior
2012年12月26日 16:00:10benjamin.petersonsetmessages: + msg178232
2012年12月26日 07:08:18techtoniksetmessages: + msg178190
2012年12月26日 03:25:40benjamin.petersonsetmessages: + msg178184
2012年12月26日 01:25:04r.david.murraysetnosy: + r.david.murray, benjamin.peterson
title: execfile/exec messes up with imports in executed file -> execfile/exec execution of class statement does not access locals()
messages: + msg178180

versions: - Python 3.1
type: behavior
2012年12月26日 00:22:29techtoniksetfiles: + b2.py
2012年12月26日 00:22:21techtoniksetfiles: + a.py
2012年12月26日 00:15:50techtonikcreate

AltStyle によって変換されたページ (->オリジナル) /