How to use ast to evaluate built-in function using python?

Question

>>> list("abc")
['a', 'b', 'c']
>>> list = 42
>>> list("xyz")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable

How do i find such bugs if one has accidentally assigned values to class-names and function definitions? I wanted to use AST but not really sure how do i do it?

• Sayse: You make sure you have a good candidate interview process, no good programmer in a real world working environment would give their variables such names
• chepner: The AST doesn't really help you here (at least, not by itself). You need to know what names are already in use to determine if assigning to list will result in a bug.
• Wizard.Ritvik: this is an example also of why not to name a variable as dict
• Yevhen Kuzmovych: The same way you find other bugs, i.e. debug, write tests, run code and print stuff all around
• Wizard.Ritvik: maybe import builtins; builtins.list('xyz') might help though.

Answer 1

One approach is to extract ast nodes for constructs that could contain builtin shadowing; for example, assignments, function signatures, for-loops, and comprehensions, and then check if any target names belonging to these ast objects are builtins:

import warnings
import builtins, ast
def extract_names(node):
 if isinstance(node, ast.Name):
 yield node.id
 elif isinstance(node, ast.arg):
 yield node.arg
 elif isinstance(node, list):
 for i in node: yield from extract_names(i)
 else:
 for i in getattr(node, '_fields', []): yield from extract_names(getattr(node, i))
def log_shadowing(node, names):
 for i in names:
 if i in dir(builtins):
 warnings.warn(f"On line {node.lineno}: shadowing of '{i}'")
def check_node(node):
 if isinstance(node, ast.Assign):
 log_shadowing(node, extract_names(node.targets))
 if isinstance(node, ast.FunctionDef):
 log_shadowing(node, [node.name, *extract_names(node.args)])
 if isinstance(node, ast.For):
 log_shadowing(node, extract_names(node.target))
 if isinstance(node, (ast.ListComp, ast.SetComp, ast.DictComp)):
 for i in node.generators:
 log_shadowing(node, extract_names(i.target)) 

---

sample = """ 
list = 10
def f(a, b, dict, c):
 pass
for set in range(10):
 pass
r = [next for next in range(10)]
"""
for i in ast.walk(ast.parse(sample)):
 check_node(i)

Output:

<stdin>:4: UserWarning: On line 2: shadowing of 'list'
<stdin>:4: UserWarning: On line 3: shadowing of 'dict'
<stdin>:4: UserWarning: On line 6: shadowing of 'set'
<stdin>:4: UserWarning: On line 9: shadowing of 'next'

Answer 2

you need to compare list to its original value, which can be found in the module builtins:

import builtins
print(builtins.list == list) # true
list = 42
print(builtins.list == list) # false

The better thing is to just avoid assigning to built-in functions. A good code editor will make built-in functions a different color