I am curious if it is possible to exec a string within a function as if the string were substituted for exec directly (with appropriate indentation). I understand that in 99.9% of cases, you shouldn't be using exec but I'm more interested in if this can be done rather than if it should be done.
The behavior I want is equivalent to:
GLOBAL_CONSTANT = 1
def test_func():
def A():
return GLOBAL_CONSTANT
def B():
return A()
return B
func = test_func()
assert func() == 1
But I am given instead:
GLOBAL_CONSTANT = 1
EXEC_STR = """
def A():
return GLOBAL_CONSTANT
def B():
return A()
"""
def exec_and_extract(exec_str, var_name):
# Insert code here
func = exec_and_extract(EXEC_STR, 'B')
assert func() == 1
def exec_and_extract(exec_str, var_name):
exec(EXEC_STR) # equivalent to exec(EXEC_STR, globals(), locals())
return locals()[var_name]
NameError: name 'A' is not defined when calling func() since A and B exist inside exec_and_extract's locals() but the execution context while running A or B is exec_and_extract's globals().
def exec_and_extract(exec_str, var_name):
exec(EXEC_STR, locals()) # equivalent to exec(EXEC_STR, locals(), locals())
return locals()[var_name]
NameError: name 'GLOBAL_CONSTANT' is not defined when calling A from inside func() since the execution context of A is exec_and_extract's locals() which does not contain GLOBAL_CONSTANT.
def exec_and_extract(exec_str, var_name):
exec(EXEC_STR, globals()) # equivalent to exec(EXEC_STR, globals(), globals())
return globals()[var_name]
Works but pollutes global namespace, not equivalent.
def exec_and_extract(exec_str, var_name):
locals().update(globals())
exec(EXEC_STR, locals()) # equivalent to exec(EXEC_STR, locals(), locals())
return locals()[var_name]
Works but requires copying the entire content of exec_and_extract's globals() into its locals() which is a waste of time if globals() is large (of course not applicable in this contrived example). Additionally, is subtly not the same as the "paste in code" version since if one of the arguments to exec_and_extract happened to be GLOBAL_CONSTANT (a terrible argument name), the behavior would be different ("paste in" version would use the argument value while this code would use the global constant value).
Trying to cover any "loopholes" in the problem statement:
exec_str value should represent arbitrary code that can access global or local scope variables.exec_str.exec_and_extract (in global namespace or otherwise). i.e. In this example, execution of EXEC_STR should not leave A around to be referenceable in future calls to exec_and_extract.This is impossible. exec interacts badly with local variable scope mechanics, and it is far too restricted for anything like this to work. In fact, literally any local variable binding operation in the executed string is undefined behavior, including plain assignment, function definitions, class definitions, imports, and more, if you call exec with the default locals. Quoting the docs:
The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.
Additionally, code executed by exec cannot return, break, yield, or perform other control flow on behalf of the caller. It can break loops that are part of the executed code, or return from functions defined in the executed code, but it cannot interact with its caller's control flow.
If you're willing to sacrifice the requirement to be able to interact with the calling function's locals (as you mentioned in the comments), and you don't care about interacting with the caller's control flow, then you could insert the code's AST into the body of a new function definition and execute that:
import ast
import sys
def exec_and_extract(code_string, var):
original_ast = ast.parse(code_string)
new_ast = ast.parse('def f(): return ' + var)
fdef = new_ast.body[0]
fdef.body = original_ast.body + fdef.body
code_obj = compile(new_ast, '<string>', 'exec')
gvars = sys._getframe(1).f_globals
lvars = {}
exec(code_obj, gvars, lvars)
return lvars['f']()
I've used an AST-based approach instead of string formatting to avoid problems like accidentally inserting extra indentation into triple-quoted strings in the input.
inspect lets us use the globals of whoever called exec_and_extract, rather than exec_and_extract's own globals, even if the caller is in a different module.
Functions defined in the executed code see the actual globals rather than a copy.
The extra wrapper function in the modified AST avoids some scope issues that would occur otherwise; particularly, B wouldn't be able to see A's definition in your example code otherwise.