Right Way To Turn An Input String Into A Callable Function And Store It As Variable?

I was testing some code to answer Possible to turn an input string into a callable function object in Python? and the question got closed as duplicate but I could not find the answ

Solution 1:

Roughly speaking, if you assume that inputs are fully trusted, and where the provided string is of valid Python syntax that will produce a single callable function, and if exec cannot be used, something like this can be provided

import ast
import types

defstring_to_function(source):
    tree = ast.parse(source)
    iflen(tree.body) != 1ornotisinstance(tree.body[0], ast.FunctionDef):
        raise ValueError('provided code fragment is not a single function')
    co = compile(tree, 'custom.py', 'exec')
    # first constant should be the code object for the functionreturn types.FunctionType(co.co_consts[0], {})

Example:

f = string_to_function("""
def my_function(x):
    return x + 5
"""
)

print('f = %d' % f(5))

Output

f = 10

The code ensures that the provided source is of a single function definition, and makes assumption of the organisation of the generated bytecode (i.e. only works for the compiler built into the current versions of Python, where the generated bytecode places the code object for the single function that was defined in the source in the 0th element in co_consts). The previous version to this answer made use of exec (which questioner "is not a big fan of exec anyway"), done in a way that binds the results into a specific target and this method should be more reliable as it does not touch this lower level structure, though the sanity checks using ast module used here could be included instead with that original version.

Also note that this answer is only applicable for Python 3+.

Further Edit:

I am actually still a little miffed by the remark on the usage of exec being assumed to execute arbitrary code (on fixed inputs) simply because what it actually does is often very misunderstood. In this particular case, if it is verified that only specific source is accepted, it doesn't necessarily mean every statement is executed immediately. This is especially true for this case (which isn't properly guaranteed in my original lazy answer, but this is where actual understanding of what the framework is actually doing is important for doing anything that involves dynamic compilation of code within the language framework, and given that more level of "safety" is desired (executing function immediately after the fact negates it hence I didn't implemented originally) using ast is done in the edit and it is now objectively better).

So what exactly does calling exec(co) do like essentially what the original example did when given the source input of an single function definition? It can be determined by looking at the bytecode like so:

>>> dis.dis(co)
  20 LOAD_CONST               0 (<code object my_function at 0x7fdbec44c420, file "custom.py", line 2>)
              2 LOAD_CONST               1 ('my_function')
              4 MAKE_FUNCTION            06 STORE_NAME               0 (my_function)
              8 LOAD_CONST               2 (None)
             10 RETURN_VALUE

All it does is to load the code object, make it into a proper function and assign the result to my_function (on the currently relevant scope), and essentially that's it. So yes, the correct way is to verify that the source is definitely a function definition like so here (as verification through checking the AST is more safe than a more naive verification that only one statement is present), then running exec on a specific dict and extract the assignment from there. Using exec this way is not inherently less (or more) safe in this instance, given that any function that was provided would be executed immediately anyway.