I'm trying to dynamically generate a function from user input. The user gives a function as a string input, gives the differents parameters, and I want to write a function that will change this string into a usable function. My idea so far was to use the exec function to create a lambda function constructed with a string like so : exec("f = lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string)). This would give, for example, a string like "f = lambda x, a:x+a" to the exec function.
This technique works fine if I define the scope of exec to globals(), but I'd like my function to be local so that I can do something like this:
def define_function(function_string, parameter_list):
exec("f = lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string))
return f
or like this :
def define_function(function_string, parameter_list):
exec("return lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string))
However, in the first case, it gives an error along the lines of "f is not defined". In the second case, meanwhile, it says that "return can only be used in a function". I understand why the second error occurs, but I wonder why my first try didn't work. Using globals() makes it work, but I'd rather not use it.
If you have any suggestions for other methods for achieving the same result, I'm interested in those as well.
EDIT
Using eval also raises an error :
line 9, in define_function
eval("f = lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string))
File "<string>", line 1
f = lambda x, a:x+a
^
SyntaxError: invalid syntax
Solution 1: use eval():
def get_func(param_list, result_str):
return eval("lambda {}: {}".format(', '.join(['x'] + param_list), result_str))
Solution 2: if you really want to use exec(), do something like this:
def get_func(param_list, result_str):
exec("f = lambda {}: {}".format(', '.join(['x'] + param_list), result_str))
return locals()['f']
Some good info on the differences between exec and eval here: What's the difference between eval, exec, and compile?
I can see what you're trying to do in your question, by the way, and it's worth noting that the documentation pages for exec() and locals() both explicitly warn that this won't work. Ditto, if you type help(locals) into the interactive interpreter, you get this message:
Help on built-in function locals in module builtins:
locals()
Return a dictionary containing the current scope's local variables.
NOTE: Whether or not updates to this dictionary will affect name lookups in
the local scope and vice-versa is *implementation dependent* and not
covered by any backwards compatibility guarantees.
Related
I have this python code that runs other code and I can get the result as such:
def codeTester():
try:
loc = {}
exec(code, {}, loc)
except Exception as e:
#Excpetion handling....
result = loc['result']
loc after running would be a dictionary and result key would have the output of the function.
However this does not work with recursive functions. According to Using exec() with recursive functions and Python3: inject a recursive function into exec() in a function I need to wrap the code which I do using the function inside the second link:
def wrap(s):
return "def foo():\n" \
"{}\n" \
"foo()".format(textwrap.indent(s, ' ' * 4))
This wraps the 'code' with that function and now The recursive code runs but loc[] does not hold the result. It just holds a key called 'foo' which has this value <function foo at 0x000001F208386F70>. How can I get the output in this situation? or is there a better method to achieving this?
edit: I should note that something like exec(code, locals(), locals()) does also work but its still the same problem where I cant access the output of exec()
I am trying to make a script in Python, that when executed, asks the user for a function name and prints the function .__doc__
For example:
>>> print abs.__doc__
abs(number) -> number
Return the absolute value of the argument.
The problem is, it doesn't work with raw_input. The following is my code and what happens when it gets executed.
Code:
f = raw_input("Your function: ")
print f.__doc__
Execution:
Your function: abs
str(object='') -> string
Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
What am I doing wrong?
Well you ask to print the __doc__ of f and f is in this case something like 'abs'. So you call 'abs'.__doc__ which is the __doc__ of a string.
Now you can however query for a builtin function with:
func = getattr(__builtins__,f)
print func.__doc__
this will however only work for builtin functions. You can also look for globals() or locals() which are dictionaries storing the global and local variables respectively.
As others have said, your problem is trying to use the string reply from raw_input() as a function object. You could get the function object by calling getattr(), but you need to known which module it is in. OK, so you could go through the global namespace looking for it, but there is a much simpler solution, just use pydoc:
f = raw_input("Your function: ")
help(f)
In this case, abs is a method that you're querying the __doc__ on. The raw_input is converting the input value to a string, so what you're really executing the __doc__ on is a string.
You get the same results if you do this:
z = ''
print z.__doc__
I want to read some input, which contains python assignment statements like this string:
"VARIABLE = 'something' + OTHER_VAR"
So I use one of these:
exec("VARIABLE = 'something' + OTHER_VAR")
exec("VARIABLE = 'something' + OTHER_VAR", globals(), locals())
I want to use this variable in other code, but after exec(...) it is not in current namespace.
It is possible to get the variable value like this:
locals()['VARIABLE']
however, if I dont know the name of variable it is not solution for me.
So how to get that new variable into my namespace?
UPDATE:
My data for exec are like this:
COMPBLOCK = StringLeft(MyPlatform.HierarchicalName,6) +"_IADI." + CMP + "." + BLOCK ;
SetCustomPropertyValue("DEVLM2",COMPBLOCK + ".DEVLM2",false);
SetCustomPropertyValue("CRIT",COMPBLOCK + ".CRIT",false);
SetCustomPropertyValue("UNACK",COMPBLOCK + ".UNACK",false);
SetCustomPropertyValue("ALMSTA_26",COMPBLOCK + ".ALMSTA#B26",false);
I defined functions SetCustomPropertyValue and StringLeft. I want to avoid some complicated translation of this script to python with all possible inputs. Exec() seems to be very quick solution, but after reading this post - Modifying locals in python I am little bit stuck.
pay attention to the comments warning about how dangerous it is to execute arbitrary code from a foreign source.
if the statements have a consistent format, for example like the one in the example, you could easly parse it and extract the variable name:
varname = stmt.split('=')[0].strip()
or something more sophisticated using regular expressions
if the statement always introduces exactly one new variable, you could compare locals() before and after execution and check which new variable has been added:
old_locals = set(locals().keys())
exec(stmt)
new_locals = set(locals().keys())
varname = (new_locals-old_locals).pop()
How about using a small domain specific language as found in configparser to declare your new variables?
This way you don't need to run untrusted code, and get simple things like variable expansion (though probably in a slightly different syntax).
E.g. considering the following input
FOO = world
BAR = hello, #FOO#
as simple parser could look like:
lines=["FOO = world", "BAR = hello, #FOO#" ]
vars={}
# store variables in dictionary, with expansion
for line in lines:
key, value = [x.strip() for x in line.split('=', 1)]
for k in vars:
value=value.replace('#%s#' % (k), str(vars[k]))
vars[key]=value
# make variables available in local namespace
for k in vars:
locals()[k]=vars[k]
## use new variable
print(BAR)
There is some issues around locals() in Python 3 (see this post) so generally speaking, changes in locals() during runtime is not allowed.
So I made some workaround, where I defined my own namespace dictonary my_dict_locals and my_dict_globals. To my_dict_globals I copied some necessary definitions from current namespace (like SetCustomPropertyValue function definition ...). Then I just called
exec(each, my_dict_globals, my_dict_locals)
where each could be like one of following:
COMPBLOCK = StringLeft(MyPlatform.HierarchicalName,6) +"_IADI." + CMP + "." + BLOCK ;
SetCustomPropertyValue("DEVLM2",COMPBLOCK + ".DEVLM2",false);
SetCustomPropertyValue("CRIT",COMPBLOCK + ".CRIT",false);
SetCustomPropertyValue("UNACK",COMPBLOCK + ".UNACK",false);
SetCustomPropertyValue("ALMSTA_26",COMPBLOCK + ".ALMSTA#B26",false);`
This works for me as I expected and I have in my_dict_locals all variables from above script defined.
>>> list=[None]
>>> def list[0](x,y):
File "<stdin>", line 1
def list[0](x,y):
^
SyntaxError: invalid syntax
How can I define a function as an element of a list?
Python's def isn't flexible enough to handle generic lvalues such as list[0]. The language only allows you to use an identifier as function name. Here are the relevant parts of the grammar rule for the def-statement:
funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite
funcname ::= identifier
Instead, you can use a series of assignment and definition statements:
s = [None]
def f(x, y):
return x + y
s[0] = f
As an alternative, you could also store a lambda expression directly in a list:
s = [lambda x,y : x+y]
def f(whatever):
do_stuff()
l[0] = f
The function definition syntax doesn't allow you to define a function directly into a data structure, but you can just create the function and then assign it wherever it needs to go.
def someFunctionA(x, y):
return x+y
def someFunctionB(x, y):
return x*y
someList = [someFunctionA, someFunctionB]
print someList[0](2, 3)
print someList[1](5, 5)
Allowing such a freedom would make parsing harder... for example the parenthesized expression
...(x, y, z=3)
can be either a parameter declaration (where 3 is the default for keyword parameter z) or a call (that is passing z keyword parameter value as 3).
If you want to allow a generic assignable expression in def you also need to allow
def foo(x, y, z=3)[3](x, y, z=3):
...
where the first parenthesized part has a different semantic meaning and syntax rules from the second part.
Writing a parser for this is annoying (basically because you need to process an arbitrary unbounded amount of source code without understanding it) and is what for example lead to the worst parsing rule in the whole universe I know (it's the dreaded most vexing parse of C++) that basically just gave up on trying to get a decent language by resigning to ambiguity.
Note that in many cases when it's harder for a program to do the parsing it's because of ambiguity that would make also harder for a human to understand it.
Python correctly values readability as very important.
Functions in Python are however first class objects so you can solve your problem easily enough:
def foo(...):
...
mylist[index] = foo
or, only if the function is a single expression, with
mylist[index] = lambda ... : ...
(but lambda is very limited, both because it's sort of "hated" in the Python community and also because it would create some annoyance at the syntax level because of the need of handling indentation inside parenthesis).
Note also that something that a few Python novices don't know is that you can use def even inside a function; for example:
def register_http():
def handle_http(connection):
...
global_register['http'] = handle_http
that will assign a function as element of a global map without polluting the global (module) namespace with its name. A local def can also create a closure by capturing local state variables (read-only in 2.x or even read/write in 3.x).
Note also that if you need some processing of a function may be decorators can be useful. For example by defining
def register(name):
def do_registering(f):
global_register[name] = f
return f
return do_registering
you can just use
#register('http')
def handle_http(connection):
...
Is there a way to pass a list as a function argument to eval() Or do I have to convert it to a string and then parse it as a list in the function?
My simple example looks like:
eval("func1(\'" + fArgs + "\')")
I'm just not sure if there is a better way of taking fArgs as a list instead of a string
Note:
The list is provided from a JSON response
EDIT: Ok here's a bit more of my class so there's a better understanding of how I'm using eval
def test(arg):
print arg
#Add all allowed functions to this list to be mapped to a dictionary
safe_list = ['test']
safe_dict = dict([ (k, locals().get(k, None)) for k in safe_list ])
class Validate:
def __init__(self, Value, fName, fArgs):
eval(fName + "(\'" + fArgs + "\')", {"__builtins__":None},safe_dict)
I may be wrong in thinking this, but to my understanding this is a safe use of eval because the only functions that can be called are the ones that are listed in the safe_list dictionary. The function to be run and the arguments for that function are being extracted out of a JSON object. The arguments are to be structured as a list, Will joining the list together with ", " be interpreted as actual arguments or just a single argument?
If you're using Python 2.6.x, then you should be able to use the json module (see py doc 19.2). If not, then there is python-json available through the python package index. Both of these packages will provide a reader for parsing JSON data into an appropriate Python data type.
For your second problem of calling a function determined by a message, you can do the following:
def foo():
print 'I am foo!'
def bar():
pass
def baz():
pass
funcs = {'func_a':foo, 'func_b':bar, 'func_c':baz}
funcs['func_a']()
This approach can be a bit more secure than eval because it prevents 'unsafe' python library functions from being injected into the JSON. However, you still need to be cautious that the data supplied to your functions can't be manipulated to cause problems.
Specifying parameters the following way works:
root#parrot$ more test.py
def func1(*args):
for i in args:
print i
l = [1,'a',9.1]
func1(*l)
root#parrot$ python test.py
1
a
9.1
so, no direct need for eval(), unless I'm misunderstanding something.
Using a library to parse JSON input may be a better approach than eval, something like:
import json
func1(json.loads(fArgs))
Assert-ing that user input is correct would be a good idea, too.
The others have a good point, that you shouldn't be using eval. But, if you must:
eval("func1(%s)" % ", ".join(fArgs))
will call the function with all the arguments in the list. This:
eval("func1([%s])" % ", ".join(fArgs))
will call it with the list of arguments in just one argument. Maybe you even want this?
eval("func1([%s])" % ", ".join(map(eval, fArgs)))
which would eval the arguments as well?