Develop Reference

How can I dynamically define a function from user input?

I'm trying to dynamically generate a function from user input. The user gives a function as a string input, gives the differents parameters, and I want to write a function that will change this string into a usable function. My idea so far was to use the exec function to create a lambda function constructed with a string like so : exec("f = lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string)). This would give, for example, a string like "f = lambda x, a:x+a" to the exec function.
This technique works fine if I define the scope of exec to globals(), but I'd like my function to be local so that I can do something like this:
def define_function(function_string, parameter_list):
exec("f = lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string))
return f
or like this :
def define_function(function_string, parameter_list):
exec("return lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string))
However, in the first case, it gives an error along the lines of "f is not defined". In the second case, meanwhile, it says that "return can only be used in a function". I understand why the second error occurs, but I wonder why my first try didn't work. Using globals() makes it work, but I'd rather not use it.
If you have any suggestions for other methods for achieving the same result, I'm interested in those as well.
EDIT
Using eval also raises an error :
line 9, in define_function
eval("f = lambda {}:{}".format(', '.join(['x'] + parameter_list), function_string))
File "<string>", line 1
f = lambda x, a:x+a
^
SyntaxError: invalid syntax

Solution 1: use eval():
def get_func(param_list, result_str):
return eval("lambda {}: {}".format(', '.join(['x'] + param_list), result_str))
Solution 2: if you really want to use exec(), do something like this:
def get_func(param_list, result_str):
exec("f = lambda {}: {}".format(', '.join(['x'] + param_list), result_str))
return locals()['f']
Some good info on the differences between exec and eval here: What's the difference between eval, exec, and compile?
I can see what you're trying to do in your question, by the way, and it's worth noting that the documentation pages for exec() and locals() both explicitly warn that this won't work. Ditto, if you type help(locals) into the interactive interpreter, you get this message:
Help on built-in function locals in module builtins:
locals()
Return a dictionary containing the current scope's local variables.
NOTE: Whether or not updates to this dictionary will affect name lookups in
the local scope and vice-versa is *implementation dependent* and not
covered by any backwards compatibility guarantees.

In a Python atexit function, strings set inside a function aren't visible?

I ran into (what seems to me like) an unusual scope problem with atexit under Python 2.7.10 yesterday (though the problem is also in Python 3). A global string set inside a function seems not to be visible to my atexit function, whereas global lists are fine. I've reduced the problem case to this:
import atexit
myString1 = None
myString2 = None
myArray1 = []
myArray2 = []
def setString1(newString):
myString1 = newString
def setArray1(newString):
myArray1.append(newString)
def testFinishUp():
print("myString1 = " + str(myString1))
print("myString2 = " + str(myString2))
print("myArray1 = " + str(myArray1))
print("myArray2 = " + str(myArray2))
setString1("Hello")
myString2 = "World"
setArray1("HELLO")
myArray2.append("WORLD")
atexit.register(testFinishUp)
What this outputs is:
myString1 = None
myString2 = World
myArray1 = ['HELLO']
myArray2 = ['WORLD']
Now, I'm sure that there's a logical scope-related reason why myString1 isn't behaving in the same way as all the others, but I for one can't see what it is. :-(
Can anyone please explain what is going on here? Thanks!

There's nothing strange about this. Your problem is that this statement:
A global string set inside a function seems not to be visible to my atexit function, whereas global lists are fine.
is false. You are not setting the global string inside the function. Instead, you create a local version of it inside the function scope that shadows the global one. If you actually want to change the global value, you should do this:
def setString1(newString):
global myString1
myString1 = newString
The reason that it works for your lists is that you modify them in-place (via append) instead of creating a copy. If you would instead do
def setArray1(newString):
myArray1 = myArray1 + [newString]
you would see the exact same behavior as in the string case.
A word of warning though: Modifying global state from functions is often considered bad style, since it obfuscates where values are set. Usually, it is better to modify the values in global scope, e.g. by assigning to the result of a function call.

If you want to manipulate a global immutable var you need to add global var in that function
In your case, you are adding something to the array, so the array reference does not get manipulated, thus you don't need to specify, that this is a global var, but the string would get manipulated itself.
adding: global myString1
above: myString1 = newString
should solve the problem.
for further reading see python doc

Alternative to eval to create a generic function that can process multiple variables

For a set of games I'm creating I have a line a code that allows a question to be answered only once. If it has been answered, it adds points to a player's score (the code below sits inside of an if function that checks the answer) and then shuts off the ability to answer the question again. Here's the code I'm currently using:
while self.game1_question1_not_answered:
self.game1_player1_score += self.score_increment
self.game1_question1_not_answered = False`
I would like to use the 'game1' in the code as a generic identifier that can be used to identify any one of the multiple games I'm creating. I tried using a variable called game_name (e.g., game_name = game1) and inserting the variable into the code using an eval function but haven't gotten the code to work. In addition, I realize the eval function has some security concerns. What function could I use to get this to work? The code I've tried that doesn't work looks like this:
while eval('self.' + game_name + 'question1_not_answered'):
eval('self.' + game_name + 'player1_score') += self.score_increment
eval('self.' + game_name + 'question1_not_answered') = False
Is there another function I could use instead of eval to get this to work?

You should use a dict instead. That will allow you to create dynamically-named variables, as it were, to store that information.
self.game_dict = {}
self.game_dict[game_name + 'question1_not_answered'] = True
Then you can modify it as above, and access it in a couple ways:
>>> game_obj.game_dict.get(game_name + 'question1_not_answered')
True
>>> game_obj.game_dict[game_name + 'question1_not_answered']
True
But as jonrsharpe said, your variable names should not include data. The best solution would be to make multiple game objects, each with variables like question1_not_answered. Then if you need to, assign all of those objects to variables in whatever self is in this case.

I'd actually follow #jonrsharpe's comment, but if you really need to have them in one instance, you can use getattr and setattr builtins:
game_name = "game1"
question1_na_line = "{}_question1_not_answered"
player1_sc_line = "{}_player1_score"
question1_na = question1_na_line.format(game_name)
player1_sc = player1_sc_line.format(game_name)
while getattr(self, question1_na):
setattr(self,
player1_sc,
getattr(self, player1_sc) + self.score_increment)
setattr(self, question1_na, False)

The usual way to access attributes by name is the getattr(obj, name) function. Now given the more specific problem you describe I think you'd be better using a dict (or dict of dicts etc) to store your games states (scores etc).

Python - Pass variable handle to evaluate

I am writing some program using python and the z3py module.
What I am trying to do is the following: I extract a constraint of an if or a while statement from a function which is located in some other file. Additionally I extract the used variables in the statement as well as their types.
As I do not want to parse the constraint by hand into a z3py friendly form, I tried to use evaluate to do this for me. Therefore I used the tip of the following page: Z3 with string expressions
Now the problem is: I do not know how the variables in the constraint are called. But it seems as I have to name the handle of each variable like the actual variable. Otherwise evaluate won't find it. My code looks like this:
solver = Solver()
# Look up the constraint:
branch = bd.getBranchNum(0)
constr = branch.code
# Create handle for each variable, depending on its type:
for k in mapper.getVariables():
var = mapper.getVariables()[k]
if k in constr:
if var.type == "intNum":
Int(k)
else:
Real(k)
# Evaluate constraint, insert the result and solve it:
f = eval(constr)
solver.insert(f)
solve(f)
As you can see I saved the variables and constraints in classes. When executing this code I get the following error:
NameError: name 'real_x' is not defined
If I do not use the looping over the variables, but instead the following code, everything works fine:
solver = Solver()
branch = bd.getBranchNum(0)
constr = branch.code
print(constr)
real_x = Real('real_x')
int_y = Int('int_y')
f = eval(constr)
print(f)
solver.insert(f)
solve(f)
The problem is: I do not know, that the variables are called "real_x" or "int_y". Furthermore I do not know how many variables there are used, which means I have to use some dynamic thing like a loop.
Now my question is: Is there a way around this? What can I do to tell python that the handles already exist, but have a different name? Or is my approach completely wrong and I have to do something totally different?

This kind of thing is almost always a bad idea (see Why eval/exec is bad for more details), but "almost always" isn't "always", and it looks like you're using a library that was specifically designed to be used this way, in which case you've found one of the exceptions.
And at first glance, it seems like you've also hit one of the rare exceptions to the Keep data out of your variable names guideline (also see Why you don't want to dynamically create variables). But you haven't.
The only reason you need these variables like real_x to exist is so that eval can see them, right? But the eval function already knows how to look for variables in a dictionary instead of in your global namespace. And it looks like what you're getting back from mapper.getVariables() is a dictionary.
So, skip that whole messy loop, and just do this:
variables = mapper.getVariables()
f = eval(constr, globals=variables)
(In earlier versions of Python, globals is a positional-only argument, so just drop the globals= if you get an error about that.)
As the documentation explains, this gives the eval function access to your actual variables, plus the ones the mapper wants to generate, and it can do all kinds of unsafe things. If you want to prevent unsafe things, do this:
variables = dict(mapper.getVariables())
variables['__builtins__'] = {}
f = eval(constr, globals=variables)

Python string interpolation implementation

[EDIT 00]: I've edited several times the post and now even the title, please read below.
I just learned about the format string method, and its use with dictionaries, like the ones provided by vars(), locals() and globals(), example:
name = 'Ismael'
print 'My name is {name}.'.format(**vars())
But I want to do:
name = 'Ismael'
print 'My name is {name}.' # Similar to ruby
So I came up with this:
def mprint(string='', dictionary=globals()):
print string.format(**dictionary)
You can interact with the code here:
http://labs.codecademy.com/BA0B/3#:workspace
Finally, what I would love to do is to have the function in another file, named my_print.py, so I could do:
from my_print import mprint
name= 'Ismael'
mprint('Hello! My name is {name}.')
But as it is right now, there is a problem with the scopes, how could I get the the main module namespace as a dictionary from inside the imported mprint function. (not the one from my_print.py)
I hope I made myself uderstood, if not, try importing the function from another module. (the traceback is in the link)
It's accessing the globals() dict from my_print.py, but of course the variable name is not defined in that scope, any ideas of how to accomplish this?
The function works if it's defined in the same module, but notice how I must use globals() because if not I would only get a dictionary with the values within mprint() scope.
I have tried using nonlocal and dot notation to access the main module variables, but I still can't figure it out.
[EDIT 01]: I think I've figured out a solution:
In my_print.py:
def mprint(string='',dictionary=None):
if dictionary is None:
import sys
caller = sys._getframe(1)
dictionary = caller.f_locals
print string.format(**dictionary)
In test.py:
from my_print import mprint
name = 'Ismael'
country = 'Mexico'
languages = ['English', 'Spanish']
mprint("Hello! My name is {name}, I'm from {country}\n"
"and I can speak {languages[1]} and {languages[0]}.")
It prints:
Hello! My name is Ismael, I'm from Mexico
and I can speak Spanish and English.
What do you think guys? That was a difficult one for me!
I like it, much more readable for me.
[EDIT 02]: I've made a module with an interpolate function, an Interpolate class and an attempt for a interpolate class method analogous to the function.
It has a small test suite and its documented!
I'm stuck with the method implementation, I don't get it.
Here's the code: http://pastebin.com/N2WubRSB
What do you think guys?
[EDIT 03]: Ok I have settled with just the interpolate() function for now.
In string_interpolation.py:
import sys
def get_scope(scope):
scope = scope.lower()
caller = sys._getframe(2)
options = ['l', 'local', 'g', 'global']
if scope not in options[:2]:
if scope in options[2:]:
return caller.f_globals
else:
raise ValueError('invalid mode: {0}'.format(scope))
return caller.f_locals
def interpolate(format_string=str(),sequence=None,scope='local',returns=False):
if type(sequence) is str:
scope = sequence
sequence = get_scope(scope)
else:
if not sequence:
sequence = get_scope(scope)
format = 'format_string.format(**sequence)'
if returns is False:
print eval(format)
elif returns is True:
return eval(format)
Thanks again guys! Any opinions?
[EDIT 04]:
This is my last version, it has a test, docstrings and describes some limitations I've found:
http://pastebin.com/ssqbbs57
You can quickly test the code here:
http://labs.codecademy.com/BBMF#:workspace
And clone grom git repo here:
https://github.com/Ismael-VC/python_string_interpolation.git

Modules don't share namespaces in python, so globals() for my_print is always going to be the globals() of my_print.py file ; i.e the location where the function was actually defined.
def mprint(string='', dic = None):
dictionary = dic if dic is not None else globals()
print string.format(**dictionary)
You should pass the current module's globals() explicitly to make it work.
Ans don't use mutable objects as default values in python functions, it can result in unexpected results. Use None as default value instead.
A simple example for understanding scopes in modules:
file : my_print.py
x = 10
def func():
global x
x += 1
print x
file : main.py
from my_print import *
x = 50
func() #prints 11 because for func() global scope is still
#the global scope of my_print file
print x #prints 50

Part of your problem - well, the reason its not working - is highlighted in this question.
You can have your function work by passing in globals() as your second argument, mprint('Hello my name is {name}',globals()).
Although it may be convenient in Ruby, I would encourage you not to write Ruby in Python if you want to make the most out of the language.

Language Design Is Not Just Solving Puzzles: ;)
http://www.artima.com/forums/flat.jsp?forum=106&thread=147358
Edit: PEP-0498 solves this issue!
The Template class from the string module, also does what I need (but more similar to the string format method), in the end it also has the readability I seek, it also has the recommended explicitness, it's in the Standard Library and it can also be easily customized and extended.
http://docs.python.org/2/library/string.html?highlight=template#string.Template
from string import Template
name = 'Renata'
place = 'hospital'
job = 'Dr.'
how = 'glad'
header = '\nTo Ms. {name}:'
letter = Template("""
Hello Ms. $name.
I'm glad to inform, you've been
accepted in our $place, and $job Red
will ${how}ly recieve you tomorrow morning.
""")
print header.format(**vars())
print letter.substitute(vars())
The funny thing is that now I'm getting more fond of using {} instead of $ and I still like the string_interpolation module I came up with, because it's less typing than either one in the long run. LOL!
Run the code here:
http://labs.codecademy.com/BE3n/3#:workspace