I find it very useful to be able to create new variables during runtime and create a dictionary of the results for processing later, i.e. writing to a file:
myDict = {}
for i in range (1,10):
temp = "variable"+str(i)
vars()[temp] = myFunctionThatReturnsData() # variable1= data1, variable2 = data2,etc.
myDict[temp] = vars(temp)
which creates the dictionary entry [result1:data1] which i can call with myDict[result1]. I have been using vars() without really understanding what I'm doing. I take it vars() returns a dictionary with the local variables(?), and
vars()[x] = y
creates a new dictionary entry of [x:y] ?
I have a script where I pass in a dictionary prepared with {input1:data1,input2:data2}, and i use this method to iterate through all the values, store all the results, and output it to a file. This bit of code is inside a function within a class, and is working.
My source of confusion is that I have read various posts on how locals() shouldn't be messed with, and how vars() is equivalent(?) to locals(), or globals()..
So my question is (at least) two-fold:
1.What exactly does vars(),or in particular, vars()[x] = y do,
2.What the scope of this dictionary is (what I need to keep in mind as I write bigger programs
3.Whether this is good programming practice.
Thanks in advance!
The pythonic way to create a sequence of variables
If you want a sequence of variables, create a sequence. Instead of trying to create independent variables like:
variable0
variable1
variable2
variable3
You should look at creating a list. This is similar to what S.Lott is suggesting (S.Lott usually has good advice), but maps more neatly onto your for loop:
sequence = []
for _ in xrange(10):
sequence.append(function_that_returns_data())
(Notice that we discard the loop variable (_). We're just trying to get 10 passes.)
Then your data will be available as:
sequence[0]
sequence[1]
sequence[2]
sequence[3]
[...]
sequence[9]
As an added bonus, you can do:
for datum in sequence:
process_data(datum)
At first, you may twitch at having your sequence start at 0. You can go through various contortions to have your actual data start at 1, but it's more pain than it's worth. I recommend just getting used to having zero-based lists. Everything is built around them, and they start to feel natural pretty quickly.
vars() and locals()
Now, to answer another part of your question. vars() (or locals()) provides low level access to variables created by python. Thus the following two lines are equivalent.
locals()['x'] = 4
x = 4
The scope of vars()['x'] is exactly the same as the scope of x. One problem with locals() (or vars()) is that it will let you put stuff in the namespace that you can't get out of the namespace by normal means. So you can do something like this: locals()[4] = 'An integer', but you can't get that back out without using locals again, because the local namespace (as with all python namespaces) is only meant to hold strings.
>>> x = 5
>>> dir()
['__builtins__', '__doc__', '__name__', 'x']
>>> locals()[4] = 'An integer'
>>> dir()
[4, '__builtins__', '__doc__', '__name__', 'x']
>>> x
5
>>> 4
4
>>> locals()[4]
'An integer'
Note that 4 does not return the same thing as locals()[4]. This can lead to some unexpected, difficult to debug problems. This is one reason to avoid using locals(). Another is that it's generally a lot of complication just to do things that python provides simpler, less error prone ways of doing (like creating a sequence of variables).
Do this instead. It's simpler.
myDict = {}
for i in range (1,10):
temp = "variable"+str(i)
myDict[temp] = myFunctionThatReturnsData() # variable1= data1, variable2 = data2,etc.
That's all you ever need to do.
The results will be myDict['variable1'] through myDict['variable9']
You rarely need vars() or locals(). Just stop using them and use ordinary variables and ordinary dictionaries. Try to avoid things you don't understand and stick to the simple, obvious stuff.
From the help for vars,
vars(...)
vars([object]) -> dictionary
Without arguments, equivalent to locals().
With an argument, equivalent to object.__dict__.
You are using it without vars, so let's look at the help for locals()
locals(...)
locals() -> dictionary
Update and return a dictionary containing the current scope's local
variables.
So this answers you first two questions. vars() returns a dictionary to the local variables that is indexed by the name of the variable as a string. The scope is local.
I'm not sure about the third question, but it does seem like kind of a hack which isn't a good sign. I guess if you're careful about using this only in the correct scope you can get by with it.
jcdyer explains the concepts very well and Justin Peel clearly states what vars() and locals() do. But a small example always speeds up understanding.
class Bull(object):
def __init__(self):
self.x = 1
self.y = "this"
def __repr__(self):
return "Bull()"
def test1(self):
z = 5
return vars()
def test2(self):
y = "that"
return vars(self)
def test3(self):
return locals()
def test4(self):
y = 1
return locals()
if __name__ == "__main__":
b = Bull()
print b.test1()
print b.test2()
print b.test3()
print b.test4()
print vars(b).get("y")
Which results in:
{'self': Bull(), 'z': 5}
{'y': 'this', 'x': 1}
{'self': Bull()}
{'y': 1, 'self': Bull()}
this
I can answer number 3: this isn't good programming practice. I don't exactly see what you are trying to accomplish, but I am sure there is a more elegant way of doing it without using locals() (which is the same as vars() according to help(vars) in the interactive Python shell).
Using vars / locals or globals in this way is (a) poor practice and (b) doesn't work in all cases. See Dynamically set local variable for more details. Bottom line: just use dicts -- that's what they're for.
I also search for this answer at many places. The best one I see so far is:
var_list = ["Var_Name{}".format(i) for i in range(1,100)]
Related
After reading about vars(), I am still a little confused on what it is actually doing. In looking at the Python documentation, "The vars() function returns the __ dict __ attribute of the given object if the object has __ dict __ attribute." So, it returns a dictionary object? Here are the questions I am really getting at:
1) What is vars() actually doing in the below code?
dict = {}
dict['H'] = atom
vars()[dict['H']] = 20
2) Why is the vars() necessary in front of the dictionary I created and could I leave it out? (I know the code fails if I leave it out, but what would be a different way of accomplishing the same task?)
dict = {}
dict['H'] = atom
dict['H'] = 20
What is vars() actually doing in the below code?
It does what it always does: it retrieves all local variables as a dictionary. That's how it works when calling vars without arguments.
Why is the vars() necessary in front of the dictionary I created and could I leave it out?
It seems that someone is doing shenanigans with the local scope. It's just that it is wrapped with dict['H'] which is just some variable (possibly atom is a string). In other words he's trying to retrieve a variable by dynamic name. For example
>>> test = 15
>>> x = 'test'
>>> vars()[x]
15
This should be avoided at all costs. Forget you even know about the existance of vars(). I do not know a single case when you have to use vars() instead of a dict or some other "normal" (and in this context safe) structure. The code you are looking at is insanely difficult to maintain, error prone and has serious security issues.
1) What is vars() doing in the below code
Honestly, it's hard to tell. vars() (without an argument) returns the local namespace as a dictionary. In other words, it does the same thing as locals(). In your code, you then look up a name (dict['H']) in the locals dict and set an item in there. Setting an item in the dict returned by locals() does nothing if you are in a function... If you are in the global namespace, it adds an attribute.
2) Why is the vars() necessary in front of the dictionary I created and could I leave it out?
It probably isn't necessary and you probably should leave it out. Normally dynamically setting attributes on the global namespace is a really bad idea. You should generally pass around a dictionary of the data that you need instead. In other words, you can do:
v = 'foo'
globals()[v] = 'bar'
# ...
value = globals()[v] # 'bar'
But you shouldn't. It's much better to just pass a dictionary around:
v = 'foo'
dct = {v: 'bar'}
# ...
value = dct[v]
First, you shouldn't overwrite __builtin__ functions. dict is a reserved name for a function used to create dictionary objects.
Second, var gives you access to name declarations within the current running scope.
I have a list of items in python something like this:
input:
a=['nt','mt','pt']
I want to make each item in the above list as a variable and then assign that variable to a regex involving that variable.
output:
I want something like this:
nt=re.compile("%s=[a-z]+&" %nt)
mt=re.compile("%s=[a-z]+&" %mt)
pt=re.compile("%s=[a-z]+&" %pt)
how do i go about doing this ???
Thanks.
[sorry didn't pose the question in the best way possible ]
Keep data out of your variable names. Don't use variables, use a dictionary:
d = {name: re.compile("%s=[a-z]+&" % name) for name in a}
Unquestionably best to use dictionary keys, not variables. But FYI in Python variables are actually stored in a dictionary anyway, which you can access by calling vars() or locals(). That means it is possible to create variables dynamically just by assigning to this dictionary, e.g.:
>>> new_var_name = 'my_var'
>>> vars()[new_var_name] = "I'm your new variable!"
>>> print my_var
"I'm your new variable!"
To be honest I don't know how tampering with vars() could ever be justifiable. But it's interesting, at least. Anyway, use Sven Marnach's answer.
Say I have a simple function:
def myfunc():
a = 4.2
b = 5.5
... many similar variables ...
I use this function one time only and I am wondering what is the easiest way to make all the variables inside the function accessible to my main name-space.
Do I have to declare global for each item? or any other suggested methods?
Thanks.
Best way, in my biased opinion, is to wrap the dictionary into a nice object where the "variables" are accessed as attributes -- the pattern I named Bunch when I introduced it many years ago, and a great example of the last item in the Zen of Python (if you don't know what that is, import this at an interpreter interactive prompt). To wit...:
class Bunch(object):
def __init__(self, d=None):
if d is not None: self.__dict__.update(d)
def myfunc():
a = 4.2
b = 5.5
...
return Bunch(locals())
x = myfunc()
print x.a, x.b
Using qualified names like x.a and x.b, rather than barenames such as a and b, is the crucial idea here: qualified names let you separate namespaces and treat them right, while barenames would make one big soup of everything and violate the Zen of Python;-). And of course there's no need to use an unwrapped dict and unsightly x['a'], x['b'] accesses!-)
If what you want is separation of your constants then put them in another module:
consts.py:
foo = 42
bar = 'quux'
main.py:
import consts
print consts.foo
print consts.bar[::-1]
I can't think of any good reason for doing this, but if you REALLY want to for some strange reason:
def myfunc():
x = 5.6
y = 7.3
# many other variables
globals().update( locals() )
This works because the globals function returns a dictionary of the global variables in your module's namespace, which you can dynamically update with the dictionary of all local variables returned by the locals function.
This is very dangerous because it will clobber everything in your global namespace, so if you already had an x or y variable, then they'll get overwritten. And as I said, I really can't think of a good reason to ever do this, so definitely think twice before resorting to this kind of trickery!
It sounds like what you want to do isn't natural because it isn't something you should generally do much.
Perhaps what you really want is to make a class and set all these things on an instance of it.
class MyState(object):
def __init__(self):
self.a = 4.2
self.b = 5.5
... many similar variables ...
Storing state on an instance of a class you create is usually very strongly preferred to mutating global state. This method also does not require any magic.
I wonder if there is a good way to bind local variables in python. Most of my work involves cobbling together short data or text processing scripts with a series of expressions (when python permits), so defining object classes (to use as namespaces) and instantiating them seems a bit much.
So what I had in mind was something like in (common) lisp, where you could do something like
(setq data '(1 2 3))
(setq output
(let ( (x (nth 2 data)) )
x + x))
In python, the best I could come up with is
data = [1,2,3]
output = ((lambda x: x + x)
(data[2]))
These are, of course, very simple examples but might there be something that is as scalable as let or let* in lisp? Are defining classes the best way to go to create a local namespace?...(but feels a little less interactive that way)
Edit: So to further explain the intention (my apologies for vagueness), I want to reduce the use of global variables. So in the case above, I meant to use the extraction operator as a general case of any type of operation that might not want to be repeated. For instance, one might write either
output = data[2] + data[2]
or
x = data[2]
output = x + x
del x
to accomplish the same result. In essence, if the desired operation on 'data' is more complicated then getting the second item, I wouldn't want to type it out multiple times, or let the computer compute the value of the same expression more times than necessary. So in most cases one would assign the result of the operation, in this case, data[2], or operator.itemgetter(2)(data), to some variable in the global space, but I have an aversion to leaving variables around in the global space if they were only necessary to store intermediate values in a computation... hence the use of the 'del' command immediately afterwards. Defining a local environment or namespace and binding intermediate results to local variables would be an ideal alternative.
I can only second Lennart and Daniel - Python is not Lisp, and trying to write language X into language Y is usually inefficient and frustrating at best.
First point: your example code
data = [1,2,3]
output = ((lambda x: x + x)
(data[2]))
would be much more readable as:
data = [1, 2, 3]
output = (lambda x=data[2] : x +x)()
but anyway, in this concrete case, using a lambda is total overkill, overcomplexificated, and mostly inefficient. A braindead
output = data[2] + data[2]
would JustWork(tm) !-)
Now wrt/ to local bindings / namespaces, the usual solution is to use... functions - eventually nested. While 100% object (as in "everything is an object"), Python is not pure object, and plain functions are just fine. FWIW, even for "scripts", you should put your logic in a function then call it - function's local namespace access is faster than "global" (really: module level) namespace access. The canonical pattern is
import whatever
def some_func(args):
code_here
def some_other_func(args)
code_here
def main(args):
parse_args
some_func(something)
some_other_func(something_else)
return some_exit_code
if __name__ == '__main__'
import sys
sys.exit(main(sys.argv))
Note also that nested functions can also access the enclosing namespace, ie
def main():
data = [1, 2, 3]
def foo():
x = data[2]
return x + x
print foo()
data = [4, 5, 6]
print foo()
# if you want the nested function to close over its arguments:
def bar(data=data):
x = data[2]
return x + x
print bar()
data = [7, 8, 9]
print bar()
HTH
It's a bit unclear what you are asking, bit I'll try to answer anyway:
You bind variables to names with = in Python. So your data = [1,2,3] binds the list [1,2,3] to the name data.
You can create local namespaces with classes and functions/methods.
The closest you get so something as powerful as let is probably def and lambda. Python is (despite where some people try to tell you) not Lisp, and not particularly functional, so you will have to adapt your mindset a bit.
Update: Ah, I see what you mean now.
All variables are pretty much local in Python. The nearest you get to global variables are variables defined in module space, because you can access them with from <module> import <variable>. You also can access them from wherever in the module, but not modify them (unless you say that you want to modify them with the global keyword. Anything you define in a function/method or class definition, will only be accessible from that namespace.
So in short: you don't have to worry about the things you worry about now. Python takes care of it for you. :)
You could combine a function decorator and default parameters to get something like let and block scoped variables:
def let(func):
return func()
data = [1,2,3]
#let
def output(x=data[2]):
return x + x
print(output) # 6
# or if a single expression is enough:
output = let(lambda x=data[2]: x+x)
But this isn't a popular idiom in Python so I advise you avoid it to make your code easier to understand for others. Just use regular local variables:
data = [1,2,3]
x = data[2]
output = x + x
If this becomes a real problem it's a good sign you are trying to do too much in a single function.
Not really knowing Lisp, I can't see what you're trying to do here. But I would say that in general you should not try to write Python as if it were Lisp, or indeed any language as if it were any other language. I've been programming in Python for five years and I've never seen a need to do what you're trying above.
Can you give an example of a use case for the above - what are you actually trying to do, in terms of the end result? Maybe then we can advise you on the best way to do it in Python, rather than Lisp.
I have a function. Inside that I'm maintainfing a dictionary of values.
I want that dictionary to be maintained between different function calls
Suppose the dic is :
a = {'a':1,'b':2,'c':3}
At first call,say,I changed a[a] to 100
Dict becomes a = {'a':100,'b':2,'c':3}
At another call,i changed a[b] to 200
I want that dic to be a = {'a':100,'b':200,'c':3}
But in my code a[a] doesn't remain 100.It changes to initial value 1.
I need an answer ASAP....I m already late...Please help me friends...
You might be talking about a callable object.
class MyFunction( object ):
def __init__( self ):
self.rememberThis= dict()
def __call__( self, arg1, arg2 ):
# do something
rememberThis['a'] = arg1
return someValue
myFunction= MyFunction()
From then on, use myFunction as a simple function. You can access the rememberThis dictionary using myFunction.rememberThis.
You could use a static variable:
def foo(k, v):
foo.a[k] = v
foo.a = {'a': 1, 'b': 2, 'c': 3}
foo('a', 100)
foo('b', 200)
print foo.a
Rather than forcing globals on the code base (that can be the decision of the caller) I prefer the idea of keeping the state related to an instance of the function. A class is good for this but doesn't communicate well what you are trying to accomplish and can be a bit verbose. Taking advantage of closures is, in my opinion, a lot cleaner.
def function_the_world_sees():
a = {'a':1,'b':2,'c':3}
def actual_function(arg0, arg1):
a[arg0] = arg1
return a
return actual_function
stateful_function = function_the_world_sees()
stateful_function("b", 100)
stateful_function("b", 200)
The main caution to keep in mind is that when you make assignments in "actual_function", they occur within "actual_function". This means you can't reassign a to a different variable. The work arounds I use are to put all of my variables I plan to reassign into either into a single element list per variable or a dictionary.
If 'a' is being created inside the function. It is going out of scope. Simply create it outside the function(and before the function is called). By doing this the list/hash will not be deleted after the program leaves the function.
a = {'a':1,'b':2,'c':3}
# call you funciton here
This question doesn't have an elegant answer, in my opinion. The options are callable objects, default values, and attribute hacks. Callable objects are the right answer, but they bring in a lot of structure for what would be a single "static" declaration in another language. Default values are a minor change to the code, but it's kludgy and can be confusing to a new python programmer looking at your code. I don't like them because their existence isn't hidden from anyone who might be looking at your API.
I generally go with an attribute hack. My preferred method is:
def myfunct():
if not hasattr(myfunct, 'state'): myfunct.state = list()
# access myfunct.state in the body however you want
This keeps the declaration of the state in the first line of the function where it belongs, as well as keeping myfunct as a function. The downside is you do the attribute check every time you call the function. This is almost certainly not going to be a bottleneck in most code.
You can 'cheat' using Python's behavior for default arguments. Default arguments are only evaluated once; they get reused for every call of the function.
>>> def testFunction(persistent_dict={'a': 0}):
... persistent_dict['a'] += 1
... print persistent_dict['a']
...
>>> testFunction()
1
>>> testFunction()
2
This isn't the most elegant solution; if someone calls the function and passes in a parameter it will override the default, which probably isn't what you want.
If you just want a quick and dirty way to get the results, that will work. If you're doing something more complicated it might be better to factor it out into a class like S. Lott mentioned.
EDIT: Renamed the dictionary so it wouldn't hide the builtin dict as per the comment below.
Personally, I like the idea of the global statement. It doesn't introduce a global variable but states that a local identifier actually refers to one in the global namespace.
d = dict()
l = list()
def foo(bar, baz):
global d
global l
l.append(bar, baz)
d[bar] = baz
In python 3.0 there is also a "nonlocal" statement.