This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 6 months ago.
The following code spits out 1 twice, but I expect to see 0 and then 1.
def pv(v) :
print v
x = []
for v in range(2):
x.append(lambda : pv(v))
for xx in x:
xx()
I expected python lambdas to bind to the reference a local variable is pointing to, behind the scenes. However that does not seem to be the case. I have encountered this problem in a large system where the lambda is doing modern C++'s equivalent of a bind ('boost::bind' for example) where in such case you would bind to a smart ptr or copy construct a copy for the lambda.
So, how do I bind a local variable to a lambda function and have it retain the correct reference when used? I'm quite gobsmacked with the behaviour since I would not expect this from a language with a garbage collector.
Change x.append(lambda : pv(v)) to x.append(lambda v=v: pv(v)).
You expect "python lambdas to bind to the reference a local variable is pointing to, behind the scene", but that is not how Python works. Python looks up the variable name at the time the function is called, not when it is created. Using a default argument works because default arguments are evaluated when the function is created, not when it is called.
This is not something special about lambdas. Consider:
x = "before foo defined"
def foo():
print x
x = "after foo was defined"
foo()
prints
after foo was defined
The lambda's closure holds a reference to the variable being used, not its value, so if the value of the variable later changes, the value in the closure also changes. That is, the closure variable's value is resolved when the function is called, not when it is created. (Python's behavior here is not unusual in the functional programming world, for what it's worth.)
There are two solutions:
Use a default argument, binding the current value of the variable to a local name at the time of definition. lambda v=v: pv(v)
Use a double-lambda and immediately call the first one. (lambda v: lambda: pv(v))(v)
Related
Im trying to build a calculator with PyQt4 and connecting the 'clicked()' signals from the buttons doesn't work as expected.
Im creating my buttons for the numbers inside a for loop where i try to connect them afterwards.
def __init__(self):
for i in range(0,10):
self._numberButtons += [QPushButton(str(i), self)]
self.connect(self._numberButtons[i], SIGNAL('clicked()'), lambda : self._number(i))
def _number(self, x):
print(x)
When I click on the buttons all of them print out '9'.
Why is that so and how can i fix this?
This is just, how scoping, name lookup and closures are defined in Python.
Python only introduces new bindings in namespace through assignment and through parameter lists of functions. i is therefore not actually defined in the namespace of the lambda, but in the namespace of __init__(). The name lookup for i in the lambda consequently ends up in the namespace of __init__(), where i is eventually bound to 9. This is called "closure".
You can work around these admittedly not really intuitive (but well-defined) semantics by passing i as a keyword argument with default value. As said, names in parameter lists introduce new bindings in the local namespace, so i inside the lambda then becomes independent from i in .__init__():
self._numberButtons[i].clicked.connect(lambda checked, i=i: self._number(i))
UPDATE: clicked has a default checked argument that would override the value of i, so it must be added to the argument list before the keyword value.
A more readable, less magic alternative is functools.partial:
self._numberButtons[i].clicked.connect(partial(self._number, i))
I'm using new-style signal and slot syntax here simply for convenience, old style syntax works just the same.
You are creating closures. Closures really capture a variable, not the value of a variable. At the end of __init__, i is the last element of range(0, 10), i.e. 9. All the lambdas you created in this scope refer to this i and only when they are invoked, they get the value of i at the time they are at invoked (however, seperate invocations of __init__ create lambdas referring to seperate variables!).
There are two popular ways to avoid this:
Using a default parameter: lambda i=i: self._number(i). This work because default parameters bind a value at function definition time.
Defining a helper function helper = lambda i: (lambda: self._number(i)) and use helper(i) in the loop. This works because the "outer" i is evaluated at the time i is bound, and - as mentioned before - the next closure created in the next invokation of helper will refer to a different variable.
Use the Qt way, use QSignalMapper instead.
I couldn't understand global and local variables in python, especially functions. I look lots of examples, but all of them don't explain in function calls clearly. My example code is below:
def called(d):
del d[0]
b=[1]
return b
def main():
a=[0,1,2,3]
print("first ", a)
c=called(a)
print("second ", a)
main()
Output:
first [0, 1, 2, 3]
second [1, 2, 3]
I expect that "a" is local variable at main. When I call the "called" function a is copyed to d. And d is local at "called". But reality is different. I solve the problem with a "called(a.copy)". But I want to understand the logic.
Python never implicitely copies anything, when you pass an object (and everything in Python is an object) to a function what you have in the function IS the object you passed in, so if you mutate it the change will be seen outside the function.
OTHO, parameters names are local to the function so rebinding a parameter within a function only change what object this name points to within the function, it has no impact on the object passed.
For more in-depth explanation the best reference is Ned Batchelder's article here.
Basically, a global variable is one that can be accessed anywhere, regardless of whether or not it is in a function (or anything else).
A local variable is one that solely exists within the function in question. You cannot declare or access it anywhere else. You could, however, make it global by explicitly including that in the function.
As far as I can see,'d' has not really been defined as a variable, but as a function parameter, as seen in called(d). This could've been changed to called(blabla) and your function would behave the same exact way if you, within the function, also changed d[0] to blabla[0].
What this means is that when you call that function, anything that has 'd' in it would be replaced by what you're calling it with. In this case, the parameter has been changed to the variable a, and the functions are then executed as you've stated.
In order to define a variable, you have to use '='.
This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
What do lambda function closures capture?
(7 answers)
Closed 6 months ago.
I was searching Stack Overflow for a solution to a problem of mine when I stumbled upon this user submitted solution (to someone else's question):
appsMenu.add_command(label=app, command=openApp(Apps[app]))
Command parameters that call functions need to be wrapped in a lambda, to prevent them from being called right away. Additionally, commands bound within a for loop need the looping variable as a default argument, in order for it to bind the right value each time.
appsMenu.add_command(label=app, command=lambda app=app: openApp(Apps[app]))
As you can see, the user wrote lambda app=app, and for the love of me I cannot figure out why. What does it do?
The lambda expression is used to define a one-line (anonymous) function. Thus, writing
f = lambda x: 2*x
is (more or less) equivalent to
def f(x):
return 2*x
Further, as for normal functions, it is possible to have default arguments also in a lambda function. Thus
k = 2
f = lambda x=k: 2*x
can be called either as f() where x will be the value of k i.e. 2 or as f(3) where x = 3. Now, to make it even more strange, since the lambda function has its own name space, it is possible to exchange the k with x as
x = 2
f = lambda x=x: 2*x
which then means that the default value of f() will be the (outer) x value (i.e. 2) if no other parameter is set.
Thus, the expression
lambda app=app: openApp(Apps[app])
is a function (unnamed) that takes one argument app with the default value of the (outer) variable app. And when called, it will execute the instruction openApp(Apps[app]) where Apps is a global (outer) object.
Now, why would we want that, instead of just writing command=openApp(Apps[app]) when calling .add_command()? The reason is that we want the command to execute not when defining the command, i.e., when calling .add_command(), but at a later time, namely when the menu item is selected.
Thus, to defer to the execution we wrap the execution of openApp(Apps[app]) in a function. This function will then be executed when the menu item is selected. Thereby we will call openApp then, instead of right now (at the time we want to add the command) which would be the case otherwise.
Now, since the lambda function is called with no arguments, it will use its default parameter (outer) app as (inner) app. That makes the whole argument a bit redundant, as the outer variable is directly accessible within the lambda expression. Therefore, it is possible to simplify the expression to lambda: openApp(Apps[app]) i.e. wiht no arguments, making the full line
appsMenu.add_command(label=app, command=lambda: openApp(Apps[app]))
To me, this looks a bit simpler, and is the way I would write it, but conceptually there is no difference between this and the code with default parameters.
This question already has answers here:
UnboundLocalError trying to use a variable (supposed to be global) that is (re)assigned (even after first use)
(14 answers)
Why does code like `str = str(...)` cause a TypeError, but only the second time?
(20 answers)
Closed 6 months ago.
Could anyone explain the exception the below code. It only works when I change the var sub in the display() to another name. There is no global variable sub as well. So what happened ?
def sub(a, b):
return a - b
def display():
sub = sub(2,1) // if change to sub1 or sth different to sub, it works
print sub
Any variable you assign to inside a scope is treated as a local variable (unless you declare it global, or, in python3, nonlocal), which means it is not looked up in the surrounding scopes.
A simplified example with the same error:
def a(): pass
def b(): a = a()
Now, consider the different scopes involved here:
The global namespace contains a and b.
The function a contains no local variables.
The function b contains an assignment to a - this means it is interpreted as a local variable and shadows the function a from the outer scope (in this case, the global scope). As a has not been defined inside of b before the call, it is an unbound local variable, hence the UnboundLocalError. This is exactly the same as if you had written this:
def b(): x = x()
The solution to this is simple: choose a different name for the result of the sub call.
It is important to note that the order of use and assignment makes no difference - the error would have still happened if you wrote the function like this:
def display():
value = sub(2,1) #UnboundLocalError here...
print value
sub = "someOtherValue" #because you assign a variable named `sub` here
This is because the list of local variables is generated when the python interpreter creates the function object.
This was originally a comment. The OP found this useful as an answer. Therefore, I am re-posting it as an answer
Initially, sub is a function. Then, it becomes the return value of a function. So when you say print sub, python doesn't know which sub you are referring to.
Edit:
First you define a function sub. Now, python knows what sub is.
When you create a variable and try to assign to it (say x = 2), python evaluates the stuff on the right hand side of the = and assigns the value of the evaluation as the value of the stuff on the left hand side of the =. Thus, everything on the right hand side should actually compute.
So if your statement was x = x+1, then x better have a value assigned to it before that line; and the previously defined x has to be of some type compatible with the addition of 1.
But suppose x is a function, and you make a variable called x in some other function, and try to assign to it, a value computed with function x, then this really starts to confuse python about which x you are referring to. This is really an oversimplification of this answer, which does a much better job of explaining variable scope and shadowing in python functions
For every variable used, Python determines whether it is a local or a nonlocal variable. Referencing a unknown variable marks it as nonlocal. Reusing the same name as a local variable later is considered a programmers mistake.
Consider this example:
def err():
print x # this line references x
x = 3 # this line creates a local variable x
err()
This gives you
Traceback (most recent call last):
File "asd.py", line 5, in <module>
err()
File "asd.py", line 2, in err
print x # this line references x
UnboundLocalError: local variable 'x' referenced before assignment
What happens is basically that Python keeps track of all references to names in code. When it reads the line print x Python knows that x is a variable from a outer scope (upvalue or global). However, in x = 3 the x is used as a local variable. As this is a inconsistency in the code, Python raises an UnboundLocalError to get the Programmers attention.
Python start executing your code and get the function first
def sub(a, b):
return a - b
So after executing this interpreter get the sub as a function. Now when come to next line it found
def display():
sub = sub(2,1) // if change to sub1 or sth different to sub, it works
print sub
so first line sub = sub (2, 1) will convert the sub function to sub variable. From this function you are returning the sub variable. So its create problem.
This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 8 years ago.
Refer to this
>>> def foo(counter=[0]):
... counter[0] += 1
... print("Counter is %i." % counter[0]);
...
>>> foo()
Counter is 1.
>>> foo()
Counter is 2.
>>>
Default values are initialized only when the function is first evaluated, not each time it is executed, so you can use a list or any other mutable object to maintain static values.
Question> Why the counter can keep its updated value during different callings? Is it true that counter refers to the same memory used to store the temporary list of the default parameter so that it can refer to the same memory address and keep the updated values during the calls?
The object created as the default argument becomes part of the function and persists until the function is destroyed.
First things first: if you are coding in Python: forget about "memory address" - you will never need one. Yes, there are objects, and they are placed in memory, and if you are referring to the same object, it is in the same "memory address" - but that does not matter - there could even be an implementation where objects don't have a memory address at all (just a place in a data structure, for example).
Then, when Python encounters the function body, as it is defined above, it does create a code object with the contents of the function body, and executes the function definition line - resolving any expressions inlined there and setting the results of those expressions as the default parameters for that function. There is nothing "temporary" about these objects. The expressions (in this case [0]) are evaluated, the resulting objects (in this case a Python list with a single element) are created, and assigned to a reference in the function object (a position in the functions's "func_defaults" attribute - remember that functions themselves are objects in Python.)
Whenever you call that function, if you don't pass a value to the counter parameter, it is assigned to the object recorded in the func_defaults attribute -in this case, a Python list. And it is the same Python list that was created at function parsing time.
What happens is that Python lists themselves are mutable: one can change its contents, add more elements, etc...but they are still the same list.
What the code above does is exactly incrementing the element in the position 0 of the list.
You can access this list at any time in the example above by typing foo.func_defaults[0]
If you want to "reset" the counter, you can just do: foo.func_defaults[0][0]=0, for example.
Of course, it is a side effect of how thigns a reimplemented in Python, and though consistent, and even docuemnted, should not be used in "real code". If you need "static variables", use a class instead of a function like the above:
class Foo(object):
def __init__(self):
self.counter = 0
def __call__(self):
self.counter += 1
print("Counter is %i." % self.counter)
And on the console:
>>> foo = Foo()
>>> foo()
Counter is 1.
>>> foo()
Counter is 2.
>>>
Default arguments are evaluated at the function definition, and not its calls. When the foo function object is created (remember, functions are first class citizens in python) its arguments, including default arguments, are all local names bound to the function object. The function object is created when the def: statement is first encountered.
In this case counter is set to a mutable list in the definition of foo(), so calling it without an argument gives the original mutable list instantiated at definition the name counter. This makes calls to foo() use and modify the original list.
Like all function arguments, counter is a local name defined when the function is declared. The default argument, too, is evaluated once when the function is declared.
If, when calling foo(), no value is passed for counter, that default value, the exact object instance provided at function definition time, is given the name counter. Since this is a mutable object (in this case, a list), any changes made to it remain after the function has completed.
The function contains a reference to the list in its default argument tuple, func_defaults, which prevents it from being destroyed.