This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 7 years ago.
This is covered by the Python Tutorial, but I still don't quite understand the reason why Python has this style. Is it purely convention, or is there some explanation behind why Python has the following style regarding default parameters:
My understanding is that Python prefers something=None as opposed to something=[] for default parameters for functions. But...why not use something=[]? Surely this is the convention in other languages, like C
As an example, take these two examples, which are equivalent
def function(arr, L=[]):
L.append(arr)
return L
and
def function(arr, L=None):
if L is None:
L = []
L.append(arr)
return L
My understanding is that the first is "incorrect style" for Python. Why?
EDIT: Ah, I finally understand. I am incorrect above: the two functions are NOT equivalent. Default arguments are evaluated once when the function is defined, not each time the function is called!
When you set a parameter to the value of a list, it is assigned when the function is defined, not when it is called. That is why you'll get different results for calling the function multiple times with the same input parameter. Beware!!!
def function(arr, L=[]):
L.append(arr)
return L
arr = [1, 2, 3]
>>> function(arr)
[[1, 2, 3]]
>>> function(arr)
[[1, 2, 3], [1, 2, 3]]
A default value that's an empty list will refer to one specific variable each time you call it, instead of creating a new empty list every time.
>>> def add(defList=[]):
defList.append(1)
return defList
>>> add()
[1]
>>> add()
[1,1]
It's a quirk of how mutable data works in Python. It could be occasionally useful, but often it's safer to use None. Then create an empty list if a value hasn't been passed.
The reason is that Python stores a value for L. In other words L is a reference to a constant. But that constant can be updated.
You can say that Python stores it as:
|--> []
|
function (arr, L)
But that [] is an ordinary object (thus with state), that can be modified as well. Now if the function can modify or return L, you start modifying the state of L. In the example:
def function(arr, L=[]):
L.append(arr)
return L
You modify L. If you call this the first time (for instance with function(123)), the object is updated so, now it is represented as:
|--> [123]
|
function (arr, L)
As a result the behavior of function depends on a global state. In general a global state is seen as a bad smell in code design and furthermore it is not what people might expect. This doesn't hold for None, since you modify the local reference L (not the object itself).
You can say that the object is the same, but each time you call the function, you copy a reference to a local variable L.
Now for the second case:
|--> None
|
def function(arr, L):
if L is None:
L = []
L.append(arr)
return L
If you call this method, you assign a value to L, but L itself is not global (only the object to which L refers). So if you call this method, after the if, the function (in progress) will look like.
Because the argument l is assigned an initial value of [] NOT when you call the function but when it's first declared and the module it's in is interpreted.
See: Python: Common Gotcha (Thanks #ferhat elmas)
Example: (showing what's going on)
$ python
Python 2.7.9 (default, Mar 19 2015, 22:32:11)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(L=[]):
... L.append(1)
... return id(L), L
...
>>> def g(L=None):
... L = [] if L is None else L
... L.append(1)
... return id(L), L
...
We'll ignore what happens when you pass in another list to the L argument because that behaviour is well defined and acceptable. (takes a list, appends to it and returns it).
>>> f()
(139978918901088, [1])
>>> g()
(139978918964112, [1])
The first time we call f() and g() we might mistakenly think that we have successfully returned a new list from an initial empty one right? What what happens when we call f() again compared with the correct g():
>>> f()
(139978918901088, [1, 1])
>>> g()
(139978918964112, [1])
So the initial value you assigned for the argument L is set once and only only when the function is defined; not when it's called. This is a common gotcha as explained in the link aove.
NB: id() here returns the unique identity of objects so you can clearly see why the first is an incorrect way to define a default value for a mutable object such as a list.
Also Note: In the above examples that the identity of L does not change when calling f().
Related
Python Tutorial 4.7.1. Default Argument Values states the following:
Important warning: The default value is evaluated only once. This makes a difference
when the default is a mutable object such as a list, dictionary, or instances of most
classes. For example, the following function accumulates the arguments passed to it on
subsequent calls:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
This will print
[1]
[1, 2]
[1, 2, 3]
I don't quite understand the meaning of "evaluated only once" in terms of memory management. Apparently, the default value of the function is evaluated once when the function is first called and stored in a separate memory address even after the function has ended. (according to my understanding, after the function ended, all local variables should be freed?)
Am I correct?
In Python, functions are objects too, and the defaults are stored with the function object. Defaults are not locals; it is just that when the function is called, the arguments are bound to a default when not given an explicit value.
When Python encounters a def <functionname>(<arguments>): statement, it creates a function object for you there and then; this is 'definition time'; the function is not called but merely created. It is then that defaults are evaluated and stored, in an attribute on the function object.
Then when you call the function, the defaults have already been created and are used when you didn't provide a more concrete value for the argument. Because the defaults are stored with the function object, you get to see changes to mutable objects between function calls.
The locals are still cleared up of course, but as they are references (all identifiers in Python are), the objects they were bound to are only cleared up if nothing else is referencing them anymore either.
You can take a look a the defaults of any function object:
>>> def foo(bar='spam', eggs=[]):
... eggs.append(bar)
... return eggs
...
>>> foo.__defaults__
('spam', [])
>>> foo()
['spam']
>>> foo.__defaults__
('spam', ['spam'])
>>> foo() is foo.__defaults__[1]
True
The foo() function has a __defaults__ attribute, a tuple of default values to use when no values for the arguments have been passed in. You can see the mutable list change as the function is called, and because the function returns the eggs list, you can also see that it is the exact same object as the second value in that tuple.
If you don't want your defaults to be shared and instead need a new value for a parameter every time the function is called, but the parameter is not given, you need to set the default value to a sentinel object. If your parameter is still set to that sentinel in the function body, you can execute code to set a fresh default value. None is usually the best choice:
def foo(bar='spam', eggs=None):
if eggs is None:
eggs = []
If it should be possible to use None as a non-default value, use a singleton sentinel created beforehand:
_sentinel = object()
def foo(bar='spam', eggs=_sentinel):
if eggs is _sentinel:
eggs = []
The function that you have defined f is an object in its own regard. When you define defaults, these defaults are bound to the function that you have created.
You can see this in action:
>>> def f(a, L=[]):
... L.append(a)
... return L
>>> print id(f)
4419902952
>>> print f.__defaults__
([],)
>>> f(1)
[1]
>>> print id(f)
4419902952
>>> print f.__defaults__
([1],)
edit, further, you can see that the list container does not change either:
>>> print id(f.__defaults__[0])
4419887544
>>> f(2)
[1, 2]
>>> print id(f.__defaults__[0])
4419887544
On each subsequent call, the default list ("L") of your f function will have your a value appended.
A function is just an object in python, that is created using the def syntax. Default values are stored within the function object when the function is defined, and they are not re-evaluated later.
This is sometimes used to create function variables that persist to subsequent invocations. You can use the __defaults__ methods to check what the default values are for your function.
A common way to initialize new objects instead of reusing the same is:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
You can check this page for more details.
Sorry this answer was meant for a different question, but I'll leave it here as a reference if anyone who wants to look at it. Define once means that at the first point when the code is executed, the default variable gets assigned to an object which is retained within the function object itself.
Notice only 1 object address gets printed, the default list object is used.
def f(a, L=[]):
print("id default: ", id(L))
L.append(a)
print("id used: ", id(L)
return L
Notice 2 different object addresses are printed, when you perform L=[] within the function, you are binding L to a different list object, therefore the default list object does not get change.
def f(a, L=[]):
print("id default: ", id(L))
if L == []:
L = []
L.append(a)
print("id used: ", id(L))
return L
The function above is basically the same as the one below except it uses the None object instead of a empty list object.
def f(a, L=None):
print("id default", id(L))
if L is None:
L = []
L.append(a)
print("id used: ", id(L))
return L
As I'm new to python I've started the topic of default arguments
As per that of the definition I've understood that the default arguments are evaluated only once and that at the point of function definition but this code fragment created the confusion
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
In the above code L being a variable
Modified to list on the first function call
ex.f(1)
But even for the second time when the function is called L is being modified to list
ex.. f(1)
f(2)
Results in [1]
[2]
Could you'll actually be precise in explaining how the above code evaluation is done
Everytime you call f without a 2nd parameter, a new list is created. If you want to reuse the list, you need to store the result of f
new_list = f(1)
f(2, new_list)
print(new_list)
Will output [1,2]
You can read this for better understanding of python arguments passing https://www.python-course.eu/passing_arguments.php
Long story short - you can't override value of argument, you can only create local variable L that points to new list, which will shadow argument L. But on next call of function argument L is still None, unless it will be passed
I saw this below piece of code in a tutorial and wondering how it works.
Generally, the lambda takes a input and returns something but here it does not take anything and still it works.
>>> for i in range(3):
... a.append(lambda:i)
...
>>> a
[<function <lambda> at 0x028930B0>, <function <lambda> at 0x02893030>, <function
<lambda> at 0x028930F0>]
lambda:i defines the constant function that returns i.
Try this:
>>> f = lambda:3
>>> f()
You get the value 3.
But there's something more going on. Try this:
>>> a = 4
>>> g = lambda:a
>>> g()
gives you 4. But after a = 5, g() returns 5. Python functions "remember" the environment in which they're executed. This environment is called a "closure". By modifying the data in the closure (e.g. the variable a in the second example) you can change the behavior of the functions defined in that closure.
In this case a is a list of function objects defined in the loop.
Each of which will return 2.
>>> a[0]()
2
To make these function objects remember i values sequentially you should rewrite the code to
>>> for i in range(3):
... a.append(lambda x=i:x)
...
that will give you
>>> a[0]()
0
>>> a[1]()
1
>>> a[2]()
2
but in this case you get side effect that allows you to not to use remembered value
>>> a[0](42)
42
I'm not sure what you mean by "it works". It appears that it doesn't work at all. In the case you have presented, i is a global variable. It changes every time the loop iterates, so after the loop, i == 2. Now, since each lambda function simply says lambda:i each function call will simply return the most recent value of i. For example:
>>> a = []
>>> for i in range(3):
a.append(lambda:1)
>>> print a[0]()
2
>>> print a[1]()
2
>>> print a[2]()
In other words, this does not likely do what you expect it to do.
lambda defines an anonymous inline function. These functions are limited compared to the full functions you can define with def - they can't do assignments, and they just return a result. However, you can run into interesting issues with them, as defining an ordinary function inside a loop is not common, but lambda functions are often put into loops. This can create closure issues.
The following:
>>> a = []
>>> for i in range(3):
... a.append(lambda:i)
adds three functions (which are first-class objects in Python) to a. These functions return the value of i. However, they use the definition of i as it existed at the end of the loop. Therefore, you can call any of these functions:
>>> a[0]()
2
>>> a[1]()
2
>>> a[2]()
2
and they will each return 2, the last iteration of the range object. If you want each to return a different number, use a default argument:
>>> for i in range(3):
... a.append(lambda i=i:i)
This will forcibly give each function an i as it was at that specific point during execution.
>>> a[0]()
0
>>> a[1]()
1
>>> a[2]()
2
Of course, since we're now able to pass an argument to that function, we can do this:
>>> b[0](5)
5
>>> b[0](range(3))
range(0, 3)
It all depends on what you're planning to do with it.
Python Tutorial 4.7.1. Default Argument Values states the following:
Important warning: The default value is evaluated only once. This makes a difference
when the default is a mutable object such as a list, dictionary, or instances of most
classes. For example, the following function accumulates the arguments passed to it on
subsequent calls:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
This will print
[1]
[1, 2]
[1, 2, 3]
I don't quite understand the meaning of "evaluated only once" in terms of memory management. Apparently, the default value of the function is evaluated once when the function is first called and stored in a separate memory address even after the function has ended. (according to my understanding, after the function ended, all local variables should be freed?)
Am I correct?
In Python, functions are objects too, and the defaults are stored with the function object. Defaults are not locals; it is just that when the function is called, the arguments are bound to a default when not given an explicit value.
When Python encounters a def <functionname>(<arguments>): statement, it creates a function object for you there and then; this is 'definition time'; the function is not called but merely created. It is then that defaults are evaluated and stored, in an attribute on the function object.
Then when you call the function, the defaults have already been created and are used when you didn't provide a more concrete value for the argument. Because the defaults are stored with the function object, you get to see changes to mutable objects between function calls.
The locals are still cleared up of course, but as they are references (all identifiers in Python are), the objects they were bound to are only cleared up if nothing else is referencing them anymore either.
You can take a look a the defaults of any function object:
>>> def foo(bar='spam', eggs=[]):
... eggs.append(bar)
... return eggs
...
>>> foo.__defaults__
('spam', [])
>>> foo()
['spam']
>>> foo.__defaults__
('spam', ['spam'])
>>> foo() is foo.__defaults__[1]
True
The foo() function has a __defaults__ attribute, a tuple of default values to use when no values for the arguments have been passed in. You can see the mutable list change as the function is called, and because the function returns the eggs list, you can also see that it is the exact same object as the second value in that tuple.
If you don't want your defaults to be shared and instead need a new value for a parameter every time the function is called, but the parameter is not given, you need to set the default value to a sentinel object. If your parameter is still set to that sentinel in the function body, you can execute code to set a fresh default value. None is usually the best choice:
def foo(bar='spam', eggs=None):
if eggs is None:
eggs = []
If it should be possible to use None as a non-default value, use a singleton sentinel created beforehand:
_sentinel = object()
def foo(bar='spam', eggs=_sentinel):
if eggs is _sentinel:
eggs = []
The function that you have defined f is an object in its own regard. When you define defaults, these defaults are bound to the function that you have created.
You can see this in action:
>>> def f(a, L=[]):
... L.append(a)
... return L
>>> print id(f)
4419902952
>>> print f.__defaults__
([],)
>>> f(1)
[1]
>>> print id(f)
4419902952
>>> print f.__defaults__
([1],)
edit, further, you can see that the list container does not change either:
>>> print id(f.__defaults__[0])
4419887544
>>> f(2)
[1, 2]
>>> print id(f.__defaults__[0])
4419887544
On each subsequent call, the default list ("L") of your f function will have your a value appended.
A function is just an object in python, that is created using the def syntax. Default values are stored within the function object when the function is defined, and they are not re-evaluated later.
This is sometimes used to create function variables that persist to subsequent invocations. You can use the __defaults__ methods to check what the default values are for your function.
A common way to initialize new objects instead of reusing the same is:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
You can check this page for more details.
Sorry this answer was meant for a different question, but I'll leave it here as a reference if anyone who wants to look at it. Define once means that at the first point when the code is executed, the default variable gets assigned to an object which is retained within the function object itself.
Notice only 1 object address gets printed, the default list object is used.
def f(a, L=[]):
print("id default: ", id(L))
L.append(a)
print("id used: ", id(L)
return L
Notice 2 different object addresses are printed, when you perform L=[] within the function, you are binding L to a different list object, therefore the default list object does not get change.
def f(a, L=[]):
print("id default: ", id(L))
if L == []:
L = []
L.append(a)
print("id used: ", id(L))
return L
The function above is basically the same as the one below except it uses the None object instead of a empty list object.
def f(a, L=None):
print("id default", id(L))
if L is None:
L = []
L.append(a)
print("id used: ", id(L))
return L
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
“Least Astonishment” in Python: The Mutable Default Argument
I recently met a problem in Python.
Code:
def f(a, L=[]):
L.append(a)
return L
print f(1)
print f(2)
print f(3)
the output would be
[1]
[1, 2]
[1, 2, 3]
but why the value in the local list: L in the function f remains unchanged?
Because L is a local variable, I think the output should be:
[1]
[2]
[3]
I tried another way to implement this function:
Code:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
This time, the output is:
[1]
[2]
[3]
I just don't understand why...
Does anyone have some ideas? many thanks.
The default parameters are in fact initialized when the function is defined, so
def f(L = []): pass
is quite similar to
global default_L = []
def f(L = default_L): pass
You can see this way that it is the same list object that is used in every invocation of the function.
The list in def f(a, L=[]) is defined as the function is defined. It is the same list every time you call the function without a keyword argument.
Setting the keyword to None and checking / creating as you have done is the usual work around for this sort of behaviour.