This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 8 years ago.
Hi I am new to python. Can someone explain how the below two pieces of code give different outputs? Does the function gets defined each time it is called?
def f(a, L=[]):
L.append(a)
return L
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
on running
print f(1)
print f(2)
print f(3)
I get these outputs respectively
[1]
[1, 2]
[1, 2, 3]
[1]
[2]
[3]
This is a very common 'gotcha' for new python developers. In the first example it looks like a new empty list should be created each time the function is called without the second parameter. It isn't. A single list is created when the function object is created, which is basically when the python script is loaded or you finish entering the function in the interactive shell. This list is then used for every invocation of the function, hence the acculumation you see.
The second is the standard way of working round this, and creates a new list instance each time the function is called without the second parameter.
Under the covers Python is putting any default values it finds in the function definition into a property of the function called defaults. You can see how the same instance is present between calls in the interactive shell:
>>> def f(a,b=[]):
... b.append(a)
>>> f.__defaults__
([],)
>>> f(1)
>>> f.__defaults__
([1],)
>>> f(2)
>>> f.__defaults__
([1,2],)
Default arguments in python are evaluated at the function declaration point - when it's first seen by the interpreter.
In your first snippet, the L argument is "given" a list as the default, and so the values are appended.
In the second snippet, L is always None on entry into the function, so gets re-created each time.
The function default parameters are evaluated when the function is defined.
In your case def f(a, L=[]):creates an empty list that is then passed every time the function is called without the L parameter. So every call to the function operates on the same list, that's why it gives different output each time.
In your second function an empty list is created every time the function is called, so it works as expected.
Related
I'm at the point in learning Python where I'm dealing with the Mutable Default Argument problem.
# BAD: if `a_list` is not passed in, the default will wrongly retain its contents between successive function calls
def bad_append(new_item, a_list=[]):
a_list.append(new_item)
return a_list
# GOOD: if `a_list` is not passed in, the default will always correctly be []
def good_append(new_item, a_list=None):
if a_list is None:
a_list = []
a_list.append(new_item)
return a_list
I understand that a_list is initialized only when the def statement is first encountered, and that's why subsequent calls of bad_append use the same list object.
What I don't understand is why good_append works any different. It looks like a_list would still be initialized only once; therefore, the if statement would only be true on the first invocation of the function, meaning a_list would only get reset to [] on the first invocation, meaning it would still accumulate all past new_item values and still be buggy.
Why isn't it? What concept am I missing? How does a_list get wiped clean every time good_append runs?
It looks like a_list would still be initialized only once
"initialization" is not something that happens to variables in Python, because variables in Python are just names. "initialization" only happens to objects, and it's done via the class' __init__ method.
When you write a = 0, that is an assignment. That is saying "a shall refer to the object that is described by the expression 0". It is not initialization; a can name anything else of any type at any later time, and that happens as a result of assigning something else to a. Assignment is just assignment. The first one is not special.
When you write def good_append(new_item, a_list=None), that is not "initializing" a_list. It is setting up an internal reference to an object, the result of evaluating None, so that when good_append is called without a second parameter, that object is automatically assigned to a_list.
meaning a_list would only get reset to [] on the first invocation
No, a_list gets set to [] any time that a_list is None to begin with. That is, when either None is passed explicitly, or the argument is omitted.
The problem with [] occurs because the expression [] is only evaluated once in this context. When the function is compiled, [] is evaluated, a specific list object is created - that happens to be empty to start - and that object is used as the default.
How does a_list get wiped clean every time good_append runs?
It doesn't. It doesn't need to be.
You know how the problem is described as being with "mutable default arguments"?
None is not mutable.
The problem occurs when you modify the object that the parameter has as a default.
a_list = [] does not modify whatever object a_list previously referred to. It cannot; arbitrary objects cannot magically transform in-place into empty lists. a_list = [] means "a_list shall stop referring to what it previously referred to, and start referring to []". The previously-referred-to object is unchanged.
When the function is compiled, and one of the arguments has a default value, that value - an object - gets baked into the function (which is also, itself, an object!). When you write code that mutates an object, the object mutates. If the object being referred to happens to be the object baked into the function, it still mutates.
But you cannot mutate None. It is immutable.
You can mutate []. It is a list, and lists are mutable. Appending an item to a list mutates the list.
The default value of a_list (or any other default value, for that matter) is stored in the function's interiors once it has been initialized and thus can be modified in any way:
>>> def f(x=[]): return x
...
>>> f.func_defaults
([],)
>>> f.func_defaults[0] is f()
True
resp. for Python 3:
>>> def f(x=[]): return x
...
>>> f.__defaults__
([],)
>>> f.__defaults__[0] is f()
True
So the value in func_defaults is the same which is as well known inside function (and returned in my example in order to access it from outside.
In other words, what happens when calling f() is an implicit x = f.func_defaults[0]. If that object is modified subsequently, you'll keep that modification.
In contrast, an assignment inside the function gets always a new []. Any modification will last until the last reference to that [] has gone; on the next function call, a new [] is created.
In order words again, it is not true that [] gets the same object on every execution, but it is (in the case of default argument) only executed once and then preserved.
The problem only exists if the default value is mutable, which None is not. What gets stored along with the function object is the default value. When the function is called, the function's context is initialized with the default value.
a_list = []
just assigns a new object to the name a_list in the context of the current function call. It does not modify None in any way.
No, in good_insert a_list is not initalised only once.
Each time the function is called without specifying the a_list argument, the default is used and a new instance of list is used and returned, the new list does not replace the default value.
The python tutorial says that
the default value is evaluated only once.
The evaluated (only once) default value is stored internally (name it x for simplicity).
case []:
When you define the function with a_list defaulted to [], if you don't provide a_list, it is assigned the internal variable x when . Therefore, when you append to a_list, you are actually appending to x (because a_list and x refer to the same variable now). When you call the function again without a_list, the updated x is re-assigned to a_list.
case None:
The value None is evaluated once and stored in x. If you don't provide, a_list, the variable x is assigned to a_list. But you don't append to x of course. You reassign an empty array to a_list. At this point x and a_list are different variables. The same way when you call the function again without a_list, it first gets the value None from x but then a_list gets assigned to an empty array again.
Note that, for the a_list = [] case, if you provide an explicit value for a_list when you call the function, the new argument does not override x because that's evaluated only once.
This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 2 years ago.
I am new guy. Here is an example from python document.
def funtest(num, L=[]):
L.append(num)
return L
print(funtest(2))
print(funtest(3))
print(funtest(4))
I thought very function call should execute the same thing, which means L=[] in very function call. so I though the output should like this:
[2]
[3]
[4]
but the real answer is in below. It looks like the funtest() inherited the output from the last function call. And then continually pass it to next function call. I am confusing to it.
[2]
[2, 3]
[2, 3, 4]
When I change the code to below case:
def funtest(L=[]):
L.append(L)
return L
print(funtest([2]))
print(funtest([3]))
print(funtest([4]))
the real output go to this:
[2, [...]]
[3, [...]]
[4, [...]]
The output are quite different with the previous one. I am more confused now. Does anyone repair my mind shot? Thanks a lot
to explain what is happening ,
1st case
so the default values in python functions are instantiated only once , only while defining the function , so the list is only created while defining the function and you keep adding to it so now the result contains all the older values
this is true for all values which are mutable when used as default value for arguments in python
what can you do ?
in general use immutable values , and for the case of list you use None as the default value and in the function you check if list is None and create one empty list inside the function
read more here ( the first gotcha ) : https://docs.python-guide.org/writing/gotchas/
2nd case
now you are actually creating infinite list by appending same list to itself when you are doing L.append(L)
and whenever a list is too large to print , python shows those three dots in its place
read more here What does [...] (an ellipsis) in a list mean in Python?
The problem is that the default value of L is a mutable object (in your case an empty list).
In Python default values are evaluated at function definition time.
Every time you call the function, you get the same default value and if you add any content to the object (the list) you'll get all the previously added content.
The correct way to get a default list (or dictionary, or set) is to create it at run time inside the function.
This is a common mistake in Python with beginners devs.
You would like to read similar questions:
"Least Astonishment" and the Mutable Default Argument
Good uses for mutable function argument default values?
move L to inside of function
def funtest(num):
L=[]
L.append(num)
return L
print(funtest(2))
print(funtest(3))
print(funtest(4))
this re set the L
This question already has answers here:
Why can a function modify some arguments as perceived by the caller, but not others?
(13 answers)
Closed 6 years ago.
I am a relative python novice. I have a simple function here that takes in a list and removes an element of the list. However, I noticed that the function actually alters the list outside of the function. For example,
def test(lista):
lista.remove(1)
return(lista)
def main():
a = [1,2,3]
print(a)
x = test(lista=a)
print(a)
It turns out that the first call to print(a), I get [1, 2, 3] as expected, but the second call to print(a), I get [2, 3] which doesn't quite make sense to me because I'm not sure why the function test would remove the element from a. I understand that I pass a in as a parameter, but I'm not sure why lista.remove(1) would remove the element 1 from both a and lista.
Thanks!
Python objects are passed-by-object-reference to functions (ruby calls this pass-by-reference but it's different than C++ pass-by-reference). This means that the argument you assign to lista is the original object a, simply with a new named reference object to the same data.
If you instead copied the list it would not affect the original
x = test(lista=list(a))
This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 7 years ago.
I was reading the python tutorial from Python Documentation Release 2.7.10 and I came across something like this.
Code
def fun1(a,L=[]):
L.append(a)
return L
print fun1(1)
print fun1(2)
print fun1(3)
def fun2(a,L = None):
if L is None:
L=[]
L.append(a)
return L
print fun2(1)
print fun2(2)
print fun2(3)
Output
[1]
[1, 2]
[1, 2, 3]
[1]
[2]
[3]
Process finished with exit code 0
If the L=[] in the first function fun1() is getting called only once , the output of fun1()is fine. But then why L=None is getting called every time in fun2().
L=[] in the function declaration makes Python essentially do this:
this function has a parameter named L
its default argument is [], let's set this particular [] aside and use it anytime there's no parameter passed for L
every time the function is called, create a variable L, and assign it either the passed parameter or the value we set aside earlier
So, the [] part is getting executed once, which creates a list object, which is set aside and kept around, which is why it accumulates changes if you modify it. Exactly the same thing happens with None, however None is not being modified nor is it mutable to begin with, so you're not seeing any weird side effects. None is still only being "executed" once, and that particular None value is set aside just as the list was, it's just that you're not doing anything to the None value itself.
When you define a function the values of the default arguments get evaluated, but the body of the function only get compiled. You can examine the result of the function definition via attributes. Theres a __defaults__ attribute containing the defaults and __code__ attribute containing the body (so these are created when the function is defined).
What's happening in the second example is that None do get evaluated at definition (it evaluates to None duh!), but the code that conditionally assigns [] to L only gets compiled and is run each time (the condition passes).
default arguments are evaluated only once. In fun1 you have the same list which you keep adding to. In fun2 you assign a new [] to the parameter and then append to it. As any assignment, its scope will be limited to the block it occurred in.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
“Least Astonishment” in Python: The Mutable Default Argument
Using python 2.7 I came across strange behaviour and I'm not sure how to explain it or if it even exists in any python docs. Using the code
class MyClass:
#staticmethod
def func(objects=[], a=None, b=None):
objects.append(a)
print 'objects: %s'%objects
print 'b: %s'%b
MyClass.func(a='one')
MyClass.func(a='two', b='foo')
MyClass.func(a='three')
I get the get output
objects: ['one']
b: None
objects: ['one', 'two']
b: foo
objects: ['one', 'two', 'three']
b: None
As you can see, the first list parameter (objects) of the method retains it's values across calls.. new values being appended to the last list even though in it's header declaration it has a default value of []. But the last parameter (b) does not retain it's value, it is reset to the default value between calls.
The expected (for me anyway) is that the objects parameter should be reset to it's default on any call to the method (like the b parameter is), but this doesn't seem to happen and only seems to occur on the first call.
Can anyone explain this behaviour? Is it a bug in this version of python or is it intended behaviour? Possibly something to do with the list reference being retained across calls but the string variable (b) is not? I'm very confused by this behaviour.
Thanks
It has nothing to do with being an staticmethod. This is a very common mistake in Python.
Functions in Python are first class objects, not just a block of code, so the empty list you assign to objects in the parameters is a real list attached to the function object. Everytime you call it and append something to that list, you are using the same list. It's easy to see it happening this way:
>>> def func(x, objects=[]):
... objects.append(x)
...
>>> func(1)
>>> func.func_defaults
([1],)
>>> func(2)
>>> func.func_defaults
([1, 2],)
>>> func(3)
>>> func.func_defaults
([1, 2, 3],)
func_defaults is the function object attribute that constains the default keyword parameters you set. See how the list is there and get changed?
The proper way to do what you want is:
class MyClass:
#staticmethod
def func(objects=None, a=None, b=None):
if objects is None:
objects = []
objects.append(a)
print 'objects: %s'%objects
print 'b: %s'%b
This behaviour is broadly known and by many treated as a feature, and is not staticmethod-specific. It works for every function. It happens when you assign mutable object as the default value to the argument.
See this answer on StackOverflow: Default Argument Gotchas / Dangers of Mutable Default arguments
If you understand mutability vs. immutability issue, think of the function / method like that:
when it is defined, the default argument values are assigned,
when it is invoked, the body is executed and if it changes mutable default argument value in-place, the changed value is then being used on every subsequent call where the default argument value has not been overriden by providing different value instead of it,