This question already has answers here:
What is the scope of a defaulted parameter in Python?
(7 answers)
Closed 1 year ago.
I understand that one should not use mutable default parameter value in Python (with some exceptions) because this value is evaluated and stored only once when the function is defined, and not each time the function is later called.
My understanding of that is this (using the example below; please excuse my imprecise language as I'm just a beginner in Python programming, who's stuck at the Function chapter of my textbook because of this):
def f(x = [1, 2, 3]):
x.append(4)
print(x)
f()
f()
1) The function f is defined, and x (local variable in f) assumes the default variable of [1, 2, 3] (even before the function is called)
2) When f() is called, x is still [1, 2, 3] due to no argument passed to it, and x continues having its default value
3) x is modified in place with append, becomes [1, 2, 3, 4], and is printed as such
However, this is where my confusion arises. I'd assume that:
4) When f ends, x is destroyed (in the stack or whatever you'd call it) and is no longer associated with the list object [1, 2, 3, 4]**
5) The list object [1, 2, 3, 4] is reclaimed since there's no variable that refers to it anymore
Therefore,
6) When f() is called the second time, I'd expect Python to output an error since x now no longer has a value associated with it. In other words, how can Python reuse the default value from the last evaluation when it's been reclaimed/destroyed?
Appreciate all your help and explanation!
** this understanding I got from Ned Batchelder's page on variable name assignment (see below)
While it may seems to you that at the end of the execution x, the default value, is disposed, it is not.
In fact, Python has a global namespace with all the names available for you to use (built-in functions, classes and functions you import or define).
The content of this namespace is made of objects. Function are objects too.
As a test, if you try this in a script or in the python command line, you will see what I mean:
def f(x = [1, 2, 3]):
x.append(4)
print(x)
print dir(f)
you will see the object nature of the function f. As an objects, the default values are referenced in an attribute, f.func_defaults, therefore they are always available and if mutable they retain the changes, giving you side effects with you may not want.
EDIT: in python 3 the attribute has been replaced by f.__defaults__
There are two references to the list in your case, one is store in the background of the function as the default value to the argument x.
When the function is called without x, a new reference to the same list is created as the local variable x. Then you append to the list via the second reference. And after the call, the second reference is garbage collected. The first reference still points to the same list, which has one element more now.
Or in short: there is only one list all the time.
Related
This question already has answers here:
How to prevent a list from changing after being used as parameter in function?
(3 answers)
Closed 1 year ago.
My (limited) understanding of Python is that objects defined outside a function should remain the same even if modified inside a function, i.e.
def fun():
i=1
return('done')
i=0
fun()
i==0 ## True
However, lists (and other objects like numpy arrays) change when indexed inside a function:
def fun():
img[0] = img[0] + 100
return('done')
img = [0, 1]
fun()
img == [0, 1] ## False
Obviously I am missing a core concept with respect to how global and local variables are handled inside functions but can't seem to find any information online. Could someone please explain why objects change inside functions when indexed? Also, could someone describe how to avoid this "feature" so that when I index objects (lists, arrays, etc...) within a function, I don't inadvertently change the objects defined outside that function? Thanks!
Please see my recent answer to a closely related question: https://stackoverflow.com/a/69303154/8431111
There's two concepts in play for your example:
behavior of an assignment versus a reference
mutating a single container object
For item (1.) when fun assigns i = 1,
it creates a new local variable
due to the lack of a global declaration.
For item (2.) let's consider this slightly simpler code instead,
as it doesn't change the essential problem:
img[0] = 100
The assignment operator locates img
by first consulting the function's local scope (not found)
and then finding it in the module's global scope.
With a reference to the existing img object in hand,
it then calls __setitem__
to alter what the zero-th element points to.
In this case it will point at an immutable int object.
The setter's assignment is performed by a STORE_SUBSCR bytecode instruction.
This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 2 years ago.
I am new guy. Here is an example from python document.
def funtest(num, L=[]):
L.append(num)
return L
print(funtest(2))
print(funtest(3))
print(funtest(4))
I thought very function call should execute the same thing, which means L=[] in very function call. so I though the output should like this:
[2]
[3]
[4]
but the real answer is in below. It looks like the funtest() inherited the output from the last function call. And then continually pass it to next function call. I am confusing to it.
[2]
[2, 3]
[2, 3, 4]
When I change the code to below case:
def funtest(L=[]):
L.append(L)
return L
print(funtest([2]))
print(funtest([3]))
print(funtest([4]))
the real output go to this:
[2, [...]]
[3, [...]]
[4, [...]]
The output are quite different with the previous one. I am more confused now. Does anyone repair my mind shot? Thanks a lot
to explain what is happening ,
1st case
so the default values in python functions are instantiated only once , only while defining the function , so the list is only created while defining the function and you keep adding to it so now the result contains all the older values
this is true for all values which are mutable when used as default value for arguments in python
what can you do ?
in general use immutable values , and for the case of list you use None as the default value and in the function you check if list is None and create one empty list inside the function
read more here ( the first gotcha ) : https://docs.python-guide.org/writing/gotchas/
2nd case
now you are actually creating infinite list by appending same list to itself when you are doing L.append(L)
and whenever a list is too large to print , python shows those three dots in its place
read more here What does [...] (an ellipsis) in a list mean in Python?
The problem is that the default value of L is a mutable object (in your case an empty list).
In Python default values are evaluated at function definition time.
Every time you call the function, you get the same default value and if you add any content to the object (the list) you'll get all the previously added content.
The correct way to get a default list (or dictionary, or set) is to create it at run time inside the function.
This is a common mistake in Python with beginners devs.
You would like to read similar questions:
"Least Astonishment" and the Mutable Default Argument
Good uses for mutable function argument default values?
move L to inside of function
def funtest(num):
L=[]
L.append(num)
return L
print(funtest(2))
print(funtest(3))
print(funtest(4))
this re set the L
This question already has answers here:
What is the scope of a defaulted parameter in Python?
(7 answers)
Closed 1 year ago.
I understand that one should not use mutable default parameter value in Python (with some exceptions) because this value is evaluated and stored only once when the function is defined, and not each time the function is later called.
My understanding of that is this (using the example below; please excuse my imprecise language as I'm just a beginner in Python programming, who's stuck at the Function chapter of my textbook because of this):
def f(x = [1, 2, 3]):
x.append(4)
print(x)
f()
f()
1) The function f is defined, and x (local variable in f) assumes the default variable of [1, 2, 3] (even before the function is called)
2) When f() is called, x is still [1, 2, 3] due to no argument passed to it, and x continues having its default value
3) x is modified in place with append, becomes [1, 2, 3, 4], and is printed as such
However, this is where my confusion arises. I'd assume that:
4) When f ends, x is destroyed (in the stack or whatever you'd call it) and is no longer associated with the list object [1, 2, 3, 4]**
5) The list object [1, 2, 3, 4] is reclaimed since there's no variable that refers to it anymore
Therefore,
6) When f() is called the second time, I'd expect Python to output an error since x now no longer has a value associated with it. In other words, how can Python reuse the default value from the last evaluation when it's been reclaimed/destroyed?
Appreciate all your help and explanation!
** this understanding I got from Ned Batchelder's page on variable name assignment (see below)
While it may seems to you that at the end of the execution x, the default value, is disposed, it is not.
In fact, Python has a global namespace with all the names available for you to use (built-in functions, classes and functions you import or define).
The content of this namespace is made of objects. Function are objects too.
As a test, if you try this in a script or in the python command line, you will see what I mean:
def f(x = [1, 2, 3]):
x.append(4)
print(x)
print dir(f)
you will see the object nature of the function f. As an objects, the default values are referenced in an attribute, f.func_defaults, therefore they are always available and if mutable they retain the changes, giving you side effects with you may not want.
EDIT: in python 3 the attribute has been replaced by f.__defaults__
There are two references to the list in your case, one is store in the background of the function as the default value to the argument x.
When the function is called without x, a new reference to the same list is created as the local variable x. Then you append to the list via the second reference. And after the call, the second reference is garbage collected. The first reference still points to the same list, which has one element more now.
Or in short: there is only one list all the time.
I'm passing the value i to th = threading.Thread(target=func, args=(i,)) and start the thread immediately by th.start().
Because this is executed inside a loop with changing index i, I'm wondering if i inside the thread retaines its value from the time the thread has been created, or if the thread is working on the reference of i. In the latter case the value wouldn't necessarily be the sames as it was at creation of th.
Are values passed by reference or value?
I would say, passing mutable objects to functions is calling by reference.
You are not alone in wanting to say that, but thinking that way limits your ability to communicate with others about how Python programs actually work.
Consider the following Python fragment:
a = [1, 2, 3]
b = a
foobar(a)
if not b is a:
print('Impossible!')
If Python was a "pass-by-reference" programming language, then the foobar(a) call could cause this program to print Impossible! by changing local variable a to refer to some different object.
But, Python does not do pass-by-reference. It only does pass-by-value, and that means there is no definition of foobar that could make the fragment execute the print call. The foobar function can mutate the object to which local variable a refers, but it has no ability to modify a itself.
See https://en.wikipedia.org/wiki/Evaluation_strategy
Pass by value and pass by reference are two terms that can be misleading sometimes, and they don't always mean the same in every language. I'm going to assume we're taking about what the two terms mean in C (where a reference is passing a pointer to the variable).
Python is really neither of those, I'll give you an example I took from an article (all credit to the original writer, I'll link the article at the end)
def spam(eggs):
eggs.append(1)
eggs = [2, 3]
ham = [0]
spam(ham)
print(ham)
When spam is called, both ham and eggs point to the same value ([0]), to the same object. So, when eggs.append (1) is executed, [0] becomes [0, 1]. That sounds like pass by reference.
However, when eggs = [2, 3], now both eggs and ham should become the new list in pass by reference. But that does not happen; now eggs points to a list in memory containing [2, 3], but ham still points to the original list with the 1 appended to it. That bit sound more like pass by value.
EDIT
As explained above, if a parameter of the thread is modified inside it, the changes will be seen in the original thread as long as the parameter is mutable. That way, passing a list to the thread and appending something to it will be reflected in the caller thread, for example.
However, an immutable object can't be modified. If you do i += 1, you didn't modify the integer, integers are immutable in Python. You are assigning to i a new integer with a value one unit higher than the one before. It's the same thing that happened with eggs = [2, 3]. So, in that particular example, changes will not be reflected in the original thread.
Hope this helps!
Here's the article I promised, it has a much better explanation of the matter. http://stupidpythonideas.blogspot.com/2013/11/does-python-pass-by-value-or-by.html?m=1
This question already has answers here:
"Least Astonishment" and the Mutable Default Argument
(33 answers)
Closed 7 years ago.
I was reading the python tutorial from Python Documentation Release 2.7.10 and I came across something like this.
Code
def fun1(a,L=[]):
L.append(a)
return L
print fun1(1)
print fun1(2)
print fun1(3)
def fun2(a,L = None):
if L is None:
L=[]
L.append(a)
return L
print fun2(1)
print fun2(2)
print fun2(3)
Output
[1]
[1, 2]
[1, 2, 3]
[1]
[2]
[3]
Process finished with exit code 0
If the L=[] in the first function fun1() is getting called only once , the output of fun1()is fine. But then why L=None is getting called every time in fun2().
L=[] in the function declaration makes Python essentially do this:
this function has a parameter named L
its default argument is [], let's set this particular [] aside and use it anytime there's no parameter passed for L
every time the function is called, create a variable L, and assign it either the passed parameter or the value we set aside earlier
So, the [] part is getting executed once, which creates a list object, which is set aside and kept around, which is why it accumulates changes if you modify it. Exactly the same thing happens with None, however None is not being modified nor is it mutable to begin with, so you're not seeing any weird side effects. None is still only being "executed" once, and that particular None value is set aside just as the list was, it's just that you're not doing anything to the None value itself.
When you define a function the values of the default arguments get evaluated, but the body of the function only get compiled. You can examine the result of the function definition via attributes. Theres a __defaults__ attribute containing the defaults and __code__ attribute containing the body (so these are created when the function is defined).
What's happening in the second example is that None do get evaluated at definition (it evaluates to None duh!), but the code that conditionally assigns [] to L only gets compiled and is run each time (the condition passes).
default arguments are evaluated only once. In fun1 you have the same list which you keep adding to. In fun2 you assign a new [] to the parameter and then append to it. As any assignment, its scope will be limited to the block it occurred in.