This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Understanding Python's call-by-object style of passing function arguments
I recently came across this:
x = [1,2,3]
def change_1(x):
x = x.remove(x[0])
return x
Which results in:
>>> change_1(x)
>>> x
[2, 3]
I find this behavior surprising, as I thought that whatever goes in inside a function has no effect on outside variables. Furthermore, I constructed an example where basically do the same thing but without using remove:
x = [1,2,3]
def change_2(x):
x = x[1:]
return x
Which results in:
>>> change_2(x)
[2, 3] # Also the output prints out here not sure why this is
>>> x
[1, 2, 3]
And I get the result that I would expect, the function does not change x.
So it must be something remove specific that has effect. What is going on here?
One of the things that is confusing is that you've called lots of different things 'x'. For example:
def change_1(x): # the x parameter is a reference to the global 'x' list
x = x.remove(x[0]) # on the left, though, is a new 'x' that is local to the function
return x # the local x is returned
>>> x = [1, 2, 3]
>>> y = change_1(x) # assign the return value to 'y'
>>> print y
None # this is None because x.remove() assigned None to the local 'x' inside the function
>>> print x
[2, 3] # but x.remove() modified the global x inside the function
def change_2(x):
x = x[1:] # again, x on left is local, it gets a copy of the slice, but the 'x' parameter is not changed
return x # return the slice (copy)
>>> x = [1, 2, 3]
>>> y = change_2(x)
>>> print x
[1, 2, 3] # the global 'x' is not changed!
>>> print y
[2, 3] # but the slice created in the function is assigned to 'y'
You would get the same result if you called the parameter of your function n, or q.
It's not the variable name that's being affected. The x in the scope of your list and the x outside that scope are two different "labels". Since you passed the value that x was attached to, to change_1() however, they are both referring to the same object. When you do x.remove() on the object in the function, you are basically saying: "get the object that x refers to. Now remove an element from that object. This is very different than a lhs assignment. If you did y=0; x=y inside your function. Your not doing anything to the list object. You're basically just tearing the 'x' label of of [1, 2. 3] and putting it on whatever y is pointing to.
Your variable x is just a reference to the list you've created. When you call that function, you are passing that reference by value. But in the function, you have a reference to the same list So when the function modifies it, it is modified at any scope.
Also, when executing commands in the interactive interpreter, python prints the return value, if it is not assigned to a variable.
Related
While trying to implement an algorithm, I couldn't get python lists to mutate via a function. After reading up on the issue I was suggested by this StackOverflow answer to use [:] in order to mutate the array passed in the function argumemt.
However, as seen in the following code snippet, the issue still persists when trying to mutate the list l. I am expecting the output to be Before: [1,2,3,4]
After: [69, 69, 69, 69], but instead I get back the original value of l as shown below.
def mutate_list(a, b):
c = [69] * 4
a[:] = c[:2] # changed the elements, but array's still unchanged outside function
b[:] = c[2:]
if __name__ == '__main__':
l = [1, 2, 3, 4]
print("Before: {}" .format(l))
mutate_list(l[:2], l[2:])
print("After: {}" .format(l))
Output:
Before: [1, 2, 3, 4]
After : [1, 2, 3, 4]
Any insights into why this is happening?
The error is that you not pass actually the l but two slices of it. You should change it, for example:
def mutate_list(a):
c = [69] * 4
a[:2] = c[:2]
a[2:] = c[2:]
if __name__ == '__main__':
l = [1, 2, 3, 4]
print("Before: {}" .format(l))
mutate_list(l)
print("After: {}" .format(l))
its all about the scope, mutable concept is applicable on list but not to reference variable.
The variables a,b are local variables, hence the scope of the variable will be always function scope.
The operations which you have performed :
a[:]=c[:2]
b[:]=c[2:]
Note: a and b both are list now so you will get following output in the function:
[69,69],[69,69]
but if you use + operator which is use for adding operations then the out out will be like:
[69,69,69,69]
Now whatever I told you that will be a local scope, if you want that the list should be mutable across the program then you have to specify the scope of the list as global inside function and on that variable you can do changes. in this case you also dont need to pass any arguments:
def mutate_list():
global l # telling python to use this global variable in a local function
c = [69] * 4
l=c # assigning new values to actual list i.e l
Now before output will be [1,2,3,4]
and after will be [69,69,69,69]
As pointed out by others, the issue arose from the fact that the function parameters were slices of the original array and as such, the parameters were being passed by value (instead of being passed by reference).
According to #Selcuk 's suggestion, the correct way of doing such an operation would be to pass the original array along with its indices to the function and then perform any slicing inside the function.
NOTE: This concept comes in handy for (recursive) divide-and-conquer algorithms where subarrays must be mutated and combined to form the solution.
This question already has answers here:
How do I pass a variable by reference?
(39 answers)
Closed 8 months ago.
I am not sure I understand the concept of Python's call by object style of passing function arguments (explained here http://effbot.org/zone/call-by-object.htm). There don't seem to be enough examples to clarify this concept well (or my google-fu is probably weak! :D)
I wrote this little contrived Python program to try to understand this concept
def foo( itnumber, ittuple, itlist, itdict ):
itnumber +=1
print id(itnumber) , itnumber
print id(ittuple) , ittuple
itlist.append(3.4)
print id(itlist) , itlist
itdict['mary'] = 2.3
print id(itdict), itdict
# Initialize a number, a tuple, a list and a dictionary
tnumber = 1
print id( tnumber ), tnumber
ttuple = (1, 2, 3)
print id( ttuple ) , ttuple
tlist = [1, 2, 3]
print id( tlist ) , tlist
tdict = tel = {'jack': 4098, 'sape': 4139}
print '-------'
# Invoke a function and test it
foo(tnumber, ttuple, tlist , tdict)
print '-------'
#Test behaviour after the function call is over
print id(tnumber) , tnumber
print id(ttuple) , ttuple
print id(tlist) , tlist
print id(tdict), tdict
The output of the program is
146739376 1
3075201660 (1, 2, 3)
3075103916 [1, 2, 3]
3075193004 {'sape': 4139, 'jack': 4098}
---------
146739364 2
3075201660 (1, 2, 3)
3075103916 [1, 2, 3, 3.4]
3075193004 {'sape': 4139, 'jack': 4098, 'mary': 2.3}
---------
146739376 1
3075201660 (1, 2, 3)
3075103916 [1, 2, 3, 3.4]
3075193004 {'sape': 4139, 'jack': 4098, 'mary': 2.3}
As you can see , except for the integer that was passed, the object id's (which as I understand refers to memeory location) remain unchanged.
So in the case of the integer, it was (effectively) passed by value and the other data structure were (effectively) passed by reference. I tried changing the list , the number and the dictionary to just test if the data-structures were changed in place. The number was not bu the list and the
dictionary were.
I use the word effectively above, since the 'call-by-object' style of argument passing seems to behave both ways depending on the data-structure passed in the above code
For more complicated data structures, (say numpy arrays etc), is there any quick rule of thumb to
recognize which arguments will be passed by reference and which ones passed by value?
The key difference is that in C-style language, a variable is a box in memory in which you put stuff. In Python, a variable is a name.
Python is neither call-by-reference nor call-by-value. It's something much more sensible! (In fact, I learned Python before I learned the more common languages, so call-by-value and call-by-reference seem very strange to me.)
In Python, there are things and there are names. Lists, integers, strings, and custom objects are all things. x, y, and z are names. Writing
x = []
means "construct a new thing [] and give it the name x". Writing
x = []
foo = lambda x: x.append(None)
foo(x)
means "construct a new thing [] with name x, construct a new function (which is another thing) with name foo, and call foo on the thing with name x". Now foo just appends None to whatever it received, so this reduces to "append None to the the empty list". Writing
x = 0
def foo(x):
x += 1
foo(x)
means "construct a new thing 0 with name x, construct a new function foo, and call foo on x". Inside foo, the assignment just says "rename x to 1 plus what it used to be", but that doesn't change the thing 0.
Others have already posted good answers. One more thing that I think will help:
x = expr
evaluates expr and binds x to the result. On the other hand:
x.operate()
does something to x and hence can change it (resulting in the same underlying object having a different value).
The funny cases come in with things like:
x += expr
which translate into either x = x + expr (rebinding) or x.__iadd__(expr) (modifying), sometimes in very peculiar ways:
>>> x = 1
>>> x += 2
>>> x
3
(so x was rebound, since integers are immutable)
>>> x = ([1], 2)
>>> x
([1], 2)
>>> x[0] += [3]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> x
([1, 3], 2)
Here x[0], which is itself mutable, was mutated in-place; but then Python also attempted to mutate x itself (as with x.__iadd__), which errored-out because tuples are immutable. But by then x[0] was already mutated!
Numbers, strings, and tuples in Python are immutable; using augmented assignment will rebind the name.
Your other types are merely mutated, and remain the same object.
I study programming languages and a quiz question and solution was this:
def foo(x):
x.append (3)
x = [8]
return x
x=[1, 5]
y= foo(x)
print x
print y
Why does this print as follows:
[1 5 3 ]
[8]
Why doesn't x equal to 8 ??
The other two answers are great. I suggest you to try id to get address.
See the following example
def foo(x):
x.append (3)
print "global",id(x)
x = [8]
print "local ",id(x)
return x
x=[1, 5]
print "global",id(x)
y= foo(x)
print "global",id(x)
print x
print y
And the output
global 140646798391920
global 140646798391920
local 140646798392928
global 140646798391920
[1, 5, 3]
[8]
As you can see, the address of the variable x remains same when you manipulate it but changes when you use =. Variable assignment inside a function makes the variable local to the function
You have a lot of things going on there. So let's go step by step.
x = [1,5] You are assigning a list of 1,5 to x
y=foo(x) You are calling foo and passing in x and assigning whatever gets returned from foo
inside foo you call x.append(3) which appends 3 to the list that was passed in.
you then set x = [8] which is now a reference to a local variable x which then gets returned from foo ultimately setting y = [8]
Object References
The key to understanding this is that Python passes variables around using object references. These are similar to pointers in a language like c++, but are different in very key ways.
When an assignment is made (using the assignment operator, =):
x = [1, 5]
Actually TWO things have been created. First is the object itself, which is the list [1, 5]. This object is a separate entity from the second thing, which is the (global) object reference to the object.
Object(s) Object Reference(s)
[1, 5] x (global) <--- New object created
In python, objects are passed into functions by object reference; they are not passed "by reference" or "by value" like in c++. This means that when x is passed into the foo function, there is a new, local object reference to the object created.
Object(s) Object Reference(s)
[1, 5] x (global), x (local to foo) <--- Now two object references
Now inside of foo we call x.append(3), which directly changes the object (referred to by the foo-local x object reference) itself:
Object(s) Object Reference(s)
[1, 5, 3] x (global), x (local to foo) <--- Still two object references
Next, we do something different. We assign the local-foo x object reference (or re-assign, since the object reference already existed previously) to a NEW LIST.
Object(s) Object Reference(s)
[1, 5, 3] x (global) <--- One object reference remains
[8] x (local to foo) <--- New object created
Notice that the global object reference, x, is still there! It has not been impacted. We only re-assigned the local-foo x object reference to a NEW list.
And finally, it should be clear that when the function returns, we have this:
Object(s) Object Reference(s)
[1, 5, 3] x (global) <--- Unchanged
[8] y (global), x (local to foo) <--- New object reference created
Sidebar
Notice that the local-foo x object reference is still there! This is important behavior to understand, because if we do something like this:
def f(a = []):
a.append(1)
return a
f()
f()
f()
We will NOT get:
[1]
[1]
[1]
Instead, we will get:
[1]
[1, 1]
[1, 1, 1]
The statement a = [] is only evaluated ONCE by the interpreter when the program first runs, and that object reference never gets deleted (unless we delete the function itself).
As a result, when f() is called, local-f a is not changed back to []. It "remembers" its previous value; this is because that local object reference is still valid (i.e., it has not been deleted), and therefore the object does not get garbage collected between function calls.
Contrast with Pointers
One of the ways object references are different from pointers is in assignment. If Python used actual pointers, you would expect behavior such as the following:
a = 1
b = a
b = 2
assert a == 2
However, this assertion produces an error. The reason is that b = 2 does NOT impact the object "pointed to" by the object reference, b. It creates a NEW object (2) and re-assigns b to that object.
Another way object references are different from pointers is in deletion:
a = 1
b = a
del a
assert b is None
This assertion also produces an error. The reason is the same as in the example above; del a does NOT impact the object "pointed to" by the object reference, b. It simply deletes the object reference, a. The object reference, b, and the object it points to, 1, are not impacted.
You might be asking, "Well then how do I delete the actual object?" The answer is YOU CAN'T! You can only delete all the references to that object. Once there are no longer any references to an object, the object becomes eligible for garbage collection and it will be deleted for you (although you can force this to happen using the gc module). This feature is known as memory management, and it is one of the primary strengths of Python, and it is also one of the reasons why Python uses object references in the first place.
Mutability
Another subject that needs to be understood is that there are two types of objects: mutable, and immutable. Mutable objects can be changed, while immutable objects cannot be changed.
A list, such as [1, 5], is mutable. A tuple or int, on the other hand, is immutable.
The append Method
Now that all of this is clear, we should be able to intuit the answer to the question "How does append work in Python?" The result of the append() method is an operation on the mutable list object itself. The list is changed "in place", so to speak. There is not a new list created and then assigned to the foo-local x. This is in contrast to the assignment operator =, which creates a NEW object and assigns that object to an object reference.
The append function modifies the x that was passed into the function, whereas assigning something new to x changed the locally scoped value and returned it.
The scope of x inside foo is specific to the function and is independent from the main calling context. x inside foo starts out referencing the same object as x in the main context because that's the parameter that was passed in, but the moment you use the assignment operator to set it to [8] you have allocated a new object to which x inside foo points, which is now totally different from x in the main context. To illustrate further, try changing foo to this:
def foo(x):
print("Inside foo(): id(x) = " + str(id(x)))
x.append (3)
print("After appending: id(x) = " + str(id(x)))
x = [8]
print("After setting x to [8], id(x) = " + str(id(x)))
return x
When I executed, I got this output:
Inside foo(): id(x) = 4418625336
After appending: id(x) = 4418625336
After setting x to [8], id(x) = 4418719896
[1, 5, 3]
[8]
(the IDs you see will vary, but the point will still be clear I hope)
You can see that append just mutates the existing object - no need to allocate a new one. But once the = operator executes, a new object gets allocated (and eventually returned to the main context, where it is assigned to y).
I am loading a file "data.imputation" which is 2 dimensional in variable 'x'. Variable 'y' is a copy of 'x'. I pop the first array from 'y' (y is 2D). Why is the change reflected on x? (The first array from 'x' is also popped)
ip = open('data.meanimputation','r')
x = pickle.load(ip)
y = x
y.pop(0)
At the start, len(x) == len(y). Even after y.pop(0), len(x) == len(y). Why is that? And how can I avoid it?
use y = x[:] instead of y = x. y = x means both y and x are now pointing to the same object.
Take a look at this example:
>>> x=[1,2,3,4]
>>> y=x
>>> y is x
True # it means both y and x are just references to a same object [1,2,3,4], so changing either of y or x will affect [1,2,3,4]
>>> y=x[:] # this makes a copy of x and assigns that copy to y,
>>> y is x # y & x now point to different object, so changing one will not affect the other.
False
If x is a list is list of list then [:] is of no use:
>>> x= [[1,2],[4,5]]
>>> y=x[:] #it makes a shallow copy,i.e if the objects inside it are mutable then it just copies their reference to the y
>>> y is x
False # now though y and x are not same object but the object contained in them are same
>>> y[0].append(99)
>>> x
[[1, 2, 99], [4, 5]]
>>> y
[[1, 2, 99], [4, 5]]
>>> y[0] is x[0]
True #see both point to the same object
in such case you should use copy module's deepcopy() function , it makes non-shallow copies of object.
y = x does not copy anything. It binds the name y to the same object already referred to by x. Assignment to a bare name never copies anything in Python.
If you want to copy an object, you need to copy it explicitly, using whatever methods are available for the object you're trying to copy. You don't say what kind of object x is, so there's no way to say how you might be able to copy it, but the copy module provides some functions that work for many types. See also this answer.
folks, i got a question about passing mutable object to a function
with the following code, I was expecting the output to be [0,0,0], while the output is [0,1,2,3]
does it mean the argument is actually copied and then send to the inside of the function?
def lala(a):
n = [0, 0 , 0]
a = n
a = [0,1,2,3]
lala(a)
print a
if i want to fulfill the above task inside the function, how shall i write it elegantly?
thanks very much!
Python makes more sense if you think of attaching name tags to objects, rather than stuffing objects into named boxes.
def lala(a):
n = [0, 0 , 0]
a = n
Here's what's happening.
You're receiving a parameter (a list in this case) and giving it the name a.
You're creating a new list and giving it the name n.
You are giving the list you named n the additional name a.
Both names, a and n, are local to the lala function, so they "expire" when the function ends.
The list you created in the function now has no names, and Python discards it.
In other words, a is not a box into which you can put a new list. It is a name you gave a list you received. Later, you reassign that name to the list you have also named n.
Others have suggested a slice assignment, a[:] = n, which uses a reference to the items in the list rather than the list itself. When you do this, the name a still points to the same list; its contents have just been replaced. Since it is the same list that was passed into the function, any names by which it is known outside the function will "see" the new contents.
kindall provided a great explanation in my opinion, but my preferred method for doing something like this is to have the function return the change instead of messing with references.
def lala():
n = [0, 0, 0]
return n
a = [0, 1, 2, 3]
a = lala()
print a # prints [0, 0, 0]
you can use a[:] = n inside the function. this is called a slice assignment
python does not have a call-by-reference feature. There's no general way to do this.
If you know your argument is going to be a list, and you want it to take a different value, you can write it like so:
def lala(a):
a[:] = [0,0,0]
That's because a function makes new label "a" for its scope. Then this "a" overshadows the "a" you defined outside the function. So the new label a is assigned to new object [0,0,0,0]
If you would write:
def lala(a):
a.pop()
a = [0,1,2,3,4]
lala(a)
print(a)
You would see that a = [0,1,2,3] because pop() actually change the object which label 'a' points to. (while assignment just change to what object given label points to)
Note the difference between your function, and this one:
def lala(a):
a[0] = 7
a = [0,1,2,3]
lala(a)
print a # prints [7, 1, 2, 3]