Why does the variable L gets manipulated in the sorting(L) function call? In other languages, a copy of L would be passed through to sorting() as a copy so that any changes to x would not change the original variable?
def sorting(x):
A = x #Passed by reference?
A.sort()
def testScope():
L = [5,4,3,2,1]
sorting(L) #Passed by reference?
return L
>>> print testScope()
>>> [1, 2, 3, 4, 5]
Long story short: Python uses pass-by-value, but the things that are passed by value are references. The actual objects have 0 to infinity references pointing at them, and for purposes of mutating that object, it doesn't matter who you are and how you got a reference to the object.
Going through your example step by step:
L = [...] creates a list object somewhere in memory, the local variable L stores a reference to that object.
sorting (strictly speaking, the callable object pointed to be the global name sorting) gets called with a copy of the reference stored by L, and stores it in a local called x.
The method sort of the object pointed to by the reference contained in x is invoked. It gets a reference to the object (in the self parameter) as well. It somehow mutates that object (the object, not some reference to the object, which is merely more than a memory address).
Now, since references were copied, but not the object the references point to, all the other references we discussed still point to the same object. The one object that was modified "in-place".
testScope then returns another reference to that list object.
print uses it to request a string representation (calls the __str__ method) and outputs it. Since it's still the same object, of course it's printing the sorted list.
So whenever you pass an object anywhere, you share it with whoever recives it. Functions can (but usually won't) mutate the objects (pointed to by the references) they are passed, from calling mutating methods to assigning members. Note though that assigning a member is different from assigning a plain ol' name - which merely means mutating your local scope, not any of the caller's objects. So you can't mutate the caller's locals (this is why it's not pass-by-reference).
Further reading: A discussion on effbot.org why it's not pass-by-reference and not what most people would call pass-by-value.
Python has the concept of Mutable and Immutable objects. An object like a string or integer is immutable - every change you make creates a new string or integer.
Lists are mutable and can be manipulated in place. See below.
a = [1, 2, 3]
b = [1, 2, 3]
c = a
print a is b, a is c
# False True
print a, b, c
# [1, 2, 3] [1, 2, 3] [1, 2, 3]
a.reverse()
print a, b, c
# [3, 2, 1] [1, 2, 3] [3, 2, 1]
print a is b, a is c
# False True
Note how c was reversed, because c "is" a. There are many ways to copy a list to a new object in memory. An easy method is to slice: c = a[:]
It's specifically mentioned in the documentation the .sort() function mutates the collection. If you want to iterate over a sorted collection use sorted(L) instead. This provides a generator instead of just sorting the list.
a = 1
b = a
a = 2
print b
References are not the same as separate objects.
.sort() also mutates the collection.
Related
This question already has answers here:
How do I operate on the actual object, not a copy, in a python for loop?
(3 answers)
Closed 2 years ago.
I am trying to increment the elements of a list by passing it into a increment() function that I have defined.
I have tried two ways to do this.
Accessing using the index.
# List passed to a function
def increment(LIST):
for i in range(len(LIST)):
LIST[i] += 1
return LIST
li = [1, 2, 3, 4]
li = increment(li)
print(li)
This outputs the desired result: [2, 3, 4, 5]
Accessing using iterator variables.
# List passed to a function
def increment(LIST):
for item in LIST:
item += 1
return LIST
li = [1, 2, 3, 4]
li = increment(li)
print(li)
This outputs: [1, 2, 3, 4]
I wish to know the reason behind this difference.
Python's in-place operators can be confusing. The "in-place" refers to the current binding of the object, not necessarily the object itself. Whether the object mutates itself or creates a new object for the in-place binding, depends on its own implementation.
If the object implements __iadd__, then the object performs the operation and returns a value. Python binds that value to the current variable. That's the "in-place" part. A mutable object may return itself whereas an immutable object returns a different object entirely. If the object doesn't implement __iadd__, python falls back to several other operators, but the result is the same. Whatever the object chooses to return is bound to the current variable.
In this bit of code
for item in LIST:
item += 1
a value of the list is bound to a variable called "item" on each iteration. It is still also bound to the list. The inplace add rebinds item, but doesn't do anything to the list. If this was an object that mutated itself with iadd, its still bound to the list and you'll see the mutated value. But python integers are immmutable. item was rebound to the new integer, but the original int is still bound to the list.
Which way any given object works, you kinda just have to know. Immutables like integers and mutables like lists are pretty straight forward. Packages that rely heavily on fancy meta-coding like pandas are all over the map.
The reasoning behind this is because integers are immutable in python. You are essentially creating a new integer when performing the operation item +=1
This post has more information on the topic
If you wished to update the list, you would need to create a new list or update the list entry.
def increment(LIST):
result = []
for item in LIST:
result.append(item+1)
return result
li = [1, 2, 3, 4]
li = increment(li)
print(li)
While trying to implement an algorithm, I couldn't get python lists to mutate via a function. After reading up on the issue I was suggested by this StackOverflow answer to use [:] in order to mutate the array passed in the function argumemt.
However, as seen in the following code snippet, the issue still persists when trying to mutate the list l. I am expecting the output to be Before: [1,2,3,4]
After: [69, 69, 69, 69], but instead I get back the original value of l as shown below.
def mutate_list(a, b):
c = [69] * 4
a[:] = c[:2] # changed the elements, but array's still unchanged outside function
b[:] = c[2:]
if __name__ == '__main__':
l = [1, 2, 3, 4]
print("Before: {}" .format(l))
mutate_list(l[:2], l[2:])
print("After: {}" .format(l))
Output:
Before: [1, 2, 3, 4]
After : [1, 2, 3, 4]
Any insights into why this is happening?
The error is that you not pass actually the l but two slices of it. You should change it, for example:
def mutate_list(a):
c = [69] * 4
a[:2] = c[:2]
a[2:] = c[2:]
if __name__ == '__main__':
l = [1, 2, 3, 4]
print("Before: {}" .format(l))
mutate_list(l)
print("After: {}" .format(l))
its all about the scope, mutable concept is applicable on list but not to reference variable.
The variables a,b are local variables, hence the scope of the variable will be always function scope.
The operations which you have performed :
a[:]=c[:2]
b[:]=c[2:]
Note: a and b both are list now so you will get following output in the function:
[69,69],[69,69]
but if you use + operator which is use for adding operations then the out out will be like:
[69,69,69,69]
Now whatever I told you that will be a local scope, if you want that the list should be mutable across the program then you have to specify the scope of the list as global inside function and on that variable you can do changes. in this case you also dont need to pass any arguments:
def mutate_list():
global l # telling python to use this global variable in a local function
c = [69] * 4
l=c # assigning new values to actual list i.e l
Now before output will be [1,2,3,4]
and after will be [69,69,69,69]
As pointed out by others, the issue arose from the fact that the function parameters were slices of the original array and as such, the parameters were being passed by value (instead of being passed by reference).
According to #Selcuk 's suggestion, the correct way of doing such an operation would be to pass the original array along with its indices to the function and then perform any slicing inside the function.
NOTE: This concept comes in handy for (recursive) divide-and-conquer algorithms where subarrays must be mutated and combined to form the solution.
the id of the object before and after should be same but its not happening. can someone explain me why a new object is being made.
L = [1, 2, 3]
print(id(L))
L = L + [4]
print(id(L))
both id's are that are being printed is different shouldn't it be the same its a mutable object. but when i use the append method of list to add 4 then the id is same
While lists are mutable, that doesn't mean that all operations involving them mutate the list in place. In your example, you're doing L + [4] to concatenate two lists. The list.__add__ method that gets invoked to implement that creates a new list, rather than modifying L. You're binding the old name L to the new list, so the value you get from id(L) changes.
If you want to mutate L while adding a value onto the end, there are several ways you can do it. L.append(4) is the obvious pick if you have just a single item to add. L.extend([4]) or the nearly synonymous L += [4] can work if the second list has more items in it than one.
Note that sometimes creating a new list will be what you want to do! If want to keep an unmodified reference to the old list, it may be desirable to create a new list with most of its contents at the same time you add new values. While you could copy the list then use one of the in place methods I mentioned above, you can also just use + to copy and add values to the list at the same time (just bind the result to a new name):
L = [1, 2, 3]
M = L + [4] # this is more convenient than M = list(L); M.append(4)
print(L) # unchanged, still [1, 2, 3]
print(M) # new list [1, 2, 3, 4]
its a mutable object
yes, you can change the value without creating a new object. But with the +, you are creating a new object.
To mute a mutable value, use methods (such as append) or set items (a[0] = ...). As soon as you have L=, the object formerly referenced by L is lost (if it doesn't have any other references) and L gets a new value.
This makes sense because, in fact, with L = L+[0], you are saying "calculate the value of L+[0] and assign it to L" not "add [0] to L".
This question already has answers here:
Variable assignment and modification (in python) [duplicate]
(6 answers)
Closed 4 years ago.
How come the following code:
a = [1,2,3]
b = a
b[0] = 3
print(a)
will print list b after it has been altered?[3,2,3].
Also why is this true but that the following code:
a = [1,2,3]
b = a
b = [0,0,0]
print(a,b)
prints [1, 2, 3] [0, 0, 0]?? This seems inconsistent. If the first code is true, then shouldn't the second code print [0,0,0][0,0,0]? Can someone please provide an explanation for this?
In python there are two types of data... mutable and immutable. Numbers, strings, boolean, tuples, and other simple types are immutable. Dicts, lists, sets, objects, classes, and other complex types are mutable.
When you say:
a = [1,2,3]
b = a
You've created a single mutable list in memory, assigned a to point to it, and then assigned b to point to it. It's the same thing in memory.
Therefore when you mutate it (modify it):
b[0] = 3
It is a modification (mutation) of the index [0] of the value which b points to at that same memory location.
However, when you replace it:
b = [0,0,0]
It is creating a new mutable list in memory and assigning b to point at it.
Check out the id() function. It will tell you the "address" of any variable. You can see which names are pointing to the same memory location with id(varname).
Bonus: Every value in python is passed by reference... meaning that when you assign it to a variable it simply causes that variable to point to that value where it was in memory. Having immutable types allows python to "reuse" the same memory location for common immutable types.
Consider some common values when the interpreter starts up:
>>> import sys
>>> sys.getrefcount('abc')
68
>>> sys.getrefcount(100)
110
>>> sys.getrefcount(2)
6471
However, a value that is definitely not present would return 2. This has to do with the fact that a couple of references to that value were in-use during the call to sys.getrefcount
>>> sys.getrefcount('nope not me. I am definitely not here already.')
2
Notice that an empty tuple has a lot of references:
>>> sys.getrefcount(tuple())
34571
But an empty list has no extra references:
>>> sys.getrefcount(list())
1
Why is this? Because tuple is immutable so it is fine to share that value across any number of variables. However, lists are mutable so they MUST NOT be shared across arbitrary variables or changes to one would affect the others.
Incidentally, this is also why you must NEVER use mutable types as default argument values to functions. Consider this innocent little function:
>>> def foo(value=[]):
... value.append(1)
... print(value)
...
...
When you call it you might expect to get [1] printed...
>>> foo()
[1]
However, when you call it again, you prob. won't expect to get [1,1] out... ???
>>> foo()
[1, 1]
And on and on...
>>> foo()
[1, 1, 1]
>>> foo()
[1, 1, 1, 1]
WHY IS THIS? Because default arguments to functions are evaluated once during function definition, and not at function run time. That way if you use a mutable value as a default argument value, then you will be stuck with that one value, mutating in unexpected ways as the function is called multiple times.
The proper way to do it is this:
>>> def foo(value=None):
... if value is None:
... value = []
... value.append(1)
... print(value)
...
...
>>>
>>> foo()
[1]
>>> foo()
[1]
>>> foo()
[1]
I study programming languages and a quiz question and solution was this:
def foo(x):
x.append (3)
x = [8]
return x
x=[1, 5]
y= foo(x)
print x
print y
Why does this print as follows:
[1 5 3 ]
[8]
Why doesn't x equal to 8 ??
The other two answers are great. I suggest you to try id to get address.
See the following example
def foo(x):
x.append (3)
print "global",id(x)
x = [8]
print "local ",id(x)
return x
x=[1, 5]
print "global",id(x)
y= foo(x)
print "global",id(x)
print x
print y
And the output
global 140646798391920
global 140646798391920
local 140646798392928
global 140646798391920
[1, 5, 3]
[8]
As you can see, the address of the variable x remains same when you manipulate it but changes when you use =. Variable assignment inside a function makes the variable local to the function
You have a lot of things going on there. So let's go step by step.
x = [1,5] You are assigning a list of 1,5 to x
y=foo(x) You are calling foo and passing in x and assigning whatever gets returned from foo
inside foo you call x.append(3) which appends 3 to the list that was passed in.
you then set x = [8] which is now a reference to a local variable x which then gets returned from foo ultimately setting y = [8]
Object References
The key to understanding this is that Python passes variables around using object references. These are similar to pointers in a language like c++, but are different in very key ways.
When an assignment is made (using the assignment operator, =):
x = [1, 5]
Actually TWO things have been created. First is the object itself, which is the list [1, 5]. This object is a separate entity from the second thing, which is the (global) object reference to the object.
Object(s) Object Reference(s)
[1, 5] x (global) <--- New object created
In python, objects are passed into functions by object reference; they are not passed "by reference" or "by value" like in c++. This means that when x is passed into the foo function, there is a new, local object reference to the object created.
Object(s) Object Reference(s)
[1, 5] x (global), x (local to foo) <--- Now two object references
Now inside of foo we call x.append(3), which directly changes the object (referred to by the foo-local x object reference) itself:
Object(s) Object Reference(s)
[1, 5, 3] x (global), x (local to foo) <--- Still two object references
Next, we do something different. We assign the local-foo x object reference (or re-assign, since the object reference already existed previously) to a NEW LIST.
Object(s) Object Reference(s)
[1, 5, 3] x (global) <--- One object reference remains
[8] x (local to foo) <--- New object created
Notice that the global object reference, x, is still there! It has not been impacted. We only re-assigned the local-foo x object reference to a NEW list.
And finally, it should be clear that when the function returns, we have this:
Object(s) Object Reference(s)
[1, 5, 3] x (global) <--- Unchanged
[8] y (global), x (local to foo) <--- New object reference created
Sidebar
Notice that the local-foo x object reference is still there! This is important behavior to understand, because if we do something like this:
def f(a = []):
a.append(1)
return a
f()
f()
f()
We will NOT get:
[1]
[1]
[1]
Instead, we will get:
[1]
[1, 1]
[1, 1, 1]
The statement a = [] is only evaluated ONCE by the interpreter when the program first runs, and that object reference never gets deleted (unless we delete the function itself).
As a result, when f() is called, local-f a is not changed back to []. It "remembers" its previous value; this is because that local object reference is still valid (i.e., it has not been deleted), and therefore the object does not get garbage collected between function calls.
Contrast with Pointers
One of the ways object references are different from pointers is in assignment. If Python used actual pointers, you would expect behavior such as the following:
a = 1
b = a
b = 2
assert a == 2
However, this assertion produces an error. The reason is that b = 2 does NOT impact the object "pointed to" by the object reference, b. It creates a NEW object (2) and re-assigns b to that object.
Another way object references are different from pointers is in deletion:
a = 1
b = a
del a
assert b is None
This assertion also produces an error. The reason is the same as in the example above; del a does NOT impact the object "pointed to" by the object reference, b. It simply deletes the object reference, a. The object reference, b, and the object it points to, 1, are not impacted.
You might be asking, "Well then how do I delete the actual object?" The answer is YOU CAN'T! You can only delete all the references to that object. Once there are no longer any references to an object, the object becomes eligible for garbage collection and it will be deleted for you (although you can force this to happen using the gc module). This feature is known as memory management, and it is one of the primary strengths of Python, and it is also one of the reasons why Python uses object references in the first place.
Mutability
Another subject that needs to be understood is that there are two types of objects: mutable, and immutable. Mutable objects can be changed, while immutable objects cannot be changed.
A list, such as [1, 5], is mutable. A tuple or int, on the other hand, is immutable.
The append Method
Now that all of this is clear, we should be able to intuit the answer to the question "How does append work in Python?" The result of the append() method is an operation on the mutable list object itself. The list is changed "in place", so to speak. There is not a new list created and then assigned to the foo-local x. This is in contrast to the assignment operator =, which creates a NEW object and assigns that object to an object reference.
The append function modifies the x that was passed into the function, whereas assigning something new to x changed the locally scoped value and returned it.
The scope of x inside foo is specific to the function and is independent from the main calling context. x inside foo starts out referencing the same object as x in the main context because that's the parameter that was passed in, but the moment you use the assignment operator to set it to [8] you have allocated a new object to which x inside foo points, which is now totally different from x in the main context. To illustrate further, try changing foo to this:
def foo(x):
print("Inside foo(): id(x) = " + str(id(x)))
x.append (3)
print("After appending: id(x) = " + str(id(x)))
x = [8]
print("After setting x to [8], id(x) = " + str(id(x)))
return x
When I executed, I got this output:
Inside foo(): id(x) = 4418625336
After appending: id(x) = 4418625336
After setting x to [8], id(x) = 4418719896
[1, 5, 3]
[8]
(the IDs you see will vary, but the point will still be clear I hope)
You can see that append just mutates the existing object - no need to allocate a new one. But once the = operator executes, a new object gets allocated (and eventually returned to the main context, where it is assigned to y).