This question already has answers here:
When is "i += x" different from "i = i + x" in Python?
(3 answers)
Closed 9 years ago.
I always thought x += 1 was just syntactic shorthand (and exactly equivalent to) x = x + 1, until I spent a while trying to figure out why this code wasn't acting as intended:
[ipython/euler 72 ]$ def func(mylist):
mylist += random.sample(range(100),2)
# do stuff with the random result, then restore original list
mylist = mylist[:-2]
It's supposed to return the same list it gets, but it doesn't seem to work that way:
[ipython/euler 81 ]$ x = [1,2,3]
[ipython/euler 82 ]$ func(x)
[1, 2, 3, 23, 7]
[ipython/euler 83 ]$ func(x)
[1, 2, 3, 23, 7, 42, 36]
[ipython/euler 84 ]$ func(x)
[1, 2, 3, 23, 7, 42, 36, 0, 5]
If I change the assignment statement to the long form mylist = mylist + ..., it works as expected and leaves the list unchanged.
Why is this happening? I assume it has to do with lists being mutable and possibly iadd not being 'real' addition when called as an overloaded method of list, but I still expected the interpreter to see them as equivalent.
The first code (mylist += random.sample(range(100),2)) modifies the list in place, as you've realised.
Both of these:
mylist = mylist + ...
mylist = mylist[:-2]
create a new object called mylist, which is then given the same value as (or one derived from) the previous mylist.
The latter line does not restore the original list for this reason; it's creating a new copy, and leaving the original reference alone.
You can get your desired behaviour by changing the list in place:
mylist[:] = mylist[:-2]
The line
mylist += random.sample(range(100),2)
mutates the list mylist in-place (that's why it's called iadd: in-place add). This means that it changes the original list in the caller's scope.
mylist = mylist[:-2]
creates a new local object mylist and assigns the contents of global mylist[:-2] to it. This new local object is then discarded immediately upon returning from the function.
Related
This question already has answers here:
Passing values in Python [duplicate]
(8 answers)
Closed 2 years ago.
Consider the Python code below:
def change_list(in_list):
in_list = [1,2,3]
def change_list_append(in_list):
in_list.append(7)
my_list = [1,2,3,4,5,6]
change_list(my_list)
print(my_list)
change_list_append(my_list)
print(my_list)
The output is:
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6, 7]
I don't understand why the first call (change_list) did not result in my_list to become [1,2,3]. Can someone explain? Look at what happens with the call to change_list_append: number 7 is appended to my_list.
Assigning to a name never changes the value previously referenced by the list. This is true even without invoking function parameters.
>>> x = [1,2,3]
>>> y = x
>>> y = [4,5,6]
>>> x
[1, 2, 3]
Changing what y refers to does not alter what x refers to.
In your second example, you are invoking a mutating method on the list referenced by the parameter. Both my_list and in_list refer to the same list, so it any changes made to the list via either reference are visible from either reference.
See https://nedbatchelder.com/text/names.html for a more in-depth discussion of how references work in Python.
I think this post sums it up nicely => Why can a function modify some arguments as perceived by the caller, but not others?
You can assign items to the in_list using methods (like the .clear() method or the .assign() ) not rename.
As mentioned by others, the 1st function does not modify the global my_list. Instead it modifies the local variable
If you want change_list to modify the global variable you need to the function to return the new value and assign it to my_list.
Something like this will do:
def change_list():
newList = [1,2,3]
return newList
def change_list_append(in_list):
in_list.append(7)
my_list = [1,2,3,4,5,6]
my_list = change_list()
print(my_list)
change_list_append(my_list)
print(my_list)
Output:
[1, 2, 3]
[1, 2, 3, 7]
What strange magic is this?
def rotate_list(lst, n):
n = n % len(lst)
lst = lst[-n:] + lst[:-n]
def rotate_list_2(lst):
lst[0], lst[1], lst[2], lst[3] = lst[3], lst[0], lst[1], lst[2]
s1 = [1, 2, 5, 4]
rotate_list(s1, 1)
print(s1)
s1 = [1, 2, 5, 4]
rotate_list_2(s1)
print(s1)
Output:
[1, 2, 5, 4]
[4, 1, 2, 5]
It appears that although lists are generally mutable within functions, if a list with the same name is created, then the original list is unaffected by changes to the new list. Could someone please explain what is happening here please, in terms of scope and references?
How would I rotate the original list without having to manually update each value as in rotate_list_2()? Or would this kind of thing generally be done by working with new lists returned from a function?
Assigning to list in function doesn't change the original reference.
The assignment just references the local parameter lst on the new value.
The original list referenced outside ('before') the function remains intact.
Insead assign to it's elements with this syntax:
def rotate_list(lst, n):
n = n % len(lst)
lst[:] = lst[-n:] + lst[:-n]
s1 = [1, 2, 5, 4]
rotate_list(s1, 1)
# And it works like magic! :)
# [4, 1, 2, 5]
print(s1)
If you reassign the argument of a function, the value does not change outside of the function scope.
def test0(a):
a = 10
print(a)
x = 4
test0(x)
print(x)
This would result in
10
4
The reason why assigning values of an array works is that you're not assigning a new value to the argument itself. You're instead accessing the memory that the array reads from, and you're changing it. Thus, those changes will happen even for outer scopes.
After change in the function you can return the list and capture it in function call like s=function(s)
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.
I am trying to write a function which removes the first item in a Python list. This is what I've tried. Why doesn't remove_first_wrong change l when I call the function on it? And why does the list slicing approach work when I do it in the main function?
def remove_first_wrong(lst):
lst = lst[1:]
def remove_first_right(lst):
lst.pop(0)
if __name__ == '__main__':
l = [1, 2, 3, 4, 5]
remove_first_wrong(l)
print(l)
l_2 = [1, 2, 3, 4, 5]
remove_first_right(l_2)
print(l_2)
# Why does this work and remove_first_wrong doesn't?
l_3 = [1, 2, 3, 4, 5]
l_3 = l_3[1:]
print(l_3)
Slicing a list returns a new list object, which is a copy of the original list indices you indicated in the slice. You then rebound lst (a local name in the function) to reference that new list instead. The old list is never altered in that process.
list.pop() on the other hand, operates on the list object itself. It doesn't matter what reference you used to reach the list.
You'd see the same thing without functions:
>>> a = [1, 2]
>>> b = a[:] # slice with all the elements, produces a *copy*
>>> b
[1, 2]
>>> a.pop() # remove an element from a won't change b
2
>>> b
[1, 2]
>>> a
[1]
Using [:] is one of two ways of making a shallow copy of a list, see How to clone or copy a list?
You may want to read or watch Ned Batchelder's Names and Values presestation, to further help understand how Python names and objects work.
Inside the function remove_first_wrong the = sign reassigns the name lst to the object on the right. Which is a brand new object, created by slicing operation lst[1:]. Thus, the object lst assigned to is local to that function (and it actually will disappear on return).
That is what Martijn means by "You then rebound lst (a local name in the function) to reference that new list instead."
On contrary, lst.pop(0) is a call to the given object -- it operates on the object.
For example, this will work right too:
def remove_first_right2(lst):
x = lst # x is assigned to the same object as lst
x.pop(0) # pop the item from the object
Alternately, you can use del keyword:
def remove_first_element(lst):
del lst[0]
return lst
This question already has answers here:
a mutable type inside an immutable container
(3 answers)
Closed 6 years ago.
So I have this code:
tup = ([1,2,3],[7,8,9])
tup[0] += (4,5,6)
which generates this error:
TypeError: 'tuple' object does not support item assignment
While this code:
tup = ([1,2,3],[7,8,9])
try:
tup[0] += (4,5,6)
except TypeError:
print tup
prints this:
([1, 2, 3, 4, 5, 6], [7, 8, 9])
Is this behavior expected?
Note
I realize this is not a very common use case. However, while the error is expected, I did not expect the list change.
Yes it's expected.
A tuple cannot be changed. A tuple, like a list, is a structure that points to other objects. It doesn't care about what those objects are. They could be strings, numbers, tuples, lists, or other objects.
So doing anything to one of the objects contained in the tuple, including appending to that object if it's a list, isn't relevant to the semantics of the tuple.
(Imagine if you wrote a class that had methods on it that cause its internal state to change. You wouldn't expect it to be impossible to call those methods on an object based on where it's stored).
Or another example:
>>> l1 = [1, 2, 3]
>>> l2 = [4, 5, 6]
>>> t = (l1, l2)
>>> l3 = [l1, l2]
>>> l3[1].append(7)
Two mutable lists referenced by a list and by a tuple. Should I be able to do the last line (answer: yes). If you think the answer's no, why not? Should t change the semantics of l3 (answer: no).
If you want an immutable object of sequential structures, it should be tuples all the way down.
Why does it error?
This example uses the infix operator:
Many operations have an “in-place” version. The following functions
provide a more primitive access to in-place operators than the usual
syntax does; for example, the statement x += y is equivalent to x =
operator.iadd(x, y). Another way to put it is to say that z =
operator.iadd(x, y) is equivalent to the compound statement z = x; z
+= y.
https://docs.python.org/2/library/operator.html
So this:
l = [1, 2, 3]
tup = (l,)
tup[0] += (4,5,6)
is equivalent to this:
l = [1, 2, 3]
tup = (l,)
x = tup[0]
x = x.__iadd__([4, 5, 6]) # like extend, but returns x instead of None
tup[0] = x
The __iadd__ line succeeds, and modifies the first list. So the list has been changed. The __iadd__ call returns the mutated list.
The second line tries to assign the list back to the tuple, and this fails.
So, at the end of the program, the list has been extended but the second part of the += operation failed. For the specifics, see this question.
Well I guess tup[0] += (4, 5, 6) is translated to:
tup[0] = tup[0].__iadd__((4,5,6))
tup[0].__iadd__((4,5,6)) is executed normally changing the list in the first element. But the assignment fails since tuples are immutables.
Tuples cannot be changed directly, correct. Yet, you may change a tuple's element by reference. Like:
>>> tup = ([1,2,3],[7,8,9])
>>> l = tup[0]
>>> l += (4,5,6)
>>> tup
([1, 2, 3, 4, 5, 6], [7, 8, 9])
The Python developers wrote an official explanation about why it happens here: https://docs.python.org/2/faq/programming.html#why-does-a-tuple-i-item-raise-an-exception-when-the-addition-works
The short version is that += actually does two things, one right after the other:
Run the thing on the right.
assign the result to the variable on the left
In this case, step 1 works because you’re allowed to add stuff to lists (they’re mutable), but step 2 fails because you can’t put stuff into tuples after creating them (tuples are immutable).
In a real program, I would suggest you don't do a try-except clause, because tup[0].extend([4,5,6]) does the exact same thing.