>>> a = [1,2,3]
>>> b = []
>>> b.append(a)
>>> print(b)
[[1, 2, 3]]
>>> num = a.pop(0)
>>> a.append(num)
>>> print(a)
[2, 3, 1]
>>> b.append(a)
>>> print(b)
[[2, 3, 1], [2, 3, 1]]
>>>
Why is this happening and how to fix it? I need the list like
[[1, 2, 3], [2, 3, 1]]
Thank you.
Edit:
Also, why is this working?
>>> a = []
>>> b = []
>>> a = [1,2,3]
>>> b.append(a)
>>> a = [1,2,3,4]
>>> b.append(a)
>>> print(b)
[[1, 2, 3], [1, 2, 3, 4]]
>>>
'''
Append a copy of your list a, at least the first time. Otherwise, you've appended the same list both times.
b.append(a[:])
When you append the list a, python creates a reference to that variable inside the list b. So when you edit the list a, it is reflected again in the list b. You need to create a copy of your variable and then append it to get the desired result.
Every variable name in Python should be thought of as a reference to a piece of data. In your first listing, b contains two references to the same underlying object that is also referenced by the name a. That object gets changed in-place by the operations you’re using to rotate its members. The effect of that change is seen when you look at either of the two references to the object found in b, or indeed when you look at the reference associated with the name a.
Their identicality can be seen by using the id() function: id(a), id(b[0]) and id(b[1]) all return the same number, which is the unique identifier of the underlying list object that they all refer to. Or you can use the is operator: b[0] is b[1] evaluates to True.
By contrast, in the second listing, you reassign a—in other words, by using the assignment operator = you cause that name to become associated with a different object: in this case, a new list object that you just created with your square-bracketed literal expression. b still contains one reference to the old list, and now you append a new reference that points to this different piece of underlying data. So the two elements of b now look different from each other—and indeed they are different objects and accordingly have different id() numbers, only one of which is the same as the current id(a). b[0] is b[1] now evaluates to False
How to fix it? Reassign the name a before changing it: for example, create a copy:
a = list(a)
or:
import copy
a = copy.copy(a)
(or you could even use copy.deepcopy()—study the difference). Alternatively, rotate the members a using methods that entail reassignment rather than in-place changes—e.g.:
a = a[1:] + a[:1]
(NB immutable objects such as the tuple avoid this whole confusion —not because they behave fundamentally differently but because they lack methods that produce in-place changes and therefore force you to use reassignment strategies.)
In addition to making the copy of a by doing a[:] and assigning it to b.
You can also use collections.deque.rotate to rotate your list
from collections import deque
a = [1,2,3]
#Make a deque of copy of a
b = deque(a[:])
#Rotate the deque
b.rotate(len(a)-1)
#Create the list and print it
print([a,list(b)])
#[[1, 2, 3], [2, 3, 1]]
Related
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 7 years ago.
I have a global list (of lists) variable. I send a shallow copy of the global list to another function as a parameter.
Surprisingly, the original list gets changed when I remove some elements from the parameter within the invoked function.
Can someone please tell me why it's happening and how to prevent this from happening?
Here is the simplified code example:
def saveCandidateRoutes(candidateRoutes):
for route in candidateRoutes:
if route: # check if the list, 'route', is empty.
tweetId = route.pop(0)
stanox = route.pop(-1)
....
def main():
global allCandidatePaths
copyOfAllCandidatePaths= list(allCandidatePaths) # making a deep copy
result = saveCandidateRoutes(copyOfAllCandidatePaths)
I think you need a quick reminder on shallow and deep copies, and how to make a deep copy.
>>> a = [[1,2], [3,4]] # a list of mutable elements
>>> b = a[:]
>>> c = list(a)
Both b and c are shallow copies of a, you can check that a, b and c are different objects because they do not share the same id.
>>> id(a)
140714873892592
>>> id(b)
140714810215672
>>> id(c)
140714873954744
However, each element of a, b and c still is a reference to the lists [1,2] and [3,4] we created when defining a. That becomes clear when we mutate an item inside the lists:
>>> c[1][1] = 42
>>> a
[[1, 2], [3, 42]]
>>> b
[[1, 2], [3, 42]]
>>> c
[[1, 2], [3, 42]]
As you can see, the second element of the second list changed in a, b and c.
Now, to make a deep copy of a, you have several options. One is a list comprehension where you copy each of the sublists:
>>> d = [sublist[:] for sublist in a]
>>> d
[[1, 2], [3, 42]]
>>> d[1][1] = 23
>>> d
[[1, 2], [3, 23]]
>>> a
[[1, 2], [3, 42]]
As you can see, the 42 in a did not change to 23, because the second list in a and d are different objects:
>>> id(a[1])
140714873800968
>>> id(d[1])
140714810230904
Another way to create a deep copy is with copy.deepcopy:
>>> from copy import deepcopy
>>> e = deepcopy(a)
>>> e
[[1, 2], [3, 42]]
>>> e[1][1] = 777
>>> e
[[1, 2], [3, 777]]
>>> a
[[1, 2], [3, 42]]
python uses references to object, it means that both allCandidatePaths and candidateRoutes point to the same list in memory, and you can use both of them to change the list.
To prevent this from happening, in the beginning of your function saveCandidateRoutes add this instruction candidateRoutes = list(candidateRoutes). The list() function will create another copy of your original list in memory and assign its reference to candidateRoutes.
So when you use candidateRoutes, you will not be working on your original list that is in the main function, but you will be working on another list.
I think you are confused about the terms.
Shallow copy is the type of copy where the elements of the copied list are still bound to same memory value with the original list's elements.
What you are looking for is deepcopy
Here is a good source to find out.
And also:
Wikipedia
A shallow copy copies the object but not any of its attributes. For a list, that means that that its elements are assigned to the elements of the new list. If the elements are ints, you get a completely new list, because int is a primitive type*, so it doesn't need to be copied. If the elements are lists, as in your two-dimesional list, they get assigned to the copy, so you have two references to each element, one in each list. If you want to copy the elements of the inner lists, you'll need a deep copy, which recursively copies the attributes (elements, in this case) of each object.
*There isn't actually a distinction between types that are primitive and those that aren't, but this is equivalent to the way it works in the scope of this explanation.
I am trying to write a function which removes the first item in a Python list. This is what I've tried. Why doesn't remove_first_wrong change l when I call the function on it? And why does the list slicing approach work when I do it in the main function?
def remove_first_wrong(lst):
lst = lst[1:]
def remove_first_right(lst):
lst.pop(0)
if __name__ == '__main__':
l = [1, 2, 3, 4, 5]
remove_first_wrong(l)
print(l)
l_2 = [1, 2, 3, 4, 5]
remove_first_right(l_2)
print(l_2)
# Why does this work and remove_first_wrong doesn't?
l_3 = [1, 2, 3, 4, 5]
l_3 = l_3[1:]
print(l_3)
Slicing a list returns a new list object, which is a copy of the original list indices you indicated in the slice. You then rebound lst (a local name in the function) to reference that new list instead. The old list is never altered in that process.
list.pop() on the other hand, operates on the list object itself. It doesn't matter what reference you used to reach the list.
You'd see the same thing without functions:
>>> a = [1, 2]
>>> b = a[:] # slice with all the elements, produces a *copy*
>>> b
[1, 2]
>>> a.pop() # remove an element from a won't change b
2
>>> b
[1, 2]
>>> a
[1]
Using [:] is one of two ways of making a shallow copy of a list, see How to clone or copy a list?
You may want to read or watch Ned Batchelder's Names and Values presestation, to further help understand how Python names and objects work.
Inside the function remove_first_wrong the = sign reassigns the name lst to the object on the right. Which is a brand new object, created by slicing operation lst[1:]. Thus, the object lst assigned to is local to that function (and it actually will disappear on return).
That is what Martijn means by "You then rebound lst (a local name in the function) to reference that new list instead."
On contrary, lst.pop(0) is a call to the given object -- it operates on the object.
For example, this will work right too:
def remove_first_right2(lst):
x = lst # x is assigned to the same object as lst
x.pop(0) # pop the item from the object
Alternately, you can use del keyword:
def remove_first_element(lst):
del lst[0]
return lst
Sorry if this is a duplicate question, I searched and couldn't find anything to help.
I'm currently trying to compare two lists. If there are any matching items I will remove them all from one of the lists.
However the results I have are buggy. Here is a rough but accurate representation of the method I'm using:
>>> i = [1,2,3,4,5,6,7,8,9]
>>> a = i
>>> c = a
>>> for b in c:
if b in i:
a.remove(b)
>>> a
[2, 4, 6, 8]
>>> c
[2, 4, 6, 8]
So I realised that the main issue is that as I remove items it shortens the list, so Python then skips over the intermediate item (seriously annoying). As a result I made a third list to act as an intermediate that can be looped over.
What really baffles me is that this list seems to change also even when I haven't directly asked it to!
In python, when you write this:
i = [1,2,3,4,5,6,7,8,9]
You create an Object (in this case, a list) and you assign it to the name i. Your next line, a = i, tells the interpreter that the name a refers to the same Object. If you want them to be separate Object you need to copy the original list. You can do that via the slicing shorthand, i[:], or you can use a = list(i) to be more explicit.
The easiest way to do this is use a set to determine shared items in a and b:
for x in set(a).intersection(b):
a.remove(x)
Your statements a = i and c = a merely make new names that reference the same object. Then as you removed things from a, it's removed from b and i, since they are the same object. You'll want to make copies of the lists instead, like so
a = i[:]
c = a[:]
a = i Doesn't make a copy of a list, it just sets another variable, i to point at your list a. Try something like this:
>>> i = [1, 2, 3, 2, 5, 6]
>>> s = []
>>> for i in t:
if i not in s:
s.append(i)
>>> s
[1, 2, 3, 5, 6]
You can also use set which guarantees no duplicates, but doesn't preserve the order:
list(set(i))
Say I have a list:
L = [1,2,3]
and I assigned L[0] to a variable a
a = L[0]
then if I change a, it won't affect L.
a = a + 1
print L # [1,2,3] not affected
Why is this happening? isn't python passing everything around with references? I thought that a is pointing to L[0]
The problem is that a and L[0] are references to an immutable object, so changing any one of them won't affect the other references:
>>> L = [1, 2, [3]]
>>> a = L[0]
>>> a = a + 1
a now points to a new object, while L[0] still points to the same object.
>>> a, L[0]
(2, 1)
Now in this case b and L[2] are references to a mutable object(list), any in-place operation on them will affect all the references:
>>> b = L[2]
>>> b.append(4) #list.append is an in-place operation
>>> b, L[2]
([3, 4], [3, 4])
>>> b = b + [5] #mutable object, but not an in-place operation
>>> b #b is assigned to new list object
[3, 4, 5]
>>> L[2] #L[2] is unchanged
[3, 4]
L[0] is a name, and when you create the list L, you assign an object to that name, the integer 1. a is also a name, and when you assign a as in a = L[0], you make a to point to the same object that L[0] points to.
But when you later do a = a + 1, this is another assignment. You are not modifying the object that a points to -- the = sign can't do that. You are creating a new object, the integer 2, and assigning that to a.
So in the end, you have two objects in memory; one is referred to by L[0] and the other is referred to by a.
Integers are immutable, which means that there is no possible way to change the properties of the objects in this example; however, that's not salient in this example exactly, because even if the object was mutable it wouldn't change the fact that you're doing assignment (with the = sign). In a case where the object in question was mutable, you could theoretically change the properties of the object when it is still referenced by L[0] and a, instead of doing any additional assignment with = as you are doing. At that point, you would see the properties change regardless of which name you used to inspect the object.
Since L[0] in your case is immutable, changing a doesn't affect the value of L[0]. When you change a, the new object is created and a starts to pointing to it.
See what happens if L[0] is of a mutable type:
>>> L = [[1],2,3]
>>> a = L[0]
>>> a.append(2)
>>> L
[[1, 2], 2, 3]
In this case a and L[0] both point to the same object.
Also see Raymond Hettinger's answer in the relevant thread.
Change the assignment to:
a = L
then when you change L as:
L[0] += 1
you will see that a also changes. This is the reference magic.
I cannot figure out at all why this is happening:
A = [[1,0], [2,2]]
B = list(A)
print('start A:', A, 'start B:', B)
A[0][0] = 999
print('end A:', A, 'end B:', B)
This returns:
start A: [[1, 0], [2, 2]] start B: [[1, 0], [2, 2]]
end A: [[999, 0], [2, 2]] end B: [[999, 0], [2, 2]]
The lists A and B end up being the same, even though I explicitly copied B from A. This only happens when I do something like A[0][0] = 999; if I replace that with A[0] = 999 then A and B are different at the end.
What's the reason behind this, and is there any way to change A in this manner without affecting B?
You are creating a shallow copy of the original list, that is a new list containing new references to the same objects as the original list.
Modifying the new list object does not alter the original list. Modifying the objects in the new list does modify the objects in the old list because they are the same.
To get a completely separate list, use copy.deepcopy() to create a deep copy.
Both A and B contain the same two lists.
Your code is roughly equivalent to this:
x = [1, 0]
y = [2, 2]
A = [x, y]
B = [x, y]
The operation A[0][0] = 999 is effectively just doing x[0] = 999. That is, it doesn't modify A itself, it modifies the first element of the list x. Since both A and B have references to x, both will see the change.
A simple copy operation like you did is shallow, it only copies the items one level deep and does not recurse into nested structures. You need
>>> import copy
>>> A = [[1,0], [2,2]]
>>> B = copy.deepcopy(A)
>>> print('start A:', A, 'start B:', B)
start A: [[1, 0], [2, 2]] start B: [[1, 0], [2, 2]]
>>> A[0][0] = 999
>>> print('end A:', A, 'end B:', B)
end A: [[999, 0], [2, 2]] end B: [[1, 0], [2, 2]]
A and B are two different names for the same chunk of memory within your computer.
A and B are two separate list objects, but A[0] and B[0] are two different names for the same chunk of memory within your computer. Try the following from the interpreter:
id(B)
id(A)
id(B[0])
id(A[0])
Python code manipulates references to objects.
Assigning to a variable is just binding a name to refer to an object.
A list consists of a bunch of references to objects. list(A) finds all the objects referenced in A and makes a new list with references to all the same objects. So if A is a list of lists, list(A) makes a new list with references to the same lists that were in A. So changing any of the sub-lists will be visible from both A and the new list.
copy.deepcopy exists to help you get around this, when you need a full "deep" copy of something.
Once you learn to think about Python code as manipulating references to objects like this, you will intuitively understand when code is likely to end up referring to the same object from multiple places like this, though there will probably always be obscure cases that surprise you.