I have a list currentCell which contains info like coordinates and values and stuff. I then create a copy of that list using currentCellNew = currentCell.copy(). Im told that doing that and modifying the copy will NOT effect the original. But when i remove something from the copy, it also removes it from the original list.
Here is the code that removes from the copy:
#some variables exist outside of scope
currentCell = next(x for x in self.cells if x[0] == currentCellCoords)
currentCellNew = currentCell.copy()
if currentCellNew[3] == 0:
if **condition**:
wallAxis = next(x[0] for x in list(self.Orientations.items()) if x[1] == "f")
#### Here is the code that removes the element from the copy #####
currentCellNew[2].remove(next(x for x in currentCellNew[2] if x[0] == wallAxis))
list.copy() is a shallow copy; it's fine when the original list contains immutable objects, but it looks like you've got a list of lists, and those inner lists aren't being copied (the new outer list has references to the same inner lists as the first one).
You need to use the copy.deepcopy function to perform the original copy, thereby copying the contents, not just the top-level references:
import copy # At top of file
...
currentCellNew = copy.deepcopy(currentCell)
Related
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 2 years ago.
This one is making me absolutely crazy, so any help would be much appreciated. I have a program where I'm iterating through a list in a function. Here's a toy model of the problem:
masterList = ["person","woman","man","camera","television"]
workingList = masterList
def removeItem (workingList):
item = workingList.pop(2)
print("Test 1:",workingList)
removeItem(workingList)
print("Test 2:", workingList)
print("Test 3:", masterList)
As expected, "Test 1" prints out the list with an item removed.
However, "Test 2" also prints out the list with an item removed. I wasn't expecting that, but no matter. That's not the real problem, but I'm sure there's something I don't understand here about variable scopes and variable shadowing.
No the real problem is "Test 3", which as you can see, is printing out the masterList, which shouldn't even be touched by the removeItem function or the pop function within it. And yet, it too is printing out the list with an item removed.
How can this be happening, and how can I prevent it?
Thanks so much!
Cheers,
Ari
Python lists are mutable objects.
m = list([1, 2, 3])
n = m
a = 5
b = a
id(a) == id(b)
# id() return "identity" of the object.
# True, Both a and b are references to the same object.
id(m) == id(n)
# True, Both m and n are references to the same object.
b = b + 2
id(a) == id(b)
# False, a new object on separate location is created, which b will point.
n.pop()
id(m) == id(n)
# True, the object is mutated, and m and n still references to the same object.
Since, python lists are mutable, m and n will still be reference to the same object after mutation. Whereas, for immutable objects like int, a new object will be created and the identifier will refer to the new object.
Gist is, in your scenario, there has been only one object since python lists are mutable.
However, if you need the original list unchanged when the new list is modified, you can use the copy() method.
new_list = original_list.copy()
The ids of new_list and original_list is different.
Learn here about mutability in detail: https://medium.com/#meghamohan/mutable-and-immutable-side-of-python-c2145cf72747.
You have to make a copy of the masterList, otherwise workingList is nothing more than a reference.
If your list does not contain other containers (which yours doesn't), then a shallow copy is sufficient. There are numerous ways to make a shallow copy but slicing is the most optimal.
masterList = ["person","woman","man","camera","television"]
workingList = masterList[:] #copy via slice
a bunch of other ways to make a shallow copy
workingList = masterList * 1
workingList = masterList.copy()
workingList = list(masterList)
workingList = [*masterList]
import copy
workingList = copy.copy(masterList)
If you have a list that does possess something that holds a reference (like other containers, classes, etc), then you need to make a deepcopy.
import copy
a = [[1, 2, 3], ['a', 'b', 'c']]
b = copy.deepcopy(a)
Looks like I figured it out. The two lists are actually the same list unless you use list.copy()
So replacing the top two lines with:
masterList = ["person","woman","man","camera","television"]
workingList = masterList.copy()
makes everything perform as expected! Well, I still can't say I understand how the variable scopes work in their entirety, but at least this solves the major problem.
I'm terribly sorry if this was already asked, but while I could find something similar I didn't find my specific issue. I have Python 3.7.4 - 64 bit. Basically I want to initialize a dictionary where each element is a list of empty lists. The problem is that in the way I'm doing it now I get that every single empty sub-list from the different items' list is the same object even though I am assigning a copy of the list to each item. As you can see in the code below, each sub-list in empty_list_of_lists is a different object. Then I assign the items to the dictionary as a copy of empty_list_of_lists. When I call my_dict['a'] is my_dict['b'] I get an expected False, but when I call my_dict['a'][0] is my_dict['b'][0] I get a True which puzzles me because empty_list_of_lists[0] is empty_list_of_lists[1] returns False and I don't get the logic. How should I go about that?
Here is my code:
empty_list_of_lists = [[] for i in range(5)]
print(empty_list_of_lists[0] is empty_list_of_lists[1]) # returns False --> expected
dict1 = {'a': empty_list_of_lists.copy(), 'b': empty_list_of_lists.copy()}
print(dict1['a'] is dict1['b']) # returns False --> expected
print(dict1['a'][0] is dict1['b'][0]) # returns True --> What?
you can use:
dict1 = {'a': [[] for _ in range(5)], 'b': [[] for _ in range(5)]}
or you can use copy.deepcopy
import copy
dict1 = {'a': copy.deepcopy(empty_list_of_lists), 'b': copy.deepcopy(empty_list_of_lists)}
you can read more about shallow and deep copy operations here
in your code you are using a shallow copy but what you need a deep copy, from the above docs:
A shallow copy constructs a new compound object and then (to the
extent possible) inserts references into it to the objects found in
the original.
A deep copy constructs a new compound object and then, recursively,
inserts copies into it of the objects found in the original.
This question already has answers here:
What is the difference between shallow copy, deepcopy and normal assignment operation?
(12 answers)
Closed 6 years ago.
Say I have a list a with some values, and I did a b = a[:]. Then modifying the contents of list b won't change list a as per what I've read. So, this means its a deep copy. But python documentation still refers to this as shallow copy. Can someone clear this for me?
To demonstrate what shallow copy means:
a = [ [1,2], [3,4,5] ]
b = a[:] # make a shallow copy
a is b # not the same object, because this is a copy
=> False
a == b # same value, because this is a copy
=> True
a[0] is b[0] # elements are the *same objects*, because this is a *shallow* copy
=> True
Changing the structure of a will not be reflected in b, because this is a copy:
a.pop()
len(a)
=> 1
len(b)
=> 2
To demonstrate the difference from a deep copy: changing an object contained in a (as opposed to a's structure) in-place, is reflected in b, because b references the same objects as a.
a[0][0] = 'XYZ'
b[0]
=> ['XYZ', 2]
From python docs
The difference between shallow and deep copying is only relevant for
compound objects (objects that contain other objects, like lists or
class instances):
A shallow copy constructs a new compound object and then (to the
extent possible) inserts references into it to the objects found in
the original. A deep copy constructs a new compound object and then,
recursively, inserts copies into it of the objects found in the
original.
Shallow copy creates a new object only for the top level/object, then copies by reference all sub-objects. Deep copy creates new object for the top object/level and for all sub-objects too.
Then modifying the contents of list "b" won't change list "a" as per what I've read.
As in, if you take out some of the contents or switch them out for other contents, it won't affect a. Mutations to the objects held in b will be visible through both lists, because both lists hold the same objects.
>>> class Mutable(object):
... def __init__(self, x):
... self.x = x
... def __repr__(self):
... return 'Mutable({})'.format(self.x)
...
>>> a = [Mutable(1), Mutable(2)]
>>> b = a[:]
>>> del b[1] # Doesn't affect a
>>> a
[Mutable(1), Mutable(2)]
>>> b[0].x = 5 # Visible through a
>>> a
[Mutable(5), Mutable(2)]
Then modifying the contents of list "b" won't change list "a" as per what I've read. So, this means its a deep copy.
No, it does not. A shallow copy differs from a deep copy in whether contained values are copied or not.
In your case, the list is copied, but the two resulting lists will contain the same objects. Adding or removing a value in one list won't affect the other list, but changes to a contained object will be reflected.
I've searched around and tried a lot of stuff but I can't get this to work. I think the problem is something to do with how Python list names point to the list, rather than being the actual list, but I still can't figure it out. The situation is this (it's a list of dictionaries):
list_original = [dictionary1, dictionary2, dictionary3]
dictionary2_modified = dictionarymodifier(dictionary2) #some function that modifies the chosen element
list_modified = [i for i in list_original] #makes copy of original list
for i,n in enumerate(dictionary_original):
if i==1:
list_modified[1] = dictionary2_modified #replaces element with modified version
return list_original, list_modified
And many similar things, but I always either get two of the original list or two of the new list! I add that I'm using python 2.4 and don't have a choice in that.
Many thanks for any help
Mutable vs. Immutable
You need to know the difference between mutable and immutable elements. Namely, both dictionaries and lists in Python are mutable. Which means that if you modify it in one place, it is also modified in the other place.
In addition, the variable of mutable type (like list or dict) can contain immutable elements (eg. str), as well as the other way around: variable of immutable type (eg. tuple) can contain mutable elements (such as list or dict).
Example for mutability
So, this shows the mutability using the example of list:
>>> a = [1, 2, 3, 4]
>>> b = a
>>> a
[1, 2, 3, 4]
>>> b
[1, 2, 3, 4]
>>> a[2] = 'x'
>>> a
[1, 2, 'x', 4]
>>> b
[1, 2, 'x', 4]
How to obtain a copy of list or dict
To obtain a copy of list, you simply can do this instead:
new_list = old_list[:] # the slicing at the end just takes the whole list
In case of dict this is generally sufficient:
new_dict = old_dict.copy()
Nested lists / dicts
However, although lists / dicts that are flat or contain only mutable elements, can be copied the way I showed, to obtain a copy of more complex mutable data structures you need to do something more...
In such case very helpful may be the copy module with its deepcopy function. Documentation of copy module says more about its purpose:
Assignment statements in Python do not copy objects, they create bindings between a target and an object. For collections that are mutable or contain mutable items, a copy is sometimes needed so one can change one copy without changing the other. This module provides generic shallow and deep copy operations (explained below).
Is your dictionarymodifier actually mutating dictionary2 in place? If so, the way you build your list is irrelevant.
Simply using list_modified = list(list_original) works fine to create a shallow copy of the list, which you can then modify to your heart's content, but only if you don't modify the items in the original list (which you can't if they're immutable built-in things like numbers or strings, so beginners often mistake this for a deep copy).
If you really need to copy the list, you can use copy.deepcopy to do so.
You need to create a copy of the list.
copy=list(original)
original[0] = None
Situation: After making a copy of the original list I use pop to modify said copy. As it turns out, the original list gets affected by the change.
I even after checking the original list and the copy are not the same object, poping an element of the copy will will pop the same element in the original.
See below for an example of the script. Thanks in advance for your help.
l = [['1412898', 'Jack', 'headache med', '8ET-500'],
['1423859', 'Sonny', 'prostate med', '8ET-800'],
['1413836', 'Paco', 'headache med', '8ET-500']]
class App(object):
def __init__(self, info):
self.fp_rows= info
def sortbyauditor(self):
self.fp_rows_copy = self.fp_rows[:]
print self.fp_rows is self.fp_rows_copy
for i in self.fp_rows_copy:
i.pop(1)
print self.fp_rows_copy
print self.fp_rows
app= App(l)
app.sortbyauditor()
some_list[:] is only a shallow copy. You seem to need a deep copy
from copy import deepcopy
copy = deepcopy(some_list)
Edit
To understand why "one objects affects the other" take a look at the id of each list:
original = [[1, 2], [3, 4]]
shallow = original[:]
deep = deepcopy(original)
print([id(l) for l in original])
# [2122937089096, 2122937087880]
print([id(l) for l in shallow])
# [2122937089096, 2122937087880]
print([id(l) for l in deep])
# [2122937088968, 2122937089672]
You can see that the ids of the lists in original are the same as the ids in shallow. That means the nested lists are the exact same objects. When you modify one nested list the changes are also in the other list.
The ids for deep are different. That are just copies. Changing them does not affect the original list.