This question already has answers here:
"Deep copy" nested list without using the deepcopy function
(5 answers)
Closed 4 years ago.
Let's say i have list a, and i want to copy it to b so that i can alter a but have its original form intact:
I use the traditional list() function...
a = [1,[2,3],4]
b = list(a)
print id(a), id(b)
# 2941136 35748600
a and b have different id, so the copying is a success. but list() did not copy the sublist -- altering a[1][0] will change b
a[1][0]=3
print b
# [1, [3, 3], 4]
I'm aware of copy.deepcopy() to solve this sort of problem, but i'm wondering if there are other ways of handling this without using a module.
One way to copy nested lists (given your example) is:
def deepcopy_nested_list(data):
out = []
for el in data:
if isinstance(el, list):
out.append(deepcopy_nested_list(el))
else:
out.append(el)
return out
This function copies the list to a new list and then recursively copies all nested lists to achieve a deep copy.
Please note that this does only create copies of lists and immutable objects (e.g., dicts are not copied). It shows only the idea how you would implement such a function and does not give a full implementation.
In real world code you would of course use copy.deepcopy().
Related
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 2 years ago.
This one is making me absolutely crazy, so any help would be much appreciated. I have a program where I'm iterating through a list in a function. Here's a toy model of the problem:
masterList = ["person","woman","man","camera","television"]
workingList = masterList
def removeItem (workingList):
item = workingList.pop(2)
print("Test 1:",workingList)
removeItem(workingList)
print("Test 2:", workingList)
print("Test 3:", masterList)
As expected, "Test 1" prints out the list with an item removed.
However, "Test 2" also prints out the list with an item removed. I wasn't expecting that, but no matter. That's not the real problem, but I'm sure there's something I don't understand here about variable scopes and variable shadowing.
No the real problem is "Test 3", which as you can see, is printing out the masterList, which shouldn't even be touched by the removeItem function or the pop function within it. And yet, it too is printing out the list with an item removed.
How can this be happening, and how can I prevent it?
Thanks so much!
Cheers,
Ari
Python lists are mutable objects.
m = list([1, 2, 3])
n = m
a = 5
b = a
id(a) == id(b)
# id() return "identity" of the object.
# True, Both a and b are references to the same object.
id(m) == id(n)
# True, Both m and n are references to the same object.
b = b + 2
id(a) == id(b)
# False, a new object on separate location is created, which b will point.
n.pop()
id(m) == id(n)
# True, the object is mutated, and m and n still references to the same object.
Since, python lists are mutable, m and n will still be reference to the same object after mutation. Whereas, for immutable objects like int, a new object will be created and the identifier will refer to the new object.
Gist is, in your scenario, there has been only one object since python lists are mutable.
However, if you need the original list unchanged when the new list is modified, you can use the copy() method.
new_list = original_list.copy()
The ids of new_list and original_list is different.
Learn here about mutability in detail: https://medium.com/#meghamohan/mutable-and-immutable-side-of-python-c2145cf72747.
You have to make a copy of the masterList, otherwise workingList is nothing more than a reference.
If your list does not contain other containers (which yours doesn't), then a shallow copy is sufficient. There are numerous ways to make a shallow copy but slicing is the most optimal.
masterList = ["person","woman","man","camera","television"]
workingList = masterList[:] #copy via slice
a bunch of other ways to make a shallow copy
workingList = masterList * 1
workingList = masterList.copy()
workingList = list(masterList)
workingList = [*masterList]
import copy
workingList = copy.copy(masterList)
If you have a list that does possess something that holds a reference (like other containers, classes, etc), then you need to make a deepcopy.
import copy
a = [[1, 2, 3], ['a', 'b', 'c']]
b = copy.deepcopy(a)
Looks like I figured it out. The two lists are actually the same list unless you use list.copy()
So replacing the top two lines with:
masterList = ["person","woman","man","camera","television"]
workingList = masterList.copy()
makes everything perform as expected! Well, I still can't say I understand how the variable scopes work in their entirety, but at least this solves the major problem.
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 4 years ago.
I have a python script were I was experimenting with minmax AI. And so tried to make a tic tac toe game.
I had a self calling function to calculate the values and it used a variable called alist(not the one below) which would be given to it by the function before. it would then save it as new list and modify it.
This worked but when it came to back-tracking and viewing all the other possibilities the original alist variable had been changed by the function following it e.g.
import sys
sys.setrecursionlimit(1000000)
def somefunction(alist):
newlist = alist
newlist[0][0] = newlist[0][0] + 1
if newlist[0][0] < 10:
somefunction(newlist)
print(newlist)
thelist = [[0, 0], [0, 0]]
somefunction(thelist)
it may be that this is difficult to solve but if someone could help me it would be greatly appreciated
newlist = alist does not make a copy of the list. You just have two variable names for the same list.
There are several ways to actually copy a list. I usually do this:
newlist = alist[:]
On the other hand, that will make a new list with the same elements. To make a deep copy of the list:
import copy
newlist = copy.deepcopy(alist)
You probably want to deepcopy your list, as it contains other lists:
from copy import deepcopy
And then change:
newlist = alist
to:
newlist = deepcopy(alist)
This question already has answers here:
What is the difference between shallow copy, deepcopy and normal assignment operation?
(12 answers)
Closed 6 years ago.
Say I have a list a with some values, and I did a b = a[:]. Then modifying the contents of list b won't change list a as per what I've read. So, this means its a deep copy. But python documentation still refers to this as shallow copy. Can someone clear this for me?
To demonstrate what shallow copy means:
a = [ [1,2], [3,4,5] ]
b = a[:] # make a shallow copy
a is b # not the same object, because this is a copy
=> False
a == b # same value, because this is a copy
=> True
a[0] is b[0] # elements are the *same objects*, because this is a *shallow* copy
=> True
Changing the structure of a will not be reflected in b, because this is a copy:
a.pop()
len(a)
=> 1
len(b)
=> 2
To demonstrate the difference from a deep copy: changing an object contained in a (as opposed to a's structure) in-place, is reflected in b, because b references the same objects as a.
a[0][0] = 'XYZ'
b[0]
=> ['XYZ', 2]
From python docs
The difference between shallow and deep copying is only relevant for
compound objects (objects that contain other objects, like lists or
class instances):
A shallow copy constructs a new compound object and then (to the
extent possible) inserts references into it to the objects found in
the original. A deep copy constructs a new compound object and then,
recursively, inserts copies into it of the objects found in the
original.
Shallow copy creates a new object only for the top level/object, then copies by reference all sub-objects. Deep copy creates new object for the top object/level and for all sub-objects too.
Then modifying the contents of list "b" won't change list "a" as per what I've read.
As in, if you take out some of the contents or switch them out for other contents, it won't affect a. Mutations to the objects held in b will be visible through both lists, because both lists hold the same objects.
>>> class Mutable(object):
... def __init__(self, x):
... self.x = x
... def __repr__(self):
... return 'Mutable({})'.format(self.x)
...
>>> a = [Mutable(1), Mutable(2)]
>>> b = a[:]
>>> del b[1] # Doesn't affect a
>>> a
[Mutable(1), Mutable(2)]
>>> b[0].x = 5 # Visible through a
>>> a
[Mutable(5), Mutable(2)]
Then modifying the contents of list "b" won't change list "a" as per what I've read. So, this means its a deep copy.
No, it does not. A shallow copy differs from a deep copy in whether contained values are copied or not.
In your case, the list is copied, but the two resulting lists will contain the same objects. Adding or removing a value in one list won't affect the other list, but changes to a contained object will be reflected.
This question already has answers here:
What is the difference between shallow copy, deepcopy and normal assignment operation?
(12 answers)
Closed 8 years ago.
I was under the impression that deepcopy copied everything recursively down a tree, but I came upon a situation that seemed to go against what I previously believed.
>>> item = "hello"
>>> a["hello"] = item
>>> b = copy.deepcopy(a)
>>> id(a)
31995776
>>> id(b)
32733616 # I expected this
>>> id(a["hello"])
140651836041376
>>> id(b["hello"])
140651836041376 # I did not expect this
The id of a and b are different, which I expected, but the internal item is still the same object. Does deepcopy only copy to a certain depth? Or is this something specific to the way Python stores strings? (I got a similar result with integers as well)
deepcopy only needs to create copies of mutable objects, like lists and dictionaries. Strings and integers are immutable; they can't be changed in-place, so there's no need to explicitly create a copy, and a reference to the same object is inserted instead.
Here is a quick demo, showing the difference between lists (mutable) and tuples (immutable):
>>> import copy
>>> l = [[1, 2], (3, 4)]
>>> l2 = copy.deepcopy(l)
>>> l2[0] is l[0]
False # created new list
>>> l2[1] is l[1]
True # didn't create new tuple
I've searched around and tried a lot of stuff but I can't get this to work. I think the problem is something to do with how Python list names point to the list, rather than being the actual list, but I still can't figure it out. The situation is this (it's a list of dictionaries):
list_original = [dictionary1, dictionary2, dictionary3]
dictionary2_modified = dictionarymodifier(dictionary2) #some function that modifies the chosen element
list_modified = [i for i in list_original] #makes copy of original list
for i,n in enumerate(dictionary_original):
if i==1:
list_modified[1] = dictionary2_modified #replaces element with modified version
return list_original, list_modified
And many similar things, but I always either get two of the original list or two of the new list! I add that I'm using python 2.4 and don't have a choice in that.
Many thanks for any help
Mutable vs. Immutable
You need to know the difference between mutable and immutable elements. Namely, both dictionaries and lists in Python are mutable. Which means that if you modify it in one place, it is also modified in the other place.
In addition, the variable of mutable type (like list or dict) can contain immutable elements (eg. str), as well as the other way around: variable of immutable type (eg. tuple) can contain mutable elements (such as list or dict).
Example for mutability
So, this shows the mutability using the example of list:
>>> a = [1, 2, 3, 4]
>>> b = a
>>> a
[1, 2, 3, 4]
>>> b
[1, 2, 3, 4]
>>> a[2] = 'x'
>>> a
[1, 2, 'x', 4]
>>> b
[1, 2, 'x', 4]
How to obtain a copy of list or dict
To obtain a copy of list, you simply can do this instead:
new_list = old_list[:] # the slicing at the end just takes the whole list
In case of dict this is generally sufficient:
new_dict = old_dict.copy()
Nested lists / dicts
However, although lists / dicts that are flat or contain only mutable elements, can be copied the way I showed, to obtain a copy of more complex mutable data structures you need to do something more...
In such case very helpful may be the copy module with its deepcopy function. Documentation of copy module says more about its purpose:
Assignment statements in Python do not copy objects, they create bindings between a target and an object. For collections that are mutable or contain mutable items, a copy is sometimes needed so one can change one copy without changing the other. This module provides generic shallow and deep copy operations (explained below).
Is your dictionarymodifier actually mutating dictionary2 in place? If so, the way you build your list is irrelevant.
Simply using list_modified = list(list_original) works fine to create a shallow copy of the list, which you can then modify to your heart's content, but only if you don't modify the items in the original list (which you can't if they're immutable built-in things like numbers or strings, so beginners often mistake this for a deep copy).
If you really need to copy the list, you can use copy.deepcopy to do so.
You need to create a copy of the list.
copy=list(original)
original[0] = None