Given that in Python:
element = element + [0]
should be equal to:
element += [0]
Why does one modify a list and the other does not? Here is a example:
>>> a = [[0, 0], [0,0]]
>>> for element in a:
... element = element + [0]
...
>>> a
[[0, 0], [0, 0]]
a is not modified. But if I increment:
>>> a = [[0, 0], [0,0]]
>>> for element in a:
... element += [0]
...
>>> a
[[0, 0, 0], [0, 0, 0]]
a is modified.
Thanks,
Frank
This is a fun side-effect of += operatior, which calls __iadd__ instead of __add__.
The statement x = x + y is equivalent to x = x.__add__(y), while x += y is equivalent to x = x.__iadd__(y).
This lets the list class optimize += by extending the existing (ex, x += y is roughly equivalent to x.extend(y)) list instead of creating an entirely new list (which is what + needs to do).
For example:
>>> a = [1, 2, 3]
>>> original_a = a
>>> b = [1, 2, 3]
>>> original_b = b
>>> a += [4]
>>> b = b + [4]
>>> a is original_a
True
>>> b is original_b
False
You can see that using += maintains the identity of the left hand side (ie, a new list isn't created) while using + does not maintain the identity (ie, a new list is created).
For more, see: http://docs.python.org/library/operator.html#operator.iadd and the paragraph directly above the documentation for operator.iadd.
In the first case, element = element + [0], you are creating a new list.
In the second case, element += [0], you are modifying an existing list.
Since the list of lists, a, contains pointers to the elements, only modifying the elements will actually change things. (That is, creating a new list does not change the pointers in a.)
This is seen more clearly if we take a simple example showing how lists work:
>>> a = [1, 2, 3]
>>> b = a
>>> a = [4, 5, 6]
>>> a
[4, 5, 6]
>>> b
[1, 2, 3]
>>> a = [1, 2, 3]
>>> b = a
>>> a += [4, 5, 6]
>>> b
[1, 2, 3, 4, 5, 6]
>>> a
[1, 2, 3, 4, 5, 6]
Assigning a variable to a list simply assigns a pointer.
Adding to what others said, there is a difference in what these statements do:
element = element + [0]
does
element = element.__add__([0])
while
element += [0]
does
element = element.__iadd__([0])
__iadd__(), in this case, is free to determine what to return: the original object with a modification or a new object.
In the case of a immutable object, it must return a different one (e.g., a = b = 8; a += 9 => a is not b.
But in the case of a mutable object, such as a list, it normally modifies this one:
a = b = []
a += [8]
=> a is b.
This different behaviour reflects in your for loop:
for element in a:
element = element + [0]
=> name element gets rebound to a different object; original one remains untouched
for element in a:
element += [0]
=> original object, which is as well contained in the outer list, a, gets modified. The fact that element is reassigned is irrelevant; it is not used.
Related
I'm a bit confused about why both outputs get changed after the restore when surely it should be just one(outputs are illustrated in the notes). Surely just the first one should change? If anyone could give me a suggestion as to why this happens I'd appreciate it
def switcher(y):
# shifts two characters
temp = y[0]
y[0] = y[1]
y[1] = temp
sub = [[1,2,3],[1,2,3]]
switcher(sub[0])
sub
#[[2, 1, 3], [1, 2, 3]]
#restore
sub[0] = sub[1]
sub
# [[1, 2, 3], [1, 2, 3]]
switcher(sub[0])
sub
#[[2, 1, 3], [2, 1, 3]]
With sub[0] = sub[1] you are defining both lists to be the same object, that's why the subsequent change is applied to both of them. Do sub[0] = sub[1][:] to create a copy, for example (there is more ways of doing this for a list).
When you are doing sub[0] = sub[1], you are assigning the reference to the value at index 1 i.e. [1, 2, 3] in your case to index 0, so ultimately both the lists reside in the same memory location, and change in either makes the corresponding change to the other.
You can verify this using id builtin which represents the memory reference for a given value:
ids after initialization:
>>> sub = [[1,2,3],[1,2,3]]
>>> [id(s) for s in sub]
[1385461417096, 1385461338824]
ids after calling switcher:
>>> switcher(sub[0])
>>> [id(s) for s in sub]
[1385461417096, 1385461338824]
ids after assigning sub[0] = sub[1]:
>>> sub[0] = sub[1]
>>> [id(s) for s in sub]
[1385461338824, 1385461338824]
As you can see, the ids are same after assigning sub[0] = sub[1], both the sub-lists get changed when modifying one of them
The offending line is the assignment
sub[0] = sub[1]
Assignment never copies data.
You are telling Python that the references sub[0] and sub[1] now both point to the same list object in memory (with the content [1,2,3]).
In your specific case this is easily fixed by taking a (shallow) copy of the list on the right hand side of the assignment.
sub[0] = sub[1][:]
You have a problem with references. By defining:
sub = [[1,2,3],[1,2,3]]
you create a list of two different list, but when you do:
sub[0] = sub[1]
you are telling python to copy sub[1] into sub[0] hence for python your new vector will be:
sub <- [ reference_to_memory_where_sub1_is, sub1 ]
To avoid this behaviour you can explicitly tell python to duplicate the objects in memory. You can do this with the module "copy":
import copy
def switcher(y):
# shifts two characters
temp = y[0]
y[0] = y[1]
y[1] = temp
l1 = [1,2,3]
l2 = [1,2,3]
sub = [copy.deepcopy(l1),copy.deepcopy(l2)]
switcher(sub[0])
print(sub)
#[[2, 1, 3], [1, 2, 3]]
#restore
sub[0] = l1
print(sub)
# [[1, 2, 3], [1, 2, 3]]
switcher(sub[0])
print(sub)
#[[2, 1, 3], [2, 1, 3]]
I have the following code:
sales_data = [[12, 17, 22], [2, 10, 3], [5, 12, 13]]
scoops_sold = 0
I originally wrote:
for location in sales_data:
print(location)
for element in location:
scoops_sold + element
print(scoops_sold)
After messing around I saw I was just missing an = after the + sign.
Why does += add the sum int. of one list to another and not just +?
Currently, all of the previous answers, including the accepted answer, are entirely misleading, note the difference between these two different python constructs:
The + operator, generates a new list from the two operands:
>>> a = b = [1]
>>> a = a + [2]
>>> a, b
[1, 2], [1]
Here, both a and b are initially pointing to the same list. Then we have created a new list and rebound it to the variable a
The += operator, modifies an existing list
>>> a = b = [1]
>>> a += [2]
>>> a, b
[1, 2], [1, 2]
Here, both a and b are initially pointing to the same list. Then we have modified this same list.
The takeaway ... the syntax a = a + b is not short for a += b
+= is short for scoops_sold = scoops_sold + element. Having scoops_sold + element is computing the result but not storing it to any variable where as scoops_sold = scoops_sold + element is computing the result and assigning it to scoops_sold.
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 3 years ago.
I have a function that generates all permutations of a string. It prints out all the possible permutations just fine. But now I want a list of all such permutations.
I tried making the global list as well as tried passing it as a parameter, but post appending the permutation all the lists previously in the main list get changed to the list last appended. Please explain this behavior
def permutationNum(a,lower,upper,perm):
if(lower==upper):
print(a)
print(perm)
perm.append(a)
# perm = perm.append(a)
else:
for i in range(lower,upper+1):
a[lower],a[i] = a[i],a[lower]
permutationNum(a,lower+1,upper, perm)
a[lower],a[i] = a[i],a[lower]
listy = [1,2,3]
perm = []
permutationNum(listy, 0, len(listy)-1, perm)
print(perm)
Output : [[1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3]]
Expected Output : [[1, 2, 3], [1, 3, 2], [2, 1, 3], [2, 3, 1], [3, 2, 1], [3, 1, 2]]
UPDATE:
Turns out it was indeed deep copy problem after all. I just had a temp variable store a deep copy of a and appended that temp variable to the list. It all worked out.
In python, certain data types are copied when passed into functions as arguments, while others are referenced.
If an argument is copied, any changes to it inside the function will not affect the original variable that was passed in.
If an argument is referenced, any changes to it inside the function will affect the original.
Strings, Ints, Floats are copied, while objects and lists are referenced. This behaviour is also replicated when assigning one variable to another:
a = 5
b = a
b = 6
print(a)
>>> 5
a = [5]
b = a
b.append(6)
print(a)
>>> [5, 6]
If you want to copy a list and not just reference it, there's multiple ways you can achieve this:
Copy Module
import copy
a = [5]
b = copy.copy(a)
b.append(6)
print(a)
>>> [5]
Slicing
a = [5]
b = a[:]
b.append(6)
print(a)
>>> [5]
.copy()
a = [5]
b = a.copy()
b.append(6)
print(a)
>>> [5]
list()
a = [5]
b = list(a)
b.append(6)
print(a)
>>> [5]
So in your case, you would change the following line from:
permutationNum(a,lower+1,upper, perm) to
permutationNum(a[:],lower+1,upper, perm)
Change this line - to append new instance of list everytime
perm.append(list(a))
Another way to get permutations:
import itertools
def permutationNum(a):
for x in itertools.permutations(a):
perm.append(list(x))
listy = [1,2,3]
perm = []
permutationNum(listy)
print(perm)
or
import itertools
def permutationNum(a):
return [list(x) for x in itertools.permutations(a)]
listy = [1,2,3]
print(permutationNum(listy))
I am a little confused on how shallow copy works, my understanding is when we do new_obj = copy.copy(mutable_obj) a new object is created with elements of it still pointing to the old object.
Example of where I am confused -
## assignment
i = [1, 2, 3]
j = i
id(i[0]) == id (j[0]) # True
i[0] = 10
i # [10, 2, 3]
j # [10, 2, 3]
## shallow copy
k = copy.copy(i)
k # [10, 2, 3]
id(i) == id(k) # False (as these are two separate objects)
id(i[0]) == id (k[0]) # True (as the reference the same location, right?)
i[0] = 100
id(i[0]) == id (k[0]) # False (why did that value in that loc change?)
id(i[:]) == id (k[:]) # True (why is this still true if an element just changed?)
i # [100, 2, 3]
k # [10, 2, 3]
In shallow copy, isn't k[0] just pointing to i[0] similar to assignment? Shouldn't k[0] change when i[0] changes?
Why I expect these to be same, because -
i = [1, 2, [3]]
k = copy(i)
i # [1, 2, [3]]
k # [1, 2, [3]]
i[2].append(4)
i # [1, 2, [3, 4]]
k # [1, 2, [3, 4]]
id(i[0]) == id (k[0]) # True
id(i[2]) == id (k[2]) # True
id(i[:]) == id (k[:]) # True
id(i) == id(k) # False (as these are two separate objects)
Correct.
id(i[0]) == id (k[0]) # True (as the reference the same location, right?)
Correct.
i[0] = 100
id(i[0]) == id (k[0]) # False (why did that value in that loc change?)
It changed because you changed it in the previous line. i[0] was pointing 10, but you changed it to point to 100. Therefore, i[0] and k[0] now no longer point to the same spot.
Pointers (references) are one way. 10 does not know what is pointing to it. Neither does 100. They are just locations in memory. So if you change where i's first element is pointing to, k doesn't care (since k and i are not the same reference). k's first element is still pointing to what it always was pointing to.
id(i[:]) == id (k[:]) # True (why is this still true if an element just changed?)
This one's a bit more subtle, but note that:
>>> id([1,2,3,4,5]) == id([1,2,3])
True
whereas
>>> x = [1,2,3,4,5]
>>> y = [1,2,3]
>>> id(x) == id(y)
False
It has to do with some subtleties of garbage collection and id, and it's answered in depth here: Unnamed Python objects have the same id.
Long story short, when you say id([1,2,3,4,5]) == id([1,2,3]), the first thing that happens is we create [1,2,3,4,5]. Then we grab where it is in memory with the call to id. However, [1,2,3,4,5] is anonymous, and so the garbage collector immediately reclaims it. Then, we create another anonymous object, [1,2,3], and CPython happens to decide that it should go in the spot that it just cleaned up. [1,2,3] is also immediately deleted and cleaned up. If you store the references, though, GC can't get in the way, and then the references are different.
Mutables example
The same thing happens with mutable objects if you reassign them. Here's an example:
>>> import copy
>>> a = [ [1,2,3], [4,5,6], [7,8,9] ]
>>> b = copy.copy(a)
>>> a[0].append(123)
>>> b[0]
[1, 2, 3, 123]
>>> a
[[1, 2, 3, 123], [4, 5, 6], [7, 8, 9]]
>>> b
[[1, 2, 3, 123], [4, 5, 6], [7, 8, 9]]
>>> a[0] = [123]
>>> b[0]
[1, 2, 3, 123]
>>> a
[[123], [4, 5, 6], [7, 8, 9]]
>>> b
[[1, 2, 3, 123], [4, 5, 6], [7, 8, 9]]
The difference is when you say a[0].append(123), we're modifying whatever a[0] is pointing to. It happens to be the case that b[0] is pointing to the same object (a[0] and b[0] are references to the same object).
But if you point a[0] to a new object (through assignment, as in a[0] = [123]), then b[0] and a[0] no longer point to the same place.
In Python all things are objects. This includes integers. All lists only hold references to objects. Replacing an element of the list doesn't mean that the element itself changes.
Consider a different example:
class MyInt:
def __init__(self, v):
self.v = v
def __repr__(self):
return str(self.v)
>>> i = [MyInt(1), MyInt(2), MyInt(3)]
[1, 2, 3]
>>> j = i[:] # This achieves the same as copy.copy(i)
[1, 2, 3]
>>> j[0].v = 7
>>> j
[7, 2, 3]
>>> i
[7, 2, 3]
>>> i[0] = MyInt(1)
>>> i
[1, 2, 3]
>>> j
[7, 2, 3]
I am creating a class MyInt here which just holds an int.
By modifying an instance of the class, both lists "change". However as I replace a list entry, the lists are now different.
The same happens with integers. You just can't modify them.
In the first case j = i is an assignment, both j and i point to the same list object. When you change an element of the list object and print i and j, since both i and j point to same list object, and it is the element and not the list object which has changed, so both will print the same output.
In the second case k = copy.copy(i) is a shallow copy, in which a copy of list object and copy of nested references is made but the internal immutable objects are not copied.
A shallow copy doesn't create a copy of nested objects, instead it just copies the reference of nested objects. Please refer this https://www.programiz.com/python-programming/shallow-deep-copy
Thus i and k have different set of references pointing to the same immutable objects. When you do i[0] = 100, the reference in list i points to a new int object with value 100, but the reference in k still references the old int object with value 10.
I have a script that requires to transfer values of one vector onto another.
Code looks like:
b = [1,2,3]
for i ranging from (0,2):
a[i] = b[i] #(transfer all three elements of b to a)
Doing this gets an error
- IndexError: list assignment index out of range
What am i missing ? Thanks for help.
a = b[:]
should more than suffice
the list needs to be the right size if you are referencing it by index. ie. The list a doesn't have 3 elements. I'd just create a new list with a list constructor or even easier do this
a = list(b)
I have a script that requires to transfer values of one vector onto another
I think there is some confusion here between the variable and its content:
>>> a = [10,20]
>>> b = [1,2,3]
>>> c = a
>>> a,b,c
([10, 20], [1, 2, 3], [10, 20])
This create two lists. With two variables (a and c) referencing the same one.
If you write:
>>> a = [10,20]
>>> b = [1,2,3]
>>> c = a
>>> a = b[:]
>>> a,b,c
([1, 2, 3], [1, 2, 3], [10, 20])
You actually create a third list. An bind it to the variable a. But c still hold a reference to the original list.
If you want to really alter the original list, write that instead:
>>> a = [10,20]
>>> b = [1,2,3]
>>> c = a
>>> a[:] = b[:] # "replace every item of the first list
# by every item of the second list"
>>> a,b,c
([1, 2, 3], [1, 2, 3], [1, 2, 3])