See my code in python 3.4. I can get around it fine. It bugs me a little. I'm guessing it's something to do with foo2 resetting a rather than treating it as list 1.
def foo1(a):
a.append(3) ### add element 3 to end of list
return()
def foo2(a):
a=a+[3] #### add element 3 to end of list
return()
list1=[1,2]
foo1(list1)
print(list1) ### shows [1,2,3]
list1=[1,2]
foo2(list1)
print(list1) #### shows [1,2]
In foo2 you do not mutate the original list referred to by a - instead, you create a new list from list1 and [3], and bind the result which is a new list to the local name a. So list1 is not changed at all.
There is a difference between append and +=
>>> a = []
>>> id(a)
11814312
>>> a.append("hello")
>>> id(a)
11814312
>>> b = []
>>> id(b)
11828720
>>> c = b + ["hello"]
>>> id(c)
11833752
>>> b += ["hello"]
>>> id(b)
11828720
As you can see, append and += have the same result; they add the item to the list, without producing a new list. Using + adds the two lists and produces a new list.
In the first example, you're using a method that modifies a in-place. In the second example, you're making a new a that replaces the old a but without modifying the old a - that's usually what happens when you use the = to assign a new value. One exception is when you use slicing notation on the left-hand side: a[:] = a + [3] would work as your first example did.
Related
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 4 years ago.
When I ran this script (Python v2.6):
a = [1,2]
b = a
a.append(3)
print a
>>>> [1,2,3]
print b
>>>> [1,2,3]
I expected print b to output [1,2]. Why did b get changed when all I did was change a? Is b permanently tied to a? If so, can I make them independent? How?
Memory management in Python involves a private heap memory location containing all Python objects and data structures.
Python's runtime only deals in references to objects (which all live in the heap): what goes on Python's stack are always references to values that live elsewhere.
>>> a = [1, 2]
>>> b = a
>>> a.append(3)
Here we can clearly see that the variable b is bound to the same object as a.
You can use the is operator to tests if two objects are physically the same, that means if they have the same address in memory. This can also be tested also using the id() function.
>>> a is b
>>> True
>>> id(a) == id(b)
>>> True
So, in this case, you must explicitly ask for a copy.
Once you've done that, there will be no more connection between the two distinct list objects.
>>> b = list(a)
>>> a is b
>>> False
Objects in Python are stored by reference—you aren't assigning the value of a to b, but a pointer to the object that a is pointing to.
To emulate assignation by value, you can make a copy like so:
import copy
b = copy.copy(a)
# now the code works as "expected"
Be aware this has performance disadvantages.
In the case of an array, there's a special method that relies on slices:
b = a[:]
# code also works as expected here
Update– In addition to this, with some objects you can use the constructor—this includes lists:
b = list(a)
Short answer - Pointers.
When you type b = a it is setting b to look at the same array that a looks at. You have to make a new array with copies of the elements to separate them. In this case, something like b = [n for n in a] would work fine. For more complex operations you may want to check out http://docs.python.org/library/copy.html.
You might want to look at this link. The problem you have here is a and b both point to the same memory location, so changing one changes the other. Instead, you want to do something like this:
a = [1,2]
b = list(a)
a is a pointer to the list [1,2].
When you do the assignment b = a the value of b is the address of the list [1,2].
So when you do a.append(3) you are not actually changing a, you are changing the list that a points to. Since a and b both point to the same list, they both appear to change when you modify the other.
If you simply want to copy the contents of list a to b, instead of making b a pointer to a:
b = a[:]
Using the slice operator will copy the contents of the list into b such that you example would become:
a = [1,2]
b = a[:]
a.append(3)
print a
>>>> [1,2,3]
print b
>>>> [1,2]
Still new, I searched google but I didn't really find an explanation so thank you in advance. When writing Python, why is is that during a while loop, a variable that is assigned a list outside of the while loop can be modified and retain it's modifications with each iteration, but a variable that is assigned a list during the loop will revert back to it's original assignment with each iteration? Here is an example code:
x = [1,1,1]
while True:
a = [1,1,1]
b = x
a.append(1)
b.append(2)
print(a)
print(b)
why does variable a stay the same, while variable b can be appended with each iteration and maintain its modifications? Thanks.
Since b and x are references to the same list object, your code is equivalent to
x = [1,1,1]
while True:
a = [1,1,1]
a.append(1)
x.append(2)
print(a)
print(x)
In each iteration of the loop, the list referenced by x gains another element. a is a reference to a new 3-element list in each iteration, though, so it never gets bigger than 4 elements before being discarded. It's easier to see if you unroll the (original) loop:
x = [1,1,1]
a = [1,1,1] # new list
b = x # reference to old list
a.append(1)
b.append(2)
print(a)
print(b)
a = [1,1,1] # new list
b = x # reference to old list
a.append(1)
b.append(2)
print(a)
print(b)
...
You can also sprinkle in a few calls to print(id(a)) and print(id(b)) to see that id(b) never changes, while id(a) changes in each iteration. (Due to how the garbage collector works, id(a) will probably oscillate back and forth between 2 distinct values, though.)
It's because a is redefined at each iteration, while b isn't. When you write b = x, you only link b to x, and modifying b in your loop is in fact changing only the list that it refers to, the same list that x refers to which isn't redefined at each iteration (unlike a).
This is a classic case of Shallow Copy vs Deep Copy.
This code below will give you results as expected by you.
import copy
x = [1,1,1]
while True:
a = [1,1,1]
b = copy.deepcopy(x)
a.append(1)
b.append(2)
print(a)
print(b)
Assignment statements by default do not copy objects, they create bindings between a target and an object.
Shallow copy is by default.
To completely copy the values, we need deepcopy as shown in the above code
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 4 years ago.
When I ran this script (Python v2.6):
a = [1,2]
b = a
a.append(3)
print a
>>>> [1,2,3]
print b
>>>> [1,2,3]
I expected print b to output [1,2]. Why did b get changed when all I did was change a? Is b permanently tied to a? If so, can I make them independent? How?
Memory management in Python involves a private heap memory location containing all Python objects and data structures.
Python's runtime only deals in references to objects (which all live in the heap): what goes on Python's stack are always references to values that live elsewhere.
>>> a = [1, 2]
>>> b = a
>>> a.append(3)
Here we can clearly see that the variable b is bound to the same object as a.
You can use the is operator to tests if two objects are physically the same, that means if they have the same address in memory. This can also be tested also using the id() function.
>>> a is b
>>> True
>>> id(a) == id(b)
>>> True
So, in this case, you must explicitly ask for a copy.
Once you've done that, there will be no more connection between the two distinct list objects.
>>> b = list(a)
>>> a is b
>>> False
Objects in Python are stored by reference—you aren't assigning the value of a to b, but a pointer to the object that a is pointing to.
To emulate assignation by value, you can make a copy like so:
import copy
b = copy.copy(a)
# now the code works as "expected"
Be aware this has performance disadvantages.
In the case of an array, there's a special method that relies on slices:
b = a[:]
# code also works as expected here
Update– In addition to this, with some objects you can use the constructor—this includes lists:
b = list(a)
Short answer - Pointers.
When you type b = a it is setting b to look at the same array that a looks at. You have to make a new array with copies of the elements to separate them. In this case, something like b = [n for n in a] would work fine. For more complex operations you may want to check out http://docs.python.org/library/copy.html.
You might want to look at this link. The problem you have here is a and b both point to the same memory location, so changing one changes the other. Instead, you want to do something like this:
a = [1,2]
b = list(a)
a is a pointer to the list [1,2].
When you do the assignment b = a the value of b is the address of the list [1,2].
So when you do a.append(3) you are not actually changing a, you are changing the list that a points to. Since a and b both point to the same list, they both appear to change when you modify the other.
If you simply want to copy the contents of list a to b, instead of making b a pointer to a:
b = a[:]
Using the slice operator will copy the contents of the list into b such that you example would become:
a = [1,2]
b = a[:]
a.append(3)
print a
>>>> [1,2,3]
print b
>>>> [1,2]
Context: I needed to randomly erase some precise element of a few lists of numbers, extracting some random indexes and saving them in a set called aleaindex (done, it properly works, thanks to some SO users' help). Now, I'd like to substitute the old lists a, b, etc with the new, eventually shorter ones newa, newb, etc. Here is the function:
def myfunction(N, other_parameters, a, b, c):
...
while (...):
aleaindex.add(random.randint(..., ...))
...
new_a = [v for i, v in enumerate(a) if i not in aleaindex]
while a: a.pop()
a = new_a[:]
...
and so on for the other lists b, c, etc.
Problem: the function seems to correctly modify them within the module (checked by printing) but then, when I print the modified lists outside the module, that is in the "main" file, lists are as they had not modified. Where am I wrong?
This line:
a=new_a[:]
overwrites the variable a with a new object. Outside the function or module, the old object is still pointed at by a (or whatever it was called there). Try:
new_a = [v for i, v in enumerate(a) if i not in aleaindex]
while a:
a.pop()
a[:] = new_a[:]
Explanation
To see this, just try the following.
>>> a = [1,2,3,4]
>>> b = a
>>> print b
[1, 2, 3, 4]
>>> a[:] = [2,3]
>>> print b
[2, 3]
>>> a = [5]
>>> print b
[2, 3]
Example in function!
If the variable is mutable (and a normal list is), this works:
>>> def f(a):
... a[0] = 2
>>> b = [3]
>>> f(b)
>>> print b
[2]
Variables are not passed by value - you can edit a mutable value.
I do not know what you are trying to do but from your snippets you are clearly lost. Your code does not make much sense and there are more more than one problem. Nonetheless, the problem you asked about - why the list is not fully changed? - seems to be related to this loop:
while a: a.pop()
a = new_a[:]
Suppose we call your function this way:
list1 = [1, 2, 3, 4, 5, 6, 7]
myfunction(N, other_parameters, list1, [], [])
What will happen is, when you call the first line, you will get a variable called list1 and it will point to a list:
When you call the function myfunction(), the function, among other things, create a variable called a which will point to the same list pointed by list1:
So far, so good. Then we get at the loop below:
while a:
a.pop()
a = new_a[:]
In the first line of it (a.pop()), you get an item out of the list. Since both variables a and list1 points to the same list, you would see the same result...
...if it were not for the next line of the loop (a = new_a[:]). In this line, you are making the a variable to point to another list:
Now, every operation you execute on a will be in this list, which is in no way related to list1. For example, you can execute a.pop() at the next iteration to get it:
However, it makes no sense at all, because the line a = new_a[:] will replace the list pointed to a again for yet another different list:
So, what is the solution? I don't know. As I have said, it is not possible (to me, at least) to make sense from your code. You have to reflect a bit more about what you are trying to do and explain it to us, with a bit more of context.
There is no function in the code you have posted. I suspect the problem is that you are not returning the new value.
Your code likely does something like:
a = "foo"
def func():
a = "bar" # uh-oh this not the same as the global a
func()
At this point global a is unchanged because the a local to func is not the same variable.
You want to do:
a = "foo"
def func():
return "bar"
a = func()
That code assigns to the a in global scope, changing it.
I occasionally see the list slice syntax used in Python code like this:
newList = oldList[:]
Surely this is just the same as:
newList = oldList
Or am I missing something?
[:] Shallow copies the list, making a copy of the list structure containing references to the original list members. This means that operations on the copy do not affect the structure of the original. However, if you do something to the list members, both lists still refer to them, so the updates will show up if the members are accessed through the original.
A Deep Copy would make copies of all the list members as well.
The code snippet below shows a shallow copy in action.
# ================================================================
# === ShallowCopy.py =============================================
# ================================================================
#
class Foo:
def __init__(self, data):
self._data = data
aa = Foo ('aaa')
bb = Foo ('bbb')
# The initial list has two elements containing 'aaa' and 'bbb'
OldList = [aa,bb]
print OldList[0]._data
# The shallow copy makes a new list pointing to the old elements
NewList = OldList[:]
print NewList[0]._data
# Updating one of the elements through the new list sees the
# change reflected when you access that element through the
# old list.
NewList[0]._data = 'xxx'
print OldList[0]._data
# Updating the new list to point to something new is not reflected
# in the old list.
NewList[0] = Foo ('ccc')
print NewList[0]._data
print OldList[0]._data
Running it in a python shell gives the following transcript. We can see the
list being made with copies of the old objects. One of the objects can have
its state updated by reference through the old list, and the updates can be
seen when the object is accessed through the old list. Finally, changing a
reference in the new list can be seen to not reflect in the old list, as the
new list is now referring to a different object.
>>> # ================================================================
... # === ShallowCopy.py =============================================
... # ================================================================
... #
... class Foo:
... def __init__(self, data):
... self._data = data
...
>>> aa = Foo ('aaa')
>>> bb = Foo ('bbb')
>>>
>>> # The initial list has two elements containing 'aaa' and 'bbb'
... OldList = [aa,bb]
>>> print OldList[0]._data
aaa
>>>
>>> # The shallow copy makes a new list pointing to the old elements
... NewList = OldList[:]
>>> print NewList[0]._data
aaa
>>>
>>> # Updating one of the elements through the new list sees the
... # change reflected when you access that element through the
... # old list.
... NewList[0]._data = 'xxx'
>>> print OldList[0]._data
xxx
>>>
>>> # Updating the new list to point to something new is not reflected
... # in the old list.
... NewList[0] = Foo ('ccc')
>>> print NewList[0]._data
ccc
>>> print OldList[0]._data
xxx
Like NXC said, Python variable names actually point to an object, and not a specific spot in memory.
newList = oldList would create two different variables that point to the same object, therefore, changing oldList would also change newList.
However, when you do newList = oldList[:], it "slices" the list, and creates a new list. The default values for [:] are 0 and the end of the list, so it copies everything. Therefore, it creates a new list with all the data contained in the first one, but both can be altered without changing the other.
As it has already been answered, I'll simply add a simple demonstration:
>>> a = [1, 2, 3, 4]
>>> b = a
>>> c = a[:]
>>> b[2] = 10
>>> c[3] = 20
>>> a
[1, 2, 10, 4]
>>> b
[1, 2, 10, 4]
>>> c
[1, 2, 3, 20]
Never think that 'a = b' in Python means 'copy b to a'. If there are variables on both sides, you can't really know that. Instead, think of it as 'give b the additional name a'.
If b is an immutable object (like a number, tuple or a string), then yes, the effect is that you get a copy. But that's because when you deal with immutables (which maybe should have been called read only, unchangeable or WORM) you always get a copy, by definition.
If b is a mutable, you always have to do something extra to be sure you have a true copy. Always. With lists, it's as simple as a slice: a = b[:].
Mutability is also the reason that this:
def myfunction(mylist=[]):
pass
... doesn't quite do what you think it does.
If you're from a C-background: what's left of the '=' is a pointer, always. All variables are pointers, always. If you put variables in a list: a = [b, c], you've put pointers to the values pointed to by b and c in a list pointed to by a. If you then set a[0] = d, the pointer in position 0 is now pointing to whatever d points to.
See also the copy-module: http://docs.python.org/library/copy.html
Shallow Copy: (copies chunks of memory from one location to another)
a = ['one','two','three']
b = a[:]
b[1] = 2
print id(a), a #Output: 1077248300 ['one', 'two', 'three']
print id(b), b #Output: 1077248908 ['one', 2, 'three']
Deep Copy: (Copies object reference)
a = ['one','two','three']
b = a
b[1] = 2
print id(a), a #Output: 1077248300 ['one', 2, 'three']
print id(b), b #Output: 1077248300 ['one', 2, 'three']