arr = [1]
def f1(lst):
lst.append(2)
print(lst)
lst = 2
print(lst)
f1(arr)
print(arr) # [1,2]
why python call by reference parameter does not change to value?
what does lst variable indeicating when do "lst = 2"
(not connected to arr?)
Assigning lst = 2 doesn't affect the value of arr. In fact, python doesn't do "call by reference" at all.
Annotating your code with comments that might help clear it up:
arr = [1]
def f1(lst):
# lst and arr both refer to the same [1] list at this point.
# Two different and independent names for the same object.
lst.append(2) # appends 2 to [1], aka lst, aka arr
print(lst) # lst/arr is now [1, 2]
lst = 2 # reassign the name 'lst' to the value 2!
# At this point, lst refers to 2 instead of [1, 2].
# lst and arr are no longer connected.
# arr is still [1, 2] even though lst is 2.
print(lst) # indeed, lst is now 2
# but arr is still [1, 2], as seen below:
f1(arr)
print(arr) # [1,2]
https://nedbatchelder.com/text/names.html is highly recommended reading on this topic! The main thing to understand is that lst and arr are just different names that at different points in the code might refer to the same value or different values.
When you call lst.append, you are modifying the value that lst is a name for, which arr also happens to be a name for. When you say lst = 2, you are rebinding the name lst, but you are not modifying the value that it previously referred to (and to which arr still refers).
Related
What strange magic is this?
def rotate_list(lst, n):
n = n % len(lst)
lst = lst[-n:] + lst[:-n]
def rotate_list_2(lst):
lst[0], lst[1], lst[2], lst[3] = lst[3], lst[0], lst[1], lst[2]
s1 = [1, 2, 5, 4]
rotate_list(s1, 1)
print(s1)
s1 = [1, 2, 5, 4]
rotate_list_2(s1)
print(s1)
Output:
[1, 2, 5, 4]
[4, 1, 2, 5]
It appears that although lists are generally mutable within functions, if a list with the same name is created, then the original list is unaffected by changes to the new list. Could someone please explain what is happening here please, in terms of scope and references?
How would I rotate the original list without having to manually update each value as in rotate_list_2()? Or would this kind of thing generally be done by working with new lists returned from a function?
Assigning to list in function doesn't change the original reference.
The assignment just references the local parameter lst on the new value.
The original list referenced outside ('before') the function remains intact.
Insead assign to it's elements with this syntax:
def rotate_list(lst, n):
n = n % len(lst)
lst[:] = lst[-n:] + lst[:-n]
s1 = [1, 2, 5, 4]
rotate_list(s1, 1)
# And it works like magic! :)
# [4, 1, 2, 5]
print(s1)
If you reassign the argument of a function, the value does not change outside of the function scope.
def test0(a):
a = 10
print(a)
x = 4
test0(x)
print(x)
This would result in
10
4
The reason why assigning values of an array works is that you're not assigning a new value to the argument itself. You're instead accessing the memory that the array reads from, and you're changing it. Thus, those changes will happen even for outer scopes.
After change in the function you can return the list and capture it in function call like s=function(s)
>>> a = [1,2,3]
>>> b = []
>>> b.append(a)
>>> print(b)
[[1, 2, 3]]
>>> num = a.pop(0)
>>> a.append(num)
>>> print(a)
[2, 3, 1]
>>> b.append(a)
>>> print(b)
[[2, 3, 1], [2, 3, 1]]
>>>
Why is this happening and how to fix it? I need the list like
[[1, 2, 3], [2, 3, 1]]
Thank you.
Edit:
Also, why is this working?
>>> a = []
>>> b = []
>>> a = [1,2,3]
>>> b.append(a)
>>> a = [1,2,3,4]
>>> b.append(a)
>>> print(b)
[[1, 2, 3], [1, 2, 3, 4]]
>>>
'''
Append a copy of your list a, at least the first time. Otherwise, you've appended the same list both times.
b.append(a[:])
When you append the list a, python creates a reference to that variable inside the list b. So when you edit the list a, it is reflected again in the list b. You need to create a copy of your variable and then append it to get the desired result.
Every variable name in Python should be thought of as a reference to a piece of data. In your first listing, b contains two references to the same underlying object that is also referenced by the name a. That object gets changed in-place by the operations you’re using to rotate its members. The effect of that change is seen when you look at either of the two references to the object found in b, or indeed when you look at the reference associated with the name a.
Their identicality can be seen by using the id() function: id(a), id(b[0]) and id(b[1]) all return the same number, which is the unique identifier of the underlying list object that they all refer to. Or you can use the is operator: b[0] is b[1] evaluates to True.
By contrast, in the second listing, you reassign a—in other words, by using the assignment operator = you cause that name to become associated with a different object: in this case, a new list object that you just created with your square-bracketed literal expression. b still contains one reference to the old list, and now you append a new reference that points to this different piece of underlying data. So the two elements of b now look different from each other—and indeed they are different objects and accordingly have different id() numbers, only one of which is the same as the current id(a). b[0] is b[1] now evaluates to False
How to fix it? Reassign the name a before changing it: for example, create a copy:
a = list(a)
or:
import copy
a = copy.copy(a)
(or you could even use copy.deepcopy()—study the difference). Alternatively, rotate the members a using methods that entail reassignment rather than in-place changes—e.g.:
a = a[1:] + a[:1]
(NB immutable objects such as the tuple avoid this whole confusion —not because they behave fundamentally differently but because they lack methods that produce in-place changes and therefore force you to use reassignment strategies.)
In addition to making the copy of a by doing a[:] and assigning it to b.
You can also use collections.deque.rotate to rotate your list
from collections import deque
a = [1,2,3]
#Make a deque of copy of a
b = deque(a[:])
#Rotate the deque
b.rotate(len(a)-1)
#Create the list and print it
print([a,list(b)])
#[[1, 2, 3], [2, 3, 1]]
Observe the following code:
class permcom:
def __init__(self, INPUT_SET, IS_PERM, REPETITION):
self.end_set = []
self.input_set = INPUT_SET
self.is_perm = IS_PERM
self.repetition = REPETITION
def helpfunc(self, seen, depth, current):
if depth == 0:
self.end_set.append(seen)
else:
for i in range(0, len(self.input_set)):
if(self.repetition):
seen.append(self.input_set[i])
if(self.is_perm):
self.helpfunc(seen, depth - 1, 0)
else:
self.helpfunc(seen, depth - 1, i)
del seen[-1]
# return all permutations with repetition
def rapwr(INPUT_SET, subset_size):
instance = permcom(INPUT_SET, True, True)
A = []
instance.helpfunc(A, subset_size, 0)
return instance.end_set
A = [1,2,3]
B = rapwr(A, 2)
for i in range(0, len(B)):
print B[i]
The output is the following:
[]
[]
[]
[]
[]
[]
[]
[]
[]
However, the intended output is this:
[1, 1]
[1, 2]
[1, 3]
[2, 1]
[2, 2]
[2, 3]
[3, 1]
[3, 2]
[3, 3]
I've spent way too much time looking at this code and, unfortunately, I still cannot figure out exactly what's wrong. There must be something fundamental that I'm not understanding about how member variables work in Python, but I still don't quite understand what's going on here and why the code isn't working. Can somebody explain this?
Short answer
What you need is list slicing [:]. Changing the statement
if depth == 0:
self.end_set.append(seen)
to
if depth == 0:
self.end_set.append(seen[:])
Gives the expected answer
Long answer
Try this sample code in a python interpreter
a = [1,2]
b = []
b.append(a)
a[0] = 3
print b
# output is [[3, 2]]
Now try this code
a = [1,2]
b = []
b.append(a[:])
a[0] = 3
print b
# output is [[1, 2]]
Why did this happen? In the first case when you appended a to the list b, it was not the value of a that was appended, it was a reference/tag to the [1,2] value. You can verify this by printing id(b[0]) and id(a). Both will be the same value. Hence when you modify any value in the a list, the value in the b list also changes.
Same is the case in your code. Since you are doing del seen[-1], the corresponding value in self.end_set is also removed. You can confirm this by printing the value of self.end_set in the depth == 0 block.
To avoid this you append a clone of one list to the other list. This is done by using the splicing syntax [:]. This creates a copy of the list from the start to the end of the list. You can learn more about slicing here.
PS: Try printing the id() of the two lists when you use slicing, the values will be different
Here is what I got
a = [1,2]
b = []
b.append(a)
print id(b[0])
#output is 43337352L
print id(a)
#output is 43337352L
b = []
b.append(a[:])
print id(b[0])
#output is 43337608L
Take a look at this python memory model diagram for a better understanding of the above
Update: some advice
Since B and self.input_set are both lists, prefer using the idiomatic for i in B and for i in self.input_set.
Make sure your function names are understandable. It might help you out someday. Generally if you are made to write a comment for a variable or function name, it is better to name the function/variable with a shortened version of the comment itself. So rapwr can be renamed to return_all_permutations_with repetition. Though the name is large, its easy to understand what it does without looking at the method body now.
I am trying to write a function which removes the first item in a Python list. This is what I've tried. Why doesn't remove_first_wrong change l when I call the function on it? And why does the list slicing approach work when I do it in the main function?
def remove_first_wrong(lst):
lst = lst[1:]
def remove_first_right(lst):
lst.pop(0)
if __name__ == '__main__':
l = [1, 2, 3, 4, 5]
remove_first_wrong(l)
print(l)
l_2 = [1, 2, 3, 4, 5]
remove_first_right(l_2)
print(l_2)
# Why does this work and remove_first_wrong doesn't?
l_3 = [1, 2, 3, 4, 5]
l_3 = l_3[1:]
print(l_3)
Slicing a list returns a new list object, which is a copy of the original list indices you indicated in the slice. You then rebound lst (a local name in the function) to reference that new list instead. The old list is never altered in that process.
list.pop() on the other hand, operates on the list object itself. It doesn't matter what reference you used to reach the list.
You'd see the same thing without functions:
>>> a = [1, 2]
>>> b = a[:] # slice with all the elements, produces a *copy*
>>> b
[1, 2]
>>> a.pop() # remove an element from a won't change b
2
>>> b
[1, 2]
>>> a
[1]
Using [:] is one of two ways of making a shallow copy of a list, see How to clone or copy a list?
You may want to read or watch Ned Batchelder's Names and Values presestation, to further help understand how Python names and objects work.
Inside the function remove_first_wrong the = sign reassigns the name lst to the object on the right. Which is a brand new object, created by slicing operation lst[1:]. Thus, the object lst assigned to is local to that function (and it actually will disappear on return).
That is what Martijn means by "You then rebound lst (a local name in the function) to reference that new list instead."
On contrary, lst.pop(0) is a call to the given object -- it operates on the object.
For example, this will work right too:
def remove_first_right2(lst):
x = lst # x is assigned to the same object as lst
x.pop(0) # pop the item from the object
Alternately, you can use del keyword:
def remove_first_element(lst):
del lst[0]
return lst
For an assignment we were asked to create a function that would reverse all the elements in an arbitrarily nested list. So inputs to the function should return something like this:
>>> seq = [1,[2,[3]]]
>>> print arb_reverse(seq)
[[[3],2],1]
>>> seq = [9,[[],[0,1,[[],[2,[[],3]]]],[],[[[4],5]]]]
>>> print arb_reverse(seq)
[[[[5,[4]]],[],[[[[3,[]],2],[]],1,0],[]],9]
I came up with a recursive solution which works well:
def arb_reverse(seq):
result = []
for element in reversed(seq):
if not is_list(element):
result.append(element)
else:
result.append(arb_reverse(element))
return result
But for a bit of a personal challenge I wanted to create a solution without the use of recursion. One version of this attempt resulted in some curious behavior which I am not understanding. For clarification, I was NOT expecting this version to work properly but the resulting input mutation does not make sense. Here is the iterative version in question:
def arb_reverse(seq):
elements = list(seq) #so input is not mutated, also tried seq[:] just to be thorough
result = []
while elements:
item = elements.pop()
if isinstance(item, list):
item.reverse() #this operation seems to be the culprit
elements += item
else:
result.append(item)
return result
This returns a flattened semi-reversed list (somewhat expected), but the interesting part is what it does to the input (not expected)...
>>> a = [1, [2, [3]]]
>>> arb_reverse(a)
[2, 3, 1]
>>> a
[1, [[3], 2]]
>>> p = [1, [2, 3, [4, [5, 6]]]]
>>> print arb_reverse(p)
[2, 3, 4, 5, 6, 1]
>>> print p
[1, [[[6, 5], 4], 3, 2]]
I was under the impression that by passing the values contained in the input to a variable using list() or input[:] as i did with elements, that I would avoid mutating the input. However, a few print statements later revealed that the reverse method had a hand in mutating the original list. Why is that?
The list() call is making a new list with shallow-copied lists from the original.
Try this (stolen from here):
from copy import deepcopy
listB = deepcopy(listA)
Try running the following code through this tool http://people.csail.mit.edu/pgbovine/python/tutor.html
o1 = [1, 2, 3]
o2 = [4, 5, 6]
l1 = [o1, o2]
l2 = list(l1)
l2[0].reverse()
print l2
print l1
Specifically look at what happens when l2[0].reverse() is called.
You'll see that when you call list() to create a copy of the list, the lists still reference the same objects.