This question already has answers here:
Why does list(my_list) modify the object?
(2 answers)
Closed 9 years ago.
So I came across something very weird in python. I tried adding a reference to the list to itself. The code might help demonstrate what I am saying better than I can express. I am using IDLE editor(interactive mode).
>>>l=[1,2,3]
>>>l.append(l)
>>>print(l)
[1,2,3,[...]]
>>>del l[:-1]
>>>print(l)
[[...]]
So far the output is as expected. But when I do this.
y=l[:]
print(y)
To me it seems that the output should be
[[...]]
But it is
[[[...]]]
Apparently instead of creating a copy of the list, it puts a reference to the list in y.
y[0] is l returns True. I can't seem to find a good explanation for this. Any ideas?
The difference is only in the way the list is displayed. I.e. the value of y is exactly what you'd expect.
The difference in the way the lists are displayed results from the fact that, unlike l, y is not a self-referencing list:
l[0] is l
=> True
y[0] is y
=> False
y is not self-referencing, because y does not reference y. It references l, which is self-referencing.
Therefor, the logic which translates the list to a string detects the potential infinite-recursion one level deeper when working on y, than on l.
This is perfectly expected. When Python prints recursive lists, it checks that the list it is printing hasn't yet been encountered and if it has prints [...]. An important point to understand is that it doesn't test for equality (as in ==) but for identity (as in is). Therefore,
when you print l = [l]. You have l[0] is l returns True and therefore it prints [[...]].
now y = l[:] makes a copy of l and therefore y is l returns False. So here is what happens. It starts printing y so it prints [ ??? ] where ??? is replaced by the printing of y[0]. Now y[0] is l and is not y. So it prints [[???]] with ??? replaced by y[0][0]. Now y[0][0] is l which has already been encountered. So it prints [...] for it giving finally [[[...]]].
You need to have a full copy of the objects. You need to use copy.deepcopy and you would see the expected results.
>>> from copy import deepcopy
>>> l=[1,2,3]
>>> l.append(l)
>>> print(l)
[1, 2, 3, [...]]
>>> del l[:-1]
>>> print(l)
[[...]]
>>> y=deepcopy(l)
>>> print(y)
[[...]]
>>> y[0] is l
False
>>>
When you use the slice notation to copy the list, the inner references are retained which cause the behavior that you observe.
Slicing generates list of items. There is only one item - list "l". So, we have new list of one element - list "l".
Related
This question already has answers here:
Variable assignment and modification (in python) [duplicate]
(6 answers)
Closed 4 years ago.
How come the following code:
a = [1,2,3]
b = a
b[0] = 3
print(a)
will print list b after it has been altered?[3,2,3].
Also why is this true but that the following code:
a = [1,2,3]
b = a
b = [0,0,0]
print(a,b)
prints [1, 2, 3] [0, 0, 0]?? This seems inconsistent. If the first code is true, then shouldn't the second code print [0,0,0][0,0,0]? Can someone please provide an explanation for this?
In python there are two types of data... mutable and immutable. Numbers, strings, boolean, tuples, and other simple types are immutable. Dicts, lists, sets, objects, classes, and other complex types are mutable.
When you say:
a = [1,2,3]
b = a
You've created a single mutable list in memory, assigned a to point to it, and then assigned b to point to it. It's the same thing in memory.
Therefore when you mutate it (modify it):
b[0] = 3
It is a modification (mutation) of the index [0] of the value which b points to at that same memory location.
However, when you replace it:
b = [0,0,0]
It is creating a new mutable list in memory and assigning b to point at it.
Check out the id() function. It will tell you the "address" of any variable. You can see which names are pointing to the same memory location with id(varname).
Bonus: Every value in python is passed by reference... meaning that when you assign it to a variable it simply causes that variable to point to that value where it was in memory. Having immutable types allows python to "reuse" the same memory location for common immutable types.
Consider some common values when the interpreter starts up:
>>> import sys
>>> sys.getrefcount('abc')
68
>>> sys.getrefcount(100)
110
>>> sys.getrefcount(2)
6471
However, a value that is definitely not present would return 2. This has to do with the fact that a couple of references to that value were in-use during the call to sys.getrefcount
>>> sys.getrefcount('nope not me. I am definitely not here already.')
2
Notice that an empty tuple has a lot of references:
>>> sys.getrefcount(tuple())
34571
But an empty list has no extra references:
>>> sys.getrefcount(list())
1
Why is this? Because tuple is immutable so it is fine to share that value across any number of variables. However, lists are mutable so they MUST NOT be shared across arbitrary variables or changes to one would affect the others.
Incidentally, this is also why you must NEVER use mutable types as default argument values to functions. Consider this innocent little function:
>>> def foo(value=[]):
... value.append(1)
... print(value)
...
...
When you call it you might expect to get [1] printed...
>>> foo()
[1]
However, when you call it again, you prob. won't expect to get [1,1] out... ???
>>> foo()
[1, 1]
And on and on...
>>> foo()
[1, 1, 1]
>>> foo()
[1, 1, 1, 1]
WHY IS THIS? Because default arguments to functions are evaluated once during function definition, and not at function run time. That way if you use a mutable value as a default argument value, then you will be stuck with that one value, mutating in unexpected ways as the function is called multiple times.
The proper way to do it is this:
>>> def foo(value=None):
... if value is None:
... value = []
... value.append(1)
... print(value)
...
...
>>>
>>> foo()
[1]
>>> foo()
[1]
>>> foo()
[1]
This question already has answers here:
Understanding slicing
(38 answers)
Closed 5 years ago.
list = sorted(set(list))
list[:] = sorted(set(list))
list[::] = sorted(set(list))
I am new to Python, and the first thing I am noticing is that the syntax is concise, but non obvious.
For example, it is not clear what is going on in the three statements above. I ran them and got some results and seems like statement 1 is not updating the list, while statement 2 and statement 3 are. But, I am sure there is more going on here.
What do each of the above assignments mean?
2 and 3 do the same (the step argument of a slice is optional, and both these slices use the default step of 1), but they both are inherently different from 1. Slice assignment (lst[:] = ...) mutates the original object while a common assignment (lst = ...) rebinds the variable to a new object.
>>> lst = [3,3,2,2,1,1]
>>> id(lst)
139793325565704
>>> lst[:] = sorted(set(lst))
>>> lst
[1, 2, 3]
>>> id(lst)
139793325565704 # same object
>>> lst = sorted(set(lst))
>>> id(lst)
139793325524744 # different object
A point worth noting is that slice assignment can have any iterable on the rhs (for partial slices their number of elements must match the length of the slice):
>>> lst = [1,2,3]
>>> lst[1:] = 'ab'
>>> lst
[1, 'a', 'b']
See some of the slice docs for more detailed information.
I'm still new to Python, and I've been making a small function that reverses a list of lists, both the original list and the lists inside. This is my code:
def deep_reverse(L):
L.reverse()
L = [i.reverse() for i in L]
Now this code works perfectly, but if I do a small change and rearrange the lines like this:
def deep_reverse(L):
L = [i.reverse() for i in L]
L.reverse()
suddenly it stops working! It only reverses the internal lists but not the original one. Putting some debugging print() statements inside, I can see the first code reverses the original list after the first line and it's printed, but the second code actually prints a list containing 'None' as elements after reversing the list. Can anyone please explain why this behavior and what is the difference between the two codes?
The reverse() function reverses a list in-place and returns None, that explains the weird behavior. A correct implementation would be:
def deep_reverse(L):
ans = [i[::-1] for i in L]
ans.reverse()
return ans
Also, it's a bad idea to reassign and/or mutate a parameter to a function, it can lead to unexpected results. Sometimes functions in the standard library do it for efficiency reasons (for example, sort() and reverse()), that's ok - but it can lead to confusions, like the one you just experienced. Your code doesn't have to be written in that fashion unless strictly necessary.
Your first deep_reverse function reassigned L, but it is not a global parameter and is not returned in your function. Hence this variable is lost. HOWEVER, you are mutating the list in place, hence the modifications remain which is why it still works! Your original function is equivalent to the following (note there is no final assignment):
def deep_reverse(L):
L.reverse()
[i.reverse() for i in L]
This should probably be written using a for-loop:
def deep_reverse_2(L):
L.reverse()
for i in L:
i.reverse()
L = [[1, 2, 3], [2, 3, 4]]
deep_reverse_2(L)
>>> L
[[4, 3, 2], [3, 2, 1]]
The second function does not work because you reassign L inside the function (it is now local to the function and not the same L variable you passed in to the function). They would have different memory locations if you checked using id. Given that nothing is returned, this new L list is lost, as are the modifications made to it.
This question already has answers here:
Create list of single item repeated N times
(9 answers)
Closed 5 years ago.
I need to get same raw multiple times like bellow.
A=[2,3,4,5]
B=[[2,3,4,5],[2,3,4,5],[2,3,4,5],[2,3,4,5],...,[2,3,4,5]]
Number of rows can be change.
How can I do this problem?
Try this:
B = [[i for i in A] for n in range(5)]
Change the value in the range () call to change the number of copies. This doesn't suffer from reference sharing like the other answers.
online example
Or a shorter equivalent:
B = [A[:] for n in range(5)]
This also doesn't suffer from reference sharing because the slice operator creates a deep copy* as well.
online example
* at least in terms of this input
As has been pointed out, just doing list comprehension in the simplest way will only make multiple references to the same old list. To instead ensure we get a new copy of the list, we can do:
A = [2, 3, 4, 5]
B = [list(A) for i in range(10)]
where list(A) ensures that we get a copy of A instead of a reference to the same list.
You can also do:
B=([A.copy()]*10)
However, it can produce references between the lists in B if their elements are complex object. But if you're using it for ints, strings and others basic stuffs it's going to work well.
A = [1,2,3]
B= []
n= 5
for i in range(5):
B.append(A)
where A is list, B is resultant list and n is number of rows
Is there a a better way to remove the last N elements of a list.
for i in range(0,n):
lst.pop( )
Works for n >= 1
>>> L = [1,2,3, 4, 5]
>>> n=2
>>> del L[-n:]
>>> L
[1, 2, 3]
if you wish to remove the last n elements, in other words, keep first len - n elements:
lst = lst[:len(lst)-n]
Note: This is not an in memory operation. It would create a shallow copy.
As Vincenzooo correctly says, the pythonic lst[:-n] does not work when n==0.
The following works for all n>=0:
lst = lst[:-n or None]
I like this solution because it is kind of readable in English too: "return a slice omitting the last n elements or none (if none needs to be omitted)".
This solution works because of the following:
x or y evaluates to x when x is logically true (e.g., when it is not 0, "", False, None, ...) and to y otherwise. So -n or None is -n when n!=0 and None when n==0.
When slicing, None is equivalent to omitting the value, so lst[:None] is the same as lst[:] (see here).
As noted by #swK, this solution creates a new list (but immediately discards the old one unless it's referenced elsewhere) rather than editing the original one. This is often not a problem in terms of performance as creating a new list in one go is often faster than removing one element at the time (unless n<<len(lst)). It is also often not a problem in terms of space as usually the members of the list take more space than the list itself (unless it's a list of small objects like bytes or the list has many duplicated entries). Please also note that this solution is not exactly equivalent to the OP's: if the original list is referenced by other variables, this solution will not modify (shorten) the other copies unlike in the OP's code.
A possible solution (in the same style as my original one) that works for n>=0 but: a) does not create a copy of the list; and b) also affects other references to the same list, could be the following:
lst[-n:n and None] = []
This is definitely not readable and should not be used. Actually, even my original solution requires too much understanding of the language to be quickly read and univocally understood by everyone. I wouldn't use either in any real code and I think the best solution is that by #wonder.mice: a[len(a)-n:] = [].
Just try to del it like this.
del list[-n:]
I see this was asked a long ago, but none of the answers did it for me; what if we want to get a list without the last N elements, but keep the original one: you just do list[:-n]. If you need to handle cases where n may equal 0, you do list[:-n or None].
>>> a = [1,2,3,4,5,6,7]
>>> b = a[:-4]
>>> b
[1, 2, 3]
>>> a
[1, 1, 2, 3, 4, 5, 7]
As simple as that.
Should be using this:
a[len(a)-n:] = []
or this:
del a[len(a)-n:]
It's much faster, since it really removes items from existing array. The opposite (a = a[:len(a)-1]) creates new list object and less efficient.
>>> timeit.timeit("a = a[:len(a)-1]\na.append(1)", setup="a=range(100)", number=10000000)
6.833014965057373
>>> timeit.timeit("a[len(a)-1:] = []\na.append(1)", setup="a=range(100)", number=10000000)
2.0737061500549316
>>> timeit.timeit("a[-1:] = []\na.append(1)", setup="a=range(100)", number=10000000)
1.507638931274414
>>> timeit.timeit("del a[-1:]\na.append(1)", setup="a=range(100)", number=10000000)
1.2029790878295898
If 0 < n you can use a[-n:] = [] or del a[-n:] which is even faster.
This is one of the cases in which being pythonic doesn't work for me and can give hidden bugs or mess.
None of the solutions above works for the case n=0.
Using l[:len(l)-n] works in the general case:
l=range(4)
for n in [2,1,0]: #test values for numbers of points to cut
print n,l[:len(l)-n]
This is useful for example inside a function to trim edges of a vector, where you want to leave the possibility not to cut anything.