Related
This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 3 years ago.
I have a function that generates all permutations of a string. It prints out all the possible permutations just fine. But now I want a list of all such permutations.
I tried making the global list as well as tried passing it as a parameter, but post appending the permutation all the lists previously in the main list get changed to the list last appended. Please explain this behavior
def permutationNum(a,lower,upper,perm):
if(lower==upper):
print(a)
print(perm)
perm.append(a)
# perm = perm.append(a)
else:
for i in range(lower,upper+1):
a[lower],a[i] = a[i],a[lower]
permutationNum(a,lower+1,upper, perm)
a[lower],a[i] = a[i],a[lower]
listy = [1,2,3]
perm = []
permutationNum(listy, 0, len(listy)-1, perm)
print(perm)
Output : [[1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3]]
Expected Output : [[1, 2, 3], [1, 3, 2], [2, 1, 3], [2, 3, 1], [3, 2, 1], [3, 1, 2]]
UPDATE:
Turns out it was indeed deep copy problem after all. I just had a temp variable store a deep copy of a and appended that temp variable to the list. It all worked out.
In python, certain data types are copied when passed into functions as arguments, while others are referenced.
If an argument is copied, any changes to it inside the function will not affect the original variable that was passed in.
If an argument is referenced, any changes to it inside the function will affect the original.
Strings, Ints, Floats are copied, while objects and lists are referenced. This behaviour is also replicated when assigning one variable to another:
a = 5
b = a
b = 6
print(a)
>>> 5
a = [5]
b = a
b.append(6)
print(a)
>>> [5, 6]
If you want to copy a list and not just reference it, there's multiple ways you can achieve this:
Copy Module
import copy
a = [5]
b = copy.copy(a)
b.append(6)
print(a)
>>> [5]
Slicing
a = [5]
b = a[:]
b.append(6)
print(a)
>>> [5]
.copy()
a = [5]
b = a.copy()
b.append(6)
print(a)
>>> [5]
list()
a = [5]
b = list(a)
b.append(6)
print(a)
>>> [5]
So in your case, you would change the following line from:
permutationNum(a,lower+1,upper, perm) to
permutationNum(a[:],lower+1,upper, perm)
Change this line - to append new instance of list everytime
perm.append(list(a))
Another way to get permutations:
import itertools
def permutationNum(a):
for x in itertools.permutations(a):
perm.append(list(x))
listy = [1,2,3]
perm = []
permutationNum(listy)
print(perm)
or
import itertools
def permutationNum(a):
return [list(x) for x in itertools.permutations(a)]
listy = [1,2,3]
print(permutationNum(listy))
I want to understand small snippet here in python:
>>> x = ['foo', [1,2,3], 10.4]
>>> y = list(x)
>>> y[0]
'foo'
>>> y[0] = "fooooooo"
>>> y[1]
[1, 2, 3]
>>> y[1][0]=4
>>> print x
['foo', [4, 2, 3], 10.4]
>>> print y
['fooooooo', [4, 2, 3], 10.4]
>>> z = ['foo', [1,2,3], 10.4]
>>> x = ['foo', [1,2,3], 10.4]
>>> y = list(x)
>>> y[0] = "fooooooo"
>>> y[1]
[1, 2, 3]
>>> y[1][0]=4
>>> print x
['foo', [4, 2, 3], 10.4]
>>> print y
['fooooooo', [4, 2, 3], 10.4]
>>> print z
['foo', [1, 2, 3], 10.4]
>>> y = list(z)
>>> y[1][0]=6
>>> print y
['foo', [6, 2, 3], 10.4]
>>> y = list(z)
>>> print z
['foo', [6, 2, 3], 10.4]
>>> print x
['foo', [4, 2, 3], 10.4]
How this works.
if change the list element of y its getting reflecting to x. It may be very basic of python but still i am not getting hold on this
y = list(x)
Above statement creates a shallow copy of the list x.
From the documentation:
A shallow copy constructs a new compound object and then (to the extent possible) inserts references into it to the objects found in the original.
In here, you do get a new object y but since the list inside it is a mutable object, you get the reference to the original object. And, if you keep creating these shallow copies the list object will be shared among all the copies.
You can check it using id:
>>> id(x)
140183460314288
>>> id(y)
140183460372992 # this is different from y
>>> id(x[1])
140183460314864
>>> id(y[1]) # this is same as x[1]
140183460314864
>>> y1 = list(y) # another shallow copy from y
>>> id(y1[1])
140183460314864 # this is still same
If you expect a behavior where you do need to modify the contents of y without affecting x, you need to perform deepcopy:
>>> >>> from copy import deepcopy
>>> z = deepcopy(x)
>>> id(z[1])
140183460405400 # this is different now because of deepcopy
You can see that this id is different than id(x[1]), and now if you try to modify the contents they won't be reflected in x.
I'm guessing the only thing you're confused by is the fact that setting the elements of the nested list of one variable will change the nested lists of all other variables. The reason is simple: Python is using the same nested list (as in, the exact same memory) in each variable. When you say y = list(x), Python copies all of the atomic elements of x into y but merely copies a reference to the nested list. Since the same nested list is used everywhere, modifying it one place modifies it everywhere.
You can also see similar behavior by playing around with l1 = [0]*3 and l2 = [[0]]*3; the differences between how l1 and l2 behave are another example of the behavior you're observing.
>>> x = ['foo', [1,2,3], 10.4]
>>> y = list(x)
y is a copy of x. y[1] contains a copy of the reference in x to the list [1,2,3]. So y[1] and x[1] are 2 references to the same list.
Is it possible to pass a slice of a list into a function and modify the list via the slice?
This doesn't seem to work:
def foo(a_list):
a_list[0]='abc'
x=[1,2,3,4]
foo(x[0:2])
I want x to now be x=['abc',2,3,4]
No. A "list slice" in the sense you describe is nothing but a new list. When you do x[0:2], the resulting list does not "know" that it was created as a slice of another list. It's just a new list.
What you could do is pass in the slice object itself, separately from the list:
def foo(a_list, a_slice):
a_list[a_slice]='abc'
>>> foo(x, slice(0, 2))
>>> x
['a', 'b', 'c', 3, 4]
No! Slices create copies of lists (see the documentation). You can do this:
>>> x = [1, 2, 3, 4]
>>> x[1:3] = [7, 8, 9]
>>> X
[1, 7, 8, 9, 4]
But, when you get a new list from a slicing operation, it's a copy, and thus changes to it won't affect the original:
>>> x = [1, 2, 3, 4]
>>> y = x[1:3]
>>> y[0] = 5
>>> y
[5, 3]
>>> x
[1, 2, 3, 4]
def foo(a_list):
a_list[0]='abc'
return a_list
x=[1,2,3,4]
foo(x) #it returns x=['abc',2,3,4]
Assumably you're trying to pass in x[0:2], rather than x[0,2], but the reason it doesn't work is because when you create a slice you are creating a subarray copy of x.
You are not operating on the same instance of that array, what you are doing is passing an entirely new array. Passing in 'x' alone would work, but passing in x[0:2] would not unless you specifically wrote it as x[0:2] = foo(x[0:2]) and had foo() return a_list.
As brenns10 explained, slices create a copy (even in python 3.0) of your origional data.
You could do something like the following:
def foo(x):
x[0] = 'abc'
return x
x = [0, 1, 2, 3]
x[0:2] = foo(x[0:2]) # x = ['abc', 1, 2, 3]
While this gives you the desired outcome, it doesn't exactly work as you would want. This could be problematic if needing to perform large slices as you'd have to perform a lot of copying.
So say I have the following lists:
a = [1,2,3]
b = [5,2,3]
c = [5,4,2]
I would like to append the unique items from each list as they are looped over into a new array to end up with:
unique_list = [1,2,3,5,4]
I still don't full comprehend list comprehension for more advanced cases, however I'm thinking the following clearly incorrect code would convey my intentions:
def append_unique(new_list):
unique_list.append(item) for item in new_list if item not in unique_list
unique_list = []
append_unique([1,2,3])
append_unique([5,2,3])
append_unique([5,4,2])
Is this even possible with a one-liner, or should I concede and go for a nested solution?
UPDATE
Sorry I don't think I've conveyed this particularly well, I need to add each additional list's unique items as part of a loop, hence why each needs to individually pass through append_unique()
I attempted to modify append_unique() to use set() as per the following:
def append_unique(new_list):
unique_list = list(set(unique_list + new_list))
unique_list = []
append_unique([1,2,3])
append_unique([5,2,3])
append_unique([5,4,2])
The problem here of course is that I get the error, which I don't fully understand how to get around:
local variable 'unique_list' referenced before assignment
If order does not matter, you can use sets:
>>> a = [1,2,3]
>>> b = [5,2,3]
>>> c = [5,4,2]
>>> set(a+b+c)
set([1, 2, 3, 4, 5])
If it does, then you can use itertools.groupby():
>>> from itertools import groupby
>>> res = []
>>> for ele, _ in groupby(a+b+c):
... if ele not in res:
... res.append(ele)
...
>>> res
[1, 2, 3, 5, 4]
If you don't mind using sets, the following code might work for you:
>>> a = [1,2,3]
>>> b = [5,2,3]
>>> c = [5,4,2]
>>> my_set = set(a) | set(b) | set(c)
>>> my_set
set([1, 2, 3, 4, 5])
>>> unique_list = list(my_set)
>>> unique_list
[1, 2, 3, 4, 5]
If I have list=[1,2,3] and I want to add 1 to each element to get the output [2,3,4],
how would I do that?
I assume I would use a for loop but not sure exactly how.
new_list = [x+1 for x in my_list]
The other answers on list comprehension are probably the best bet for simple addition, but if you have a more complex function that you needed to apply to all the elements then map may be a good fit.
In your example it would be:
>>> map(lambda x:x+1, [1,2,3])
[2,3,4]
>>> mylist = [1,2,3]
>>> [x+1 for x in mylist]
[2, 3, 4]
>>>
list-comprehensions python.
if you want to use numpy there is another method as follows
import numpy as np
list1 = [1,2,3]
list1 = list(np.asarray(list1) + 1)
Edit: this isn't in-place
Firstly don't use the word 'list' for your variable. It shadows the keyword list.
The best way is to do it in place using splicing, note the [:] denotes a splice:
>>> _list=[1,2,3]
>>> _list[:]=[i+1 for i in _list]
>>> _list
[2, 3, 4]
>>> [x.__add__(1) for x in [1, 3, 5]]
3: [2, 4, 6]
My intention here is to expose if the item in the list is an integer it supports various built-in functions.
Python 2+:
>>> mylist = [1,2,3]
>>> map(lambda x: x + 1, mylist)
[2, 3, 4]
Python 3+:
>>> mylist = [1,2,3]
>>> list(map(lambda x: x + 1, mylist))
[2, 3, 4]
import numpy as np
np.add([1, 2, 3], 1).tolist()
which gives
[2, 3, 4]
Came across a not so efficient, but unique way of doing it. So sharing it across.And yes it requires extra space for another list.
from operator import add
test_list1 = [4, 5, 6, 2, 10]
test_list2 = [1] * len(test_list1)
res_list = list(map(add, test_list1, test_list2))
print(test_list1)
print(test_list2)
print(res_list)
#### Output ####
[4, 5, 6, 2, 10]
[1, 1, 1, 1, 1]
[5, 6, 7, 3, 11]
list = [1,2,3,4,5]
for index in range(len(list)):
list[index] = list[index] +1
print(list)
Just in case anyone was looking for a solution that only uses built-ins and no lambdas:
from functools import partial
from operator import add
my_list = range(1, 4) # list(my_list) #=> [1, 2, 3]
my_list_plus_one = list(map(partial(add, 1), my_list) #=> [2, 3, 4]
Many of the answers above are very good. I've also seen some weird answers that will do the job. Also, the last answer seen was through a normal loop. This willingness to give answers leads me to itertools and numpy, which will do the same job in a different way.
Here I present different ways to do the job, not answered above.
import operator
import itertools
x = [3, 5, 6, 7]
integer = 89
"""
Want more vairaint can also use zip_longest from itertools instead just zip
"""
#lazy eval
a = itertools.starmap(operator.add, zip(x, [89] * len(x))) # this is not subscriptable but iterable
print(a)
for i in a:
print(i, end = ",")
# prepared list
a = list(itertools.starmap(operator.add, zip(x, [89] * len(x)))) # this returns list
print(a)
# With numpy (before this, install numpy if not present with `pip install numpy`)
import numpy
res = numpy.ones(len(x), dtype=int) * integer + x # it returns numpy array
res = numpy.array(x) + integer # you can also use this, infact there are many ways to play around
print(res)
print(res.shape) # prints structure of array, i.e. shape
# if you specifically want a list, then use tolist
res_list = res.tolist()
print(res_list)
Output
>>> <itertools.starmap object at 0x0000028793490AF0> # output by lazy val
>>> 92,94,95,96, # output of iterating above starmap object
>>> [92, 94, 95, 96] # output obtained by casting to list
>>> __
>>> # |\ | | | |\/| |__| \ /
>>> # | \| |__| | | | |
>>> [92 94 95 96] # this is numpy.ndarray object
>>> (4,) # shape of array
>>> [92, 94, 95, 96] # this is a list object (doesn't have a shape)
My sole reason to highlight the use of numpy is that one should always do such manipulations with libraries like numpy because it is performance efficient for very large arrays.