I want to get all the different choices from a list of lists in python. for example, take a list like,
list=[[a,b,c],[a,d,e]]
I just want to select choices from one at each list and get all the possible different selections.
so above list possible choices look like,
[a,a]
[a,d]
[a,e]
[b,a]
[b,d]
[b,e]
[c,a]
[c,d]
[c,e]
without using any external library, how can I do this?
Easiest to implement with a recursive generator.
def yield_combos(lst):
if lst:
for el in lst[0]:
for combo in yield_combos(lst[1:]):
yield [el] + combo
else:
yield []
lst = [['a','b','c'], ['a','d','e']]
for combo in yield_combos(lst):
print(combo)
If you prefer the output as a list, you can always convert it into one:
print(list(yield_combos(lst)))
Minor notes:
I've called the variable lst where the question uses list; it's not a good idea to override a builtin, and in this case doing so would prevent the call to list from working.
I've set some string values in the caller for sake of a self-contained program (the question had a in place of 'a', etc).
The simplest way to do this is with two functions
def funcA(x, arr):
return [(x,c) for c in arr]
def funcB(arr1, arr2):
ans = []
for x in arr1:
ans.extend(funcA(x, arr2))
return ans
Call it whichever way to need:
funcB(your_list[0], your_list[1])
The most pythonic way is with itertools
you can simply use product from the itertools library like so:
my_list = [[a,b,c],[a,d,e]]
# will contain: [(a,a), (a,d)...]
all_combinations = list(itertools.product(*my_list))
The "*" unpacks the list into the lists it contains, its like writing:
# will contain: [(a,a), (a,d)...]
list(itertools.product([a,b,c],[a,d,e]))
It is worth mentioning that this works for any number of sublists.
my_list = [[a,b,c],[a,d,e], [1,2,3]]
# will contain: [(a,a,1), (a,a,2)...]
all_combinations = list(itertools.product(*my_list))
Hope that helps
you could use generators,
perm = [(i,j) for i in list[0] for j in list[1]]
furthermore if you could convert this into a generator object, if you memory concerns,
perm = ((i,j) for i in list[0] for j in list[1]])
I was told you have n lists and want to permutate all of them,
you can use a while loop
while(len(list)>1):
a = list.pop()
b = list.pop()
list.append([(i, j) for i in a for j in b])
or make a recursive function. Do note, that the answer will be list[0] and not list itself.
l=[['a','b','c'],['a','d','e']]
def combinations(L, tmp=None):
if tmp is None:
tmp = []
if L==[]:
print (tmp)
else:
for i in L[0]:
product(L[1:], tmp+[i])
combinations(l)
Does this help?
Related
I am working on a function that can reverse the list similar to reverse(). I tried both building a function using slicing and also tried looking old posts and following a similar logic. I understand the logic behind reversing the elements but mechanically I don't understand why the elements remain unreversed at the end of the function.
def reverse_list(listofval):
newlist = []
index = 0
while index < len(listofval):
newlist.append(listofval[len(listofval) - 1 - index])
index += 1
return newlist
So the above function is just taking the old list (list of val) and keep reading the old list backwards then adding each element in reverse order (last element in old list became first, first became last). But "return newlist" seems to return an unmodified list.
def reverse_list(listofval):
newlist = listofval[::-1]
return newlist
Similarly I have build another function which is more straight forward using slicing and when new list returned, nothing is changed. I guess it must be something wrong with "return newlist" but I am not entirely sure what mistakes I made there.
Thanks a lot guys!
If you want to reverse a list in place, you have to modify the passed list:
E.g. by swapping elements:
def reverse_list(lst):
for i in range(len(lst) // 2):
lst[i], lst[len(lst)-1-i] = lst[len(lst)-1-i], lst[i]
or slice assignment:
def reverse_list(lst):
lst[:] = lst[::-1]
Note that the function no longer returns anything. But the passed list will be reversed after calling it:
>>> lst = [1,2,3]
>>> reverse_list(lst)
>>> lst
[3, 2, 1]
The way to get returned vale is this.
def retList():
list = []
for i in range(0,10):
list.append(i)
return list
a = retList()
print a
you can also use global variable for new list before defining the newlist[] in your code
global newlist
newlist = []
you can also use append left
from collections import deque
def reverse_list(listofval):
global newlist
newlist = []
for i in listofval:
newlist = deque(newlist)
newlist.appendleft(i)
print(newlist)
listofval = [1, 2, 3, 4, 5]
reverse_list(listofval)
print(newlist)
I have a list of strings that look like:
my_list = ['https://www.google.com/', 'http://www.google.com/',
'https://www.google.com', 'http://www.google.com']
As you can see they are not the same but they all look very similar.
I also have a function which is:
from fuzzywuzzy import fuzz
def similar(a, b):
return fuzz.ratio(a,b)
I want to use this functions and say something like:
for a,b in my_list:
print (a,b)
if similar(a,b) > 0.95:
my_list.remove(b)
So I'm trying to remove similar looking strings from a list if they are above a certain similarity ratio. I want to do this so that in this list I would end up with just the first url, in this case my_list would end up being:
my_list = ['https://www.google.com/']
After doing some googling, I found fuzzywuzzy has an inbuilt function which is pretty great.
from fuzzywuzzy.process import dedupe
deduped_list = list(dedupe(my_list, threshold=97, scorer=fuzz.ratio))
In general you should never use list.remove() within an iteration loop, because the list iterator will get confused when you remove an item from the same list you are iterating over.
And because you always want to keep the first item you can exclude it from the loop:
idx = 1
while idx < len(my_list):
if similar(my_list[idx - 1], my_list[idx]) > 0.95:
my_list.remove(my_list[idx])
print(my_list)
alternative solution with list comprehension
first_item = my_list[0]
my_list = [first_item] + [item for item in my_list[1:] if similar(first_item, item) <= 0.95]
print(my_list)
This question already has answers here:
How can I use list comprehensions to process a nested list?
(13 answers)
Closed 7 months ago.
I recently looked for a way to flatten a nested python list, like this: [[1,2,3],[4,5,6]], into this: [1,2,3,4,5,6].
Stackoverflow was helpful as ever and I found a post with this ingenious list comprehension:
l = [[1,2,3],[4,5,6]]
flattened_l = [item for sublist in l for item in sublist]
I thought I understood how list comprehensions work, but apparently I haven't got the faintest idea. What puzzles me most is that besides the comprehension above, this also runs (although it doesn't give the same result):
exactly_the_same_as_l = [item for item in sublist for sublist in l]
Can someone explain how python interprets these things? Based on the second comprension, I would expect that python interprets it back to front, but apparently that is not always the case. If it were, the first comprehension should throw an error, because 'sublist' does not exist. My mind is completely warped, help!
Let's take a look at your list comprehension then, but first let's start with list comprehension at it's easiest.
l = [1,2,3,4,5]
print [x for x in l] # prints [1, 2, 3, 4, 5]
You can look at this the same as a for loop structured like so:
for x in l:
print x
Now let's look at another one:
l = [1,2,3,4,5]
a = [x for x in l if x % 2 == 0]
print a # prints [2,4]
That is the exact same as this:
a = []
l = [1,2,3,4,5]
for x in l:
if x % 2 == 0:
a.append(x)
print a # prints [2,4]
Now let's take a look at the examples you provided.
l = [[1,2,3],[4,5,6]]
flattened_l = [item for sublist in l for item in sublist]
print flattened_l # prints [1,2,3,4,5,6]
For list comprehension start at the farthest to the left for loop and work your way in. The variable, item, in this case, is what will be added. It will produce this equivalent:
l = [[1,2,3],[4,5,6]]
flattened_l = []
for sublist in l:
for item in sublist:
flattened_l.append(item)
Now for the last one
exactly_the_same_as_l = [item for item in sublist for sublist in l]
Using the same knowledge we can create a for loop and see how it would behave:
for item in sublist:
for sublist in l:
exactly_the_same_as_l.append(item)
Now the only reason the above one works is because when flattened_l was created, it also created sublist. It is a scoping reason to why that did not throw an error. If you ran that without defining the flattened_l first, you would get a NameError
The for loops are evaluated from left to right. Any list comprehension can be re-written as a for loop, as follows:
l = [[1,2,3],[4,5,6]]
flattened_l = []
for sublist in l:
for item in sublist:
flattened_l.append(item)
The above is the correct code for flattening a list, whether you choose to write it concisely as a list comprehension, or in this extended version.
The second list comprehension you wrote will raise a NameError, as 'sublist' has not yet been defined. You can see this by writing the list comprehension as a for loop:
l = [[1,2,3],[4,5,6]]
flattened_l = []
for item in sublist:
for sublist in l:
flattened_l.append(item)
The only reason you didn't see the error when you ran your code was because you had previously defined sublist when implementing your first list comprehension.
For more information, you may want to check out Guido's tutorial on list comprehensions.
For the lazy dev that wants a quick answer:
>>> a = [[1,2], [3,4]]
>>> [i for g in a for i in g]
[1, 2, 3, 4]
While this approach definitely works for flattening lists, I wouldn't recommend it unless your sublists are known to be very small (1 or 2 elements each).
I've done a bit of profiling with timeit and found that this takes roughly 2-3 times longer than using a single loop and calling extend…
def flatten(l):
flattened = []
for sublist in l:
flattened.extend(sublist)
return flattened
While it's not as pretty, the speedup is significant. I suppose this works so well because extend can more efficiently copy the whole sublist at once instead of copying each element, one at a time. I would recommend using extend if you know your sublists are medium-to-large in size. The larger the sublist, the bigger the speedup.
One final caveat: obviously, this only holds true if you need to eagerly form this flattened list. Perhaps you'll be sorting it later, for example. If you're ultimately going to just loop through the list as-is, this will not be any better than using the nested loops approach outlined by others. But for that use case, you want to return a generator instead of a list for the added benefit of laziness…
def flatten(l):
return (item for sublist in l for item in sublist) # note the parens
Note, of course, that the sort of comprehension will only "flatten" a list of lists (or list of other iterables). Also if you pass it a list of strings you'll "flatten" it into a list of characters.
To generalize this in a meaningful way you first want to be able to cleanly distinguish between strings (or bytearrays) and other types of sequences (or other Iterables). So let's start with a simple function:
import collections
def non_str_seq(p):
'''p is putatively a sequence and not a string nor bytearray'''
return isinstance(p, collections.Iterable) and not (isinstance(p, str) or isinstance(p, bytearray))
Using that we can then build a recursive function to flatten any
def flatten(s):
'''Recursively flatten any sequence of objects
'''
results = list()
if non_str_seq(s):
for each in s:
results.extend(flatten(each))
else:
results.append(s)
return results
There are probably more elegant ways to do this. But this works for all the Python built-in types that I know of. Simple objects (numbers, strings, instances of None, True, False are all returned wrapped in list. Dictionaries are returned as lists of keys (in hash order).
lis a list that I want to explore in order to suppress some items. The function do.i.want.to.suppres.i returns TRUE or FALSE in order to tell me whether I want the suppression. The details of this function is not important.
I tried this:
l = [1,4,2,3,5,3,5,2]
for i in l:
if do.i.want.to.suppress.i(i):
del i
print l
but l does not change! So I tried
l = [1,4,2,3,5,3,5,2]
for position,i in enumerate(l):
if do.i.want.to.suppress.i(i):
del l[position]
But then the problem is that the position does not match the object i as lget modified during the loop.
I could do something like this:
l = [1,4,2,3,5,3,5,2]
for position,i in enumerate(l):
if do.i.want.to.suppress.i(i):
l[position] = 'bulls'
l = [x for x in l if x!='bulls']
But I guess there should have a smarter solution. Do you have one?
l = [item for item in my_list if not do_I_suppress(item)]
List comprehensions! learn them! love them! live them!
The list comprehension approach is the most pythonic way, but if you really need to modify the list itself then I found this to be the best approach, nicer than the while loop approach:
for position in xrange(len(l) - 1, -1, -1):
i = l[position]
if do.i.want.to.suppress.i(i):
del l[position]
This is a good place to use a while loop
i = 0
while i < len(l):
if do.i.want.to.suppress.i(i):
del l[i]
else:
i = i + 1
Besides List Comprehension (which returns a list, creating the full list in memory):
filtered_list = [itm for itm in lst if i_want_to_keep(itm)]
You can use filter() (same result as List Comprehensions)
filtered_list = filter(i_want_to_keep, lst)
or itertools.ifilter() (which returns an iterator and avoid creating the whole list in memory, specially useful for iterating)
import itertools
filtered_list = itertools.ifilter(i_want_to_keep, lst)
for itm in filtered_list:
do_whatever(itm)
filter will also work:
answer = filter(lambda x: not do_I_suppress(x), lis)
Note that in Python 3.x, you will need to put filter in list:
answer = list(filter(lambda x: not do_I_suppress(x), lis))
I have a list of strings. I have a function that given a string returns 0 or 1. How can I delete all strings in the list for which the function returns 0?
[x for x in lst if fn(x) != 0]
This is a "list comprehension", one of Python's nicest pieces of syntactical sugar that often takes lines of code in other languages and additional variable declarations, etc.
See:
http://docs.python.org/tutorial/datastructures.html#list-comprehensions
I would use a generator expression over a list comprehension to avoid a potentially large, intermediate list.
result = (x for x in l if f(x))
# print it, or something
print list(result)
Like a list comprehension, this will not modify your original list, in place.
edit: see the bottom for the best answer.
If you need to mutate an existing list, for example because you have another reference to it somewhere else, you'll need to actually remove the values from the list.
I'm not aware of any such function in Python, but something like this would work (untested code):
def cull_list(lst, pred):
"""Removes all values from ``lst`` which for which ``pred(v)`` is false."""
def remove_all(v):
"""Remove all instances of ``v`` from ``lst``"""
try:
while True:
lst.remove(v)
except ValueError:
pass
values = set(lst)
for v in values:
if not pred(v):
remove_all(v)
A probably more-efficient alternative that may look a bit too much like C code for some people's taste:
def efficient_cull_list(lst, pred):
end = len(lst)
i = 0
while i < end:
if not pred(lst[i]):
del lst[i]
end -= 1
else:
i += 1
edit...: as Aaron pointed out in the comments, this can be done much more cleanly with something like
def reversed_cull_list(lst, pred):
for i in range(len(lst) - 1, -1, -1):
if not pred(lst[i]):
del lst[i]
...edit
The trick with these routines is that using a function like enumerate, as suggested by (an) other responder(s), will not take into account the fact that elements of the list have been removed. The only way (that I know of) to do that is to just track the index manually instead of allowing python to do the iteration. There's bound to be a speed compromise there, so it may end up being better just to do something like
lst[:] = (v for v in lst if pred(v))
Actually, now that I think of it, this is by far the most sensible way to do an 'in-place' filter on a list. The generator's values are iterated before filling lst's elements with them, so there are no index conflict issues. If you want to make this more explicit just do
lst[:] = [v for v in lst if pred(v)]
I don't think it will make much difference in this case, in terms of efficiency.
Either of these last two approaches will, if I understand correctly how they actually work, make an extra copy of the list, so one of the bona fide in-place solutions mentioned above would be better if you're dealing with some "huge tracts of land."
>>> s = [1, 2, 3, 4, 5, 6]
>>> def f(x):
... if x<=2: return 0
... else: return 1
>>> for n,x in enumerate(s):
... if f(x) == 0: s[n]=None
>>> s=filter(None,s)
>>> s
[3, 4, 5, 6]
With a generator expression:
alist[:] = (item for item in alist if afunction(item))
Functional:
alist[:] = filter(afunction, alist)
or:
import itertools
alist[:] = itertools.ifilter(afunction, alist)
All equivalent.
You can also use a list comprehension:
alist = [item for item in alist if afunction(item)]
An in-place modification:
import collections
indexes_to_delete= collections.deque(
idx
for idx, item in enumerate(alist)
if afunction(item))
while indexes_to_delete:
del alist[indexes_to_delete.pop()]