Related
How can a list of indices (called "indlst"), such as [[1,0], [3,1,2]] which corresponds to elements [1][0] and [3][1][2] of a given list (called "lst"), be used to access their respective elements? For example, given
indlst = [[1,0], [3,1,2]]
lst = ["a", ["b","c"], "d", ["e", ["f", "g", "h"]]]
(required output) = [lst[1][0],lst[3][1][2]]
The output should correspond to ["b","h"]. I have no idea where to start, let alone find an efficient way to do it (as I don't think parsing strings is the most pythonic way to go about it).
EDIT: I should mention that the nested level of the indices is variable, so while [1,0] has two elements in it, [3,1,2] has three, and so forth. (examples changed accordingly).
Recursion can grab arbitrary/deeply indexed items from nested lists:
indlst = [[1,0], [3,1,2]]
lst = ["a", ["b","c"], "d", ["e", ["f", "g", "h"]]]
#(required output) = [lst[1][0],lst[3][1][2]]
def nested_lookup(nlst, idexs):
if len(idexs) == 1:
return nlst[idexs[0]]
return nested_lookup(nlst[idexs[0]], idexs[1::])
reqout = [nested_lookup(lst, i) for i in indlst]
print(reqout)
dindx = [[2], [3, 0], [0], [2], [3, 1, 2], [3, 0], [0], [2]]
reqout = [nested_lookup(lst, i) for i in dindx]
print(reqout)
['b', 'h']
['d', 'e', 'a', 'd', 'h', 'e', 'a', 'd']
I also found that arbitrary extra zero indices are fine:
lst[1][0][0]
Out[36]: 'b'
lst[3][1][2]
Out[37]: 'h'
lst[3][1][2][0][0]
Out[38]: 'h'
So if you actually know the max nesting depth you can fill in the index list values by overwriting your (variable number, shorter) index list values into the max fixed length dictionary primed with zeros using the .update() dictonary method
Then directly hard code the indices of nested list, which ignores any "extra" hard coded zero valued indices
below hard coded 4 depth:
def fix_depth_nested_lookup(nlst, idexs):
reqout = []
for i in idexs:
ind = dict.fromkeys(range(4), 0)
ind.update(dict(enumerate(i)))
reqout.append(nlst[ind[0]][ind[1]][ind[2]][ind[3]])
return reqout
print(fix_depth_nested_lookup(lst, indlst))
['b', 'h']
you can try this code block:
required_output = []
for i,j in indlst:
required_output.append(lst[i][j])
You can just iterate through and collect the value.
>>> for i,j in indlst:
... print(lst[i][j])
...
b
f
Or, you can use a simple list comprehension to form a list from those values.
>>> [lst[i][j] for i,j in indlst]
['b', 'f']
Edit:
For variable length, you can do the following:
>>> for i in indlst:
... temp = lst
... for j in i:
... temp = temp[j]
... print(temp)
...
b
h
You can form a list with functions.reduce and list comprehension.
>>> from functools import reduce
>>> [reduce(lambda temp, x: temp[x], i,lst) for i in indlst]
['b', 'h']
N.B. this is a python3 solution. For python2, you can just ignore the import statement.
I have a very long list in wich I would like to replace strings. I have made a simplified example below to illustrate my problem.
my_list = ['a7_1_1', 'a7_2_1', 'a7_3_1','a7_1_2', 'a7_2_2', 'a7_3_2','a7_1_3', 'a7_2_3', 'a7_3_3']
Out[12]:
['a7_1_1',
'a7_2_1',
'a7_3_1',
'a7_1_2',
'a7_2_2',
'a7_3_2',
'a7_1_3',
'a7_2_3',
'a7_3_3'
I would like to replace the strings with a suffix added to the first 3 strings so the final list should look like:
my_new_list = ['a7_1_1', 'a7_2_1', 'a7_3_1','a7_1_1.1', 'a7_2_1.1', 'a7_3_1.1','a7_1_1.2', 'a7_2_1.2', 'a7_3_1.2']
Out[15]:
['a7_1_1',
'a7_2_1',
'a7_3_1',
'a7_1_1.1',
'a7_2_1.1',
'a7_3_1.1',
'a7_1_1.2',
'a7_2_1.2',
'a7_3_1.2']
Is there an easy way to do this?
Using itertools.cycle() function
import itertools as it #1
def cycle_first_n(lst, n):
""" cycles through first n elements of the list """
c = it.cycle(lst[:n]) #2
for idx in xrange(len(lst)): #3
sfx = idx / n
yield c.next() + ('.' + str(sfx) if sfx > 0 else '') #4
itertools is a library for creating iterators for
efficient looping
creates an iterator to cycles through a slice of n elements of
the list
use xrange rather than range to avoid creating a presumably long
list in memory (see the question)
yield means we are creating a generator. Again to avoid creating a
long list in memory
How to use the function
lst = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']
for o in cycle_first_n(lst, 3):
print o,
Output
a b c a.1 b.1 c.1 a.2 b.2
I'm not very clear with what you mean. Check if you want to do this:
>>> my_list = ['a7_1_1', 'a7_2_1', 'a7_3_1','a7_1_2', 'a7_2_2', 'a7_3_2','a7_1_3', 'a7_2_3', 'a7_3_3']
>>> my_new_list = sum([[x, x+'.1', x+'.2'] for x in my_list[:3]], [])
>>> print(my_new_list)
['a7_1_1', 'a7_1_1.1', 'a7_1_1.2', 'a7_2_1', 'a7_2_1.1', 'a7_2_1.2', 'a7_3_1', 'a7_3_1.1', 'a7_3_1.2']
I want to get all the 3 letter permutations possible from every letter in the alphabet using itertools. This comes back blank:
import itertools
def permutations(ABCDEFGHIJKLMNOPQRSTUVWXYZ, r=3):
pool = tuple(iterable)
n = len(pool)
r = n if r is None else r
for indices in product(range(n), repeat=r):
if len(set(indices)) == r:
yield tuple(pool[i] for i in indices)
What am I doing wrong?
You are a bit mixed up, that is just code explaining what permutations does. itertools is actually written in C code, the python equivalent is just given to show how it works.
>>> from itertools import permutations
>>> from string import ascii_uppercase
>>> for x in permutations(ascii_uppercase, r=3):
print x
('A', 'B', 'C')
('A', 'B', 'D')
('A', 'B', 'E')
('A', 'B', 'F')
.....
That should work fine
The code in the itertools.permutations documentation explains how the function is implemented, not how to use it. You want to do this:
perms = itertools.permutations('ABCDEFGHIJKLMNOPQRSTUVWXYZ', r=3)
You can print them all out by converting it to a list (print(list(perms))), but you can just iterate over them in a for loop if you want to do something else with them - eg,
for perm in perms:
...
I am having problems with 'splitting' a larger list into several of it's combinations. Here is an example:
Let's say I have this list:
x = [['a','b'],['c','f'],['q','w','t']]
and I want to end up with
x = [['a','b'],['c','f'],['q','w'],['q','t'],['w','t']]
so essentially
['q','w','t']
becomes
['q','w'],['q','t'],['w','t']
I see how I can convert
['q','w','t']
to
[['q','w'],['q','t'],['w','t']] #notice the extra brackets
with itertools combinations, but then I am stuck with
x = [['a','b'],['c','f'],[['q','w'],['q','t'],['w','t']]] #notice the extra brackets
Which is not what I want.
How should I do this?
EDIT:
Here is the "solution", that does not give me the result that I want:
from itertools import combinations
x = [['a','b'],['c','f'],['q','w','t']]
new_x = []
for sublist in x:
if len(sublist) == 2:
new_x.append(sublist)
if len(sublist) > 2:
new_x.append([list(ele) for ele in (combinations(sublist,2))])
Thank You
I generally use a nested list comprehension to flatten a list like this:
>>> x = [['a','b'],['c','f'],['q','w','t']]
>>> [c for s in x for c in itertools.combinations(s, 2)]
[('a', 'b'), ('c', 'f'), ('q', 'w'), ('q', 't'), ('w', 't')]
Not the best way to do it but pretty clear for understanding:
from itertools import combinations
a = [['a','b'],['c','f'],['q','w','t']]
def get_new_list(x):
newList = []
for l in x:
if len(l) > 2:
newList.extend([list(sl) for sl in combinations(l, 2)])
else:
newList.append(l)
return newList
print get_new_list(a)
>>> [['a','b'],['c','f'],['q','w'],['q','t'],['w','t']]
This question already has answers here:
Python group by
(9 answers)
Closed last month.
Let's say I have a list like this:
mylist = [["A",0], ["B",1], ["C",0], ["D",2], ["E",2]]
How can I most elegantly group this to get this list output in Python:
[["A", "C"], ["B"], ["D", "E"]]
So the values are grouped by the secound value but the order is preserved...
values = set(map(lambda x:x[1], mylist))
newlist = [[y[0] for y in mylist if y[1]==x] for x in values]
from operator import itemgetter
from itertools import groupby
lki = [["A",0], ["B",1], ["C",0], ["D",2], ["E",2]]
lki.sort(key=itemgetter(1))
glo = [[x for x,y in g]
for k,g in groupby(lki,key=itemgetter(1))]
print glo
.
EDIT
Another solution that needs no import , is more readable, keeps the orders, and is 22 % shorter than the preceding one:
oldlist = [["A",0], ["B",1], ["C",0], ["D",2], ["E",2]]
newlist, dicpos = [],{}
for val,k in oldlist:
if k in dicpos:
newlist[dicpos[k]].extend(val)
else:
newlist.append([val])
dicpos[k] = len(dicpos)
print newlist
Howard's answer is concise and elegant, but it's also O(n^2) in the worst case. For large lists with large numbers of grouping key values, you'll want to sort the list first and then use itertools.groupby:
>>> from itertools import groupby
>>> from operator import itemgetter
>>> seq = [["A",0], ["B",1], ["C",0], ["D",2], ["E",2]]
>>> seq.sort(key = itemgetter(1))
>>> groups = groupby(seq, itemgetter(1))
>>> [[item[0] for item in data] for (key, data) in groups]
[['A', 'C'], ['B'], ['D', 'E']]
Edit:
I changed this after seeing eyequem's answer: itemgetter(1) is nicer than lambda x: x[1].
>>> import collections
>>> D1 = collections.defaultdict(list)
>>> for element in L1:
... D1[element[1]].append(element[0])
...
>>> L2 = D1.values()
>>> print L2
[['A', 'C'], ['B'], ['D', 'E']]
>>>
I don't know about elegant, but it's certainly doable:
oldlist = [["A",0], ["B",1], ["C",0], ["D",2], ["E",2]]
# change into: list = [["A", "C"], ["B"], ["D", "E"]]
order=[]
dic=dict()
for value,key in oldlist:
try:
dic[key].append(value)
except KeyError:
order.append(key)
dic[key]=[value]
newlist=map(dic.get, order)
print newlist
This preserves the order of the first occurence of each key, as well as the order of items for each key. It requires the key to be hashable, but does not otherwise assign meaning to it.
len = max(key for (item, key) in list)
newlist = [[] for i in range(len+1)]
for item,key in list:
newlist[key].append(item)
You can do it in a single list comprehension, perhaps more elegant but O(n**2):
[[item for (item,key) in list if key==i] for i in range(max(key for (item,key) in list)+1)]
>>> xs = [["A",0], ["B",1], ["C",0], ["D",2], ["E",2]]
>>> xs.sort(key=lambda x: x[1])
>>> reduce(lambda l, x: (l.append([x]) if l[-1][0][1] != x[1] else l[-1].append(x)) or l, xs[1:], [[xs[0]]]) if xs else []
[[['A', 0], ['C', 0]], [['B', 1]], [['D', 2], ['E', 2]]]
Basically, if the list is sorted, it is possible to reduce by looking at the last group constructed by the previous steps - you can tell if you need to start a new group, or modify an existing group. The ... or l bit is a trick that enables us to use lambda in Python. (append returns None. It is always better to return something more useful than None, but, alas, such is Python.)
if using convtools library, which provides a lot of data processing primitives and generates ad hoc code under the hood, then:
from convtools import conversion as c
my_list = [["A", 0], ["B", 1], ["C", 0], ["D", 2], ["E", 2]]
# store the converter somewhere because this is where code generation
# takes place
converter = (
c.group_by(c.item(1))
.aggregate(c.ReduceFuncs.Array(c.item(0)))
.gen_converter()
)
assert converter(my_list) == [["A", "C"], ["B"], ["D", "E"]]
An answer inspired by #Howard's answer.
from operator import itemgetter
def group_by(nested_iterables: Iterable[Iterable], key_index: int) \
-> List[Tuple[Any, Iterable[Any]]]:
""" Groups elements nested in <nested_iterables> based on their <key_index>_th element.
Behaves similarly to itertools.groupby when the input to the itertools function is sorted.
E.g. If <nested_iterables> = [(1, 2), (2, 3), (5, 2), (9, 3)] and
<key_index> = 1, we will return [(2, [(1, 2), (5, 2)]), (3, [(2, 3), (9,3)])].
Returns:
A list of (group_key, values) tuples where <values> is an iterator of the iterables in
<nested_iterables> that all have their <key_index>_th element equal to <group_key>.
"""
group_keys = set(map(itemgetter(key_index), nested_iterables))
return [(key, list(filter(lambda x: x[key_index] == key, nested_iterables)))
for key in group_keys]