I'm trying to combine to different nested list into a list of tuples (x,y)
where x comes from the first nested list and y from the second nested list.
nested_list1 = [[1, 2, 3],[3],[0, 3],[1]]
nested_list2 = [[.0833, .0833, .0833], [.2], [.175, .175], [.2]]
when you combine them it should be:
result = [(1,.0833), (2,.0833), (3,.0833), (3,.2), (0,.175), (3,.175), (1,.2)]
my approach is that i need to iterate through the list of lists and join them 1 at a time.
I know to iterate through 1 nested list like so:
for list in nested_list1:
for number in list:
print(number)
but I can't iterate through 2 nested list at the same time.
for list, list in zip(nested_list1, nested_list2):
for number, prob in zip(list,list):
print(tuple(number, prob)) #will not work
any ideas?
You could do a double zip through lists:
lst1 = [[1, 2, 3],[3],[0, 3],[1]]
lst2 = [[.0833, .0833, .0833], [.2], [.175, .175], [.2]]
print([(u, v) for x, y in zip(lst1, lst2) for u, v in zip(x, y)])
Or use itertools.chain.from_iterable to flatten list and zip:
from itertools import chain
lst1 = [[1, 2, 3],[3],[0, 3],[1]]
lst2 = [[.0833, .0833, .0833], [.2], [.175, .175], [.2]]
print(list(zip(chain.from_iterable(lst1), chain.from_iterable(lst2))))
Use itertools.chain:
>>> nested_list1 = [[1, 2, 3],[3],[0, 3],[1]]
>>> nested_list2 = [[.0833, .0833, .0833], [.2], [.175, .175], [.2]]
>>> import itertools
>>> res = list(zip(itertools.chain.from_iterable(nested_list1), itertools.chain.from_iterable(nested_list2)))
>>> res
[(1, 0.0833), (2, 0.0833), (3, 0.0833), (3, 0.2), (0, 0.175), (3, 0.175), (1, 0.2)]
Flatten your lists and then pass to zip():
list1 = [item for sublist in nested_list1 for item in sublist]
list2 = [item for sublist in nested_list2 for item in sublist]
final = list(zip(list1, list2))
Yields:
[(1, 0.0833), (2, 0.0833), (3, 0.0833), (3, 0.2), (0, 0.175), (3, 0.175), (1, 0.2)]
There are 2 errors in your code:
You shadow built-in list twice and in a way that you can't differentiate between two variables. Don't do this.
You use tuple(x, y) to create a tuple form 2 variables. This is incorrect, as tuple takes one argument only. To construct a tuple of two variables just use syntax (x, y).
So this will work:
for L1, L2 in zip(nested_list1, nested_list2):
for number, prob in zip(L1, L2):
print((number, prob))
More idiomatic would be to flatten your nested lists; for example, via itertools.chain:
from itertools import chain
res = list(zip(chain.from_iterable(nested_list1),
chain.from_iterable(nested_list2)))
[(1, 0.0833), (2, 0.0833), (3, 0.0833), (3, 0.2), (0, 0.175), (3, 0.175), (1, 0.2)]
This one liner will achieve what you want.
reduce(lambda x, y: x+y, [[(i, j) for i, j in zip(x,y)] for x, y in zip(nested_list1, nested_list2)])
One way is to convert both the nested lists into full lists and then use zip. Sample code below:
>>> nested_list1 = [[1, 2, 3],[3],[0, 3],[1]]
>>> nested_list2 = [[.0833, .0833, .0833], [.2], [.175, .175], [.2]]
>>> new_list1 = [x for val in nested_list1 for x in val]
>>> new_list2 = [x for val in nested_list2 for x in val]
>>> print new_list1
[1, 2, 3, 3, 0, 3, 1]
>>> print new_list2
[0.0833, 0.0833, 0.0833, 0.2, 0.175, 0.175, 0.2]
>>> new_val = zip(new_list1, new_list2)
>>> print new_val
[(1, 0.0833), (2, 0.0833), (3, 0.0833), (3, 0.2), (0, 0.175), (3, 0.175), (1, 0.2)]
result = []
[result.extend(list(zip(x, y))) for x in nested_list1 for y in nested_list2]
print(result)
Use two times zip and flatten the list
from functools import reduce
reduce(lambda x,y: x+y,[(zip(i,j)) for i,j in zip(nested_list1,nested_list2)])
You can flatten using chain as well
from itertools import chain
list(chain(*[(zip(i,j)) for i,j in zip(nested_list1,nested_list2)]))
output
[(1, 0.0833), (2, 0.0833), (3, 0.0833), (3, 0.2), (0, 0.175), (3, 0.175), (1, 0.2)]
Related
I am trying to slice a list in python so that I get two separate lists: one with the keys, and one with the values. My list looks like this:
[(b'a', 2), (b'surprise,', 1), (b'but', 1), (b'welcome', 1), (b'one', 1), (b'for', 1), (b'sure.', 1)]
I am relatively new at coding and so I am just wondering how I would be able to do something like that.
Use zip unpacking the data:
>>> l = [(b'a', 2), (b'surprise,', 1), (b'but', 1), (b'welcome', 1), (b'one', 1), (b'for', 1), (b'sure.', 1)]
>>> [*zip(*l)] # similar to list(zip(*l))
[(b'a', b'surprise,', b'but', b'welcome', b'one', b'for', b'sure.'), (2, 1, 1, 1, 1, 1, 1)]
If you want lists instead of tuples, use a comprehension:
>>> [list(x) for x in zip(*l)]
[[b'a', b'surprise,', b'but', b'welcome', b'one', b'for', b'sure.'], [2, 1, 1, 1, 1, 1, 1]]
For fixing if l is an empty list you can use combiantion with an or expresion:
>>> keys, values = [*zip(*[])] or ([], [])
>>> keys
[]
>>> values
[]
Netwave has a very correct answer, but here is a straightforward way to do it:
Given that:
data = [(b'a', 2), (b'surprise,', 1), (b'but', 1), (b'welcome', 1), (b'one', 1), (b'for', 1), (b'sure.', 1)]
You can use a loop to unpack it:
keylist = []
vallist = []
for item in data:
keylist.append(item[0])
vallist.append(item[1])
# You end up with keys in keylist and values in vallist
You can also use "tuple unpacking" to put each tuple into two vars:
keylist = []
vallist = []
for k,v in data:
keylist.append(k)
vallist.append(v)
# You end up with keys in keylist and values in vallist
You can streamline with list comprehensions:
keylist = [item[0] for item in data]
vallist = [item[1] for item in data]
Just to be obnoxious, you can use tuple unpacking as well:
kl,vl = [item[0] for item in data], [item[1] for item in data]
Since you specifically use the terms "keys" and "values", you can do it via a dictionary:
my_dict = dict(my_list)
my_keys = list(my_dict.keys())
my_values = list(my_dict.values())
If all you care about is some kind of (read only) iterable, don't bother wrapping in list.
Use map:
l = [(b'a', 2), (b'surprise,', 1), (b'but', 1), (b'welcome', 1), (b'one', 1), (b'for', 1), (b'sure.', 1)]
l1, l2 = map(lambda x: x[0], l), map(lambda x: x[1], l)
print('l1 is: {}\nl2 is: {}'.format(list(l1), list(l2)))
This question already has answers here:
How do I remove duplicates from a list, while preserving order?
(30 answers)
Closed 4 years ago.
I want to remove those tuples which had same values at index 0 except the first occurance. I looked at other similar questions but did not get a particular answer I am looking for. Can somebody please help me?
Below is what I tried.
from itertools import groupby
import random
Newlist = []
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
Newlist = [random.choice(tuple(g)) for _, g in groupby(abc, key=lambda x: x[0])]
print Newlist
my expected output : [(1,2,3), (2,3,4), (0,2,0), (5,4,3)]
A simple way is to loop over the list and keep track of which elements you've already found:
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
found = set()
NewList = []
for a in abc:
if a[0] not in found:
NewList.append(a)
found.add(a[0])
print(NewList)
#[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
found is a set. At each iteration we check if the first element in the tuple is already in found. If not, we append the whole tuple to NewList. At the end of each iteration we add the first element of the tuple to found.
A better alternative using OrderedDict:
from collections import OrderedDict
abc = [(1,2,3), (2,3,4), (1,0,3), (0,2,0), (2,4,5),(5,4,3), (0,4,1)]
d = OrderedDict()
for t in abc:
d.setdefault(t[0], t)
abc_unique = list(d.values())
print(abc_unique)
Output:
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
Simple although not very efficient:
abc = [(1,2,3), (2,3,4), (1,0,3), (0,2,0), (2,4,5),(5,4,3), (0,4,1)]
abc_unique = [t for i, t in enumerate(abc) if not any(t[0] == p[0] for p in abc[:i])]
print(abc_unique)
Output:
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
The itertools recipes (Python 2: itertools recipes, but basically no difference in this case) contains a recipe for this, which is a bit more general than the implementation by #pault. It also uses a set:
Python 2:
from itertools import ifilterfalse as filterfalse
Python 3:
from itertools import filterfalse
def unique_everseen(iterable, key=None):
"List unique elements, preserving order. Remember all elements ever seen."
# unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D
seen = set()
seen_add = seen.add
if key is None:
for element in filterfalse(seen.__contains__, iterable):
seen_add(element)
yield element
else:
for element in iterable:
k = key(element)
if k not in seen:
seen_add(k)
yield element
Use it with:
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
Newlist = list(unique_everseen(abc, key=lambda x: x[0]))
print Newlist
# [(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
This should be slightly faster because of the caching of the set.add method (only really relevant if your abc is large) and should also be more general because it makes the key function a parameter.
Apart from that, the same limitation I already mentioned in a comment applies: this only works if the first element of the tuple is actually hashable (which numbers, like in the given example, are, of course).
#PatrickHaugh claims:
but the question is explicitly about maintaining the order of the
tuples. I don't think there's a solution using groupby
I never miss an opportunity to (ab)use groupby(). Here's my solution sans sorting (once or twice):
from itertools import groupby, chain
abc = [(1, 2, 3), (2, 3, 4), (1, 0, 3), (0, 2, 0), (2, 4, 5), (5, 4, 3), (0, 4, 1)]
Newlist = list((lambda s: chain.from_iterable(g for f, g in groupby(abc, lambda k: s.get(k[0]) != s.setdefault(k[0], True)) if f))({}))
print(Newlist)
OUTPUT
% python3 test.py
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
%
To use groupby correctly, the sequence must be sorted:
>>> [next(g) for k,g in groupby(sorted(abc, key=lambda x:x[0]), key=lambda x:x[0])]
[(0, 2, 0), (1, 2, 3), (2, 3, 4), (5, 4, 3)]
or if you need that very exact order of your example (i.e. maintaining original order):
>>> [t[2:] for t in sorted([next(g) for k,g in groupby(sorted([(t[0], i)+t for i,t in enumerate(abc)]), lambda x:x[0])], key=lambda x:x[1])]
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
the trick here is to add one field for keeping the original order to restore after the groupby() step.
Edit: even a bit shorter:
>>> [t[1:] for t in sorted([next(g)[1:] for k,g in groupby(sorted([(t[0], i)+t for i,t in enumerate(abc)]), lambda x:x[0])])]
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
I have a list of tuples, for example:
(1,3)
(1,2)
(1,7)
(2,4)
(2,10)
(3,8)
I need to be able to sum up the second values based upon what the first value is, getting this result for the example list:
(1,12)
(2,14)
(3,8)
This question is very similar in nature to this one, however, my solution may not use any imports or for loops, and all answers to that question use one or the other. It's supposed to rely on list and set comprehension.
my_set = {x[0] for x in my_tuples}
my_sums = [(i,sum(x[1] for x in my_tuples if x[0] == i)) for i in my_set]
I guess. ... those requirements are not very good for this problem (this solution will be slow ...)
If you can use dictionaries the following should work
x = [(1,3), (1, 2), (1, 7), (2, 4), (2, 10), (3, 8)]
d = {}
[d.__setitem__(first, d.get(first, 0) + second) for first, second in x]
print(list(d.items()))
If you are using python2, you can use map to behave like izip_longest and get the index of where the groups end:
def sums(l):
st = set()
inds = [st.add(a) or ind for ind, (a, b) in enumerate(l) if a not in st]
return [(l[i][0], sum(sub[1] for sub in l[i:j])) for i, j in map(None, inds, inds[1:])]
Output:
In [10]: print(sums(l))
[(1, 12), (2, 14), (3, 8)]
for python 2 or 3 you can just use enumerate and check the index:
def sums(l):
st = set()
inds = [st.add(a) or ind for ind, (a, b) in enumerate(l) if a not in st]
return [(l[j][0], sum(sub[1] for sub in (l[j:inds[i]] if i < len(inds) else l[inds[-1]:])))
for i, j in enumerate(inds, 1)]
same output:
In [12]: print(sums(l))
[(1, 12), (2, 14), (3, 8)]
This is pretty straightforward, but it's definitely O(n**2), so keep your input data small:
data = (
(1,3),
(1,2),
(1,7),
(2,4),
(2,10),
(3,8),
)
d = { k:v for k,v in data }
d2 = [(t1,sum( v for k,v in data if k == t1 )) for t1 in d.keys() ]
print(d2)
Output is
[(1, 12), (2, 14), (3, 8)]
I'd use a defaultdict
from collections import defaultdict
x = [(1,3), (1, 2), (1, 7), (2, 4), (2, 10), (3, 8)]
d = defaultdict(int)
for k, v in x:
d[k] += v
print(list(d.items()))
if you need a one-liner (lambda inline function) using itertools
from itertools import groupby
myfunc = lambda tu : [(k, sum(v2[1] for v2 in v)) for k, v in groupby(tu, lambda x: x[0])])
print(myfunc(x))
I want to compare to lists and return the different indices and elements.
So I write the following code:
l1 = [1,1,1,1,1]
l2 = [1,2,1,1,3]
ind = []
diff = []
for i in range(len(l1)):
if l1[i] != l2[i]:
ind.append(i)
diff.append([l1[i], l2[i]])
print ind
print diff
# output:
# [1, 4]
# [[1, 2], [1, 3]]
The code works, but are there any better ways to do that?
Update the Question:
I want to ask for another solutions, for example with the iterator, or ternary expression like [a,b](expression) (Not the easiest way like what I did. I want to exclude it.) Thanks very much for the patient! :)
You could use a list comprehension to output all the information in a single list.
>>> [[idx, (i,j)] for idx, (i,j) in enumerate(zip(l1, l2)) if i != j]
[[1, (1, 2)], [4, (1, 3)]]
This will produce a list where each element is: [index, (first value, second value)] so all the information regarding a single difference is together.
An alternative way is the following
>>> l1 = [1,1,1,1,1]
>>> l2 = [1,2,1,1,3]
>>> z = zip(l1,l2)
>>> ind = [i for i, x in enumerate(z) if x[0] != x[1]]
>>> ind
[1, 4]
>>> diff = [z[i] for i in ind]
>>> diff
[(1, 2), (1, 3)]
In Python3 you have to add a call to list around zip.
You can try functional style:
res = filter(lambda (idx, x): x[0] != x[1], enumerate(zip(l1, l2)))
# [(1, (1, 2)), (4, (1, 3))]
to unzip res you can use:
zip(*res)
# [(1, 4), ((1, 2), (1, 3))]
I have a 2D list of lists, I am doing some stuff to it and getting, as a result a slightly modified 2d lists of lists. I cannot track what changes are being made until after I get the new list back. I want to get a list of all the items that have been changed such that
[[1,2,3], [4,5,6], [7,8,9]] becomes [[1,None,3], [4,None,6], [7,None, None]] and I would get a list [(0,1), (1,1), (2, 1), (2,2)] I know you can normally do list(set(a)-set(b)) but when I tried it I got TypeError: unhashable type: 'list' So what is the most efficient way of doing this?
Using zip, enumerate and a generator function:
def diff(lis1, lis2):
for i, (x, y) in enumerate(zip(lis1, lis2)):
for j, (x1, y1) in enumerate(zip(x, y)):
if x1 != y1:
yield i, j
...
>>> lis1 = [[1,2,3], [4,5,6], [7,8,9]]
>>> lis2 = [[1,None,3], [4,None,6], [7,None, None]]
>>> list(diff(lis1, lis2))
[(0, 1), (1, 1), (2, 1), (2, 2)]
Using list comprehension:
>>> a = [[1,2,3], [4,5,6], [7,8,9]]
>>> b = [[1,None,3], [4,None,6], [7,None, None]]
>>> [(i,j) for i, row in enumerate(a) for j, x in enumerate(row) if b[i][j] != x]
[(0, 1), (1, 1), (2, 1), (2, 2)]
If the lists have a regular structure, that is each of the sub-lists has the same length, and you don't mind using external packages, numpy can help.
import numpy as np
a = np.array([[1,2,3], [4,5,6], [7,8,9]])
b = np.array([[1,None,3], [4,None,6], [7,None, None]])
print(np.where(a!=b))
>>>(array([0, 1, 2, 2]), array([1, 1, 1, 2]))