Related
This question already has answers here:
How do I remove duplicates from a list, while preserving order?
(30 answers)
Closed 4 years ago.
I want to remove those tuples which had same values at index 0 except the first occurance. I looked at other similar questions but did not get a particular answer I am looking for. Can somebody please help me?
Below is what I tried.
from itertools import groupby
import random
Newlist = []
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
Newlist = [random.choice(tuple(g)) for _, g in groupby(abc, key=lambda x: x[0])]
print Newlist
my expected output : [(1,2,3), (2,3,4), (0,2,0), (5,4,3)]
A simple way is to loop over the list and keep track of which elements you've already found:
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
found = set()
NewList = []
for a in abc:
if a[0] not in found:
NewList.append(a)
found.add(a[0])
print(NewList)
#[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
found is a set. At each iteration we check if the first element in the tuple is already in found. If not, we append the whole tuple to NewList. At the end of each iteration we add the first element of the tuple to found.
A better alternative using OrderedDict:
from collections import OrderedDict
abc = [(1,2,3), (2,3,4), (1,0,3), (0,2,0), (2,4,5),(5,4,3), (0,4,1)]
d = OrderedDict()
for t in abc:
d.setdefault(t[0], t)
abc_unique = list(d.values())
print(abc_unique)
Output:
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
Simple although not very efficient:
abc = [(1,2,3), (2,3,4), (1,0,3), (0,2,0), (2,4,5),(5,4,3), (0,4,1)]
abc_unique = [t for i, t in enumerate(abc) if not any(t[0] == p[0] for p in abc[:i])]
print(abc_unique)
Output:
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
The itertools recipes (Python 2: itertools recipes, but basically no difference in this case) contains a recipe for this, which is a bit more general than the implementation by #pault. It also uses a set:
Python 2:
from itertools import ifilterfalse as filterfalse
Python 3:
from itertools import filterfalse
def unique_everseen(iterable, key=None):
"List unique elements, preserving order. Remember all elements ever seen."
# unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D
seen = set()
seen_add = seen.add
if key is None:
for element in filterfalse(seen.__contains__, iterable):
seen_add(element)
yield element
else:
for element in iterable:
k = key(element)
if k not in seen:
seen_add(k)
yield element
Use it with:
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
Newlist = list(unique_everseen(abc, key=lambda x: x[0]))
print Newlist
# [(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
This should be slightly faster because of the caching of the set.add method (only really relevant if your abc is large) and should also be more general because it makes the key function a parameter.
Apart from that, the same limitation I already mentioned in a comment applies: this only works if the first element of the tuple is actually hashable (which numbers, like in the given example, are, of course).
#PatrickHaugh claims:
but the question is explicitly about maintaining the order of the
tuples. I don't think there's a solution using groupby
I never miss an opportunity to (ab)use groupby(). Here's my solution sans sorting (once or twice):
from itertools import groupby, chain
abc = [(1, 2, 3), (2, 3, 4), (1, 0, 3), (0, 2, 0), (2, 4, 5), (5, 4, 3), (0, 4, 1)]
Newlist = list((lambda s: chain.from_iterable(g for f, g in groupby(abc, lambda k: s.get(k[0]) != s.setdefault(k[0], True)) if f))({}))
print(Newlist)
OUTPUT
% python3 test.py
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
%
To use groupby correctly, the sequence must be sorted:
>>> [next(g) for k,g in groupby(sorted(abc, key=lambda x:x[0]), key=lambda x:x[0])]
[(0, 2, 0), (1, 2, 3), (2, 3, 4), (5, 4, 3)]
or if you need that very exact order of your example (i.e. maintaining original order):
>>> [t[2:] for t in sorted([next(g) for k,g in groupby(sorted([(t[0], i)+t for i,t in enumerate(abc)]), lambda x:x[0])], key=lambda x:x[1])]
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
the trick here is to add one field for keeping the original order to restore after the groupby() step.
Edit: even a bit shorter:
>>> [t[1:] for t in sorted([next(g)[1:] for k,g in groupby(sorted([(t[0], i)+t for i,t in enumerate(abc)]), lambda x:x[0])])]
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
Let's say I have a list like so: [(1, (1,2)), (2, (23, -10)), (3, (4, 5))...]
I want to get (1,2), (23, -10), etc
edit: Thanks for help. I didn't know about list comprehension as I'm not too familiar with python
Try Something like this:-
Here is List and get other list of tuples:-
a = [(1, (1,2)), (2, (23, -10)), (3, (4, 5))...]
b = map(lambda item: item[1], a)
print b
This will solve your problem.
Here is list of tuples where second elem of each tuple is also a tuple. To get that second elem we are going to use lambda that is going to take elements from list and just return second item from that element, in this case being desired tuple. The map function also creates a list of returned values.
>>> list_of_nested_tuples = [(1, (1, 2)), (2, (23, -10)), (3, (4, 5))]
>>> b = map(lambda item: item[1], list_of_nested_tuples)
>>> b
[(1, 2), (23, -10), (4, 5)]
Take note that it would be more clear to just use list comprehension like so
>>> [elem[1] for elem in list_of_nested_tuples]
[(1, 2), (23, -10), (4, 5)]
Yes, you can iterate over all tuples and then take the second element:
list = [(1, (1,2)), (2, (23, -10)), (3, (4, 5))]
for elem in list:
print(elem[1])
In each iteration elem value its (1,(1,2)) -> (2,(23,-10)) -> ....
Then you take the second item of the tuple (index 1)
Let's say I have the following two lists of tuples
myList = [(1, 7), (3, 3), (5, 9)]
otherList = [(2, 4), (3, 5), (5, 2), (7, 8)]
returns => [(1, 7), (2, 4), (3, 8), (5, 11), (7, 8)]
I would like to design a merge operation that merges these two lists by checking for any intersections on the first element of the tuple, if there are intersections, add the second elements of each tuple in question (merge the two). After the operation I would like to sort based upon the first element.
I am also posting this because I think its a pretty common problem that has an obvious solution, but I feel that there could be very pythonic solutions to this question ;)
Use a dictionary for the result:
result = {}
for k, v in my_list + other_list:
result[k] = result.get(k, 0) + v
If you want a list of tuples, you can get it via result.items(). The resulting list will be in arbitrary order, but of course you can sort it if desired.
(Note that I renamed your lists to conform with Python's style conventions.)
Use defaultdict:
from collections import defaultdict
results_dict = defaultdict(int)
results_dict.update(my_list)
for a, b in other_list:
results_dict[a] += b
results = sorted(results_dict.items())
Note: When sorting sequences, sorted sorts by the first item in the sequence. If the first elements are the same, then it compares the second element. You can give sorted a function to sort by, using the key keyword argument:
results = sorted(results_dict.items(), key=lambda x: x[1]) #sort by the 2nd item
or
results = sorted(results_dict.items(), key=lambda x: abs(x[0])) #sort by absolute value
A method using itertools:
>>> myList = [(1, 7), (3, 3), (5, 9)]
>>> otherList = [(2, 4), (3, 5), (5, 2), (7, 8)]
>>> import itertools
>>> merged = []
>>> for k, g in itertools.groupby(sorted(myList + otherList), lambda e: e[0]):
... merged.append((k, sum(e[1] for e in g)))
...
>>> merged
[(1, 7), (2, 4), (3, 8), (5, 11), (7, 8)]
This first concatenates the two lists together and sorts it. itertools.groupby returns the elements of the merged list, grouped by the first element of the tuple, so it just sums them up and places it into the merged list.
>>> [(k, sum(v for x,v in myList + otherList if k == x)) for k in dict(myList + otherList).keys()]
[(1, 7), (2, 4), (3, 8), (5, 11), (7, 8)]
>>>
tested for both Python2.7 and 3.2
dict(myList + otherList).keys() returns an iterable containing a set of the keys for the joined lists
sum(...) takes 'k' to loop again through the joined list and add up tuple items 'v' where k == x
... but the extra looping adds processing overhead. Using an explicit dictionary as proposed by Sven Marnach avoids it.
I want to filter a list of tuples like [(1,22,1),(5,1,8),(8,3,4),(7,5,6)] using a list like [1,7] which would eventually give me the result [(1,22,1),(5,1,8),(7,5,6)]; since (8,3,4) does not have either 1 or 7, it is eliminated.
I can write a comprehensive function for this. But I am looking for a short list comprehension if possible.
Thanks.
>>> tup_list = [(1,22,1),(5,1,8),(8,3,4),(7,5,6)]
>>> filter_list = [1,7]
>>> [tup for tup in tup_list if any(i in tup for i in filter_list)]
[(1, 22, 1), (5, 1, 8), (7, 5, 6)]
try with this one :
items = [(1,22,1),(5,1,8),(8,3,4),(7,5,6)]
result = [ item for item in items if (set([1,7]) & set(item))]
>>> [(1, 22, 1), (5, 1, 8), (7, 5, 6)]
I have a list of lists of tuples
A= [ [(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)] ]
The outer list can have any number of inner lists, the inner lists can have any number of tuples, a tuple always has 3 integers.
I want to generate all combination of tuples, one from each list:
[(1,2,3),(7,8,9),(2,1,0)]
[(1,2,3),(7,8,9),(1,3,5)]
[(1,2,3),(8,7,6),(2,1,0)]
...
[(4,5,6),(5,4,3),(1,3,5)]
A simple way to do it is to use a function similar to itertools.poduct()
but it must be called like this
itertools.product([(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)])
i.e the outer list is removed. And I don't know how to do that. Is there a better way to generate all combinations of tuples?
itertools.product(*A)
For more details check the python tutorial
This works for your example, if there is only one level of nested lists (no lists of lists of lists):
itertools.product(*A)
you can probably call itertools.product like so:
itertools.product(*A) # where A is your list of lists of tuples
This way it expands your list's elements into arguments for the function you are calling.
Late to the party but ...
I'm new to python and come from a lisp background. This is what I came up with (check out the var names for lulz):
def flatten(lst):
if lst:
car,*cdr=lst
if isinstance(car,(list)):
if cdr: return flatten(car) + flatten(cdr)
return flatten(car)
if cdr: return [car] + flatten(cdr)
return [car]
Seems to work. Test:
A = [ [(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)] ]
flatten(A)
Result:
[(1, 2, 3), (4, 5, 6), (7, 8, 9), (8, 7, 6), (5, 4, 3), (2, 1, 0), (1, 3, 5)]
Note: the line car,*cdr=lst only works in Python 3.0
This is not exactly one step, but this would do what you want if for some reason you don't want to use the itertools solution:
def crossprod(listoflists):
if len(listoflists) == 1:
return listoflists
else:
result = []
remaining_product = prod(listoflists[1:])
for outertupe in listoflists[0]:
for innercombo in remaining_product[0]:
newcombo = [outertupe]
newcombo.append(innercombo)
result.append(newcombo)
return result
def flatten(A)
answer = []
for i in A:
if type(i) == list:
ans.extend(i)
else:
ans.append(i)
return ans
This may also be achieved using list comprehension.
In [62]: A = [ [(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)] ]
In [63]: improved_list = [num for elem in A for num in elem]
In [64]: improved_list
Out[64]: [(1, 2, 3), (4, 5, 6), (7, 8, 9), (8, 7, 6), (5, 4, 3), (2, 1, 0), (1, 3, 5)]