Related
This question already has answers here:
How do I remove duplicates from a list, while preserving order?
(30 answers)
Closed 4 years ago.
I want to remove those tuples which had same values at index 0 except the first occurance. I looked at other similar questions but did not get a particular answer I am looking for. Can somebody please help me?
Below is what I tried.
from itertools import groupby
import random
Newlist = []
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
Newlist = [random.choice(tuple(g)) for _, g in groupby(abc, key=lambda x: x[0])]
print Newlist
my expected output : [(1,2,3), (2,3,4), (0,2,0), (5,4,3)]
A simple way is to loop over the list and keep track of which elements you've already found:
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
found = set()
NewList = []
for a in abc:
if a[0] not in found:
NewList.append(a)
found.add(a[0])
print(NewList)
#[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
found is a set. At each iteration we check if the first element in the tuple is already in found. If not, we append the whole tuple to NewList. At the end of each iteration we add the first element of the tuple to found.
A better alternative using OrderedDict:
from collections import OrderedDict
abc = [(1,2,3), (2,3,4), (1,0,3), (0,2,0), (2,4,5),(5,4,3), (0,4,1)]
d = OrderedDict()
for t in abc:
d.setdefault(t[0], t)
abc_unique = list(d.values())
print(abc_unique)
Output:
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
Simple although not very efficient:
abc = [(1,2,3), (2,3,4), (1,0,3), (0,2,0), (2,4,5),(5,4,3), (0,4,1)]
abc_unique = [t for i, t in enumerate(abc) if not any(t[0] == p[0] for p in abc[:i])]
print(abc_unique)
Output:
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
The itertools recipes (Python 2: itertools recipes, but basically no difference in this case) contains a recipe for this, which is a bit more general than the implementation by #pault. It also uses a set:
Python 2:
from itertools import ifilterfalse as filterfalse
Python 3:
from itertools import filterfalse
def unique_everseen(iterable, key=None):
"List unique elements, preserving order. Remember all elements ever seen."
# unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D
seen = set()
seen_add = seen.add
if key is None:
for element in filterfalse(seen.__contains__, iterable):
seen_add(element)
yield element
else:
for element in iterable:
k = key(element)
if k not in seen:
seen_add(k)
yield element
Use it with:
abc = [(1,2,3), (2,3,4), (1,0,3),(0,2,0), (2,4,5),(5,4,3), (0,4,1)]
Newlist = list(unique_everseen(abc, key=lambda x: x[0]))
print Newlist
# [(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
This should be slightly faster because of the caching of the set.add method (only really relevant if your abc is large) and should also be more general because it makes the key function a parameter.
Apart from that, the same limitation I already mentioned in a comment applies: this only works if the first element of the tuple is actually hashable (which numbers, like in the given example, are, of course).
#PatrickHaugh claims:
but the question is explicitly about maintaining the order of the
tuples. I don't think there's a solution using groupby
I never miss an opportunity to (ab)use groupby(). Here's my solution sans sorting (once or twice):
from itertools import groupby, chain
abc = [(1, 2, 3), (2, 3, 4), (1, 0, 3), (0, 2, 0), (2, 4, 5), (5, 4, 3), (0, 4, 1)]
Newlist = list((lambda s: chain.from_iterable(g for f, g in groupby(abc, lambda k: s.get(k[0]) != s.setdefault(k[0], True)) if f))({}))
print(Newlist)
OUTPUT
% python3 test.py
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
%
To use groupby correctly, the sequence must be sorted:
>>> [next(g) for k,g in groupby(sorted(abc, key=lambda x:x[0]), key=lambda x:x[0])]
[(0, 2, 0), (1, 2, 3), (2, 3, 4), (5, 4, 3)]
or if you need that very exact order of your example (i.e. maintaining original order):
>>> [t[2:] for t in sorted([next(g) for k,g in groupby(sorted([(t[0], i)+t for i,t in enumerate(abc)]), lambda x:x[0])], key=lambda x:x[1])]
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
the trick here is to add one field for keeping the original order to restore after the groupby() step.
Edit: even a bit shorter:
>>> [t[1:] for t in sorted([next(g)[1:] for k,g in groupby(sorted([(t[0], i)+t for i,t in enumerate(abc)]), lambda x:x[0])])]
[(1, 2, 3), (2, 3, 4), (0, 2, 0), (5, 4, 3)]
I am looking to take as input a list and then create another list which contains tuples (or sub-lists) of adjacent elements from the original list, wrapping around for the beginning and ending elements. The input/output would look like this:
l_in = [0, 1, 2, 3]
l_out = [(3, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, 0)]
My question is closely related to another titled getting successive adjacent elements of a list, but this other question does not take into account wrapping around for the end elements and only handles pairs of elements rather than triplets.
I have a somewhat longer approach to do this involving rotating deques and zipping them together:
from collections import deque
l_in = [0, 1, 2, 3]
deq = deque(l_in)
deq.rotate(1)
deq_prev = deque(deq)
deq.rotate(-2)
deq_next = deque(deq)
deq.rotate(1)
l_out = list(zip(deq_prev, deq, deq_next))
# l_out is [(3, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, 0)]
However, I feel like there is probably a more elegant (and/or efficient) way to do this using other built-in Python functionality. If, for instance, the rotate() function of deque returned the rotated list instead of modifying it in place, this could be a one- or two-liner (though this approach of zipping together rotated lists is perhaps not the most efficient). How can I accomplish this more elegantly and/or efficiently?
One approach may be to use itertools combined with more_itertools.windowed:
import itertools as it
import more_itertools as mit
l_in = [0, 1, 2, 3]
n = len(l_in)
list(it.islice(mit.windowed(it.cycle(l_in), 3), n-1, 2*n-1))
# [(3, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, 0)]
Here we generated an infinite cycle of sliding windows and sliced the desired subset.
FWIW, here is an abstraction of the latter code for a general, flexible solution given any iterable input e.g. range(5), "abcde", iter([0, 1, 2, 3]), etc.:
def get_windows(iterable, size=3, offset=-1):
"""Return an iterable of windows including an optional offset."""
it1, it2 = it.tee(iterable)
n = mit.ilen(it1)
return it.islice(mit.windowed(it.cycle(it2), size), n+offset, 2*n+offset)
list(get_windows(l_in))
# [(3, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, 0)]
list(get_windows("abc", size=2))
# [('c', 'a'), ('a', 'b'), ('b', 'c')]
list(get_windows(range(5), size=2, offset=-2))
# [(3, 4), (4, 0), (0, 1), (1, 2), (2, 3)]
Note: more-itertools is a separate library, easily installed via:
> pip install more_itertools
This can be done with slices:
l_in = [0, 1, 2, 3]
l_in = [l_in[-1]] + l_in + [l_in[0]]
l_out = [l_in[i:i+3] for i in range(len(l_in)-2)]
Well, or such a perversion:
div = len(l_in)
n = 3
l_out = [l_in[i % div: i % div + 3]
if len(l_in[i % div: i % div + 3]) == 3
else l_in[i % div: i % div + 3] + l_in[:3 - len(l_in[i % div: i % div + 3])]
for i in range(3, len(l_in) + 3 * n + 2)]
You can specify the number of iterations.
Well I figured out a better solution as I was writing the question, but I already went through the work of writing it, so here goes. This solution is at least much more concise:
l_out = list(zip(l_in[-1:] + l_in[:-1], l_in, l_in[1:] + l_in[:1]))
See this post for different answers on how to rotate lists in Python.
The one-line solution above should be at least as efficient as the solution in the question (based on my understanding) since the slicing should not be more expensive than the rotating and copying of the deques (see https://wiki.python.org/moin/TimeComplexity).
Other answers with more efficient (or elegant) solutions are still welcome though.
as you found there is a list rotation slicing based idiom lst[i:] + lst[:i]
using it inside a comprehension taking a variable n for the number of adjacent elements wanted is more general [lst[i:] + lst[:i] for i in range(n)]
so everything can be parameterized, the number of adjacent elements n in the cyclic rotation and the 'phase' p, the starting point if not the 'natural' 0 base index, although the default p=-1 is set to -1 to fit the apparant desired output
tst = list(range(4))
def rot(lst, n, p=-1):
return list(zip(*([lst[i+p:] + lst[:i+p] for i in range(n)])))
rot(tst, 3)
Out[2]: [(3, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, 0)]
showing the shortend code as per the comment
This question already has answers here:
Iterating over every two elements in a list [duplicate]
(22 answers)
Closed 7 years ago.
I'm trying to turn a list into pairs, but only for as long as possible (i.e. my list can be odd, in that case I want to ignore the last element).
E.g. my input is x = [0, 1, 2, 3, 4], which I would want to turn into [(0, 1), (2, 3)]. Similarly, x = [0, 1, 2, 3, 4, 5] should become [(0, 1), (2, 3), (4, 5)].
What I'm currently doing is [(x[i], x[i+1]) for i in range(0, len(x), 2)]. This breaks, as range(0, len(x), 2) still includes x[-1] if len(x) is odd. Note that something of the form [(l, r) for l, r in ...] would also be preferable, rather than having to fiddle with indices.
Bonus points: Here's some more context. I'm not completely ignoring the last element of an odd sequence, of course. I'm applying a function to each pair, but I do not want to apply this function H to the singleton element. Currently, I'm doing the following:
next_layer = [H(layer[i], layer[i+1]) for i in range(0, len(layer), 2)]
if len(layer) & 1: # if there is a lone node left on this layer
next_layer.append(layer[-1])
An extra elegant solution would incorporate this into the above as well.
Use a zip
This function returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The returned list is truncated in length to the length of the shortest argument sequence.
>>> a = [1, 2, 3, 4, 5]
>>> b = [0, 1, 2, 3, 4, 5]
>>> zip(a[::2], a[1::2])
[(1, 2), (3, 4)]
>>> zip(b[::2], b[1::2])
[(0, 1), (2, 3), (4, 5)]
I want to take a list, for instance List = [1,2,2], and generate its permutations. I can do this with:
NewList = [list(itertools.permutations(List))]
and the output is:
[[(1, 2, 2), (1, 2, 2), (2, 1, 2), (2, 2, 1), (2, 1, 2), (2, 2, 1)]]
The Problem: itertools.permutations returns a list of length 1 whose only entry is the list of all permutations of List. That is:
NewList[0] == [(1,2,2),(1,2,2),(2,1,2),(2,2,1),(2,1,2),(2,2,1)]
and
NewList[1] does not exist.
I want the output to be a list where each entry is one of the permutations. That is
NewList[0] == (1,2,2)
NewList[1] == (1,2,2)
NewList[2] == (2,1,2)
...
NewList[5] == (2,2,1)
The Question: Is there a command that will produce the permutations of List in this way? Failing that, is there a way to access the 'individual elements' of [list(itertools.permutations(List))] so I can do things with them?
>>> from itertools import permutations
>>> list(permutations([1,2,2]))
[(1, 2, 2), (1, 2, 2), (2, 1, 2), (2, 2, 1), (2, 1, 2), (2, 2, 1)]
You don't need to put it in a list again. i.e Don't do [list(permutations(...))] (By doing [] you are making a nested list and hence are unable to access the elements using testList[index], though you could do testList[0][index] but it would be better to just keep it as a list of tuples.)
>>> newList = list(permutations([1, 2, 2]))
>>> newList[0]
(1, 2, 2)
>>> newList[3]
(2, 2, 1)
Now you can access the elements by using their indices.
Why can't you do this:
NewList = [list(itertools.permutations(List))][0]
or even just this:
NewList = list(itertools.permutations(List))
By doing [ list(itertools.permutations(List)) ], you put the list of permutations (the one that you want) inside of another list. So the fix would be to remove the outer []'s
I have a list of lists of tuples
A= [ [(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)] ]
The outer list can have any number of inner lists, the inner lists can have any number of tuples, a tuple always has 3 integers.
I want to generate all combination of tuples, one from each list:
[(1,2,3),(7,8,9),(2,1,0)]
[(1,2,3),(7,8,9),(1,3,5)]
[(1,2,3),(8,7,6),(2,1,0)]
...
[(4,5,6),(5,4,3),(1,3,5)]
A simple way to do it is to use a function similar to itertools.poduct()
but it must be called like this
itertools.product([(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)])
i.e the outer list is removed. And I don't know how to do that. Is there a better way to generate all combinations of tuples?
itertools.product(*A)
For more details check the python tutorial
This works for your example, if there is only one level of nested lists (no lists of lists of lists):
itertools.product(*A)
you can probably call itertools.product like so:
itertools.product(*A) # where A is your list of lists of tuples
This way it expands your list's elements into arguments for the function you are calling.
Late to the party but ...
I'm new to python and come from a lisp background. This is what I came up with (check out the var names for lulz):
def flatten(lst):
if lst:
car,*cdr=lst
if isinstance(car,(list)):
if cdr: return flatten(car) + flatten(cdr)
return flatten(car)
if cdr: return [car] + flatten(cdr)
return [car]
Seems to work. Test:
A = [ [(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)] ]
flatten(A)
Result:
[(1, 2, 3), (4, 5, 6), (7, 8, 9), (8, 7, 6), (5, 4, 3), (2, 1, 0), (1, 3, 5)]
Note: the line car,*cdr=lst only works in Python 3.0
This is not exactly one step, but this would do what you want if for some reason you don't want to use the itertools solution:
def crossprod(listoflists):
if len(listoflists) == 1:
return listoflists
else:
result = []
remaining_product = prod(listoflists[1:])
for outertupe in listoflists[0]:
for innercombo in remaining_product[0]:
newcombo = [outertupe]
newcombo.append(innercombo)
result.append(newcombo)
return result
def flatten(A)
answer = []
for i in A:
if type(i) == list:
ans.extend(i)
else:
ans.append(i)
return ans
This may also be achieved using list comprehension.
In [62]: A = [ [(1,2,3),(4,5,6)], [(7,8,9),(8,7,6),(5,4,3)],[(2,1,0),(1,3,5)] ]
In [63]: improved_list = [num for elem in A for num in elem]
In [64]: improved_list
Out[64]: [(1, 2, 3), (4, 5, 6), (7, 8, 9), (8, 7, 6), (5, 4, 3), (2, 1, 0), (1, 3, 5)]