Related
For example,
a = [1, 1, 2, 4, 4, 4, 5, 6, 7, 100]
b = [1, 2, 2, 2, 2, 4, 5, 7, 8, 100]
I can find the matching elements using:
np.intersect1d(a,b)
Output:
array([ 1, 2, 4, 5, 7, 100])
Then, how can I get the indices of matched elements in arrays a and b, respectively ?
There is a function in IDL as "match" - https://www.l3harrisgeospatial.com/docs/match.html
Is there a similar function in Python?
Use return_indices in numpy.intersect1d:
intersect, ind_a, ind_b = np.intersect1d(a,b, return_indices=True)
Output:
intersect
# array([ 1, 2, 4, 5, 7, 100])
ind_a
# array([0, 2, 3, 6, 8, 9], dtype=int64)
ind_b
# array([0, 1, 5, 6, 7, 9], dtype=int64)
Which can then be reused like:
np.array(a)[ind_a]
np.array(b)[ind_b]
array([ 1, 2, 4, 5, 7, 100])
you can track indexes using enumerate as follows:
a = [1, 1, 2, 4, 4, 4, 5, 6, 7, 100]
b = [1, 2, 2, 2, 2, 4, 5, 7, 8, 100]
# 0 1 2 3 4 5 6 7 8 9
print([i for i,x in enumerate(zip(a,b)) if x[0] == x[1]])
[0, 2, 5, 6, 9]
so what's going on here?!
We're taking advantage of the amazing enumerate function! This function generates a tuple for each element in an iterable, with the first element being the enumeration (or the index in this case) and the second being the iterable.
Heres what the enumeration of zip(a,b) looks like
[(0, (1, 1)), (1, (1, 2)), (2, (2, 2)), (3, (4, 2)), (4, (4, 2)), (5, (4, 4)), (6, (5, 5)), (7, (6, 7)), (8, (7, 8)), (9, (100, 100))]
# lets look a little closer at one element
(0, (1, 1))
# ^ ^
# index iterable
From there on it's simple! Unpack the iterable and check if both elements are equal, and if they are then use append the enumeration # to a list!
Use range like so:
matching_idxs = [idx for idx in range(len(a)) if a[idx] == b[idx]]
print(matching_idxs)
# [0, 2, 5, 6, 9]
Using enumerate and zip:
a = [1, 1, 2, 4, 4, 4, 5, 6, 7, 100]
b = [1, 2, 2, 2, 2, 4, 5, 7, 8, 100]
output = [(i, x) for i, (x, y) in enumerate(zip(a, b)) if x == y]
print(output)
[(0, 1), (2, 2), (5, 4), (6, 5), (9, 100)]
This results in a list of tuples being (index, value)
this is very straightforward with the help of zip function to loop two lists in parallel:
>>> count = 0
>>> indices = []
>>> for x, y in zip(a, b):
if x == y:
indices.append(count)
count += 1
>>> indices
[0, 2, 5, 6, 9]
I have a list of things, and I need to apply some processing on successive (and overlapping) groups of 3 elements:
I can do it with:
for i in range(len(things)-2):
process(things[i:i+3])
So for instance:
things=[0, 1, 2, 3, 4, 5, 6, 7]
And I want to process:
[0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 7]
But is there a clever (but readable) way to do it without explicitly using len(things)?
Yes, what you're looking for is called a sliding/moving window. There are different ways to achieve this but the easiest is to use tee() and islice() functions from itertools. Using this you can define a window() function like below with default window size of 2.
import itertools
def window(iterable, n=2):
iters = itertools.tee(iterable, n)
for i, it in enumerate(iters):
next(itertools.islice(it, i, i), None)
return zip(*iters)
Then you can use it as
>>> things=[0, 1, 2, 3, 4, 5, 6, 7]
>>> list(window(things, n = 3))
[(0, 1, 2), (1, 2, 3), (2, 3, 4), (3, 4, 5), (4, 5, 6), (5, 6, 7)]
>>> for elem in window(things, n = 3):
... print(elem)
...
(0, 1, 2)
(1, 2, 3)
(2, 3, 4)
(3, 4, 5)
(4, 5, 6)
(5, 6, 7)
Edit: For one time use a more simpler option may be
>>> list(zip(things, things[1:], things[2:]))
[(0, 1, 2), (1, 2, 3), (2, 3, 4), (3, 4, 5), (4, 5, 6), (5, 6, 7)]
Another way of doing it could be:
for i in things[0:-2]:
a=things.index(i)
process(things[a:a+3])
Let's try using enumerate, Here len(things[i : i+len_]) == len_ is to drop uneven sized list that get's accumulated at the end iterations.
len_ = 3
[things[i : i+len_] for i, j in enumerate(things) if len(things[i : i+len_]) == len_]
[[0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 7]]
len_ = 4
[things[i : i+len_] for i, j in enumerate(things) if len(things[i : i+len_]) == len_]
[[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 6], [4, 5, 6, 7]]
If I have a list of lists and want to find all the possible combination from each different indices, how could I do that?
For example:
list_of_lists = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
I want to find
all_possibility = [[1, 5, 9], [1, 8, 6], [4, 2, 9], [4, 8, 3], [7, 2, 6], [7, 5, 3]]
where
[1,5,9]: 1 is 1st element of [1, 2, 3], 5 is 2nd element of [4, 5, 6], 9 is 3rd element of [7, 8, 9].
[1,8,6]: 1 is 1st element of [1, 2, 3], 8 is 2nd element of [7, 8, 9], 6 is 3rd element of [4, 5, 6].
and so on.
(Edited) Note: I would like the result to be in the same order as the original element of the list. [1, 8, 6] but not [1, 6, 8] because 8 is the 2nd element of [7, 8, 9].
What you're looking for is the Cartesian product, in Python itertools.product:
>>> import itertools
>>> list_of_lists = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> all_possibility = list(itertools.product(*list_of_lists))
>>> print(all_possibility)
[(1, 4, 7), (1, 4, 8), (1, 4, 9), (1, 5, 7), (1, 5, 8), (1, 5, 9), (1, 6, 7), (1, 6, 8),
(1, 6, 9), (2, 4, 7), (2, 4, 8), (2, 4, 9), (2, 5, 7), (2, 5, 8), (2, 5, 9), (2, 6, 7),
(2, 6, 8), (2, 6, 9), (3, 4, 7), (3, 4, 8), (3, 4, 9), (3, 5, 7), (3, 5, 8), (3, 5, 9),
(3, 6, 7), (3, 6, 8), (3, 6, 9)]
If you want permutations based on the indices rather than the values, you can use itertools.combinations to get the possible indices, then use those indices to get the respective values from the sub-lists, like this:
>>> list_of_lists = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> length = 3
>>> all_indices = list(itertools.permutations(range(length), length))
>>> all_possibility = [[l[i] for l,i in zip(list_of_lists, indices)] for indices in all_indices]
>>> print(all_possibility)
[[1, 5, 9], [1, 6, 8], [2, 4, 9], [2, 6, 7], [3, 4, 8], [3, 5, 7]]
I have to consider the indices as well. For example, (1, 4, 7) is excluded because 1, and 4 both are the 1st element from the list of the lists (from [1, 2, 3] and [4, 5, 6]). And actually (1, 4, 7) all of them are from the first component of the nested list. I need the cases with all the different indices.
So you actually just want to get the possible permutations of a “list selector” for each index in the output, i.e. these are what you are trying to get:
>>> list(itertools.permutations(range(3), 3))
[(0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0)]
And once you have that, you just need to translate into your list_of_lists where you access each index from the specified sublist:
>>> [[list_of_lists[k][i] for i, k in enumerate(comb)] for comb in itertools.permutations(range(3), 3)]
[[1, 5, 9], [1, 8, 6], [4, 2, 9], [4, 8, 3], [7, 2, 6], [7, 5, 3]]
In a spirit of #poke's approach, here is the cases for number of elements differ than the number of the list in the lists. (Previously, there are 3 elements in individual list where 3 sub-lists in the list).
list_of_lists = [[1, 2], [3, 4], [5, 6], [7, 8]]
We expect every (0, 1) pairs from the list of lists, or
all_possibility = [[1, 4], [1, 6], [1, 8], [3, 2], [3, 6], [3, 8], \
[5, 2], [5, 4], [5, 8], [7, 2], [7, 4], [7, 6]]
The code:
permutation_cases = list(itertools.permutations(range(2), 2))
select_from = list(itertools.combinations(range(len(list_of_lists)), 2))
all_possibility = []
for selecting_index in select_from:
selected = [list_of_lists[i] for i in selecting_index ]
cases = list([selected[k][i] for i, k in enumerate(comb)] for comb in permutation_cases)
for item in cases:
all_possibility.append(item)
print(all_possibility)
I am doing a question based on combinations and just stuck in it. And yes I am not so good in python.
The itertools combinations function using ncr just return the r possible combinations from n. I want something which will return the r possible combinations selected and also the other remaining elements from n numbers which were not selected in that iteration.
Example:
>>>from itertools import combinations
>>>list = [1, 2, 3, 4, 5]
>>>rslt = combinations(list, 2)
when [2, 4] is selected it should also return [1, 3, 5]
so it should return like [[2, 4], [1, 3, 5]]
Thanks in advance
itertools.combinations
Return r length subsequences of elements from the input iterable.
You can use a list comprehension to get the other items [j for j in l if j not in i]:
from itertools import combinations
l = [1, 2, 3, 4, 5]
for i in combinations(l,2):
print(list(i),[j for j in l if j not in i])
And you will get :
[1, 2] [3, 4, 5]
[1, 3] [2, 4, 5]
[1, 4] [2, 3, 5]
[1, 5] [2, 3, 4]
[2, 3] [1, 4, 5]
[2, 4] [1, 3, 5]
[2, 5] [1, 3, 4]
[3, 4] [1, 2, 5]
[3, 5] [1, 2, 4]
[4, 5] [1, 2, 3]
By the way, it's not recommended to use list as a variable name.
The simplest way would be to make a copy of the original list, removing the elements that are in the combination:
from itertools import combinations
def combinations_and_remaining(l, n):
for c in combinations(l, n):
diff = [i for i in l if i not in c]
yield c, diff
for i in combinations_and_remaining([1, 2, 3, 4, 5], 2):
print(i)
Will output
((1, 2), [3, 4, 5])
((1, 3), [2, 4, 5])
((1, 4), [2, 3, 5])
((1, 5), [2, 3, 4])
((2, 3), [1, 4, 5])
((2, 4), [1, 3, 5])
((2, 5), [1, 3, 4])
((3, 4), [1, 2, 5])
((3, 5), [1, 2, 4])
((4, 5), [1, 2, 3])
(The combinations returns tuples; remaining elements are returned as lists for efficiency)
One slightly extravagant but fun way would be to use combinations twice:
from itertools import combinations
n = 5
k = 2
lst = list(range(1, n+1))
rslt = zip(combinations(lst, k), map(tuple, reversed(list(combinations(lst, n-k)))))
print(list(rslt))
# -> [((1, 2), (3, 4, 5)), ((1, 3), (2, 4, 5)), ((1, 4), (2, 3, 5)),
# ((1, 5), (2, 3, 4)), ((2, 3), (1, 4, 5)), ((2, 4), (1, 3, 5)),
# ((2, 5), (1, 3, 4)), ((3, 4), (1, 2, 5)), ((3, 5), (1, 2, 4)),
# ((4, 5), (1, 2, 3))]
You can avoid loops and improve readability by using sets, as such:
from itertools import combinations
input = [1,2,3,4,5]
for n in itertools.combinations(input, 2):
print(n , set(n) ^ set(input))
That is, of course, assuming there are no duplicates in the original, which will be lost in the conversion to a set.
Let's say I have some_lst = [1, 2, 3, 4, 5, 6, 7, 8, 9] and I want to find all 3 element consecutive sub-lists: [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 7], [6, 7, 8], [7, 8, 9].
What's the most elegant way of doing this?
>>> some_lst = [1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> l = [some_lst[i:i+3] for i in xrange(len(some_lst)-2)]
Another option is slices and zip:
>>> some_lst = [1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> zip(some_lst, some_lst[1:], some_lst[2:])
[(1, 2, 3), (2, 3, 4), (3, 4, 5), (4, 5, 6), (5, 6, 7), (6, 7, 8), (7, 8, 9)]
With itertools.islice and itertools.izip you can make it more memory-efficient:
from itertools import islice, izip
izip(islice(some_lst, 0, None),
islice(some_lst, 1, None),
islice(some_lst, 2, None))
or
izip(*[islice(some_lst, s, None) for s in range(3)])
yet another way:
subLists = map(lambda x: some_lst[x-1:x+2], some_lst[:-2])