Given two lists of lists find specific matches - fastest solution - python

Given two lists of lists of arbitrary length, let's say list1 and list2 I want to divide the lists in list1 into subsets of lists, if they contain only one of the lists of list2.
I give you a specific example:
list1 = [[1, 2, 3, 4], [1, 2, 3, 5, 6, 8], [1, 2, 3, 6, 7], [1, 2, 3, 6, 8, 9, 10],
[1, 2, 3, 6, 8, 11, 12], [1, 2, 4, 5, 9, 10], [1, 2, 4, 5, 11, 12],
[1, 2, 5, 6, 7, 9, 10], [1, 2, 5, 6, 7, 11, 12], [1, 2, 5, 6, 8, 9, 10],
[1, 2, 5, 6, 8, 11, 12], [3, 4, 5, 6, 8], [3, 5, 9, 10], [3, 5, 11, 12],
[4, 6, 7], [4, 6, 8, 9, 10], [4, 6, 8, 11, 12], [9, 10, 11, 12]]
list2 = [[2], [6, 7], [6, 8], [9,9]]
and then desired outcome of the function would be for "inner" matches:
[[1, 2, 3, 4],
[1, 2, 4, 5, 11, 12],
[4, 6, 7],
[3, 4, 5, 6, 8],
[4, 6, 8, 11, 12],
[3, 5, 9, 10],
[9, 10, 11, 12]]
and for the "outer" matches (that are consequently the remaining items in list_1):
[(1, 2, 5, 6, 8, 11, 12),
(1, 2, 5, 6, 7, 11, 12),
(4, 6, 8, 9, 10),
(1, 2, 5, 6, 7, 9, 10),
(1, 2, 3, 5, 6, 8),
(1, 2, 3, 6, 8, 11, 12),
(1, 2, 3, 6, 7),
(3, 5, 11, 12),
(1, 2, 4, 5, 9, 10),
(1, 2, 5, 6, 8, 9, 10),
(1, 2, 3, 6, 8, 9, 10)]
I coded a quick and dirty solution that produces the desired outcome, but does not scale well for very long lists (for example 100000 & 2500).
My solution:
from itertools import chain
def find_all_sets(list1,list2):
d = {}
d2 = {}
count = 0
for i in list2:
count = count + 1
set2 = set(i)
d['set'+str(count)] = set2
d['lists'+str(count)] = []
first = []
d2['match'+str(count)] = []
for a in list1:
set1 = set(a)
if d['set'+str(count)].issubset(set1) == True:
first.append(a)
d['lists'+str(count)].append(first)
d2['match'+str(count)].append(d['lists'+str(count)])
count = 0
count2 = -1
d3 = {}
all_sub_lists = []
for i in d2.values():
count = count + 1
count2 = count2 + 1
d3['final'+str(count)] = []
real = []
for item in i:
for each_item in item:
for each_each_item in each_item:
seta= set(each_each_item)
save = []
for i in list2:
setb = set(i)
a=setb.issubset(seta)
save.append(a)
index_to_remove = count2
new_save = save[:index_to_remove] + save[index_to_remove + 1:]
if True not in new_save:
real.append(each_each_item)
d3['final'+str(count)].append(real)
all_sub_lists.append(real)
inner_matches = list(chain(*all_sub_lists))
setA = set(map(tuple, inner_matches))
setB = set(map(tuple, list1))
outer_matches = [i for i in setB if i not in setA]
return inner_matches, outer_matches
inner_matches, outer_matches = find_all_sets(list1,list2)
I am looking for a faster way to process large lists. Please excuse if the terminology of "inner" an "outer" matches is unclear. I did not know how else to call them.

Here is my suggestion (let me know if you need it as a function):
inner_matches=[]
outer_matches=[]
for i in list1:
if sum(1 for k in list2 if set(k).intersection(set(i))==set(k))==1:
inner_matches.append(i)
else:
outer_matches.append(i)
print(inner_matches)
#[[1, 2, 3, 4], [1, 2, 4, 5, 11, 12], [3, 4, 5, 6, 8], [3, 5, 9, 10], [4, 6, 7], [4, 6, 8, 11, 12], [9, 10, 11, 12]]
print(outer_matches)
#[[1, 2, 3, 5, 6, 8], [1, 2, 3, 6, 7], [1, 2, 3, 6, 8, 9, 10], [1, 2, 3, 6, 8, 11, 12], [1, 2, 4, 5, 9, 10], [1, 2, 5, 6, 7, 9, 10], [1, 2, 5, 6, 7, 11, 12], [1, 2, 5, 6, 8, 9, 10], [1, 2, 5, 6, 8, 11, 12], [3, 5, 11, 12], [4, 6, 8, 9, 10]]

Here's a solution that uses issubset() to detect the inner lists. Using your sample data it's faster than your algorithm by a factor of nearly 4.
inner = []
outer = []
search_sets = [set(l) for l in list2]
for l in list1:
if sum(s.issubset(l) for s in search_sets) == 1:
inner.append(l)
else:
outer.append(l)
print(f'{inner = }')
print()
print(f'{outer = }')
Output
inner = [[1, 2, 3, 4], [1, 2, 4, 5, 11, 12], [3, 4, 5, 6, 8], [3, 5, 9, 10], [4, 6, 7], [4, 6, 8, 11, 12], [9, 10, 11, 12]]
outer = [[1, 2, 3, 5, 6, 8], [1, 2, 3, 6, 7], [1, 2, 3, 6, 8, 9, 10], [1, 2, 3, 6, 8, 11, 12], [1, 2, 4, 5, 9, 10], [1, 2, 5, 6, 7, 9, 10], [1, 2, 5, 6, 7, 11, 12], [1, 2, 5, 6, 8, 9, 10], [1, 2, 5, 6, 8, 11, 12], [3, 5, 11, 12], [4, 6, 8, 9, 10]]

Related

Filtering all dictionary key values based on single key?

Say, if we have a dictionary like below:
{'time' : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
'x_coordinates': [3, 1, 2, 4, 6, 8, 2, 4, 8, 9],
'y_coordinates': [3, 5, 8, 1, 7, 3, 7, 2, 5, 2]
}
And one like:
{'time' : [2, 6, 8, 10]}
I want to filter out all the key values belonging to the first dict by the key values belonging to the second dict. That is my desired output would be:
{'time': [2, 6, 8, 10],
'x_coordinates': [1, 8, 4, 9],
'y_coordinates': [5, 3, 2, 2]
}
How can I do this in the most efficient way possible?
You can try this.
a={'time' : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
'x_coordinates': [3, 1, 2, 4, 6, 8, 2, 4, 8, 9],
'y_coordinates': [3, 5, 8, 1, 7, 3, 7, 2, 5, 2]
}
search={'time' : [2, 6, 8, 10]}
idx=[a['time'].index(i) for i in search['time']]
#[1, 5, 7, 9]
final_dict={key:[a[key][i] for i in idx] for key in a.keys()}
{'time': [2, 6, 8, 10],
'x_coordinates': [1, 8, 4, 9],
'y_coordinates': [5, 3, 2, 2]}
It appears that you are looking for corresponding list elements based on the list provided for time. This can be accomplished with zip, and to construct your dictionary, you could leverage a defaultdict
from collections import defaultdict
d = {'time' : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
'x_coordinates': [3, 1, 2, 4, 6, 8, 2, 4, 8, 9],
'y_coordinates': [3, 5, 8, 1, 7, 3, 7, 2, 5, 2]
}
# sets provide faster lookup times than lists
vals = set([2, 6, 8, 10])
new_values = defaultdict(list)
for time, x, y in zip(d['time'], d['x_coordinates'], d['y_coordinates']):
if time in vals:
# you only have to do a single membership test, then you
# simply append the desired values
new_values['time'].append(time)
new_values['x_coordinates'].append(x)
new_values['y_coordinates'].append(y)
defaultdict(<class 'list'>, {'time': [2, 6, 8, 10], 'x_coordinates': [1, 8, 4, 9], 'y_coordinates': [5, 3, 2, 2]})
The benefit here is that you only iterate once over all of the values

shuffle a list and then append it to another list

I use a for loop to shuffle a list and append it to another empty list (List A).
I can see each shuffled list is different, but the List A has been appended with multiple of the last shuffled list only.
print('---------shuffle list-------------------------------------')
matr=[ ]
entry=[1, 2, 3, 4, 5, 6, 7, 8, 9]
for i in range(9):
shuffle(entry )
print(entry )
matr.append(entry)
print(matr)
'''
results below:
---------shuffle list-------------------------------------
[3, 1, 7, 5, 8, 9, 2, 6, 4]
[5, 4, 6, 8, 1, 9, 7, 2, 3]
[6, 4, 7, 5, 1, 3, 2, 9, 8]
[4, 9, 8, 1, 7, 3, 6, 5, 2]
[5, 1, 9, 2, 8, 6, 4, 7, 3]
[3, 5, 1, 4, 2, 6, 8, 9, 7]
[1, 2, 4, 6, 7, 8, 3, 9, 5]
[4, 8, 1, 6, 7, 3, 5, 9, 2]
[6, 4, 2, 1, 9, 8, 3, 5, 7]
[[6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7], [6, 4, 2, 1, 9, 8, 3, 5, 7]]
'''
it should have appended each of the shuffled list rather the last shuffled list.
Their same objects, so you gotta do, shuffle is an exception where you need to do this:
matr=[]
entry=[1, 2, 3, 4, 5, 6, 7, 8, 9]
for i in range(9):
entry = entry.copy()
shuffle(entry)
print(entry)
matr.append(entry)
print(matr)
Output:
[9, 7, 6, 4, 3, 2, 8, 5, 1]
[4, 3, 2, 8, 7, 1, 5, 9, 6]
[3, 9, 7, 4, 5, 8, 2, 1, 6]
[6, 1, 9, 5, 2, 3, 4, 7, 8]
[5, 4, 7, 9, 8, 2, 6, 3, 1]
[2, 3, 5, 8, 6, 7, 9, 4, 1]
[5, 4, 9, 8, 3, 6, 1, 7, 2]
[5, 1, 2, 3, 7, 8, 6, 9, 4]
[6, 8, 9, 2, 1, 5, 3, 7, 4]
[[9, 7, 6, 4, 3, 2, 8, 5, 1], [4, 3, 2, 8, 7, 1, 5, 9, 6], [3, 9, 7, 4, 5, 8, 2, 1, 6], [6, 1, 9, 5, 2, 3, 4, 7, 8], [5, 4, 7, 9, 8, 2, 6, 3, 1], [2, 3, 5, 8, 6, 7, 9, 4, 1], [5, 4, 9, 8, 3, 6, 1, 7, 2], [5, 1, 2, 3, 7, 8, 6, 9, 4], [6, 8, 9, 2, 1, 5, 3, 7, 4]]

combinations of a list [duplicate]

This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 4 years ago.
I'm trying to get all the combinations of my list, but everytime two elements need to be removed, how do i remove those elements?
I tried to make a for loop two times and every time its removes two elements, but at the end it does not restore the list
indexes = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
for een in indexes:
for twee in indexes:
temp = indexes
if een == twee:
pass
else:
if een in temp:
temp.remove(een)
temp.remove(twee)
print(temp)
temp = indexes
i expected to output every time a list of length of 9 but the list keeps getting shorter.
the output i got was:
[2, 3, 4, 5, 6, 7, 8, 9, 10]
[2, 3, 5, 6, 7, 8, 9, 10]
[2, 3, 5, 7, 8, 9, 10]
[2, 3, 5, 7, 9, 10]
[2, 3, 5, 7, 9]
[5, 7, 9]
[5, 9]
the first list is correct, but on the next one, the 1 does not return to the list. what am i doing wrong? Also after this one is done een should equal to 1 and do it all over again, after that een should equal to 2.....
this should be the output
[2, 3, 4, 5, 6, 7, 8, 9, 10]
[1, 3, 4, 5, 6, 7, 8, 9, 10]
[1, 2, 4, 5, 6, 7, 8, 9, 10]
[1, 2, 3, 5, 6, 7, 8, 9, 10]
[1, 2, 3, 4, 6, 7, 8, 9, 10]
[1, 2, 3, 4, 5, 7, 8, 9, 10]
[1, 2, 3, 4, 5, 6, 8, 9, 10]
[1, 2, 3, 4, 5, 6, 7, 9, 10]
[1, 2, 3, 4, 5, 6, 7, 8, 10]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
[2, 3, 4, 5, 6, 7, 8, 9, 10]
[0, 3, 4, 5, 6, 7, 8, 9, 10]
[0, 2, 4, 5, 6, 7, 8, 9, 10]
[0, 2, 3, 5, 6, 7, 8, 9, 10]
[0, 2, 3, 4, 6, 7, 8, 9, 10]
[0, 2, 3, 4, 5, 7, 8, 9, 10]
[0, 2, 3, 4, 5, 6, 8, 9, 10]
[0, 2, 3, 4, 5, 6, 7, 9, 10]
[0, 2, 3, 4, 5, 6, 7, 8, 10]
[0, 2, 3, 4, 5, 6, 7, 8, 9]
[1, 3, 4, 5, 6, 7, 8, 9, 10]
[0, 3, 4, 5, 6, 7, 8, 9, 10]
[0, 1, 4, 5, 6, 7, 8, 9, 10]
and should go one untill every combination is reached
Replace temp = indexes by temp = indexes[:]

How to Make my Merge output Horizontally instead of Vertically in python

I have this python3 code that merges my sub-list to a single list:
l=[[4, 5, 6], [10], [1, 2, 3], [10], [1, 2, 3], [10], [4, 5, 6], [1, 2, 3], [4, 5, 6], [4, 5, 6], [7, 8, 9], [1, 2, 3], [7, 8, 9], [1, 2, 3], [4, 5, 6], [7, 8, 9], [4, 5, 6], [10], [7, 8, 9], [7, 8, 9]]
import itertools
merged = list(itertools.chain(*l))
from collections import Iterable
def flatten(items):
"""Yield items from any nested iterable; see Reference."""
for x in items:
if isinstance(x, Iterable) and not
isinstance(x, (str, bytes)):
for sub_x in flatten(x):
yield sub_x
else:
yield x
merged = list(itertools.chain(*l))
merged
The Undesired Shape of the Output
Though the output produces what I want but the shape of the output is not what I want
the output comes out in vertical shape as shown bellow:
[4,
5,
6,
10,
1,
2,
3,
10,
1,
2,
3,
10,
4,
5,
6,
1,
2,
3,
4,
5,
6,
4,
5,
6,
7,
8,
9,
1,
2,
3,
7,
8,
9,
1,
2,
3,
4,
5,
6,
7,
8,
9,
4,
5,
6,
10,
7,
8,
9,
7,
8,
9]
The Desirable Shape of the Output as I Would Want It
I would rather want the output to come out horizontally as I present bellow:
[4, 5, 6, 10, 1, 2, 3, 10, 1, 2, 3, 10, 4, 5, 6, 1, 2, 3, 4, 5, 6, 4, 5, 6, 7, 8, 9, 1, 2, 3, 7, 8, 9, 1, 2, 3, 4, 5, 6, 7, 8, 9, 4, 5, 6, 10, 7, 8, 9, 7, 8, 9]
Please help me out, I will not mind if there is a way to make this happen different from my code.
Instead just use list comprehention:
l=[[4, 5, 6], [10], [1, 2, 3], [10], [1, 2, 3], [10], [4, 5, 6], [1, 2, 3], [4, 5, 6], [4, 5, 6], [7, 8, 9], [1, 2, 3], [7, 8, 9], [1, 2, 3], [4, 5, 6], [7, 8, 9], [4, 5, 6], [10], [7, 8, 9], [7, 8, 9]]
new_l=[j for i in l for j in i]
print(new_l)
Output :
C:\Users\Desktop>py x.py
[4, 5, 6, 10, 1, 2, 3, 10, 1, 2, 3, 10, 4, 5, 6, 1, 2, 3, 4, 5, 6, 4, 5, 6, 7, 8, 9, 1, 2, 3, 7, 8, 9, 1, 2, 3, 4, 5, 6, 7, 8, 9, 4, 5, 6, 10, 7, 8, 9, 7, 8, 9]
You can do it like this:
In [5]: for x in merged:
...: print(x, end=' ')
...:
4 5 6 10 1 2 3 10 1 2 3 10 4 5 6 1 2 3 4 5 6 4 5 6 7 8 9 1 2 3 7 8 9 1 2 3 4 5 6 7 8 9 4 5 6 10 7 8 9 7 8 9
from collections import Counter
import itertools
import operator
list1=[[4, 5, 6], [10], [1, 2, 3], [10], [1, 2, 3], [10], [4, 5, 6], [1, 2, 3], [4, 5, 6], [4, 5, 6], [7, 8, 9], [1, 2, 3], [7, 8, 9], [1, 2, 3], [4, 5, 6], [7, 8, 9], [4, 5, 6], [10], [7, 8, 9], [7, 8, 9]]
dd = [item for sublist in list1 for item in sublist]
print(dd) # method 1
out1 = reduce(operator.concat,list1)
print(out1) # method 2
merged1 = list(itertools.chain.from_iterable(list1))
print(merged1) # method 3
merged2 = list(itertools.chain(*list1))
print(merged2) # method 4

Two Combination Lists from One List

I am a python beginner. I am trying to get two combination lists from one list.
For example, I have a list:
c = [1, 2, 3, 4]
I want to get every possible combination using every four items to fill two lists. There are going to be ((2^4)/2)-1 possibilities.
c1 = [1] c2 = [2, 3, 4]
c1 = [2] c2 = [1, 3, 4]
c1 = [3] c2 = [2, 3, 4]
c1 = [4] c2 = [1, 2, 3]
c1 = [1, 2] c2 = [3, 4]
c1 = [1, 3] c2 = [2, 4]
c1 = [1, 4] c2 = [2, 3]
The function usually works for this kind of task is itertools, but I cannot choose the number of lists produced by itertools.combination.
The function only allows me to choose how many items per one list should be.
For example, If I try following function,
print list(itertools.combinations(c, 2))
I can get an outcome only like this.
[(1,2),(1,3),(1,4),(2,3),(2,4),(3,4)]
I searched pretty hard to find this, but I couldn't find anything.
Update
Oh my poor English is causing such a confusion! I totally changed my example. I wanted to allocate the 4 items to 2 lists. Sorry for the confusion!
I'm unsure as to what your understanding of 10 choose 2 is. The output you receive from list(itertools.combinations(c, 2)) is what is mathematically defined as 10C2.
EDIT
From the edit to your question, it appears that you want an entirely different kind of combinations. The number of outcomes would still not be 45, but instead: 10C1 + 10C2 + 10C3 + 10C4 + 10C5.
I expect the following should help you move forward:
for i in range(1, 6):
for c1 in itertools.combinations(c, i):
c1 = set(c1)
c2 = set(c) - c1
print c1, c2
The above code was inspired by this (deleted) answer by CSZ.
The output received when range(1, 3) is used:
[1] [2, 3, 4, 5, 6, 7, 8, 9, 10]
[2] [1, 3, 4, 5, 6, 7, 8, 9, 10]
[3] [1, 2, 4, 5, 6, 7, 8, 9, 10]
[4] [1, 2, 3, 5, 6, 7, 8, 9, 10]
[5] [1, 2, 3, 4, 6, 7, 8, 9, 10]
[6] [1, 2, 3, 4, 5, 7, 8, 9, 10]
[7] [1, 2, 3, 4, 5, 6, 8, 9, 10]
[8] [1, 2, 3, 4, 5, 6, 7, 9, 10]
[9] [1, 2, 3, 4, 5, 6, 7, 8, 10]
[10] [1, 2, 3, 4, 5, 6, 7, 8, 9]
[1, 2] [3, 4, 5, 6, 7, 8, 9, 10]
[1, 3] [2, 4, 5, 6, 7, 8, 9, 10]
[1, 4] [2, 3, 5, 6, 7, 8, 9, 10]
[1, 5] [2, 3, 4, 6, 7, 8, 9, 10]
[1, 6] [2, 3, 4, 5, 7, 8, 9, 10]
[1, 7] [2, 3, 4, 5, 6, 8, 9, 10]
[8, 1] [2, 3, 4, 5, 6, 7, 9, 10]
[1, 9] [2, 3, 4, 5, 6, 7, 8, 10]
[1, 10] [2, 3, 4, 5, 6, 7, 8, 9]
[2, 3] [1, 4, 5, 6, 7, 8, 9, 10]
[2, 4] [1, 3, 5, 6, 7, 8, 9, 10]
[2, 5] [1, 3, 4, 6, 7, 8, 9, 10]
[2, 6] [1, 3, 4, 5, 7, 8, 9, 10]
[2, 7] [1, 3, 4, 5, 6, 8, 9, 10]
[8, 2] [1, 3, 4, 5, 6, 7, 9, 10]
[9, 2] [1, 3, 4, 5, 6, 7, 8, 10]
[2, 10] [1, 3, 4, 5, 6, 7, 8, 9]
[3, 4] [1, 2, 5, 6, 7, 8, 9, 10]
[3, 5] [1, 2, 4, 6, 7, 8, 9, 10]
[3, 6] [1, 2, 4, 5, 7, 8, 9, 10]
[3, 7] [1, 2, 4, 5, 6, 8, 9, 10]
[8, 3] [1, 2, 4, 5, 6, 7, 9, 10]
[9, 3] [1, 2, 4, 5, 6, 7, 8, 10]
[10, 3] [1, 2, 4, 5, 6, 7, 8, 9]
[4, 5] [1, 2, 3, 6, 7, 8, 9, 10]
[4, 6] [1, 2, 3, 5, 7, 8, 9, 10]
[4, 7] [1, 2, 3, 5, 6, 8, 9, 10]
[8, 4] [1, 2, 3, 5, 6, 7, 9, 10]
[9, 4] [1, 2, 3, 5, 6, 7, 8, 10]
[10, 4] [1, 2, 3, 5, 6, 7, 8, 9]
[5, 6] [1, 2, 3, 4, 7, 8, 9, 10]
[5, 7] [1, 2, 3, 4, 6, 8, 9, 10]
[8, 5] [1, 2, 3, 4, 6, 7, 9, 10]
[9, 5] [1, 2, 3, 4, 6, 7, 8, 10]
[10, 5] [1, 2, 3, 4, 6, 7, 8, 9]
[6, 7] [1, 2, 3, 4, 5, 8, 9, 10]
[8, 6] [1, 2, 3, 4, 5, 7, 9, 10]
[9, 6] [1, 2, 3, 4, 5, 7, 8, 10]
[10, 6] [1, 2, 3, 4, 5, 7, 8, 9]
[8, 7] [1, 2, 3, 4, 5, 6, 9, 10]
[9, 7] [1, 2, 3, 4, 5, 6, 8, 10]
[10, 7] [1, 2, 3, 4, 5, 6, 8, 9]
[8, 9] [1, 2, 3, 4, 5, 6, 7, 10]
[8, 10] [1, 2, 3, 4, 5, 6, 7, 9]
[9, 10] [1, 2, 3, 4, 5, 6, 7, 8]
l = [1,2,3,4, 5, 6, 7, 8]
print [[l[:i], l[i:]] for i in range(1, len(l))]
If you want all combinations. you can do like this.
print [l[i:i+n] for i in range(len(l)) for n in range(1, len(l)-i+1)]
or
itertools.combinations

Categories

Resources