Related
I have n multi-lists. I want to merge them into one big multi-list.
Example:
listA = [[1,2],[3,4],[6,7]]
listB = [[10,20],[30,40],[60,70]]
output:
listAB = [[1,2,10,20],[3,4,30,40],[6,7,60,70]]
This is the code I wrote which computes the above:
listAB,tmp = [],[]
for i in range(len(listA)):
for j in range(len(listA[i])):
tmp.append(listA[i][j])
tmp.append(listB[i][j])
listAB.append(tmp)
tmp = []
This code only works when you have two list, and the sub-list have the same length. I want to merge multiple multi-lists which have different lengths of sub-lists. Is there some function in itertools which can do these?
For example:
listA = [[1,2,3],[3,4,5],[6,7,8]]
listB = [[10,20],[30,40],[60,70]]
listC = [[4,4],[5,5],[6]]
listABC = [[1,2,3,10,20,4,4],
[3,4,5,30,40,5,5],
[6,7,8,60,70,6]]
zip the lists together and use itertools.chain to concatenate the output tuples:
from itertools import chain
listABC = [list(chain.from_iterable(i)) for i in zip(listA, listB, listC)]
Output:
[[1, 2, 3, 10, 20, 4, 4], [3, 4, 5, 30, 40, 5, 5], [6, 7, 8, 60, 70, 6]]
Although directly appending the lists is simpler for 3 inputs, with this approach you can also pass an arbitrary number of lists using the star operator:
# this can also be a tuple or any iterable
list_of_lists = [listA, listB, listC, listD ... ]
[list(chain.from_iterable(i)) for i in zip(*list_of_lists)]
UPDATE: an alternative solution (not recommended) using functools instead of itertools (as per the discussion with ReblochonMasque):
from functools import reduce
[reduce(lambda x,y: x+y, i) for i in zip(listA, listB, listC)]
For two sequences, you can add them in a list comprehension:
[a+b for a, b in zip(A, B)]
For more than two, #meowgoesthedog answer is appropriate.
If for some reasons, you cannot use itertools, you could maybe do like this:
def merge(seq1, seq2):
return (a+b for a, b in zip(seq1, seq2))
def merge_all(s):
res = merge(s[0], s[1])
for sn in s[2:]:
res = merge(res, sn)
return list(res)
A = [[1,2],[3,4],[6,7]]
B = [[10,20],[30,40],[60,70]]
C = [[4,4],[5,5],[6]]
seq = [A, B, C]
merge_all(seq)
I believe you can merge multiple multi-lists which have different lengths of sub-lists just by some simple code w/o using any itertools:
listCom = []
for n in range(len(listA)):
listCom.append(listA[n] + listB[n] + listC[n])
listCom
The output is:
[[1, 2, 3, 10, 20, 4, 4], [3, 4, 5, 30, 40, 5, 5], [6, 7, 8, 60, 70, 6]]
As last part of a bigger project here is what I am trying to solve:
I have a list of lists of which I need to extract exactlty one element based on the value of a second list.
a = [[6,2,3,9], [10,19,14,11], [27,28,21,24]]
b = [0,2,2]
The values in b indicate the positions of the elements in the sublists. Also, the index in b is the true for the index of elements in list a.
The result I am looking for is:
c = [6, 14, 21]
I have tried many versions of this:
c = [i[j] for i in a for j in b]
But as a result I get a list over all emements of all lists looking like this:
c = [6, 3, 3, 10, 14, 14, 27, 21, 21]
By using nested loops, you are combining every element from a with every element from b. What you want is pair-wise iteration, using zip:
c = [x[y] for x, y in zip(a, b)]
# [6, 14, 21]
This is roughly equivalent to:
c = [a[i][b[i]] for i in range(min(len(a), len(b)))]
Or:
[v[b[i]] for i,v in enumerate(a)]
Example:
>>> a = [[6,2,3,9], [10,19,14,11], [27,28,21,24]]
>>> b = [0,2,2]
>>> [v[b[i]] for i,v in enumerate(a)]
[6, 14, 21]
>>>
Try this:
c = [a[i][b[i]] for i in xrange(len(b))]
You can try the following.
a = [[6,2,3,9], [10,19,14,11], [27,28,21,24]]
b = [0,2,2]
c = []
for i in range(0, len(b)):
c.append(a[i][b[i]])
print (c)
What I want to do is to choose one item in list A and another one in list B, pair them like:
A[0]+B[n], A[1]+B[n-1],.....,A[n]+B[1]
I use two for loops but it doesn't work:
class Solution(object):
def plusOne(self, digits):
sum=0
for j in range(len(digits)-1,0,-1) :
for i in range(0,len(digits),1):
sum=sum+digits[i]*pow(10,j)
return sum+1
I inputted [1,2,3] and what I want to get is 124,
but I got 661.
Edit:
Sorry, the example I gave above is not so clear.
Let us think about A[1,2,3] and B[6,5,4].
I want output [5,7,9], because 5 is 1+4, 7 is 2+5, 9 is 3+6
What you are trying to do is turn a list of digits into the according number (and add 1). You can enumerate the reversed list in order to pair a digit with its appropriate power of 10:
digits = [1, 2, 3]
sum(10**i * y for i, y in enumerate(digits[::-1])) + 1
# 124
You can apply that to your other example as follows, using zip:
A = [1,2,3]
B = [6,5,4]
sum(10**i * (x+y) for i, (x, y) in enumerate(zip(B, A[::-1])))
# 579
You can do this without a loop:
A = [1,2,3]
B = [6,5,4]
C = list(map(sum,zip(A,B[::-1]) ))
print(C)
zip() - creates pairs of all elements of iterables, you feed it A and B reversed (via slicing). Then you sum up each pair and create a list from those sums.
map( function, iterable) - applies the function to each element of the iterable
zip() works when both list have the same length, else you would need to leverage itertools.zip_longest() with a defaultvalue of 0.
K = [1,2,3,4,5,6]
P = list(map(sum, zip_longest(K,C,fillvalue=0)))
print(P)
Output:
[5, 7, 9] # zip of 2 same length lists A and B reversed
[6, 9, 12, 4, 5, 6] # ziplongest for no matter what length lists
You only need one loop if you want to search in same list back and forth or different list with same length (i and len(lst)-1-i).
Try not use build-ins such as sum, list, tuple, str, int as variable names, it will give you some nasty result in some case.
class Solution(object):
def plusOne(self, digits):
sum_val = 0
for i in range(len(digits)):
sum_val += digits[i]*pow(10, len(digits)-1-i)
return sum_val+1
sol = Solution()
dig = [1, 2, 3]
print(sol.plusOne(dig))
Output:
124
for A = [1, 2, 3] and B = [6, 5, 4].
You can use a list comprehension:
res = [A[i]+B[len(A)-i-1] for i in range(len(A))]
Or the zip() function and a list comprehension:
res = [a+b for (a, b) in zip(A, reversed(B))]
Result:
[5, 7, 9]
I hope to write the join_lists function to take an arbitrary number of lists and concatenate them. For example, if the inputs are
m = [1, 2, 3]
n = [4, 5, 6]
o = [7, 8, 9]
then we I call print join_lists(m, n, o), it will return [1, 2, 3, 4, 5, 6, 7, 8, 9]. I realize I should use *args as the argument in join_lists, but not sure how to concatenate an arbitrary number of lists. Thanks.
Although you can use something which invokes __add__ sequentially, that is very much the wrong thing (for starters you end up creating as many new lists as there are lists in your input, which ends up having quadratic complexity).
The standard tool is itertools.chain:
def concatenate(*lists):
return itertools.chain(*lists)
or
def concatenate(*lists):
return itertools.chain.from_iterable(lists)
This will return a generator which yields each element of the lists in sequence. If you need it as a list, use list: list(itertools.chain.from_iterable(lists))
If you insist on doing this "by hand", then use extend:
def concatenate(*lists):
newlist = []
for l in lists: newlist.extend(l)
return newlist
Actually, don't use extend like that - it's still inefficient, because it has to keep extending the original list. The "right" way (it's still really the wrong way):
def concatenate(*lists):
lengths = map(len,lists)
newlen = sum(lengths)
newlist = [None]*newlen
start = 0
end = 0
for l,n in zip(lists,lengths):
end+=n
newlist[start:end] = list
start+=n
return newlist
http://ideone.com/Mi3UyL
You'll note that this still ends up doing as many copy operations as there are total slots in the lists. So, this isn't any better than using list(chain.from_iterable(lists)), and is probably worse, because list can make use of optimisations at the C level.
Finally, here's a version using extend (suboptimal) in one line, using reduce:
concatenate = lambda *lists: reduce((lambda a,b: a.extend(b) or a),lists,[])
One way would be this (using reduce) because I currently feel functional:
import operator
from functools import reduce
def concatenate(*lists):
return reduce(operator.add, lists)
However, a better functional method is given in Marcin's answer:
from itertools import chain
def concatenate(*lists):
return chain(*lists)
although you might as well use itertools.chain(*iterable_of_lists) directly.
A procedural way:
def concatenate(*lists):
new_list = []
for i in lists:
new_list.extend(i)
return new_list
A golfed version: j=lambda*x:sum(x,[]) (do not actually use this).
You can use sum() with an empty list as the start argument:
def join_lists(*lists):
return sum(lists, [])
For example:
>>> join_lists([1, 2, 3], [4, 5, 6])
[1, 2, 3, 4, 5, 6]
Another way:
>>> m = [1, 2, 3]
>>> n = [4, 5, 6]
>>> o = [7, 8, 9]
>>> p = []
>>> for (i, j, k) in (m, n, o):
... p.append(i)
... p.append(j)
... p.append(k)
...
>>> p
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>
This seems to work just fine:
def join_lists(*args):
output = []
for lst in args:
output += lst
return output
It returns a new list with all the items of the previous lists. Is using + not appropriate for this kind of list processing?
Or you could be logical instead, making a variable (here 'z') equal to the first list passed to the 'join_lists' function
then assigning the items in the list (not the list itself) to a new list to which you'll then be able add the elements of the other lists:
m = [1, 2, 3]
n = [4, 5, 6]
o = [7, 8, 9]
def join_lists(*x):
z = [x[0]]
for i in range(len(z)):
new_list = z[i]
for item in x:
if item != z:
new_list += (item)
return new_list
then
print (join_lists(m, n ,o)
would output:
[1, 2, 3, 4, 5, 6, 7, 8, 9]
I need to verify if a list is a subset of another - a boolean return is all I seek.
Is testing equality on the smaller list after an intersection the fastest way to do this? Performance is of utmost importance given the number of datasets that need to be compared.
Adding further facts based on discussions:
Will either of the lists be the same for many tests? It does as one of them is a static lookup table.
Does it need to be a list? It does not - the static lookup table can be anything that performs best. The dynamic one is a dict from which we extract the keys to perform a static lookup on.
What would be the optimal solution given the scenario?
>>> a = [1, 3, 5]
>>> b = [1, 3, 5, 8]
>>> c = [3, 5, 9]
>>> set(a) <= set(b)
True
>>> set(c) <= set(b)
False
>>> a = ['yes', 'no', 'hmm']
>>> b = ['yes', 'no', 'hmm', 'well']
>>> c = ['sorry', 'no', 'hmm']
>>>
>>> set(a) <= set(b)
True
>>> set(c) <= set(b)
False
Use set.issubset
Example:
a = {1,2}
b = {1,2,3}
a.issubset(b) # True
a = {1,2,4}
b = {1,2,3}
a.issubset(b) # False
The performant function Python provides for this is set.issubset. It does have a few restrictions that make it unclear if it's the answer to your question, however.
A list may contain items multiple times and has a specific order. A set does not. Additionally, sets only work on hashable objects.
Are you asking about subset or subsequence (which means you'll want a string search algorithm)? Will either of the lists be the same for many tests? What are the datatypes contained in the list? And for that matter, does it need to be a list?
Your other post intersect a dict and list made the types clearer and did get a recommendation to use dictionary key views for their set-like functionality. In that case it was known to work because dictionary keys behave like a set (so much so that before we had sets in Python we used dictionaries). One wonders how the issue got less specific in three hours.
one = [1, 2, 3]
two = [9, 8, 5, 3, 2, 1]
all(x in two for x in one)
Explanation: Generator creating booleans by looping through list one checking if that item is in list two. all() returns True if every item is truthy, else False.
There is also an advantage that all return False on the first instance of a missing element rather than having to process every item.
Assuming the items are hashable
>>> from collections import Counter
>>> not Counter([1, 2]) - Counter([1])
False
>>> not Counter([1, 2]) - Counter([1, 2])
True
>>> not Counter([1, 2, 2]) - Counter([1, 2])
False
If you don't care about duplicate items eg. [1, 2, 2] and [1, 2] then just use:
>>> set([1, 2, 2]).issubset([1, 2])
True
Is testing equality on the smaller list after an intersection the fastest way to do this?
.issubset will be the fastest way to do it. Checking the length before testing issubset will not improve speed because you still have O(N + M) items to iterate through and check.
One more solution would be to use a intersection.
one = [1, 2, 3]
two = [9, 8, 5, 3, 2, 1]
set(one).intersection(set(two)) == set(one)
The intersection of the sets would contain of set one
(OR)
one = [1, 2, 3]
two = [9, 8, 5, 3, 2, 1]
set(one) & (set(two)) == set(one)
Set theory is inappropriate for lists since duplicates will result in wrong answers using set theory.
For example:
a = [1, 3, 3, 3, 5]
b = [1, 3, 3, 4, 5]
set(b) > set(a)
has no meaning. Yes, it gives a false answer but this is not correct since set theory is just comparing: 1,3,5 versus 1,3,4,5. You must include all duplicates.
Instead you must count each occurrence of each item and do a greater than equal to check. This is not very expensive, because it is not using O(N^2) operations and does not require quick sort.
#!/usr/bin/env python
from collections import Counter
def containedInFirst(a, b):
a_count = Counter(a)
b_count = Counter(b)
for key in b_count:
if a_count.has_key(key) == False:
return False
if b_count[key] > a_count[key]:
return False
return True
a = [1, 3, 3, 3, 5]
b = [1, 3, 3, 4, 5]
print "b in a: ", containedInFirst(a, b)
a = [1, 3, 3, 3, 4, 4, 5]
b = [1, 3, 3, 4, 5]
print "b in a: ", containedInFirst(a, b)
Then running this you get:
$ python contained.py
b in a: False
b in a: True
one = [1, 2, 3]
two = [9, 8, 5, 3, 2, 1]
set(x in two for x in one) == set([True])
If list1 is in list 2:
(x in two for x in one) generates a list of True.
when we do a set(x in two for x in one) has only one element (True).
Pardon me if I am late to the party. ;)
To check if one set A is subset of set B, Python has A.issubset(B) and A <= B. It works on set only and works great BUT the complexity of internal implementation is unknown. Reference: https://docs.python.org/2/library/sets.html#set-objects
I came up with an algorithm to check if list A is a subset of list B with following remarks.
To reduce complexity of finding subset, I find it appropriate to
sort both lists first before comparing elements to qualify for
subset.
It helped me to break the loop when value of element of second list B[j] is greater than value of element of first list A[i].
last_index_j is used to start loop over list B where it last left off. It helps avoid starting comparisons from the start of
list B (which is, as you might guess unnecessary, to start list B from index 0 in subsequent iterations.)
Complexity will be O(n ln n) each for sorting both lists and O(n) for checking for subset.
O(n ln n) + O(n ln n) + O(n) = O(n ln n).
Code has lots of print statements to see what's going on at each iteration of the loop. These are meant for understanding
only.
Check if one list is subset of another list
is_subset = True;
A = [9, 3, 11, 1, 7, 2];
B = [11, 4, 6, 2, 15, 1, 9, 8, 5, 3];
print(A, B);
# skip checking if list A has elements more than list B
if len(A) > len(B):
is_subset = False;
else:
# complexity of sorting using quicksort or merge sort: O(n ln n)
# use best sorting algorithm available to minimize complexity
A.sort();
B.sort();
print(A, B);
# complexity: O(n^2)
# for a in A:
# if a not in B:
# is_subset = False;
# break;
# complexity: O(n)
is_found = False;
last_index_j = 0;
for i in range(len(A)):
for j in range(last_index_j, len(B)):
is_found = False;
print("i=" + str(i) + ", j=" + str(j) + ", " + str(A[i]) + "==" + str(B[j]) + "?");
if B[j] <= A[i]:
if A[i] == B[j]:
is_found = True;
last_index_j = j;
else:
is_found = False;
break;
if is_found:
print("Found: " + str(A[i]));
last_index_j = last_index_j + 1;
break;
else:
print("Not found: " + str(A[i]));
if is_found == False:
is_subset = False;
break;
print("subset") if is_subset else print("not subset");
Output
[9, 3, 11, 1, 7, 2] [11, 4, 6, 2, 15, 1, 9, 8, 5, 3]
[1, 2, 3, 7, 9, 11] [1, 2, 3, 4, 5, 6, 8, 9, 11, 15]
i=0, j=0, 1==1?
Found: 1
i=1, j=1, 2==1?
Not found: 2
i=1, j=2, 2==2?
Found: 2
i=2, j=3, 3==3?
Found: 3
i=3, j=4, 7==4?
Not found: 7
i=3, j=5, 7==5?
Not found: 7
i=3, j=6, 7==6?
Not found: 7
i=3, j=7, 7==8?
not subset
Below code checks whether a given set is a "proper subset" of another set
def is_proper_subset(set, superset):
return all(x in superset for x in set) and len(set)<len(superset)
In python 3.5 you can do a [*set()][index] to get the element. It is much slower solution than other methods.
one = [1, 2, 3]
two = [9, 8, 5, 3, 2, 1]
result = set(x in two for x in one)
[*result][0] == True
or just with len and set
len(set(a+b)) == len(set(a))
Here is how I know if one list is a subset of another one, the sequence matters to me in my case.
def is_subset(list_long,list_short):
short_length = len(list_short)
subset_list = []
for i in range(len(list_long)-short_length+1):
subset_list.append(list_long[i:i+short_length])
if list_short in subset_list:
return True
else: return False
Most of the solutions consider that the lists do not have duplicates. In case your lists do have duplicates you can try this:
def isSubList(subList,mlist):
uniqueElements=set(subList)
for e in uniqueElements:
if subList.count(e) > mlist.count(e):
return False
# It is sublist
return True
It ensures the sublist never has different elements than list or a greater amount of a common element.
lst=[1,2,2,3,4]
sl1=[2,2,3]
sl2=[2,2,2]
sl3=[2,5]
print(isSubList(sl1,lst)) # True
print(isSubList(sl2,lst)) # False
print(isSubList(sl3,lst)) # False
Since no one has considered comparing two strings, here's my proposal.
You may of course want to check if the pipe ("|") is not part of either lists and maybe chose automatically another char, but you got the idea.
Using an empty string as separator is not a solution since the numbers can have several digits ([12,3] != [1,23])
def issublist(l1,l2):
return '|'.join([str(i) for i in l1]) in '|'.join([str(i) for i in l2])
If you are asking if one list is "contained" in another list then:
>>>if listA in listB: return True
If you are asking if each element in listA has an equal number of matching elements in listB try:
all(True if listA.count(item) <= listB.count(item) else False for item in listA)