Related
My code find all combinations of a list of numbers with a given sum. The code is working well, but when trying big numbers (like 100 or 200), the code is taking way too long.
Any advices on how to make the code much faster ?
def check(target, lst):
def _a(idx, l, r, t):
if t == sum(l): r.append(l)
elif t < sum(l): return
for u in range(idx, len(lst)):
_a(u, l + [lst[u]], r, t)
return r
return len(_a(0, [], [], target))
print(check(200, (1, 2, 5, 10, 20, 50, 100, 200, 500)))
Make the inner function simpler (only give it index and remaining target, and return the number) and then memoize it?
from functools import lru_cache
def check(target, lst):
#lru_cache(None)
def a(idx, t):
if t == 0: return 1
elif t < 0: return 0
return sum(a(u, t - lst[u])
for u in range(idx, len(lst)))
return a(0, target)
print(check(200, (1, 2, 5, 10, 20, 50, 100, 200, 500)))
you could use itertools to iterate through every combination of every possible size, and filter out everything that doesn't sum to 10:
import itertools
numbers = [1, 2, 3, 7, 7, 9, 10]
result = [seq for i in range(len(numbers), 0, -1) for seq in
itertools.combinations(numbers, i) if sum(seq) == 10]
print result
result
[(1, 2, 7), (1, 2, 7), (1, 9), (3, 7), (3, 7), (10,)]
Unfortunately this is something like O(2^N) complexity, so it isn't suitable for input lists larger than, say, 20 elements.
Here's the task:
I have ten cards in a list with these values: [2,3,4,5,6,7,8,9,10,11]
Im supposed to gather three cards with the total value of 21. I have to print every possible combinations of three cards with the total value of 21 as sorted tuples. Example of the output:
(2, 8, 11)
(2, 9, 10)
(3, 7, 11)
(3, 8, 10)
(4, 6, 11)
...
I have NO idea how to do this and where to start.
itertools.combinations will give all possible combinations. Mathematically, nCr = n!/((n-r)! *r!).
# If n = 4 and r =3 then
# nCr = 4!/1!*3! = 4*3*2*1 / (3*2*1)*(1) = 4.
More help on itertools -
Init signature: itertools.combinations(iterable, r)
Docstring:
Return successive r-length combinations of elements in the iterable.
combinations(range(4), 3) --> (0,1,2), (0,1,3), (0,2,3), (1,2,3)
Type: type
Subclasses:
So, by itertools.combination we can get all the possible combinations of length 3.
import itertools
cards = [2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
all_combinations = itertools.combinations(cards, 3)
Now, to filter only those combination which has sum =21 we can use list comprehension. Python has an in-built function sum(iterable) to do this task.
Signature: sum(iterable, /, start=0)
Docstring:
Return the sum of a 'start' value (default: 0) plus an iterable of numbers
When the iterable is empty, return the start value.
This function is intended specifically for use with numeric values and may
reject non-numeric types.
Type: builtin_function_or_method
The last requirement was to sort the tuple which can be done via another in-built function sorted.
result = [
tuple(sorted(combination))
for combination in all_combinations
if sum(combination) == 21
]
print(result)
Use itertools.combinations to generate all possible 3-card combinations, and sort each in ascending order
import itertools
cards = [2,3,4,5,6,7,8,9,10,11]
for combo in itertools.combinations(cards, 3):
print(tuple(sorted(combo)))
# alternatively, if you need to collect them in a list for later processing...
output = [ tuple(sorted(combo)) for combo in itertools.combinations(cards, 3) ]
Before you suggest that this question is similar to another, read P.P.S, pls.
Simillarly to this question, I am looking to find all the non-repeating combinations of a list of factors: But beeing looking for a python solution, and also having a slighty diferent premise, I decided it's worth opening a new question.
The input to the combinations function would be of the form [2,5,3,1,5,1,11,2], a list where the odd entries are the primes and the even are the number of times they are present. This number would be (2^5)*3*5*(11^2), or 58080. My goal is to print all the combinations (in this case products) of the different factors.
My try is the following (b is the list where i have the primes, and div a blank list where I put the divisors(don't mind 1 as a divisor):
n=len(b)//2
a=1
if a<=n:
for i in range (n):
for g in range (1,b[2*i+1]+1):
div.append (b[2*i]**g)
a+=1
if a<=n:
for i in range (n):
for o in range (i+1,n):
for g in range(1,b[2*i+1]+1):
for h in range(1,b[2*o+1]+1):
div.append ((b[2*i]**g)*(b[2*o]**h))
This adds to the list all the combinations of at most two different prime factors, but there must be a way to continue this to numbers of n different prime factors without mannualy adding more code. But the most important is that it will not generate repeated products. If there is an answer out there please redirect me to it. Thanks in advance to all.
P.S. For example, take 60. 60 will be factored by a function (not showed here) into [2, 2, 3, 1, 5, 1]. My desired output is all the divisors of 60, in order or not, like this [1,2,3,4,5,6,10,12,15,30,60] All the combinations of the products of the factors. (2,2*2,3,5,2*3,2*5,2*2*3,2*2*5,3*5,2*2*3*5 (and 1, that is added to div before or after))
P.P.S. The difference to this (another) question relies on 2 things.
First, and most important, the point of this question isn't finding divisors, but combinations. Divisors is just the context for it, but I would like to know for future problems how to make such iterations. Second, like I said in the comments, even if were about divisors, finding the primes and only then combining them is more efficient (for large N) than looking for numbers until the sqrt, and the refered post revolves around that (for an example why, see in comments).
Your friend is itertools:
from itertools import product
from functools import reduce
def grow(factor, power):
#returns list [1, factor, factor^2, ..., factor^power]
array = []
for pw in range(power+1):
if pw != 0:
k *= factor
else:
k = 1
array.append(k)
return array
x = [2,2,3,1,5,1]
prime_factors = [x[i] for i in range(0, len(x), 2)]
powers = [x[i] for i in range(1, len(x), 2)]
divisor_tree = [grow(*n) for n in zip(prime_factors, powers)]
divisor_groups = product(*divisor_tree)
# returns iterator of [(1, 1, 1), (1, 1, 5), (1, 3, 1), (1, 3, 5), (2, 1, 1), (2, 1, 5), (2, 3, 1), (2, 3, 5), (4, 1, 1), (4, 1, 5), (4, 3, 1), (4, 3, 5)]
result = [reduce(lambda x,y: x*y, n) for n in divisor_groups]
print(result)
Output:
[1, 5, 3, 15, 2, 10, 6, 30, 4, 20, 12, 60]
Now I introduce what it does:
Extracts prime_factors and their powers from your list
zip(prime_factors, powers) pairs them with each other
grow returns list consecutive powers as commented
divisor_groups is iterable of all possible selections from group of these lists, each item taken from separate list
reduce(lambda x,y: x*y, n) maps selected tuple of factors to a product of these factors, e.g. (2,3,5) -> 30
Edit:
You might like to implement grow in this way, less efficient but more readable:
def grow(factor, power):
return [factor**i for i in range(power+1)]
Pardon in advance if my question appears too naive or too basic. I'm a still learning to use SE.
Is there a quick way to convert tree-form tuple to a matrix-form tuple (triangular form)?
That is this:
t=((1,),(2,3,),(4,5,6,),(7,8,9,10,))
to this:
t=((1,0,0,0,),(2,3,0,0,),(4,5,6,0,),(7,8,9,10,))
Just an idea would suffice. I can try implementing it myself.
You could add 0s if the length is less than 4
tuple(a+(0,)*(4-len(a)) for a in t)
((1, 0, 0, 0), (2, 3, 0, 0), (4, 5, 6, 0), (7, 8, 9, 10))
take advantage of tuple concatenating: (this is just something to start with like you said)
>>> (1,)+(0,)*2
(1, 0, 0)
and max() and len() function:
>>> len(max(((1,),(2,3,),(4,5,6,),(7,8,9,10,)),key = lambda x: len(x)))
4
I would do something like that even if it's not optimised:
def tuple_to_serie(t)
s = []
max_len = max( [len(tup) for tup in t] )
for tup in t :
s.append( tup+(0,)*(max_len-len(tup)) )
return tuple(s)
Then :
t = tuple_to_serie(t)
I want an algorithm to iterate over list slices. Slices size is set outside the function and can differ.
In my mind it is something like:
for list_of_x_items in fatherList:
foo(list_of_x_items)
Is there a way to properly define list_of_x_items or some other way of doing this using python 2.5?
edit1: Clarification Both "partitioning" and "sliding window" terms sound applicable to my task, but I am no expert. So I will explain the problem a bit deeper and add to the question:
The fatherList is a multilevel numpy.array I am getting from a file. Function has to find averages of series (user provides the length of series) For averaging I am using the mean() function. Now for question expansion:
edit2: How to modify the function you have provided to store the extra items and use them when the next fatherList is fed to the function?
for example if the list is lenght 10 and size of a chunk is 3, then the 10th member of the list is stored and appended to the beginning of the next list.
Related:
What is the most “pythonic” way to iterate over a list in chunks?
If you want to divide a list into slices you can use this trick:
list_of_slices = zip(*(iter(the_list),) * slice_size)
For example
>>> zip(*(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
If the number of items is not dividable by the slice size and you want to pad the list with None you can do this:
>>> map(None, *(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]
It is a dirty little trick
OK, I'll explain how it works. It'll be tricky to explain but I'll try my best.
First a little background:
In Python you can multiply a list by a number like this:
[1, 2, 3] * 3 -> [1, 2, 3, 1, 2, 3, 1, 2, 3]
([1, 2, 3],) * 3 -> ([1, 2, 3], [1, 2, 3], [1, 2, 3])
And an iterator object can be consumed once like this:
>>> l=iter([1, 2, 3])
>>> l.next()
1
>>> l.next()
2
>>> l.next()
3
The zip function returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. For example:
zip([1, 2, 3], [20, 30, 40]) -> [(1, 20), (2, 30), (3, 40)]
zip(*[(1, 20), (2, 30), (3, 40)]) -> [[1, 2, 3], [20, 30, 40]]
The * in front of zip used to unpack arguments. You can find more details here.
So
zip(*[(1, 20), (2, 30), (3, 40)])
is actually equivalent to
zip((1, 20), (2, 30), (3, 40))
but works with a variable number of arguments
Now back to the trick:
list_of_slices = zip(*(iter(the_list),) * slice_size)
iter(the_list) -> convert the list into an iterator
(iter(the_list),) * N -> will generate an N reference to the_list iterator.
zip(*(iter(the_list),) * N) -> will feed those list of iterators into zip. Which in turn will group them into N sized tuples. But since all N items are in fact references to the same iterator iter(the_list) the result will be repeated calls to next() on the original iterator
I hope that explains it. I advice you to go with an easier to understand solution. I was only tempted to mention this trick because I like it.
If you want to be able to consume any iterable you can use these functions:
from itertools import chain, islice
def ichunked(seq, chunksize):
"""Yields items from an iterator in iterable chunks."""
it = iter(seq)
while True:
yield chain([it.next()], islice(it, chunksize-1))
def chunked(seq, chunksize):
"""Yields items from an iterator in list chunks."""
for chunk in ichunked(seq, chunksize):
yield list(chunk)
Use a generator:
big_list = [1,2,3,4,5,6,7,8,9]
slice_length = 3
def sliceIterator(lst, sliceLen):
for i in range(len(lst) - sliceLen + 1):
yield lst[i:i + sliceLen]
for slice in sliceIterator(big_list, slice_length):
foo(slice)
sliceIterator implements a "sliding window" of width sliceLen over the squence lst, i.e. it produces overlapping slices: [1,2,3], [2,3,4], [3,4,5], ... Not sure if that is the OP's intention, though.
Do you mean something like:
def callonslices(size, fatherList, foo):
for i in xrange(0, len(fatherList), size):
foo(fatherList[i:i+size])
If this is roughly the functionality you want you might, if you desire, dress it up a bit in a generator:
def sliceup(size, fatherList):
for i in xrange(0, len(fatherList), size):
yield fatherList[i:i+size]
and then:
def callonslices(size, fatherList, foo):
for sli in sliceup(size, fatherList):
foo(sli)
Answer to the last part of the question:
question update: How to modify the
function you have provided to store
the extra items and use them when the
next fatherList is fed to the
function?
If you need to store state then you can use an object for that.
class Chunker(object):
"""Split `iterable` on evenly sized chunks.
Leftovers are remembered and yielded at the next call.
"""
def __init__(self, chunksize):
assert chunksize > 0
self.chunksize = chunksize
self.chunk = []
def __call__(self, iterable):
"""Yield items from `iterable` `self.chunksize` at the time."""
assert len(self.chunk) < self.chunksize
for item in iterable:
self.chunk.append(item)
if len(self.chunk) == self.chunksize:
# yield collected full chunk
yield self.chunk
self.chunk = []
Example:
chunker = Chunker(3)
for s in "abcd", "efgh":
for chunk in chunker(s):
print ''.join(chunk)
if chunker.chunk: # is there anything left?
print ''.join(chunker.chunk)
Output:
abc
def
gh
I am not sure, but it seems you want to do what is called a moving average. numpy provides facilities for this (the convolve function).
>>> x = numpy.array(range(20))
>>> x
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19])
>>> n = 2 # moving average window
>>> numpy.convolve(numpy.ones(n)/n, x)[n-1:-n+1]
array([ 0.5, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5,
9.5, 10.5, 11.5, 12.5, 13.5, 14.5, 15.5, 16.5, 17.5, 18.5])
The nice thing is that it accomodates different weighting schemes nicely (just change numpy.ones(n) / n to something else).
You can find a complete material here:
http://www.scipy.org/Cookbook/SignalSmooth
Expanding on the answer of #Ants Aasma: In Python 3.7 the handling of the StopIteration exception changed (according to PEP-479). A compatible version would be:
from itertools import chain, islice
def ichunked(seq, chunksize):
it = iter(seq)
while True:
try:
yield chain([next(it)], islice(it, chunksize - 1))
except StopIteration:
return
Your question could use some more detail, but how about:
def iterate_over_slices(the_list, slice_size):
for start in range(0, len(the_list)-slice_size):
slice = the_list[start:start+slice_size]
foo(slice)
For a near-one liner (after itertools import) in the vein of Nadia's answer dealing with non-chunk divisible sizes without padding:
>>> import itertools as itt
>>> chunksize = 5
>>> myseq = range(18)
>>> cnt = itt.count()
>>> print [ tuple(grp) for k,grp in itt.groupby(myseq, key=lambda x: cnt.next()//chunksize%2)]
[(0, 1, 2, 3, 4), (5, 6, 7, 8, 9), (10, 11, 12, 13, 14), (15, 16, 17)]
If you want, you can get rid of the itertools.count() requirement using enumerate(), with a rather uglier:
[ [e[1] for e in grp] for k,grp in itt.groupby(enumerate(myseq), key=lambda x: x[0]//chunksize%2) ]
(In this example the enumerate() would be superfluous, but not all sequences are neat ranges like this, obviously)
Nowhere near as neat as some other answers, but useful in a pinch, especially if already importing itertools.
A function that slices a list or an iterator into chunks of a given size. Also handles the case correctly if the last chunk is smaller:
def slice_iterator(data, slice_len):
it = iter(data)
while True:
items = []
for index in range(slice_len):
try:
item = next(it)
except StopIteration:
if items == []:
return # we are done
else:
break # exits the "for" loop
items.append(item)
yield items
Usage example:
for slice in slice_iterator([1,2,3,4,5,6,7,8,9,10],3):
print(slice)
Result:
[1, 2, 3]
[4, 5, 6]
[7, 8, 9]
[10]