I am trying to create a method (sum) that takes a variable number of vectors and adds them in. For educational purposes, I have written my own Vector class, and the underlying data is stored in an instance variable named data.
My code for the #classmethod sum works (for each of the vectors passed in, loop through each element in the data variable and add it to a result list), but it seems non-Pythonic, and wondering if there is a better way?
class Vector(object):
def __init__(self, data):
self.data = data
#classmethod
def sum(cls, *args):
result = [0 for _ in range(len(args[0].data))]
for v in args:
if len(v.data) != len(result): raise
for i, element in enumerate(v.data):
result[i] += element
return cls(result)
itertools.izip_longest may come very handy in your situation:
a = [1, 2, 3, 4]
b = [1, 2, 3, 4, 5, 6]
c = [1, 2]
lists = (a, b, c)
result = [sum(el) for el in itertools.izip_longest(*lists, fillvalue=0)]
And here you got what you wanted:
>>> result
[3, 6, 6, 8, 5, 6]
What it does is simply zips up your lists together, by filling empty value with 0. e.g. izip_longest(a, b) would be [(1, 1), (2, 2), (3, 0), (4, 0)]. Then just sums up all the values in each tuple element of the intermediate list.
So here you go step by step:
>>> lists
([1, 2, 3, 4], [1, 2, 3, 4, 5, 6], [1, 2])
>>> list(itertools.izip_longest(*lists, fillvalue=0))
[(1, 1, 1), (2, 2, 2), (3, 3, 0), (4, 4, 0), (0, 5, 0), (0, 6, 0)]
So if you run a list comprehension, summing up all sub-elements, you get your result.
Another thing that you could do (and that might be more "pythonic") would be to implement the __add__ magic method, so you can use + and sum directly on vectors.
class Vector(object):
def __init__(self, data):
self.data = data
def __add__(self, other):
if isinstance(other, Vector):
return Vector([s + o for s, o in zip(self.data, other.data)])
if isinstance(other, int):
return Vector([s + other for s in self.data])
raise TypeError("can not add %s to vector" % other)
def __radd__(self, other):
return self.__add__(other)
def __repr__(self):
return "Vector(%r)" % self.data
Here, I also implemented addition of Vector and int, adding the number on each of the Vector's data elements, and the "reverse addition" __radd__, to make sum work properly.
Example:
>>> v1 = Vector([1,2,3])
>>> v2 = Vector([4,5,6])
>>> v3 = Vector([7,8,9])
>>> v1 + v2 + v3
Vector([12, 15, 18])
>>> sum([v1,v2,v3])
Vector([12, 15, 18])
args = [[1, 2, 3],
[10, 20, 30],
[7, 3, 15]]
result = [sum(data) for data in zip(*args)]
# [18, 25, 48]
Is this what you want?
Related
Good day everyone, I need to get all possible combination from the list with parameters.
for example, we can have a list with parameters, where:
the parameter has only value, in this case, this parameter will not be changed
the parameter has a lower and high limit, in this case, the parameter will be changed from the lower limit to high limit wit some step
here is my a code
from typing import List
class Parameter:
def __init__ (self, value = None, lowLimit = None, hightLimit = None, step = None):
self.value = value
self.lowLimit = lowLimit
self.hightLimit = hightLimit
self.step = step
data: List[Parameter] = [Parameter(1), Parameter(2, 2, 3, 1), Parameter(3, 3, 4, 1)]
def recursion(array: List[Parameter], skip: int):
for index, value in enumerate(array):
if index < skip: continue
if value.lowLimit is not None and value.hightLimit is not None:
temp = [it.value for it in array]
init = value.lowLimit
while init <= value.hightLimit :
temp[index] = init
init += value.step
yield temp
def combination(array: List[Parameter]):
for index, value in enumerate(array):
if value.lowLimit is None and value.hightLimit is None : continue
for d in recursion(array, index):
yield d
for d in combination(data):
print(d)
and this what I would like to get:
[1, 2, 3]
[1, 2, 4]
[1, 2, 5]
[1, 3, 3]
[1, 2, 3]
[1, 3, 3]
but my code does not work, and I need some help to implement this task. Could someone help, please?
Not sure I really understand what you are asking for, but I think you can convert your Parameters to ranges and then get the itertools.product of those:
import itertools
data = [Parameter(1), Parameter(2, 2, 3, 1), Parameter(3, 3, 5, 2)]
ranges = [[p.value] if p.lowLimit is None else
range(p.lowLimit, p.hightLimit+1, p.step) for p in data]
print(ranges)
# [[1], range(2, 4), range(3, 6, 2)]
for d in itertools.product(*ranges):
print(d)
# (1, 2, 3), (1, 2, 5), (1, 3, 3), (1, 3, 5)
(Note: Output differs as I slights changed the data to actually use a step != 1)
I have a nested list with unequal length:
[[1,2,3],[4,5],[6,7,8]] and I have a start_index=(i,j) and end_index=(a,b) and I need to print all elements between start_index and end_index. For example if start_index=(1,1) and end_index=(2,2) then I will print (5,6,7,8)
You can use the following function:
def nested_index(arr, start, end):
res = arr[start[0]][start[1]:]
for i in range(start[0] + 1, end[0]):
res.extend(arr[i])
res.extend(arr[end[0]][:end[1] + 1])
return res
>>> print(nested_index([[1,2,3],[4,5],[6,7,8]], (1, 1), (2, 2)))
[5, 6, 7, 8]
This is example from Bratt Slatkin's book
def sort_priority(values, group):
def helper(x):
if x in group:
return (0, x)
return (1, x)
values.sort(key=helper)
Furthermore they gave these values
numbers = [8, 3, 1, 2, 5, 4, 7, 6]
group = {2, 3, 5, 7}
sort_priority(numbers, group)
print(numbers)
And we have
[2, 3, 5, 7, 1, 4, 6, 8]
I do not understand this example.Why do we have return two times and what does helper function actually do?
You read the function as:
def helper(x):
if x in group:
return (0, x)
else:
return (1, x)
Or, more concisely,
def helper(x):
return (x not in group, x)
The intuition behind this is that sort accepts a key callback which is called on each element. For each element, helper is invoked which returns a tuple (could be either (0, x) or (1, x) depending on whether x exists in the VIP list).
You should understand that tuples are sorted based on multiple predicates, meaning both items in the tuples are considered when deciding the order of elements. This would imply that elements for which group returns (0, x) will be ordered first compared to those returning (1, x) because 0 < 1.
After this, we have two groups, those with first element 0 and those with first element 1. All 0 group elements will come first, but the order of those elements depends on the second item in the tuples - x. And similar for 1 group elements.
For your input:
Group0: [2, 3, 5, 7]
Group1: [8, 1, 4, 6]
Ordering within Group0: [2, 3, 5, 7]
Ordering within Group1: [1, 4, 6, 8]
Overall ordering: [Group0, Group1]
Result: [2, 3, 5, 7, 1, 4, 6, 8]
Why do we have return two times?
This has nothing to do with closures or nested functions.
def helper(x):
if x in group:
return (0, x)
return (1, x)
Can be written as
def helper(x):
if x in group:
return (0, x)
else:
return (1, x)
Either way, the return value depends on what the if statement is evaluated to.
If it is True then (0, x) will be returned. If it is False then (1, x) will be returned.
Note that the first return statement is within the if block. In python whenever a function encounters a return statement, the execution is handed back to the caller
In your example, the two returns are just a shortcut way to avoid if else statements. When a particular value is in the group, (0,x) is returned and if the if condition is not satisfied, then (1,x) is returned.
It's a bit easier to understand the code when it's written without nested functions:
def helper(x):
global group
if x in group:
return 0, x
return 1, x
def sort_priority(values):
values.sort(key=helper)
numbers = [8, 3, 1, 2, 5, 4, 7, 6]
group = {2, 3, 5, 7}
sort_priority(numbers)
print(numbers)
Now it's easy to see that sort_priority() simply sorts the values by calling the helper function which creates an order by assigning a value to each x.
When the helper function is called with a value that's in group - it gets "lower" priority (zero) while if the value is not in group, it gets higher priority (one).
A closer look at helper indeed shows:
def helper(x):
global group
if x in group:
return 0, x # <-- we're in the `if` so `x` gets zero
return 1, x # <-- if we got here it means we didn't get into the `if` so `x` gets one
So by using the helper as a key function in the sorting, we'll get and ordered lists which puts the items that are in group first and only then the items that are not in group:
[2, 3, 5, 7, 1, 4, 6, 8]
^
The first item that is not in group
It is more obvious for me to use sorted(values) function instead of values.sort(), otherwise it is little ambiguous "what is returned?", "how actually helper is used?".
def sort_priority(values, group):
def helper(x):
if x in group:
return (0, x)
return (1, x)
sorted_values = sorted(values, key=helper)
return sorted_values
numbers = [8, 3, 1, 2, 5, 4, 7, 6]
group = {2, 3, 5, 7}
print('Sorted Numbers List: ', sort_priority(numbers, group))
Of course after sorted() is used, sorted list returned it explicitly.
I'm trying to figure out how to iterate over an arbitrary number of loops where each loop depends on the most recent outer loop. The following code is an example of what I want to do:
def function(z):
n = int(log(z))
tupes = []
for i_1 in range(1, n):
for i_2 in range(1, i_1):
...
...
...
for i_n in range(1, i_{n - 1}):
if i_1*i_2*...*i_n > z:
tupes.append((i_1, i_2,..., i_n))
return tupes
While I'd like this to work for any z > e**2, it's sufficient for it to work for zs up to e**100. I know that if I take the Cartesian product of the appropriate ranges that I'll end up with a superset of the tuples I desire, but I'd like to obtain only the tuples I seek.
If anyone can help me with this, I'd greatly appreciate it. Thanks in advance.
Combinations can be listed in ascending order; in fact, this is the default behavior of itertools.combinations.
The code:
for i1 in range(1,6):
for i2 in range(1,i1):
for i3 in range(1,i2):
print (i3, i2, i1)
# (1, 2, 3)
# (1, 2, 4)
# ...
# (3, 4, 5)
Is equivalent to the code:
from itertools import combinations
for combination in combinations(range(1,6), 3):
print combination
# (1, 2, 3)
# (1, 2, 4)
# ...
# (3, 4, 5)
Using the combinations instead of the Cartesian product culls the sample space down to what you want.
The logic in your question implemented recursively (note that this allows for duplicate tuples):
import functools
def f(n, z, max_depth, factors=(), depth=0):
res = []
if depth == max_depth:
product = functools.reduce(lambda x, y: x*y, factors, 1)
if product > z:
res.append(factors)
else:
for i in range(1, n):
new_factors = factors + (i,)
res.extend(f(i, z, factors=new_factors, depth=depth+1, max_depth=max_depth))
return res
z = np.e ** 10
n = int(np.log(z))
print(f(n, z, max_depth=8))
yields
[(8, 7, 6, 5, 4, 3, 2, 1),
(9, 7, 6, 5, 4, 3, 2, 1),
(9, 8, 6, 5, 4, 3, 2, 1),
(9, 8, 7, 5, 4, 3, 2, 1),
(9, 8, 7, 6, 4, 3, 2, 1),
(9, 8, 7, 6, 5, 3, 2, 1),
(9, 8, 7, 6, 5, 4, 2, 1),
(9, 8, 7, 6, 5, 4, 3, 1),
(9, 8, 7, 6, 5, 4, 3, 2)]
As zondo suggested, you'll need to use a function and recursion to accomplish this task. Something along the lines of the following should work:
def recurse(tuplesList, potentialTupleAsList, rangeEnd, z):
# No range to iterate over, check if tuple sum is large enough
if rangeEnd = 1 and sum(potentialTupleAsList) > z:
tuplesList.append(tuple(potentialTupeAsList))
return
for i in range(1, rangeEnd):
potentialTupleAsList.append(i)
recurse(tuplesList, potentialTupleAsList, rangeEnd - 1, z)
# Need to remove item you used to make room for new value
potentialTupleAsList.pop(-1)
Then you could call it as such to get the results:
l = []
recurse(l, [], int(log(z)), z)
print l
Your innermost loop can (if reached at all) only go over range(1, 1). Since the endpoint is not included, the loop will not iterate over any values. The shortest implementation of your function is thus:
def function(z):
return []
If you are content with tuples of length smaller than n, then I propose the following solution:
import math
def function(z):
def f(tuples, loop_variables, product, end):
if product > z:
tuples.append(loop_variables)
for i in range(end - 1, 0, -1):
f(tuples, loop_variables + (i,), product * i, i)
n = int(math.log(z))
tuples = []
f(tuples, (), 1, n)
return tuples
The time complexity is not good though: With n nested loops over O(n) elements, we are on the order of n**n steps.
From a previous question I learned something interesting. If Python's itertools.product is fed a series of iterators, these iterators will be converted into tuples before the Cartesian product begins. Related questions look at the source code of itertools.product to conclude that, while no intermediate results are stored in memory, tuple versions of the original iterators are created before the product iteration begins.
Question: Is there a way to create an iterator to a Cartesian product when the (tuple converted) inputs are too large to hold in memory? Trivial example:
import itertools
A = itertools.permutations(xrange(100))
itertools.product(A)
A more practical use case would take in a series of (*iterables[, repeat]) like the original implementation of the function - the above is just an example. It doesn't look like you can use the current implementation of itertools.product, so I welcome in submission in pure python (though you can't beat the C backend of itertools!).
Here's an implementation which calls callables and iterates iterables, which are assumed restartable:
def product(*iterables, **kwargs):
if len(iterables) == 0:
yield ()
else:
iterables = iterables * kwargs.get('repeat', 1)
it = iterables[0]
for item in it() if callable(it) else iter(it):
for items in product(*iterables[1:]):
yield (item, ) + items
Testing:
import itertools
g = product(lambda: itertools.permutations(xrange(100)),
lambda: itertools.permutations(xrange(100)))
print next(g)
print sum(1 for _ in g)
Without "iterator recreation", it may be possible for the first of the factors. But that would save only 1/n space (where n is the number of factors) and add confusion.
So the answer is iterator recreation. A client of the function would have to ensure that the creation of the iterators is pure (no side-effects). Like
def iterProduct(ic):
if not ic:
yield []
return
for i in ic[0]():
for js in iterProduct(ic[1:]):
yield [i] + js
# Test
x3 = lambda: xrange(3)
for i in iterProduct([x3,x3,x3]):
print i
This can't be done with standard Python generators, because some of the iterables must be cycled through multiple times. You have to use some kind of datatype capable of "reiteration." I've created a simple "reiterable" class and a non-recursive product algorithm. product should have more error-checking, but this is at least a first approach. The simple reiterable class...
class PermutationsReiterable(object):
def __init__(self, value):
self.value = value
def __iter__(self):
return itertools.permutations(xrange(self.value))
And product iteslf...
def product(*reiterables, **kwargs):
if not reiterables:
yield ()
return
reiterables *= kwargs.get('repeat', 1)
iterables = [iter(ri) for ri in reiterables]
try:
states = [next(it) for it in iterables]
except StopIteration:
# outer product of zero-length iterable is empty
return
yield tuple(states)
current_index = max_index = len(iterables) - 1
while True:
try:
next_item = next(iterables[current_index])
except StopIteration:
if current_index > 0:
new_iter = iter(reiterables[current_index])
next_item = next(new_iter)
states[current_index] = next_item
iterables[current_index] = new_iter
current_index -= 1
else:
# last iterable has run out; terminate generator
return
else:
states[current_index] = next_item
current_index = max_index
yield tuple(states)
Tested:
>>> pi2 = PermutationsReiterable(2)
>>> list(pi2); list(pi2)
[(0, 1), (1, 0)]
[(0, 1), (1, 0)]
>>> list(product(pi2, repeat=2))
[((0, 1), (0, 1)), ((0, 1), (1, 0)), ((1, 0), (0, 1)), ((1, 0), (1, 0))]
>>> giant_product = product(PermutationsReiterable(100), repeat=5)
>>> len(list(itertools.islice(giant_product, 0, 5)))
5
>>> big_product = product(PermutationsReiterable(10), repeat=2)
>>> list(itertools.islice(big_product, 0, 5))
[((0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)),
((0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 7, 9, 8)),
((0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 8, 7, 9)),
((0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 8, 9, 7)),
((0, 1, 2, 3, 4, 5, 6, 7, 8, 9), (0, 1, 2, 3, 4, 5, 6, 9, 7, 8))]
I'm sorry to up this topic but after spending hours debugging a program trying to iterate over recursively generated cartesian product of generators. I can tell you that none of the solutions above work if not working with constant numbers as in all the examples above.
Correction :
from itertools import tee
def product(*iterables, **kwargs):
if len(iterables) == 0:
yield ()
else:
iterables = iterables * kwargs.get('repeat', 1)
it = iterables[0]
for item in it() if callable(it) else iter(it):
iterables_tee = list(map(tee, iterables[1:]))
iterables[1:] = [it1 for it1, it2 in iterables_tee]
iterable_copy = [it2 for it1, it2 in iterables_tee]
for items in product(*iterable_copy):
yield (item, ) + items
If your generators contain generators, you need to pass a copy to the recursive call.