Related
Making my first steps in Python.
I have a list of lists and i'm trying to return the sub list with largest sum of all sub lists.
For now I just have the max sum itself.
Ex: this code returns 18, but I need to return [3,3,3,3,3,3]
Any directions?
Thanks
def find_biggest(lst):
inner_list_sum = []
for i in range(len(lst)):
inner_list_sum.append(sum(lst[i])) # list of elements sums
return max(inner_list_sum) # I actually need the element itself...not the max sum
print(find_biggest([[1,2,3,4], [1,2,3,3], [1,1], [3,3,3,3,3,3]]))
Use max with key=sum
Ex:
data = [[1,2,3,4], [1,2,3,3], [1,1], [3,3,3,3,3,3]]
print(max(data, key=sum))
Output:
[3, 3, 3, 3, 3, 3]
OK
Since I'm not allowed to use (max(data, key=sum)).
I did this not very elegant code, but it was accepted a correct answer
def find_biggest(lst):
inner_list_sum = []
for i in range(len(lst)):
inner_list_sum.append(sum(lst[i])) # list of elements sums
max_element=max(inner_list_sum)
seq_index= inner_lis
import functools
def find_biggest(lst):
return functools.reduce(lambda x, y : x if sum(x) > sum(y) else y, lst)
Here, the used lambda expression is the function equivalent to find the greatest sum and lst is iterable.
Please note: reduce will directly work for python2, you need to import functools
for python3only.
Using functools.reduce
I was wondering if map can be used at all to sum the elements of a list.
assume a = [1, 2, 3, 4]
list(map(sum, a)) will give an error that int object is not iterable because list wants iterables.
map(sum, a) is a valid statement but given the object, I do not see an easy way to dereference it.
[map(sum, a)] will return an object inside the list
this answer states that it should be easy. What am I missing here?
map applies a function to every element in the list. Instead, you can use reduce:
a = [1, 2, 3, 4]
sum_a = reduce(lambda x, y:x+y, a)
In this case, purely sum can be used, however, to be more functional, reduce is a better option.
Or, in Python3:
from functools import reduce
a = [1, 2, 3, 4]
sum_a = reduce(lambda x, y:x+y, a)
x = list(map(sum,a))
Is equivalent to
x = []
for i in a:
x.append(sum(i))
Sum needs a iterable to apply sum across. If you see the docs syntax goes this way sum(iterable[, start]). Since int is not an iterable you get that error.
Of course if one just want to sum the elements of a list, he should simply call sum(list_).
Now, comming to your question: map, both the Python built-in and the pattern refer to applying a function to a data sequence, and yield another sequence, with a separate result for each element in the initial sequence.
sum does not do that - it yields a single result for the whole sequence. That pattern is called reduce, and so is the Python ex-built-in to do that. In Python 3, it was "demoted" to the functools module, as it is rarely used when compared to the map pattern.
The sum built-in itself employs the "reduce" pattern alone - but if you were to explicitly recreate sum using the reduce pattern it goes like:
from functools import reduce
a = [1, 2, 3, 4]
reduce(lambda result, value: result + value, a, 0)
The first parameter is a callable that takes the "accumulated result so far", the second value is the sequence of items you want to run reduce at, and the third parameter is the initial value to be passed as the accumulated result. (so,it starts at zero). For a multiplicatory, we could use:
reduce(lambda result, value: result * value, a, 1)
update: Python 3.8 implemented the "multiplicatory" in the standard library as math.prod.
The error int object is not iterable is not because list expects an iterable, but sum expected an iterable.
The following code:
map(sum , [1,2,3,4])
Is somewhat equivalent to:
[sum(x) for x in [1,2,3,4]]
Executing the last expression yields the same error.
reduce(lambda x,y:x+y, L) #summing all elements of a list L
Using map reduce and printing the elapsed time in seconds
import time
from six.moves import reduce
import numpy as np
start=time.time()
L = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
result = reduce(lambda x,y:x+y, L)
end=time.time()
print("sum of list L ", L, " is equal to", result)
print("elapsed time is ", end-start, ' seconds')
output:
sum of list L [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] is equal to 55
elapsed time is 0.00014519691467285156 seconds
using python's build-in sum function and elapsed time
start=time.time()
s = sum([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
end=time.time()
print("elapsed time is ", end-start, ' seconds')
output:
elapsed time is 9.226799011230469e-05 seconds
sum is a slightly faster method since 9e-05 is less than 1e-04
Here's one way to do it purely functionally.
from operator import add
from functools import reduce
result = reduce(add, a)
Indirectly you can add all the elements of a list using a map function using a global variable like below:
# reading the file
with open('numbers.txt') as f:
lines = [line.strip() for line in f]
numbers = [int(line) for line in lines]
all_sum = 0
def add(n):
global all_sum
all_sum += n
return(all_sum)
result = map(add, numbers)
print(list(result)[-1])
There is only one number in one line in the text file.
I am trying to use itertools.product to manage the bookkeeping of some nested for loops, where the number of nested loops is not known in advance. Below is a specific example where I have chosen two nested for loops; the choice of two is only for clarity, what I need is a solution that works for an arbitrary number of loops.
This question provides an extension/generalization of the question appearing here:
Efficient algorithm for evaluating a 1-d array of functions on a same-length 1d numpy array
Now I am extending the above technique using an itertools trick I learned here:
Iterating over an unknown number of nested loops in python
Preamble:
from itertools import product
def trivial_functional(i, j): return lambda x : (i+j)*x
idx1 = [1, 2, 3, 4]
idx2 = [5, 6, 7]
joint = [idx1, idx2]
func_table = []
for items in product(*joint):
f = trivial_functional(*items)
func_table.append(f)
At the end of the above itertools loop, I have a 12-element, 1-d array of functions, func_table, each element having been built from the trivial_functional.
Question:
Suppose I am given a pair of integers, (i_1, i_2), where these integers are to be interpreted as the indices of idx1 and idx2, respectively. How can I use itertools.product to determine the correct corresponding element of the func_table array?
I know how to hack the answer by writing my own function that mimics the itertools.product bookkeeping, but surely there is a built-in feature of itertools.product that is intended for exactly this purpose?
I don't know of a way of calculating the flat index other than doing it yourself. Fortunately this isn't that difficult:
def product_flat_index(factors, indices):
if len(factors) == 1: return indices[0]
else: return indices[0] * len(factors[0]) + product_flat_index(factors[1:], indices[1:])
>> product_flat_index(joint, (2, 1))
9
An alternative approach is to store the results in a nested array in the first place, making translation unnecessary, though this is more complex:
from functools import reduce
from operator import getitem, setitem, itemgetter
def get_items(container, indices):
return reduce(getitem, indices, container)
def set_items(container, indices, value):
c = reduce(getitem, indices[:-1], container)
setitem(c, indices[-1], value)
def initialize_table(lengths):
if len(lengths) == 1: return [0] * lengths[0]
subtable = initialize_table(lengths[1:])
return [subtable[:] for _ in range(lengths[0])]
func_table = initialize_table(list(map(len, joint)))
for items in product(*map(enumerate, joint)):
f = trivial_functional(*map(itemgetter(1), items))
set_items(func_table, list(map(itemgetter(0), items)), f)
>>> get_items(func_table, (2, 1)) # same as func_table[2][1]
<function>
So numerous answers were quite useful, thanks to everyone for the solutions.
It turns out that if I recast the problem slightly with Numpy, I can accomplish the same bookkeeping, and solve the problem I was trying to solve with vastly improved speed relative to pure python solutions. The trick is just to use Numpy's reshape method together with the normal multi-dimensional array indexing syntax.
Here's how this works. We just convert func_table into a Numpy array, and reshape it:
func_table = np.array(func_table)
component_dimensions = [len(idx1), len(idx2)]
func_table = np.array(func_table).reshape(component_dimensions)
Now func_table can be used to return the correct function not just for a single 2d point, but for a full array of 2d points:
dim1_pts = [3,1,2,1,3,3,1,3,0]
dim2_pts = [0,1,2,1,2,0,1,2,1]
func_array = func_table[dim1_pts, dim2_pts]
As usual, Numpy to the rescue!
This is a little messy, but here you go:
from itertools import product
def trivial_functional(i, j): return lambda x : (i+j)*x
idx1 = [1, 2, 3, 4]
idx2 = [5, 6, 7]
joint = [enumerate(idx1), enumerate(idx2)]
func_map = {}
for indexes, items in map(lambda x: zip(*x), product(*joint)):
f = trivial_functional(*items)
func_map[indexes] = f
print(func_map[(2, 0)](5)) # 40 = (3+5)*5
I'd suggest using enumerate() in the right place:
from itertools import product
def trivial_functional(i, j): return lambda x : (i+j)*x
idx1 = [1, 2, 3, 4]
idx2 = [5, 6, 7]
joint = [idx1, idx2]
func_table = []
for items in product(*joint):
f = trivial_functional(*items)
func_table.append(f)
From what I understood from your comments and your code, func_table is simply indexed by the occurence of a certain input in the sequence. You can access it back again using:
for index, items in enumerate(product(*joint)):
# because of the append(), index is now the
# position of the function created from the
# respective tuple in join()
func_table[index](some_value)
Let's say we have a function add as follows
def add(x, y):
return x + y
we want to apply map function for an array
map(add, [1, 2, 3], 2)
The semantics are I want to add 2 to every element of the array. But the map function requires a list in the third argument as well.
Note: I am putting the add example for simplicity. My original function is much more complicated. And of course option of setting the default value of y in add function is out of question as it will be changed for every call.
One option is a list comprehension:
[add(x, 2) for x in [1, 2, 3]]
More options:
a = [1, 2, 3]
import functools
map(functools.partial(add, y=2), a)
import itertools
map(add, a, itertools.repeat(2, len(a)))
The docs explicitly suggest this is the main use for itertools.repeat:
Make an iterator that returns object over and over again. Runs indefinitely unless the times argument is specified. Used as argument to map() for invariant parameters to the called function. Also used with zip() to create an invariant part of a tuple record.
And there's no reason for pass len([1,2,3]) as the times argument; map stops as soon as the first iterable is consumed, so an infinite iterable is perfectly fine:
>>> from operator import add
>>> from itertools import repeat
>>> list(map(add, [1,2,3], repeat(4)))
[5, 6, 7]
In fact, this is equivalent to the example for repeat in the docs:
>>> list(map(pow, range(10), repeat(2)))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
This makes for a nice lazy-functional-language-y solution that's also perfectly readable in Python-iterator terms.
Use a list comprehension.
[x + 2 for x in [1, 2, 3]]
If you really, really, really want to use map, give it an anonymous function as the first argument:
map(lambda x: x + 2, [1,2,3])
Map can contain multiple arguments, the standard way is
map(add, a, b)
In your question, it should be
map(add, a, [2]*len(a))
The correct answer is simpler than you think.
Simply do:
map(add, [(x, 2) for x in [1,2,3]])
And change the implementation of add to take a tuple i.e
def add(t):
x, y = t
return x+y
This can handle any complicated use case where both add parameters are dynamic.
Sometimes I resolved similar situations (such as using pandas.apply method) using closures
In order to use them, you define a function which dynamically defines and returns a wrapper for your function, effectively making one of the parameters a constant.
Something like this:
def add(x, y):
return x + y
def add_constant(y):
def f(x):
return add(x, y)
return f
Then, add_constant(y) returns a function which can be used to add y to any given value:
>>> add_constant(2)(3)
5
Which allows you to use it in any situation where parameters are given one at a time:
>>> map(add_constant(2), [1,2,3])
[3, 4, 5]
edit
If you do not want to have to write the closure function somewhere else, you always have the possibility to build it on the fly using a lambda function:
>>> map(lambda x: add(x, 2), [1, 2, 3])
[3, 4, 5]
If you have it available, I would consider using numpy. It's very fast for these types of operations:
>>> import numpy
>>> numpy.array([1,2,3]) + 2
array([3, 4, 5])
This is assuming your real application is doing mathematical operations (that can be vectorized).
If you really really need to use map function (like my class assignment here...), you could use a wrapper function with 1 argument, passing the rest to the original one in its body; i.e. :
extraArguments = value
def myFunc(arg):
# call the target function
return Func(arg, extraArguments)
map(myFunc, itterable)
Dirty & ugly, still does the trick
I believe starmap is what you need:
from itertools import starmap
def test(x, y, z):
return x + y + z
list(starmap(test, [(1, 2, 3), (4, 5, 6)]))
def func(a, b, c, d):
return a + b * c % d
map(lambda x: func(*x), [[1,2,3,4], [5,6,7,8]])
By wrapping the function call with a lambda and using the star unpack, you can do map with arbitrary number of arguments.
You can include lambda along with map:
list(map(lambda a: a+2, [1, 2, 3]))
To pass multiple arguments to a map function.
def q(x,y):
return x*y
print map (q,range(0,10),range(10,20))
Here q is function with multiple argument that map() calls.
Make sure, the length of both the ranges i.e.
len (range(a,a')) and len (range(b,b')) are equal.
In :nums = [1, 2, 3]
In :map(add, nums, [2]*len(nums))
Out:[3, 4, 5]
Another option is:
results = []
for x in [1,2,3]:
z = add(x,2)
...
results += [f(z,x,y)]
This format is very useful when calling multiple functions.
#multi argument
def joke(r):
if len(r)==2:
x, y = r
return x + y
elif len(r)==3:
x,y,z=r
return x+y+z
#using map
print(list(map(joke,[[2,3],[3,4,5]])))
output = [6,12]
if the case like above and just want use function
def add(x,y):
ar =[]
for xx in x:
ar.append(xx+y)
return ar
print(list(map(add,[[3,2,4]],[2]))[0])
output = [5,4,6]
Note: you can modified as you want.
What is the best way to touch two following values in an numpy array?
example:
npdata = np.array([13,15,20,25])
for i in range( len(npdata) ):
print npdata[i] - npdata[i+1]
this looks really messed up and additionally needs exception code for the last iteration of the loop.
any ideas?
Thanks!
numpy provides a function diff for this basic use case
>>> import numpy
>>> x = numpy.array([1, 2, 4, 7, 0])
>>> numpy.diff(x)
array([ 1, 2, 3, -7])
Your snippet computes something closer to -numpy.diff(x).
How about range(len(npdata) - 1) ?
Here's code (using a simple array, but it doesn't matter):
>>> ar = [1, 2, 3, 4, 5]
>>> for i in range(len(ar) - 1):
... print ar[i] + ar[i + 1]
...
3
5
7
9
As you can see it successfully prints the sums of all consecutive pairs in the array, without any exceptions for the last iteration.
You can use ediff1d to get differences of consecutive elements. More generally, a[1:] - a[:-1] will give the differences of consecutive elements and can be used with other operators as well.