I have a function which takes two parameters and perform a binary operation:
def foo(arg1,arg2):
return operation(arg1,arg2)
I need to generalize this function such that if three args are passed it returns operation(arg1,operation(arg2,arg3)), if four are provided operation(arg1,operation(arg2,operation(arg3,arg4))) and so on. Is it possible to do that in python?
You can do this using the *args form of declaring a function; check if the length of the arguments is 2 and if so return the value of the operation, otherwise return the value of the operation of the first argument with foo of the remaining arguments:
def operation(arg1, arg2):
return arg1 + arg2
def foo(*args):
if (len(args) == 2):
return operation(*args)
return operation(args[0], foo(*args[1:]))
print(foo(1, 3))
print(foo(2, 3, 5))
print(foo(1, 2, 3, 4, 5, 6, 7))
Output:
4
10
28
Note you may also want to check if 0 or 1 arguments are passed to prevent "index out of range" errors. For 1 argument you could just return the input value e.g.
if (len(args) == 1):
return args[0]
As pointed out by #wallefan in the comments, there is a standard library function for this: functools.reduce. You can use that like this:
from functools import reduce
print(reduce(operation, (1, 3)))
print(reduce(operation, (2, 3, 5)))
print(reduce(operation, (1, 2, 3, 4, 5, 6, 7)))
The output is the same as the foo function above.
Yes, and in fact it's built into the standard library: https://docs.python.org/3/library/functools.html#functools.reduce
import functools
def operation(a,b):
return a + b
# returns 15
functools.reduce(operation, (1, 2, 3, 4, 5))
If you'd like, you can combine this with varargs mentioned in Nick's answer:
import functools
def operation(a,b):
return a + b
def foo(*args):
return functools.reduce(operation, args)
# returns 15
foo(1,2,3,4,5)
Related
I have a question about passing arguments in a function.
In python when you want to unpack an iterable obj on the right-hand side, you have to do it within some context, like a tuple or a set or etc.
For example, you can not say:
a, b = *(1, 2)
and you should say:
a, b = (*(1, 2),) // a=1, b=2
or you can say:
a, b = {*(1, 2)} //a=1, b=2 or a=2, b=1
Am I rigth?
But when you want to unpack iterable and then pass it as arguments in a function, you do not need any context at all and you just unpack your iterable object.
For example:
def f(param1, param2):
pass
f(*(1, 2))
and you do not need to use some kind of context like before. For example, you do not say:
f({*(1, 2)}) // it will be f({1, 2})
I think we don't use {} or any other context in this case because we were looking for pass 2 values as arguments in our f function. Thus, I assume we have to say f(*(1, 2)) not f({*(1, 2)}) .
If I was right, Could you please explain more about how f(*(1, 2)) is worked without using any context under the hood?
f(*(1, 2)) does have context! The context is the list (and dictionary) of function arguments itself. In essence, in a function call f(...) the stuff between the parentheses can be seen as a hybrid between a tuple and a dictionary literal (only using x=4 instead of 'x': 4), which then gets passed to the function:
>>> def f(*args, **kwargs):
... print(args, kwargs)
>>> l = [10, 11]; d = {'a': 12, 'b': 13}
>>> f(1, *l, 2, 3, x=4, y=5, **d, z=6)
(1, 10, 11, 2, 3) {'x': 4, 'y': 5, 'a': 12, 'b': 13, 'z': 6}
Viewed like this it makes perfect sense that you can unpack sequences and dictionaries into this context.
Recently, I have studied 'Programming language' using standard ML, and I've learned currying method(or something), so I applied it in Python.
The below is simple function and currying.
def range_new(x, y):
return [i for i in range(x, y+1)]
def curry_2(f):
return lambda x: lambda y: f(x, y)
def uncurry_2(f):
pass # I don't know it...
print(range_new(1, 10))
curried_range = curry_2(range_new)
countup = curried_range(1)
print(countup(10))
print(curried_range(1)(10))
The result is below. And it works well; with curry_2 we can make a new function(countup). But, then I want to make an uncurried function.
However, I don't know how I can make it.
How can I do it?
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
The easiest solution is to wrap the curried function again with code that uncurries it:
def uncurry_2(f):
return lambda x, y: f(x)(y)
uncurried_range = uncurry_2(curried_range)
print(uncurried_range(1, 10))
It's not exactly good style but you can access the variables in the closure using the (maybe CPython-only) __closure__ attribute of the returned lambda:
>>> countup.__closure__[0].cell_contents
<function __main__.range_new>
This accesses the content of the innermost closure (the variable used in the innermost lambda) of your function curry_2 and thus returns the function you used there.
However in production code you shouldn't use that. It would be better to create a class (or function) for currying that supports accessing the uncurried function (which is something lambda does not provide). However some functools in Python support accessing the "decorated" function, for example partial:
>>> from functools import partial
>>> countup = partial(range_new, 1)
>>> print(countup(10))
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> countup.func
<function __main__.range_new>
I believe by uncurry you mean you'd like to allow the function to accept more arguments. Have you considered using the "partial" function? It allows you to use as many arguments as desired when calling the method.
from functools import partial
def f(a, b, c, d):
print(a, b, c, d)
g = partial(partial(f, 1, 2), 3)
g(4)
Implementing it should be pretty straight forward
def partial(fn, *args):
def new_func(*args2):
newArgs = args + args2
fn(*newArgs)
return new_func;
Note both the code presented in the original question, and the code above is known as partial application. Currying is more flexible than this typically - here's how you can do it with Python 3 (it is more tricky in Python 2).
def curry(fn, *args1):
current_args = args1
sig = signature(fn)
def new_fn(*args2):
nonlocal current_args
current_args += args2
if len(sig.parameters) > len(current_args):
return new_fn
else:
return fn(*current_args)
return new_fn
j = curry(f)
j(1)(2, 3)(4)
Now back to your code. range_new can now be used in a few new ways:
print(range_new(1, 10))
curried_range = curry(range_new)
countup = curried_range(1)
print(countup(10))
countup_again = curried_range
print(countup_again(1, 10))
Function _mget_search definition will be as,
def _mget_search(query, with_scores=False, count=False, record_term=True,
**kwargs):
This function called with keyword arguments.
_mget_search(**search_kwargs)
Function parameter search_kwargs contains as,
{'count': False, 'min_score': 20, 'end': 49, 'with_scores': False, 'start': 0, 'query': u'ouuuuu', 'max_score': inf, 'record_term': True}
I am getting this error -
_mget_search() got multiple values for keyword argument 'query'
I am unable to understand why it is happening and how to correct it.
**kwargs allows you to pass keyworded variable length of arguments to a function. You should use **kwargs if you want to handle named arguments in a function. Here is an example to get you going with it:
def greet_me(**kwargs):
if kwargs is not None:
for key, value in kwargs.iteritems():
print "%s == %s" %(key,value)
>>> greet_me(name="yasoob")
name == yasoob
It's because you are unpacking arguments you are trying to pass. Try to use:
_mget_search(search_kwargs)
EDIT
Let's dive more into this problem. We'll define two functions that and see how they behave when passing various arguments.
In [1]: def fun1(a, b, c):
...: print('Passed arguments: a={} b={} c={}'.format(a, b, c))
...:
In [2]: def fun2(*args):
...: print('Passed args: {}'.format(args))
...:
In [3]: fun1(1, 2, 3)
Passed arguments: a=1 b=2 c=3
In [4]: fun2(1, 2, 3)
Passed args: (1, 2, 3)
In [5]: fun2([1, 2, 3])
Passed args: ([1, 2, 3],)
In [6]: fun2(*[1, 2, 3])
Passed args: (1, 2, 3)
In [7]: fun1(*[1, 2, 3])
Passed arguments: a=1 b=2 c=3
In 3rd call we passed 3 arguments separately which is the same as if we used 7th call where an unpacked list was called. Compare it with situations when fun2 was called.
I believe the issue is because query is defined as a positional arg, so when the dictionary you pass with ** is unpacked, the first entry (regardless of its name) is being used for query, then query also appears in the dictionary, hence the multiple values for query.
Try fixing by making query a named arg:
def _mget_search(query='', with_scores=False, count=False, record_term=True, **kwargs):
Alternatively, don't include query in the dictionary when you pass it.
I would bet that something somewhere is also passing a parameter to that function, in addition to the search_kwargs. If it's part of a callback function, there's a good chance the callback is calling it with some parameters and those are getting assigned to the first parameter query.
The only way that I can see you getting that error is if you called your function with a parameter and kwargs, like so:
assuming:
def _mget_search(query, with_scores=False, count=False, record_term=True,
**kwargs):
then the way to get that error message is:
>>> search_kwargs= {'query': u'ouuuuu', 'count': False, 'min_score': 20, 'end': 49, 'with_scores': False, 'start': 0, 'max_score': inf, 'record_term': True}
>>> _mget_search(some_random_parameter, **search_kwargs)
Here's what I used to try and recreate the error.
def func(a, b=2, **kwargs):
return '{}{}{}'.format(a, b, kwargs)
Only the last one succeeded in getting that error.
>>> func(**{'a':2, 'b': 3, 'other': 3})
"23{'other': 3}"
>>> func(7, **{'stuff': 5})
"72{'stuff': 5}"
>>> func(2, **{'b': 7})
'27{}'
>>> func(**{'b': 3, 'other': 3, 'a': 5})
"53{'other': 3}"
>>> func(2, **{'a': 5})
TypeError: func() got multiple values for argument 'a'
def sub3(n):
return n - 3
def square(n):
return n * n
It's easy to compose functions in Python:
>>> my_list
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [square(sub3(n)) for n in my_list]
[9, 4, 1, 0, 1, 4, 9, 16, 25, 36]
Unfortunately, to use the composition as a key it's awkward, you have to use them in another function which calls both functions in turn:
>>> sorted(my_list, key=lambda n: square(sub3(n)))
[3, 2, 4, 1, 5, 0, 6, 7, 8, 9]
This should really just be sorted(my_list, key=square*sub3), because heck, function __mul__ isn't used for anything else anyway:
>>> square * sub3
TypeError: unsupported operand type(s) for *: 'function' and 'function'
Well let's just define it then!
>>> type(sub3).__mul__ = 'something'
TypeError: can't set attributes of built-in/extension type 'function'
D'oh!
>>> class ComposableFunction(types.FunctionType):
... pass
...
TypeError: Error when calling the metaclass bases
type 'function' is not an acceptable base type
D'oh!
class Hack(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
return self.function(*args, **kwargs)
def __mul__(self, other):
def hack(*args, **kwargs):
return self.function(other(*args, **kwargs))
return Hack(hack)
Hey, now we're getting somewhere..
>>> square = Hack(square)
>>> sub3 = Hack(sub3)
>>> [square(sub3(n)) for n in my_list]
[9, 4, 1, 0, 1, 4, 9, 16, 25, 36]
>>> [(square*sub3)(n) for n in my_list]
[9, 4, 1, 0, 1, 4, 9, 16, 25, 36]
>>> sorted(my_list, key=square*sub3)
[3, 2, 4, 1, 5, 0, 6, 7, 8, 9]
But I don't want a Hack callable class! The scoping rules are different in ways I don't fully understand, and it's arguably even uglier than just using the "lameda". Is it possible to get composition working directly with functions somehow?
You can use your hack class as a decorator pretty much as it's written, though you'd likely want to choose a more appropriate name for the class.
Like this:
class Composable(object):
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
return self.function(*args, **kwargs)
def __mul__(self, other):
#Composable
def composed(*args, **kwargs):
return self.function(other(*args, **kwargs))
return composed
def __rmul__(self, other):
#Composable
def composed(*args, **kwargs):
return other(self.function(*args, **kwargs))
return composed
You can then decorate your functions like so:
#Composable
def sub3(n):
return n - 3
#Composable
def square(n):
return n * n
And compose them like so:
(square * sub3)(n)
Basically it's the same thing you've accomplished using your hack class, but using it as a decorator.
Python does not (and likely will never) have support for function composition either at the syntactic level or as a standard library function. There are various 3rd party modules (such as functional) that provide a higher-order function that implements function composition.
Maybe something like this:
class Composition(object):
def __init__(self, *args):
self.functions = args
def __call__(self, arg):
result = arg
for f in reversed(self.functions):
result = f(result)
return result
And then:
sorted(my_list, key=Composition(square, sub3))
You can compose functions using SSPipe library:
from sspipe import p, px
sub3 = px - 3
square = px * px
composed = sub3 | square
print(5 | composed)
I have two class instances which have list attributes, e.g.:
foo.range = [1,2]
bar.range = [3,4]
I also have a function which takes multiple arguments:
def permutations(*args):
return list(itertools.product(arg.range for arg in args))
I want permutations(foo, bar) to return all permutations of those two (or more) lists (i.e. [(1,3), (1,4) …]) but I actually get [([1, 2],), ([3, 4],)]
Could someone please help me to understand where am I going wrong and how to achieve the result I'm hoping for?
itertools.product expects the iterables as individual arguments(itertools.product(*iterables[, repeat])), so you need to unpack the items to it using the the splat(*) operator:
>>> def permutations(*args):
return list(itertools.product(*(arg.range for arg in args)))
>>> permutations(foo, bar)
[(1, 3), (1, 4), (2, 3), (2, 4)]
Consider this simple function, it will collect all positional arguments passed to it in args:
>>> def func(*args):
print args
print list(args[0])
...
When we pass it a generator expression it is collected as one individual item in the args tuple:
>>> func(x for x in range(5))
(<generator object <genexpr> at 0x7fda2ab4e780>,)
[0, 1, 2, 3, 4]
To fix this we need to unpack our generator expression, also as somethine like *x for x in range(5) is not a valid syntax in Python, we need to add extra parenthesis around the generator expression:
>>> func(*(x for x in range(5)))
(0, 1, 2, 3, 4)
# Failed as expected for args[0]
Traceback (most recent call last):
File "<ipython-input-215-13af4340dad1>", line 1, in <module>
func(*(x for x in range(5)))
File "<ipython-input-213-08941e2a3c06>", line 3, in func
print list(args[0])
TypeError: 'int' object is not iterable