I am aware that Python reduce only accepts a function with two arguments. However, is there a way to make more than two arguments available to the function? I don't want to make it a global variable because that would be visible for all other imports. The following snippet might help describing the problem (Please read the comments in the snippets):
# The reduce function
def apply_something(something, config):
# Consrtuct a class object based on the truth value of some other variable.
# some_var can be changed and is being accessed in different threads so its
# not safe to make it global. The reduce function is being called from
# inside some other function so It would be better to make
# some_var only accessible in the function context.
if some_var:
obj = Klass(some_var)
else:
obj = Klass()
def callee():
# This is how I apply the reduce operation.
reduce(apply_something, [1, 2, 3], something_initializer)
# I want something like this:
some_var = True # So this can be accessed in apply_something
Please provide some insight into this kind of problem.
I think what you're looking for is partial function application, which you can do using functools.
def apply_something(something, config, some_var):
pass # ...
import functools
functools.reduce(functools.partial(apply_something, some_var=True),
[1, 2, 3], something_initializer)
Example:
>>> def foo(a, b, c):
... return a + b if c else a * b
>>> functools.reduce(functools.partial(foo, c=True), [1,2,3,4,5], 0)
15
>>> functools.reduce(functools.partial(foo, c=False), [1,2,3,4,5], 1)
120
Strictly speaking, the function passed to reduce will always be called with two arguments. However, these types of functions will often access variables in an outer scope. Since I'm unclear on your exact problem, let's implement join in terms of reduce:
def join(joining_string, strings_to_join):
# i.e., join('-', ['a', 'b', 'c']) -> 'a-b-c'
# Here, we need an extra piece of data in our reduce function:
# joining_string.
def do_reduce(a, b):
# Joining string comes from the outer scope:
return a + joining_string + b
return reduce(do_reduce, strings_to_join)
Also, Chris Martin's solution involving partial is perfectly correct as well. Applying it to the above would look like this:
# This didn't *need* to move out of join, but it can. Before, it needed
# to be in join to be able to "capture" the local variable joining_string.
def do_reduce(a, b, joiner):
# Joining string comes from the outer scope:
return a + joiner + b
def join(joining_string, strings_to_join):
# i.e., join('-', ['a', 'b', 'c']) -> 'a-b-c'
# Here, we need an extra piece of data in our reduce function:
# joining_string.
return reduce(
functools.partial(do_reduce, joiner=joining_string),
strings_to_join)
You could as well just use a lambda:
def foo(total, cur, some_var=False):
pass # ...
reduce(lambda total, cur: foo(total, cur, some_var=True), [1, 2, 3], init)
Related
Lets say I have three functions in one module as defined below:
def add_nums(a, b):
return a + b
def sum_and_square_one(add_nums, a, b):
result = add_nums(a,b)
return result*result
def sum_and_square_two(a, b):
result = add_nums(a,b)
return result*result
Both functions sum_and_square_one and sum_and_square_two do the same task. But the former takes add_nums as an argument while the latter calls add_nums inside. My question is which one is the better way. Passing a function as an argument or calling inside a function?
If you always want to call the same function then there is no point in passing it as an argument every time. (this is the case in your example)
If you, however, want to dynamically call different (maybe similar) functions, then passing the appropiate function as an argument would make sense.
Depends on your use case. Do you need the function to change the way it works depending on that argument?
Or, rewriting your meaningless example into a bit less meaningless example (but still quite):
def add_nums(a, b):
return a + b
def do_something_then_square(a, b, what_to_do):
result = what_to_do(a, b)
return result * result
def sum_then_square(a, b):
result = add_nums(a, b)
return result * result
# Then you do:
sum_then_square(2, 5) # 49
do_something_then_square(2, 5, what_to_do=add_nums) # 49
do_something_then_square(2, 5, what_to_do=lambda a, b: a + b) # 49
do_something_then_square(2, 5, what_to_do=lambda a, b: a * b) # 100
do_something_then_square(2, 5, what_to_do=min) # 4
do_something_then_square(2, 5, what_to_do=complex) # (-21+20j)
Only question is: do you need that added flexibility? Otherwise it's just useless additional typing. That question must be answered on a case by case basis; it needs a complete example to give a useful answer.
Is there an equivalent to R's do.call in python?
do.call(what = 'sum', args = list(1:10)) #[1] 55
do.call(what = 'mean', args = list(1:10)) #[1] 5.5
?do.call
# Description
# do.call constructs and executes a function call from a name or a function and a list of arguments to be passed to it.
There is no built-in for this, but it is easy enough to construct an equivalent.
You can look up any object from the built-ins namespace using the __builtin__ (Python 2) or builtins (Python 3) modules then apply arbitrary arguments to that with *args and **kwargs syntax:
try:
# Python 2
import __builtin__ as builtins
except ImportError:
# Python 3
import builtins
def do_call(what, *args, **kwargs):
return getattr(builtins, what)(*args, **kwargs)
do_call('sum', range(1, 11))
Generally speaking, we don't do this in Python. If you must translate strings into function objects, it is generally preferred to build a custom dictionary:
functions = {
'sum': sum,
'mean': lambda v: sum(v) / len(v),
}
then look up functions from that dictionary instead:
functions['sum'](range(1, 11))
This lets you strictly control what names are available to dynamic code, preventing a user from making a nuisance of themselves by calling built-ins for their destructive or disruptive effects.
do.call is pretty much the equivalent of the splat operator in Python:
def mysum(a, b, c):
return sum([a, b, c])
# normal call:
mysum(1, 2, 3)
# with a list of arguments:
mysum(*[1, 2, 3])
Note that I’ve had to define my own sum function since Python’s sum already expects a list as an argument, so your original code would just be
sum(range(1, 11))
R has another peculiarity: do.call internally performs a function lookup of its first argument. This means that it finds the function even if it’s a character string rather than an actual function. The Python equivalent above doesn’t do this — see Martijn’s answer for a solution to this.
Goes similar to previous answer, but why so complicated?
def do_call(what, args=[], kwargs = {}):
return what(*args, **kwargs)
(Which is more elegant than my previously posted definition:)
def do_call(which, args=None, kwargs = None):
if args is None and kwargs is not None:
return which(**kwargs)
elif args is not None and kwargs is None:
return which(*args)
else:
return which(*args, **kwargs)
Python's sum is different than R's sum (1 argument a list expected vs.
arbitraily many arguments expected in R). So we define our own sum (mysum)
which behaves similarly to R's sum. In a similar way we define mymean.
def mysum(*args):
return sum(args)
def mymean(*args):
return sum(args)/len(args)
Now we can recreate your example in Python - as a reasonable 1:1 translation of the R function call.
do_call(what = mymean, args=[1, 2, 3])
## 2.0
do_call(what = mysum, args=[1, 2, 3])
## 6
For functions with argument names, we use a dict for kwargs, where the parameter
names are keys of the dictionary (as strings) and their values the values.
def myfunc(a, b, c):
return a + b + c
do_call(what = myfunc, kwargs={"a": 1, "b": 2, "c": 3})
## 6
# we can even mix named and unnamed parts
do_call(what = myfunc, args = [1, 2], kwargs={"c": 3})
## 6
Is it possible to store a function with predefined arguments to be called by another function?
For example:
def function(num):
print num
trigger=function #store function(1)
trigger() #prints 1
trigger=function #store function(2)
trigger() #prints 2
trigger calls whatever is stored without passing any arguments. I can change trigger but that would require me to rewrite the function that calls the trigger, so I'm wondering if there is a way to store functions along with arguments when storing a function in a variable.
You're looking for functools.partial:
>>> import functools
>>> def foo(number):
... print number
...
>>> bar = functools.partial(foo, 1)
>>> bar()
1
Someone already mentioned functools.partial which is preferred, but you can also use lambda functions without arguments:
trigger1 = lambda: function(1)
trigger2 = lambda: function(2)
Note: As someone mentioned, be careful about defining functions like this in loops or by referencing any value in the lambda body that might change.
You might end up in a situation like this:
a = []
for i in range(5):
a.append(lambda: i)
b = [func() for func in a]
# equals [4, 4, 4, 4, 4]
# to avoid this, do a.append(lambda i=i: i)
I've got some old code where I stored lists of functions in Python as class attributes. These lists are used as a sort of event hook.
To call each function in the list with appropriate arguments, I've used one-liners, mixing map with lambda expressions. I'm now concerned that there is unnecessary overhead in using lambda expressions like this.. I guess the recommended way would be to drop both map and lambda and just use a standard for loop, for readability.
Is there a better (read faster) one-liner to do this, though?
For example:
class Foo:
"""Dummy class demonstrating event hook usage."""
pre = [] # list of functions to call before entering loop.
mid = [] # list of functions to call inside loop, with value
post = [] # list of functions to call after loop.
def __init__(self, verbose=False, send=True):
"""Attach functions when initialising class."""
self._results = []
if verbose:
self.mid.append( self._print )
self.mid.append( self._store )
if send:
self.post.append( self._send )
def __call__(self, values):
# call each function in self.pre (no functions there)
map( lambda fn: fn(), self.pre )
for val in values:
# call each function in self.mid, with one passed argument
map( lambda fn: fn(val), self.mid )
# call each fn in self.post, with no arguments
map( lambda fn: fn(), self.post )
def _print(self, value):
"""Print argument, when verbose=True."""
print value
def _store(self, value):
"""Store results"""
self._results.append(value)
def _send(self):
"""Send results somewhere"""
# create instance of Foo
foo = Foo(verbose=True)
# equivalent to: foo.__call__( ... )
foo( [1, 2, 3, 4] )
Is there a better way to write those one-liner map calls?
The recommended way is definitely to use for loops, however, if you insist on using map, then operator.methodcaller might be just what you need:
>>> def foo(*args):
... print 'foo',args
...
>>> def bar(*args):
... print 'bar',args
...
>>> from operator import methodcaller
>>>
>>> map(methodcaller('__call__',1,2,3),[foo,bar])
foo (1, 2, 3)
bar (1, 2, 3)
[None, None]
A word of caution about using map for this -- It won't work if you port your code to python 3 since map became lazy.
You could also use list comprehensions pretty trivially (and that works on python3 also):
[fn() for fn in self.pre]
[fn(val) for fn in self.mid]
etc.
First of all "I'm concerned that there is unnecessary overhead" is no way to optimise your code. Use a profiler to find the hotspots.
Secondly, your code could do with comments to let the reader know what is going on.
Finally, until proven otherwise, the following is a fine way to accomplish the task:
for func in self.pre: func()
#apply every function in self.mid to every value in values
for func,val in itertools.product(self.mid, values):
func(val)
If you wanted to capture the values, you could use a list comprehension; if you wanted to delay evaluation, you could use a generator expression.
>>> def chain(*fn):
>>> return lambda *args, **kwargs: [_(*args, **kwargs) for _ in fn]
>>>
>>> def add(x, y):
>>> return(x + y)
>>>
>>> def multiply(x, y):
>>> return(x * y)
>>>
>>> chained = chain(add, multiply)
>>> chained(2, 6)
[8, 12]
Say I have a Python function that returns multiple values in a tuple:
def func():
return 1, 2
Is there a nice way to ignore one of the results rather than just assigning to a temporary variable? Say if I was only interested in the first value, is there a better way than this:
x, temp = func()
You can use x = func()[0] to return the first value, x = func()[1] to return the second, and so on.
If you want to get multiple values at a time, use something like x, y = func()[2:4].
One common convention is to use a "_" as a variable name for the elements of the tuple you wish to ignore. For instance:
def f():
return 1, 2, 3
_, _, x = f()
If you're using Python 3, you can you use the star before a variable (on the left side of an assignment) to have it be a list in unpacking.
# Example 1: a is 1 and b is [2, 3]
a, *b = [1, 2, 3]
# Example 2: a is 1, b is [2, 3], and c is 4
a, *b, c = [1, 2, 3, 4]
# Example 3: b is [1, 2] and c is 3
*b, c = [1, 2, 3]
# Example 4: a is 1 and b is []
a, *b = [1]
The common practice is to use the dummy variable _ (single underscore), as many have indicated here before.
However, to avoid collisions with other uses of that variable name (see this response) it might be a better practice to use __ (double underscore) instead as a throwaway variable, as pointed by ncoghlan. E.g.:
x, __ = func()
Remember, when you return more than one item, you're really returning a tuple. So you can do things like this:
def func():
return 1, 2
print func()[0] # prints 1
print func()[1] # prints 2
The best solution probably is to name things instead of returning meaningless tuples (unless there is some logic behind the order of the returned items). You can for example use a dictionary:
def func():
return {'lat': 1, 'lng': 2}
latitude = func()['lat']
You could even use namedtuple if you want to add extra information about what you are returning (it's not just a dictionary, it's a pair of coordinates):
from collections import namedtuple
Coordinates = namedtuple('Coordinates', ['lat', 'lng'])
def func():
return Coordinates(lat=1, lng=2)
latitude = func().lat
If the objects within your dictionary/tuple are strongly tied together then it may be a good idea to even define a class for it. That way you'll also be able to define more complex operations. A natural question that follows is: When should I be using classes in Python?
Most recent versions of python (≥ 3.7) have dataclasses which you can use to define classes with very few lines of code:
from dataclasses import dataclass
#dataclass
class Coordinates:
lat: float = 0
lng: float = 0
def func():
return Coordinates(lat=1, lng=2)
latitude = func().lat
The primary advantage of dataclasses over namedtuple is that its easier to extend, but there are other differences. Note that by default, dataclasses are mutable, but you can use #dataclass(frozen=True) instead of #dataclass to force them being immutable.
Here is a video that might help you pick the right data class for your use case.
Three simple choices.
Obvious
x, _ = func()
x, junk = func()
Hideous
x = func()[0]
And there are ways to do this with a decorator.
def val0( aFunc ):
def pick0( *args, **kw ):
return aFunc(*args,**kw)[0]
return pick0
func0= val0(func)
This seems like the best choice to me:
val1, val2, ignored1, ignored2 = some_function()
It's not cryptic or ugly (like the func()[index] method), and clearly states your purpose.
If this is a function that you use all the time but always discard the second argument, I would argue that it is less messy to create an alias for the function without the second return value using lambda.
def func():
return 1, 2
func_ = lambda: func()[0]
func_() # Prints 1
This is not a direct answer to the question. Rather it answers this question: "How do I choose a specific function output from many possible options?".
If you are able to write the function (ie, it is not in a library you cannot modify), then add an input argument that indicates what you want out of the function. Make it a named argument with a default value so in the "common case" you don't even have to specify it.
def fancy_function( arg1, arg2, return_type=1 ):
ret_val = None
if( 1 == return_type ):
ret_val = arg1 + arg2
elif( 2 == return_type ):
ret_val = [ arg1, arg2, arg1 * arg2 ]
else:
ret_val = ( arg1, arg2, arg1 + arg2, arg1 * arg2 )
return( ret_val )
This method gives the function "advanced warning" regarding the desired output. Consequently it can skip unneeded processing and only do the work necessary to get your desired output. Also because Python does dynamic typing, the return type can change. Notice how the example returns a scalar, a list or a tuple... whatever you like!
When you have many output from a function and you don't want to call it multiple times, I think the clearest way for selecting the results would be :
results = fct()
a,b = [results[i] for i in list_of_index]
As a minimum working example, also demonstrating that the function is called only once :
def fct(a):
b=a*2
c=a+2
d=a+b
e=b*2
f=a*a
print("fct called")
return[a,b,c,d,e,f]
results=fct(3)
> fct called
x,y = [results[i] for i in [1,4]]
And the values are as expected :
results
> [3,6,5,9,12,9]
x
> 6
y
> 12
For convenience, Python list indexes can also be used :
x,y = [results[i] for i in [0,-2]]
Returns : a = 3 and b = 12
It is possible to ignore every variable except the first with less syntax if you like. If we take your example,
# The function you are calling.
def func():
return 1, 2
# You seem to only be interested in the first output.
x, temp = func()
I have found the following to works,
x, *_ = func()
This approach "unpacks" with * all other variables into a "throwaway" variable _. This has the benefit of assigning the one variable you want and ignoring all variables behind it.
However, in many cases you may want an output that is not the first output of the function. In these cases, it is probably best to indicate this by using the func()[i] where i is the index location of the output you desire. In your case,
# i == 0 because of zero-index.
x = func()[0]
As a side note, if you want to get fancy in Python 3, you could do something like this,
# This works the other way around.
*_, y = func()
Your function only outputs two potential variables, so this does not look too powerful until you have a case like this,
def func():
return 1, 2, 3, 4
# I only want the first and last.
x, *_, d = func()