Develop Reference

How to specify arg position for functool partial()

As per manual, functools partial() is 'used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature.'
What's the best way to specify the positions of the arguments that one wishes to evaluate?
EDIT
Note as per comments, the function to be partially evaluated may contain named and unnamed arguments (these functions should be completely arbitrary and may be preexisting)
END EDIT
For example, consider:
def f(x,y,z):
return x + 2*y + 3*z
Then, using
from functools import partial
both
partial(f,4)(5,6)
and
partial(f,4,5)(6)
give 32.
But what if one wants to evaluate, say the third argument z or the first and third arguments x, and z?
Is there a convenient way to pass the position information to partial, using a decorator or a dict whose keys are the desired arg positions and the respective values are the arg values? eg to pass the x and z positions something like like this:
partial_dict(f,{0:4,2:6})(5)

No, partial is not designed to freeze positional arguments at non-sequential positions.
To achieve the desired behavior outlined in your question, you would have to come up with a wrapper function of your own like this:
def partial_positionals(func, positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(positionals[i] if i in positionals else next(arg)
for i in range(len(args) + len(positionals))), **{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, {0: 4, 2: 6})(5))
outputs:
32

Simply use keyword arguments. Using your definition of f above,
>>> g = partial(f, z=10)
>>> g(2, 4)
40
>>> h = partial(f, y=4, z=10)
>>> h(2)
40
Note that once you use a keyword argument for a given parameter, you must use keyword arguments for all remaining arguments. For example, the following would not be valid:
>>> j = partial(f, x=2, z=10)
>>> j(4)
TypeError: f() got multiple values for argument 'x'
But continuing to use keyword arguments is:
>>> j = partial(f, x=2, z=10)
>>> j(y=4)
40
When you use functools.partial, you store the values of *args and **kwargs for later interpolation. When you later call the "partially applied" function, the implementation of functools.partial effectively adds the previously provided *args and **kwargs to the argument list at the front and end, respectively, as though you had inserted these argument-unpackings yourself. I.e., calling
h = partial(1, z=10)
f(4)
is roughly equivalent to writing
args = [1]
kwargs = {'z': 10}
f(*args, 4, **kwargs)
As such, the semantics of how you provide arguments to functools.partial is the same as how you would need to store arguments in the args and kwargs variables above such that the final call to f is sensible. For more information, take a look at the pseduo-implementation of functools.partial given in the functools module documentation

For easier usage, you can create a new object specifically to specify a positional argument that is to be skipped when sequentially listing values for positional arguments to be frozen with partial:
SKIP = object()
def partial_positionals(func, *positionals, **keywords):
def wrapper(*args, **kwargs):
arg = iter(args)
return func(*(*(next(arg) if i is SKIP else i for i in positionals), *arg),
**{**keywords, **kwargs})
return wrapper
so that:
def f(x, y, z):
return x + 2 * y + 3 * z
print(partial_positionals(f, 4, SKIP, 6)(5))
outputs:
32

Is it possible to pass the same optional arguments to multiple functions?

I want to ask if there is a way to prevent unnecessary duplicate of code when passing the same arguments into a function's optional arguments.
Hopefully the following example provides a good idea of what I am trying to do:
def f(arg1):
def g(optional_1=0, optional_2=0, optional_3=0):
return arg1+optional_1+optional_2+optional_3
return g
b, c = 2, 3
f1 = f(1)
f2 = f(2)
calc_f1 = f1(optional_2=b, optional_3=c)
calc_f2 = f2(optional_2=b, optional_3=c)
As you can see, f1 and f2 only differ in the arg1 passed into f and afterwards I call them with the same variables for the same optional arguments.
It is fine when the code is short, but when I have over 10 optional arguments, it becomes unnecessarily long and redundant.
Is it possible to do something like
optional_variable_pair = #some way to combine them
calc_f1 = f1(optional_variable_pair)
calc_f2 = f2(optional_variable_pair)
so I get a more succinct and easy to read code?

Any function with multiple optional arguments is a bit smelly because:
you get so many argument combinations that it requires a large amount of testing.
because of all the options the function has to have alot of conditionals and routes which increase its cyclomatic complexity.
You can apply a refactoring to extract the whole argument list into an Object and have the function work on that object. This works really well if you can find a unifying name that describes your argument list and fits whatever metaphor you are using around the function. You can even invert the call so that the function becomes a method of the Object, so you get some encapsulation.

To answer the question you asked, the answer is yes. You can do almost exactly what you want using keyword argument unpacking.
def f(arg1):
def g(optional_1=0, optional_2=0, optional_3=0):
return arg1+optional_1+optional_2+optional_3
return g
optional_variable_pair = {
'optional_2': 2,
'optional_3': 3
}
f1 = f(1)
f2 = f(2)
calc_f1 = f1(**optional_variable_pair)
calc_f2 = f2(**optional_variable_pair)
If I'm reading your intent correctly, though, the essence of your question is wanting to pass new first arguments with the same successive arguments to a function. Depending on your use case, the wrapper function g may be unnecessary.
def f(arg1, *, optional_1=0, optional_2=0, optional_3=0):
return optional_1 + optional_2+optional_3
optional_variable_pair = {
'optional_2': 2,
'optional_3': 3
}
calc_f1 = f(1, **optional_variable_pair)
calc_f2 = f(2, **optional_variable_pair)
Obviously, if the first argument continues incrementing by one, a for loop is in order. Obviously, if you are never using the optional_1 parameter, you do not need to include it. But, moreover, if you find yourself using numbered arguments, there is a good chance you really should be working with tuple unpacking instead of keyword unpacking:
def f(*args):
return sum(args)
optional_variable_pair = (2, 3)
for i in range(1, 3):
calc = f(i, *optional_variable_pair)
# ...do something with calc...
You may also be interested in researching functools.partial, as well, which can take the place of your wrapper function g, and allow this:
import functools
def f(*args):
return sum(args)
f1 = functools.partial(f, 1)
f2 = functools.partial(f, 2)
calc_f1 = f1(2, 3) # = 1 + 2 + 3 = 6
calc_f2 = f2(2, 3) # = 2 + 2 + 3 = 7

You use key-value pairs as function argsuments, for this purpose you can use *args and **kwargs:
optional_variable_pair = {
"optional_1": 1,
"optional_2": 2,
"optional_3": 3,
}
calc_f1 = f1(**optional_variable_pair)
calc_f2 = f2(**optional_variable_pair)

equivalent to R's `do.call` in python

Is there an equivalent to R's do.call in python?
do.call(what = 'sum', args = list(1:10)) #[1] 55
do.call(what = 'mean', args = list(1:10)) #[1] 5.5
?do.call
# Description
# do.call constructs and executes a function call from a name or a function and a list of arguments to be passed to it.

There is no built-in for this, but it is easy enough to construct an equivalent.
You can look up any object from the built-ins namespace using the __builtin__ (Python 2) or builtins (Python 3) modules then apply arbitrary arguments to that with *args and **kwargs syntax:
try:
# Python 2
import __builtin__ as builtins
except ImportError:
# Python 3
import builtins
def do_call(what, *args, **kwargs):
return getattr(builtins, what)(*args, **kwargs)
do_call('sum', range(1, 11))
Generally speaking, we don't do this in Python. If you must translate strings into function objects, it is generally preferred to build a custom dictionary:
functions = {
'sum': sum,
'mean': lambda v: sum(v) / len(v),
}
then look up functions from that dictionary instead:
functions['sum'](range(1, 11))
This lets you strictly control what names are available to dynamic code, preventing a user from making a nuisance of themselves by calling built-ins for their destructive or disruptive effects.

do.call is pretty much the equivalent of the splat operator in Python:
def mysum(a, b, c):
return sum([a, b, c])
# normal call:
mysum(1, 2, 3)
# with a list of arguments:
mysum(*[1, 2, 3])
Note that I’ve had to define my own sum function since Python’s sum already expects a list as an argument, so your original code would just be
sum(range(1, 11))
R has another peculiarity: do.call internally performs a function lookup of its first argument. This means that it finds the function even if it’s a character string rather than an actual function. The Python equivalent above doesn’t do this — see Martijn’s answer for a solution to this.

Goes similar to previous answer, but why so complicated?
def do_call(what, args=[], kwargs = {}):
return what(*args, **kwargs)
(Which is more elegant than my previously posted definition:)
def do_call(which, args=None, kwargs = None):
if args is None and kwargs is not None:
return which(**kwargs)
elif args is not None and kwargs is None:
return which(*args)
else:
return which(*args, **kwargs)
Python's sum is different than R's sum (1 argument a list expected vs.
arbitraily many arguments expected in R). So we define our own sum (mysum)
which behaves similarly to R's sum. In a similar way we define mymean.
def mysum(*args):
return sum(args)
def mymean(*args):
return sum(args)/len(args)
Now we can recreate your example in Python - as a reasonable 1:1 translation of the R function call.
do_call(what = mymean, args=[1, 2, 3])
## 2.0
do_call(what = mysum, args=[1, 2, 3])
## 6
For functions with argument names, we use a dict for kwargs, where the parameter
names are keys of the dictionary (as strings) and their values the values.
def myfunc(a, b, c):
return a + b + c
do_call(what = myfunc, kwargs={"a": 1, "b": 2, "c": 3})
## 6
# we can even mix named and unnamed parts
do_call(what = myfunc, args = [1, 2], kwargs={"c": 3})
## 6

Calling function with unknown number of parameters

I am trying to create a set of functions in python that will all do a similar operation on a set of inputs. All of the functions have one input parameter fixed and half of them also need a second parameter. For the sake of simplicity, below is a toy example with only two functions.
Now, I want, in my script, to run the appropriate function, depending on what the user input as a number. Here, the user is the random function (so the minimum example works). What I want to do is something like this:
def function_1(*args):
return args[0]
def function_2(*args):
return args[0] * args[1]
x = 10
y = 20
i = random.randint(1,2)
f = function_1 if i==1 else function_2
return_value = f(x,y)
And it works, but it seems messy to me. I would rather have function_1 defined as
def function_1(x):
return x
Another way would be to define
def function_1(x,y):
return x
But that leaves me with a dangling y parameter.
but that will not work as easily. Is my way the "proper" way of solving my problem or does there exist a better way?

There are couple of approaches here, all of them adding more boiler-plate code.
There is also this PEP which may be interesting to you.
But 'pythonic' way of doing it is not as elegant as usual function overloading due to the fact that functions are just class attributes.
So you can either go with function like that:
def foo(*args):
and then count how many args you've got which will be very broad but very flexible as well.
another approach is the default arguments:
def foo(first, second=None, third=None)
less flexible but easier to predict, and then lastly you can also use:
def foo(anything)
and detect the type of anything in your function acting accordingly.
Your monkey-patching example can work too, but it becomes more complex if you use it with class methods, and does make introspection tricky.
EDIT: Also, for your case you may want to keep the functions separate and write single 'dispatcher' function that will call appropriate function for you depending on the arguments, which is probably best solution considering above.
EDIT2: base on your comments I believe that following approach may work for you
def weigh_dispatcher(*args, **kwargs):
#decide which function to call base on args
if 'somethingspecial' in kwargs:
return weight2(*args, **kwargs)
def weight_prep(arg):
#common part here
def weight1(arg1, arg2):
weitht_prep(arg1)
#rest of the func
def weight2(arg1, arg2, arg3):
weitht_prep(arg1)
#rest of the func
alternatively you can move the common part into the dispatcher

You may also have a function with optional second argument:
def function_1(x, y = None):
if y != None:
return x + y
else:
return x
Here's the sample run:
>>> function_1(3)
3
>>> function_1(3, 4)
7
Or even optional multiple arguments! Check this out:
def function_2(x, *args):
return x + sum(args)
And the sample run:
>>> function_2(3)
3
>>> function_2(3, 4)
7
>>> function_2(3, 4, 5, 6, 7)
25
You may here refer to args as to list:
def function_3(x, *args):
if len(args) < 1:
return x
else:
return x + sum(args)
And the sample run:
>>> function_3(1,2,3,4,5)
15

Expanding tuples into arguments

Suppose I have a function like:
def myfun(a, b, c):
return (a * 2, b + c, c + b)
Given a tuple some_tuple = (1, "foo", "bar"), how can I use some_tuple to call myfun, to get the result (2, "foobar", "barfoo")
I know could define myfun so that it accepts the tuple directly, but I want to call the existing myfun.
See also: What do the * (star) and ** (double star) operators mean in a function call?.

myfun(*some_tuple) does exactly what you request. The * operator simply unpacks the tuple (or any iterable) and passes them as the positional arguments to the function. Read more about unpacking arguments.

Note that you can also expand part of argument list:
myfun(1, *("foo", "bar"))

Take a look at the Python tutorial section 4.7.3 and 4.7.4.
It talks about passing tuples as arguments.
I would also consider using named parameters (and passing a dictionary) instead of using a tuple and passing a sequence. I find the use of positional arguments to be a bad practice when the positions are not intuitive or there are multiple parameters.

This is the functional programming method. It lifts the tuple expansion feature out of syntax sugar:
apply_tuple = lambda f, t: f(*t)
Redefine apply_tuple via curry to save a lot of partial calls in the long run:
from toolz import curry
apply_tuple = curry(apply_tuple)
Example usage:
from operator import add, eq
from toolz import thread_last
thread_last(
[(1,2), (3,4)],
(map, apply_tuple(add)),
list,
(eq, [3, 7])
)
# Prints 'True'

Similar to #Dominykas's answer, this is a decorator that converts multiargument-accepting functions into tuple-accepting functions:
apply_tuple = lambda f: lambda args: f(*args)
Example 1:
def add(a, b):
return a + b
three = apply_tuple(add)((1, 2))
Example 2:
#apply_tuple
def add(a, b):
return a + b
three = add((1, 2))