Develop Reference

Overriding function signature (in help) when using functools.wraps

I'm creating a wrapper for a function with functools.wraps. My wrapper has the effect of overriding a default parameter (and it doesn't do anything else):
def add(*, a=1, b=2):
"Add numbers"
return a + b
#functools.wraps(add)
def my_add(**kwargs):
kwargs.setdefault('b', 3)
return add(**kwargs)
This my_add definition behaves the same as
#functools.wraps(add)
def my_add(*, a=1, b=3):
return add(a=a, b=b)
except that I didn't have to manually type out the parameter list.
However, when I run help(my_add), I see the help string for add, which has the wrong function name and the wrong default argument for the parameter b:
add(*, a=1, b=2)
Add numbers
How can I override the function name and the default argument in this help() output?
(Or, is there a different way to define my_add, using for example some magic function my_add = magic(add, func_name='my_add', kwarg_defaults={'b': 3}) that will do what I want?)

Let me try and explain what happens.
When you call the help functions, this is going to request information about your function using the inspect module. Therefore you have to change the function signature, in order to change the default argument.
Now this is not something that is advised, or often preferred, but who cares about that right? The provided solution is considered hacky and probably won't work for all versions of Python. Therefore you might want to reconsider how important the help function is... Any way let's start with some explanation on how it was done, followed by the code and test case.
Copying functions
Now the first thing we will do is copy the entire function, this is because I only want to change the signature of the new function and not the original function. This decouples the new my_add signature (and default values) from the original add function.
See:
How to create a copy of a python function
How can I make a deepcopy of a function in Python?
For ideas of how to do this (I will show my version in a bit).
Copying / updating signature
The next step is to get a copy of the function signature, for that this post was very useful. Except for the part where we have to adjust the signature parameters to match the new keyword default arguments.
For that we have to change the value of a mappingproxy, which we can see when running the debugger on the return value of inspect.signature(g). Now so far this can only be done by changing the private variables (the values with leading underscores _private). Therefore this solution will be considered hacky and is not guaranteed to withstand possible updates. That said, let's see the solution!
Full code
import inspect
import types
import functools
def update_func(f, func_name='', update_kwargs: dict = None):
"""Based on http://stackoverflow.com/a/6528148/190597 (Glenn Maynard)"""
g = types.FunctionType(
code=f.__code__,
globals=f.__globals__.copy(),
name=f.__name__,
argdefs=f.__defaults__,
closure=f.__closure__
)
g = functools.update_wrapper(g, f)
g.__signature__ = inspect.signature(g)
g.__kwdefaults__ = f.__kwdefaults__.copy()
# Adjust your arguments
for key, value in (update_kwargs or {}).items():
g.__kwdefaults__[key] = value
g.__signature__.parameters[key]._default = value
g.__name__ = func_name or g.__name__
return g
def add(*, a=1, b=2):
"Add numbers"
return a + b
my_add = update_func(add, func_name="my_add", update_kwargs=dict(b=3))
Example
if __name__ == '__main__':
a = 2
print("*" * 50, f"\nMy add\n", )
help(my_add)
print("*" * 50, f"\nOriginal add\n", )
help(add)
print("*" * 50, f"\nResults:"
f"\n\tMy add : a = {a}, return = {my_add(a=a)}"
f"\n\tOriginal add: a = {a}, return = {add(a=a)}")
Output
**************************************************
My add
Help on function my_add in module __main__:
my_add(*, a=1, b=3)
Add numbers
**************************************************
Original add
Help on function add in module __main__:
add(*, a=1, b=2)
Add numbers
**************************************************
Results:
My add : a = 2, return = 5
Original add: a = 2, return = 4
Usages
f: is the function that you want to update
func_name: is optionally the new name of the function (if empty, keeps the old name)
update_kwargs: is a dictionary containing the key and value of the default arguments that you want to update.
Notes
The solution is using copy variables to make full copies of dictionaries, such that there is no impact on the original add function.
The _default value is a private variable, and can be changed in future releases of python.

Can I implement a function or better a decorator that makes func(a1)(a2)(a3)...(an) == func(a1, a2, a3,...,an)? [duplicate]

On Codewars.com I encountered the following task:
Create a function add that adds numbers together when called in succession. So add(1) should return 1, add(1)(2) should return 1+2, ...
While I'm familiar with the basics of Python, I've never encountered a function that is able to be called in such succession, i.e. a function f(x) that can be called as f(x)(y)(z).... Thus far, I'm not even sure how to interpret this notation.
As a mathematician, I'd suspect that f(x)(y) is a function that assigns to every x a function g_{x} and then returns g_{x}(y) and likewise for f(x)(y)(z).
Should this interpretation be correct, Python would allow me to dynamically create functions which seems very interesting to me. I've searched the web for the past hour, but wasn't able to find a lead in the right direction. Since I don't know how this programming concept is called, however, this may not be too surprising.
How do you call this concept and where can I read more about it?

I don't know whether this is function chaining as much as it's callable chaining, but, since functions are callables I guess there's no harm done. Either way, there's two ways I can think of doing this:
Sub-classing int and defining __call__:
The first way would be with a custom int subclass that defines __call__ which returns a new instance of itself with the updated value:
class CustomInt(int):
def __call__(self, v):
return CustomInt(self + v)
Function add can now be defined to return a CustomInt instance, which, as a callable that returns an updated value of itself, can be called in succession:
>>> def add(v):
... return CustomInt(v)
>>> add(1)
1
>>> add(1)(2)
3
>>> add(1)(2)(3)(44) # and so on..
50
In addition, as an int subclass, the returned value retains the __repr__ and __str__ behavior of ints. For more complex operations though, you should define other dunders appropriately.
As #Caridorc noted in a comment, add could also be simply written as:
add = CustomInt
Renaming the class to add instead of CustomInt also works similarly.
Define a closure, requires extra call to yield value:
The only other way I can think of involves a nested function that requires an extra empty argument call in order to return the result. I'm not using nonlocal and opt for attaching attributes to the function objects to make it portable between Pythons:
def add(v):
def _inner_adder(val=None):
"""
if val is None we return _inner_adder.v
else we increment and return ourselves
"""
if val is None:
return _inner_adder.v
_inner_adder.v += val
return _inner_adder
_inner_adder.v = v # save value
return _inner_adder
This continuously returns itself (_inner_adder) which, if a val is supplied, increments it (_inner_adder += val) and if not, returns the value as it is. Like I mentioned, it requires an extra () call in order to return the incremented value:
>>> add(1)(2)()
3
>>> add(1)(2)(3)() # and so on..
6

You can hate me, but here is a one-liner :)
add = lambda v: type("", (int,), {"__call__": lambda self, v: self.__class__(self + v)})(v)
Edit: Ok, how this works? The code is identical to answer of #Jim, but everything happens on a single line.
type can be used to construct new types: type(name, bases, dict) -> a new type. For name we provide empty string, as name is not really needed in this case. For bases (tuple) we provide an (int,), which is identical to inheriting int. dict are the class attributes, where we attach the __call__ lambda.
self.__class__(self + v) is identical to return CustomInt(self + v)
The new type is constructed and returned within the outer lambda.

If you want to define a function to be called multiple times, first you need to return a callable object each time (for example a function) otherwise you have to create your own object by defining a __call__ attribute, in order for it to be callable.
The next point is that you need to preserve all the arguments, which in this case means you might want to use Coroutines or a recursive function. But note that Coroutines are much more optimized/flexible than recursive functions, specially for such tasks.
Here is a sample function using Coroutines, that preserves the latest state of itself. Note that it can't be called multiple times since the return value is an integer which is not callable, but you might think about turning this into your expected object ;-).
def add():
current = yield
while True:
value = yield current
current = value + current
it = add()
next(it)
print(it.send(10))
print(it.send(2))
print(it.send(4))
10
12
16

Simply:
class add(int):
def __call__(self, n):
return add(self + n)

If you are willing to accept an additional () in order to retrieve the result you can use functools.partial:
from functools import partial
def add(*args, result=0):
return partial(add, result=sum(args)+result) if args else result
For example:
>>> add(1)
functools.partial(<function add at 0x7ffbcf3ff430>, result=1)
>>> add(1)(2)
functools.partial(<function add at 0x7ffbcf3ff430>, result=3)
>>> add(1)(2)()
3
This also allows specifying multiple numbers at once:
>>> add(1, 2, 3)(4, 5)(6)()
21
If you want to restrict it to a single number you can do the following:
def add(x=None, *, result=0):
return partial(add, result=x+result) if x is not None else result
If you want add(x)(y)(z) to readily return the result and be further callable then sub-classing int is the way to go.

The pythonic way to do this would be to use dynamic arguments:
def add(*args):
return sum(args)
This is not the answer you're looking for, and you may know this, but I thought I would give it anyway because if someone was wondering about doing this not out of curiosity but for work. They should probably have the "right thing to do" answer.

how to partially apply arbitrary argument of a function?

I want to use partial from functools to partially apply a function's second argument, I know it is easy to do with lambda rather than partial as follows
>>> def func1(a,b):
... return a/b
...
>>> func2 = lambda x:func1(x,2)
>>> func2(4)
2
but I strictly want to use partial here (for the sake of learning) so i came up with this.
>>> def swap_binary_args(func):
... return lambda x,y: func(y,x)
...
>>> func3 = partial(swap_binary_args(func1),2)
>>> func3(4)
2
Is it possible to extend this strategy to a level where I can partial apply any arguments at any place like in the following pseudocode
>>>def indexed_partial(func, list_of_index, *args):
... ###do_something###
... return partially_applied_function
>>>func5=indexed_partial(func1, [1,4,3,5], 2,4,5,6)
in our case I can use this function as follows
>>>func6=indexed_partial(func1, [1], 2)
Is it possible to have an indexed partial like I want ? is there anything similar to this already which I am not aware of ? and more importantly is the idea of indexed partial generally a good or bad idea why ?
This question has been marked as possible duplicate of Can one partially apply the second argument of a function that takes no keyword arguments?
in that question the OP asked is it possible to partially apply second argument but here i am asking how to cook a function that can partially apply any arbitrary argument

I, too, think what you ask can't be done (easily?) with functools.partial. Probably the best (and most readable) solution is to use partial with keyword-arguments.
However, in case you want to use positional arguments (and hence indexed partial arguments), here is a possible definition of indexed_partial:
def indexed_partial(func, list_of_index, *args):
def partially_applied_function(*fargs, **fkwargs):
nargs = len(args) + len(fargs)
iargs = iter(args)
ifargs = iter(fargs)
posargs = ((ifargs, iargs)[i in list_of_index].next() for i in range(nargs))
return func(*posargs, **fkwargs)
return partially_applied_function

Is taking advantage of the one-time binding of function arguments a bad idea?

New python users often get tripped up by mutable argument defaults. What are the gotchas and other issues of using this 'feature' on purpose, for example, to get tweakable defaults at runtime that continue to display properly in function signatures via help()?
class MutableString (str):
def __init__ (self, value):
self.value = value
def __str__ (self):
return self.value
def __repr__ (self):
return "'" + self.value + "'"
defaultAnimal = MutableString('elephant')
def getAnimal (species=defaultAnimal):
'Return the given animal, or the mutable default.'
return species
And in use:
>>> help(getAnimal)
getAnimal(species='elephant')
Return the given animal, or the mutable default.
>>> print getAnimal()
elephant
>>> defaultAnimal.value = 'kangaroo'
>>> help(getAnimal)
getAnimal(species='kangaroo')
Return the given animal, or the mutable default.
>>> print getAnimal()
kangaroo

First, read Why are default values shared between objects. That doesn't answer your question, but it provides some background.
There are different valid uses for this feature, but they pretty much all share something in common: the default value is a transparent, simple, obviously-mutable, built-in type. Memoization caches, accumulators for recursive calls, optional output variables, etc. all look like this. So, experienced Python developers will usually spot one of these use cases—if they see memocache={} or accum=[], they'll know what to expect. But your code will not look like a use for mutable default values at all, which will be as misleading to experts as it is to novices.
Another problem is that your function looks like it's returning a string, but it's lying:
>>> print getAnimal()
kangaroo
>>> print getAnimal()[0]
e
Of course the problem here is that you've implemented MutableString wrong, not that it's impossible to implement… but still, this should show why trying to "trick" the interpreter and your users tends to open the door to unexpected bugs.
--
The obvious way to handle it is to store the changing default in a module, function, or (if it's a method) instance attribute, and use None as a default value. Or, if None is a valid value, use some other sentinel:
defaultAnimal = 'elephant'
def getAnimal (species=None):
if species is None:
return defaultAnimal
return species
Note that this is pretty much exactly what the FAQ suggests. Even if you inherently have a mutable value, you should do this dance to get around the problem. So you definitely shouldn't create a mutable value out of an inherently immutable one to create the problem.
Yes, this means that help(getAnimal) doesn't show the current default. But nobody will expect it to.
They will probably expect you to tell them that the default value is a hook, of course, but that's a job for a docstring:
defaultAnimal = 'elephant'
def getAnimal (species=None):
"""getAnimal([species]) -> species
If the optional species parameter is left off, a default animal will be
returned. Normally this is 'elephant', but this can be configured by setting
foo.defaultAnimal to another value.
"""
if species is None:
return defaultAnimal
return species

The only useful use I've seen for it is as a cache:
def fibo(n, cache={}):
if n < 2:
return 1
else:
if n in cache:
return cache[n]
else:
fibo_n = fibo(n-1) + fibo(n-2) # you can still hit maximum recursion depth
cache[n] = fibo_n
return fibo_n
...but then it's cleaner to use the #lru_cache decorator.
#lru_cache
def fibo(n):
if n < 2:
return 1
else:
return fibo(n-1) + fibo(n-2)

Deal with undefined arguments more elegantly

The accepted paradigm to deal with mutable default arguments is:
def func(self, a = None):
if a is None:
a = <some_initialisation>
self.a = a
As I might have to do this for several arguments, I would need to write very similar 3 lines over and over again. I find this un-pythonically a lot of text to read for a very very standard thing to do when initialising class objects or functions.
Isn't there an elegant one-liner to replace those 3 lines dealing with the potentially undefined argument and the standard required copying to the class instance variables?

If a "falsy" value (0, empty string, list, dict, etc.) is not a valid value for a, then you can cut down the initialization to one line:
a = a or <initialize_object>

Another way of doing the same thing is as follows:
def func(self,**kwargs):
self.a=kwargs.get('a',<a_initialization>)
...
This has the added bonus that the value of a passed to the function could be None and the initialization won't overwrite it. The disadvantage is that a user using the builtin help function won't be able to tell what keywords your function is looking for unless you spell it out explicitly in the docstring.
EDIT
One other comment. The user could call the above function with keywords which are not pulled out of the kwargs dictionary. In some cases, this is good (if you want to pass the keywords to another function for instance). In other cases, this is not what you want. If you want to raise an error if the user provides an unknown keyword, you can do the following:
def func(self,**kwargs):
self.a=kwargs.pop('a',"Default_a")
self.b=kwargs.pop('b',"Default_b")
if(kwargs):
raise ... #some appropriate exception...possibly using kwargs.keys() to say which keywords were not appropriate for this function.

You could do this
def func(self, a=None):
self.a = <some_initialisation> if a is None else a
But why the obsession with one liners? I would usually use the 3 line version even if it gets repeated all over the place because if makes your code very easy for experienced Python programmers to read

just a little solution I came up by using an extra function, can be improved of course:
defaultargs.py:
def doInit(var, default_value,condition):
if condition:
var = default_value
return var
def func(a=None, b=None, c=None):
a = doInit(a,5,(a is None or not isinstance(a,int)))
b = doInit(b,10.0,(a is None or not isinstance(a,float)))
c = doInit(c,"whatever",(a is None or not isinstance(c, str)))
print a
print b
print c
if __name__ == "__main__":
func(10)
func(None,12341.12)
func("foo",None,"whowho")
output:
10
10.0
whatever
5
10.0
whatever
5
10.0
whowho
I like your question. :)
Edit: If you dont care about the variables type, please dont use isinstance().