I'm trying to understand the python data model better and ran into something odd.
def foo(a, b = 2):
return a / b
assert foo(20) == 10.0
# note: for sanity purposes, should also change signature, but not needed for effect
foo.__defaults__ = (10,)
assert foo(20) == 2.0
foo.__defaults__ = ()
foo.__kwdefaults__ = {'b': 10}
foo(20) # raises TypeError: foo() missing 1 required positional argument: 'b'
An error is expected: __kwdefaults__ is for keyword-only arguments, so let's make b a keyword-only argument to try to solve this problem:
from inspect import signature
foo.__signature__ = signature(lambda a, *, b=10: None)
foo(20) # still raises TypeError: foo() missing 1 required positional argument: 'b'
How does the error message relate to what's happening?.
What I find strange is that neither the original function, nor my doctored one required b (it always had a default!). Also, b has never been a positional-only argument.
What is happening here? How can one transform foo to make b be a keyword-only argument with default 10.
If my original function had the signature I "injected" above, all goes well though:
def foo(a, *, b=2): # same as previous `foo`, with signature we want
return a / b
foo.__kwdefaults__ = {'b': 10} # change kwdefault
assert foo(20) == 2.0 # it works!!
Preemptive note: I know of functools wraps and partial, which I could use -- though in my context, I'd rather change the function itself, not a wrapped version. My question is about the behavior I created in the code above: How did it come about?
Purpose of __signature__
Your issue is, that you think that you change a function's signature by setting foo.__signature__. However, this is not what's happening. It is equally useless to set it to foo.signature or foo.any_other_name. You just set a signature object to the respective property of the function, which changes nothing with regards to the function's behaviour.
The only thing that __signature__ does is to change the behaviour of inspect.signature(), since it will return the signature of the function as stored in function.__signature__ iff it is set. I.e. the only thing, that __signature__ changes is the behaviour of inspect.signature(), but not the function itself.
See ekhumoro's comment for the link to the appropriate PEP.
TypeError
As for the type error: In foo() b is not a kwarg-only argument:
def foo(a, b = 2):
return a / b
It is a positional argument with a default value. Hence its default value is stored in foo.__defaults__. When you set foo.__defaults__ = () you erased those defaults. After that, b hence has no longer a default value and needs to be passed explicitly.
Changing signatures
How can one transform foo to make b be a keyword-only argument with default 10.
You cannot change a function's signature during runtime. Period.
Changing default values
You can, however, change b's default value to 10 via
>>> foo.__defaults__ = (10,)
>>> foo(2)
0.2
Since positional arguments with default values cannot be followed by positional arguments without defaults, the tuple __defaults__ is applied to the positional arguments from right to left.
So you can also give a a default value of e.g. 20 via
>>> foo.__defaults__ = (20, 10)
>>> foo()
2.0
Related
In Python, what value can a variable take, so that when a function is invoked with the variable as an argument, the function uses its default value for the parameter instead?
Consider the following code:
def foo(a=100):
print(a)
b = None #blank value
foo(b)
Desired output:
100
Actual output:
None
I hypothesized that None would work, but clearly it doesn't. What value can I choose for b, so that foo(b) is equivalent to foo()? Or is this simply not possible? I'm in a situation where the value for b can either be defined, or I would like to use the default value of the parameter.
(This answer assumes that you cannot modify foo, and that you cannot use reflection or introspection to determine what the default argument value is.)
It's the absence of an argument, not any particular value used as an argument, that triggers the use of the default value. The only way you can produce nothing out of something is to unpack an empty mapping
foo(**{})
or an empty sequence
foo(*())
Both * and ** are part of the function-call syntax, though, not part of the argument value, so with a variable, it still looks like
b = {}
foo(**b)
b = ()
foo(*b)
If you want None to revert to a default value, the easiest way is to do the logic in the function itself.
def foo(a=None):
if a is None:
a = 100
print(a)
Function default parameter uses when there don't pass any parameter for the argument. And None is not a blank value. None is an object of NoneType Datatype in python similar to Other Datatype Object.
Instead, you can use
def foo(a=100):
a=100 if a is None else a //Ternary operator
print(a)
b = None #blank value
foo(b)
Output:
100
In my case, I ended up using the inspect module to create a helper function which extracts the default values of the function as described here:
import inspect
def get_defaults(func):
signature = inspect.signature(func)
return { k: v.default for k, v
in signature.parameters.items()
if v.default is not inspect.Parameter.empty }
def foo(a=100):
print(a)
b = get_defaults(foo)['a']
foo(b)
Output:
100
What does a bare asterisk in the parameters of a function do?
When I looked at the pickle module, I see this:
pickle.dump(obj, file, protocol=None, *, fix_imports=True)
I know about a single and double asterisks preceding parameters (for variable number of parameters), but this precedes nothing. And I'm pretty sure this has nothing to do with pickle. That's probably just an example of this happening. I only learned its name when I sent this to the interpreter:
>>> def func(*):
... pass
...
File "<stdin>", line 1
SyntaxError: named arguments must follow bare *
If it matters, I'm on python 3.3.0.
Bare * is used to force the caller to use named arguments - so you cannot define a function with * as an argument when you have no following keyword arguments.
See this answer or Python 3 documentation for more details.
While the original answer answers the question completely, just adding a bit of related information. The behaviour for the single asterisk derives from PEP-3102. Quoting the related section:
The second syntactical change is to allow the argument name to
be omitted for a varargs argument. The meaning of this is to
allow for keyword-only arguments for functions that would not
otherwise take a varargs argument:
def compare(a, b, *, key=None):
...
In simple english, it means that to pass the value for key, you will need to explicitly pass it as key="value".
def func(*, a, b):
print(a)
print(b)
func("gg") # TypeError: func() takes 0 positional arguments but 1 was given
func(a="gg") # TypeError: func() missing 1 required keyword-only argument: 'b'
func(a="aa", b="bb", c="cc") # TypeError: func() got an unexpected keyword argument 'c'
func(a="aa", b="bb", "cc") # SyntaxError: positional argument follows keyword argument
func(a="aa", b="bb") # aa, bb
the above example with **kwargs
def func(*, a, b, **kwargs):
print(a)
print(b)
print(kwargs)
func(a="aa",b="bb", c="cc") # aa, bb, {'c': 'cc'}
Semantically, it means the arguments following it are keyword-only, so you will get an error if you try to provide an argument without specifying its name. For example:
>>> def f(a, *, b):
... return a + b
...
>>> f(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() takes 1 positional argument but 2 were given
>>> f(1, b=2)
3
Pragmatically, it means you have to call the function with a keyword argument. It's usually done when it would be hard to understand the purpose of the argument without the hint given by the argument's name.
Compare e.g. sorted(nums, reverse=True) vs. if you wrote sorted(nums, True). The latter would be much less readable, so the Python developers chose to make you to write it the former way.
Suppose you have function:
def sum(a,key=5):
return a + key
You can call this function in 2 ways:
sum(1,2) or sum(1,key=2)
Suppose you want function sum to be called only using keyword arguments.
You add * to the function parameter list to mark the end of positional arguments.
So function defined as:
def sum(a,*,key=5):
return a + key
may be called only using sum(1,key=2)
I've found the following link to be very helpful explaining *, *args and **kwargs:
https://pythontips.com/2013/08/04/args-and-kwargs-in-python-explained/
Essentially, in addition to the answers above, I've learned from the site above (credit: https://pythontips.com/author/yasoob008/) the following:
With the demonstration function defined first below, there are two examples, one with *args and one with **kwargs
def test_args_kwargs(arg1, arg2, arg3):
print "arg1:", arg1
print "arg2:", arg2
print "arg3:", arg3
# first with *args
>>> args = ("two", 3,5)
>>> test_args_kwargs(*args)
arg1: two
arg2: 3
arg3: 5
# now with **kwargs:
>>> kwargs = {"arg3": 3, "arg2": "two","arg1":5}
>>> test_args_kwargs(**kwargs)
arg1: 5
arg2: two
arg3: 3
So *args allows you to dynamically build a list of arguments that will be taken in the order in which they are fed, whereas **kwargs can enable the passing of NAMED arguments, and can be processed by NAME accordingly (irrespective of the order in which they are fed).
The site continues, noting that the correct ordering of arguments should be:
some_func(fargs,*args,**kwargs)
What does a bare asterisk in the parameters of a function do?
When I looked at the pickle module, I see this:
pickle.dump(obj, file, protocol=None, *, fix_imports=True)
I know about a single and double asterisks preceding parameters (for variable number of parameters), but this precedes nothing. And I'm pretty sure this has nothing to do with pickle. That's probably just an example of this happening. I only learned its name when I sent this to the interpreter:
>>> def func(*):
... pass
...
File "<stdin>", line 1
SyntaxError: named arguments must follow bare *
If it matters, I'm on python 3.3.0.
Bare * is used to force the caller to use named arguments - so you cannot define a function with * as an argument when you have no following keyword arguments.
See this answer or Python 3 documentation for more details.
While the original answer answers the question completely, just adding a bit of related information. The behaviour for the single asterisk derives from PEP-3102. Quoting the related section:
The second syntactical change is to allow the argument name to
be omitted for a varargs argument. The meaning of this is to
allow for keyword-only arguments for functions that would not
otherwise take a varargs argument:
def compare(a, b, *, key=None):
...
In simple english, it means that to pass the value for key, you will need to explicitly pass it as key="value".
def func(*, a, b):
print(a)
print(b)
func("gg") # TypeError: func() takes 0 positional arguments but 1 was given
func(a="gg") # TypeError: func() missing 1 required keyword-only argument: 'b'
func(a="aa", b="bb", c="cc") # TypeError: func() got an unexpected keyword argument 'c'
func(a="aa", b="bb", "cc") # SyntaxError: positional argument follows keyword argument
func(a="aa", b="bb") # aa, bb
the above example with **kwargs
def func(*, a, b, **kwargs):
print(a)
print(b)
print(kwargs)
func(a="aa",b="bb", c="cc") # aa, bb, {'c': 'cc'}
Semantically, it means the arguments following it are keyword-only, so you will get an error if you try to provide an argument without specifying its name. For example:
>>> def f(a, *, b):
... return a + b
...
>>> f(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() takes 1 positional argument but 2 were given
>>> f(1, b=2)
3
Pragmatically, it means you have to call the function with a keyword argument. It's usually done when it would be hard to understand the purpose of the argument without the hint given by the argument's name.
Compare e.g. sorted(nums, reverse=True) vs. if you wrote sorted(nums, True). The latter would be much less readable, so the Python developers chose to make you to write it the former way.
Suppose you have function:
def sum(a,key=5):
return a + key
You can call this function in 2 ways:
sum(1,2) or sum(1,key=2)
Suppose you want function sum to be called only using keyword arguments.
You add * to the function parameter list to mark the end of positional arguments.
So function defined as:
def sum(a,*,key=5):
return a + key
may be called only using sum(1,key=2)
I've found the following link to be very helpful explaining *, *args and **kwargs:
https://pythontips.com/2013/08/04/args-and-kwargs-in-python-explained/
Essentially, in addition to the answers above, I've learned from the site above (credit: https://pythontips.com/author/yasoob008/) the following:
With the demonstration function defined first below, there are two examples, one with *args and one with **kwargs
def test_args_kwargs(arg1, arg2, arg3):
print "arg1:", arg1
print "arg2:", arg2
print "arg3:", arg3
# first with *args
>>> args = ("two", 3,5)
>>> test_args_kwargs(*args)
arg1: two
arg2: 3
arg3: 5
# now with **kwargs:
>>> kwargs = {"arg3": 3, "arg2": "two","arg1":5}
>>> test_args_kwargs(**kwargs)
arg1: 5
arg2: two
arg3: 3
So *args allows you to dynamically build a list of arguments that will be taken in the order in which they are fed, whereas **kwargs can enable the passing of NAMED arguments, and can be processed by NAME accordingly (irrespective of the order in which they are fed).
The site continues, noting that the correct ordering of arguments should be:
some_func(fargs,*args,**kwargs)
What does a bare asterisk in the parameters of a function do?
When I looked at the pickle module, I see this:
pickle.dump(obj, file, protocol=None, *, fix_imports=True)
I know about a single and double asterisks preceding parameters (for variable number of parameters), but this precedes nothing. And I'm pretty sure this has nothing to do with pickle. That's probably just an example of this happening. I only learned its name when I sent this to the interpreter:
>>> def func(*):
... pass
...
File "<stdin>", line 1
SyntaxError: named arguments must follow bare *
If it matters, I'm on python 3.3.0.
Bare * is used to force the caller to use named arguments - so you cannot define a function with * as an argument when you have no following keyword arguments.
See this answer or Python 3 documentation for more details.
While the original answer answers the question completely, just adding a bit of related information. The behaviour for the single asterisk derives from PEP-3102. Quoting the related section:
The second syntactical change is to allow the argument name to
be omitted for a varargs argument. The meaning of this is to
allow for keyword-only arguments for functions that would not
otherwise take a varargs argument:
def compare(a, b, *, key=None):
...
In simple english, it means that to pass the value for key, you will need to explicitly pass it as key="value".
def func(*, a, b):
print(a)
print(b)
func("gg") # TypeError: func() takes 0 positional arguments but 1 was given
func(a="gg") # TypeError: func() missing 1 required keyword-only argument: 'b'
func(a="aa", b="bb", c="cc") # TypeError: func() got an unexpected keyword argument 'c'
func(a="aa", b="bb", "cc") # SyntaxError: positional argument follows keyword argument
func(a="aa", b="bb") # aa, bb
the above example with **kwargs
def func(*, a, b, **kwargs):
print(a)
print(b)
print(kwargs)
func(a="aa",b="bb", c="cc") # aa, bb, {'c': 'cc'}
Semantically, it means the arguments following it are keyword-only, so you will get an error if you try to provide an argument without specifying its name. For example:
>>> def f(a, *, b):
... return a + b
...
>>> f(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() takes 1 positional argument but 2 were given
>>> f(1, b=2)
3
Pragmatically, it means you have to call the function with a keyword argument. It's usually done when it would be hard to understand the purpose of the argument without the hint given by the argument's name.
Compare e.g. sorted(nums, reverse=True) vs. if you wrote sorted(nums, True). The latter would be much less readable, so the Python developers chose to make you to write it the former way.
Suppose you have function:
def sum(a,key=5):
return a + key
You can call this function in 2 ways:
sum(1,2) or sum(1,key=2)
Suppose you want function sum to be called only using keyword arguments.
You add * to the function parameter list to mark the end of positional arguments.
So function defined as:
def sum(a,*,key=5):
return a + key
may be called only using sum(1,key=2)
I've found the following link to be very helpful explaining *, *args and **kwargs:
https://pythontips.com/2013/08/04/args-and-kwargs-in-python-explained/
Essentially, in addition to the answers above, I've learned from the site above (credit: https://pythontips.com/author/yasoob008/) the following:
With the demonstration function defined first below, there are two examples, one with *args and one with **kwargs
def test_args_kwargs(arg1, arg2, arg3):
print "arg1:", arg1
print "arg2:", arg2
print "arg3:", arg3
# first with *args
>>> args = ("two", 3,5)
>>> test_args_kwargs(*args)
arg1: two
arg2: 3
arg3: 5
# now with **kwargs:
>>> kwargs = {"arg3": 3, "arg2": "two","arg1":5}
>>> test_args_kwargs(**kwargs)
arg1: 5
arg2: two
arg3: 3
So *args allows you to dynamically build a list of arguments that will be taken in the order in which they are fed, whereas **kwargs can enable the passing of NAMED arguments, and can be processed by NAME accordingly (irrespective of the order in which they are fed).
The site continues, noting that the correct ordering of arguments should be:
some_func(fargs,*args,**kwargs)
In Python, is it possible to redefine the default parameters of a function at runtime?
I defined a function with 3 parameters here:
def multiplyNumbers(x,y,z):
return x*y*z
print(multiplyNumbers(x=2,y=3,z=3))
Next, I tried (unsuccessfully) to set the default parameter value for y, and then I tried calling the function without the parameter y:
multiplyNumbers.y = 2;
print(multiplyNumbers(x=3, z=3))
But the following error was produced, since the default value of y was not set correctly:
TypeError: multiplyNumbers() missing 1 required positional argument: 'y'
Is it possible to redefine the default parameters of a function at runtime, as I'm attempting to do here?
Just use functools.partial
multiplyNumbers = functools.partial(multiplyNumbers, y = 42)
One problem here: you will not be able to call it as multiplyNumbers(5, 7, 9); you should manually say y=7
If you need to remove default arguments I see two ways:
Store original function somewhere
oldF = f
f = functools.partial(f, y = 42)
//work with changed f
f = oldF //restore
use partial.func
f = f.func //go to previous version.
Technically, it is possible to do what you ask… but it's not a good idea. RiaD's answer is the Pythonic way to do this.
In Python 3:
>>> def f(x=1, y=2, z=3):
... print(x, y, z)
>>> f()
1 2 3
>>> f.__defaults__ = (4, 5, 6)
4 5 6
As with everything else that's under the covers and hard to find in the docs, the inspect module chart is the best place to look for function attributes.
The details are slightly different in Python 2, but the idea is the same. (Just change the pulldown at the top left of the docs page from 3.3 to 2.7.)
If you're wondering how Python knows which defaults go with which arguments when it's just got a tuple… it just counts backward from the end (or the first of *, *args, **kwargs—anything after that goes into the __kwdefaults__ dict instead). f.__defaults = (4, 5) will set the defaults to y and z to 4 and 5, and with default for x. That works because you can't have non-defaulted parameters after defaulted parameters.
There are some cases where this won't work, but even then, you can immutably copy it to a new function with different defaults:
>>> f2 = types.FunctionType(f.__code__, f.__globals__, f.__name__,
... (4, 5, 6), f.__closure__)
Here, the types module documentation doesn't really explain anything, but help(types.FunctionType) in the interactive interpreter shows the params you need.
The only case you can't handle is a builtin function. But they generally don't have actual defaults anyway; instead, they fake something similar in the C API.
yes, you can accomplish this by modifying the function's func.__defaults__ tuple
that attribute is a tuple of the default values for each argument of the function.
for example, to make pandas.read_csv always use sep='\t', you could do:
import inspect
import pandas as pd
default_args = inspect.getfullargspec(pd.read_csv).args
default_arg_values = list(pd.read_csv.__defaults__)
default_arg_values[default_args.index("sep")] = '\t'
pd.read_csv.__defaults__ = tuple(default_arg_values)
use func_defaults as in
def myfun(a=3):
return a
myfun.func_defaults = (4,)
b = myfun()
assert b == 4
check the docs for func_defaults here
UPDATE: looking at RiaD's response I think I was too literal with mine. I don't know the context from where you're asking this question but in general (and following the Zen of Python) I believe working with partial applications is a better option than redefining a function's defaults arguments