Develop Reference

passing **kwargs to a de-referencing function

Hope the title is conveying the correct information.
My problem is that I don't understand why call kwarg_function(some_func, a=1, b=2, c=3) fails. I would have thought that as 'c' isn't referenced with some_func() it would simply be ignored. Can anyone explain why 'c' isn't simply ignored.
def kwarg_function(function, **kwargs):
print(kwargs)
function(**kwargs)
def some_func(a, b):
print(f"type: {type(a)} values: {a}")
print(f"type: {type(b)} values: {b}")
kwarg_function(some_func, a=1, b=2) # called successfully
kwarg_function(some_func, a=1, b=2, c=3) # fails with unexpected keyword arg 'c'

Think of the ** as "unpack what's on my right side as keyword arguments" in this case.
def foo(a,b,**kwargs):
# This will take any number of arguments provided that starting from the 3rd one they are keyword args
# Those are equivalent (and working) calls
foo(1,2, x = 7)
foo(1,2, **{"x":7})
# Those will fail
foo(1,2,7)
foo(1,2, {"x":7})
The function you declared expects 2 arguments
def some_func(a, b):
And you are calling it with three under the hood, because this:
kwarg_function(some_func, a=1, b=2, c=3) # fails with unexpected keyword arg 'c'
Does this (inside kwarg_function body):
funtion(a=1,b=2,c=3)

In python, * and ** are for unpacking iterables. They don't consider what's are in them, and just unpack whatever you pass in them.
You can find more info about it in this link.
So, when you pass a=1, b=2, c=3, ... as kwargs to your kwargs_function, you will get them as kwargs param, regardless of what you have passed.
And then, when you pass **kwargs to another function, all of your data would be passed to your another function, regardless of what's in that.
If you want your some_func be more flexible with your data and accept whatever you pass to it, you can add **kwargs param to it too:
def some_func(a, b, **kwargs):
print(f"type: {type(a)} values: {a}")
print(f"type: {type(b)} values: {b}")

The rationale of `functools.partial` behavior

I'm wondering what the story -- whether sound design or inherited legacy -- is behind these functools.partial and inspect.signature facts (talking python 3.8 here).
Set up:
from functools import partial
from inspect import signature
def bar(a, b):
return a / b
All starts well with the following, which seems compliant with curry-standards.
We're fixing a to 3 positionally, a disappears from the signature and it's value is indeed bound to 3:
f = partial(bar, 3)
assert str(signature(f)) == '(b)'
assert f(6) == 0.5 == f(b=6)
If we try to specify an alternate value for a, f won't tell us that we got an unexpected keyword, but rather that it got multiple values for argument a:
f(a=2, b=6) # TypeError: bar() got multiple values for argument 'a'
f(c=2, b=6) # TypeError: bar() got an unexpected keyword argument 'c'
But now if we fix b=3 through a keyword, b is not removed from the signature, it's kind changes to keyword-only, and we can still use it (overwrite the default, as a normal default, which we couldn't do with a in the previous case):
f = partial(bar, b=3)
assert str(signature(f)) == '(a, *, b=3)'
assert f(6) == 2.0 == f(6, b=3)
assert f(6, b=1) == 6.0
Why such asymmetry?
It gets even stranger, we can do this:
f = partial(bar, a=3)
assert str(signature(f)) == '(*, a=3, b)' # whaaa?! non-default argument follows default argument?
Fine: For keyword-only arguments, there can be no confusing of what parameter a default is assigned to, but I still wonder what design-thinking or constraints are behind these choices.

Using partial with a Positional Argument
f = partial(bar, 3)
By design, upon calling a function, positional arguments are assigned first. Then logically, 3 should be assigned to a with partial. It makes sense to remove it from the signature as there is no way to assign anything to it again!
when you have f(a=2, b=6), you are actually doing
bar(3, a=2, b=6)
when you have f(2, 2), you are actually doing
bar (3, 2, 2)
We never get rid of 3
For the new partial function:
We can't give a a different value with another positional argument
We can't use the keyword a to assign a different value to it as it is already "filled"
If there is a parameter with the same name as the keyword, then the argument value is assigned to that parameter slot. However, if the parameter slot is already filled, then that is an error.
I recommend reading the function calling behavior section of pep-3102 to get a better grasp of this matter.
Using partial with a Keyword Argument
f = partial(bar, b=3)
This is a different use case. We are applying a keyword argument to bar.
You are functionally turning
def bar(a, b):
...
into
def f(a, *, b=3):
...
where b becomes a keyword-only argument
instead of
def f(a, b=3):
...
inspect.signature correctly reflects a design decision of partial. The keyword arguments passed to partial are designed to append additional positional arguments (source).
Note that this behavior does not necessarily override the keyword arguments supplied with f = partial(bar, b=3), i.e., b=3 will be applied regardless of whether you supply the second positional argument or not (and there will be a TypeError if you do so). This is different from a positional argument with a default value.
>>> f(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() takes 1 positional argument but 2 were given
where f(1, 2) is equivalent to bar(1, 2, b=3)
The only way to override it is with a keyword argument
>>> f(2, b=2)
An argument that can only be assigned with a keyword but positionally? This is a keyword-only argument. Thus (a, *, b=3) instead of (a, b=3).
The Rationale of Non-default Argument follows Default Argument
f = partial(bar, a=3)
assert str(signature(f)) == '(*, a=3, b)' # whaaa?! non-default argument follows default argument?
You can't do def bar(a=3, b). a and b are so called positional-or-keyword arguments.
You can do def bar(*, a=3, b). a and b are keyword-only arguments.
Even though semantically, a has a default value and thus it is optional, we can't leave it unassigned because b, which is a positional-or-keyword argument needs to be assigned a value if we want to use b positionally. If we do not supply a value for a, we have to use b as a keyword argument.
Checkmate! There is no way for b to be a positional-or-keyword argument as we intended.
The PEP for positonal-only arguments also kind of shows the rationale behind it.
This also has something to do with the aforementioned "function calling behavior".
partial != Currying & Implementation Details
partial by its implementation wraps the original function while storing the fixed arguments you passed to it.
IT IS NOT IMPLEMENTED WITH CURRYING. It is rather partial application instead of currying in the sense of functional programming. partial is essentially applying the fixed arguments first, then the arguments you called with the wrapper:
def __call__(self, /, *args, **keywords):
keywords = {**self.keywords, **keywords}
return self.func(*self.args, *args, **keywords)
This explains f(a=2, b=6) # TypeError: bar() got multiple values for argument 'a'.
See also: Why is partial called partial instead of curry
Under the Hood of inspect
The outputs of inspect is another story.
inspect itself is a tool that produces user-friendly outputs. For partial() in particular (and partialmethod(), similarly), it follows the wrapped function while taking the fixed parameters into account:
if isinstance(obj, functools.partial):
wrapped_sig = _get_signature_of(obj.func)
return _signature_get_partial(wrapped_sig, obj)
Do note that it is not inspect.signature's goal to show you the actual signature of the wrapped function in the AST.
def _signature_get_partial(wrapped_sig, partial, extra_args=()):
"""Private helper to calculate how 'wrapped_sig' signature will
look like after applying a 'functools.partial' object (or alike)
on it.
"""
...
So we have a nice and ideal signature for f = partial(bar, 3)
but get f(a=2, b=6) # TypeError: bar() got multiple values for argument 'a' in reality.
Follow-up
If you want currying so badly, how do you implement it in Python, in the way which gives you the expected TypeError?

When you provide positional or keyword arguments to partial, the new function is constructed
f = partial(bar, 3)
f(a=2, b=6) # TypeError: bar() got multiple values for argument 'a'
f(c=2, b=6) # TypeError: bar() got an unexpected keyword argument 'c'
This is actually consistent with the idea of partial, which is that arguments are passed to the wrapped function with the addition of positional and keyword arguments passed to partial
These cases behave as expected:
bar(3, a=2, b=6) # TypeError: bar() got multiple values for argument 'a'
bar(3, c=2, b=6) # TypeError: bar() got an unexpected keyword argument 'c'
But now if we fix b=3 through a keyword, b is not removed from the signature,
f = partial(bar, b=3)
assert str(signature(f)) == '(a, *, b=3)'
assert f(6) == 2.0 == f(6, b=3)
assert f(6, b=1) == 6.0
This case is different from the above because in the previous case, a positional argument was provided to partial, not a keyword argument. When positional arguments are provided to partial, then it makes sense to remove them from the signature. Arguments provided as keywords are not removed from the signature.
So far, there is no inconsistency or asymmetry.
f = partial(bar, a=3)
assert str(signature(f)) == '(*, a=3, b)' # whaaa?! non-default argument follows default argument?
The signature here makes sense and is the expectation for partial(bar, a=3) -- it works the same as def f(*, a=3, b): ... and is the correct signature in this case. Note that when you provide a=3 to partial in this case, a becomes a keyword-only argument, as does b.
This is because when a positional argument is provided as a keyword, all following arguments must be specified keyword arguments.
sig = signature(f)
sig.parameters['a'].kind # <_ParameterKind.KEYWORD_ONLY: 3>
inspect.getfullargspec(f)
# FullArgSpec(args=[], varargs=None, varkw=None, defaults=None, kwonlyargs=['a', 'b'], kwonlydefaults={'a': 3}, annotations={})

Why Does Python Allow *args After Keyword Arguments?

Example:
def foo(a, b=2, *args, **kwargs): pass
Why does this not result in a SyntaxError? *args will not catch additional non-keyword arguments because it is illegal to pass them after keyword arguments.
For python3.x the correct use of *args, **kwargs in this case looks like:
def foo(a, *args, b=2, **kwargs): pass
Thanks for any insights into this curious behavior.
Edit:
Thanks to Jab for pointing me to PEP 3102, which explains this behavior concisely. Check it out!
And also thanks to jsbueno for the additional excellent explanation, which I am updating as the best answer due to its thoroughness.

Given:
def foo(a, b=2, *args, **kwargs): pass
b is not a keyword-only parameter - it is just a parameter for which arguments can be positional or named, but have a default value. It is not possible to pass any value into args and omit passing b or passing b out of order in the signature you suggest.
This signature makes sense and is quite unambiguous - you can pass from 0 to n positional arguments, but if you pass 2 or more, the second argument is assigned to "b", and end of story.
If you pass 0 positional arguments, you can still assign values to "a" or "b" as named arguments, but trying anything like: foo(0, 1, 2, a=3, b=4) will fail as more than one value is attempted to be passed to both parameters.
Where as in:
def foo(a, *args, b=2, **kwargs): pass
it is also an unambiguous situation: the first positional argument goes to "a", the others go to "args", and you can only pass a value to "b" as a named argument.
The new / syntax in signature definition coming with Python 3.8 gives more flexibility to this, allowing one to require that "a" and "b" are passed as positional-only arguments. Again, there is no ambiguity:
def foo(a, b=2, /, *args, **kwargs): pass
A curious thing on this new syntax: one is allowed to pass named arguments to "a" and "b", but the named arguments will come up as key/value pairs inside "kwargs" - while the local variables "a" and "b" will be assigned the positional only arguments:
def foo(a, b=2, /, *args, **kwargs):
print(a, b, args, kwargs)
...
In [9]: foo(1, 2, a=3, b=4)
1 2 () {'a': 3, 'b': 4}
Whereas with the traditional syntax you ask about - def foo(a, b=2, *args, **kwargs): - one gets a TypeError if that is tried:
In [11]: foo(1,2, a=3, b=4)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-11-d002c7717dba> in <module>
----> 1 foo(1,2, a=3, b=4)
TypeError: foo() got multiple values for argument 'a'

This was implemented into 3.X for multiple reasons. Best way I can answer this is refer to
PEP 3102
Also take a look at the New Syntax section in the Python 3.0.1 docs.
TLDR:
Named parameters occurring after
*args in the parameter list must be specified using keyword syntax in the call. You can also use a bare * in the parameter list to indicate
that you don’t accept a variable-length argument list, but you do have
keyword-only arguments.

How to create a python function which take only positional arguments and no keyword arguments?

A lot of inbuilt functions in python don't take keyword arguments. For example, the chr function.
>>> help(chr)
Help on built-in function chr in module builtins:
chr(i, /)
Return a Unicode string of one character with ordinal i; 0 <= i <= 0x10ffff.
Trying to pass values to chr using keyword arguments don't work.
>>> chr(i=65)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: chr() takes no keyword arguments
I know that the / character in the help text of the chr function means that it won't take keyword arguments.
How can I define a function that does not take keyword arguments? And of course, I want to define a function that takes arguments, but only positional arguments.
This will probably be marked as a duplicate but at least that way I'll get the answer. I can't find a StackOverflow answer for this question.
Another similar feature I learnt is to create a function that does not take positional arguments.
>>> def f(*, a, b):
... print(a, b)
...
>>> f(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() takes 0 positional arguments but 2 were given
>>> f(a=1, b=2)
1 2
This question is similar to mine, but it doesn't actually answer my question. I still don't know how to define a function that will not accept keyword arguments, like several of the built-in functions.

There's PEP 570, which is only a draft, so one cannot create positional-only arguments in pure Python. This can, however, be done in a function written in C for Python.

Seeing as how the previous answer never got updated to 3.8 here's a brief answer for future readers
the / character in a function declaration marks all arguments before as positional only
def func(a, b, /):
print(a ** b)
func(2, 4) # safe
func(a=2, b=4) # got some positional-only arguments passed as keyword arguments: 'a, b'
and the * character in a function declaration marks all arguments after as keyword only
def func(*, a, b):
print(a ** b)
func(a=2, b=4) # safe
func(2, 4) # takes 0 positional arguments but 2 were given
these declarations can be combined to create a function with all three options
positional only-
default(both)- keyword only
def func(a, b, /, c, *, d, e):
pass #too lazy to think of a use case
func(1, 2, 3, d=4, e=5) # safe
func(6, 7, c=8, d=9, e=10) # safe
func(1, b=2, c=3, d=4, e=5) # got some positional-only arguments passed as keyword arguments: 'b'
# etc

Force function caller decide which default value is used

Lets say I have the following function:
def myFun(a=None, b=None):
print a, b
Now I call the function:
myFun("hi") # Case 1
>>>hi None
myFun(b="hi") # Case 2
>>>None hi
myFun(a="hi") # Case 3
>>>hi None
Is there a way to throw an exception if the function caller did not decide which variable the value "hi" is assigned to? That means I would like to have an exception in the 1st case but not in the 2nd and 3rd case. I use python 2.7.

In Python3 you can specify keyword-only arguments:
def myFun(*, a=None, b=None):
print(a, b)
myFun('hi')
raises
TypeError: myFun() takes 0 positional arguments but 1 was given
In Python2, as Ricardo Silveira points out you could use **kwargs to force all arguments to be keyword arguments.
def myFun(**kwargs):
a, b = kwargs.get('a'), kwargs.get('b')
print(a, b)
myFun('hi')
# TypeError: myFun() takes exactly 0 arguments (1 given)

def my_fun(**kwargs):
a = kwargs.get("a", None)
b = kwargs.get("b", None)
# have your code here...
Then: my_fun("hi") shouldn't work...

You can actually allow the user to provide any arguments they'd like by using **kwargs.
def myFun(**kwargs):
print kwargs[a]
print kwargs[b]
This will cause an error if a or b aren't defined, but they're not very helpful.
You can make your own errors by checking if the value exists. For example:
def myFun(**kwargs):
if not kwargs.get(a):
raise Exception('a is not here!')
if not kwargs.get(b):
raise Exception ('b is not here!')
print kwargs[a], kwargs[b]