Develop Reference

Get a dictionary of current function parameters

I recently had to use a function conditionally dispatching tasks to other functions, with a lot of mandatory and optional named arguments (e.g. manipulating connection strings, spark connectors configs and so on), and it occurred to me that It would have been really much "cleaner" (or "pythonesque") to have a syntax allowing me to pass every arguments from a function to another similar to this :
def sisterFunction(**kwargs) : # Doing things with a bunch of mandatory and optional args
<do various things/>
def motherFunction(a,b,**kwargs) :
<do various things/>
sisterFunction(**allArgs)
where allArgs would be a dictionary containing keys a,b, and everything in kwargs. This sounds like something python would be inclined to allow and ease but I can't seem to find something similar to a "super kwargs" implemented. Is there a straightforward way to do this ? Is there an obvious good reason it's not a thing ?

def sisterFunction(**kwargs):
pass
def motherFunction(a, b, **kwargs):
sisterFunction(a=a, b=b, **kwargs)
kwargs in sisterFunction will contain a and b keys with corresponding values.
UPDATE
If you don't want to pass long list of function parameters via a=a, there is some workaround to get allArgs:
def motherFunction(a, b, **kwargs):
allArgs = locals().copy()
allArgs.update(allArgs.pop('kwargs', {}))
sisterFunction(**allArgs)

I would probably go with just using kwargs
def sisterFunction(**kwargs):
pass
def motherFunction(**kwargs):
# use the values directly from 'kwargs'
print(kwargs['a'])
# or assign them to local variables for this function
b = kwargs['b']
sisterFunction(**kwargs)
This will probably be the option with the least code in your function signatures (the definitions of all the parameters to the function).
A KeyError will be raised if some parameters were not passed to the function and the function tries to use them.

Python function loses identity after being decorated

(Python 3)
First of all, I feel my title isn't quite what it should be, so if you stick through the question and come up with a better title, please feel free to edit it.
I have recently learned about Python Decorators and Python Annotations, and so I wrote two little functions to test what I have recently learned.
One of them, called wraps is supposed to mimic the behaviour of the functools wraps, while the other, called ensure_types is supposed to check, for a given function and through its annotations, if the arguments passed to some function are the correct ones.
This is the code I have for those functions:
def wraps(original_func):
"""Update the decorated function with some important attributes from the
one that was decorated so as not to lose good information"""
def update_attrs(new_func):
# Update the __annotations__
for key, value in original_func.__annotations__.items():
new_func.__annotations__[key] = value
# Update the __dict__
for key, value in original_func.__dict__.items():
new_func.__dict__[key] = value
# Copy the __name__
new_func.__name__ = original_func.__name__
# Copy the docstring (__doc__)
new_func.__doc__ = original_func.__doc__
return new_func
return update_attrs # return the decorator
def ensure_types(f):
"""Uses f.__annotations__ to check the expected types for the function's
arguments. Raises a TypeError if there is no match.
If an argument has no annotation, object is returned and so, regardless of
the argument passed, isinstance(arg, object) evaluates to True"""
#wraps(f) # say that test_types is wrapping f
def test_types(*args, **kwargs):
# Loop through the positional args, get their name and check the type
for i in range(len(args)):
# function.__code__.co_varnames is a tuple with the names of the
##arguments in the order they are in the function def statement
var_name = f.__code__.co_varnames[i]
if not(isinstance(args[i], f.__annotations__.get(var_name, object))):
raise TypeError("Bad type for function argument named '{}'".format(var_name))
# Loop through the named args, get their value and check the type
for key in kwargs.keys():
if not(isinstance(kwargs[key], f.__annotations__.get(key, object))):
raise TypeError("Bad type for function argument named '{}'".format(key))
return f(*args, **kwargs)
return test_types
Supposedly, everything is alright until now. Both the wraps and the ensure_types are supposed to be used as decorators. The problem comes when I defined a third decorator, debug_dec that is supposed to print to the console when a function is called and its arguments. The function:
def debug_dec(f):
"""Does some annoying printing for debugging purposes"""
#wraps(f)
def profiler(*args, **kwargs):
print("{} function called:".format(f.__name__))
print("\tArgs: {}".format(args))
print("\tKwargs: {}".format(kwargs))
return f(*args, **kwargs)
return profiler
That also works cooly. The problem comes when I try to use debug_dec and ensure_types at the same time.
#ensure_types
#debug_dec
def testing(x: str, y: str = "lol"):
print(x)
print(y)
testing("hahaha", 3) # raises no TypeError as expected
But if I change the order with which the decorators are called, it works just fine.
Can someone please help me understand what is going wrong, and if is there any way of solving the problem besides swapping those two lines?
EDIT
If I add the lines:
print(testing.__annotations__)
print(testing.__code__.co_varnames)
The output is as follows:
#{'y': <class 'str'>, 'x': <class 'str'>}
#('args', 'kwargs', 'i', 'var_name', 'key')

Although wraps maintains the annotations, it doesn't maintain the function signature. You see this when you print out the co_varnames. Since ensure_types does its checking by comparing the names of the arguments with the names in the annotation dict, it fails to match them up, because the wrapped function has no arguments named x and y (it just accepts generic *args and **kwargs).
You could try using the decorator module, which lets you write decorators that act like functools.wrap but also preserve the function signature (including annotations).
There is probably also a way to make it work "manually", but it would be a bit of a pain. Basically what you would have to do is have wraps store the original functions argspec (the names of its arguments), then have ensure_dict use this stored argspec instead of the wrapper's argspec in checking the types. Essentially your decorators would pass the argspec in parallel with the wrapped functions. However, using decorator is probably easier.

why doesn't ** unpack kwargs in function calls?

This is something that's bugged me for awhile now:
def test (*args, **kwargs):
print target
test(foo='bar', target='baz')
I would presume that target='test' in the aFunc call at the bottom would end up in kwargs (and it does), and I would also presume that **would unpack kwargs in the function call, so target would exist as a keyword argument inside of aFunc. It doesn't. I know that it comes in as a dict, but I need to have that dict unpack in the argument list. Is this possible? In short, is there any way to have *args and **kwargs disappear and have the actual args and kwargs go into the call?
Edit: I threw together a case where unpacking of *args and **kwargs might help:
Let's say I have a function that prints a list:
def printList (inputList=None):
print inputList
I want to be able to pass no list and have a default list supplied:
def ensureList (listFunc):
def wrapper (inputList=None):
listFunc(inputList=inputList or ['a','default','list'])
return wrapper
#ensureList
def printList (inputList=None):
print inputList
Now I want to get a bit more complicated with a list repeater:
#ensureList
def repeatList (inputList=None):
print inputList*2
That works fine. But now I want variable repeating:
#ensureList
def repeatList (times, inputList=None):
print inputList*times
Now you would be able to say:
repeatList(5)
It would generate the default list and repeat it 5 times.
This fails, of course, because wrapper can't handle the times argument. I could of course do this:
#ensureList
def repeatList (inputList=None, times=1)
But then I always have to do this:
repeatList(times=5)
And maybe in some cases I want to enforce supplying a value, so a non-keyword arg makes sense.
When I first encountered problems like this last year, I thought a simple solution would be to remove the requirements on the wrapper:
def ensureList (listFunc):
"info here re: operating on/requiring an inputList keyword arg"
def wrapper (*args, **kwargs):
listFunc(inputList=inputList or ['a','default','list'])
return wrapper
That doesn't work, though. This is why I'd like to have args and kwargs actually expand, or I'd like to have a way to do the expansion. Then whatever args and kwargs I supply, they actually fill in the arguments, and not a list and a dict. The documentation in the wrapper would explain requirements. If you pass in inputList, it would actually go in, and inputList in the call back to repeatList from the wrapper would be valid. If you didn't pass in inputList, it would create it in the call back to repeatList with a default list. If your function didn't care, but used *kwargs, it would just gracefully accept it without issue.
Apologies if any of the above is wrong (beyond the general concept). I typed it out in here, untested, and it's very late.

The answer to "why doesn't ** unpack kwargs in function calls?" is: Because it's a bad idea, the person who develop a function does not want local variable to just appear depending on the call arguments.
So, this is not how it's working and you surely do not want python to behave like that.
To access the target variable in the function, you can either use:
def test(target='<default-value>', *args, **kwargs):
print target
or
def test(*args, **kwargs):
target = kwargs.get('target', '<default-value>')
print target
However, if you want a hack (educational usage only) to unpack **kwargs, you can try that:
def test(*args, **kwargs):
for i in kwargs:
exec('%s = %s' % (i, repr(kwargs[i])))
print target

The obvious way for this particular case is
def test(foo=None, target=None):
print target
test(foo='bar', target='baz')
If you want to access a parameter inside a function by name, name it explicitly in the argument list.

Python required variable style

What is the best style for a Python method that requires the keyword argument 'required_arg':
def test_method(required_arg, *args, **kwargs):
def test_method(*args, **kwargs):
required_arg = kwargs.pop('required_arg')
if kwargs:
raise ValueError('Unexpected keyword arguments: %s' % kwargs)
Or something else? I want to use this for all my methods in the future so I'm kind of looking for the best practices way to deal with required keyword arguments in Python methods.

The first method by far. Why duplicate something the language already provides for you?
Optional arguments in most cases should be known (only use *args and **kwargs when there is no possible way of knowing the arguments). Denote optional arguments by giving them their default value (def bar(foo = 0) or def bar(foo = None)). Watch out for the classic gotcha of def bar(foo = []) which doesn't do what you expect.

The first method offers you the opportunity to give your required argument a meaningful name; using *args doesn't. Using *args is great when you need it, but why give up the opportunity for clearer expression of your intent?

If you don't want arbitrary keyword arguments, leave out the ** parameter. For the love of all that is holy, if you have something that is required, just make it a normal argument.
Instead of this:
def test_method(*args, **kwargs):
required_arg = kwargs.pop('required_arg')
if kwargs:
raise ValueError('Unexpected keyword arguments: %s' % kwargs)
Do this:
def test_method(required_arg, *args):
pass

Dynamic Keyword Arguments in Python?

Does python have the ability to create dynamic keywords?
For example:
qset.filter(min_price__usd__range=(min_price, max_price))
I want to be able to change the usd part based on a selected currency.

Yes, It does. Use **kwargs in a function definition.
Example:
def f(**kwargs):
print kwargs.keys()
f(a=2, b="b") # -> ['a', 'b']
f(**{'d'+'e': 1}) # -> ['de']
But why do you need that?

If I understand what you're asking correctly,
qset.filter(**{
'min_price_' + selected_currency + '_range' :
(min_price, max_price)})
does what you need.

You can easily do this by declaring your function like this:
def filter(**kwargs):
your function will now be passed a dictionary called kwargs that contains the keywords and values passed to your function. Note that, syntactically, the word kwargs is meaningless; the ** is what causes the dynamic keyword behavior.
You can also do the reverse. If you are calling a function, and you have a dictionary that corresponds to the arguments, you can do
someFunction(**theDictionary)
There is also the lesser used *foo variant, which causes you to receive an array of arguments. This is similar to normal C vararg arrays.

Yes, sort of.
In your filter method you can declare a wildcard variable that collects all the unknown keyword arguments. Your method might look like this:
def filter(self, **kwargs):
for key,value in kwargs:
if key.startswith('min_price__') and key.endswith('__range'):
currency = key.replace('min_price__', '').replace('__range','')
rate = self.current_conversion_rates[currency]
self.setCurrencyRange(value[0]*rate, value[1]*rate)