Related
When a function is called, I would like to print (or log) the names and values of the parameters it was called with. Something like:
>>> def foo(bar, baz):
>>> do_something()
>>> foo(5, 'test')
# would print: foo(bar=5, baz='test')
I can manually add print/logging calls to each function, but this is a pain as the functions I am concerned with grow/change over time and are part of an abstract interface that can be extended in other places.
This answer mentions how to get the parameter names, but doesn't mention how to get their values when called. I also found this answer which uses eval and works on a known set of parameters. I'd really like to avoid eval and be able to do it for an arbitrary set of parameters.
For context, this code is used for debugging a hardware interface. When the hardware isn't present it prints (or logs) the IO requests instead of sending them to the hardware.
After you've add explanation how you want to apply this, I think best way will be to use decorator. It's more universal solution, because you can add it to any function in your code and it will print all debug info if debug mode is on.
Code:
from functools import wraps
DEBUG = True
def debug_log(function):
#wraps(function)
def wrapper(*args, **kwargs):
if DEBUG:
print(">> Called", function.__name__, "\n",
{**dict(zip(function.__code__.co_varnames, args)), **kwargs})
result = function(*args, **kwargs)
if DEBUG:
print(">>", function.__name__, "return:\n", result)
return result
return wrapper
#debug_log
def first_example(a, b, c):
return 100
#debug_log
def second_example(d, e, f):
return 200
first_example(10, 11, 12)
first_example(c=12, a=10, b=11)
second_example(13, 14, 15)
second_example(e=14, d=13, f=15)
DEBUG = False
first_example(0, 0, 0)
second_example(1, 1, 1)
Output:
>> Called first_example
{'a': 10, 'b': 11, 'c': 12}
>> first_example return:
100
>> Called first_example
{'c': 12, 'a': 10, 'b': 11}
>> first_example return:
100
>> Called second_example
{'d': 13, 'e': 14, 'f': 15}
>> second_example return:
200
>> Called second_example
{'e': 14, 'd': 13, 'f': 15}
>> second_example return:
200
You could try to use func.__code__.co_varnames like this:
def foo(bar, baz):
magic_parameter_printing()
bar=0
baz=None
for var in foo.__code__.co_varnames:
print(var,'= ',eval(var))
Output:
bar=0
baz=None
We have a somewhat flawed answer, but it is different than some of the other answers I have seen. Hopefully, someone can post an even better answer, because my solution to the problem has some serious issues.
import functools
import inspect
def print_calling_args(f):
assert(callable(f))
#functools.wraps(f)
def wrapper(*args, **kwargs):
siggy = inspect.signature(f)
bound_siggy = siggy.bind(*args, **kwargs)
all_sargs = ", ".join(kw + " = " + str(arg) for kw, arg in bound_siggy.arguments.items())
print(f.__qualname__, "(", all_sargs, ")", sep = "")
return f(*args, **kwargs)
return wrapper
Issue: str(arg) is bad way to convert arguments into strings
I converted each argument arg into a string by writing str(arg)
Suppose that you pass a pandas dataframe into a function.
We can get some very ugly output.
The resulting string can contain new-line characters \n or carriage returns \r.
The strings could also be a mile long.
Issue: Cannot Get the Name of a Functor
Another issue with my attempts is that Functors (also known as "function objects") do not have an attribute named __qualname__. That is, if you define a class which has a __call__ method, then we cannot easily print the name of the functor to a string.
funky = Functor()
funky = decorate(funky)
r = funky(1, 2, 3, kwargy = int("python", 36))
# should print:
# funky(1, 2, 3 kwargy = 1570137287)
Below is an example of a functor:
# EXAMPLE OF A `FUNCTOR`
class Multiplier:
def __init__(self, m:int):
self._m = int(str(m))
def __call__(self, num:float)
return num * self._m;
multi_the_multiplier = Multiplier(10**6)
# Decorate `multi_the_multiplier` with ` print_calling_args`
multi_the_multiplier = print_calling_args(multi_the_multiplier)
# ERROR: `multi_the_multiplier` does not have a `__qualname__`
r = multi_the_multiplier(1)
r = multi_the_multiplier(2)
r = multi_the_multiplier(3)
When I have a function with bunch of optional arguments
def function_name(arg1=0, arg2=0, arg3=0, arg4=0, ...):
.....doing anything
and I want to pass same value to some of them
function_name(arg1=1, arg3=1, arg6=1)
Is there a more compact way to do it?
so I don't have to keep copying =value and filling up the line.
After several helpful replies I think I came to the answer I was looking for:
def myargs(value, *args):
kargs = {}
for arg in args:
kargs[arg] = value
return kargs
function_name(**myargs(1, 'arg1', 'arg3', 'arg6'))
Thanks guys, sorry if I was bad at explaining what I needed.
I think what you are wanting to do is keyword argument unpacking. You can pass a dictionary to the function and it will unpack the arguments you have included. Anything not included uses the default values.
def test(w=0, x=0, y=0, z=0):
return w+x+y+z
kwargs_1 = {'y': 1}
kwargs_2 = {'x': 2, 'z': 3}
test(**kwargs_1)
# returns:
1
test(**kwargs_2)
# returns:
5
You can use default arguments:
def func(arg1, arg2, def_arg1="1", def_arg2="2"):
pass
You can just use **kwargs
def function_name(arg, argOpt1=0, argOpt2=0, **kwargs):
pass
kwargs = {"argOpt1" : 123, "argOpt2" : 234}
function_name('argument1', kwargs)
Look at this
and this
If the arguments that are to have the same value are contiguous, then:
def function_name(arg1=0,arg2=0,arg3=0,arg4=0):
print(arg1, arg2, arg3, arg4)
function_name(2, *[1] * 3)
Prints:
2 1 1 1
If you didn't write that function, and you can't change its signature, you can pass the right number of arguments that way :
function_name(*(0,) * function_name.__code__.co_argcount)
It will pass the same value (in that case 0) to all the arguments the function expects.
If you need to pass only 10 arguments for instance, you can write :
function_name(*(0,) * 10)
If you want to target specific arguments by their position and not their names, you can use this :
import inspect
def func(a, b, c=None, d='foo', e=False):
print(a, b, c, d, e)
def fill_args(fun, value, arg_numbers):
return {
name: value
for name in (
tuple(inspect.signature(fun).parameters)[i]
for i in arg_numbers
)
}
if __name__ == '__main__':
func(**fill_args(func, 0, (0, 1, 2)))
func(**fill_args(func, 'bar', (0, 1, 4)))
func(**fill_args(func, True, (0, 1, 3)))
What's the best way NOT to have to extracts the SAME KWARGS twice: once in the decorator (wrapper function) and once in the function itself.
Here is the function:
#close_logger
def close(**kwargs):
""" returns close object"""
# Get params
session_attributes = kwargs.get('sessionAttrbiutes', {})
message = kwargs.get('message', '')
LID = kwargs.get('LIData', {})
SANS = kwargs.get('SANS', [])
FS = kwargs.get('fulfillmentState', 'Fulfilled')
response = {
'sessionAttributes': session_attributes,
'dialogAction': {
'type': SANS,
'fulfillmentState': FS,
'message': {
'contentType': LID,
'content': message
}
}
}
return response
and here is the decorator (used for logging the close event):
def close_logger(func):
#functools.wraps(func)
def wrapper(**kwargs):
# Get params
session_attributes = kwargs.get('sessionAttrbiutes', {})
message = kwargs.get('message', '')
LID = kwargs.get('LIData', {})
SANS = kwargs.get('SANS', [])
FS = kwargs.get('fulfillmentState', 'Fulfilled')
logger.debug('Logging:\n Function:{} Session Attributes: {}\n \
Message{}\n: LID: {}\n SANS: {}\n FS: {}'.format(
func.__name__,
session_attributes,
message,
LID,
SANS,
FS
))
return func(**kwargs)
return wrapper
Start by making your keyword arguments explicit for the close function.
def close(sessionAttrbiutes=None, message='', LIData=None, SANS=None, fulfillmentState='Fulfilled'):
if sessionAttrbiutes is None:
sessionAttrbiutes = {}
...
Note that I used None as default values for the mutable default values to avoid a common pitfall.
Then in your decorator use inspect.getfullargspec, similar to this answer:
import inspect
import functools
def get_default_args(func):
"""
returns a dictionary of arg_name:default_values for the input function
"""
argspec = inspect.getfullargspec(func)
return dict(zip(reversed(argspec.args), reversed(argspec.defaults)))
def close_logger(func):
#functools.wraps(func)
def wrapper(**kwargs):
kwargs_local = get_default_args(func)
kwargs_local.update(kwargs)
logger.debug("""Logging:
Function: {}
Session Attributes: {sessionAttrbiutes}
Message: {message}
LID: {LIData}
SANS: {SANS}
FS: {fulfillmentState}""".format(func.__name__, **kwargs_local))
return func(**kwargs)
return wrapper
This will raise a KeyError if one of those fields is not defined (either in the signature or the passed keywords). Of course it will show None for those where that is the default in the signature.
The full argspec will also contain positional arguments, so you might be able to figure those out as well.
**-notation can be used within function argument lists to both pack and unpack arguments.
**-notation within a function definition collects all the values into a dictionary.
>>> def kwargs_in_definition(**kwargs): return kwargs
>>> kwargs_in_definition(arg1 = 1, arg2 = 2, arg3 = 3)
{'arg1': 1, 'arg2': 2, 'arg3': 3}
**-notation within a function call unpacks all the values as keyword argumnts.
def kwargs_in_call(arg1 =0, arg2 = 0, arg3 = 0): return arg1, arg2, arg3
So when you pass kwargs (kwargs := {'arg1': 1, 'arg2': 2, 'arg3': 3}) to kwargs_in_call you get this:
>>> kwargs_in_call(kwargs)
({'arg1': 1, 'arg2': 2, 'arg3': 3}, 0, 0) # kwargs is used as arg1.
...but if you unpack it first:
>>> kwargs_in_call(**kwargs)
(1, 2, 3) # kwargs unapcks to "arg1 = 1, arg2 = 2, arg3 = 3"
All of this applies for * as well
Your specific case:
**kwargs in a function defintion gives you a regular old dict. You can turn that dictionary into a list -- however you wish to do that --, unpack that list and pass it on to another function. You'll have to account for ordering though, so it's difficult to make this work for all functions.
I'm not sure why you insist on your decorator function taking keyword arguments while your decorated function doesn't. You can, of-course, do whatever you want, but I would argue against such an approach, because you're decorator function is changing how your actual function behaves.
If you manage to sort those keyword arguments in to a valid argument list you'll be good.
Hope this helps.
EDIT: Oh, I forgot to point out that you're unpacking **kwargs in your decorator function (return func(**kwargs)) and then you're repacking them in your actual function (def close(**kwargs):. I'll be honest, that's a silly thing to do.
EDIT: You can use inspect.getargspec or inspect.signature to get a functions argument list. If you just have your decorator take in *args, you can match each value to a corresponding name from the argument list.
I am struggling to pass a list of functions with a list of corresponding parameters. I also checked here, but it wasn't very helpful. for example (a naive approach which doesn't work):
def foo(data, functions_list, **kwarg):
for func_i in functions_list:
print func_i(data, **kwarg)
def func_1(data, par_1):
return some_function_1(data, par_1)
def func_2(data, par_2_0, par_2_1):
return some_function_2(data, par_2_0, par_2_1)
foo(data, [func_1, func_2], par_1='some_par', par_2_0=5, par_2_1=11)
Importantly, par_1 cannot be used in func_2, so each function consumes a unique set of parameters.
You could use the function's name as the keyword arguments. When indexing kwargs, you'd use func_i.__name__ as the key.
def foo(data, function_list, **kwargs):
for func_i in function_list:
print(func_i(data, kwargs[func_i.__name__]))
And now,
foo(data, [func_1, func_2], func_1='some_par', func_2=[5, 11])
You could use inspect.getargspec (I assume you use Python 2, you shouldn't use that function in Python 3 because it has been deprecated) to find out which argument names a function has and build a new dictionary based on those:
import inspect
def foo(data, functions_list, **kwargs):
for func_i in functions_list:
newkwargs = {name: kwargs[name]
for name in inspect.getargspec(func_i).args
if name in kwargs}
print(func_i(data, **newkwargs))
def func_1(data, par_1):
return data, par_1
def func_2(data, par_2_0, par_2_1):
return data, par_2_0, par_2_1
>>> data = 10
>>> foo(data, [func_1, func_2], par_1='some_par', par_2_0=5, par_2_1=11)
(10, 'some_par')
(10, 5, 11)
But a better way would be to simply associate parameters with functions that doesn't rely on introspection.
If you want to keep the foo function with that exact same declaration and you don't mind each function receiving the whole set of parameters you could do it like this:
You just need to add to each 'my_*' function the **kwargs parameter.
def foo(data, functions_list, **kwargs):
for my_function in functions_list:
print(my_function(data, **kwargs))
def my_sum(a, b, **kwargs):
return a + b
def my_sub(a, c, **kwargs):
return a - c
foo(0, [my_sum, my_sub], b=3, c=10)
Python automatically parses kwargs setting the b and c parameters where it has the value.
Another approach can be like this:
def foo(data, function_list, **kwargs):
function_dict = {
"func_1": func_1,
"func_2": func_2
}
for func_i in function_list:
print function_dict[func_i](data, **kwargs)
def func_1(data, **arg):
filtered_argument = {key: value for key, value in arg.items() if key.startswith('par_1')}
return list([data, filtered_argument])
def func_2(data, **arg):
filtered_argument = {key: value for key, value in arg.items() if key.startswith('par_2_')}
return list([data, filtered_argument])
data = [1,2,3]
foo(data, ['func_1', 'func_2'], par_1='some_par', par_2_0=5, par_2_1=11)
Output:
[[1, 2, 3], {'par_1': 'some_par'}]
[[1, 2, 3], {'par_2_0': 5, 'par_2_1': 11}]
I am sure that you can improvise your current code as it gets ugly in this way.
I like #COLDSPEED's approach, but want to present yet another solution. Pass always 3 values: function, args, keyword args:
Usage:
foo(
func_1, ('some_par',), {},
func_2, (5, 11), {},
)
Implementation (Python3 syntax):
def foo(*args3):
while args3:
func, args, kwargs, *args3 = args3
func(*args, **kwargs)
An approach would be making the 3rd argument of foo a positional argument and pass in a list of args with functions list:
def foo(data, functions_list, args):
for func, arg in zip(functions_list, args):
print(func(data, arg))
def func1(data, par_1):
return 'func1 called with {}'.format(par_1)
def func2(data, par_2):
return 'func2 called with {}'.format(par_2)
foo('some_data', [func1, func2],
[
{'par_1_1': 11, 'par_1_2': 12},
{'par_2_1': 21, 'par_2_2': 22}
])
zip() is used to map each function with the corresponding args.
Output:
func1 called with {'par_1_1': 11, 'par_1_2': 12}
func2 called with {'par_2_1': 21, 'par_2_2': 22}
You can do it something like that, "close" each parameters for function in a list item and then let "foo" split it backwards:
def foo(data, functions_list, kwarg):
for func_i, args in zip(functions_list, kwarg):
func_i(data, **args)
def func_1(data, par_1):
print("func_1 %s %s" % (data, par_1))
def func_2(data, par_2_0, par_2_1):
print("func_2 %s "
"%s %s" % (data, par_2_0, par_2_1))
data = "Some Data"
foo(data, [func_1, func_2], [{"par_1":'some_par'}, {"par_2_0":5, "par_2_1":11}])
I read a solution in the Python Cookbooks for creating a function that only allows name arguments. I wrote my own code to try it out:
class Reporter(object):
def __init__(self, *, testline=None, sw_ver= None, directory=None):
pass
if __name__ == "__main__"
r = Reporter()
However the interpreter shows this error:
File "Reporter.py", line 6
def __init__(self, *, testline=None, sw_ver= None, directory=None):
^
SyntaxError: invalid syntax
Why is it showing this?
The code you are using is valid syntax but for python3 so the book must be using python3 syntax, it allows keyword arguments only, pep-3102:
python 3 new syntax
You can also use a bare * in the parameter list to indicate that you don’t accept a variable-length argument list, but you do have keyword-only arguments.
Using your code and passing a non keyword in python 3 would error but for a different reason:
TypeError Traceback (most recent call last)
<ipython-input-2-b4df44fa1e0c> in <module>()
1 if __name__ == "__main__":
----> 2 r = Reporter(4)
3
TypeError: __init__() takes 1 positional argument but 2 were given
Once you use a keyword it works fine:
In [4]: if __name__ == "__main__":
r = Reporter(testline=4)
...:
Using a function with the same syntax would maybe give a more obvious error:
def f(*, foo=None, bar=None):
return foo, bar
In [6]: f(4)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-a122d87dbe99> in <module>()
----> 1 f(4)
TypeError: f() takes 0 positional arguments but 1 was given
Is is also useful if you want to allow some positional args and have optional keywords args passed but only by name:
def f(a,b, *, foo=None, bar=None):
return a, b, foo, bar
Then passing 3 positional args will error:
In [8]: f(1,2,3)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-8-b61741968103> in <module>()
----> 1 f(1,2,3)
TypeError: f() takes 2 positional arguments but 3 were given
I received a recent comment about this answer.
→ Please note it targets python2, since OP explicitly tagged his question with python2.7. For python3, see Padraic Cunningham's answer.
[original answer]
You may not use the * alone. In the function declaration, it means "unpack any other unnamed argument in this variable", so you have to give it a variable name.
You can achieve what you want by giving it a name, then checking it is empty, like this:
class Reporter(object):
def __init__(self, *args):
assert not args, "Reporter.__ini__ only accepts named arguments"
Then you'd want to add arguments you can allow, like this:
# won't work
def __init__(self, *args, testline=None, sw_ver= None, directory=None):
...except *args needs to be at the end. But if you put them at the end, you'll see that you can still pass the other arguments unnamed first.
You have to reverse the logic, and take only kwargs.
class Reporter(object):
def __init__(self, **kwargs):
testline = kwargs.pop('testline', None)
sw_ver = kwargs.pop('sw_ver', None)
directory = kwargs.pop('directory', None)
assert not kwargs, 'Unknown arguments: %r' % kwargs
Now, if someone tries to give unnamed argments, they will be rejected.
The star operator (*) is used for unpacking. You can't use it as an argument.
You may want to read Variable-Length Argument Tuples:
Functions can take a variable number of arguments. A parameter name
that begins with * gathers arguments into a tuple. For example,
printall takes any number of arguments and prints them:
def printall(*args):
print args
The gather parameter can have any name you like, but args is
conventional. Here’s how the function works:
>>> printall(1, 2.0, '3') (1, 2.0, '3')
These are a few utility decorators I wrote on a tangent for a Python 2 project I was working on. The exceptions raised mirror, as closely as possible, the ones raised by functions in Python 3 that use the keyword-only arguments syntax.
They don't disallow positional arguments, but they can require/restrict keyword arguments. You could create another decorator that disallowed positional arguments.
import functools
def original_wrapped_function(f):
try:
while True:
f = f.__wrapped__
except AttributeError:
return f
def restrict_kwargs(*allowed_keywords):
def restrict_kwargs_decorator(func):
#functools.wraps(original_wrapped_function(func))
def restrict_wrapper(*args, **kwargs):
for keyword in kwargs:
if keyword not in allowed_keywords:
msg = "%s() got an unexpected keyword argument '%s'"
raise TypeError(msg % (func.__name__, keyword))
return func(*args, **kwargs)
restrict_wrapper.__wrapped__ = func
return restrict_wrapper
return restrict_kwargs_decorator
def require_kwargs(*required_keywords):
def require_kwargs_decorator(func):
#functools.wraps(original_wrapped_function(func))
def require_wrapper(*args, **kwargs):
missing_keywords = []
for keyword in required_keywords:
if keyword not in kwargs:
missing_keywords.append(keyword)
if missing_keywords:
func_name = func.__name__
count = len(missing_keywords)
if count == 1:
arg_word = 'argument'
missing_keywords_str = "'%s'" % missing_keywords[0]
else:
arg_word = 'arguments'
and_join_str = ' and ' if count == 2 else ', and '
missing_keywords_str = ', '.join(
("'%s'" % mk) for mk in missing_keywords[:-1])
missing_keywords_str = and_join_str.join((
missing_keywords_str, ("'%s'" % missing_keywords[-1])))
msg = "%s() missing %d required keyword-only %s: %s"
raise TypeError(msg % (func_name, count, arg_word,
missing_keywords_str))
return func(*args, **kwargs)
require_wrapper.__wrapped__ = func
return require_wrapper
return require_kwargs_decorator
def exact_kwargs(*exact_keywords):
def exact_kwargs_decorator(func):
#restrict_kwargs(*exact_keywords)
#require_kwargs(*exact_keywords)
#functools.wraps(original_wrapped_function(func))
def exact_wrapper(*args, **kwargs):
return func(*args, **kwargs)
exact_wrapper.__wrapped__ = func
return exact_wrapper
return exact_kwargs_decorator
Some examples:
>>> #restrict_kwargs('five', 'six')
... def test_restrict_kwargs(arg1, arg2, *moreargs, **kwargs):
... return (arg1, arg2, moreargs, kwargs)
...
>>>
>>> #require_kwargs('five', 'six')
... def test_require_kwargs(arg1, arg2, *moreargs, **kwargs):
... return (arg1, arg2, moreargs, kwargs)
...
>>>
>>> #exact_kwargs('five', 'six')
... def test_exact_kwargs(arg1, arg2, *moreargs, **kwargs):
... return (arg1, arg2, moreargs, kwargs)
...
>>>
>>>
>>>
>>> test_restrict_kwargs(1, 2, 3, 4, five=5)
(1, 2, (3, 4), {'five': 5})
>>>
>>> test_restrict_kwargs(1, 2, 3, 4, five=5, six=6)
(1, 2, (3, 4), {'six': 6, 'five': 5})
>>>
>>> test_restrict_kwargs(1, 2, 3, 4, five=5, six=6, seven=7)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "SO_31939890.py", line 19, in restrict_wrapper
raise TypeError(msg % (func.__name__, keyword))
TypeError: test_restrict_kwargs() got an unexpected keyword argument 'seven'
>>>
>>>
>>>
>>> test_require_kwargs(1, 2, 3, 4, five=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "SO_31939890.py", line 49, in require_wrapper
missing_keywords_str))
TypeError: test_require_kwargs() missing 1 required keyword-only argument: 'six'
>>>
>>> test_require_kwargs(1, 2, 3, 4, five=5, six=6)
(1, 2, (3, 4), {'six': 6, 'five': 5})
>>>
>>> test_require_kwargs(1, 2, 3, 4, five=5, six=6, seven=7)
(1, 2, (3, 4), {'seven': 7, 'six': 6, 'five': 5})
>>>
>>>
>>>
>>> test_exact_kwargs(1, 2, 3, 4, five=5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "SO_31939890.py", line 20, in restrict_wrapper
return func(*args, **kwargs)
File "SO_31939890.py", line 49, in require_wrapper
missing_keywords_str))
TypeError: test_exact_kwargs() missing 1 required keyword-only argument: 'six'
>>>
>>> test_exact_kwargs(1, 2, 3, 4, five=5, six=6)
(1, 2, (3, 4), {'six': 6, 'five': 5})
>>>
>>> test_exact_kwargs(1, 2, 3, 4, five=5, six=6, seven=7)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "SO_31939890.py", line 19, in restrict_wrapper
raise TypeError(msg % (func.__name__, keyword))
TypeError: test_exact_kwargs() got an unexpected keyword argument 'seven'
>>>
A star (*) is not a valid Python 2.7 identifier, it's an operator. I think you made a mistake while copying a code from the cookbook.
However, it's a valid code in Python 3, as Padraic Cunningham answered.
In python 2.*:
Parameters names derived as variables name laws in any programming languages. but in python 2.* STAR sign used before parameters names for determining special situation and single STAR sign as a parameter name raising error.
But in python 3.*:
A parameter that equal to STAR can't assign any value to this and next position parameters can't give value by position, and only can value by parameter name.