What I want to accomplish:
dct = {'foo':0, 'bar':1, 'baz':2}
def func(**dct):
pass
#function signature is now func(foo=0, bar=1, baz=2)
However, the ** syntax is obviously clashing here between expanding a dict (what I want to do) and declaring a parameter that holds the keyword arguments (what I don't want to do).
Is this possible?
Based on my interpretation of your requirements -- you want to dynamically define a function with a signature that matches the content of adict provided at runtime -- there are two issues here which makes it impractical.
If the arguments are defined at run-time, how can your function reference the variables? Are you planning to build the function body at run-time as well?
dicts are unordered, so you cannot reliably use them to define positional arguments
I suspect this is an XY problem. If you can explain what you're trying to achieve then perhaps we can be of better help.
However, assuming you're trying to assign default keyword arguments using a dict then one way to achieve this would be to use decorators. For example:
def defaultArgs(default_kw):
"decorator to assign default kwargs"
def wrap(f):
def wrapped_f(**kwargs):
kw = {}
kw.update(default_kw) # apply defaults
kw.update(kwargs) # apply from input args
f(**kw) # run actual function with updated kwargs
return wrapped_f
return wrap
You can then use this decorator to assign default keyword arguments to a function that expects only keyword arguments:
defaults = {'foo':0, 'bar':1, 'baz':2}
#defaultArgs(defaults)
def func(**kwargs):
print kwargs # args accessible via the kwargs dict
Results:
func() # prints {'baz': 2, 'foo': 0, 'bar': 1}
func(foo=2) # prints {'baz': 2, 'foo': 2, 'bar': 1}
params = {'bar':1000, 'hello':'world'}
func(**params) # prints {'baz': 2, 'foo': 0, 'bar': 1000, 'hello': 'world'}
Note that you will not be able to use positional arguments:
func(1, 2, 3) # raises TypeError
what you want i believe is eval() link
an answer i gave on a similar question: https://stackoverflow.com/a/11865190/1561176
I'm really not sure what you plan on accomplishing here. The following works (sort of):
def func(**dct):
pass
dct = {'foo':0, 'bar':1, 'baz':2}
func(**dct)
How do you plan on using foo, bar or baz in the function if they're created dynamically? If you give some more details on what you're actually trying to accomplish, we might be able to be a little more helpful.
Related
I know there are similar questions but still could not figure it out the proper syntax for this issue.
I would like to define a function under a library created by myself based on plotly functions.
The function i would like to create needs multiple kwargs to be declared for using them separate function under my function.Below is my failing example code
def plotly_overlaid_histogram(self,series1,series2,**{**kwargs1,**kwargs2}):
self.series1=series1
self.series2=series2
fig=go.Figure()
fig.add_trace(go.Histogram(x=series1))
fig.add_trace(go.Histogram(x=series2))
fig.update_layout(**kwargs1)
fig.update_traces(**kwargs2)
fig.show()
I also try first line like followings but did not help either
def plotly_overlaid_histogram(self,series1,series2,**{**kwargs1,**kwargs2}):
pass
def plotly_overlaid_histogram(self,series1,series2,**kwargs1,**kwargs2):
pass
As #MisterMiyagi mentioned in the comments, by using that syntax, Python wouldn't know which keyword arguments go with which set, e.g., when you unpack the dictionary {"a": 1, "b": 2, "c": 3} in a function call, all the function sees is a=1, b=2, c=3 as keyword arguments.
Solution 1: do it differently
For your case, I suggest making the traces and layout keyword-only arguments, and type hint them as dict:
def plotly_overlaid_histogram(self, series1, series2, *, layout: dict, traces: dict) -> go.Figure:
...
fig.update_layout(**layout)
fig.update_traces(**traces)
Then just pass separate "kwargs" dictionaries when calling:
layout = dict(title="hello world")
traces = dict(marker_color="pink")
obj.plotly_overload_histogram(series1, series2, layout=layout, traces=traces)
Or whatever.
Solution 2: if you really want to stick with your current interface
Another option is to use go.Figure.update() and rework your function a little bit. In order to update traces using go.Figure.update() you need to pass a list of dictionaries under the keyword "data".
fig = go.Figure(...) # create some figure with multiple traces
traces = [{"opacity": 0.2}]*2 # because there are two traces
layout = dict(title="hello world")
update_dict = dict(layout=layout, data=traces)
fig.update(**update_dict)
And now you can redo your function signature:
def plotly_overlaid_histogram(self, series1, series2, **update_dict) -> go.Figure:
...
fig.update(**update_dict)
Just making sure to construct your update dictionary properly.
Of course, if all you're doing is passing a keyword argument dictionary to an inner function, you don't really need to use the unpacking operator in the signature, just pass your keyword arguments as a dictionary:
def plotly_overlaid_histogram(self, series1, series2, update_dict) -> go.Figure:
...
fig.update(**update_dict)
Clarification
To clarify, you'd build your update_dict outside the function call, and then pass it as a dictionary (similar to how you were doing before), so:
traces_kwargs = dict(opacity=0.2) # whatever kwargs you WOULD'VE passed to fig.update_traces()
data = [traces_kwargs]*2 # because there are two traces in this case
layout = dict(title="hello world") # whatever kwargs you WOULD'VE passed to fig.update_layout()
update_dict = dict(layout=layout, data=data) # package all your 'kwargs' together
obj.plotly_overlaid_histogram(series1, series2, update_dict) # call your function
can someone say what the best method to do this would be?
I want to have a function, overall_function that I pass other functions to. But the other functions don't always have the same or type of arguments.
So what would be the correct syntax to do:
def overall_function(function, arguments):
function(arguments)
do other stuff
I'd like arguments to be:
arg1 = 'foo', arg2 = 53, ...
Thank you!
Take variable number of arguments:
def overall_function(function, *positional, **keyword):
function(*positional, **keyword)
Now, you can pass the arguments like:
overall_function(function, 1, 2, foo='bar')
and this will execute the function as:
function(1, 2, foo='bar')
positional would be a tuple: (1, 2) (these two are positional arguments).
keyword would be a dict: {'foo': 'bar'} (this is a keyword argument).
Looking for *args and **kwargs
def overall_function(function, *args, **kwargs):
function(*args, **kwargs)
do other stuff
You might be better off passing in a lambda as opposed to two arguments so you can delay the execution of the function till you need it, and still keeping the parameters contained together
def overall_function(lambda_func):
lambda_func()
overall_function(lambda: function(used,as,normal))
I've got some trouble dealing with defining a user friendly function interface when passing two keyworded arguments with the same key.
Question
What is the best way to make it possible to call a function where two keyworded arguments have the same key and the second keyworded argument has precedence?
If this problem occurs, the first keyworded argument always stems from an unzipped database in a dict, while the second keyworded argument is always passed by giving it "directly" as a keyworded argument.
The database dictionary values must not be overwritten in the outer scopy of the functions, since they may be used multiple times.
edit: To keep up the usability of the function for the user, a backend-implementation is preferred. This means that the user can simply pass arguments to the function without the use of additional modules, while the function itself does all the magic.
Problem
I've got a function, called fun_one here, which receives a multitude of arguments defined directly by the user of my program. This may be length and width of a heat exchanger for example. To ease the use of the function and make the calling code as short as possible, the use of databases is encouraged. These databases contain the data in a dict (or in a pandas Series), in this case called inputs.
To pass the database-dict inputs to the function, it is unzipped with **inputs and thus passed as keyworded arguments.
Now if the user wants to overwrite a specific argument of the database, my understanding of a user-friendly approach would be to just let him pass the preceded argument again, for example with length=23.7, and internally overwrite the argument from the database. But of course (see example code) this raises the error before I can even enter the function where I could try/except:
TypeError: fun_one() got multiple values for keyword argument 'length'
Code example reproducing the error
def fun_one(*args, **kwargs): # short example function
print(kwargs)
inputs = {'length': 15.8, 'width': 1.1, 'some_other_args': np.random.rand(3)}
fun_one(**inputs, length=23.7)
My current solution
My current solution fun_two involves not unzipping the database and passing it to *args. It checks *args for dicts and sets values which are not yet in kwargs to kwargs, as shown in the code example below.
def fun_two(*args, **kwargs): # example function printing kwargs
print(kwargs) # print kwargs before applying changes
for arg in args: # find dicts
if type(arg) is dict:
for key, val in arg.items(): # loop over dict
_ = kwargs.setdefault(key, val) # set val if key not in dict
print(kwargs) # print kwargs after applying changes
inputs = {'length': 15.8, 'width': 1.1, 'some_other_args': np.random.rand(3)}
fun_two(inputs, length=23.7)
But this approach is imho quite obscure for the user and requires looping and checking at the beginning of quite alot functions, since this will apply to numerous functions. (I'll outsource it to a module, so it is one line per function. But it still deviates from my understanding of an easy and clear function definition).
Is there any better (more Pythonic) way to do this? Did I oversee some way to do it in the process of calling the function? Performance does not matter.
Thanks in advance!
Easiest solution is using ChainMap from collections (manual pages). That way you can chose which arguments have precedence. Example:
from collections import ChainMap
def fun_one(*args, **kwargs): # short example function
print(kwargs)
inputs = {'length': 15.8, 'width': 1.1, 'some_other_args': 1}
c = ChainMap({'length': 23.7}, inputs) # we overwrite length here
fun_one(**c)
Outputs:
{'some_other_args': 1, 'width': 1.1, 'length': 23.7}
If we call fun_one just with inputs:
c = ChainMap(inputs)
# or c = inputs
fun_one(**c)
Output will be:
{'width': 1.1, 'length': 15.8, 'some_other_args': 1}
From manual:
A ChainMap groups multiple dicts or other mappings together to create
a single, updateable view. If no maps are specified, a single empty
dictionary is provided so that a new chain always has at least one
mapping.
You can wrap this ChainMap in decorator, one change is that don't call fun_one() with **input, only input:
from collections import ChainMap
def function_with_many_arguments(func):
orig_func = func
def _f(*args, **kwargs):
if args:
c = ChainMap(kwargs, args[0])
return orig_func(**c)
else:
return orig_func(*args, **kwargs)
return _f
#function_with_many_arguments
def fun_one(*args, **kwargs): # short example function
print(kwargs)
inputs = {'length': 15.8, 'width': 1.1, 'some_other_args': 1}
fun_one(inputs, length=23)
Prints:
{'some_other_args': 1, 'length': 23, 'width': 1.1}
As extension to Andrej Kesely's answer (thanks again!), I added a loop of ChainMaps to enable the use of multiple databases in the same function and to be able to use all kinds of positional arguments. The precedence of the multiple databases is first-come-first-served, but this is ok in this case. This is the decorator:
def function_with_many_arguments(func):
orig_func = func
def _f(*args, **kwargs):
if args:
c = ChainMap(kwargs)
for arg in args:
if type(arg) is dict:
c = ChainMap(c, arg)
orig_func(*args, **c)
else:
orig_func(*args, **kwargs)
return _f
And here is my extended example function with some code to test it. I just added all kind of random arguments, not considering any Pythonic way to do it... ;)
#function_with_many_arguments
def fun_one(a, b, *args, name, database=None, **kwargs):
print(name)
print(a, b)
print(kwargs)
inputs = {'length': 15.8, 'width': 1.1, 'some_other_args': np.random.rand(3)}
inputs2 = inputs.copy()
inputs2['width'] = 123
inputs2['weight'] = 3.8
fun_one(4, 8, inputs, database=inputs2, name='abc', length=23.8, weight=55)
I am wondering if it is possible to list the variables expected by a Python function, prior to calling it, in order to pass the expected variables from a bigger dict containing a lot of variables.
I have searched the net but couldn't find anything. However, the python interpreter can show the list of expected variables, so there surely must be some way to do it in a script?
You can use either the inspect.signature() or inspect.getfullargspec() functions:
import inspect
argspec = inspect.getfullargspec(somefunction)
signature = inspect.signature(somefunction)
inspect.fullargspec returns a named tuple with 7 elements:
A list with the argument names
The name of the catchall *args parameter, if defined (None otherwise)
The name of the catchall **kwargs parameter, if defined (None otherwise)
A tuple with default values for the keyword arguments; they go with the last elements of the arguments; match these by length of the default values tuple.
A list of keyword-only parameter names
A dictionary of default values for the keyword-only parameter names, if any
and a dictionary containing the annotations
With inspect.signature() you get a Signature object, a rich object that models not only the above data as a more structured set of objects but also lets you bind values to parameters the same way a call to the function would.
Which one is better will depend on your use cases.
Demo:
>>> import inspect
>>> def foo(bar, baz, spam='eggs', *monty, python: "kwonly", spanish=42, **inquisition) -> "return annotation":
... pass
...
>>> inspect.getfullargspec(foo)
FullArgSpec(args=['bar', 'baz', 'spam'], varargs='monty', varkw='inquisition', defaults=('eggs',), kwonlyargs=['python', 'spanish'], kwonlydefaults={'spanish': 42}, annotations={'return': 'return annotation', 'python': 'kwonly'})
>>> signature = inspect.signature(foo)
>>> signature
<Signature (bar, baz, spam='eggs', *monty, python: 'kwonly', spanish=42, **inquisition) -> 'return annotation'>
>>> signature.parameters['python'].kind.description
'keyword-only'
>>> signature.bind('Eric', 'Idle', 'John', python='Cleese')
<BoundArguments (bar='Eric', baz='Idle', spam='John', python='Cleese')>
If you have a dictionary named values of possible parameter values, I'd use inspect.signature() and use the Signature.parameters mapping to match names:
posargs = [
values[param.name]
for param in signature.parameters.values()
if param.kind is Parameter.POSITIONAL_ONLY
]
skip_kinds = {Parameter.POSITIONAL_ONLY, Parameter.VAR_POSITIONAL, Parameter.VAR_KEYWORD}
kwargs = {
param.name: values[param.name]
for param in signature.parameters.values()
if param.name in values and param.kind not in skip_kinds
}
The above gives you a list of values for the positional-only parameters, and a dictionary for the rest (excepting any *args or **kwargs parameters).
Just as a side answer, I now use another approach to pass to functions the variables they expect: I pass them all.
What I mean is that I maintain a kind of global/shared dictionnary of variables in my root object (which is the parent of all other objects), eg:
shareddict = {'A': 0, 'B':'somestring'}
Then I simply pass this dict to any method of any other object that is to be called, just like this:
shareddict.update(call_to_func(**shareddict))
As you can see, we unpack all the keys/values in shareddict as keyword arguments to call_to_func(). We also update shareddict with the returned result, we'll see below why.
Now with this technic, I can simply and clearly define in my functions/methods if I need one or several variables from this dict:
my_method1(A=None, *args, **kwargs):
''' This method only computes on A '''
new_A = Do_some_stuff(A)
return {'A': new_A} # Return the new A in a dictionary to update the shared value of A in the shareddict
my_method2(B=None, *args, **kwargs):
''' This method only computes on B '''
new_B = Do_some_stuff(B)
return {'B': new_B} # Return the new B in a dictionary to update the shareddict
my_method3(A=None, B=None, *args, **kwargs):
''' This method swaps A and B, and then create a new variable C '''
return {'A': B, 'B': A, 'C': 'a_new_variable'} # Here we will update both A and B and create the new variable C
As you can notice, all the methods above return a dict of variables, which will update the shareddict, and which will get passed along to other functions.
This technic has several advantages:
Quite simple to implement
Elegant way to maintain a shared list of variables but without using a global variable
Functions and methods clearly show in their definitions what they expect (but of course one caveat is that even mandatory variables will need to be set as a keyword argument with a default value such as None, which usually means that the variable is optional, but here it's not
The methods are inheritable and overloadable
Low memory footprint since the same shareddict is passed all along
The children functions/methods define what they need (bottom-up), instead of the root defining what arguments will be passed to children (top-down)
Very easy to create/update variables
Optionally, it's VERY easy to dump all those variables in a file, eg by using json.dumps(finaldict, sort_keys=True).
Nice and easy:
import inspect #library to import
def foo(bar, baz, spam='eggs', *monty, **python): pass #example function
argspec = inspect.signature(foo)
print(argspec) #print your output
prints: (bar, baz, spam='eggs', *monty, **python)
It also works for methods inside classes (very useful!):
class Complex: #example Class
def __init__(self, realpart, imagpart): #method inside Class
... self.r = realpart
... self.i = imagpart
argspec = inspect.signature(Complex)
print(argspec)
prints: (realpart, imagpart)
When can I pass * and ** in the argument of a Python function? i.e.:
def fun_name(arg1, *arg2 , ** arg3):
As you've stated your question, you aren't using them in the arguments (which occur when you are calling a function), you are using them in the parameters which occur when you are creating a function. The * and ** operators serve different purposes in each of those situations.
When you are defining a function, they specify that positional arguments will be placed in a tuple and that keyword arguments will be placed in a dict. Yes I did just say arguments, but they are applied to paramaters in this case.
def example(*args, **kwargs):
print "args: {0}".format(args)
print "kwargs: {0}".format(kwargs)
example(1, 2, 'a', foo='bar', bar='foo')
when run, this outputs:
args: (1, 2, 'a')
kwargs: {'foo': 'bar', 'bar': 'foo'}
Do you see what I mean when I say that we applied it to the paramaters in the function definition? the arguments are 1, 2, 'a', foo='bar', bar='foo'. the paramaters are *args, **kwargs.
Now here's an example applying them to arguments.
def example2(a, b, foo=None, bar=None):
print "a: {0}, b:{1}, foo:{2}, bar: {3}".format(a, b, foo, bar)
args = (1, 2)
kwargs = {'foo': 'bar', 'bar': 'foo'}
example2(*args, **kwargs)
This outputs:
a: 1, b:2, foo:bar, bar: foo
You can see that when we apply them to arguments (that is when we are calling the function), * has the effect of expanding a list or tuple to fill the positional arguments of a function and ** has the effect of expanding a dictionary to fill the keyword arguments of a function. You just need to make sure that there are enough and not too much arguments in total after the expansions have taken place.
in the last example, the arguments are *args, **kwargs and the parameters are a, b, foo=None, bar=None
When can I pass * and ** in the argument of a Python function? i.e.:
Short answer: when you require variable number of argument to be passed to your function.
That said, I honestly think that this a very broad question. You will be much better off reading more about these concepts, trying them off and then asking an specific questions here.
#Wooble's answer will help you; but what will help you even more is to understand what *args and **kwargs do. Then you can use them as befits the situations you encounter.
You can learn more about the variable and keyword arguments concepts for example at:
http://www.network-theory.co.uk/docs/pytut/KeywordArguments.html
http://www.saltycrane.com/blog/2008/01/how-to-use-args-and-kwargs-in-python/
Any time you want your function to accept non-named positional arguments (*), or additional named arguments (**).
It's certainly possible to include *args, **kwargs in every function definition, although this might not be a great idea if passing more arguments won't actually have any effect.