I am using a 3rd party library function which has a large number of positional and named arguments. The function is called from numerous points in my code with identical arguments/values.
For ease of maintenance I don't want to hard-code the dozens of identical arguments multiple times throughout my code. I was hoping there was a way of storing them once in a data structure so I just need to pass the data structure. Along the lines of the following:
Assume the signature of the function I'm calling looks like:
def lib_function(arg1, arg2, arg3=None, arg4=None):
Assume that throughout my code I want to call it with values of
a for arg1,
b for arg2
d for arg4
(and I'm not using arg3).
I've tried defining a data structure as follows:
arguments = ('a', 'b', {'arg4':'d'})
and using it like this:
res = lib_function(arguments)
but this is obviously not correct - the tuple and dict are not unpacked during the call but are handled as a single first argument.
Is there a way of doing this in Python?
The obvious alternative is to proxy lib_function in my code, with the arguments hard-coded in the proxy which is then called with no arguments. Something like:
def proxy_lib_function():
return lib_function('a', 'b', arg4='d')
res = proxy_lib_function()
However I wanted to check there isn't a more Pythonic way of doing this.
Separate positional and named arguments and use asterisk unpacking:
def lib_function(arg1, arg2, arg3=None, arg4=None):
print(locals())
args = ("a", "b")
kwargs = {"arg4": "d"}
res = lib_function(*args, **kwargs)
Your function is:
def lib_function(arg1, arg2, arg3=None, arg4=None):
Define another function:
def yourFunction(x):
return lib_function(x[0], x[1], x[2], x[3])
The "data structure" would be:
data = [yourData1, yourData2, yourData3, yourData4]
Then you can call your new function with this:
yourFunction(data)
Related
I would like to create functions in my Python3 code, making use of data supplied during run-time. I'm stuck on how I can write a function along the lines of
def foo(varname1, varname2):
return varname1 + varname2
where the strings varname1 and varname2 that give the parameter names are specified as arguments to some constructor function, e.g.:
def makeNewFooFunc(varname1, varname2):
# do magic
return foo
fooFunc = makeNewFooFunc('first', 'second')
print(fooFunc(first=1, second=2))
# should return 3
what would be the #do magic step? Is this a job for eval, or is there an alternative?
You don't need to write a function like that.
just use **kwargs
def foo_func(**kwargs):
return sum(kwargs.values())
foo_func(any_name=1, any_name_2=2)
but if you still need to do what you want, you can try
def make_new_func(var_name_1, var_name_2):
def foo(**kwargs):
# make sure in kwargs only expected parameters
assert set(kwargs) == {var_name_1, var_name_2}
return kwargs[var_name_1] + kwargs[var_name_2]
return foo
foo_func = make_new_func('a', 'b')
foo_func(a=1, b=2)
In python we can do this:
def myFun1(one = '1', two = '2'):
...
Then we can call the function and pass the arguments by their name:
myFun1(two = 'two', one = 'one')
Also, we can do this:
def myFun2(**kwargs):
print kwargs.get('one', 'nothing here')
myFun2(one='one')
So I was wondering if it is possible to combine both methods like:
def myFun3(name, lname, **other_info):
...
myFun3(lname='Someone', name='myName', city='cityName', otherInfo='blah')
In general what combinations can we do?
Thanks and sorry for my silly question.
The general idea is:
def func(arg1, arg2, ..., kwarg1=default, kwarg2=default, ..., *args, **kwargs):
...
You can use as many of those as you want. The * and ** will 'soak up' any remaining values not otherwise accounted for.
Positional arguments (provided without defaults) can't be given by keyword, and non-default arguments can't follow default arguments.
Note Python 3 also adds the ability to specify keyword-only arguments by having them after *:
def func(arg1, arg2, *args, kwonlyarg=default):
...
You can also use * alone (def func(a1, a2, *, kw=d):) which means that no arguments are captured, but anything after is keyword-only.
So, if you are in 3.x, you could produce the behaviour you want with:
def myFun3(*, name, lname, **other_info):
...
Which would allow calling with name and lname as keyword-only.
Note this is an unusual interface, which may be annoying to the user - I would only use it in very specific use cases.
In 2.x, you would need to manually make this by parsing **kwargs.
You can add your named arguments along with kwargs. If the keys are available in the calling function It will taken to your named argument otherwise it will be taken by the kwargs dictionary.
def add(a=1, b=2,**c):
res = a+b
for items in c:
res = res + c[items]
print(res)
add(2,3)
5
add(b=4, a =3)
7
add(a =1,b=2,c=3,d=4)
10
It's possible at least for Python 2.7. Keyword arguments get assigned to positional parameters by name, so you can do
In [34]: def func(name, lname, **kwargs):
print 'name='+name, 'lname='+lname
print kwargs
....:
In [35]: func(lname='lname_val', name='name_val', city='cityName', otherInfo='blah')
name=name_val lname=lname_val
{'city': 'cityName', 'otherInfo': 'blah'}
Official docs state it that way:
"If keyword arguments are present, they are first converted to positional arguments, as follows. First, a list of unfilled slots is created for the formal parameters. If there are N positional arguments, they are placed in the first N slots. Next, for each keyword argument, the identifier is used to determine the corresponding slot (if the identifier is the same as the first formal parameter name, the first slot is used, and so on). If the slot is already filled, a TypeError exception is raised. Otherwise, the value of the argument is placed in the slot, filling it (even if the expression is None, it fills the slot)."
https://docs.python.org/2/reference/expressions.html#calls
Excuse my poor wording in the title, but here's a longer explanation:
I have a function which as arguments takes some functions which are used to determine which data to retrieve from a database, as such:
def customer_data(customer_name, *args):
# initialize dictionary with ids
codata = dict([(data.__name__, []) for data in args])
codata['customer_observer_id'] = _customer_observer_ids(customer_name)
# add values to dictionary using function name as key
for data in args:
for coid in codata['customer_observer_id']:
codata[data.__name__].append(data(coid))
return codata
Which makes the call to the function looking something like this:
customer_data('customername', target_parts, source_group, ...)
One of these functions is defined with an extra parameter:
def polarization_value(customer_observer_id, timespan='day')
What I would like is a way to change the timespan variable in a clever way. One obvious way is to include a keyword argument in customer_observer and add an exception when the function name being called is 'polarization_value', but I have a feeling there is a better way to do this.
You can use functools.partial and pass polarization_value as :
functools.partial(polarization_value, timespan='day')
Example:
>>> import functools
def func(x, y=1):
print x, y
...
>>> new_func = functools.partial(func, y=20)
>>> new_func(100)
100 20
You may also find this helpful: Python: Why is functools.partial necessary?
In python we can do this:
def myFun1(one = '1', two = '2'):
...
Then we can call the function and pass the arguments by their name:
myFun1(two = 'two', one = 'one')
Also, we can do this:
def myFun2(**kwargs):
print kwargs.get('one', 'nothing here')
myFun2(one='one')
So I was wondering if it is possible to combine both methods like:
def myFun3(name, lname, **other_info):
...
myFun3(lname='Someone', name='myName', city='cityName', otherInfo='blah')
In general what combinations can we do?
Thanks and sorry for my silly question.
The general idea is:
def func(arg1, arg2, ..., kwarg1=default, kwarg2=default, ..., *args, **kwargs):
...
You can use as many of those as you want. The * and ** will 'soak up' any remaining values not otherwise accounted for.
Positional arguments (provided without defaults) can't be given by keyword, and non-default arguments can't follow default arguments.
Note Python 3 also adds the ability to specify keyword-only arguments by having them after *:
def func(arg1, arg2, *args, kwonlyarg=default):
...
You can also use * alone (def func(a1, a2, *, kw=d):) which means that no arguments are captured, but anything after is keyword-only.
So, if you are in 3.x, you could produce the behaviour you want with:
def myFun3(*, name, lname, **other_info):
...
Which would allow calling with name and lname as keyword-only.
Note this is an unusual interface, which may be annoying to the user - I would only use it in very specific use cases.
In 2.x, you would need to manually make this by parsing **kwargs.
You can add your named arguments along with kwargs. If the keys are available in the calling function It will taken to your named argument otherwise it will be taken by the kwargs dictionary.
def add(a=1, b=2,**c):
res = a+b
for items in c:
res = res + c[items]
print(res)
add(2,3)
5
add(b=4, a =3)
7
add(a =1,b=2,c=3,d=4)
10
It's possible at least for Python 2.7. Keyword arguments get assigned to positional parameters by name, so you can do
In [34]: def func(name, lname, **kwargs):
print 'name='+name, 'lname='+lname
print kwargs
....:
In [35]: func(lname='lname_val', name='name_val', city='cityName', otherInfo='blah')
name=name_val lname=lname_val
{'city': 'cityName', 'otherInfo': 'blah'}
Official docs state it that way:
"If keyword arguments are present, they are first converted to positional arguments, as follows. First, a list of unfilled slots is created for the formal parameters. If there are N positional arguments, they are placed in the first N slots. Next, for each keyword argument, the identifier is used to determine the corresponding slot (if the identifier is the same as the first formal parameter name, the first slot is used, and so on). If the slot is already filled, a TypeError exception is raised. Otherwise, the value of the argument is placed in the slot, filling it (even if the expression is None, it fills the slot)."
https://docs.python.org/2/reference/expressions.html#calls
I have created a class MyClassthat contains a lot of simulation data. The class groups simulation results for different simulations that have a similar structure. The results can be retreived with a MyClass.get(foo) method. It returns a dictionary with simulationID/array pairs, array being the value of foo for each simulation.
Now I want to implement a method in my class to apply any function to all the arrays for foo. It should return a dictionary with simulationID/function(foo) pairs.
For a function that does not need additional arguments, I found the following solution very satisfying (comments always welcome :-) ):
def apply(self, function, variable):
result={}
for k,v in self.get(variable).items():
result[k] = function(v)
return result
However, for a function requiring additional arguments I don't see how to do it in an elegant way. A typical operation would be the integration of foo with bar as x-values like np.trapz(foo, x=bar), where both foo and bar can be retreived with MyClass.get(...)
I was thinking in this direction:
def apply(self, function_call):
"""
function_call should be a string with the complete expression to evaluate
eg: MyClass.apply('np.trapz(QHeat, time)')
"""
result={}
for SID in self.simulations:
result[SID] = eval(function_call, locals=...)
return result
The problem is that I don't know how to pass the locals mapping object. Or maybe I'm looking in a wrong direction. Thanks on beforehand for your help.
Roel
You have two ways. The first is to use functools.partial:
foo = self.get('foo')
bar = self.get('bar')
callable = functools.partial(func, foo, x=bar)
self.apply(callable, variable)
while the second approach is to use the same technique used by partial, you can define a function that accept arbitrary argument list:
def apply(self, function, variable, *args, **kwds):
result={}
for k,v in self.get(variable).items():
result[k] = function(v, *args, **kwds)
return result
Note that in both case the function signature remains unchanged. I don't know which one I'll choose, maybe the first case but I don't know the context on you are working on.
I tried to recreate (the relevant part of) the class structure the way I am guessing it is set up on your side (it's always handy if you can provide a simplified code example for people to play/test).
What I think you are trying to do is translate variable names to variables that are obtained from within the class and then use those variables in a function that was passed in as well. In addition to that since each variable is actually a dictionary of values with a key (SID), you want the result to be a dictionary of results with the function applied to each of the arguments.
class test:
def get(self, name):
if name == "valA":
return {"1":"valA1", "2":"valA2", "3":"valA3"}
elif name == "valB":
return {"1":"valB1", "2":"valB2", "3":"valB3"}
def apply(self, function, **kwargs):
arg_dict = {fun_arg: self.get(sim_args) for fun_arg, sim_args in kwargs.items()}
result = {}
for SID in arg_dict[kwargs.keys()[0]]:
fun_kwargs = {fun_arg: sim_dict[SID] for fun_arg, sim_dict in arg_dict.items()}
result[SID] = function(**fun_kwargs)
return result
def joinstrings(string_a, string_b):
return string_a+string_b
my_test = test()
result = my_test.apply(joinstrings, string_a="valA", string_b="valB")
print result
So the apply method gets an argument dictionary, gets the class specific data for each of the arguments and creates a new argument dictionary with those (arg_dict).
The SID keys are obtained from this arg_dict and for each of those, a function result is calculated and added to the result dictionary.
The result is:
{'1': 'valA1valB1', '3': 'valA3valB3', '2': 'valA2valB2'}
The code can be altered in many ways, but I thought this would be the most readable. It is of course possible to join the dictionaries instead of using the SID's from the first element etc.