Develop Reference

Passing arguments to Python function as a data structure

I am using a 3rd party library function which has a large number of positional and named arguments. The function is called from numerous points in my code with identical arguments/values.
For ease of maintenance I don't want to hard-code the dozens of identical arguments multiple times throughout my code. I was hoping there was a way of storing them once in a data structure so I just need to pass the data structure. Along the lines of the following:
Assume the signature of the function I'm calling looks like:
def lib_function(arg1, arg2, arg3=None, arg4=None):
Assume that throughout my code I want to call it with values of
a for arg1,
b for arg2
d for arg4
(and I'm not using arg3).
I've tried defining a data structure as follows:
arguments = ('a', 'b', {'arg4':'d'})
and using it like this:
res = lib_function(arguments)
but this is obviously not correct - the tuple and dict are not unpacked during the call but are handled as a single first argument.
Is there a way of doing this in Python?
The obvious alternative is to proxy lib_function in my code, with the arguments hard-coded in the proxy which is then called with no arguments. Something like:
def proxy_lib_function():
return lib_function('a', 'b', arg4='d')
res = proxy_lib_function()
However I wanted to check there isn't a more Pythonic way of doing this.

Separate positional and named arguments and use asterisk unpacking:
def lib_function(arg1, arg2, arg3=None, arg4=None):
print(locals())
args = ("a", "b")
kwargs = {"arg4": "d"}
res = lib_function(*args, **kwargs)

Your function is:
def lib_function(arg1, arg2, arg3=None, arg4=None):
Define another function:
def yourFunction(x):
return lib_function(x[0], x[1], x[2], x[3])
The "data structure" would be:
data = [yourData1, yourData2, yourData3, yourData4]
Then you can call your new function with this:
yourFunction(data)

How to efficiently unravel arguments to a database in Python

I have the following function which takes 4 arguments (we don't know which are set and which aren't before runtime).
myFuct(self, arg1, arg2, arg3, arg4):
# db insert function for only those args that are set
Is there an efficient way of unraveling the set arguments in a mysql database with a single sql query?
P.S. I am looking for a pythonic way to do this without having to say if-this-then-that for a million times...
Thanks in advance.

Not entirely sure what you need.
def myfunc(*args, **kwargs):
...
Can take any number of arguments, is this what you're looking for?
print(*(1,2,3,4)) * "unravels" the arguments in the order they are given but can be passed as one object or any number of objects.
However, passing these into a SQL statement without knowing the precise length of the arguments. Since SQL to my knowledge is a bit strict on how you pass the arguments.
You could do something like:
def myfunc(*args):
statement = "INSERT INTO table VALUES(" + ', '.join(str(x) for x in args) + ");"
db.execute(statement)
Note of course that this can only be used if you actually pass arguments to myfunc that are of the correct type, order and length.
in postgresql you have something called a prepared statement which would come in handy here.
def myfunc(*args):
statement = "INSERT INTO table VALUES(" + ', '.join('$'+str(i) for i in range(0, len(args)+1)) + ");"
handle = db.prepare(statement)
handle(args)
Where you simply define $1, $2 and so on in the prepared statement and then pass the equivilant number of arguments to the handler which will execute the query with the correct format on each of the values enabling you to pass integers, strings and arrays.. any way you like :)
Also, if the argument is a dictionary (mentioned in a comment)...
You'd have to do:
def myfunc(*args):
keys = ', '.join(str(x) for x in list(args.keys()))
values = ', '.join(str(val) for k, val in list(args.items()))
statement = "INSERT INTO table (" + keys + ") VALUES(" + values + ");"
db.execute(statement)
Again, assumes the dictionary is passed with proper variables for a execution to proceed.

You can use variable-length argument list construct:
def myfunc(*args):
args_non_empty = [a for a in args if a]
This allows you to pass any number of argument to your method. You can easily then filter them by traversing the list.

Can I change function parameters when passing them as variables?

Excuse my poor wording in the title, but here's a longer explanation:
I have a function which as arguments takes some functions which are used to determine which data to retrieve from a database, as such:
def customer_data(customer_name, *args):
# initialize dictionary with ids
codata = dict([(data.__name__, []) for data in args])
codata['customer_observer_id'] = _customer_observer_ids(customer_name)
# add values to dictionary using function name as key
for data in args:
for coid in codata['customer_observer_id']:
codata[data.__name__].append(data(coid))
return codata
Which makes the call to the function looking something like this:
customer_data('customername', target_parts, source_group, ...)
One of these functions is defined with an extra parameter:
def polarization_value(customer_observer_id, timespan='day')
What I would like is a way to change the timespan variable in a clever way. One obvious way is to include a keyword argument in customer_observer and add an exception when the function name being called is 'polarization_value', but I have a feeling there is a better way to do this.

You can use functools.partial and pass polarization_value as :
functools.partial(polarization_value, timespan='day')
Example:
>>> import functools
def func(x, y=1):
print x, y
...
>>> new_func = functools.partial(func, y=20)
>>> new_func(100)
100 20
You may also find this helpful: Python: Why is functools.partial necessary?

SQLAlchemy Insert w/ Dictionary

For this situation assume there is a table declared with the declarative base called Game, with column names "espn_game_id" and "a_name". There is a session object open under the variable "s". So this works:
s.add(Game(espn_game_id=888, a_name='lol'))
s.commit()
This fails:
n = {"espn_game_id": 888, "a_name": 'lol'}
s.add(Game(n))
s.commit()
This works (assuming there is already an entry with espn_game_id==888):
n = {"a_name": 'lol'}
s.query(Game).filter(Game.espn_game_id==888).update(n)
s.commit()
Why does the second case fail and is there a way to get that type of syntax to work here?

Try to replace:
s.add(Game(n))
with:
s.add(Game(**n))
Let's assume you have function like this:
def foo(**kwargs):
print [item for item in kwargs.items()]
It expects keyword not positional arguments so this will work:
foo(**{'foo': 1, 'bar': 2})
and this will fail:
foo({'foo': 1, 'bar': 2})
For a better explanation you should read *args and **kwargs?

how to make my own mapping type in python

I have created a class MyClassthat contains a lot of simulation data. The class groups simulation results for different simulations that have a similar structure. The results can be retreived with a MyClass.get(foo) method. It returns a dictionary with simulationID/array pairs, array being the value of foo for each simulation.
Now I want to implement a method in my class to apply any function to all the arrays for foo. It should return a dictionary with simulationID/function(foo) pairs.
For a function that does not need additional arguments, I found the following solution very satisfying (comments always welcome :-) ):
def apply(self, function, variable):
result={}
for k,v in self.get(variable).items():
result[k] = function(v)
return result
However, for a function requiring additional arguments I don't see how to do it in an elegant way. A typical operation would be the integration of foo with bar as x-values like np.trapz(foo, x=bar), where both foo and bar can be retreived with MyClass.get(...)
I was thinking in this direction:
def apply(self, function_call):
"""
function_call should be a string with the complete expression to evaluate
eg: MyClass.apply('np.trapz(QHeat, time)')
"""
result={}
for SID in self.simulations:
result[SID] = eval(function_call, locals=...)
return result
The problem is that I don't know how to pass the locals mapping object. Or maybe I'm looking in a wrong direction. Thanks on beforehand for your help.
Roel

You have two ways. The first is to use functools.partial:
foo = self.get('foo')
bar = self.get('bar')
callable = functools.partial(func, foo, x=bar)
self.apply(callable, variable)
while the second approach is to use the same technique used by partial, you can define a function that accept arbitrary argument list:
def apply(self, function, variable, *args, **kwds):
result={}
for k,v in self.get(variable).items():
result[k] = function(v, *args, **kwds)
return result
Note that in both case the function signature remains unchanged. I don't know which one I'll choose, maybe the first case but I don't know the context on you are working on.

I tried to recreate (the relevant part of) the class structure the way I am guessing it is set up on your side (it's always handy if you can provide a simplified code example for people to play/test).
What I think you are trying to do is translate variable names to variables that are obtained from within the class and then use those variables in a function that was passed in as well. In addition to that since each variable is actually a dictionary of values with a key (SID), you want the result to be a dictionary of results with the function applied to each of the arguments.
class test:
def get(self, name):
if name == "valA":
return {"1":"valA1", "2":"valA2", "3":"valA3"}
elif name == "valB":
return {"1":"valB1", "2":"valB2", "3":"valB3"}
def apply(self, function, **kwargs):
arg_dict = {fun_arg: self.get(sim_args) for fun_arg, sim_args in kwargs.items()}
result = {}
for SID in arg_dict[kwargs.keys()[0]]:
fun_kwargs = {fun_arg: sim_dict[SID] for fun_arg, sim_dict in arg_dict.items()}
result[SID] = function(**fun_kwargs)
return result
def joinstrings(string_a, string_b):
return string_a+string_b
my_test = test()
result = my_test.apply(joinstrings, string_a="valA", string_b="valB")
print result
So the apply method gets an argument dictionary, gets the class specific data for each of the arguments and creates a new argument dictionary with those (arg_dict).
The SID keys are obtained from this arg_dict and for each of those, a function result is calculated and added to the result dictionary.
The result is:
{'1': 'valA1valB1', '3': 'valA3valB3', '2': 'valA2valB2'}
The code can be altered in many ways, but I thought this would be the most readable. It is of course possible to join the dictionaries instead of using the SID's from the first element etc.