Develop Reference

python interface with optional variable [duplicate]

I have a Python function which takes several arguments. Some of these arguments could be omitted in some scenarios.
def some_function (self, a, b, c, d = None, e = None, f = None, g = None, h = None):
#code
The arguments d through h are strings which each have different meanings. It is important that I can choose which optional parameters to pass in any combination. For example, (a, b, C, d, e), or (a, b, C, g, h), or (a, b, C, d, e, f, or all of them (these are my choices).
It would be great if I could overload the function - but I read that Python does not support overloading. I tried to insert some of the required int arguments in the list - and got an argument mismatch error.
Right now I am sending empty strings in place of the first few missing arguments as placeholders. I would like to be able to call a function just using actual values.
Is there any way to do this? Could I pass a list instead of the argument list?
Right now the prototype using ctypes looks something like:
_fdll.some_function.argtypes = [c_void_p, c_char_p, c_int, c_char_p, c_char_p, c_char_p, c_char_p, c_char_p]

Just use the *args parameter, which allows you to pass as many arguments as you want after your a,b,c. You would have to add some logic to map args->c,d,e,f but its a "way" of overloading.
def myfunc(a,b, *args, **kwargs):
for ar in args:
print ar
myfunc(a,b,c,d,e,f)
And it will print values of c,d,e,f
Similarly you could use the kwargs argument and then you could name your parameters.
def myfunc(a,b, *args, **kwargs):
c = kwargs.get('c', None)
d = kwargs.get('d', None)
#etc
myfunc(a,b, c='nick', d='dog', ...)
And then kwargs would have a dictionary of all the parameters that are key valued after a,b

Try calling it like: obj.some_function( '1', 2, '3', g="foo", h="bar" ). After the required positional arguments, you can specify specific optional arguments by name.

It is very easy just do this
def foo(a = None):
print(a)
Instead of None you can type anything that should be in place if there was no argument for example if you will not write the value of the parameter like this foo() then it will print None because no argument is given and if you will GIVE it an argument like foo("hello world") then it will print hello world... oh well I just forgot to tell y'all that these types of parameters i.e optional parameters, need to be behind all the other parameters. This means that, let's take the previous function and add another parameter b
def foo(a = None, b):
print(a)
Now if you'll execute your python file it is going to raise an exception saying that Non-default arguments follow default arguments,
SyntaxError: non-default argument follows default argument
so you gotta put the optional or non-default argument after the arguments which are required
which means
def foo (a, b=None): ... #This one is right
def foo(b=None, a): ... #and this isn't

Required parameters first, optional parameters after. Optional parameters always with a =None.
Easy and fast example:
def example_function(param1, param2, param3=None, param4=None):
pass
# Doesn't work, param2 missing
example_function("hello")
# Works
example_function("hello", "bye")
# Works. Both the same
example_function("hello", "bye", "hey")
example_function("hello", "bye", param3="hey")
# Works. Both the same
example_function("hello", "bye", "hey", "foo")
example_function("hello", "bye", param3="hey", param4="foo")

Check this:
from typing import Optional
def foo(a: str, b: Optional[str] = None) -> str or None:
pass

To get a better sense of what's possible when passing parameters it's really helpful to refer to the various options: positional-or-keyword (arg or arg="default_value"), positional-only (before /, in the parameter list), keyword-only (after *, in the parameter list), var-positional (typically *args) or var-keyword (typically **kwargs). See the Python documentation for an excellent summary; the various other answers to the question make use of most of these variations.
Since you always have parameters a, b, c in your example and you appear to call them in a positional manner, you could make this more explicit by adding /,,
def some_function (self, a, b, c, /, d = None, e = None, f = None, g = None, h = None):
#code

To make Avión's answer work for vector argument inputs;
def test(M,v=None):
try:
if (v==None).all() == False:
print('argument passed')
return M + v
except:
print('no argument passed')
return M
Where M is some matrix and v some vector. Both test(M) and test(M,v) produce errors when I attempted to use if statements without using 'try/ except' statements.
As mentioned by cem, upgrading to python 3.10 would allow the union (x|y) (or the Optional[...])functionality which might open some doors for alternative methods, but I'm using Anaconda spyder so I think I have to wait for a new release to use python 3.10.

Overriding function signature (in help) when using functools.wraps

I'm creating a wrapper for a function with functools.wraps. My wrapper has the effect of overriding a default parameter (and it doesn't do anything else):
def add(*, a=1, b=2):
"Add numbers"
return a + b
#functools.wraps(add)
def my_add(**kwargs):
kwargs.setdefault('b', 3)
return add(**kwargs)
This my_add definition behaves the same as
#functools.wraps(add)
def my_add(*, a=1, b=3):
return add(a=a, b=b)
except that I didn't have to manually type out the parameter list.
However, when I run help(my_add), I see the help string for add, which has the wrong function name and the wrong default argument for the parameter b:
add(*, a=1, b=2)
Add numbers
How can I override the function name and the default argument in this help() output?
(Or, is there a different way to define my_add, using for example some magic function my_add = magic(add, func_name='my_add', kwarg_defaults={'b': 3}) that will do what I want?)

Let me try and explain what happens.
When you call the help functions, this is going to request information about your function using the inspect module. Therefore you have to change the function signature, in order to change the default argument.
Now this is not something that is advised, or often preferred, but who cares about that right? The provided solution is considered hacky and probably won't work for all versions of Python. Therefore you might want to reconsider how important the help function is... Any way let's start with some explanation on how it was done, followed by the code and test case.
Copying functions
Now the first thing we will do is copy the entire function, this is because I only want to change the signature of the new function and not the original function. This decouples the new my_add signature (and default values) from the original add function.
See:
How to create a copy of a python function
How can I make a deepcopy of a function in Python?
For ideas of how to do this (I will show my version in a bit).
Copying / updating signature
The next step is to get a copy of the function signature, for that this post was very useful. Except for the part where we have to adjust the signature parameters to match the new keyword default arguments.
For that we have to change the value of a mappingproxy, which we can see when running the debugger on the return value of inspect.signature(g). Now so far this can only be done by changing the private variables (the values with leading underscores _private). Therefore this solution will be considered hacky and is not guaranteed to withstand possible updates. That said, let's see the solution!
Full code
import inspect
import types
import functools
def update_func(f, func_name='', update_kwargs: dict = None):
"""Based on http://stackoverflow.com/a/6528148/190597 (Glenn Maynard)"""
g = types.FunctionType(
code=f.__code__,
globals=f.__globals__.copy(),
name=f.__name__,
argdefs=f.__defaults__,
closure=f.__closure__
)
g = functools.update_wrapper(g, f)
g.__signature__ = inspect.signature(g)
g.__kwdefaults__ = f.__kwdefaults__.copy()
# Adjust your arguments
for key, value in (update_kwargs or {}).items():
g.__kwdefaults__[key] = value
g.__signature__.parameters[key]._default = value
g.__name__ = func_name or g.__name__
return g
def add(*, a=1, b=2):
"Add numbers"
return a + b
my_add = update_func(add, func_name="my_add", update_kwargs=dict(b=3))
Example
if __name__ == '__main__':
a = 2
print("*" * 50, f"\nMy add\n", )
help(my_add)
print("*" * 50, f"\nOriginal add\n", )
help(add)
print("*" * 50, f"\nResults:"
f"\n\tMy add : a = {a}, return = {my_add(a=a)}"
f"\n\tOriginal add: a = {a}, return = {add(a=a)}")
Output
**************************************************
My add
Help on function my_add in module __main__:
my_add(*, a=1, b=3)
Add numbers
**************************************************
Original add
Help on function add in module __main__:
add(*, a=1, b=2)
Add numbers
**************************************************
Results:
My add : a = 2, return = 5
Original add: a = 2, return = 4
Usages
f: is the function that you want to update
func_name: is optionally the new name of the function (if empty, keeps the old name)
update_kwargs: is a dictionary containing the key and value of the default arguments that you want to update.
Notes
The solution is using copy variables to make full copies of dictionaries, such that there is no impact on the original add function.
The _default value is a private variable, and can be changed in future releases of python.

Can I change function parameters when passing them as variables?

Excuse my poor wording in the title, but here's a longer explanation:
I have a function which as arguments takes some functions which are used to determine which data to retrieve from a database, as such:
def customer_data(customer_name, *args):
# initialize dictionary with ids
codata = dict([(data.__name__, []) for data in args])
codata['customer_observer_id'] = _customer_observer_ids(customer_name)
# add values to dictionary using function name as key
for data in args:
for coid in codata['customer_observer_id']:
codata[data.__name__].append(data(coid))
return codata
Which makes the call to the function looking something like this:
customer_data('customername', target_parts, source_group, ...)
One of these functions is defined with an extra parameter:
def polarization_value(customer_observer_id, timespan='day')
What I would like is a way to change the timespan variable in a clever way. One obvious way is to include a keyword argument in customer_observer and add an exception when the function name being called is 'polarization_value', but I have a feeling there is a better way to do this.

You can use functools.partial and pass polarization_value as :
functools.partial(polarization_value, timespan='day')
Example:
>>> import functools
def func(x, y=1):
print x, y
...
>>> new_func = functools.partial(func, y=20)
>>> new_func(100)
100 20
You may also find this helpful: Python: Why is functools.partial necessary?

Python: Best way to deal with functions with long list of arguments?

I've found various detailed explanations on how to pass long lists of arguments into a function, but I still kinda doubt if that's proper way to do it.
In other words, I suspect that I'm doing it wrong, but I can't see how to do it right.
The problem: I have (not very long) recurrent function, which uses quite a number of variables and needs to modify some content in at least some of them.
What I end up with is sth like this:
def myFunction(alpha, beta, gamma, zeta, alphaList, betaList, gammaList, zetaList):
<some operations>
myFunction(alpha, beta, modGamma, zeta, modAlphaList, betaList, gammaList, modZetaList)
...and I want to see the changes I did on original variables (in C I would just pass a reference, but I hear that in Python it's always a copy?).
Sorry if noob, I don't know how to phrase this question so I can find relevant answers.

You could wrap up all your parameters in a class, like this:
class FooParameters:
alpha = 1.0
beta = 1.0
gamma = 1.0
zeta = 1.0
alphaList = []
betaList = []
gammaList = []
zetaList = []
and then your function takes a single parameter instance:
def myFunction(params):
omega = params.alpha * params.beta + exp(params.gamma)
# more magic...
calling like:
testParams = FooParameters()
testParams.gamma = 2.3
myFunction(testParams)
print params.zetaList
Because the params instance is passed by reference, changes in the function are preserved.

This is commonly used in matplotlib, for example. They pass the long list of arguments using * or **, like:
def function(*args, **kwargs):
do something
Calling function:
function(1,2,3,4,5, a=1, b=2, b=3)
Here 1,2,3,4,5 will go to args and a=1, b=2, c=3 will go to kwargs, as a dictionary. So that they arrive at your function like:
args = [1,2,3,4,5]
kwargs = {a:1, b:2, c:3}
And you can treat them in the way you want.

I don't know where you got the idea that Python copies values when passing into a function. That is not at all true.
On the contrary: each parameter in a function is an additional name referring to the original object. If you change the value of that object in some way - for example, if it's a list and you change one of its members - then the original will also see that change. But if you rebind the name to something else - say by doing alpha = my_completely_new_value - then the original remains unchanged.

You may be tempted to something akin to this:
def myFunction(*args):
var_names = ['alpha','beta','gamma','zeta']
locals().update(zip(var_names,args))
myFunction(alpha,beta,gamma,zeta)
However, this 'often' won't work. I suggest introducing another namespace:
from collections import OrderedDict
def myFunction(*args):
var_names = ['alpha','beta','gamma','zeta']
vars = OrderedDict(zip(var_names,args))
#get them all via vars[var_name]
myFunction(*vars.values()) #since we used an orderedDict we can simply do *.values()

you can capture the non-modfied values in a closure:
def myFunction(alpha, beta, gamma, zeta, alphaList, betaList, gammaList, zetaList):
def myInner(g=gamma, al, zl):
<some operations>
myInner(modGamma, modAlphaList, modZetaList)
myInner(al=alphaList, zl=zetaList)
(BTW, this is about the only way to write a truly recursive function in Python.)

You could pass in a dictionary and return a new dictionary. Or put your method in a class and have alpha, beta etc. be attributes.

You should put myFunction in a class. Set up the class with the appropriate attributes and call the appropriate functions. The state is then well contained in the class.

Do you change variables AFTER you run a function in python?

So I wrote this function from a book I am reading, and this is how it starts:
def cheese_and_crackers(cheese_count, boxes_of_crackers):
print "You have %d cheeses!" % cheese_count
print "You have %d boxes of crackers!" % boxes_of_crackers
print "Man that's enough for a party!"
print "Get a blanket.\n"
ok, makes sense. and then, this is when this function is run where I got a little confused and wanted to confirm something:
print "OR, we can use variables from our script:"
amount_of_cheese = 10
amount_of_crackers = 50
cheese_and_crackers(amount_of_cheese, amount_of_crackers)
the thing that confused me here is that the amount_of_cheese and amount_of_crackers is changing the variables (verbage? not sure if i am saying the right lingo) from cheese_count and boxes_of_crackers repectively from the first inital variable labels in the function.
so my question is, when you are using a different variable from the one that is used in the initial function you wrote, why would you change the name of the AFTER you wrote out the new variable names? how would the program know what the new variables are if it is shown after it?
i thought python reads programs top to bottom, or does it do it bottom to top?
does that make sense? i'm not sure how to explain it. thank you for any help. :)
(python 2.7)

I think you are just a bit confused on the naming rules for parameter passing.
Consider:
def foo(a, b):
print a
print b
and you can call foo as follows:
x = 1
y = 2
foo(x, y)
and you'll see:
1
2
The variable names of the arguments (a, b) in the function signature (1st line of function definition) do not have to agree with the actual variable names used when you invoke the function.
Think of it as this, when you call:
foo(x, y)
It's saying: "invoke the function foo; pass x in as a, pass y in as b". Furthermore, the arguments here are passed in as copies, so if you were to modify them inside the function, it won't change the values outside of the function, from where it was invoked. Consider the following:
def bar(a, b):
a = a + 1
b = b + 2
print a
x = 0
y = 0
bar(x, y)
print x
print y
and you'll see:
1
2
0
0

The script runs from top to bottom. The function executes when you call it, not when you define it.
I'd suggest trying to understand concepts like variables and function argument passing first.

def change(variable):
print variable
var1 = 1
change(var1)
In the above example, var1 is a variable in the main thread of execution.
When you call a function like change(), the scope changes. Variables you declared outside that function cease to exist so long as you're still in the function's scope. However, if you pass it an argument, such as var1, then you can use that value inside your function, by the name you give it in the function declaration: in this case, variable. But it is entirely separate from var! The value is the same, but it is a different variable!

Your question relates to function parameter transfer.
There are two types of parameter transfer into a function:
By value ------- value changed in function domain but not global domain
By reference ------- value changed in global domain
In python, non-atomic types are transferred by reference; atomic types (like string, integer) is transferred by value.
For example,
Case 1:
x = 20
def foo(x):
x+=10
foo()
print x // 20, rather than 30
Case 2:
d = {}
def foo(x): x['key']=20
foo(d)
print d // {'key': 20}