Develop Reference

pass one function to another, where inner function may access variable in the larger function, python

I have a library function to which a user can pass their own function that will be executed at various points during the library function's execution. Sometimes, these user functions may want to access (not modify) variables that only exist inside the library function. This is the current implementation:
def lib_func(user_func=None):
lib_var1 = 'somevalue'
lib_var2 = 'othervalue'
...
if user_func: user_func(lib_var1,lib_var2)
...
if user_func: user_func(lib_var1,lib_var2)
The problem is that sometimes the user's function may not need lib_var1 or lib_var2, and currently they still need to write them as arguments:
def my_func(a,b):
<some-code-not-involving-a-or-b>
lib_func(my_func)
Is there a better way to deal with this situation where the user doesn't need to have unnecessary arguments in their function?

One approach is to require different functions for different contexts. If some argument is not passed to the function in some case, then the function does not have the same role in that context.
Ask yourself if the function has the same role in both cases, if it does not maybe require multiple functions.
def lib_func(fun_context1=None, fun_context2=None):
# case with all arguments
if fun_context1:
fun_context1(lib_var1, lib_var2)
# specialized case with only one argument
if fun_context2:
fun_context2(lib_var1)
# User's function
def my_func1(foo, bar):
...
def my_func2(foo):
...
lib_func(my_func1, my_func2)
The above solution works to a point... there may well be an unpractical amount of special cases. If so, you could require the user to provide a function which takes a **kwargs argument. This will give the user's function the flexibility to check which parameters were passed to it and what to do in each case.
def lib_func(user_func=None):
if user_func:
user_func(lib_var1=lib_var1, lib_var2=lib_var2)
# case with only one parameters
if user_func:
user_func(lib_var1=lib_var1)
# User's function
def my_func(**params):
lib_var1 = params.get("lib_var1")
lib_var2 = params.get("lib_var2")
lib_func(my_func)
Another benefit of this approach in maintainability. If your library ever provide additional keyword arguments to the user-defined function, this will not break the user's code.

Optional arguments with few mandatory arguments

I am new to python and came across a problem.
I have to get a dictionary from the user via a function that has few elements (key-value). At the same time, if the user is not providing the dictionary, they can provide individual elements as arguments.
How do I check if user has provided dictA, if not, src msg. If the user doesn't provides either one of them, return an error back to the calling function.
Lets say:
def myfunc(dictA, src, msg, e=True)
where dictA is the dictonary and src, msg are arguements if the user didn't provide dictA.

I wouldn't support them passing in dictA or the other arguments. I would just have them pass in dictA as kwargs if that want to use that. Then you can just check if the values have been there
New function prototype might be
def myfunc(src, msg, e=True)
And then usages could be
myfunc(a_source, a_msg)
Or
myfunc(**dictA)
This way your function will automatically check that those values are at least present. And any additional value checking can be done on only one input
For example the following call would fail (without you needing to do anything else) because it would still be missing the positional argument of msg.
myfunc(**{'src': 'a_source'})
Similarly it will fail if you send too many positional arguments as well (i.e. passing in a positional argument and a kwarg via a dictionary)

forcing value of lambda inner scope variable to outer variable - python

I have already found various answers to this question (eg. lambda function acessing outside variable) and all point to the same hack, namely (eg.) lambda n=i : n*2 with i a variable in the external scope of lambda (hoping I'm not misusing the term scope). However, this is not working and given that all answers I found are generally from couple of years ago, I thought that maybe this has been deprecated and only worked with older versions of python. Does anybody have an idea or suggestion on how to solve this?
SORRY, forgot the MWE
from inspect import getargspec
params = ['a','b']
def test(*args):
return args[0]*args[1]
func = lambda p=params : test(p)
I expected the signature of func to be ['a','b'] but if I try
func(3,2)
I get a Type error (TypeError: <lambda>() takes at most 1 argument (2 given) )
and it's true signature (from getargspec(func)[0] ) is ['p']
In my real code the thing is more complicated. Shortly:
def fit(self, **kwargs):
settings = self.synch()
freepars = self.loglike.get_args()
func = lambda p=freeparams : self.loglike(p)
minuit = Minuit(func,**settings)
I need lambda because it's the only way I could think to create inplace a function object depending on a non-hardcoded list of variables (extracted via a method get_params() of the instance self.loglike). So func has to have the correct signature, to match the info inside the dict settings
The inspector gives ['p'] as argument of func, not the list of parameters which should go in loglike. Hope you can easily spot my mistake. Thank you

There's no way to do exactly what you want. The syntax you're trying to use to set the signature of the function you're creating doesn't do what you want. It instead sets a default value for the argument you've defined. Python's function syntax allows you to define a function that accepts an arbitrary number of arguments, but it doesn't let you define a function with argument names in a variable.
What you can do is accept *args (or **kwargs) and then do some processing on the value to match it up with a list of argument names. Here's an example where I turn positional arguments in a specific order into keyword arguments to be passed on to another function:
arg_names = ['a', 'b']
def foo(*args):
if len(args) != len(arg_names):
raise ValueError("wrong number of arguments passed to foo")
args_by_name = dict(zip(arg_names, args))
some_other_function(**args_by_name)
This example isn't terribly useful, but you could do more sophisticated processing on the args_by_name dict (e.g. combining it with another dict), which might be relevant to your actual use case.

passing the values to a function

I have a function that returns the replaced values correctly, but for some reason, the run_instances function receives the entire string as a single object (instead of 4 separate values).
import boto
ec2_conn = boto.connect_ec2(aws_access_key_id='XXX', aws_secret_access_key='XXX')
ami='ami-XXX'
key_name='XXX15a.pem'
instance_type='t1.macro'
aid="image_id='%s', placement='us-east-1a', key_name='%s', instance_type='%s'" % (ami, key_name, instance_type)
When I try to execute the run_instances function...
ec2_conn.run_instances(aid)
<Message>Invalid id: "image_id='ami-XXX', placement='us-east-1a', key_name='XXX.pem', instance_type='t1.macro'" (expecting "ami-...")</Message>
Is there any way to pass the values to the function correctly?

Simplifying the problem statement to: how to pass multiple variables around so they can be passed into a function later on without passing all variables individually...:
params = dict(ami='ami-XXX', key_name='XXX15a.pem', instance_type='t1.macro', placement='us-east-1a')
ec2_conn.run_instances(**params)
Store them in a dict and expand them to keyword arguments with **.

Is it in Python possible to set the last parameter of several in a function call?

Is it in Python possible to set the last parameter of several in a function call if all have a default value?
Example: in the ftplib.FTP the module defines this:
ftplib.FTP([host[, user[, passwd[, acct[, timeout]]]]])
All these parameters have a default value, so you don´t need to set them all, you could, for example, just call ftp = ftplib.FTP() to get a FTP object. But what if I would like to set only the timeout parameter? How would I achieve this?

You can pass this (or any other) parameter by name:
ftplib.FTP(timeout=10)

Yes, by using named arguments you can specify the value with a name-value pair.
ftp = ftplib.FTP(timeout=100)