Python and Excel have a different behaviour for defaults. Python is passing optional arguments by keyword, while Excel is only positional, even on defaults. As a result, an unused argument is in reality passed, as None. Assuming as example the scipy function brentq:
brentq(f, a, b, xtol=1e-12, rtol=4.4408920985006262e-16, maxiter=100, full_output=False, disp=True, *args)
calling it from Excel with xtol and rtol unset:
brentq(f,a,f,,,50)
will in reality be seen as
brentq(f,a,f,None,None,50)
and of course Python will not like None for xtol,rtol.
As an escamotage, till now, I have a function checking for default values:
def checkvals(f, args):
a = inspect.getargspec(f)
defs = zip(a.args[-len(a.defaults):], a.defaults)
for x in defs:
key = x[0]
if args[key] == None:
args[key] = x[1]
return args
and I wrap brentq as follows:
def brentq(f, a, b, xtol=1e-12, rtol=4.4408920985006262e-16, maxiter=100, full_output=False, disp=True, *args):
x = checkvals(brentq, locals())
return scipy.optimize.brentq (f, a, b, *args, x['xtol'], x['rtol'], x['maxiter'], x['full_output'], x['disp'])
It works, meaning that x['xtol'] and x['rtol'] are restored to their defaults. However, I was wondering if there is a better way to do it.
In other words: is it possible to modify locals() inside a function, and force the function to use the modified values?
If Excel passes None when you skip an argument, then put that in your Python function signature as the default argument. In fact, this is the Pythonic way to do it (for various reasons including default-argument mutability that aren't worth getting into here).
brentq(f, a, b, xtol=None, rtol=None, maxiter=100, full_output=None, disp=None, *args):
if xtol is None: xtol = 1e-12
if rtol is None: rtol = 4.4408920985006262e-16
if full_output is None: full_output = False
if disp is None: disp = true
# ... rest of function
I'm satisfied with final Kyle's suggestion. Following it the excel-python interface would be like (example tried in spyder):
import scipy.optimize
def f(x):
return -2*x**4 + 2*x**3 + -16*x**2 + -60*x + 100
#this is the wrapper for Excel
def brentqWrapper (f, a, b, xtol=None, rtol=None, maxiter=None, full_output=None, disp=None,*args):
return scipy.optimize.brentq (**{k: v for k, v in locals().items() if v!=None})
#this simulates the excel call with unset inner optional parameters:
print brentqWrapper(f,0,2,None,None,50)
>> 1.24078711375
Related
I have a class & method, each with several arguments: my_class(a,b,c).my_method(d,e,f) and I'd like to be able to only change a single argument, while holding the others constant.
Constantly copy-pasting the other constant arguments seems bad, so I'd like to create a new object wrapper_fct where I reference my_class but only provide the one argument I want to change, b, without always having to specify the remaining arguments. How would wrapper_fct() look like?
For example, wrapper_fct(my_class, b1) would return my_class(a,b1,c).my_method(d,e,f), wrapper_fct(my_class, b2) would return my_class(a,b2,c).my_method(d,e,f).
Here's an example in practice:
Loop through just the variable b and evaluate several classes/methods for each new instance of b, and append the results in a list.
I can currently do this in a for loop:
mylist1 = [] # init lists (append results here)
mylist2 = []
mylist2 = []
for b in [1,2,3,4,5]:
mylist1.append( my_class1(a,b,c).my_method(d,e,f) )
mylist2.append( my_class2(a,b,c).my_method(d,e,f) )
mylist3.append( my_class3(a,b,c).my_method(d,e,f) )
...
But it seems better to create a function loop_through_B() and use the wrapper_fct(my_class,b) as specified above. Not sure if it's the ideal solution, but maybe something like:
def loop_through_B(input_class, b_values = [1,2,3,4,5])
mylist = []
for b in b_values:
mylist.append( wrapper_fct(input_class,b) )
return mylist
loop_through_B(my_class1) # would I also have to specify the method here as well?
loop_through_B(my_class2)
loop_through_B(my_class3)
Extra Question: how would I add the ability to vary method arguments, or even multiple class & method arguments?
After #chepner pointed me in the right direction, I think the best solution is to use the lambda function:
wrapper_fct = lambda b: my_class1(a,b,c).my_method(d,e,f)
In this case, I can vary b as much as I want while holding the class arguments a,c, and method arguments d,e,f constant. Note that with lambda functions, I can also vary the method arguments and/or the class arguments. For example:
wrapper_fct_multiple = lambda b, e: my_class1(a,b,c).my_method(d,e,f)
It is also possible to do this with functools.partial, but it's not obvious to me how I would specify both class & method arguments with functools.
Anyway, here is the solution implementation using lambda:
# define the "wrapper function" outside the loop
wrapper_fct = lambda b: my_class1(a,b,c).my_method(d,e,f)
# define the function I want to use to loop through B:
def loop_through_B(class_wrapper, b_values)
mylist = []
for b in b_values:
mylist.append( class_wrapper(b) )
return mylist
# run:
loop_through_B(wrapper_fct, b_values=[1,2,3,4,5])
# Can make additional wrapper_fct2, wrapper_fct3, for my_class2, my_class3 ...
You can pass the method a dictionary of arguments, and change what the method sees by selectively updating it when calling the method.
Here's what I mean:
class MyClass:
def __init__(self, a, b, c):
self.a, self.b, self.c = a, b, c
def my_method(self, kwargs):
return sum((kwargs[key] for key in kwargs.keys()))
def __repr__(self):
classname = type(self).__name__
args = ', '.join((f'{v!r}' for v in (self.a, self.b, self.c)))
return f'{classname}({args})'
instance = MyClass('a','b','c')
print(instance) # -> MyClass('a', 'b', 'c')
kwargs = dict(d=1, e=2, f=3)
print(instance.my_method(kwargs)) # -> 6
print(instance.my_method(dict(kwargs, e=38))) # -> 42
Mock version of the problem
For a function
def f(a,b,c):
return a+b+c
The function
def fix(func, **kwargs):
fa = kwargs.get('a')
fb = kwargs.get('b')
if fa is not None and fb is not None:
def f(*args):
return func(a=fa, b=fb, c=args[0])
elif fa is not None:
def f(*args):
return func(a=fa, b=args[0], c=args[1])
elif fb is not None:
def f(*args):
return func(a=args[0],b=fb, c=args[1])
else:
def f(*args):
return func(args)
return f
allows to obtain a new function by fixing some of the parameters of func.
For example: fix(g, b=3) would give us a function like
def fixed_b_in_g(a,c):
return g(a,3,c)
Question: I would like to see if there is some trick to use fix in such a way that produces a function like
def fix_a_equal_b_in_g(a,c):
return g(a,a,c)
Concrete problem
The function scipy.stats.rv_continuous.fit allows to fit parameters of a distribution to an input sample. It allows to input some keyword arguments (like fix above does) to tell it to keep some of the parameters fixed to values that the user inputs. Internally scipy.stats.rv_continuous.fit has a function, scipy.stats.rv_continuous._reduce_func, that does more or less what dix does (better implemented than my fix for example).
In my case, rather than fixing some parameters to values, I would like to fit to keep two parameters (say a and b) equal to each other, but still free during the fitting.
We can use this function to copy a keyword argument whose name is base_kwarg_name to added_kwarg_name:
def with_copied_kwargs(func, added_kwarg_names_by_base):
def fixed_func(*args, **base_kwargs):
added_kwargs = {
added_kwarg_name: base_kwargs[base_kwarg_name]
for base_kwarg_name, added_kwarg_name in added_kwarg_names_by_base.items()
}
return func(*args, **base_kwargs, **added_kwargs)
return fixed_func
Given:
def add(*, a, b, c):
return a + b + c
then modified_add = with_copied_kwargs(add, {"b": "c"}) is equivalent to:
def modified_add(*, a, b):
return add(a=a, b=b, c=b)
with_copied_kwargs can then be used along with functools.partial to both both copy keyword arguments and provide values incrementally. modified_add = functools.partial(with_copied_kwargs(add, {"b": "c"}), a=1) is equivalent to:
def modified_add(*, b):
return add(a=1, b=b, c=b)
Note that I add * (see PEP 3102) before all parameters in functions I then apply with_copied_kwargs to because the minute people start using positional arguments, things would get messy. So better to restrict it to keyword-only arguments.
Is it possible to define a closure for a function which is already defined?
For example I'd like to have a "raw" function and a function which already has some predefined values set by a surrounding closure.
Here is some code showing what I can do with a closure to add predefined variables to a function definition:
def outer(a, b, c):
def fun(d):
print(a + b + c - d)
return fun
foo = outer(4, 5, 6)
foo(10)
Now I want to have a definition of fun outside of a wrapping closure function, to be able to call fun either with variables from a closure or by passing variables directly. I know that I need to redefine a function to make it usable in a closure, thus I tried using lambda for it:
def fun(a, b, c, d): # raw function
print(a + b + c - d)
def clsr(func): # make a "closure" decorator
def wrap(*args):
return lambda *args: func(*args)
return wrap
foo = clsr(fun)(5, 6, 7) # make a closure with values already defined
foo(10) # raises TypeError: fun() missing 3 required positional arguments: 'a', 'b', and 'c'
fun(5, 6, 7, 10) # prints 8
What I also tried is using wraps from functools, but I was not able to make it work.
But is this even possible? And if yes: Is there any module which already implements decorators for this?
You can just define the wrap on the fly:
def fun(a, b, c, d): # raw function
print(a + b + c - d)
def closed(d): fun(5,6,7,d)
closed(10)
You can use this with lambda, but #juanpa points out you should not if there is no reason to. The above code will result in 8. This method by the way is not Python specific, most languages would support this.
But if you need a closure in a sense that it relies on the wrapper variables, than no, and there is good reason not to. This will create essentially a non-working function, that relies on wrapping. In this case using a class maybe better:
class fun:
def __init__(self,*args): #Can use specific things, not just *args.
self.args = args #Or meaningful names
def __call__(self,a, b, c, d): # raw function
print(a + b + c - d,self.args)
def closed(d):
fun("some",3,"more",['args'])(5,6,7,d)
closed(10)
or using *args/**kwargs directly and passing extra variables through that. Otherwise I am not familiar with a "inner function" construct that only works after wrapping.
I would like to use functools.partial to set a certain argument to a constant and at the same time remove the argument altogether.
Let me explain it using a simple example.
from functools import partial
def f(a, b):
return a * b
g = partial(f, b=2)
However, this function g still has the following calling signature:
g?
Signature: g(a, *, b=1)
Call signature: g(*args, **kwargs)
Type: partial
String form: functools.partial(<function f at 0x7ff7045289d8>, b=1)
File: /opt/conda/envs/dev/lib/python3.6/functools.py
Docstring:
partial(func, *args, **keywords) - new function with partial application
of the given arguments and keywords.
I could of course do this with a lambda function like:
def f(a, b):
return a * b
g = lambda a: f(a, b=2)
with the correct calling signature:
g?
Signature: g(a)
Docstring: <no docstring>
File: ~/Work/<ipython-input-7-fc5f3f492590>
Type: function
The downside of using lamdba functions is that I would need to write down all the arguments again. In my simple example this doesn't matter, but look at this:
phase_func = lambda site1, site2, B_x, B_y, B_z, orbital, e, hbar: \
phase_func(site1, site2, B_x, B_y, B_z, orbital, e, hbar, xyz_offset=(0,0,0))
# or this
phase_func = partial(phase_func, xyz_offset=(0,0,0)
Why would I even want this?
Later in my code I use wrappers that can generate a new function by multiplying two other functions, like combine(function1, function2, operator.mul). This combine function looks at all the arguments, therefore I need to remove the argument after setting it.
How can I write a decorator so that decorated functions can accept (and ignore) arbitrary arguments?
I have some functions like this:
def foo(x):
return x
def foo2(x, y):
if bar(y):
return x
else:
return x + 1
def foo3(x, y, z):
...
foo() can calculate the return value for a given x just based on x, but foo2() needs another parameter, and foo3() needs a third parameter. I have a method elsewhere that, among other things, calls either foo(), foo2(), etc depending on a user-specified argument.
Right now, the method just grabs the appropriate function with getattr(user_arg) and calls it with all of x, y, z. To avoid a TypeError for the wrong number of arguments, the foo functions are all defined with *args like this:
def foo(x, *args):
return x
But I'd like to just have a decorator to avoid including *args in every single foo function definition. Is there a way to do that? Or can you suggest a better way to organize this code?
One way would be to write the method like this:
if user_arg == 'foo':
foo(x)
elif user_arg == 'foo2':
foo2(x, y)
elif user_arg == 'foo3':
foo3(x, y, z)
But I'd really like to avoid this, because there are a lot of foo functions, and because I'd like to make it possible to add new foo functions without also having to add another branch to the method.
Edit: To clarify, it's not the case that I need to call a different foo function based on the number of arguments. It's arbitrary (user-specified) which of the foo functions is called.
def foo3(x, y, z):
return x + y + z
def foo4(x, y, z):
return x + y - z
You can use inspect.getargspec to determine what arguments the decorated function takes:
import functools
import inspect
def ignore_extra_arguments(func):
args, varargs, kwvarargs, defaults = inspect.getargspec(func)
#functools.wraps(func)
def wrapper_func(*wrapper_args, **wrapper_kwargs):
if varargs is None:
# remove extra positional arguments
wrapper_args = wrapper_args[:len(args)]
if kwvarargs is None:
# remove extra keyword arguments
wrapper_kwargs = {k: v for k, v in wrapper_kwargs.iteritems() if k in args}
return func(*wrapper_args, **wrapper_kwargs)
return wrapper_func
This can be optimized a little by lifting the if checks out of wrapper_func, at the expense of writing more code.
You can use named arguments and always call foo() (the first function) in order to dispatch to the "right" function, as follows:
def foo(a, b=None, c=None, d=None):
if b:
return foo2(a, b, c, d)
else:
# do your thing
def foo2(a, b, c=None, d=None):
if c:
return foo3(a, b, c, d)
else:
# do your thing
def foo3(a, b, c, d=None):
...
Another option:
def foo(a, *b):
if b:
return foo2(a, *b)
else:
print "in foo"
# do your thing
def foo2(a, b, *c):
if c:
return foo3(a, b, *c)
else:
print "in foo2"
# do your thing
def foo3(a, b, c, *d):
print "in foo3"
foo(1, 2, 3)