I am trying to write some code to perform a statistical test called model-reduction. Basically what I want to know is whether each variable in my function makes a meaningful contribution (i.e. significantly explains variance). Say for example my original fit-function looks like this:
full_model(x, a, b, c, d):
return a + b*x + c*x**3 + sin(d*x)
I want to compare reduced forms of this model. The once I need to check are:
reduced = lambda x, b, c, d: full_model(x, 0, b, c, d)
reduced = lambda x, a, c, d: full_model(x, a, 0, c, d)
reduced = lambda x, a, b, d: full_model(x, a, b, 0, d)
reduced = lambda x, a, b, c: full_model(x, a, b, c, 0)
For each case, I run some sort of test that I don't go into detail:
compare_models(full_model, reduced, x, y)
In reality, my fit function has more parameters, and I want test even further reduced functions. The code will be really messy if I have to explicitly define all possible models. Is there any way to define the reduced function in a for-loop? And is there any existing python module that can achieve what I want to do?
I would harness functools.partial for that following way, consider following simplified example:
import functools
def sum3(x, y, z):
return x+y+z
args = ["x", "y", "z"]
red_dict = {}
for arg in args:
red_dict[arg] = functools.partial(sum3, **{arg: 0})
print(red_dict["x"](y=10,z=10))
print(red_dict["y"](x=10,z=10))
print(red_dict["z"](x=10,y=10))
Output:
20
20
20
Explanation: args is list of args names you want to zero, in for-loop I use argument unpacking (**) to fix selected argument value to zero, then I store result in red_dict. Use loop is equivalent to doing:
red_dict["x"] = functools.partial(sum3, x=0)
red_dict["y"] = functools.partial(sum3, y=0)
red_dict["z"] = functools.partial(sum3, z=0)
Related
I have a function-object f, which takes 4 numeric inputs and outputs two numbers. Maybe
def f(a, b, c, d):
return a+b, c+d
or maybe
def f(a, b, c, d):
return a*c, d*c
To be clear, I don't actually know what f is, I just have it as an object.
I would like to create a new function-object, h, such that h(a,b,c,d)=x*c+y where (x,y)=f(a,b,c,d). The trouble is, I have no direct access to c, only to f.
def make_h(f):
???
return h
assert( make_h(f)(a,b,c,d) == f(a,b,c,d)[0]*c+f(a,b,c,d)[1])
Is it possible to do this in python? I have tried searching and reading some documentation, but have not found an answer (yet?).
EDIT: There is a simple answer (given below) when the signature of f is fixed. Suppose I had to do this to different functions, some with inputs (a, b, c, d), some with inputs (l, m, c), and maybe some with inputs (c, r). Would it still be possible to do what I want?
This example is strongly related to the concept of a decorator. My solution is the following:
def make_h(f):
def h(a, b, c, d):
x, y = f(a,b,c,d)
return x * c + y
return h
UPDATE. In case f has any number of arguments, we can use args, and kwargs. While it is a bad practice, if we know that one of kwargs is c, we could use the following code:
def make_h(f):
def h(*args, **kwargs):
x, y = f(*args, **kwargs)
return x * kwargs["c"] + y
return h
Suppose I have an objective function f(a,b,c). I want to find the value of b that minimizes it, holding a and c constant, and to experiment with different combinations of a and c, I prefer not to write f(a,b,c) as g(b).
from scipy.optimize import minimize
def f(a,b,c):
return((a+1)**2 + b + c/2)
res = minimize(f, x0=1, args=(a,c,),)
print(res.x)
Then how do I specify that b is the parameter that f(a,b,c) should be minimized with respect to? Does that parameter have to be expressed as x? Or should I make b the first argument of f?
As the documentation states, the signature of the function should be fun(x, *args) where x is the parameter that is minimized for. So you can just use a small wrapper around your original function:
res = minimize(lambda b, a, c: f(a, b, c), x0=1, args=(a, c))
Say I have a container class containing some numeric value:
class Container:
def __init__(self, value):
self.value = value
I would like to give those Containers to a lambdified sympy expression and as a result get a new Container:
import sympy as sp
a = Container(1)
b = Container(-0.5)
x, y = sp.symbols("x y")
f = sp.lambdify((x, y), sp.atan(x)**-1*sp.pi*10 - 1/y)
result = f(a, b) # result should be a Container with value 42
Now I actually don't mind writing some wrapper around f that unpacks and packs the values, but the problem occurs here:
Like in Numpy, I also want it to be able to handle lists or lists mixed with values:
a = Container(-47)
b = Container(19)
c = [Container(0), Container(1), Container(2), Container(3), Container(4)]
x, y, z = sp.symbols("x y z")
f = sp.lambdify((x, y, z), z**4 + x/6*z**3 + y*z**2 + (x-2*y)/6*z + 3)
result = f(a, b, c) # should be a list of Containers with values 3, 1, 4, 1 and 5
What is an elegant way to achieve this?
Edit
I already wrote a wrapper function around lambdify that unpacks the values before applying the sympy expression and packs up the result, but it involves checking whether the type of an argument is a list or a Container, which is against the whole duck-typing philosophy of Python. My actual question: is there a more pythonic way to do this that doesn't involve ugly type-checking?
I am using a function in TensorFlow which maps a set of tensors to another arrangement of tensors. For example, you might write:
data = data.map(_function)
def _function(a, b, c):
return (a + 1, b, c)
So here, you pass _function as a function variable to map, and map passes it three tensors, which are mutated in some way (here, just adding one) and returned.
My question is: Is there a way to pass in additional variables to _function?
If I want to perform a + x, and not a + 1, then how could I pass in the additional variable?
You can't do something like: data.map(_function(x)) because then you're passing the result of a function, not the function itself.
I've experimented with *arg, but I can't find a way. Any help is greatly appreciated.
You can do sth like
def extra_func(x):
def _function(a, b, c):
return (a + x, b, c)
return _function
So you can do data.map(extra_func(x))
or you can use functools.partial to fix some of a function params
I'm writing Python in functional style (I think what I'm getting at is similar to a monad?). Here's what I have so far, hardcoded for three functions. What if I had 10 or 100?
# a list of (function, function, function), each of which accept
# a scalar and return a list
funcs = [(lambda a: [a, a], lambda a: [a, a, a], lambda a: [a])] * 10
possible = []
car = 3
for a, b, c in funcs:
ra = a(car)
if ra:
rb = b(ra[0])
if rb:
rc = c(rb[0])
if rc: # last function
possible.extend(rc)
That is very monadic.
What you'd probably want to do is run a fold using the monad's bind function (using the Maybe or Either monad in this case), though your individual functions would have to return a monadic value (i.e. the list value you want returned by the original function wrapped in the monad's particular value constructor).
Your end call would be something like this (in a mix of Haskell and Python):
fold Maybe.bind Maybe.mreturn(car) funcs
(You'd want the Maybe.bind to be in a lambda probably since python doesn't do partial application like Haskell, I was just being lazy.)
Here's a python monad library/script to get you started.
Perhaps something like:
funcs = [(lambda a: [a, a], lambda a: [a, a, a], lambda a: [a])] * 10
initial = [3]
result = []
for function_chain on funcs:
running = initial
for function in function_chain:
running = function(running[0])
if not running[0]:
break
else:
result.extend(running)