I am having trouble pulling out values out of a function that I am trying to optimize. The code looks similar to what follows. I want to minimize c by changing x through scipy.optimize.minimize, but am also trying to pull a and b out of the function as well. What is the best way to do this?
def function(x, inputs)
a = math
b = math
c = math
return(c)
You should be able to just compute a and b outside of the function. It doesn't need to be part of the function if the result is not optimized. Just use x, inputs, and c to compute a and b outside of the function.
My question is about fitting parameters of a complicated model composed of different parametric functions.
More precisely, I want to describe a complicated experiment.
The experiment produces a one-dimensional array of measured data data, where each its entries corresponds to (a set of) experimental control variables x.
I now a theoretical model (actually multiple models, see below) model(x,pars), which takes x and a lot of parameters pars to give a prediction for data. However, not all parameters are known and I need to fit them.
Moreover, some details of the model are not yet certain. Because of that, I actually have a family of multiple models which are in some parts very similar, but where some internal component of the model is different (but a large part of the model is the same).
Unfortunately, switching one component for another might introduce new (unknown) parameters, that is we now have modelA(x,parsA) and modelB(x,parsB)
which have different parameters.
Basically, the model is composed of functions f(x, pars, vals_of_subfuncs) where x is the independent variable, pars are some explicit parameters of f, and vals_of_subfuncs are the results of evaluating some lower-level functions, which themselves depend on their own parameters (and maybe the results of their own lower-level functions etc.)
Obviously, there are no recursions possible, and at there is a lowest level of functions that do not rely on the value of other functions.
The situation is best illustrated in this picture:
Modular model architecture
The independent variable is x (blue), parameters are a,b,c,d (red), and the values of subfunctions appear as green arrows into nodes that represent functions.
In (1), we have a lowest-level function G(x; (a,b); {}) with no sub-functions and a higher-level function F(x; c; G(x; (a,b)) whose evaluation gives the model result, which depends on x and pars=(a,b,c).
In (2) and (3) we change a component of the model, (F->F') and (G->G'), respectively. This changes the parameter dependence of the final model.
Now I am looking for a most pythonic/modular way to approach the problem of implementing parameter fitting in this situation, without having to re-write the fit function everytime I swap/change a component of my model, thereby possibly introducing a new parameter.
At the moment, I am trying to find solutions to this problem using lmfit. I also thought about maybe trying to use sympy to work with symbolic "parameters", but I don't think all the functions that appear can be easily written as expressions that can be evaluated by asteval.
Does anyone know of a natural way to approach such a situation?
I think this question would definitely be improved with a more concrete example, (that is, with actual code). If I understand correctly, you have a general model
def model_func(x, a, b, c, d):
gresult = G(x, a, b, d)
return F(x, b, c, gresult)
but you also want to control whether d and b are really variables, and whether c gets passed to F. Is that correct?
If that is correct (or at least captures the spirit), then I think you can do this with lmfit (disclaimer: I'm a lead author) with a combination of adding keyword arguments to the model function and setting some Parameter values as fixed.
For example, you might do some rearranging like this:
def G(x, a, b=None, d=None):
if b is not None and d is None:
return calc_g_without_d(x, a, b)
return calc_g_with_d(x, a, d)
def F(x, gresult, b, c=None):
if c is None:
return calc_f_without_c(x, gresult, b)
return calc_f_with_c(x, gresult, b, c)
def model_func(x, a, b, c, d, g_with_d=True, f_with_c=True):
if g_with_d:
gresult = G(x, a, d)
else:
gresult = G(x, a, b)
if f_with_c:
return F(x, gresult, b, c=c)
else:
return F(x, gresult, b)
Now, when you make your model you can override the default values f_with_c and/or g_with_d:
import lmfit
mymodel = lmfit.Model(model_func, f_with_c=False)
params = mymodel.make_params(a=100, b=0.2201, c=2.110, d=0)
and then evaluate the model with mymodel.eval() or run a fit with mymodel.fit() and passing in explicit values for the keyword arguments f_with_c and/or g_with_d, like
test = mymodel.eval(params, x=np.linspace(-1, 1, 41),
f_with_c=False, g_with_d=False)
or
result = mymodel.fit(ydata, params, x=xdata, g_with_d=False)
I think the way you have it specified, you'd want to make sure that d was not a variable in the fit when g_with_d=False, and there are cases where you would want b to not vary in the fit. You can do that with
params['b'].vary = False
params['d'].vary = False
as needed. I can imagine your actual problem is slightly more involved than that, but I hope that helps get you started in the right direction.
Thanks for the answers.
I think lmfit might be able to do what I want, but I will have to implement the "modularity" myself.
The example I have was just conceptional and a miminal model. In general, the "networks" of function and their dependencies are much more intricate that what I do in the example.
My current plan is as follows:
I will write a class Network for the "network", which contains certain Nodes.
Notes specify their possible "symbolic" dependence on subNodes, explicit parameters and independent variables.
The Network class will have routines to check that the such constructed network is consisten. Moreover, it will have a (lmfit) Parameters object (i.e. the unification of all the parameters that the nodes explicitly depend on) and provide some method to generate an lmfit Model from that.
Then I will use lmfit for the fitting.
At least this is the plan.
If I succeed in building this, I will publish update this post with my code.
Since you brought up sympy, I think you should take a look at symfit, which does precisely what you ask for in the last paragraph. With symfit you can write symbolic expressions, which are then fitted with scipy. It will make it very easy for you to combine your different submodels willy-nilly.
Let me implement your second example using symfit:
from symfit import variables, parameters, Fit, Model
a, b, c = parameters('a, b, c')
x, G, F = variables('x, G, F')
model_dict = {
G: a * x + b,
F: b * G + c * x
}
model = Model(model_dict)
print(model.connectivity_mapping)
I choose these rather trivial functions, but you can obviously choose whatever you want. To see that this model matches your illustration, this is what connectivity_mapping prints:
{F: {b, G, x, c}, G: {b, a, x}}
So you see that this is really a mapping representing what you drew. (The arguments are in no particular order within each set, but they will be evaluated in the right order, e.g. G before F.) To then fit to your data, simply do
fit = Fit(model, x=xdata, F=Fdata)
fit_results = fit.execute()
And that's it! I hope this makes it clearer why I think symfit does fit your use case. I'm sorry I couldn't clarify that earlier, I was still finalizing this feature into the API so up to now it only existed in the development branch. But I made a release with this and many other features just now :).
Disclaimer: I'm the author of symfit.
I have a problem, I have 2 lists with x and y values, and I would like to create a function based on these. But the problem is that I would like to build a function like this one:
f(x) = a * (x-b)**c
I already know scipy.interpolate but I couldn't find anything to return a function like this one.
is there a quite easy way to try to create the best function I can by searching which values of a,b and c match the most?
thanks for your help!
Edit:
here is what my current values of x and y look like:
I created this function :
def problem(values):
s = sum((y - values[0]*(x-values[1])**values[2])**2 for x,y in zip(X,Y))
return(s)
and I tried to find the best values of a,b and c with scipy.optimize.minimize but I don't know with which values of a,b and c I should start...
values = minimize(problem,(a,b,c))
(Edited to account for the OP's added code and sub-question.)
The general idea is to use a least-squares minimization to find the "best" values of a, b, and c. First define a function whose parameters are a, b, c that returns the sum of the squares of the differences between the given y values and the calculated values of a * (x-b)**c. (That function can be done as a one-liner.) Then use an optimization routine, such as one found in scipy, to minimize the value of that function value. Those values of a, b, c are what you want--use them to define your desired function.
There are a few details to examine, such as restrictions on the allowed values of a, b, c, but those depend somewhat on your lists of x and y values.
Now that you have shown a graph of your x and y values, I see that your values are all positive and the function is generally increasing. For that common situation I would use the initial values
a = 1.0
b = 0.0
c = 1.0
That gives a straight line through the origin, in fact the line y = x, which is often a decent first guess. In your case the x and y values have a very different scale, with y about a hundred times larger than x, so you would probably get better results with changing the value of a:
a = 100.0
b = 0.0
c = 1.0
I can see even better values and some restrictions on the end values but I would prefer to keep this answer more general and useful for other similar problems.
Your function problem() looks correct to me, though I would have written it a little differently for better clarity. Be sure to test it.
def problem (a , b, c, d):
return a * (x[d]-b)**c
I guess is what you are after. With D being what value of the X array. Not sure where Y comes into it.
I would like to replicate the following python function in lua/torch.
def add_and_multiply(a,b):
c=a+b;
d=a*b;
return c,d
How can I return two values simultaneously in lua/torch as above?
Also, suppose a and b were matrices(with appropriate dimensions), how would the code change for torch?
The same thing happens in lua too. You can even skip using two extra local variables:
function add_and_multiply(a,b)
return a + b, a * b
end
I have a number of methods that are independent of each other but are needed collectively to compute an output. Thus, when a variable in any of the methods changes all the methods are called in the computation which is slow and expensive. Here is a quick pesudo-code of what I have:
# o represents an origin variable
# valueA represents a variable which can change
def a (o, valueA):
# calculations
return resultA
def b (o, valueB):
# calculations
return resultA
def c (o, valueC1, valueC2):
# calculations
return resultA
def compute (A, B, C1, C2):
one = self.a(o, A)
two = self.b(one,B)
three = self.c(two, C1, C2)
return img
For example when the value of C1 changes, when calling compute all the methods are calculated despite a & b having no change. What I would like is some way of checking which of the values of A,B,C1,C2 have changed between each call to compute.
I have considered defining a list of the values then on the next call comparing it to the new values being pass to compute. Eg; 1st call: list=[1,2,3,4] on 2nd call list=[1,3,4,5] so b & c need calculating but a is the same. However, I am unsure as to how to go from the comparison to defining which method to call?
Some background on my particular application in case it is of use. I have a wxPython window with sliders that determine values for image processing and an image is drawn on each change of these sliders.
What is the best way to compare each call to compute and remove these wasted repeated computations?
If i have to solve this, I would use a Dictionary, where the key is the valueX (or a list of it if have more than one, in your example C) and the value should be the result of the function.
So, you should have something like that:
{ valueA: resultA, valueB: resultB, [valueC1, valueC2]: resultC }
To do that, in the functions you will have to add it:
def a(o, valueA):
[calcs]
dic[valueA] = resultA
return resultA
[...]
def c(o, valueC1, valueC2)
[calcs]
dic[[valueC1, valueC2]] = resultC
return resultC
And, in the function that computes, you can try to get the value for the parameters and if not get the value, calculate it
def compute (A, B, C1, C2):
one = dic.get(A) if dic.get(A) else self.a(o, A)
two = dic.get(B) if dic.get(B) else self.b(one,B)
three = dic.get([C1,C2]) if dic.get([C1,C1]) else self.c(two, C1, C2)
return img
P.D: this is the "crude" implementation of memoize functions that #holdenweb pointed in his comment.
You could consider making the methods memoizing functions that use a dict to look up the results of previously stored computations (probably best in the class namespace to allow memoizing to optimize across all instances).
The memory requirements could be quite severe, however, if the methods are called with many arguments, in which case you might want to adopt a "publish and subscribe" pattern to try and make your computation more "systolic" (driven by changes in the data, loosely).
That' a couple of approaches. I'm sure SO will think of more.