I try to calculate the constants of a known mathematical model for my measured (and already filtered) data.
A little bit of pseudo code:
#my data is saved in y_data and x_data
#my model is a function of constants (a,b,c) and the x_data
model = f(x_data, a, b, c)
#set model equal to data
y_data != model
calculate(a, b, c)
Is there any way to find the constants? I know there will be no exact result...but is it possible for a deviation (e.g. 5%) of y_data?
An alternative would be to calculate the constants for each index. Then have len(x_data) equations and somehow find the best fitting model and its constants.
I tried to simplify my issue, furthermore this is my first question, so let me know when I could to something better.
Thanks in advance!
If your model is linear, the simplest methods would be least squares (linear regression). There is a nice tutorial with examples here.
Since it appears your model is non-linear (and assuming you can't or don't want to find an analytical solution), you might want to try numerical optimization.
Scipy has a function called minimize which you could use to try to find the best solution. But it is not guaranteed to find the global minimum. So you may have to try different initial conditions.
Your code might look something like this:
from scipy.optimize import minimize
# Initial guess
x0 = [1.3, 0.7, 0.8]
res = minimize(cost_function, x0, tol=1e-5)
print(res.x)
The trick is, you need to define the function cost_function that you give to the solver, first.
It's common to use quadratic errors (sum of squared-errors or mean-squared error). Something like this:
def cost_function(x):
a, b, c = x
model_predictions = f(x_data, a, b, c)
return sum((model_predictions - y_data)**2)
You might also have to try different solvers which are built into scipy.optimize.minimize. Refer to the documentation on the pros and cons of each method.
Maybe get familiar with the examples in the Scipy documentation first, then try to set it up on your actual problem.
Related
I am looking into using Nlopt for solving optimisation problems in Python.
I have a series of simultaneous equations of the form
Ax = b
where A is an NxM matrix, with x the solution. Another way to think about this is that I have N simultaneous equations of the form x_1c_1m + x_2c_2m + .... + x_Nc_Nm = k_M, where x_i are variables to solve for, c_im is a constant associated with x_i when in equation M=m, and k_m is some constant in equation M=m. c_im and k_m are all known.
What confuses me is how to even approach this in Nlopt. Nlopt requires you to have actual callable functions, which I don't have? I suppose I could generalise each of the equations in that matrix equation above to something like:
def fn(x,c_m,k_m):
val = 0
for x_i, c_im in zip(x,c_m):
val += x_i * c_im
return val - k_m
where c_m and k_m would be already known, with the variables to solve for in x. All the examples I've seen have only been looking at a single variable problem, which has kind of thrown me a little. Would I then have to somehow define M copies of this function, and set each copy of fn as an equality constraint in the Nlopt optimisation object? It's all rather confusing. I'm looking to solve for x, which itself has multiple solutions, and I want to try to find the minimum values of x (or atleast an approximate solution if an exact solution cannot be found). Would I have to then set multiple objective functions, ie obj_fn_i = min(x_i) or something like that? It's all a little confusing to me in terms of what needs to be presented to the solver. I've already got an analytical solution to the above problem, so I can check my results reliably. Any help appreciated.
Cheers!
I have been using NLopt for a couple of problems, and what I have come to understand is the solver requires an objective function which returns a float value, so you must set the function as an MSE sum, or still as a single float value to be minimized. And it can solve for an array of variables x, in which both the objective function and constraint must depend. All equations that are involved in the system you can insert either in the objective function directly, or as constraints.
Hope this was helpful somehow!
I have a function of two variables, R(t,r), that has been constructed using a list of values for R, t, and r. This function cannot be written down, the values are found from solving a differential equation (d R(t,r)/dt). I require to take the derivatives of the function, in particular, I need
dR(t,r)/dr, d^2R(t,r)/drdt. I have tried using this answer to do this, but I cannot seem to get an answer that makes sense. (note that all derivatives should be partials). Any help would be appreciated.
Edit:
my current code. I understand getting anything to work without the `Rdata' file is impossible but the file itself is 160x1001. Really, any data could be made up to get the rest to work. Z_t does not return answers that seem like the derivative of my original function based on what I know, therefore, I know it is not differentiating my function as I'd expect.
If there are numerical routines for using the array of data I do not mind, I simply need some way of figuring out the derivatives.
import numpy as np
from scipy import interpolate
data = np.loadtxt('Rdata.txt')
rvals = np.linspace(1,160,160)
tvals = np.linspace(0,1000,1001)
f = interpolate.interp2d(tvals, rvals, data)
Z_t = interpolate.bisplev(tvals, rvals, f.tck, dx=0.8, dy=0)
I am trying to fit this function to some data
This is my function:
def first_deriv(xlist, L, k, x0):
return [k*L*(math.e**(-k*(x+x0)))/(1+math.e**(-k*(x+x0)))**2 for x in xlist]
This is how the function looks, so I expected to get a pretty good fitting.
This is the code to fit the function to the data
popt = curve_fit(first_deriv, list(range(len(data))), data, bounds=((0, -np.inf, -np.inf), (10**(9), +np.inf, 0)), maxfev=10000)
And this is how I am plotting it:
xdata = list(np.linspace(-100, 100, 2000))
plt.scatter(xdata, first_deriv(xdata, popt[0][0], popt[0][1], popt[0][2]), s=1)
The bounds are there to make the answer reasonable, but even if I make all the bounds infinite, it still gives a terrible fit.
This is the terrible fitting
I'm surprised how curve_fit seems to totally screw it up.
Can someone please explain why?
There are two problems with your example.
A trivial problem
curve_fit expects the values of the independent variable as the second argument, whereas the code above gives list(range(len(data)), which will produce integer x values from 0 to however long data is. You'll notice that your scatter plot points only go up to ~50. My guess is that this is why.
Instead you should give a list of the values of the independent variable at the points where data was observed. Depending on how data is generated/collected (you don't provide this information in your question), this might be xdata.
I am also a little confused as to how the second plot fits in with the first. The y scale doesn't seem to match up for the blue line. My guess is that the plots were generated from different examples for different parameter values. I'll ignore them.
The real problem
Not all optimisation methods are equally suited to all problems. curve_fit is capable of using 3 methods, 'lm, 'trf' and 'dogbox'. The default is lm, for the Levenberg–Marquardt method, unless bounds are given, in which case 'trf' is used instead, which is a Trust Region method.
Playing with the example a bit, I found that 'lm' performed well, while 'trf' did not.
For example, take the following code:
import math
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
def first_deriv(xlist, L, k, x0):
return [k*L*(math.e**(-k*(x+x0)))/(1+math.e**(-k*(x+x0)))**2 for x in xlist]
xdata = list(np.linspace(-100, 100, 2000))
real_parameters = (320000.0, 0.1, -30.0)
fakedata = first_deriv(xdata, *real_parameters)
plt.plot(xdata, fakedata)
Which produces the curve in your example above (ish):
A comparison of the 3 methods confirms that 'lm' is looking the best, and recovers the original parameters:
lm_parameters = curve_fit(first_deriv, xdata, fakedata)[0]
trf_parameters = curve_fit(first_deriv, xdata, fakedata, method='trf')[0]
dogbox_parameters = curve_fit(first_deriv, xdata, fakedata, method='dogbox')[0]
plt.scatter(xdata, first_deriv(xdata, *lm_parameters), s=1, label='lm')
plt.scatter(xdata, first_deriv(xdata, *trf_parameters), s=1, label='trf')
plt.scatter(xdata, first_deriv(xdata, *dogbox_parameters), s=1, label='dogbox')
plt.legend()
An interesting question (that perhaps deserves its own post) is why this is the case. While I am unqualified to make a precise mathematical argument, playing with the parameters of your function suggests some rough ideas.
For example, 'broadening the peak' of your function seems to allow all methods to perform well.
No doubt changing the parameters has altered the 'fitness landscape' in such a way to allow the trust region methods to succeed.
It is also possible that certain parameters of the 'trf' and 'dogbox' methods themselves might produce better results. This would require a more in-depth knowledge of the methods.
Having said that, 'lm' seems to be the best method for this particular problem. It is always important to be aware of which method you are using, and to experiment with different ones for each new problem, especially if you are getting bad results.
Optimization procedures can get trapped in local maxima (when any change to the parameters would first make the fit worse before it would get better). To avoid this problem (and to speed up the computations), scipy.optimize.curve_fit() allows you to specify your best guesses for the parameters with the p0 keyword argument.
So you should pass the parameter values you used for your plot of the function above to curve_fit() as a starting point.
I am using a scipy.minimize function, where I'd like to have one parameter only searching for options with two decimals.
def cost(parameters,input,target):
from sklearn.metrics import mean_squared_error
output = self.model(parameters = parameters,input = input)
cost = mean_squared_error(target.flatten(), output.flatten())
return cost
parameters = [1, 1] # initial parameters
res = minimize(fun=cost, x0=parameters,args=(input,target)
model_parameters = res.x
Here self.model is a function that performs some matrix manipulation based on the parameters. Input and target are two matrices. The function works the way I want to, except I would like to have parameter[1] to have a constraint. Ideally I'd just like to give an numpy array, like np.arange(0,10,0.01). Is this possible?
In general this is very hard to do as smoothness is one of the core-assumptions of those optimizers.
Problems where some variables are discrete and some are not are hard and usually tackled either by mixed-integer optimization (working good for MI-linear-programming, quite okay for MI-convex-programming although there are less good solvers) or global-optimization (usually derivative-free).
Depending on your task-details, i recommend decomposing the problem:
outer-loop for np.arange(0,10,0.01)-like fixing of variable
inner-loop for optimizing, where this variable is fixed
return the model with the best objective (with status=success)
This will effect in N inner-optimizations, where N=state-space of your to fix-var.
Depending on your task/data, it might be a good idea to traverse the fixing-space monotonically (like using np's arange) and use the solution of iteration i as initial-point for the problem i+1 (potentially less iterations needed if guess is good). But this is probably not relevant here, see next part.
If you really got 2 parameters, like indicated, this decomposition leads to an inner-problem with only 1 variable. Then, don't use minimize, use minimize_scalar (faster and more robust; does not need an initial-point).
I am having an equation
Ax=By
Where A and B are tridiagonal matrices. I want to calculate a matrix
C=inv (A).B
there are different x,s which will give different y,s hence calculation of C is handy.
Can someone please tell me a faster method to compute the inverse. I am using Python 3.5 and prefer if we use any method from numpy. If not possible I can use scipy or cython as second and third choice.
I have seen other similar questions but they do not fully match with my problem.
Thank you
There are many method to do it, anyway one of the simplest is the Tridiagonal matrix algorithm see the Wiki page. This algorithm work in O(n) time, there is a simple implementation in Numpy at the following Github link.
However, you may think to implement by yourself one of the known algorithm, for example something like a LU factorization
scipy.linalg.solve_banded is a wrapper for LAPACK which should in turn call MKL. It seems to run O(N). For a trivial example to show syntax
a = np.array([[1,2,0,0], [-1,2,1,0], [0,1,3,1], [0,0,1,2]])
x = np.array([1,2,3,4])
b = np.dot(a,x)
ab = np.empty((3,4))
ab[0,1:] = np.diag(a,1)
ab[1,:] = np.diag(a,0)
ab[2,:-1] = np.diag(a,-1)
y = solve_banded((1,1),ab,b)
print y