I am trying to fit this function to some data
This is my function:
def first_deriv(xlist, L, k, x0):
return [k*L*(math.e**(-k*(x+x0)))/(1+math.e**(-k*(x+x0)))**2 for x in xlist]
This is how the function looks, so I expected to get a pretty good fitting.
This is the code to fit the function to the data
popt = curve_fit(first_deriv, list(range(len(data))), data, bounds=((0, -np.inf, -np.inf), (10**(9), +np.inf, 0)), maxfev=10000)
And this is how I am plotting it:
xdata = list(np.linspace(-100, 100, 2000))
plt.scatter(xdata, first_deriv(xdata, popt[0][0], popt[0][1], popt[0][2]), s=1)
The bounds are there to make the answer reasonable, but even if I make all the bounds infinite, it still gives a terrible fit.
This is the terrible fitting
I'm surprised how curve_fit seems to totally screw it up.
Can someone please explain why?
There are two problems with your example.
A trivial problem
curve_fit expects the values of the independent variable as the second argument, whereas the code above gives list(range(len(data)), which will produce integer x values from 0 to however long data is. You'll notice that your scatter plot points only go up to ~50. My guess is that this is why.
Instead you should give a list of the values of the independent variable at the points where data was observed. Depending on how data is generated/collected (you don't provide this information in your question), this might be xdata.
I am also a little confused as to how the second plot fits in with the first. The y scale doesn't seem to match up for the blue line. My guess is that the plots were generated from different examples for different parameter values. I'll ignore them.
The real problem
Not all optimisation methods are equally suited to all problems. curve_fit is capable of using 3 methods, 'lm, 'trf' and 'dogbox'. The default is lm, for the Levenberg–Marquardt method, unless bounds are given, in which case 'trf' is used instead, which is a Trust Region method.
Playing with the example a bit, I found that 'lm' performed well, while 'trf' did not.
For example, take the following code:
import math
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
def first_deriv(xlist, L, k, x0):
return [k*L*(math.e**(-k*(x+x0)))/(1+math.e**(-k*(x+x0)))**2 for x in xlist]
xdata = list(np.linspace(-100, 100, 2000))
real_parameters = (320000.0, 0.1, -30.0)
fakedata = first_deriv(xdata, *real_parameters)
plt.plot(xdata, fakedata)
Which produces the curve in your example above (ish):
A comparison of the 3 methods confirms that 'lm' is looking the best, and recovers the original parameters:
lm_parameters = curve_fit(first_deriv, xdata, fakedata)[0]
trf_parameters = curve_fit(first_deriv, xdata, fakedata, method='trf')[0]
dogbox_parameters = curve_fit(first_deriv, xdata, fakedata, method='dogbox')[0]
plt.scatter(xdata, first_deriv(xdata, *lm_parameters), s=1, label='lm')
plt.scatter(xdata, first_deriv(xdata, *trf_parameters), s=1, label='trf')
plt.scatter(xdata, first_deriv(xdata, *dogbox_parameters), s=1, label='dogbox')
plt.legend()
An interesting question (that perhaps deserves its own post) is why this is the case. While I am unqualified to make a precise mathematical argument, playing with the parameters of your function suggests some rough ideas.
For example, 'broadening the peak' of your function seems to allow all methods to perform well.
No doubt changing the parameters has altered the 'fitness landscape' in such a way to allow the trust region methods to succeed.
It is also possible that certain parameters of the 'trf' and 'dogbox' methods themselves might produce better results. This would require a more in-depth knowledge of the methods.
Having said that, 'lm' seems to be the best method for this particular problem. It is always important to be aware of which method you are using, and to experiment with different ones for each new problem, especially if you are getting bad results.
Optimization procedures can get trapped in local maxima (when any change to the parameters would first make the fit worse before it would get better). To avoid this problem (and to speed up the computations), scipy.optimize.curve_fit() allows you to specify your best guesses for the parameters with the p0 keyword argument.
So you should pass the parameter values you used for your plot of the function above to curve_fit() as a starting point.
Related
I have a function of two variables, R(t,r), that has been constructed using a list of values for R, t, and r. This function cannot be written down, the values are found from solving a differential equation (d R(t,r)/dt). I require to take the derivatives of the function, in particular, I need
dR(t,r)/dr, d^2R(t,r)/drdt. I have tried using this answer to do this, but I cannot seem to get an answer that makes sense. (note that all derivatives should be partials). Any help would be appreciated.
Edit:
my current code. I understand getting anything to work without the `Rdata' file is impossible but the file itself is 160x1001. Really, any data could be made up to get the rest to work. Z_t does not return answers that seem like the derivative of my original function based on what I know, therefore, I know it is not differentiating my function as I'd expect.
If there are numerical routines for using the array of data I do not mind, I simply need some way of figuring out the derivatives.
import numpy as np
from scipy import interpolate
data = np.loadtxt('Rdata.txt')
rvals = np.linspace(1,160,160)
tvals = np.linspace(0,1000,1001)
f = interpolate.interp2d(tvals, rvals, data)
Z_t = interpolate.bisplev(tvals, rvals, f.tck, dx=0.8, dy=0)
I am having a problem with a function I am trying to fit to some data. I have a model, given by the equation inside the function which I am using to find a value for v. However, the order in which I write the variables in the function definition greatly effects the value the fit gives for v. If, as in the code block below, I have def MAR_fit(v,x) where x is the independent variable, the fit gives a value for v hugely different from if I have the definition def MAR_fit(x,v). I haven't had a huge amount of experience with the curve_fit function in the scipy package and the docs still left me wondering.
Any help would be great!
def MAR_fit(v,x):
return (3.*((2.-1.)**2.)*0.05*v)/(2.*(2.-1.)*(60.415**2.)) * (((3.*x*((2.-1.)**2.)*v)/(60.415**2.))+1.)**(-((5./2.)-1.)/(2.-1.))
x = newCD10_AVB1_AMIN01['time_phys'][1:]
y = (newCD10_AVB1_AMIN01['MAR'][1:])
popt_tf, pcov = curve_fit(MAR_fit, x, y)
Have a look at the documentation again, it says that the callable that you pass to curve_fit (the function you are trying to fit) must take the independent variable as its first argument. Further arguments are the parameters you are trying to fit. You must use MAR_fit(x,v) because that is what curve_fit expects.
I try to calculate the constants of a known mathematical model for my measured (and already filtered) data.
A little bit of pseudo code:
#my data is saved in y_data and x_data
#my model is a function of constants (a,b,c) and the x_data
model = f(x_data, a, b, c)
#set model equal to data
y_data != model
calculate(a, b, c)
Is there any way to find the constants? I know there will be no exact result...but is it possible for a deviation (e.g. 5%) of y_data?
An alternative would be to calculate the constants for each index. Then have len(x_data) equations and somehow find the best fitting model and its constants.
I tried to simplify my issue, furthermore this is my first question, so let me know when I could to something better.
Thanks in advance!
If your model is linear, the simplest methods would be least squares (linear regression). There is a nice tutorial with examples here.
Since it appears your model is non-linear (and assuming you can't or don't want to find an analytical solution), you might want to try numerical optimization.
Scipy has a function called minimize which you could use to try to find the best solution. But it is not guaranteed to find the global minimum. So you may have to try different initial conditions.
Your code might look something like this:
from scipy.optimize import minimize
# Initial guess
x0 = [1.3, 0.7, 0.8]
res = minimize(cost_function, x0, tol=1e-5)
print(res.x)
The trick is, you need to define the function cost_function that you give to the solver, first.
It's common to use quadratic errors (sum of squared-errors or mean-squared error). Something like this:
def cost_function(x):
a, b, c = x
model_predictions = f(x_data, a, b, c)
return sum((model_predictions - y_data)**2)
You might also have to try different solvers which are built into scipy.optimize.minimize. Refer to the documentation on the pros and cons of each method.
Maybe get familiar with the examples in the Scipy documentation first, then try to set it up on your actual problem.
I am using a scipy.minimize function, where I'd like to have one parameter only searching for options with two decimals.
def cost(parameters,input,target):
from sklearn.metrics import mean_squared_error
output = self.model(parameters = parameters,input = input)
cost = mean_squared_error(target.flatten(), output.flatten())
return cost
parameters = [1, 1] # initial parameters
res = minimize(fun=cost, x0=parameters,args=(input,target)
model_parameters = res.x
Here self.model is a function that performs some matrix manipulation based on the parameters. Input and target are two matrices. The function works the way I want to, except I would like to have parameter[1] to have a constraint. Ideally I'd just like to give an numpy array, like np.arange(0,10,0.01). Is this possible?
In general this is very hard to do as smoothness is one of the core-assumptions of those optimizers.
Problems where some variables are discrete and some are not are hard and usually tackled either by mixed-integer optimization (working good for MI-linear-programming, quite okay for MI-convex-programming although there are less good solvers) or global-optimization (usually derivative-free).
Depending on your task-details, i recommend decomposing the problem:
outer-loop for np.arange(0,10,0.01)-like fixing of variable
inner-loop for optimizing, where this variable is fixed
return the model with the best objective (with status=success)
This will effect in N inner-optimizations, where N=state-space of your to fix-var.
Depending on your task/data, it might be a good idea to traverse the fixing-space monotonically (like using np's arange) and use the solution of iteration i as initial-point for the problem i+1 (potentially less iterations needed if guess is good). But this is probably not relevant here, see next part.
If you really got 2 parameters, like indicated, this decomposition leads to an inner-problem with only 1 variable. Then, don't use minimize, use minimize_scalar (faster and more robust; does not need an initial-point).
I face a problem in scipy 'leastsq' optimisation routine, if i execute the following program it says
raise errors[info][1], errors[info][0]
TypeError: Improper input parameters.
and sometimes index out of range for an array...
from scipy import *
import numpy
from scipy import optimize
from numpy import asarray
from math import *
def func(apar):
apar = numpy.asarray(apar)
x = apar[0]
y = apar[1]
eqn = abs(x-y)
return eqn
Init = numpy.asarray([20.0, 10.0])
x = optimize.leastsq(func, Init, full_output=0, col_deriv=0, factor=100, diag=None, warning=True)
print 'optimized parameters: ',x
print '******* The End ******'
I don't know what is the problem with my func optimize.leastsq() call, please help me
leastsq works with vectors so the residual function, func, needs to return a vector of length at least two. So if you replace return eqn with return [eqn, 0.], your example will work. Running it gives:
optimized parameters: (array([10., 10.]), 2)
which is one of the many correct answers for the minimum of the absolute difference.
If you want to minimize a scalar function, fmin is the way to go, optimize.fmin(func, Init).
The issue here is that these two functions, although they look the same for a scalars are aimed at different goals. leastsq finds the least squared error, generally from a set of idealized curves, and is just one way of doing a "best fit". On the other hand fmin finds the minimum value of a scalar function.
Obviously yours is a toy example, for which neither of these really makes sense, so which way you go will depend on what your final goal is.
Since you want to minimize a simple scalar function (func() returns a single value, not a list of values), scipy.optimize.leastsq() should be replaced by a call to one of the fmin functions (with the appropriate arguments):
x = optimize.fmin(func, Init)
correctly works!
In fact, leastsq() minimizes the sum of squares of a list of values. It does not appear to work on a (list containing a) single value, as in your example (even though it could, in theory).
Just looking at the least squares docs, it might be that your function func is defined incorrectly. You're assuming that you always receive an array of at least length 2, but the optimize function is insanely vague about the length of the array you will receive. You might try writing to screen whatever apar is, to see what you're actually getting.
If you're using something like ipython or the python shell, you ought to be getting stack traces that show you exactly which line the error is occurring on, so start there. If you can't figure it out from there, posting the stack trace would probably help us.