Related
I'm trying to fit a curve with a Gaussian plus a Lorentzian function, using the curve_fit function from scipy.
def gaussian(x, a, x0, sig):
return a * np.exp(-1/2 * (x - x0)**2 / sig**2)
def lorentzian(x, a, b, c):
return a*c**2/((x-b)**2+c**2)
def decompose(x, z, n, b, *par):
hb_n = gaussian(x, par[0], 4861.3*(1+z), n)
hb_b = lorentzian(x, par[1], 4861.3*(1+z), b)
return hb_b + hb_n
And when I set the p0 parameter, I can get a reasonable result, which fits the curve well.
guess = [0.0001, 2, 10, 3e-16, 3e-16]
p, c = curve_fit(decompose, wave, residual, guess)
fitting parameters
the fitting model and data figure when I set the p0 parameter
But if I set the p0 and bounds parameters simultaneously, the curve_fit function gives the initial guess as the final fitting result, which is rather deviated from the data.
guess = [0.0001, 2, 10, 3e-16, 3e-16]
p, c = curve_fit(decompose, wave, residual, guess, bounds=([-0.001, 0, 0, 0, 0], [0.001, 10, 100, 1e-15, 1e-15]))
fitting parameters
the fitting model and data figure when I set the p0 and bounds parameters simultaneously
I have tried many different combinations of boundaries for the parameters, but the fitting results invariably return the initial guess values. I've been stuck in this problem for a long time. I would be very grateful if anyone can give me some advice to solve this problem.
This happens due to a combination of the optimization algorithm and its parameters.
From the official documentation:
method{‘lm’, ‘trf’, ‘dogbox’}, optional
Method to use for optimization. See least_squares for more details.
Default is ‘lm’ for unconstrained problems and ‘trf’ if bounds are
provided. The method ‘lm’ won’t work when the number of observations
is less than the number of variables, use ‘trf’ or ‘dogbox’ in this
case.
So when you add bound constraints curve_fit will use different optimization algorithm (trust region instead of Levenberg-Marquardt).
To debug the problem you can try to set full_output=True as Warren Weckesser noted in the comments.
In the case of the fit with bounds you will see something similar to:
'nfev': 1
`gtol` termination condition is satisfied.
So the optimization stopped after the first iteration. That's why founed parameters similar to the initial guess.
To fix this you can specify lower gtol parameter. Full list of available parameters you can find here: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html#scipy.optimize.least_squares
Example:
Code:
import numpy as np
from matplotlib import pyplot as plt
from scipy.optimize import curve_fit
def gaussian(x, a, x0, sig):
return a * np.exp(-1 / 2 * (x - x0) ** 2 / sig**2)
def lorentzian(x, a, b, c):
return a * c**2 / ((x - b) ** 2 + c**2)
def decompose(x, z, n, b, *par):
hb_n = gaussian(x, par[0], 4861.3 * (1 + z), n)
hb_b = lorentzian(x, par[1], 4861.3 * (1 + z), b)
return hb_b + hb_n
gt_parameters = [-2.42688295e-4, 2.3477827, 1.56977708e1, 4.47455820e-16, 2.2193466e-16]
wave = np.linspace(4750, 5000, num=400)
gt_curve = decompose(wave, *gt_parameters)
noisy_curve = gt_curve + np.random.normal(0, 2e-17, size=len(wave))
guess = [0.0001, 2, 10, 3e-16, 3e-16]
bounds = ([-0.001, 0, 0, 0, 0], [0.001, 10, 100, 1e-15, 1e-15])
options = [
("Levenberg-Marquardt without bounds", dict(method="lm")),
("Trust Region without bounds", dict(method="trf")),
("Trust Region with bounds", dict(method="trf", bounds=bounds)),
(
"Trust Region with bounds + fixed tolerance",
dict(method="trf", bounds=bounds, gtol=1e-36),
),
]
fig, axs = plt.subplots(len(options))
for (title, fit_params), ax in zip(options, axs):
ax.set_title(title)
p, c = curve_fit(decompose, wave, noisy_curve, guess, **fit_params)
fitted_curve = decompose(wave, *p)
ax.plot(wave, gt_curve, label="gt_curve")
ax.plot(wave, noisy_curve, label="noisy")
ax.plot(wave, fitted_curve, label="fitted_curve")
handles, labels = ax.get_legend_handles_labels()
fig.legend(handles, labels)
plt.show()
I have two equations that originate from one single matrix equation:
[x,y] = [[cos(n), -sin(n)],[sin(n), cos(n)]]*[x', y'],
where x' = Acos(w1*t+ p1) and y' = Bcos(w2*t + p2).
This is just a single matrix equation for the vector [x,y], but it can be decomposed into 2 scalar equations: x = A*cos(s)*cos(w1*t+ p1)*x' - B*sin(s)*sin(w2*t + p2)*y' and y = A*sin(s)*cos(w1*t+ p1)*x' + B*cos(s)*sin(w2*t + p2)*y'.
So I am fitting two datasets, x vs. t and y vs. t, but there are some shared parameters in these fits, namely A, B and s.
1) Can I fit directly the matrix equation, or do I have to decompose it into scalar ones? The former would be more elegant.
2) Can I fit with shared parameters on curve_fit? All relevant questions use other packages.
The below example code fits two different equations with one shared parameter using curve_fit. This is not an answer to your question, but I cannot format code in a comment so I'm posting it here.
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
y1 = np.array([ 16.00, 18.42, 20.84, 23.26, 25.68])
y2 = np.array([-20.00, -25.50, -31.00, -36.50, -42.00])
comboY = np.append(y1, y2)
h = np.array([5.0, 6.1, 7.2, 8.3, 9.4])
comboX = np.append(h, h)
def mod1(data, a, b, c): # not all parameters are used here
return a * data + c
def mod2(data, a, b, c): # not all parameters are used here
return b * data + c
def comboFunc(comboData, a, b, c):
# single data set passed in, extract separate data
extract1 = comboData[:len(y1)] # first data
extract2 = comboData[len(y2):] # second data
result1 = mod1(extract1, a, b, c)
result2 = mod2(extract2, a, b, c)
return np.append(result1, result2)
# some initial parameter values
initialParameters = np.array([1.0, 1.0, 1.0])
# curve fit the combined data to the combined function
fittedParameters, pcov = curve_fit(comboFunc, comboX, comboY, initialParameters)
# values for display of fitted function
a, b, c = fittedParameters
y_fit_1 = mod1(h, a, b, c) # first data set, first equation
y_fit_2 = mod2(h, a, b, c) # second data set, second equation
plt.plot(comboX, comboY, 'D') # plot the raw data
plt.plot(h, y_fit_1) # plot the equation using the fitted parameters
plt.plot(h, y_fit_2) # plot the equation using the fitted parameters
plt.show()
print(fittedParameters)
I want to approximate a function that I do not have an actual analytical expression for. I know that I want to compute this integral: integral a * b * c dx. Pretend that I get the a, b, and c are from observed data. How can I evaluate this integral? Can scipy do this? Is scipy.integrate.simps the right approach?
import numpy as np
from scipy.integrate import simps
a = np.random.random(10)
b = np.random.uniform(0, 10, 10)
c = np.random.normal(2, .8, 10)
x = np.linspace(0, 1, 10)
dx = x[1] - x[0]
print 'Is the integral of a * b * dx is ', simps(a * b, c, dx), ', ', simps(b * a, c, dx), ',', simps(a, b * c, dx), ', ', simps(a, c * b, dx), ', or something else?'
With your setup, the correct way to integrate is either
simps(a*b*c, x) # function values, argument values
or
simps(a*b*c, dx=dx) # function values, uniform spacing between x-values
Both yield the same result. Yes, simps is a very good choice for integrating sampled data. Most of the time it is more accurate than trapz.
If the data comes from a smooth function (even though you don't know the function), and you can somehow make the number of points to be 1 more than a power of 2, then Romberg integration will be even better. I compared trapz vs simps vs quad in this post.
I have a system of two first order ODEs, which are nonlinear, and hence difficult to solve analytically in a closed form. I want to fit the numerical solution to this system of ODEs to a data set. My data set is for only one of the two variables that are part of the ODE system. How do I go about this?
This didn't help because there's only one variable there.
My code which is currently leading to an error is:
import numpy as np
from scipy.integrate import odeint
from scipy.optimize import curve_fit
def f(y, t, a, b, g):
S, I = y # S, I are supposed to be my variables
Sdot = -a * S * I
Idot = (a - b) * S * I + (b - g - b * I) * I
dydt = [Sdot, Idot]
return dydt
def y(t, a, b, g, y0):
y = odeint(f, y0, t, args=(a, b, g))
return y.ravel()
I_data =[] # I have data only for I, not for S
file = open('./ratings_showdown.csv')
for e_raw in file.read().split('\r\n'):
try:
e=float(e_raw); I_data.append(e)
except ValueError:
continue
data_t = range(len(I_data))
popt, cov = curve_fit(y, data_t, I_data, [.05, 0.02, 0.01, [0.99,0.01]])
#want to fit I part of solution to data for variable I
#ERROR here, ValueError: setting an array element with a sequence
a_opt, b_opt, g_opt, y0_opt = popt
print("a = %g" % a_opt)
print("b = %g" % b_opt)
print("g = %g" % g_opt)
print("y0 = %g" % y0_opt)
import matplotlib.pyplot as plt
t = np.linspace(0, len(data_y), 2000)
plt.plot(data_t, data_y, '.',
t, y(t, a_opt, b_opt, g_opt, y0_opt), '-')
plt.gcf().set_size_inches(6, 4)
#plt.savefig('out.png', dpi=96) #to save the fit result
plt.show()
This type of ODE fitting becomes a lot easier in symfit, which I wrote specifically as a user friendly wrapper to scipy. I think it will be very useful for your situation because the decreased amount of boiler-plate code simplifies things a lot.
From the docs and applied roughly to your problem:
from symfit import variables, parameters, Fit, D, ODEModel
S, I, t = variables('S, I, t')
a, b, g = parameters('a, b, g')
model_dict = {
D(S, t): -a * S * I,
D(I, t): (a - b) * S * I + (b - g - b * I) * I
}
ode_model = ODEModel(model_dict, initial={t: 0.0, S: 0.99, I: 0.01})
fit = Fit(ode_model, t=tdata, I=I_data, S=None)
fit_result = fit.execute()
Check out the docs for more :)
So I figured out the problem.
The curve_fit() function apparently returns a list as it's second return value. So, instead of passing the initial conditions as a list [0.99,0.01], I passed them separately as 0.99 and 0.01.
I'm looking for a way to plot a curve through some experimental data. The data shows a small linear regime with a shallow gradient, followed by a steep linear regime after a threshold value.
My data is here: http://pastebin.com/H4NSbxqr
I could fit the data with two lines relatively easily, but I'd like to fit with a continuous line ideally - which should look like two lines with a smooth curve joining them around the threshold (~5000 in the data, shown above).
I attempted this using scipy.optimize curve_fit and trying a function which included the sum of a straight line and an exponential:
y = a*x + b + c*np.exp((x-d)/e)
although despite numerous attempts, it didn't find a solution.
If anyone has any suggestions please, either on the choice of fitting distribution / method or the curve_fit implementation, they would be greatly appreciated.
If you don't have a particular reason to believe that linear + exponential is the true underlying cause of your data, then I think a fit to two lines makes the most sense. You can do this by making your fitting function the maximum of two lines, for example:
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
def two_lines(x, a, b, c, d):
one = a*x + b
two = c*x + d
return np.maximum(one, two)
Then,
x, y = np.genfromtxt('tmp.txt', unpack=True, delimiter=',')
pw0 = (.02, 30, .2, -2000) # a guess for slope, intercept, slope, intercept
pw, cov = curve_fit(two_lines, x, y, pw0)
crossover = (pw[3] - pw[1]) / (pw[0] - pw[2])
plt.plot(x, y, 'o', x, two_lines(x, *pw), '-')
If you really want a continuous and differentiable solution, it occurred to me that a hyperbola has a sharp bend to it, but it has to be rotated. It was a bit difficult to implement (maybe there's an easier way), but here's a go:
def hyperbola(x, a, b, c, d, e):
""" hyperbola(x) with parameters
a/b = asymptotic slope
c = curvature at vertex
d = offset to vertex
e = vertical offset
"""
return a*np.sqrt((b*c)**2 + (x-d)**2)/b + e
def rot_hyperbola(x, a, b, c, d, e, th):
pars = a, b, c, 0, 0 # do the shifting after rotation
xd = x - d
hsin = hyperbola(xd, *pars)*np.sin(th)
xcos = xd*np.cos(th)
return e + hyperbola(xcos - hsin, *pars)*np.cos(th) + xcos - hsin
Run it as
h0 = 1.1, 1, 0, 5000, 100, .5
h, hcov = curve_fit(rot_hyperbola, x, y, h0)
plt.plot(x, y, 'o', x, two_lines(x, *pw), '-', x, rot_hyperbola(x, *h), '-')
plt.legend(['data', 'piecewise linear', 'rotated hyperbola'], loc='upper left')
plt.show()
I was also able to get the line + exponential to converge, but it looks terrible. This is because it's not a good descriptor of your data, which is linear and an exponential is very far from linear!
def line_exp(x, a, b, c, d, e):
return a*x + b + c*np.exp((x-d)/e)
e0 = .1, 20., .01, 1000., 2000.
e, ecov = curve_fit(line_exp, x, y, e0)
If you want to keep it simple, there's always a polynomial or spline (piecewise polynomials)
from scipy.interpolate import UnivariateSpline
s = UnivariateSpline(x, y, s=x.size) #larger s-value has fewer "knots"
plt.plot(x, s(x))
I researched this a little, Applied Linear Regression by Sanford, and the Correlation and Regression lecture by Steiger had some good info on it. They all however lack the right model, the piecewise function should be
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import lmfit
dfseg = pd.read_csv('segreg.csv')
def err(w):
th0 = w['th0'].value
th1 = w['th1'].value
th2 = w['th2'].value
gamma = w['gamma'].value
fit = th0 + th1*dfseg.Temp + th2*np.maximum(0,dfseg.Temp-gamma)
return fit-dfseg.C
p = lmfit.Parameters()
p.add_many(('th0', 0.), ('th1', 0.0),('th2', 0.0),('gamma', 40.))
mi = lmfit.minimize(err, p)
lmfit.printfuncs.report_fit(mi.params)
b0 = mi.params['th0']; b1=mi.params['th1'];b2=mi.params['th2']
gamma = int(mi.params['gamma'].value)
import statsmodels.formula.api as smf
reslin = smf.ols('C ~ 1 + Temp + I((Temp-%d)*(Temp>%d))' % (gamma,gamma), data=dfseg).fit()
print reslin.summary()
x0 = np.array(range(0,gamma,1))
x1 = np.array(range(0,80-gamma,1))
y0 = b0 + b1*x0
y1 = (b0 + b1 * float(gamma) + (b1 + b2)* x1)
plt.scatter(dfseg.Temp, dfseg.C)
plt.hold(True)
plt.plot(x0,y0)
plt.plot(x1+gamma,y1)
plt.show()
Result
[[Variables]]
th0: 78.6554456 +/- 3.966238 (5.04%) (init= 0)
th1: -0.15728297 +/- 0.148250 (94.26%) (init= 0)
th2: 0.72471237 +/- 0.179052 (24.71%) (init= 0)
gamma: 38.3110177 +/- 4.845767 (12.65%) (init= 40)
The data
"","Temp","C"
"1",8.5536,86.2143
"2",10.6613,72.3871
"3",12.4516,74.0968
"4",16.9032,68.2258
"5",20.5161,72.3548
"6",21.1613,76.4839
"7",24.3929,83.6429
"8",26.4839,74.1935
"9",26.5645,71.2581
"10",27.9828,78.2069
"11",32.6833,79.0667
"12",33.0806,71.0968
"13",33.7097,76.6452
"14",34.2903,74.4516
"15",36,56.9677
"16",37.4167,79.8333
"17",43.9516,79.7097
"18",45.2667,76.9667
"19",47,76
"20",47.1129,78.0323
"21",47.3833,79.8333
"22",48.0968,73.9032
"23",49.05,78.1667
"24",57.5,81.7097
"25",59.2,80.3
"26",61.3226,75
"27",61.9194,87.0323
"28",62.3833,89.8
"29",64.3667,96.4
"30",65.371,88.9677
"31",68.35,91.3333
"32",70.7581,91.8387
"33",71.129,90.9355
"34",72.2419,93.4516
"35",72.85,97.8333
"36",73.9194,92.4839
"37",74.4167,96.1333
"38",76.3871,89.8387
"39",78.0484,89.4516
Graph
I used #user423805 's answer (found via google groups thread: https://groups.google.com/forum/#!topic/lmfit-py/7I2zv2WwFLU ) but noticed it had some limitations when trying to use three or more segments.
Instead of applying np.maximum in the minimizer error function or adding (b1 + b2) in #user423805 's answer, I used the same linear spline calculation for both the minimizer and end-usage:
# least_splines_calc works like this for an example with three segments
# (four threshold params, three gamma params):
#
# for 0 < x < gamma0 : y = th0 + (th1 * x)
# for gamma0 < x < gamma1 : y = th0 + (th1 * x) + (th2 * (x - gamma0))
# for gamma1 < x : y = th0 + (th1 * x) + (th2 * (x - gamma0)) + (th3 * (x - gamma1))
#
def least_splines_calc(x, thresholds, gammas):
if(len(thresholds) < 2):
print("Error: expected at least two thresholds")
return None
applicable_gammas = filter(lambda gamma: x > gamma , gammas)
#base result
y = thresholds[0] + (thresholds[1] * x)
#additional factors calculated depending on x value
for i in range(0, len(applicable_gammas)):
y = y + ( thresholds[i + 2] * ( x - applicable_gammas[i] ) )
return y
def least_splines_calc_array(x_array, thresholds, gammas):
y_array = map(lambda x: least_splines_calc(x, thresholds, gammas), x_array)
return y_array
def err(params, x, data):
th0 = params['th0'].value
th1 = params['th1'].value
th2 = params['th2'].value
th3 = params['th3'].value
gamma1 = params['gamma1'].value
gamma2 = params['gamma2'].value
thresholds = np.array([th0, th1, th2, th3])
gammas = np.array([gamma1, gamma2])
fit = least_splines_calc_array(x, thresholds, gammas)
return np.array(fit)-np.array(data)
p = lmfit.Parameters()
p.add_many(('th0', 0.), ('th1', 0.0),('th2', 0.0),('th3', 0.0),('gamma1', 9.),('gamma2', 9.3)) #NOTE: the 9. / 9.3 were guesses specific to my data, you will need to change these
mi = lmfit.minimize(err_alt, p, args=(np.array(dfseg.Temp), np.array(dfseg.C)))
After minimization, convert the params found by the minimizer into an array of thresholds and gammas to re-use linear_splines_calc to plot the linear splines regression.
Reference: While there's various places that explain least splines (I think #user423805 used http://www.statpower.net/Content/313/Lecture%20Notes/Splines.pdf , which has the (b1 + b2) addition I disagree with in its sample code despite similar equations) , the one that made the most sense to me was this one (by Rob Schapire / Zia Khan at Princeton) : https://www.cs.princeton.edu/courses/archive/spring07/cos424/scribe_notes/0403.pdf - section 2.2 goes into linear splines. Excerpt below:
If you're looking to join what appears to be two straight lines with a hyperbola having a variable radius at/near the intersection of the two lines (which are its asymptotes), I urge you to look hard at Using an Hyperbola as a Transition Model to Fit Two-Regime Straight-Line Data, by Donald G. Watts and David W. Bacon, Technometrics, Vol. 16, No. 3 (Aug., 1974), pp. 369-373.
The formula is drop dead simple, nicely adjustable, and works like a charm. From their paper (in case you can't access it):
As a more useful alternative form we consider an hyperbola for which:
(i) the dependent variable y is a single valued function of the independent variable x,
(ii) the left asymptote has slope theta_1,
(iii) the right asymptote has slope theta_2,
(iv) the asymptotes intersect at the point (x_o, beta_o),
(v) the radius of curvature at x = x_o is proportional to a quantity delta. Such an hyperbola can be written y = beta_o + beta_1*(x - x_o) + beta_2* SQRT[(x - x_o)^2 + delta^2/4], where beta_1 = (theta_1 + theta_2)/2 and beta_2 = (theta_2 - theta_1)/2.
delta is the adjustable parameter that allows you to either closely follow the lines right to the intersection point or smoothly merge from one line to the other.
Just solve for the intersection point (x_o, beta_o), and plug into the formula above.
BTW, in general, if line 1 is y_1 = b_1 + m_1 *x and line 2 is y_2 = b_2 + m_2 * x, then they intersect at x* = (b_2 - b_1) / (m_1 - m_2) and y* = b_1 + m_1 * x*. So, to connect with the formalism above, x_o = x*, beta_o = y* and the two m_*'s are the two thetas.
There is a straightforward method (not iterative, no initial guess) pp.12-13 in https://fr.scribd.com/document/380941024/Regression-par-morceaux-Piecewise-Regression-pdf
The data comes from the scanning of the figure published by IanRoberts in his question. Scanning for the coordinates of the pixels in not accurate. So, don't be surprised by additional deviation.
Note that the abscisses and ordinates scales have been devised by 1000.
The equations of the two segments are
The approximate values of the five parameters are written on the above figure.