I am currently using my beginner-level knowledge of python for some econometric problems I am facing. Until now, this worked perfectly fine. However, my current problem is finding a graph + function for a few interview answers, for example for the following 6 points:
xvalues = [0, 0.2, 0.4, 0.6, 0.8, 1]
yvalues = [0, 0.15, 0.6, 0.49, 0.51, 1]
I've used curve_fit with mixed results. I have no problem with sigmoid and logarithmic functions. But when it comes to polynomial functions, I need to limit the possible y-values the function can have. For 0 <= x <= 1 the following conditions have to apply (I don't care about x < 0 and x > 1):
0 <= y <= 1
Maxima and minima of the function have to be located at said points. This doesn't apply to inflection points, though. Edit for clarity: Maxima and minima have to located only at said points.
as a basis, let's take the following, very simple code that works:
from scipy.optimize import curve_fit
def poly6(x, a, b, c, d, e, f):
return f * (x ** 6) + e * (x ** 5) + d * (x ** 4) + c * (x ** 3) + b * (x ** 2) + a * (x ** 1)
xvalues = [0, 0.2, 0.4, 0.6, 0.8, 1]
yvalues = [0, 0.15, 0.6, 0.49, 0.51, 1]
x = xvalues
y = yvalues
x_line = arange(min(x), max(x), 1)
popt, _ = curve_fit(poly6, x, y)
a, b, c, d, e, f = popt
print("Poly 6:")
print(popt)
How can I efficiently write these conditions down?
I've tried to find an answer, but with underwhelming success. I found it hard to narrow my problem down to an oneliner that other people already asked.
Using scipy.optimize.minimize to provide provide bounds of the possible y values of your function. I only implemented the limits of y being between 0 and 1. I didn't fully understand what you meant by the maxima/minima of the function having to be in the interval 0 <= x <= 1. Or do you mean minimum has to be at x=0 and maximum at x=1? If that's the case, then it's fairly easy to add two new weights for those situations.
from scipy.optimize import minimize, curve_fit
import numpy as np
import matplotlib.pyplot as plt
xvalues = np.array([0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1])
yvalues = np.array([0, 0.075, 0.15, 0.375, 0.6, 0.545, 0.49, 0.5, 0.51, 0.755, 1])
def poly6(x, a, b, c, d, e, f):
return f * (x ** 6) + e * (x ** 5) + d * (x ** 4) + c * (x ** 3) + b * (x ** 2) + a * (x ** 1)
def min_function(params, x, y):
model = poly6(x, *params)
residual = ((y - model) ** 2).sum()
if np.any(model > 1):
residual += 100 # Just some large value
if np.any(model < 0):
residual += 100
return residual
res = minimize(min_function, x0=(1, 1, 1, 1, 1, 1), args=(xvalues, yvalues))
plt.plot(xvalues, yvalues, label='data')
plt.plot(xvalues, poly6(xvalues, *res.x), label='model')
plt.legend()
This is the resulting fit:
First, at least your test dataset is very small compared to the amount of variables you are trying to fit.
The constraints for 0<x<1 are:
local minima/ maxima at given points y'(x)=0, y''(x)!=0
0<= y(x)<= 1
The first constraint cannot be fulfilled by an approximation approach like curve-fit since you will not end exactly on the given points. Thus you will have to use some interpolation function like scipy.interpolate.UnivaranteSpline by definition a spline is continous differentiable and touches all points.
If you are looking for an approximate solution you have to define
how important is each constraint
how to measure the deviation of the expected result
and define yourself some kind of loss-function, punishing bad results e.g. if y >1 return 100000. Here you should use scipy.minimize since it provides the possibility of custom residuum-functions.
Misread the OPs Question
Second, have a look into the reference of the module/ function you are useing. Here you see:
bounds2-tuple of array_like, optional
Lower and upper bounds on
parameters. Defaults to no bounds. Each element of the tuple must be
either an array with the length equal to the number of parameters, or
a scalar (in which case the bound is taken to be the same for all
parameters). Use np.inf with an appropriate sign to disable bounds on
all or some parameters.
Thus, the following change will set a limit for all parameters to [0,1]
bound=(0,1)
popt, _ = curve_fit(poly6, x, y, bounds=bound)
Related
I'm trying to fit a curve with a Gaussian plus a Lorentzian function, using the curve_fit function from scipy.
def gaussian(x, a, x0, sig):
return a * np.exp(-1/2 * (x - x0)**2 / sig**2)
def lorentzian(x, a, b, c):
return a*c**2/((x-b)**2+c**2)
def decompose(x, z, n, b, *par):
hb_n = gaussian(x, par[0], 4861.3*(1+z), n)
hb_b = lorentzian(x, par[1], 4861.3*(1+z), b)
return hb_b + hb_n
And when I set the p0 parameter, I can get a reasonable result, which fits the curve well.
guess = [0.0001, 2, 10, 3e-16, 3e-16]
p, c = curve_fit(decompose, wave, residual, guess)
fitting parameters
the fitting model and data figure when I set the p0 parameter
But if I set the p0 and bounds parameters simultaneously, the curve_fit function gives the initial guess as the final fitting result, which is rather deviated from the data.
guess = [0.0001, 2, 10, 3e-16, 3e-16]
p, c = curve_fit(decompose, wave, residual, guess, bounds=([-0.001, 0, 0, 0, 0], [0.001, 10, 100, 1e-15, 1e-15]))
fitting parameters
the fitting model and data figure when I set the p0 and bounds parameters simultaneously
I have tried many different combinations of boundaries for the parameters, but the fitting results invariably return the initial guess values. I've been stuck in this problem for a long time. I would be very grateful if anyone can give me some advice to solve this problem.
This happens due to a combination of the optimization algorithm and its parameters.
From the official documentation:
method{‘lm’, ‘trf’, ‘dogbox’}, optional
Method to use for optimization. See least_squares for more details.
Default is ‘lm’ for unconstrained problems and ‘trf’ if bounds are
provided. The method ‘lm’ won’t work when the number of observations
is less than the number of variables, use ‘trf’ or ‘dogbox’ in this
case.
So when you add bound constraints curve_fit will use different optimization algorithm (trust region instead of Levenberg-Marquardt).
To debug the problem you can try to set full_output=True as Warren Weckesser noted in the comments.
In the case of the fit with bounds you will see something similar to:
'nfev': 1
`gtol` termination condition is satisfied.
So the optimization stopped after the first iteration. That's why founed parameters similar to the initial guess.
To fix this you can specify lower gtol parameter. Full list of available parameters you can find here: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html#scipy.optimize.least_squares
Example:
Code:
import numpy as np
from matplotlib import pyplot as plt
from scipy.optimize import curve_fit
def gaussian(x, a, x0, sig):
return a * np.exp(-1 / 2 * (x - x0) ** 2 / sig**2)
def lorentzian(x, a, b, c):
return a * c**2 / ((x - b) ** 2 + c**2)
def decompose(x, z, n, b, *par):
hb_n = gaussian(x, par[0], 4861.3 * (1 + z), n)
hb_b = lorentzian(x, par[1], 4861.3 * (1 + z), b)
return hb_b + hb_n
gt_parameters = [-2.42688295e-4, 2.3477827, 1.56977708e1, 4.47455820e-16, 2.2193466e-16]
wave = np.linspace(4750, 5000, num=400)
gt_curve = decompose(wave, *gt_parameters)
noisy_curve = gt_curve + np.random.normal(0, 2e-17, size=len(wave))
guess = [0.0001, 2, 10, 3e-16, 3e-16]
bounds = ([-0.001, 0, 0, 0, 0], [0.001, 10, 100, 1e-15, 1e-15])
options = [
("Levenberg-Marquardt without bounds", dict(method="lm")),
("Trust Region without bounds", dict(method="trf")),
("Trust Region with bounds", dict(method="trf", bounds=bounds)),
(
"Trust Region with bounds + fixed tolerance",
dict(method="trf", bounds=bounds, gtol=1e-36),
),
]
fig, axs = plt.subplots(len(options))
for (title, fit_params), ax in zip(options, axs):
ax.set_title(title)
p, c = curve_fit(decompose, wave, noisy_curve, guess, **fit_params)
fitted_curve = decompose(wave, *p)
ax.plot(wave, gt_curve, label="gt_curve")
ax.plot(wave, noisy_curve, label="noisy")
ax.plot(wave, fitted_curve, label="fitted_curve")
handles, labels = ax.get_legend_handles_labels()
fig.legend(handles, labels)
plt.show()
I'm trying to fit the following data
tau = [0.0001, 0.0004, 0.0006, 0.0008, 0.001, 0.0015, 0.002, 0.004, 0.006, 0.008, 0.01, 0.05, 0.1, 0.2, 0.5, 0.6, 0.8, 1.0, 1.5, 2.0, 4.0, 6.0, 8.0, 10.0]
tet = [1.000000000, 0.993790739, 0.965602604, 0.924802378, 0.88010508, 0.778684048, 0.702773729, 0.569882533, 0.544103907, 0.54709633, 0.547347558, 0.543859156, 0.504348651, 0.691909732, 0.351717086, 0.405861814, 0.340536768, 0.301032851, 0.192656835, 0.188915355, 0.100207658, 0.059809495, 0.035968302, 0.024147687]
using a summation with the general formula
f(x) = $\sum_{i=1}^{n} a_i* exp^{-x/ti}$
I'm doing it separately, I'm sure I can do it using a for a function or something like that but I do not know how to do it. So here it goes
def fitfunc_1(x, a, t1):
return a * np.exp(- x / t1)
popt_tet_1, pcov = curve_fit(fitfunc_1, data['tau'], data['tet'], maxfev=10000, bounds = (0.0, np.inf))
def fitfunc_2(x, a, t1, b, t2):
return a * np.exp(- x / t1) + b * np.exp(- x / t2)
popt_tet_2, pcov = curve_fit(fitfunc_2, data['tau'], data['tet'], maxfev=10000, bounds = (0.0, np.inf))
def fitfunc_3(x, a, t1, b, t2, c, t3):
return a * np.exp(- x / t1) + b * np.exp(- x / t2) + c * np.exp(- x / t3)
popt_tet_3, pcov = curve_fit(fitfunc_3, data['tau'], data['tet'], maxfev=10000, bounds = (0.0, np.inf))
However, I need to make sure that the sum of the a_i indexes, a, b and c are around 1. Meaning a ~ 1, a + b ~ 1, a + b + c ~ 1
Is there a way to limit scipy's fitting function this way?
Sorry for the noob question I guess
I tried to fit to your data to a sum of two exponentials and also to a sum of three exponentials. In both cases the fitting is correct only on a part of the range but never on the whole range. The difficulty can be understood in plotting the experimental points with a logarithmic scale on the abscissa axis.
The shape of the pattern looks more like the sum of fuctions of logistic kind than the sum of functions of exponential kind.
This suggests that each term of the sum might be on this form :
Thus the whole function to be fitted is :
NOTE : The above is a preliminary study in order to find a convenient kind of function to be fitted. The above numerical values of parameters are only empirically approximated. In order to have a better fit one have still to compute the parameters thanks to non-linear regression in using iterative calculus. The initial values to start the iterative process can be the above values of parameters.
Current I'm attempting to use scipy's least squares, or any of their minimization functions to minimize a function with 5 parameters.
What I would like scipy to do is minimize some function using a standard least squares.
My code is below:
fitfunc1 = lambda p, xx, yy, zz: -(50000*(xx + (p[0] + p[1])*yy +
p[3]))/(1.67*(-p[2]*yy + zz + p[4]))
errfunc1 = lambda p,x11, xx, yy, zz: fitfunc1(p, xx, yy, zz) - x11
x0 = np.array([0.1, 0.1, 0.1, 0.1, 0.1, 0.1], dtype = float)
res3 = leastsq(errfunc1, x0[:], args=(x1, x, y, z))
where x1, x, y, z are all column numpy arrays of the same length about 90x1
I'm currently getting an error that says ' Error: Result from function call is not a proper array of floats', I've attempted many possibilities, and tried to rewrite this the way it is described in examples, but doesn't seem to work.
In addition: I actually would like to solve the problem:
min sum (f - x1)**2 + (g - x2)**2
where f = f(p, x, y, z) and g = g(p, x, y, z) and x, y, z, x1, y1 are all data, but attempting to find the parameters, p (6 of them).
Is this currently possible in least squares? I have attempted using scipy.minimize, but when this is done using the Nedler's Mead method, it doesn't seem to work either.
Here is my current code:
def f(phi, psi, theta, xnot, ynot, znot):
return sum(abs( (-50000*(x[:]+ (psi + phi)*y[:] + xnot)/(1.67*(-
theta*y[:] + z[:] + znot))) - x1[:]) //
+ abs( (-50000*(-x[:]*(psi + phi) + y[:] + theta*(z[:]) + ynot)/(1.67*(-
theta*y[:] + z[:] + znot))) - y1[:]))
x0 = np.array([0.1, 0.1, 0.1, 0.1, 0.1, 0.1], dtype = float)
res3 = leastsq(f, x0[:], args=(x1, y1, x, y, z))
I feel as if I am making some mistake that may be obvious to someone more familiar, but this is my first time using scipy. All help would be much appreciated.
I believe your problem is with the shape of the variables:
where x1, x, y, z are all column numpy arrays of the same length about 90x1
It causes your fitfunc1 and errfunc1 functions to return 2d arrays (of shape (90,1)), where the scipy optimization function expects a 1d array.
Try reshaping your arrays, e.g.,
x1 = x1.reshape((90,)),
and similarly for the rest of your input variables.
This should fix your problem.
I am solving a system of transcendental equations:
cos(x) / x = 0.48283 + a*3.46891
cos(y) / y = 0.47814 + b*28.6418
a + b = 1
1.02 * sinc(x) = 1.03 * sinc(y)
And it just so happens I tried to solve the above system in two separated programming languages (Mathematica and Python)
Mathematica
Running the code
FindRoot[{Cos[x]/x == 0.482828 + a*3.46891,
Cos[y]/y == 0.47814 + b*28.6418, a + b == 1,
1.02*Sinc[x] == 1.03*Sinc[y]}, {{x, .2}, {y, .2}, {a, 0.3}, {b,
0.3}}, PrecisionGoal -> 6]
returns
{x -> 0.261727, y -> 0.355888, a -> 0.924737, b -> 0.0752628}
Python
Running the code:
import numpy as np
from scipy.optimize import root
def fuuu(X, en,dv,tri,sti):
x, y, a, b = X
F = [np.cos(x) / x - en-a*dv,
np.cos(y) / y - tri-b*sti,
a + b - 1,
1.02 * np.sinc(x) - 1.03 * np.sinc(y)]
return F
root(fuuu, [0.2, 0.2, 0.3, 0.3], args=(0.482828,3.46891,0.47814,28.6418)).x
returns
array([ 0.26843418, 0.27872813, 0.89626625, 0.10373375])
Comparison
Let's say that the 'x' value is the same. Let's just ignore the small difference. But the y values differ by miles! The physical meaning completely changes. For some reason I believe the values from Mathematica more than I believe values from Python.
Questions:
Why do the calculations differ?
Which one is now correct? What do I have to change in python (assuming python is the problematic one)?
The calculation differ because of the sinc function.
(* Mathematica *)
In[1] := Sinc[0.26843418]
Out[1] = 0.988034
# Python
>>> np.sinc(0.26843418)
0.88561519683835599
>>> np.sin(0.26843418) / 0.26843418
0.98803370932709034
Huh? Well let's RTFM
numpy.sinc(x)
Return the sinc function.
The sinc function is sin(πx)/(πx).
Oops. NumPy's sinc has a different definition than Mathematica's Sinc.
Mathematica's Sinc uses the unnormalized definition sin(x)/x. This definition is usually used in mathematics and physics.
NumPy's sinc uses the normalized version sin(πx)/(πx). This definition is usually used in digital signal processing and information theory. It is called normalized because
∫-∞∞ sin(πx)/(πx) dx = 1.
Therefore, if you want NumPy to produce the same result as Mathematica, you need to divide x and y by np.pi.
def fuuu(X, en,dv,tri,sti):
x, y, a, b = X
F = [np.cos(x) / x - en-a*dv,
np.cos(y) / y - tri-b*sti,
a + b - 1,
1.02 * np.sinc(x/np.pi) - 1.03 * np.sinc(y/np.pi)] # <---
return F
>>> root(fuuu, [0.2, 0.2, 0.3, 0.3], args=(0.482828,3.46891,0.47814,28.6418)).x
array([ 0.26172691, 0.3558877 , 0.92473722, 0.07526278])
I am trying to fit a curve to X and Y data points using a rational function. It can be done in Matlab using the cftool (http://de.mathworks.com/help/curvefit/rational.html). However, I am looking to do the same in Python. I have tried to use scipy.optimize.curve_fit(), but it initially requires a function, which I don't have.
You have the function, it is the rational function. So you need to set up the function and perform the fitting. As curve_fit requires that you supply your arguments not as lists, I supplied an additional function which does the fitting on the specific case of third degree polynomial in both the numerator as well as the denominator.
def rational(x, p, q):
"""
The general rational function description.
p is a list with the polynomial coefficients in the numerator
q is a list with the polynomial coefficients (except the first one)
in the denominator
The zeroth order coefficient of the denominator polynomial is fixed at 1.
Numpy stores coefficients in [x**2 + x + 1] order, so the fixed
zeroth order denominator coefficent must comes last. (Edited.)
"""
return np.polyval(p, x) / np.polyval(q + [1.0], x)
def rational3_3(x, p0, p1, p2, q1, q2):
return rational(x, [p0, p1, p2], [q1, q2])
x = np.linspace(0, 10, 100)
y = rational(x, [-0.2, 0.3, 0.5], [-1.0, 2.0])
ynoise = y * (1.0 + np.random.normal(scale=0.1, size=x.shape))
popt, pcov = curve_fit(rational3_3, x, ynoise, p0=(0.2, 0.3, 0.5, -1.0, 2.0))
print popt
plt.plot(x, y, label='original')
plt.plot(x, ynoise, '.', label='data')
plt.plot(x, rational3_3(x, *popt), label='fit')