My goal is to optimize least squares ofth degree polynomial functios with some constraints, so my goal is to use scipy.optimize.minimize(...., method = 'SLSQP', ....). In optimalization, it is always good to pass Jacobian in the method.
I am not sure, however, how to desing my 'jac' function.
My objective function is desinged like this:
def least_squares(args_pol, x, y):
a, b, c, d, e = args_pol
return ((y-(a*x**4 + b*x**3 + c*x**2 + d*x + e))**2).sum()
where x and y are numpy arrays and contains the coordinates of points. I found in documentation, that 'jacobian' of scipy.ompitmize.minimize is gradient ob objective function and thus its array of first derivatives.
for args_pol its easy to find first derivatives, for example
db = (2*(a*x**4 + b*x**3 + c*x**2 + d*x + e - y)*x**3).sum()
but for each [x_i] in my numpy.array x is derivative
dx_i = 2*(a*x[i]**4 + b*x[i]**3 + c*x[i]**2 + d*x[i] + e - y[i])*
(4*a*x[i]**3 + 3*b*x[i]**2 + 2*c*x[i] + d)
and so on for each y_i. Thus, reasonable way is to compute each derivative as numpy.array dx and dy.
My question is - what form of result should my function for gradient return? For example should it look like
return np.array([[da, db, dc, dd, de], [dx[1], dx[2], .... dx[len(x)-1]],
[dy[1], dy[2],..........dy[len(y)-1]]])
or should it look like
return np.array([da, db, dc, dd, de, dx, dy])
Thanks for any explanations.
Related
I have data sample x,y and z. I want to 3rd order polynomial fitting of the median of X=(x,y) with z in python. But, I am not able to do it.
Here is the code I used.
from scipy.stats import binned_statistic
from scipy.optimize import curve_fit
x=data_x
y=data_y
z=data_z
X=(x,y) #X = np.column_stack((x, y))
# Define the function for linear regression
# Define the function for 2nd order polynomial fitting
def poly2_func(X, a, b, c, d, e, f):
x, y = X
return a*x**2 + b*x*y + c*y**2 + d*x + e*y + f
# Define the function for 3rd order polynomial fitting
def poly3_func(X, a, b, c, d, e, f, g, h, i, j):
x, y = X
return a*x**3 + b*x**2*y + c*x*y**2 + d*y**3 + e*x**2 + f*x*y + g*y**2 + h*x + i*y + j
# bin the data
bin_median, bin_edges, binnumber = binned_statistic(X,z,statistic='median', bins=bins_)
bin_width = (bin_edges[1] - bin_edges[0])
bin_centers = bin_edges[1:] - bin_width/2
# Fit the 2nd order polynomial model
popt_poly2, pcov_poly2 = curve_fit(poly2_func,bin_centers, bin_median)
# Fit the 3rd order polynomial model
popt_poly3, pcov_poly3 = curve_fit(poly3_func,bin_centers, bin_median)
#I am trying to see how good the data are fitted
plt.plot(bin_centers, polyfit_order3(bin_centers, *popt_3), 'g-', label='Third Order Polynomial Fit')
I got some errors. One of the main error is
ValueError: too many values to unpack (expected 2).
Kindly help me how to fit the median of this kind of multivariate data. Thank you so much.
The idea is to compute the line integral of the following vector field and curve:
This is the code I have tried:
import numpy as np
from sympy import *
from sympy import Curve, line_integrate
from sympy.abc import x, y, t
C = Curve([cos(t) + 1, sin(t) + 1, 1 - cos(t) - sin(t)], (t, 0, 2*np.pi))
line_integrate(y * exp(x) + x**2 + exp(x) + z**2 * exp(z), C, [x, y, z])
But the ValueError: Function argument should be (x(t), y(t)) but got [cos(t) + 1, sin(t) + 1, -sin(t) - cos(t) + 1] comes up.
How can I compute this line integral then?
I think that maybe this line integral contains integrals that don't have exact solution. It is also fine if you provide a numerical approximation method.
Thanks
In this case you can compute the integral using line_integrate because we can reduce the 3d integral to a 2d one. I'm sorry to say I don't know python well enough to write the code, but here's the drill:
If we write
C(t) = x(t),y(t),z(t)
then the thing to notice is that
z(t) = 3 - x(t) - y(t)
and so
dz = -dx - dy
So, we can write
F.dr = Fx*dx + Fy*dy + Fz*dz
= (Fx-Fz)*dx + (Fy-Fz)*dy
So we have reduced the problem to a 2d problem: we integrate
G = (Fx-Fz)*i + (Fx-Fz)*j
round
t -> x(t), y(t)
Note that in G we need to get rid of z by substituting
z = 3 - x - y
The value error you receive does not come from your call to the line_integrate function; it comes because according to the source code for the Curve class, only functions in 2D Euclidean space are supported. This integral can still be computed without using sympy according to this research blog that I found by simply searching for a workable method on Google.
The code you need looks like this:
import autograd.numpy as np
from autograd import jacobian
from scipy.integrate import quad
def F(X):
x, y, z = X
return [y * np.exp(x), x**2 + np.exp(x), z**2 * np.exp(z)]
def C(t):
return np.array([np.cos(t) + 1, np.sin(t) + 1, 1 - np.cos(t) - np.sin(t)])
dCdt = jacobian(C, 0)
def integrand(t):
return F(C(t)) # dCdt(t)
I, e = quad(integrand, 0, 2 * np.pi)
The variable I then stores the numerical solution to your question.
You can define a function:
import sympy as sp
from sympy import *
def linea3(f,C):
P = f[0].subs([(x,C[0]),(y,C[1]),(z,C[2])])
Q = f[1].subs([(x,C[0]),(y,C[1]),(z,C[2])])
R = f[2].subs([(x,C[0]),(y,C[1]),(z,C[2])])
dx = diff(C[0],t)
dy = diff(C[1],t)
dz = diff(C[2],t)
m = integrate(P*dx+Q*dy+R*dz,(t,C[3],C[4]))
return m
Then use the example:
f = [x**2*z**2,y**2*z**2,x*y*z]
C = [2*cos(t),2*sin(t),4,0,2*sp.pi]
I am trying to fit a quadratic plane to a cloud of data points in python. My plane function is of the form
f(x,y,z) = a*x**2 + b*y**2 + c*x*y + d*x + e*y + f - z
Currently, my data points do not have errors associated with them, however, some errors can be assumed if necessary. Following suggestions from here, I work out the vertical distance from a point p0=(x0,y0,z0) (which correspond to my data points) to a point on the plane p=(x,y,z) following this method. I then end up with
def vertical_distance(params,p0):
*** snip ***
nominator = f + a*x**2 + b*y**2 + c*x*y - x0*(2*a*x-c*y-d) - y0*(2*b*y-c*x-e) + z0
denominator = sqrt((2*a*x+c*y+d)**2 + (2*b*y+c*x+e)**2 + (-1)**2)
return nominator/denominator
Ultimately, I think it is the vertical_distance function that I need to minimise. I can happily feed a list of starting parameters (params) and the array of data points to it in two dimensions, however, I am unsure on how to achieve this in 3D. ODR pack seems to only allow data containing x,y or two dimensions. Furthermore, how do I implement the points on the plane (p) into the minimising routine? I guess that during the fit operations the points vary according to the parameter optimisation and thus the exact equation of the plane at that very moment.
I guess that «quadratic surface» would be a more correct term than «plane».
And the problem is to fit z = ax^2 + by^2 + cxy + dx + ey + f
to given set of points P.
To do that via optimization you need to formulate residual function (for instance, vertical distance).
For each 3D points p from P residual is
|p_2 – ap_0^2 + bp_1^2 + c*p_0*p_1 + dp_0 + ep_1 + f|
You need to minimize all residuals, i.e. sum of square of them, variating parameters a…f.
The following code technically should solve above problem. But fitting the problem is multi-extremal and such routine may fail to find right set of parameters without good starting point or globalization of search.
import numpy
import scipy.optimize
P = numpy.random.rand(3,10) # given point set
def quadratic(x,y, a, b, c, d, e, f):
#fit quadratic surface
return a*x**2 + b*y**2 + c*x*y + d*x + e*y + f
def residual(params, points):
#total residual
residuals = [
p[2] - quadratic(p[0], p[1],
params[0], params[1], params[2], params[3], params[4], params[5]) for p in points]
return numpy.linalg.norm(residuals)
result = scipy.optimize.minimize(residual,
(1, 1, 0, 0, 0, 0),#starting point
args=P)
I have the option to add bounds to sio.curve_fit. Is there a way to expand upon this bounds feature that involves a function of the parameters? In other words, say I have an arbitrary function with two or more unknown constants. And then let's also say that I know the sum of all of these constants is less than 10. Is there a way I can implement this last constraint?
import numpy as np
import scipy.optimize as sio
def f(x, a, b, c):
return a*x**2 + b*x + c
x = np.linspace(0, 100, 101)
y = 2*x**2 + 3*x + 4
popt, pcov = sio.curve_fit(f, x, y, \
bounds = [(0, 0, 0), (10 - b - c, 10 - a - c, 10 - a - b)]) # a + b + c < 10
Now, this would obviously error, but I think it helps to get the point across. Is there a way I can incorporate a constraint function involving the parameters to a curve fit?
Thanks!
With lmfit, you would define 4 parameters (a, b, c, and delta). a and b can vary freely. delta is allowed to vary, but has a maximum value of 10 to represent the inequality. c would be constrained to be delta-a-b (so, there are still 3 variables: c will vary, but not independently from the others). If desired, you could also put bounds on the values for a, b, and c. Without testing, your code would be approximately::
import numpy as np
from lmfit import Model, Parameters
def f(x, a, b, c):
return a*x**2 + b*x + c
x = np.linspace(0, 100.0, 101)
y = 2*x**2 + 3*x + 4.0
fmodel = Model(f)
params = Parameters()
params.add('a', value=1, vary=True)
params.add('b', value=4, vary=True)
params.add('delta', value=5, vary=True, max=10)
params.add('c', expr = 'delta - a - b')
result = fmodel.fit(y, params, x=x)
print(result.fit_report())
Note that if you actually get to a situation where the constraint expression or bounds dictate the values for the parameters, uncertainties may not be estimated.
curve_fit and least_squares only accept box constraints. In scipy.optimize, SLSQP can deal with more complicated constraints.
For curve fitting specifically, you can have a look at lmfit package.
it is commonly an easy task to build an n-th order polynomial
and find the roots with numpy:
import numpy
f = numpy.poly1d([1,2,3])
print numpy.roots(f)
array([-1.+1.41421356j, -1.-1.41421356j])
However, suppose you want a polynomial of type:
f(x) = a*(x-x0)**0 + b(x-x0)**1 + ... + n(x-x0)**n
Is there a simple way to construct a numpy.poly1d type function
and find the roots ? I've tried scipy.fsolve but it is very unstable as it depends highly on the choice of the starting values
in my particular case.
Thanks in advance
Best Regards
rrrak
EDIT: Changed "polygon"(wrong) to "polynomial"(correct)
First of all, surely you mean polynomial, not polygon?
In terms of providing an answer, are you using the same value of "x0" in all the terms? If so, let y = x - x0, solve for y and get x using x = y + x0.
You could even wrap it in a lambda function if you want. Say, you want to represent
f(x) = 1 + 3(x-1) + (x-1)**2
Then,
>>> g = numpy.poly1d([1,3,1])
>>> f = lambda x:g(x-1)
>>> f(0.0)
-1.0
The roots of f are given by:
f.roots = numpy.roots(g) + 1
In case x0 are different by power, such as:
f(x) = 3*(x-0)**0 + 2*(x-2)**1 + 3*(x-1)**2 + 2*(x-2)**3
You can use polynomial operation to calculate the finally expanded polynomial:
import numpy as np
import operator
ks = [3,2,3,2]
offsets = [0,2,1,2]
p = reduce(operator.add, [np.poly1d([1, -x0])**i * c for i, (c, x0) in enumerate(zip(ks, offsets))])
print p
The result is:
3 2
2 x - 9 x + 20 x - 14