I am using GEKKO for fitting purposes trying to optimise functions which are explicitly defined - so I have a fully functional form and can create equation objects for optimisation purposes.
But now I have a different problem.
I can't create equations because of the complexed functional dependence.
But I have a python function that calculates the output using some inputs - optimisation parameters and some other that can be interpreted as fixed or known.
The key moments: I have the experimental data and a complexed model that is described in f1(set_of_parameters) - python function. f1 - is nonlinear, nonconvex and it can't be expressed as one simple equation - it has a lot of conditional parameters and a lot of branches the calls of other python functions inside, etc.
So actually f1 can't be converted to a gekko model equation.
And I need to find such parameters - set_of_optimal_parameters, which will lead to the minimum of a distance so that f1(set_of_optimal_parameters) will be as close as possible to the experimental data I have, so I will find a set_of_optimal_parameters.
For each parameter of a set, I have initial values and boundaries and even some constraints.
So I need to do something like this:
m = GEKKO()
#parameters I need to find
param_1 = m.FV(value = val_1, lb = lb_1, ub=rb_1)
param_2 = m.FV(value = val_2, lb = lb_2, ub=rb_2)
...
param_n = m.FV(value = val_n, lb = lb_n, ub=rb_n) #constructing the input for the function f1()
params_dataframe = ....()# some function that collects all the parameters and arranges all of them to a proper form to an input of f1()
#exp data description
x = m.Param(value = xData)
z = m.Param(value = yData)
y = m.Var()
#model description - is it possible to use other function inside equation? because f1 is very complexed with a lot of branches and options.. I don't really want to translate it in equation form..
m.Equation(
y==f1(params_dataframe)
)
#add some constraints
min = m.Param(value=some_val_min)
m.Equation(min <= (param_1+param_2) / (param_1+param_2)**2))
# trying to solve and minimize the sum of squares
m.Minimize(((y-z))**2)
# Options for solver
param_1.STATUS = 1
param_2.STATUS = 1
...
param_n.STATUS = 1
m.options.IMODE = 2
m.options.SOLVER = 1
m.options.MAX_ITER = 1000
m.solve(disp=1)
Is it possible to use GEKKO this way or it's not allowed? and why?
Gekko compiles equations into byte-code and requires all equations in Gekko format so that it can overload equation operators to provide exact first and second derivatives in sparse form. Black-box functions do not provide the necessary first and second derivatives, but they can provide function evaluations for finite differences (derivative approximations) or for surrogate functions.
To answer your question directly, you can't use f1(params) in a Gekko problem. If you need an optimizer to evaluate arbitrary black box functions, an optimizer such as scipy.optimize.minimize() is a good choice.
If you would still like to use Gekko, there are several options to built a surrogate model for f1 that has continuous first and second derivatives. The surrogate model depends on the number of params:
1D: use cspline()
2D: use bspline()
3D+: use Machine learning such as Gaussian Processes, Neural Network, Linear Regression, etc.
Here is an example that create a surrogate model for y=f(x) where f(x)=3*np.sin(x) - (x-3). This equation could be modeled directly in Gekko, but it serves as an example of creating a cspline() object that approximates the function and finds the minimum.
from gekko import GEKKO
import numpy as np
import matplotlib.pyplot as plt
"""
minimize y
s.t. y = f(x)
using cubic spline with random sampling of data
"""
# function to generate data for cspline
def f(x):
return 3*np.sin(x) - (x-3)
x_data = np.random.rand(50)*10+10
y_data = f(x_data)
c = GEKKO()
x = c.Var(value=np.random.rand(1)*10+10)
y = c.Var()
c.cspline(x,y,x_data,y_data,True)
c.Obj(y)
c.options.IMODE = 3
c.options.CSV_READ = 0
c.options.SOLVER = 3
c.solve(disp=True)
if c.options.SOLVESTATUS == 1:
plt.figure()
plt.scatter(x_data,y_data,5,'b')
plt.scatter(x.value,y.value,200,'r','x')
else:
print ('Failed!')
print(x_data,y_data)
plt.figure()
plt.scatter(x_data,y_data,5,'b')
plt.show()
Related
I am very new to Pyomo and I am trying to implement the example given here, i.e.,
theta' = omega,
omega' = -b * omega - c * sin(theta)
with b(t) = 0.25, for 0<=t<15, b(t) = 0.025 for t>=15,
c(t) = 5, for 0<=t<7, c(t) = 50, for t>=7,
by modifying the time-varying coefficients in the following way:
b(t) = 0.25, for 0<=t<=14,
b(t) = 0.025 for t>=15, and b(t) = m*t for 14<t<15, a straight line with a slope m joining the two constant segments. In other words, b(t) is not piecewise constant anymore as in the example, but also linear in an interval.
I looked at the varying_inputs option given here but it can only be used for the piecewise constant profiles. The page also says, "time-varying inputs can be specified using a Pyomo Suffix. We currently only support piecewise constant profiles. For more complex inputs defined by a continuous function of time, we recommend adding an algebraic variable and constraint to your model"
How can I implement such a function b(t) in my problem? That is, how to add the linear part of the new b(t) in b_profile as defined in the link above?
(With this, my final goal is to calculate the time evolution of dependent variables and later, solve a parameter estimation problem)
Any comments would be much appreciated.
Best,
I have a regression of the form model = sm.GLM(y, X, w = weight).
Which ends up being a simple weighted OLS. (note that specificying w as the error weights array actually works in sm.GLM identically to sm.WLS despite it not being in the documentation).
I'm using GLM because this allows me to fit with some additional constraints using fit_constrained(). My X consists of 6 independent variables, 2 of which i want to constrain the resulting coeffecients to be positive. But i can not seem to figure out the syntax to get fit_constrained() to work. The documentation is extremely bare and i can not find any good examples anywhere. All i really need is the correct syntax for imputing these constraints. Thanks!
The function you see is meant for linear constraints, that is a combination of your coefficients fulfill some linear equalities, not meant for defining boundaries.
The closest you can get is using scipy least squares and defining the boundaries, for example, we set up some dataset with 6 coefficients:
from scipy.optimize import least_squares
import numpy as np
np.random.seed(100)
x = np.random.uniform(0,1,(30,6))
y = np.random.normal(0,2,30)
The function to basically matrix multiply and return error:
def fun(b, x, y):
return b[0] + np.matmul(x,b[1:]) - y
The first coefficient is the intercept. Let's say we require the 2nd and 6th to be always positive:
res_lsq = least_squares(fun, [1,1,1,1,1,1,1], args=(x, y),
bounds=([-np.inf,0,-np.inf,-np.inf,-np.inf,-np.inf,0],+np.inf))
And we check the result:
res_lsq.x
array([-1.74342242e-01, 2.09521327e+00, -2.02132481e-01, 2.06247855e+00,
-3.65963504e+00, 6.52264332e-01, 5.33657765e-20])
I'm trying to use a conditional asymmetric loss function with a regression model and am having issues. I want to penalize wrong way results but the direction flips depending on the sign of the variable.
import numpy as np
def CustomLoss(predict,true):
ix = np.logical_and((predict*true)>0,np.abs(true)>=np.abs(predict))
n = ((predict - true)**2)*2
y = (predict-true)**2
out = np.where(ix,y,n)
return out
# CustomLoss(1,3) = 4
# CustomLoss(1,-1) = 8 ## Bigger loss for wrong way result
# CustomLoss(-2,-4) = 4
# CustomLoss(-2, 0) = 8 ## Bigger loss for wrong way result
I tried using scipy optimize, it converges for some data but not others. The function is still convex so I'd think this should always converge.
I've typically used CVXPY but can't figure out how to implement the conditional part of the cost function.
I currently want to implement a Hammerstein model in sympy. I have now created a small example for a simple system:
import numpy as np
from sympy import *
####HAMMERSTEIN MODEL####
#time
t = symbols("t")
#inputs
u = symbols('u')
#states
y = symbols('y',cls = Function, Function = True)
#init states
y_init =symbols('y_init')
#parameters
gain = 2 #symbols('gain')
time_constant = 20000#symbols('time_constant')
#EQUATIONS
#NONLINEAR STATIC PART
u_nonlinear = u**2 # nonlinear input
#DYNAMIC PART
# first order system with inputs
rhe = (gain * u_nonlinear - y(t)) * 1/time_constant
ode = Eq(diff(y(t),t),rhe)
#solve equation
sol_step = dsolve(ode, ics = {y(0): y_init})
sol_step = sol_step.rhs
#lambdify (sympy)
system_step =lambdify((t,u, y_init),sol_step, 'sympy')
#####SIMULATE STEPWISE######
nr_steps = 10
dt=1
u_data =IndexedBase('u_data')
y_init_data =symbols('y_init_data')
#solution vector
sol =[]
for i in range(nr_steps):
#first sim. step
if i == 0:
sol.append(system_step(dt,u_data[i],y_init_data))
#uses the states of prev. solution as inits
else:
sol.append(system_step(dt,u_data[i],sol[i-1]))
#convert
system=lambdify((u_data,y_init_data),sol, 'numpy')
#EXAMPLE
t_obs = np.linspace(0,10,10)
u_obs = np.ones(10)* 40
x_obs_init =20
#RESULT
print(system(u_obs,x_obs_init))
As you can see from the example, I solve the problem step by step. I always call the Sympy function object "system_step".
The performance is not particularly good with larger systems.
However, I would also like to use the simulation in a scipy optimizer, which leads to it being called several times, which extremely increases the solution time
My problem:
1.)
Can this step-by-step calculation also be implemented using sympy (e.g. indexed objects)? Can the repeated calculation in the loop be avoided?
2.) If so, how can this be done if the length of the input variables (u) should remain flexible and not be specified by a fixed index (m) using hardcode (see nr_steps).
Thank you very much!
Thank you for the info. If I calculate the ODE system with constant input values, I do not need to calculate it step by step. Then the solution process is very quick. Therefore, my idea was to set up the system using vectors or indexed objects, which can prevent the step-by-step calculation.
My goal:
set up the system with variable input variables
solve the system symbolically, even if it takes a very long time
Lambdify and storage in a binary file
use the solved system for different operations
I've been trying to solve this for a bit and really just haven't seen an example or anything that my brain is able to use to move forward.
The goal is to find a model Gaussian curve by minimizing the total chi-squared between the real data and the model resulting from unknown parameters that require sensible estimations (the Gaussian is of unknown position, amplitude and width). scipy.optimize.fmin has come up but I've never used this before and I'm still very new to python...
Ultimately, I'd like to plot the original data along with the model - I have use pyplot before, it's just generating the model and using fmin that has me completely bewildered where I'm essentially here:
def gaussian(a, b, c, x):
return a*np.exp(-(x-b)**2/(2*c**2))
I've seen multiple ways to generate a model and this has rendered me confused and thus I have no code! I have imported my data file through np.loadtxt.
Thanks for anyone that can suggest a framework or help at all.
There are basically four (or five) main steps involved in model fitting problems like this:
Define your forward model, yhat = F(P, x), that takes a set of parameters P and your independent variable x, and estimates your response variable y
Define your loss function, loss = L(P, x, y) that you'd like to minimize over your parameters
Optional: define a function that returns the Jacobian matrix, i.e. the partial derivatives of your loss function w.r.t. your model parameters.*
Make an initial guess at your model parameters
Plug all these into one of the optimizers and get the fitted parameters for your model
Here's a worked example to get you started:
import numpy as np
from scipy.optimize import minimize
from matplotlib import pyplot as pp
# function that defines the model we're fitting
def gaussian(P, x):
a, b, c = P
return a*np.exp(-(x-b)**2 /( 2*c**2))
# objective function to minimize
def loss(P, x, y):
yhat = gaussian(P, x)
return ((y - yhat)**2).sum()
# generate a gaussian distribution with known parameters
amp = 1.3543
pos = 64.546
var = 12.234
P_real = np.array([amp, pos, var])
# we use the vector of real parameters to generate our fake data
x = np.arange(100)
y = gaussian(P_real, x)
# add some gaussian noise to make things harder
y_noisy = y + np.random.randn(y.size)*0.5
# minimize needs an initial guess at the model parameters
P_guess = np.array([1, 50, 25])
# minimize provides a unified interface to all of scipy's solvers. you
# can also access them individually in scipy.optimize, but the
# standalone versions have annoying differences in their syntax. for now
# we'll use the Nelder-Mead solver, which doesn't use the Jacobian. we
# also need to hand it x and y_noisy as additional args to loss()
res = minimize(loss, P_guess, method='Nelder-Mead', args=(x, y_noisy))
# res is a dict containing the results of the optimization. in particular we
# want the optimized model parameters:
P_fit = res['x']
# we can pass these to gaussian() to evaluate our fitted model
y_fit = gaussian(P_fit, x)
# now let's plot the results:
fig, ax = pp.subplots(1,1)
ax.hold(True)
ax.plot(x, y, '-r', lw=2, label='Real')
ax.plot(x, y_noisy, '-k', alpha=0.5, label='Noisy')
ax.plot(x, y_fit, '--b', lw=5, label='Fit')
ax.legend(loc=0, fancybox=True)
*Some solvers, e.g. conjugate gradient methods, take the Jacobian as an additional argument, and by and large these solvers are faster and more robust, but if you're feeling lazy and performance isn't all that critical then you can usually get away without providing the Jacobian, in which case it will use the finite differences method to estimate the gradients.
You can read more about the different solvers here