In my ODE function I need to iteratively solve an equation for a parameter until convergence at each time step. I'd like to pass the latest parameter value to be used as the initial value for the next time step so when the function does the iterative update of the parameter it will take less time. But I can't figure out how to do that. The code structure of the ODE function is like this:
from scipy.integrate import solve_ivp
def run(t, y):
if t==0:
a = 1e-8
nn = 0
while nn<=100:
nn = nn +1
#update a until convergence
return a*y
In some one language I can return the updated parameter to be used by the integrator, but I don't' see how that's possible with solve_ivp
It's not clear what you're after: do you want to obtain a solution for an ODE at a series of parameter values (i.e. for each value of the parameter you solve the full ODE) or you are changing the parameter along with the ODE iterations (IOW, you want inner or outer iterations).
If the former, then just do a for loop over the parameters. If the latter, it's likely easier and cleaner to use solver classes which implement specific solvers (DOPRI, Radau, RK, BDF etc), which solve_ivp delegates the work to. They offer a step method, which performs a single step. So that you can adjust you parameters, control convergence etc on a way that's most relevant to this particular case.
I think what you are looking for is something in the following form:
class test:
a = 1e-8
def f(self, t, y):
## do iter on self.a
return self.a*y
t = test()
# solve_ivp(t.f, .....)
This way you can always use the last value of a, since it is part of your instance of the test class. This is not exactly what you are asking for, since this will call the iteration each time solve_ivp evaluates f, which will be multiple times per timestep. However, I think this is the closest you can get, since solve_ivp does not appear to have a callback function to invoke after each timestep
Related
I am having a problem with a function I am trying to fit to some data. I have a model, given by the equation inside the function which I am using to find a value for v. However, the order in which I write the variables in the function definition greatly effects the value the fit gives for v. If, as in the code block below, I have def MAR_fit(v,x) where x is the independent variable, the fit gives a value for v hugely different from if I have the definition def MAR_fit(x,v). I haven't had a huge amount of experience with the curve_fit function in the scipy package and the docs still left me wondering.
Any help would be great!
def MAR_fit(v,x):
return (3.*((2.-1.)**2.)*0.05*v)/(2.*(2.-1.)*(60.415**2.)) * (((3.*x*((2.-1.)**2.)*v)/(60.415**2.))+1.)**(-((5./2.)-1.)/(2.-1.))
x = newCD10_AVB1_AMIN01['time_phys'][1:]
y = (newCD10_AVB1_AMIN01['MAR'][1:])
popt_tf, pcov = curve_fit(MAR_fit, x, y)
Have a look at the documentation again, it says that the callable that you pass to curve_fit (the function you are trying to fit) must take the independent variable as its first argument. Further arguments are the parameters you are trying to fit. You must use MAR_fit(x,v) because that is what curve_fit expects.
I'm having trouble specifying constraints using basinhopping with method='COBYLA'. Here is a test case where things go wrong. Essentially, the constraints are ignored and there are function trials outside the specified range. I specify a simple quadratic with minimum at [0,0], searching for -3<x[0], but as you can see from the output, there are lots of searches outside that range (I increased the stepsize to make it obvious)
import numpy as np
from scipy.optimize import basinhopping
def f(x):
if x[0]<-3 :
print('outside range ',x[0])
return x[0]**2+x[1]**2
cons = [{'type':'ineq','fun': lambda x: x[0]+3}]
kwargs = {'method':'COBYLA','constraints':cons}
ret=basinhopping(f, [5,1],T=1,stepsize=1000,niter=1,minimizer_kwargs=kwargs)
print(ret)
runfile('py/cobyla_test', wdir='/py', post_mortem=True)
outside range -446.14581341127945
outside range -445.14581341127945
outside range -445.14581341127945
outside range -444.14581341127945
[etc... lots of output deleted]
[-4.81217825e-05 -5.23242054e-05] 5.0535284302996725e-09
As written at scipy.optimize.basinhopping — SciPy v1.1.0 Reference Guide, Basin-hopping is a two-step method:
first, a random jump is done (take_step callback)
then a local minimum is found from that point using the specificed minimization method
finally, it's decided if the step is accepted (accept_test callback)
The constraints you've specified are for the minimization method, they don't affect the jump step. For the jump step, either adjust stepsize (the maximum displacement for the random jump), or define your own take_step.
"I thought the point of the constraint is that it would never try an x outside the constraint" -- constraints in mathematical problems, including a constrained optimization problem, don't work that way. They only specify what conditions the solution itself must satisfy. They don't limit what points can be used while obtaining that solution, it's completely up to the algorithm to choose these.
The approach to limit the area in which a numerical method searches is to tweak method parameters in some way specific to the nature of the function and the method, to "guide" the method into the right direction.
I was reading this question and I was trying to do the same, but I want the function to have a single parameter say x. And that parameter is an array of "values" to be filled by an optimization solver. For instance:
def f(x):
return x[0]**2 + 3*x[1]
That function will refer to: f(x)=x^2 + 3y, meaning x is an array of variables. Those variables will be present on the current function or not, because they are all the variables in the whole optimization problem, meaning they can be present on the constraints. So I will like to find that functions partial derivatives of all variables. So,in this case, i will need 2 callable functions so I can use it to form a new array that is the Jacobian of the function. Is there a way to do that? How?
Disclaimer: I am the author of pyneqsys.
If you are open to using a library, pyneqsys does exactly this. If not, you can look at the source of pyneqsys/symbolic.py which (approximately) does this to calculate the jacobian:
f = sympy.Matrix(self.nf, 1, self.exprs)
x = sympy.Matrix(self.nx, 1, self.x)
J = f.jacobian(x)
You then need to use sympy.lambdify to obtain a callable with the expected syntax of your particular solver.
I have two loss functions here to be minimized:
The first one is a local one, where:
min f1(x1),
min f2(x2),
min f3(x3),....
min fn(xn)
The other one is global one, where:
min f(x1,x2,...,xn) = f1(x1)+f2(x2)+...fn(xn)
For each local problem fi(x), I have 2 variables to be optimized, and I have 1000 local problems. Correspondingly, for the global problem, I have 2000 variables to be optimized. Surely the 2nd one has more parameters to be optimized, but since f1, f2, f3...fn are independent with each other, I hope they two should be comparable.
I use the scipy minimize function for optimization (scipy.optimize.minimize). But the 2nd one much much slower than the 1st one.
The only drawback of the global one, i think, is taking more gradients than it actually need to. For example, the gradient of x1 only comes from f1, but the global computes its gradient from f2, f3... fn, which is 0. Thus, making it slower. If that is the case, I do hope there would be some ways for acceleration.
BTW, since I later on need to add a global constraint to the optimization, this is why I must use the global loss function instead of the local one.
I think your guess is correct that the amount of time that it takes more is because it needs to compute the gradients. Based on the description page for scipy.optimize.minimize (https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html), it seems that the method computes the gradient numerically if you provide jac = False (optional and set to False by default).
jac : bool or callable, optional
Jacobian (gradient) of objective function. Only for CG, BFGS, Newton-CG, L-BFGS- B, TNC, SLSQP, dogleg, trust-ncg, trust-krylov, trust-region-exact. If jac is a Boolean and is True, fun is assumed to return the gradient along with the objective function. If False, the gradient will be estimated numerically. jac can also be a callable returning the gradient of the objective. In this case, it must accept the same arguments as fun.
Based on the above, you can set jac = True and then you should provide your function as a callable that returns function value as well as the gradients. This should speed up the process.
One other way is to write your own customizable minimizer as callable.
I am using a scipy.minimize function, where I'd like to have one parameter only searching for options with two decimals.
def cost(parameters,input,target):
from sklearn.metrics import mean_squared_error
output = self.model(parameters = parameters,input = input)
cost = mean_squared_error(target.flatten(), output.flatten())
return cost
parameters = [1, 1] # initial parameters
res = minimize(fun=cost, x0=parameters,args=(input,target)
model_parameters = res.x
Here self.model is a function that performs some matrix manipulation based on the parameters. Input and target are two matrices. The function works the way I want to, except I would like to have parameter[1] to have a constraint. Ideally I'd just like to give an numpy array, like np.arange(0,10,0.01). Is this possible?
In general this is very hard to do as smoothness is one of the core-assumptions of those optimizers.
Problems where some variables are discrete and some are not are hard and usually tackled either by mixed-integer optimization (working good for MI-linear-programming, quite okay for MI-convex-programming although there are less good solvers) or global-optimization (usually derivative-free).
Depending on your task-details, i recommend decomposing the problem:
outer-loop for np.arange(0,10,0.01)-like fixing of variable
inner-loop for optimizing, where this variable is fixed
return the model with the best objective (with status=success)
This will effect in N inner-optimizations, where N=state-space of your to fix-var.
Depending on your task/data, it might be a good idea to traverse the fixing-space monotonically (like using np's arange) and use the solution of iteration i as initial-point for the problem i+1 (potentially less iterations needed if guess is good). But this is probably not relevant here, see next part.
If you really got 2 parameters, like indicated, this decomposition leads to an inner-problem with only 1 variable. Then, don't use minimize, use minimize_scalar (faster and more robust; does not need an initial-point).