I'm currently trying to use SciPy's integrate.ode package to solve a pair of first-order ODEs that are coupled: say, the Lotka-Volterra predator-prey equation. However, this means during the integration loop I have to update the parameters I'm sending to the methods on every iteration, and simply keeping track of the previous value and calling set_f_params() on each iteration doesn't seem to be doing the trick.
hprev = Ho
pprev = Po
yh = np.zeros(0)
yp = np.zeros(0)
while dh.successful() and dp.successful() and dp.t < endtime and dh.t < endtime:
hparams = [alpha, beta, pprev]
pparams = [delta, gamma, hprev]
dh.set_f_params(hparams)
dp.set_f_params(pparams)
dh.integrate(dh.t + stepsize)
dp.integrate(dp.t + stepsize)
yh = np.append(yh, dh.y)
yp = np.append(yp, dp.y)
hprev = dh.y
pprev = dp.y
The values I'm setting at each iteration through set_f_params don't seem to be propagated to the callback methods, which wasn't terribly surprising given none of the examples on the web seem to involve "live" variable passing to the callbacks, but this was the only method by which I could think to get these values into the callback methods.
Does anyone have any advice on how to use SciPy to numerically integrate these ODEs?
I could be wrong, but this example seems very close to your problem. :) It uses odeint to solve the system of ODEs.
I had a similar issue. Turns out, the integrator doesn't re-evaluate the differential equation function for every call of integrate(), but does it at its own internal times. I changed max_step option of the integrator to be the same as stepsize and that worked for me.
Related
I'm facing a problem while trying to implement the coupled differential equation below (also known as single-mode coupling equation) in Python 3.8.3. As for the solver, I am using Scipy's function scipy.integrate.solve_bvp, whose documentation can be read here. I want to solve the equations in the complex domain, for different values of the propagation axis (z) and different values of beta (beta_analysis).
The problem is that it is extremely slow (not manageable) compared with an equivalent implementation in Matlab using the functions bvp4c, bvpinit and bvpset. Evaluating the first few iterations of both executions, they return the same result, except for the resulting mesh which is a lot greater in the case of Scipy. The mesh sometimes even saturates to the maximum value.
The equation to be solved is shown here below, along with the boundary conditions function.
import h5py
import numpy as np
from scipy import integrate
def coupling_equation(z_mesh, a):
ka_z = k # Global
z_a = z # Global
a_p = np.empty_like(a).astype(complex)
for idx, z_i in enumerate(z_mesh):
beta_zf_i = np.interp(z_i, z_a, beta_zf) # Get beta at the desired point of the mesh
ka_z_i = np.interp(z_i, z_a, ka_z) # Get ka at the desired point of the mesh
coupling_matrix = np.empty((2, 2), complex)
coupling_matrix[0] = [-1j * beta_zf_i, ka_z_i]
coupling_matrix[1] = [ka_z_i, 1j * beta_zf_i]
a_p[:, idx] = np.matmul(coupling_matrix, a[:, idx]) # Solve the coupling matrix
return a_p
def boundary_conditions(a_a, a_b):
return np.hstack(((a_a[0]-1), a_b[1]))
Moreover, I couldn't find a way to pass k, z and beta_zf as arguments of the function coupling_equation, given that the fun argument of the solve_bpv function must be a callable with the parameters (x, y). My approach is to define some global variables, but I would appreciate any help on this too if there is a better solution.
The analysis function which I am trying to code is:
def analysis(k, z, beta_analysis, max_mesh):
s11_analysis = np.empty_like(beta_analysis, dtype=complex)
s21_analysis = np.empty_like(beta_analysis, dtype=complex)
initial_mesh = np.linspace(z[0], z[-1], 10) # Initial mesh of 10 samples along L
mesh = initial_mesh
# a_init must be complex in order to solve the problem in a complex domain
a_init = np.vstack((np.ones(np.size(initial_mesh)).astype(complex),
np.zeros(np.size(initial_mesh)).astype(complex)))
for idx, beta in enumerate(beta_analysis):
print(f"Iteration {idx}: beta_analysis = {beta}")
global beta_zf
beta_zf = beta * np.ones(len(z)) # Global variable so as to use it in coupling_equation(x, y)
a = integrate.solve_bvp(fun=coupling_equation,
bc=boundary_conditions,
x=mesh,
y=a_init,
max_nodes=max_mesh,
verbose=1)
# mesh = a.x # Mesh for the next iteration
# a_init = a.y # Initial guess for the next iteration, corresponding to the current solution
s11_analysis[idx] = a.y[1][0]
s21_analysis[idx] = a.y[0][-1]
return s11_analysis, s21_analysis
I suspect that the problem has something to do with the initial guess that is being passed to the different iterations (see commented lines inside the loop in the analysis function). I try to set the solution of an iteration as the initial guess for the following (which must reduce the time needed for the solver), but it is even slower, which I don't understand. Maybe I missed something, because it is my first time trying to solve differential equations.
The parameters used for the execution are the following:
f2 = h5py.File(r'path/to/file', 'r')
k = np.array(f2['k']).squeeze()
z = np.array(f2['z']).squeeze()
f2.close()
analysis_points = 501
max_mesh = 1e6
beta_0 = 3e2;
beta_low = 0; # Lower value of the frequency for the analysis
beta_up = beta_0; # Upper value of the frequency for the analysis
beta_analysis = np.linspace(beta_low, beta_up, analysis_points);
s11_analysis, s21_analysis = analysis(k, z, beta_analysis, max_mesh)
Any ideas on how to improve the performance of these functions? Thank you all in advance, and sorry if the question is not well-formulated, I accept any suggestions about this.
Edit: Added some information about performance and sizing of the problem.
In practice, I can't find a relation that determines de number of times coupling_equation is called. It must be a matter of the internal operation of the solver. I checked the number of callings in one iteration by printing a line, and it happened in 133 ocasions (this was one of the fastests). This must be multiplied by the number of iterations of beta. For the analyzed one, the solver returned this:
Solved in 11 iterations, number of nodes 529.
Maximum relative residual: 9.99e-04
Maximum boundary residual: 0.00e+00
The shapes of a and z_mesh are correlated, since z_mesh is a vector whose length corresponds with the size of the mesh, recalculated by the solver each time it calls coupling_equation. Given that a contains the amplitudes of the progressive and regressive waves at each point of z_mesh, the shape of a is (2, len(z_mesh)).
In terms of computation times, I only managed to achieve 19 iterations in about 2 hours with Python. In this case, the initial iterations were faster, but they start to take more time as their mesh grows, until the point that the mesh saturates to the maximum allowed value. I think this is because of the value of the input coupling coefficients in that point, because it also happens when no loop in beta_analysisis executed (just the solve_bvp function for the intermediate value of beta). Instead, Matlab managed to return a solution for the entire problem in just 6 minutes, aproximately. If I pass the result of the last iteration as initial_guess (commented lines in the analysis function, the mesh overflows even faster and it is impossible to get more than a couple iterations.
Based on semi-random inputs, we can see that max_mesh is sometimes reached. This means that coupling_equation can be called with a quite big z_mesh and a arrays. The problem is that coupling_equation contains a slow pure-Python loop iterating on each column of the arrays. You can speed the computation up a lot using Numpy vectorization. Here is an implementation:
def coupling_equation_fast(z_mesh, a):
ka_z = k # Global
z_a = z # Global
a_p = np.empty(a.shape, dtype=np.complex128)
beta_zf_i = np.interp(z_mesh, z_a, beta_zf) # Get beta at the desired point of the mesh
ka_z_i = np.interp(z_mesh, z_a, ka_z) # Get ka at the desired point of the mesh
# Fast manual matrix multiplication
a_p[0] = (-1j * beta_zf_i) * a[0] + ka_z_i * a[1]
a_p[1] = ka_z_i * a[0] + (1j * beta_zf_i) * a[1]
return a_p
This code provides a similar output with semi-random inputs compared to the original implementation but is roughly 20 times faster on my machine.
Furthermore, I do not know if max_mesh happens to be big with your inputs too and even if this is normal/intended. It may make sense to decrease the value of max_mesh in order to reduce the execution time even more.
I am trying to solve an ODE using Python's solve_ivp. However, I want to change the right hand side of my ODE dynamically based on a comparison between the current solution and previous solution. The idea behind this is that my right hand side is a vector field, and I want to ensure directionality of the vector field by reversing the right hand side based on the direction of the previous solution.
The implementation for this is as follows: I want to check the dot product in the right hand side function definition between the previous solution and the vector field. If the dot product is negative the right hand side is multiplied by -1.
I therefore need to access the previous state of the ODE solver and use it in comparison with the current iteration. In MATLAB there is the possibility of using "OutputFcn" while solving an ODE. This function is called after every iteration of the integrator. In the function it is therefore possible to simply extract the state as a variable and use it in the next iteration. I have not been able to find something similar for Python.
def RHS(timesnotused,x):
out = solve_ivp(doubleGyreVar, [0,T/2, T], [x[0], x[1], 1, 0, 0, 1], rtol = 1e-10, atol=1e-10)
output = out.y
J = output[2:,-1].reshape(2,2)
CG = np.matmul(J.T , J)
lambdas, xis = np.linalg.eig(CG)
xi_1 = xis[np.argmin(lambdas)]
xi_2 = xis[np.argmax(lambdas)]
lambda_1 = np.min(lambdas)
lambda_2 = np.max(lambdas)
alpha = ((lambda_2-lambda_1) / (lambda_2+lambda_1))**2
sign = 1
if np.dot(xlast,xi_1) < 0:
sign = -1
return(sign*alpha*xi_1)
As can be seen I want "xlast" to be the previous solution, and check it with xi_1 of the current iteration. Somehow xlast needs to be updated every iteration.
If the differential equation you want to solve is
dx/dt = sign(g(x)) * F(x)
for some sufficiently smooth functions g, F, then you have a discontinuous right side where all advanced numerical solvers will produce nonsense as soon as they approach this jump singularity.
The clearest method to solve such a multi-phase system is to present only continuous right sides to the numerical solver and to handle the phase change via the event mechanism that is also present in scipy.integrate.solve_ivp.
I explored a mechanism to do that with the tools of scipy in a similar problem with a sign function producing a discontinuity in odeint returns wrong results for an ODE including descrete function
To check some of my results more easily I used an Excel sheet to make a few diagrams. However, I noticed something really awkward.
EDIT :
So let present the problem in another way, I found something that represent what I don't understand in my code.
import numpy as np
from scipy.integrate import odeint
A = []
def F(y, z):
global A
a = y[0]
b = y[1]
A.append(a)
return [a, b]
y0 = [1, 1]
z = np.linspace(0, 1, 101)
y = odeint(F, y0, z)
print(len(z), len(A))
The question is why the length of z and A are different (e.g. 101 and 55)?
For me ,during the solving, a should vary len(z) times and so A. So it looks like the linspace is not doing anything on the solving of the equations. Or perhaps I haven't understood the usage of linspace in Python.
The solution via odeint uses an implicit linear multi-step method with adaptive internal time stepping. This is implemented via a PECE predictor-corrector scheme. The E there stands for "evaluation". Which means that in each internal integration step, the ODE function is called twice. You might get less internal steps than the input time list has entries, the output array is interpolated from the internal time steps, so that you can have multiple output values per internal step. But the other extreme is also possible, that to reach the requested tolerances the internal step size is so small that one output time step requires multiple internal steps.
If the problem were more stiff, there would be even more calls, periodically for the numerical approximation of the Jacobian, and possibly multiple calls per step of the Newton-like corrector step or just multiple simple correction steps, which is then called PE(CE)d.
To compare with, look at the explicit RK4 method. There you have 4 evaluations of the ODE function per time step. The Dormand-Prince method of ode45 has 6+1 evaluations per time step, however there the internal time steps need not correspond to the time sample list passed to the method, the requested output samples are interpolated from the internal steps.
Here is the thing.
I am trying to use fsolve function in Python to find the root of a cubic function. This cubic function has a parameter, deltaW. What I do is change this parameter deltaW from -50 to 50, and find the root of the cubic function at the same time. Below is my script:
from scipy.optimize import fsolve
import matplotlib.pyplot as plt
import numpy as np
import pylab
g = 5.61
gamma = 6.45
kappa = 6.45
J = 6.45
rs = 1.0 #There are just parameters
m = 5.0*10**(-11)
wm = 2*3.14*23.4
X = []
X1 = []
def func(x): #Define the cubic function I need to solve
A = 1j*g**2*(kappa + 1j*deltaW)*x*x/(m*wm**2)
B = J**2 + (1j*deltaW - gamma)*(1j*deltaW + kappa)
C = A + B
D = abs(C)*x - J*np.sqrt(2*kappa)*rs
return D
for deltaW in np.linspace(-50, 50, 1000):
x0 = fsolve(func, 0.0001)
X.append(x0)
deltaW = np.linspace(-50, 50, 1000)
plt.plot(deltaW, X)
plt.show()
When I run this script, I get these two messages:
RuntimeWarning: The iteration is not making good progress, as measured by the
improvement from the last five Jacobian evaluations.
warnings.warn(msg, RuntimeWarning)
/usr/lib/python2.7/dist-packages/scipy/optimize/minpack.py:152: RuntimeWarning: The iteration is not making good progress, as measured by the
improvement from the last ten iterations.
warnings.warn(msg, RuntimeWarning)
I am sorry I do not have enough reputation to put the plot of this script here. My question is why do I get this message and why do my plot look so weird in the left part.
Is it because of my code is wrong?
As in almost all cases of finding roots, a good initial guess is imperative. Sometimes the best initial guess is, in fact, known to be wrong. That is the case here. The behavior of your script, which shows unexpected 'spikes' in the answer, can be looked at more deeply by both plotting up the function, and plotting up the found roots around those spikes (hey, you've got a Python console - this is really easy).
What you find is that the solution returned by the solver is jumping around, even though the function really doesn't look that different. The problem is that your initial guess of 0.0001 lies close to a tiny minimum of the function, and the solver can't figure out how to get out of there. Setting the initial guess to 1.0 (way far away, but on a nice, easy descending portion of the function that will head directly to the root), results instead in:
So, three things:
1. solvers need loving care and attention - they are rarely automagic.
Sometimes the 'right' initial guess can be well away from what you know is the right answer, but in such a way that the solver has an easy time of it.
the interactive Python console lets you look quickly at what is going on. Use the power of it!
I try to fit a function to my data using scipy.optimize.curvefit.
Q=optimization.curve_fit(func,X,Y, x0,ERR)
and it works well.
However, now I am trying to use an asymmetric error and I have no idea how to do that - or even if it is possible.
By asymmetric error I mean that the error is not for example: 3+-0.5 but 3 +0.6 -0.2.
So that ERR is an array with two columns.
It would be great if somebody had an idea how to do that - or could me point to a different Python routine which might be able to do it.
That a snippet of the code I am using - but I am not sure it makes it clearer:
A=numpy.genfromtxt('WF.dat')
cc=A[:,4]
def func(A,a1,b1,c1):
N=numpy.zeros(len(x))
for i in range(len(x)):
N[i]=1.0*erf(a1*(A[i,1]-c1*A[i,0]**b1))
return N
x0 = numpy.array([2.5 , -0.07 ,-5.0])
Q=optimization.curve_fit(func,A,cc, x0, Error)
And Error=[ErP,ErM] (2 columns)
Least squares algorithm like curve_fit or scipy.optimize.leastsq will not be able to do this because the loss function is quadratic, and so symmetric for positive and negative error.
I haven't seen any models for this and maybe PAIDA can handle it, as DanHickstein mentioned.
Otherwise, you could use the nonlinear optimizers like optimize.fmin and construct your own asymmetric loss function.
def loss_function(params, ...):
error = (y - func(x, params))
error_neg = (error < 0)
error_squared = error**2 / (error_neg * sigma_low + (1 - error_neg) * sigma_upp))
return error_squared.sum()
and minimize this with fmin or fmin_bfgs.
(I never tried this.)
In the current version, I am afraid it is not doable. curve_fit is a wrap around the popular Fortran library minipack. Check the source code of \scipy_install_path\optimize\minipack.py, you will see: (line 498-509):
if sigma is None:
func = _general_function
else:
func = _weighted_general_function
args += (1.0/asarray(sigma),)
Basically what it means is that of sigma is not provided, then the unweighted Levenberg-Marquardt method in minipack will be called. If sigma is provided, then the weighted LM will be called. That implies, if sigma is to be provided, it must be provided as a array of the same length of X or Y.
That means if you want to have asymmetric error residue on Y, you have to come up with some modification to your target function, as #Jaime suggested.
I'm not 100% sure, but it looks like the PAIDA package might do fits with asymmetric errors:
http://paida.sourceforge.net/documentation/fitter/index.html
A solution, which I've used frequently, is to draw realisations (say 100-1000) from a split-normal distribution, and run your fitting algorithm on each of these realisations with the error set to 0.0. You'll then have 100-1000 best-fitting parameters, from which you can simply take the median, along with any error estimate you want to use.