I currently want to implement a Hammerstein model in sympy. I have now created a small example for a simple system:
import numpy as np
from sympy import *
####HAMMERSTEIN MODEL####
#time
t = symbols("t")
#inputs
u = symbols('u')
#states
y = symbols('y',cls = Function, Function = True)
#init states
y_init =symbols('y_init')
#parameters
gain = 2 #symbols('gain')
time_constant = 20000#symbols('time_constant')
#EQUATIONS
#NONLINEAR STATIC PART
u_nonlinear = u**2 # nonlinear input
#DYNAMIC PART
# first order system with inputs
rhe = (gain * u_nonlinear - y(t)) * 1/time_constant
ode = Eq(diff(y(t),t),rhe)
#solve equation
sol_step = dsolve(ode, ics = {y(0): y_init})
sol_step = sol_step.rhs
#lambdify (sympy)
system_step =lambdify((t,u, y_init),sol_step, 'sympy')
#####SIMULATE STEPWISE######
nr_steps = 10
dt=1
u_data =IndexedBase('u_data')
y_init_data =symbols('y_init_data')
#solution vector
sol =[]
for i in range(nr_steps):
#first sim. step
if i == 0:
sol.append(system_step(dt,u_data[i],y_init_data))
#uses the states of prev. solution as inits
else:
sol.append(system_step(dt,u_data[i],sol[i-1]))
#convert
system=lambdify((u_data,y_init_data),sol, 'numpy')
#EXAMPLE
t_obs = np.linspace(0,10,10)
u_obs = np.ones(10)* 40
x_obs_init =20
#RESULT
print(system(u_obs,x_obs_init))
As you can see from the example, I solve the problem step by step. I always call the Sympy function object "system_step".
The performance is not particularly good with larger systems.
However, I would also like to use the simulation in a scipy optimizer, which leads to it being called several times, which extremely increases the solution time
My problem:
1.)
Can this step-by-step calculation also be implemented using sympy (e.g. indexed objects)? Can the repeated calculation in the loop be avoided?
2.) If so, how can this be done if the length of the input variables (u) should remain flexible and not be specified by a fixed index (m) using hardcode (see nr_steps).
Thank you very much!
Thank you for the info. If I calculate the ODE system with constant input values, I do not need to calculate it step by step. Then the solution process is very quick. Therefore, my idea was to set up the system using vectors or indexed objects, which can prevent the step-by-step calculation.
My goal:
set up the system with variable input variables
solve the system symbolically, even if it takes a very long time
Lambdify and storage in a binary file
use the solved system for different operations
Related
I'm facing a problem while trying to implement the coupled differential equation below (also known as single-mode coupling equation) in Python 3.8.3. As for the solver, I am using Scipy's function scipy.integrate.solve_bvp, whose documentation can be read here. I want to solve the equations in the complex domain, for different values of the propagation axis (z) and different values of beta (beta_analysis).
The problem is that it is extremely slow (not manageable) compared with an equivalent implementation in Matlab using the functions bvp4c, bvpinit and bvpset. Evaluating the first few iterations of both executions, they return the same result, except for the resulting mesh which is a lot greater in the case of Scipy. The mesh sometimes even saturates to the maximum value.
The equation to be solved is shown here below, along with the boundary conditions function.
import h5py
import numpy as np
from scipy import integrate
def coupling_equation(z_mesh, a):
ka_z = k # Global
z_a = z # Global
a_p = np.empty_like(a).astype(complex)
for idx, z_i in enumerate(z_mesh):
beta_zf_i = np.interp(z_i, z_a, beta_zf) # Get beta at the desired point of the mesh
ka_z_i = np.interp(z_i, z_a, ka_z) # Get ka at the desired point of the mesh
coupling_matrix = np.empty((2, 2), complex)
coupling_matrix[0] = [-1j * beta_zf_i, ka_z_i]
coupling_matrix[1] = [ka_z_i, 1j * beta_zf_i]
a_p[:, idx] = np.matmul(coupling_matrix, a[:, idx]) # Solve the coupling matrix
return a_p
def boundary_conditions(a_a, a_b):
return np.hstack(((a_a[0]-1), a_b[1]))
Moreover, I couldn't find a way to pass k, z and beta_zf as arguments of the function coupling_equation, given that the fun argument of the solve_bpv function must be a callable with the parameters (x, y). My approach is to define some global variables, but I would appreciate any help on this too if there is a better solution.
The analysis function which I am trying to code is:
def analysis(k, z, beta_analysis, max_mesh):
s11_analysis = np.empty_like(beta_analysis, dtype=complex)
s21_analysis = np.empty_like(beta_analysis, dtype=complex)
initial_mesh = np.linspace(z[0], z[-1], 10) # Initial mesh of 10 samples along L
mesh = initial_mesh
# a_init must be complex in order to solve the problem in a complex domain
a_init = np.vstack((np.ones(np.size(initial_mesh)).astype(complex),
np.zeros(np.size(initial_mesh)).astype(complex)))
for idx, beta in enumerate(beta_analysis):
print(f"Iteration {idx}: beta_analysis = {beta}")
global beta_zf
beta_zf = beta * np.ones(len(z)) # Global variable so as to use it in coupling_equation(x, y)
a = integrate.solve_bvp(fun=coupling_equation,
bc=boundary_conditions,
x=mesh,
y=a_init,
max_nodes=max_mesh,
verbose=1)
# mesh = a.x # Mesh for the next iteration
# a_init = a.y # Initial guess for the next iteration, corresponding to the current solution
s11_analysis[idx] = a.y[1][0]
s21_analysis[idx] = a.y[0][-1]
return s11_analysis, s21_analysis
I suspect that the problem has something to do with the initial guess that is being passed to the different iterations (see commented lines inside the loop in the analysis function). I try to set the solution of an iteration as the initial guess for the following (which must reduce the time needed for the solver), but it is even slower, which I don't understand. Maybe I missed something, because it is my first time trying to solve differential equations.
The parameters used for the execution are the following:
f2 = h5py.File(r'path/to/file', 'r')
k = np.array(f2['k']).squeeze()
z = np.array(f2['z']).squeeze()
f2.close()
analysis_points = 501
max_mesh = 1e6
beta_0 = 3e2;
beta_low = 0; # Lower value of the frequency for the analysis
beta_up = beta_0; # Upper value of the frequency for the analysis
beta_analysis = np.linspace(beta_low, beta_up, analysis_points);
s11_analysis, s21_analysis = analysis(k, z, beta_analysis, max_mesh)
Any ideas on how to improve the performance of these functions? Thank you all in advance, and sorry if the question is not well-formulated, I accept any suggestions about this.
Edit: Added some information about performance and sizing of the problem.
In practice, I can't find a relation that determines de number of times coupling_equation is called. It must be a matter of the internal operation of the solver. I checked the number of callings in one iteration by printing a line, and it happened in 133 ocasions (this was one of the fastests). This must be multiplied by the number of iterations of beta. For the analyzed one, the solver returned this:
Solved in 11 iterations, number of nodes 529.
Maximum relative residual: 9.99e-04
Maximum boundary residual: 0.00e+00
The shapes of a and z_mesh are correlated, since z_mesh is a vector whose length corresponds with the size of the mesh, recalculated by the solver each time it calls coupling_equation. Given that a contains the amplitudes of the progressive and regressive waves at each point of z_mesh, the shape of a is (2, len(z_mesh)).
In terms of computation times, I only managed to achieve 19 iterations in about 2 hours with Python. In this case, the initial iterations were faster, but they start to take more time as their mesh grows, until the point that the mesh saturates to the maximum allowed value. I think this is because of the value of the input coupling coefficients in that point, because it also happens when no loop in beta_analysisis executed (just the solve_bvp function for the intermediate value of beta). Instead, Matlab managed to return a solution for the entire problem in just 6 minutes, aproximately. If I pass the result of the last iteration as initial_guess (commented lines in the analysis function, the mesh overflows even faster and it is impossible to get more than a couple iterations.
Based on semi-random inputs, we can see that max_mesh is sometimes reached. This means that coupling_equation can be called with a quite big z_mesh and a arrays. The problem is that coupling_equation contains a slow pure-Python loop iterating on each column of the arrays. You can speed the computation up a lot using Numpy vectorization. Here is an implementation:
def coupling_equation_fast(z_mesh, a):
ka_z = k # Global
z_a = z # Global
a_p = np.empty(a.shape, dtype=np.complex128)
beta_zf_i = np.interp(z_mesh, z_a, beta_zf) # Get beta at the desired point of the mesh
ka_z_i = np.interp(z_mesh, z_a, ka_z) # Get ka at the desired point of the mesh
# Fast manual matrix multiplication
a_p[0] = (-1j * beta_zf_i) * a[0] + ka_z_i * a[1]
a_p[1] = ka_z_i * a[0] + (1j * beta_zf_i) * a[1]
return a_p
This code provides a similar output with semi-random inputs compared to the original implementation but is roughly 20 times faster on my machine.
Furthermore, I do not know if max_mesh happens to be big with your inputs too and even if this is normal/intended. It may make sense to decrease the value of max_mesh in order to reduce the execution time even more.
Consider an optimal control problem as follows:
min int(0,1) x(t).u(t)+u(t)**2
x.dt()= x(t)+u(t), x(0)=1
0 <= u(t) <= t**2+(1-t)**3 for 0<=t<=1
My first question is how to define the upper bound of control in Gekko. Also, suppose we want to compare this problem with the case the control is constant during the planning horizon, i.e., u(0)=...=u(t)=...=u(1). How can we define it?
In another case, how is it possible to have fixed but unknown control in different sub-intervals? For example, in [0,t1], control should be fixed, in [t1,t2] control should be fixed but can be different from control in [0,t1] (e.g. t1=0.5, t2=1, Tf=t2=1).
I would be thankful to know if it is possible to study a case where t1, t2, ... are also control and should be determined?
Here is code that gives a solution to the problem:
# min int(0,1) x(t).u(t)+u(t)**2
# x.dt()= x(t)+u(t), x(0)=1
# 0 <= u(t) <= t**2+(1-t)**3 for 0<=t<=1
import numpy as np
from gekko import GEKKO
m = GEKKO(remote=False); m.time=np.linspace(0,1,101)
t = m.Var(0); m.Equation(t.dt()==1)
ub = m.Intermediate(t**2+(1-t)**3)
u = m.MV(0,lb=0,ub=1); m.Equation(u<=ub)
u.STATUS=1; u.DCOST=0; m.free_initial(u)
x = m.Var(0); m.Equation(x.dt()==x+u)
p = np.zeros(101); p[-1]=1; final=m.Param(p)
m.Minimize(m.integral(x*u+u**2)*final)
m.options.IMODE=6; m.options.NODES=3; m.solve()
print(m.options.OBJFCNVAL)
import matplotlib.pyplot as plt
plt.plot(m.time,x.value,'b--',label='x')
plt.plot(m.time,u.value,'k-',label='u')
plt.plot(m.time,ub.value,'r--',label='ub')
plt.legend()
plt.show()
The solution isn't very interesting because the optimal objective is u(t)=0 and x(t)=0. If you add a final condition like x(1)=0.75 then the solution is more interesting.
m.Equation(final*(x-0.75)==0)
If you want all of the interval to be one value then I recommend that you use a u=m.FV() type. The u=m.MV() type is adjustable by the optimizer at every interval when you set u.STATUS=1. You can also reduce degrees of freedom with the m.options.MV_STEP_HOR=5 as a global option in Gekko for all MVs or else u.MV_STEP_HOR=5 to adjust it just for that MV. There is more information on the different Gekko types.
You can set the final time by using m.time = [0,...,1] and then scale it with time final tf. The derivatives in your problem need to be divided by tf as well. Here is a related rocket launch problem or the Jennings optimal control problem that minimize the final time. You can also set up multiple time intervals and then connect them with m.Connection().
I have a (Portfolio Optimization) python program that uses scipy to optimize n variables using constraints. However, I was wondering if it is possible to tell the program to choose m out of these n variables that will be the best for maximsing the objective?
This is my current code:
def obj(x):
return (-np.sum(array_weights.t*x))
def con_vol(x):
return np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))
where array_returns is list of returns for all the stocks that is imported from Bloomberg and cov_matrix is the covariance matrix constructed using the returns data.
cons = [{'type':'eq','fun':lambda x: np.sum(x)-1}, {'type':'ineq','fun':lambda x: con_vol}\
bnds = tuple(0.02, 0.1) for x in range(20)
opts = sco.minimize(obj, list_final_weights, bounds = bnds, method = 'SLQSP', constraints = cons)
In this program weights are allocated to all the assets. I want a way in which it can choose (say the best 10 to allocate to out of 20)
One option would be to solve this problem iteratively (running the scipy solver many times):
1st iteration: run the scipy solver, take the solution, and discard all of the n variables the coefficients of which are below a certain (initially rather small) threshold t.
Next iteration: run the scipy solver again, now not searching within the space of the previously discarded variables anymore. With a (slighlty) increased value of t, now discard further variables.
Repeat this until only m variables left.
There are also more sophisticated approaches involving e.g. genetic programming techniques to identify relevant parameters or even functional forms (so-called sparse symbolic regression in the latter case, see e.g. a python implementation here: https://github.com/snagcliffs/PDE-FIND.
I'm currently making the switch from MATLAB to Python for a project that involves solving differential equations.
In MATLAB if the t array that's passed only contains two elements, the solver outputs all the intermediate steps of the simulation. However, in Python you just get the start and end point. To get time points in between you have to explicitly specify the time points you want.
from scipy import integrate as sp_int
import numpy as np
def odeFun(t,y):
k = np.ones((2))
dy_dt = np.zeros(y.shape)
dy_dt[0]= k[1]*y[1]-k[0]*y[0]
dy_dt[1]=-dy_dt[0]
return(dy_dt)
t = np.linspace(0,10,1000)
yOut = sp_int.odeint(odeFun,[1,0],t)
I've also looked into the following method:
solver = sp_int.ode(odefun).set_integrator('vode', method='bdf')
solver.set_initial_value([1,0],0)
dt = 0.01
solver.integrate(solver.t+dt)
However, it still requires an explicit dt. From reading around I understand that Python's solvers (e.g. 'vode') calculates intermediate steps for the dt requested, and then interpolates that time point and outputs it. What I'd like though is to get all these intermediate steps directly without the interpolation. This is because they represent the minimum number of points required to fully describe the time series within the integration tolerances.
Is there an option available to do that?
I'm working in Python 3.
scipy.integrate.odeint
odeint has an option full_output that allows you to obtain a dictionary with information on the integration, including tcur which is:
vector with the value of t reached for each time step. (will always be at least as large as the input times).
(Note the second sentence: The actual steps are always as fine as your desired output. If you want use the minimum number of necessary step, you must ask for a coarse sampling.)
Now, this does not give you the values, but we can obtain those by integrating a second time using these very steps:
from scipy.integrate import odeint
import numpy as np
def f(y,t):
return np.array([y[1]-y[0],y[0]-y[1]])
start,end = 0,10 # time range we want to integrate
y0 = [1,0] # initial conditions
# Function to add the initial time and the target time if needed:
def ensure_start_and_end(times):
times = np.insert(times,0,start)
if times[-1] < end:
times = np.append(times,end)
return times
# First run to establish the steps
first_times = np.linspace(start,end,100)
first_run = odeint(f,y0,first_times,full_output=True)
first_steps = np.unique(first_run[1]["tcur"])
# Second run to obtain the results at the steps
second_times = ensure_start_and_end(first_steps)
second_run = odeint(f,y0,second_times,full_output=True,h0=second_times[0])
second_steps = np.unique(second_run[1]["tcur"])
# ensuring that the second run actually uses (almost) the same steps.
np.testing.assert_allclose(first_steps,second_steps,rtol=1e-5)
# Your desired output
actual_steps = np.vstack((second_times, second_run[0].T)).T
scipy.integrate.ode
Having some experience with this module, I am not aware of any way to obtain the step size without digging deeply into the internals.
I have a dynamic model set up as a (stiff) system of ODEs. I currently solve this with CVODE (from the SUNDIALS package in the Assimulo python package) and all is good.
I now want to add a new 3D heat sink (with temperature-dependent thermal parameters) to the problem. Instead of writing out all the equations from scratch for the 3D heat equation, my idea is to use an existing FEM or FVM framework to provide to me an interface that will allow me to easily provide the (t, y) for the 3D block to a routine, and get back the residuals y'. The principle is to use the equations from the FEM system but not the solver. CVODE can exploit sparsity, but the combined system is expected to solve slower than the FEM system would solve on its own, being tailored for such.
# pseudocode of a residuals function for CVODE
def residual(t, y):
# ODE system of n equations
res[0] = <function of t,y>;
res[1] = <function of t,y>;
...
res[n] = <function of t,y>;
# Here we add the FEM/FVM residuals
for i in range(FEMcount):
res[n+1+i] = FEMequations[FEMcount](t,y)
return res
My question is whether (a) this approach is sane, and (b) is there a FEM or FVM library that will easily let me treat it as a system of equations, such that I can "tack it on" to my existing set of ODE equations.
If can't let the two systems share the same time axis, then I will have to run them in a stepping mode, where I run the one model for a short time, update the boundary conditions for the other, run that one, update the first model's BCs, and so on.
I have some experience with the wonderful library FiPy, and I am expecting to eventually end up using that library in the manner described above. But I want to know about experience with other systems in problems of this nature, and also other approaches that I have missed.
Edit: I now have some example code that appears to be working, showing how the FiPy mesh diffusion residuals can be solved with CVODE. However, this is only one approach (using FiPy) and the remainder of my other questions and concerns still stand. Any suggestions welcome.
from fipy import *
from fipy.solvers.scipy import DefaultSolver
solverFIPY = DefaultSolver()
from assimulo.solvers import CVode as solverASSIMULO
from assimulo.problem import Explicit_Problem as Problem
# FiPy Setup - Using params from the Mesh1D example
###################################################
nx = 50; dx = 1.; D = 1.
mesh = Grid1D(nx = nx, dx = dx)
phi = CellVariable(name="solution variable", mesh=mesh, value=0.)
valueLeft, valueRight = 1., 0.
phi.constrain(valueRight, mesh.facesRight)
phi.constrain(valueLeft, mesh.facesLeft)
# Instead of eqX = TransientTerm() == ExplicitDiffusionTerm(coeff=D),
# Rather just operate on the diffusion term. CVODE will calculate the
# Transient side
edt = ExplicitDiffusionTerm(coeff=D)
timeStepDuration = 0.9 * dx**2 / (2 * D)
steps = 100
# For comparison with an analytical solution - again,
# taken from the Mesh1D.py example
phiAnalytical = CellVariable(name="analytical value", mesh=mesh)
x = mesh.cellCenters[0]
t = timeStepDuration * steps
from scipy.special import erf
phiAnalytical.setValue(1 - erf(x / (2 * numerix.sqrt(D * t))))
if __name__ == '__main__':
viewer = Viewer(vars=(phi, phiAnalytical))#, datamin=0., datamax=1.)
viewer.plot()
raw_input('Press a key...')
# Now for the Assimulo/Sundials solver setup
############################################
def residual(t, X):
# Pretty straightforward, phi is the unknown
phi.value = X # This is a vector, 50 elements
# Can immediately return the residuals, CVODE sees this vector
# of 50 elements as X'(t), which is like TransientTerm() from FiPy
return edt.justResidualVector(var=phi, solver=solverFIPY)
x0 = phi.value
t0 = 0.
model = Problem(residual, x0, t0)
simulation = solverASSIMULO(model)
tfinal = steps * timeStepDuration # s,
cell_tol = [1.0e-8]*50
simulation.atol = cell_tol
simulation.rtol = 1e-6
simulation.iter = 'Newton'
t, x = simulation.simulate(tfinal, 0)
print x[-1]
# Write back the answer to compare
phi.value = x[-1]
viewer.plot()
raw_input('Press a key...')
This will produce a graph showing a perfect match:
An ODE is a differential equation in one dimension.
An FEM model is for problems that are spacial ie, problems in higher dimensions. You want a finite difference method. It's easier to solve and understand from the perspective someone coming from ODE world.
The question I think you should really be asking is how do you take your ODE, and transfer it to a 3D problem space.
Multidimensional partial differential equations are difficult to solve, yet I'll refer you to a CFD method for doing just that however only in 2D. http://lorenabarba.com/blog/cfd-python-12-steps-to-navier-stokes/
It should take you a solid afternoon to get through that! Good Luck!