Related
As mentioned above, the function below works, however its very slow. I am very interested in using faster/optimised numpy (or other) vectorized alternatives. I have not posted the entire script here due to it being too large.
My specific question is - are there suitable numpy (or other) functions that I can use to 1) reduce run time and 2) reduce code volume of this function, specifically the for loop?
Edit: mass, temp, U and dpdh are functions that carry out simple algebraic calculations and return constants
def my_system(t, y, n, hIn, min, mAlumina, cpAlumina, sa, V):
dydt = np.zeros(3 * n) #setting up zeros array for solution (solving for [H0,Ts0,m0,H1,Ts1,m1,H2,Ts2,m2,..Hn,Tsn,mn])
# y = [h_0, Ts_0, m_0, ... h_n, Ts_n, m_n]
# y[0] = hin
# y[1] = Ts0
# y[2] = minL
i=0
## Using thermo
T = temp(y[i],P) #initial T
m = mass(y[i],P) #initial m
#initial values
dydt[i] = (min * (hIn - y[i]) + (U(hIn,P,min) * sa * (y[i + 1] - T))) / m # dH/dt (eq. 2)
dydt[i + 1] = -(U(hIn,P,min) * sa * (y[i + 1] - T)) / (mAlumina * cpAlumina) # dTs/dt from eq.3
dmdt = dydt[i] * dpdh(y[i], P) * V # dm/dt (holdup variation) eq. 4b
dydt[i + 2] = min - dmdt # mass flow out (eq.4a)
for i in range(3, 3 * n, 3): #starting at index 3, and incrementing by 3 because we are solving for 'triplets' [h,Ts,m] in each loop
## Using thermo
T = temp(y[i],P)
m = mass(y[i],P)
# [h, TS, mdot]
dydt[i] = (dydt[i-1] * (y[i - 3] - y[i]) + (U(y[i-3], P, dydt[i-1]) * sa * (y[i + 1] - T))) / m # dH/dt (eq.2), dydt[i-1] is the mass of the previous tank
dydt[i + 1] = -(U(y[i-3], P, dydt[i-1]) * sa * (y[i + 1] - T)) / (mAlumina * cpAlumina) # dTs/dt eq. (3)
dmdt = dydt[i] * dpdh(y[i], P) * V # Equation 4b
dydt[i + 2] = dydt[i-1] - dmdt # Equation 4a
return dydt
The functions mass, temp, U, and dpdh used inside the my_system function all take numbers as input, perform some simple algebraic operation and return a number (no need to optimise these I am just providing them for further context)
def temp(H,P):
# returns temperature given enthalpy (after processing function)
T = flasher.flash(H=H, P=P, zs=zs, retry=True).T
return T
def mass(H, P):
# returns mass holdup in mol
m = flasher.flash(H=H, P=P, zs=zs, retry=True).rho()*V
return m
def dpdh(H, P):
res = flasher.flash(H=H, P=P, zs=zs, retry=True)
if res.phase_count == 1:
if res.phase == 'L':
drho_dTf = res.liquid0.drho_dT()
else:
drho_dTf = res.gas.drho_dT()
else:
drho_dTf = res.bulk._equilibrium_derivative(of='rho', wrt='T', const='P')
dpdh = drho_dTf/res.dH_dT_P()
return dpdh
def U(H,P,m):
# Given T, P, m
air = Mixture(['nitrogen', 'oxygen'], Vfgs=[0.79, 0.21], H=H, P=P)
mu = air.mu*1000/mWAir #mol/m.s
cp = air.Cpm #J/mol.K
kg = air.k #W/m.K
g0 = m/areaBed #mol/m2.s
a = sa*n/vTotal #m^2/m^3 #QUESTIONABLE
psi = 1
beta = 10
pr = (mu*cp)/kg
re = (6*g0)/(a*mu*psi)
hfs = ((2.19*(re**1/3)) + (0.78*(re**0.619)))*(pr**1/3)*(kg)/diameterParticle
h = 1/((1/hfs) + ((diameterParticle/beta)/kAlumina))
return h
Reference Image:
enter image description here
For improving the speed, you can see Numba, which is useable if you use NumPy a lot but not every code can be used with Numba. Apart from that, the formulation of the equation system is confusing. You are solving 3 equations and adding the result to a single dydt list by 3 elements each. You can simply create three lists, solve each equation and add them to their respective list. For this, you need to re-write my_system as:
import numpy as np
def my_system(t, RHS, hIn, Ts0, minL, mAlumina, cpAlumina, sa, V):
# get initial boundary condition values
y1 = RHS[0]
y2 = RHS[1]
y3 = RHS[2]
## Using thermo
T = # calculate T
m = # calculate m
# [h, TS, mdot] solve dy1dt for h, dy2dt for TS and dy3dt for mdot
dy1dt = # dH/dt (eq.2), y1 corresponds to initial or previous value of dy1dt
dy2dt = # dTs/dt eq. (3), y2 corresponds to initial or previous value of dy2dt
dmdt = # Equation 4b
dy3dt = # Equation 4a, y3 corresponds to initial or previous value of dy3dt
# Left-hand side of ODE
LHS = np.zeros([3,])
LHS[0] = dy1dt
LHS[1] = dy2dt
LHS[2] = dy3dt
return LHS
In this function, you can pass RHS as a list with initial values ([dy1dt, dy2dt, dy3dt]) which will be unpacked as y1, y2, and y3 respectively and use them for respective differential equations. The solved equations (next values) will be saved to dy1dt, dy2dt, and dy3dt which will be returned as a list LHS.
Now you can solve this using scipy.integrate.odeint. Therefore, you can leave the for loop structure and solve the equations by using this method as follows:
hIn = #some val
Ts0 = #some val
minL = #some val
mAlumina = #some vaL
cpAlumina = #some val
sa = #some val
V = #some val
P = #some val
## Using thermo
T = temp(hIn,P) #initial T
m = mass(hIn,P) #initial m
#initial values
y01 = # calculate dH/dt (eq. 2)
y02 = # calculate dTs/dt from eq.3
dmdt = # calculate dm/dt (holdup variation) eq. 4b
y03 = # calculatemass flow out (eq.4a)
n = # time till where you want to solve the equation system
y0 = [y01, y02, y03]
step_size = 1
t = np.linspace(0, n, int(n/step_size)) # use that start time to which initial values corresponds
res = odeint(my_sytem, y0, t, args=(hIn, Ts0, minL, mAlumina, cpAlumina, sa, V,), tfirst=True)
print(res[:,0]) # print results for dH/dt
print(res[:,1]) # print results for dTs/dt
print(res[:,2]) # print results for Equation 4a
Here, I have passed all the initial values as y0 and chosen a step size of 1 which you can change as per your need.
I'm converting my code from Matlab to Python and stuck on how to split the state vector such that the result returns the two solution. I have a vector and a single value for the two initial conditions and I expect as the final result a matrix and a vector.
I tried joining the initial conditions (y0 = [c_pt_0, x_0]) in the same manner as the solution (soln = [dfdt,dcdt]) (which is shown below in the code). I also tried a similar approach that is used in matlab, which is concatenating the the initial conditions to one single array and unpacking the results but I think the problem is in the dimensions.
#Basic imports
import numpy as np
import pylab
import matplotlib. pyplot as plt
import scipy
from scipy.integrate import odeint
import matplotlib.pyplot as plt
# define parameters
pi = 3.14159265
V_m = 9.09
m_V__M_Pt = 1e6/195.084
rho = 21.45
R0 = 10**(-8.19)
k_d = 10**(-13)
k_r = 10**(-5)
S = 0.314 #distribution parameter
M = 0.944 #distribution parameter
## geometry
# Finite Volume Method with equidistant elements
r_max = 30.1e-9 #maximum value
n = 301 #number of elements of FVM
dr = r_max/n #length of elements, equidistant
ini_r = np.linspace(5e-10,r_max,n+1) #boundaries of elements
mid_r = ini_r[0:n]+dr/2 #center of elements
## initial conditions
#initial distribution
x0 = 1/(S*np.sqrt(2*pi)*mid_r*1e9)*np.exp((-(np.log(mid_r*1e9)-M)**2)/(2*S**2))
c_pt_0 = 0
y0 = [x0, c_pt_0]
MN_0 = scipy.trapz(np.power(mid_r, 3)*x0,
x=mid_r) # initial mass
M_0 = 4/3*pi*rho*MN_0
def f(y, t):
r = y[0]
c_pt = y[1]
#materials balance
drdt = V_m * k_r * c_pt * np.exp(-R0/ mid_r) - V_m * k_d * np.exp(R0/ mid_r)
dmdt = 4*pi*rho*mid_r**2*drdt
dMdt = np.trapz(r*dmdt, x=mid_r)
dcdt = m_V__M_Pt*(-dMdt)/M_0
dfdt = -(np.gradient(r*drdt, dr))
soln = [dfdt, dcdt]
return soln
#------------------------------------------------------
#define timespace
time = np.linspace(0, 30, 500)
#solve ode system
sln_1 = odeint (f , y0 , time,
rtol = 1e-3, atol = 1e-5)
pylab.plot(mid_r, sln_1[1,:], color = 'r', marker = 'o')
pylab.plot(mid_r, sln_1[-1,:], color = 'b', marker = 'o')
plt.show()
Traceback:
ValueError: setting an array element with a sequence.
Any help is much appreciated.
EDIT: ADDED MATLAB CODE
Here is the MATLAB code that works that I want to convert to python where the state vector is split. I have three files (one main, the f function, and the parameters). Please excuse any face palm coding errors but I do appreciate any suggestions even for this.
modified_model.m:
function modified_model
% import parameters
p = cycling_parameters;
% initial conditions
c_pt_0 = 0;
y0 = [p.x0; c_pt_0];
% call integrator
options_ODE=odeset('Stats','on', 'RelTol',1e-3,'AbsTol',1e-5);
[~, y] = ode15s(#(t,y) f(t, y, p), p.time, y0, options_ODE);
%% Post processing
% split state vector
r = y(:,1:p.n);
c_Pt = y(:,p.n+1);
%% Plot results
figure
hold on;
plot(p.r_m, r(1,:));
plot(p.r_m, r(end,:));
xlabel({'size'},'FontSize',15)
ylabel({'counts'},'FontSize',15)
f.m
function soln = f(~, y, p)
%split state vector
r = y(1:p.n);
c_pt = y(p.n+1);
% materials balance
drdt = p.Vm_Pt.*p.k_rdp.*c_pt.*exp(-p.R0./p.r_m) - p.Vm_Pt.*p.k_dis.*exp(p.R0./p.r_m);
dmdt = 4*pi*p.rho*p.r_m.^2.*drdt;
dMdt = trapz(p.r_m, r.*dmdt);
dcdt = p.I_V*p.m_V__M_Pt*(-dMdt)/p.M_0;
dfdt = - gradient(r.*drdt,p.dr);
soln = [dfdt; dcdt];
and the parameters file: cycling_parameters.m
function p=cycling_parameters
p.M = 195.084;
p.rho = 21.45;
p.time = linspace(0, 30, 500);
p.m_V__M_Pt = 1e6/p.M;
p.Vm_Pt = 9.09;
p.R0_log = -8.1963;
p.k_dis_log = -13;
p.k_rdp_log = -11;
p.R0 = 10^(p.R0_log);
p.k_dis = 10^(p.k_dis_log);
p.k_rdp = 10^(p.k_rdp_log);
%%% geometry %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% Finite Volume Method with equidistant elements
p.r_max = 10.1e-9; % [m] maximum radius of PRD
p.n = 301; % number of elements of FVM
p.dr = p.r_max/p.n; % [m] length of elements, equidistant
p.r = linspace(5e-10,p.r_max,p.n+1)'; % [m] boundaries of elements
p.r_m = p.r(1:p.n)+p.dr/2; % [m] center of elements
%log normal initial distribution
S = 0.314;
M = 0.944;
p.x0 = 1./(S.*sqrt(2.*pi).*p.r_m*1e9).*exp((-(log(p.r_m*1e9)-M).^2)./(2.*S.^2));
p.r_squared = p.r_m.^2; % [m^2] squares of the radius (center of elements)
p.r_cubed = p.r_m.^3; % [m^3] cubes of the radius (center of elements)
p.MN_0 = trapz(p.r_m, p.r_cubed.*p.x0); % Eq. 2.11 denominator
p.M_0 = 4/3*pi*p.rho*p.MN_0;
p.I_V = 1; %ionomer volume fraction in the catalyst layer
After looking at both codes, the issue is that the odeint solver only takes 1D array inputs and your y0 is [int, array(300,)] and odeint can't work with that. However, you can merge the y0 into a 1D array and then split it up in the function you are integrating over to do the calculation then recombine as the output. Here's a working code of that:
import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
class P:
def __init__(self, S, M):
self.M = 195.084
self.rho = 21.45
self.m_V__M_Pt = (1*10**6)/self.M
self.Vm_Pt = 9.09
self.R0_log = -8.1963
self.k_dis_log = -13
self.k_rdp_log = -11
self.R0 = 10**(self.R0_log)
self.k_dis = 10**(self.k_dis_log)
self.k_rdp = 10**(self.k_rdp_log)
self.r_max = 10.1*10**(-9)
self.n = 301
self.dr = self.r_max / self.n
self.r = np.linspace(5*10**(-10), self.r_max, self.n)
self.r_m = self.r[0:self.n+1]+self.dr/2
self.x0 = self.compute_x0(S, M)
self.r_squared = np.power(self.r_m, 2)
self.r_cubed = np.power(self.r_m, 3)
self.MN_0 = np.trapz(self.r_m, np.multiply(self.r_cubed, self.x0))
self.M_0 = (4 / 3)* np.pi * self.rho * self.MN_0
self.I_V = 1
def compute_x0(self, S, M):
p1 = np.multiply(2, np.power(S, 2))
p2 = np.multiply(S, np.sqrt(np.multiply(2, np.pi)))
p3 = np.log(self.r_m*1*10**(9)) - M
p4 = np.multiply(p2, self.r_m*10**(9))
p5 = np.power(-p3, 2)
p6 = np.multiply(p4, np.exp(np.divide(p5,p1)))
p7 = np.divide(1, p6)
return p7
def cycling_parameters():
S = 0.314
M = 0.944
p = P(S, M)
return p
def f(y, t):
p = cycling_parameters()
c_pt = y[0]
r = np.delete(y, 0)
p1 = np.multiply(p.Vm_Pt, p.k_rdp)
p2 = np.multiply(p1, c_pt)
p3 = np.multiply(p.Vm_Pt, p.k_dis)
drdt = np.multiply(p2, np.exp(np.divide(-p.R0, p.r_m))) - np.multiply(p3, np.exp(np.divide(p.R0, p.r_m)))
dmdt = np.multiply(4*np.pi*p.rho*np.power(p.r_m, 2), drdt)
p4 = np.multiply(r, dmdt)
dMdt = np.trapz(p.r_m, p4)
dcdt = p.I_V*p.m_V__M_Pt*(-dMdt)/p.M_0
p5 = np.multiply(r, drdt)
dfdt = - np.gradient(p5,p.dr)
ans = np.insert(dfdt, 0, dcdt)
return ans
def modified_model():
p = cycling_parameters()
c_pt_0 = 0
y0 = np.insert(p.x0, 0, c_pt_0)
t = np.linspace(0, 30, 500)
ans = odeint(f, y0, t, rtol = 1e-3, atol = 1e-5)
r = ans[:, 1:p.n+1]
c_Pt = ans[:, 0]
print(r)
print(c_Pt)
plt.plot(p.r_m, r[0, :], color='r', linewidth=0.5)
plt.plot(p.r_m, r[r.shape[0]-1, :], color='b', linewidth=0.5)
plt.show()
if __name__ == '__main__':
modified_model()
Python plot (what this script outputs):
Original Matlab Plot:
This is a sample code I am running, which generates a matrix of dimension (size x size). The matrix is sent to FFT (over multiple iterations) and their norm is the required result. Since, this is a test run, I set the size = 256 and the iterations (zaxis) to be 3. It takes 1-2 mins per matrix to be processed presently.
Actual production run requires : matrix of 512 x 512, 1024 x 1024 (or maybe more) with about 25 iterations each, and I wonder if I can speed up this python script.
In short, I generate a complex matrix => assign the non-zero values element by element in a loop => send to FFT => compute norm => save norm in an array. It works fine !
The heavy work is performed in the following piece of code, where the non-zero values are computed as val. Here, 2D integral is computed separately for real and imaginary components. Ideally, I should be able to perform this on multiple cores. (*though I think if assignment of different non-zero matrix elements can be fully offloaded to multiple cores, it would be very efficient. I am not experienced in multiprocessing. System spec : 1700X AMD, 8 cores, 32GB RAM running Python3, Win 10; alternatively Ubuntu system also available with 12 cores, 64 GB RAM )
if (r < (dim/2) ):
c1 = fy(a,r,x,y)
c2 = r*complexAmp(a,b,x,y)
re = I_real ( r ) # double integral, real
im = I_imag ( r ) # double integral, imaginary
val=c1 * c2 *(re[0]+1j*im[0])
So, my question. Are there any good ways to improve speed of such operations (and hopefully I can learn more about efficient programming in python). For now I am checking ray and multiprocessing.
The following is the full input script. Output is shown in bottom.
import time
import math
import cmath as cm
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import dblquad
#-----------------------------------------------------------
# DEFINE FUNCTIONS
def fy(a,b,x,y):
return (a*b**3+(x/2.5)+y)/50
def complexAmp(a,b,x,y):
return ( (cm.exp(-1j*x*y*cm.sqrt(a+b)))/ cm.sqrt( a ) ) *b
def wrap(r, rho, phi):
return cm.cos(phi)*cm.exp(-1j*2*math.pi*cm.sqrt(rho**2 \
+ r**2))/cm.sqrt(rho**2 + r**2)
def wrap_real(r, rho, phi):
res = cm.cos(phi)*cm.exp(-1j*2*math.pi*cm.sqrt(rho**2 +\
r**2))/cm.sqrt(rho**2 + r**2)
return res.real
def wrap_imag(r, rho, phi):
res = cm.cos(phi)*cm.exp(-1j*2*math.pi*cm.sqrt(rho**2 + \
r**2))/cm.sqrt(rho**2 + r**2)
return res.imag
rMax = 5
def I_real (value ):
return dblquad(lambda rho, phi: wrap_real (value, rho, phi) \
, 0, rMax, lambda x: 0, lambda x: 2*math.pi)
def I_imag (value ):
return dblquad(lambda rho, phi: wrap_imag (value, rho, phi) ,\
0, rMax, lambda x: 0, lambda x: 2*math.pi)
#-----------------------------------------------------------
# TEST INTEGRATION
print("\n-----------COMPLEX INTEGRATION RESULT ----------")
print (I_real ( 6 ), I_imag ( 6 ))
print("--------------------------------------------------")
# parameters governing size and step of grid
size=256
depth=10
step=1.75 # step of grid
n2 = 1.45
theta = math.asin(1.00025/n2)
# complex matrix to keep data
inp = np.zeros((size,size) , dtype=complex )
zaxis = np.arange(-60, -10, 20)
result = np.zeros(zaxis.shape[0])
n2 = 1.454
theta = math.asin(1.00025/n2) # update theta
dim = 16000
# The main program -----------------------------------------------
for z in range(zaxis.shape[0]):
print ("In the loop {0}".format(z))
start = time.time()
for i in range(inp.shape[0]):
for j in range(inp.shape[1]):
x = step*(i-(size/2))
y = step*(j-(size/2))
r = x**2 + y**2
#print(i, (i-(size/2)),j, (j-(size/2)) )
b = r*( math.sin( 1.00025 /n2)) *math.sqrt(2*r**2)
a = 250/abs(zaxis[z]-r)
rMax = abs(zaxis[z])*math.tan(theta)
val=0
if (r < (dim/2) ):
c1 = fy(a,r,x,y)
c2 = r*complexAmp(a,b,x,y)
re = I_real ( r ) # double integral, real
im = I_imag ( r ) # double integral, imaginary
val=c1 * c2 *(re[0]+1j*im[0])
inp [i][j] = val # substitue the value to matrix
end = time.time()
print("Time taken : {0} sec \n" . format( round(end - start,7 )))
b = np.fft.fft2(inp)
result [z] = np.linalg.norm(b)
Output :
-----------COMPLEX INTEGRATION RESULT ----------
(-0.0003079405888916291, 1.0879642638692853e-17) (-0.0007321233659418995, 2.5866160149768244e-17)
--------------------------------------------------
In the loop 0
Time taken : 138.8842542 sec
[plot]
In the loop 1
Time taken : 134.3815458 sec
[plot]
In the loop 2
Time taken : 56.848331 sec
[plot]
[plot]
Using Ray I was able to achieve significant speed up for the above script. The double integral is now solved in a parallel way.
A comparison of time is below.
Time in seconds
+-------+-----------------+-------------------+
| loop | serial version | parallel with Ray |
+-------+-----------------+-------------------+
| 0 | 138.8 | 34.391 |
| 1 | 134.3 | 34.303 |
| 2 | 56.84 | 32.647 |
+-------+-----------------+-------------------+
Following is the updated script.
from sys import exit
import time
import math
import cmath as cm
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import dblquad
import ray
ray.init(num_cpus=6) # initializing ray here
#-----------------------------------------------------------
# DEFINE FUNCTIONS :
def fy(a,b,x,y):
return (a*b**3+(x/2.5)+y)/50
def complexAmp(a,b,x,y):
return ( (cm.exp(-1j*x*y*cm.sqrt(a+b)))/ cm.sqrt( a ) ) *b
def wrap(r, rho, phi):
return cm.cos(phi)*cm.exp(-1j*2*math.pi*cm.sqrt(rho**2 \
+ r**2))/cm.sqrt(rho**2 + r**2)
def wrap_real(r, rho, phi):
res = cm.cos(phi)*cm.exp(-1j*2*math.pi*cm.sqrt(rho**2 +\
r**2))/cm.sqrt(rho**2 + r**2)
return res.real
def wrap_imag(r, rho, phi):
res = cm.cos(phi)*cm.exp(-1j*2*math.pi*cm.sqrt(rho**2 + \
r**2))/cm.sqrt(rho**2 + r**2)
return res.imag
rMax = 5
def I_real (value ):
return dblquad(lambda rho, phi: wrap_real (value, rho, phi) \
, 0, rMax, lambda x: 0, lambda x: 2*math.pi)
def I_imag (value ):
return dblquad(lambda rho, phi: wrap_imag (value, rho, phi) ,\
0, rMax, lambda x: 0, lambda x: 2*math.pi)
############################################################
# DEFINE RAY FUNCTIONS
#ray.remote
def I_real_mod (value ):
out= dblquad(lambda rho, phi: wrap_real (value, rho, phi) \
, 0, rMax, lambda x: 0, lambda x: 2*math.pi)
return out[0]
#-----------------------------------------------------------
#ray.remote
def I_imag_mod (value ):
out= dblquad(lambda rho, phi: wrap_imag (value, rho, phi) ,\
0, rMax, lambda x: 0, lambda x: 2*math.pi)
return out[0]
#-----------------------------------------------------------
#ray.remote
def compute_integral( v ):
i1 = I_real_mod.remote( v )
i2 = I_imag_mod.remote( v )
result_all = ray.get([i1, i2])
return (result_all[0]+1j*result_all[1])
#-----------------------------------------------------------
# TEST INTEGRATION
print("\n-----------COMPLEX INTEGRATION RESULT : SERIAL ----------")
print (I_real ( 6 ), I_imag ( 6 ))
print("--------------------------------------------------")
print("\n-----------COMPLEX INTEGRATION RESULT : PARALLEL with RAY ----------")
v1=compute_integral.remote( 6 )
print(ray.get(v1))
print("--------------------------------------------------")
#exit(0)
# parameters governing size and step of grid
size=256
depth=10
step=1.75 # step of grid
n2 = 1.45
theta = math.asin(1.00025/n2)
# complex matrix to keep data
inp = np.zeros((size,size) , dtype=complex )
zaxis = np.arange(-60, -10, 20)
result = np.zeros(zaxis.shape[0])
n2 = 1.454
theta = math.asin(1.00025/n2)
dim = 16000
# The main program -----------------------------------------------
for z in range(zaxis.shape[0]):
print ("In the loop {0}".format(z))
start = time.time()
for i in range(inp.shape[0]):
for j in range(inp.shape[1]):
x = step*(i-(size/2))
y = step*(j-(size/2))
r = x**2 + y**2
#print(i, (i-(size/2)),j, (j-(size/2)) )
b = r*( math.sin( 1.00025 /n2)) *math.sqrt(2*r**2)
a = 250/abs(zaxis[z]-r)
rMax = abs(zaxis[z])*math.tan(theta)
val=0
if (r < (dim/2) ):
c1 = fy(a,r,x,y)
c2 = r*complexAmp(a,b,x,y)
o1 = compute_integral.remote( r ) # using RAY decorated integral here
val=c1 * c2 *(ray.get(o1))
inp [i][j] = val # substitue the value to matrix
end = time.time()
print("Time taken : {0} sec \n" . format( round(end - start,7 )))
b = np.fft.fft2(inp)
result [z] = np.linalg.norm(b)
#----------------------------------------------------------
I'm trying to get the coefficients of a function p(T,x). I provided the data for p, T and x from excel sheets via panda. The following Code works quite nice for me:
import pandas as pd
import os
from scipy.optimize import curve_fit
import numpy as np
df = pd.read_excel(os.path.join(os.path.dirname(__file__), "./Data.xlsx"))
T = np.array(df['T'], dtype=float)
x = np.array(df['x'], dtype=float)
p = np.array(df['p'], dtype=float)
p_s = 67.17
def func(X, a, b, c, d, e, f):
T, x = X
return x * p_s + x * (1 - x) * (a + b * T + c * T ** 2 + d * x + e * x * T + f * x * T ** 2) * p_s
popt, pcov = curve_fit(func, (T, x), p)
print("a = %s , b = %s, c = %s, d = %s, e = %s, f = %s" % (popt[0], popt[1], popt[2], popt[3], popt[4], popt[5]))
My acutal problem is, that the function is swinging a little bit in the end.
Because of this behaviour i get two x values for one p value, which i dont want.
So to avoid this little swing i want to accomplish a boundary condition for the fitting that say something like dp/dx (for constant T) > 0. With dp/dx I mean the derivation of the function after x.
Is this possible with the normal bound paramter of curve_fit? How can I do this?
EDIT:
As suggested I've messed a little bit around with least_square function but I guess that I came to a point where I don't realy understand what I'm doing or have to do.
T = np.array(df['T'], dtype=float)
x = np.array(df['x'], dtype=float)
p = np.array(df['p'], dtype=float)
p_s = 67
def f(X, z):
T, x = X
return x * p_s + x * (1 - x) * (z[0] + z[1] * T + z[2] * T ** 2 + z[3] * x + z[4] * x * T + z[5] * x * T ** 2) * p_s
def g(X,p,z):
return p - f(X ,z)
z0 = np.array([0,0,0,0,0,0], dtype=float)
res, flag = least_squares(g,z0, args=(T,x,p))
print(res)
With this code I get the following error:
TypeError: g() takes 3 positional arguments but 4 were given
Here is an algorithm of Exponential time differencing method, using the original
Matlab code from Oxford
clc
clear
% viscosity
nu = 0.5;
% Spatial grid and initial condition:
N = 128;
x = 2*pi*(0:N-1)'/N;
u = cos(x).*(1+sin(x));
v = fft(u);
% Precompute various ETDRK4 scalar quantities:
h = 0.01;
tot = 5;
% time step
k = [0:N/2-1 0 -N/2+1:-1]';
% wave numbers
L = 1/16*k.^2 - 1/16^3*nu*k.^4;
% Fourier multipliers
E = exp(h*L);
E2 = exp(h*L/2);
M =16;
% no. of points for complex means
r = exp(1i*pi*((1:M)-.5)/M); % roots of unity
% construct things on the
LR = h*L(:,ones(M,1)) + r(ones(N,1),:);
Q = h*real(mean( (exp(LR/2)-1)./LR ,2) );
f1 = h*real(mean( (-4-LR+exp(LR).*(4-3*LR+LR.^2))./LR.^3 ,2));
f2 = h*real(mean( (2+LR+exp(LR).*(-2+LR))./LR.^3 ,2));
f3 = h*real(mean( (-4-3*LR-LR.^2+exp(LR).*(4-LR))./LR.^3 ,2));
% Main time-stepping loop:
uu=u;tt=0;
vv=v;
NvNV = [];
aa = [];
bb = [];
cc = [];
NvNv = [];
NaNa = [];
NbNb = [];
NcNc = [];
nmax = floor(tot/h);
g = -0.5i*k;
%
for n = 1:nmax
t = n*h;
Nv = g.*fft(real(ifft(v)).^2);
a = E2.*v + Q.*Nv;
Na = g.*fft(real(ifft(a)).^2);
b = E2.*v + Q.*Na;
Nb = g.*fft(real(ifft(b)).^2);
c = E2.*a + Q.*(2*Nb-Nv);
Nc = g.*fft(real(ifft(c)).^2);
v = E.*v + Nv.*f1 + 2*(Na+Nb).*f2 + Nc.*f3;
u = real(ifft(v));
uu = [uu,u];
vv = [vv,v];
tt = [tt,t];
aa = [aa,a];
bb = [bb,b];
cc = [cc,c];
NvNv = [NvNv,Nv];
NaNa = [NaNa,Na];
NbNb = [NbNb,Nb];
NcNc = [NcNc,Nc];
end
Python code
# spatial domain: 0,2pi
# time domain: 0,2
# init
import numpy as np
from matplotlib import pyplot as plt
np.seterr(all='raise')
plt.style.use('siads')
np.random.seed(0)
pi = np.pi
# viscosity
nu = 0.5
# mesh
mesh = 128
# time restriction
tot = 5
# distributed x point
x = np.linspace(0, 2.0 * pi, mesh, endpoint=False)
# force IC
u0 = np.cos(x)*(1.0 + np.sin(x))
N = mesh
k_array_noshift = np.fft.fftfreq(mesh)*mesh
u = u0
v = np.fft.fft(u)
h = 0.01
## follow them, set -N/2 to zer
k_array_noshift[mesh / 2] = 0
## Linear part in fourier space
L = 1.0/16.0*k_array_noshift**2 - nu*1.0/16.0**3 * k_array_noshift**4
## Fourier mulitplier
E = np.exp(h * L)
E2 = np.exp(h * L / 2.0)
## select number of points on the circle
M = 16
## choose radius 1, since k^2-k^4 ranges a lot... so 1 is enough to make sure only one singular point
r = np.exp(1j * np.pi * (np.arange(1, M + 1) - 0.5) / M)
r = r.reshape(1, -1)
r_on_circle = np.repeat(r, mesh, axis=0)
## define hL on M copies
LR = h * L
LR = LR.reshape(-1, 1)
LR = np.repeat(LR, M, axis=1)
## obtain hL on circle
LR = LR + r_on_circle
## important quantites used in ETDRK4
# g = -0.5*i*k
g = -0.5j * k_array_noshift
# averaged Q, f1,f2,f3
Q = h*np.real(np.mean( (np.exp(LR/2.0)-1)/LR, axis=1 ))
f1 = h*np.real(np.mean( (-4.0-LR + np.exp(LR)*(4.0-3.0*LR+LR**2))/LR**3, axis=1 ))
f2 = h*np.real(np.mean( (2.0+LR + np.exp(LR)*(-2.0 + LR))/LR**3, axis=1 ))
f3 = h*np.real(np.mean( (-4.0-3.0*LR - LR**2 + np.exp(LR)*(4.0 - LR))/LR**3, axis=1 ))
def compute_u2k_zeropad_dealiased(uk_):
u2k = np.fft.fft(np.real(np.fft.ifft(uk_))**2)
# three over two law
# N = uk.size
#
# # map uk to uk_fine
#
# uk_fine = np.hstack((uk[0:N / 2], np.zeros((N / 2)), uk[-N / 2:])) * 3.0 / 2.0
#
# # convert uk_fine to physical space
# u_fine = np.real(np.fft.ifft(uk_fine))
#
# # compute square
# u2_fine = np.square(u_fine)
#
# # compute fft on u2_fine
# u2k_fine = np.fft.fft(u2_fine)
#
# # convert u2k_fine to u2k
# u2k = np.hstack((u2k_fine[0:N / 2], u2k_fine[-N / 2:])) / 3.0 * 2.0
return u2k
print 'dt =', h
# np.linalg.norm(np.fft.ifft(uk_0)-u0) # very good
ntsnap = int(tot/h)
isnap = 0
tsnap = np.linspace(0, tot, ntsnap)
usnap = np.zeros((mesh, ntsnap))
uksnap = np.zeros((mesh, ntsnap),dtype=complex)
aasnap = np.zeros((mesh, ntsnap),dtype=complex)
bbsnap = np.zeros((mesh, ntsnap),dtype=complex)
ccsnap = np.zeros((mesh, ntsnap),dtype=complex)
Nvsnap = np.zeros((mesh, ntsnap),dtype=complex)
Nasnap = np.zeros((mesh, ntsnap),dtype=complex)
Nbsnap = np.zeros((mesh, ntsnap),dtype=complex)
Ncsnap = np.zeros((mesh, ntsnap),dtype=complex)
tcur = 0.0
## main loop time stepping
while tcur <= tot and isnap < ntsnap:
# print tcur
# record snap
# if abs(tcur - tsnap[isnap]) < 1e-2:
print ' current progress =', tcur / tsnap[-1] * 100, ' % '
u = np.real(np.fft.ifft(v))
usnap[:, isnap] = u
uksnap[:, isnap] = v
Nv = g * np.fft.fft(np.real(np.fft.ifft(v))**2)
a = E2 * v + Q * Nv
Na = g * np.fft.fft(np.real(np.fft.ifft(a))**2)
b = E2 * v + Q * Na
Nb = g * np.fft.fft(np.real(np.fft.ifft(b))**2)
c = E2 * a + Q * (2.0*Nb - Nv)
Nc = g * np.fft.fft(np.real(np.fft.ifft(c))**2)
v = E*v + Nv*f1 + 2.0*(Na + Nb)*f2 + Nc*f3
aasnap[:, isnap] = a
bbsnap[:, isnap] = b
ccsnap[:, isnap] = c
Nvsnap[:, isnap] = Nv
Nasnap[:, isnap] = Na
Nbsnap[:, isnap] = Nb
Ncsnap[:, isnap] = Nc
# algo: ETDRK4
# # 1. compute nonlinear part in first fractional step
# u2k = compute_u2k_zeropad_dealiased(uk)
# Nuk = g * u2k
#
# # 2. update fractional step
# uk_a = E2 * uk + Q * Nuk
#
# # 3. compute nonlinear part in second fractional step
# Nuk_a = g * compute_u2k_zeropad_dealiased(uk_a)
#
# # 4. update fractional step
# uk_b = E2 * uk + Q * Nuk_a
#
# # 5. compute nonlinear part in third fractional step
# Nuk_b = g * compute_u2k_zeropad_dealiased(uk_b)
#
# # 6. update fractional step
# uk_c = E2 * uk_a + Q * (2.0 * Nuk_b - Nuk)
#
# # 7 compute nonlinear part in the fourth fractional step
# Nuk_c = g * compute_u2k_zeropad_dealiased(uk_c)
#
# # final, update uk
# uk = E * uk + Nuk * f1 + 2.0 * (Nuk_a + Nuk_b) * f2 + Nuk_c * f3
# record time
tcur = tcur + h
isnap = isnap + 1
# save uksnap
np.savez('output_uk',uksnap=uksnap,f1=f1,f2=f2,f3=f3,Q=Q,LR=LR,L=L,g=g, Nasnap=Nasnap, Nvsnap=Nvsnap, Nbsnap=Nbsnap, Ncsnap=Ncsnap,
aasnap=aasnap, bbsnap=bbsnap, ccsnap=ccsnap, usnap=usnap)
# plot snapshots
plt.figure(figsize=(16,16))
for isnap in xrange(ntsnap):
plt.plot(x, usnap[:,isnap])
plt.xlabel('x')
plt.ylabel('u')
plt.savefig('snapshots.png')
# plot contours
from matplotlib import cm
fig = plt.figure(figsize=(8, 8))
X, Y = np.meshgrid(x, tsnap)
Z = usnap.transpose()
V = np.linspace(-10, 10, 500)
surf = plt.contourf(X, Y, Z, V, cmap=cm.seismic)
plt.savefig('contour.png')
Python code runs to some extent says some value goes to NaN, but not for matlab. Same code, exactly
So I wish to know why and what's going on, you can run the code yourself.
What makes me feel more weird is that the first iteration is extremely good match between the two!
For more information, please check out this linkedIn link.
My LinkedIn Post that explains this problem
Finally!!!!!!!! Everything works!!!!!!!!!!!!!!!!!
I found I can make python works the same well as matlab(note that python always blows up to inf at certain physical time), by simply
changing numpy.fft to scipy.fft
Note that I tested, numpy.fft and scipy.fft evaluated in my code is almost identical but different around 1e-16..
But this matters for such a chaotic system simulation.
If I want my code to blow up, simply change scipy.fft to numpy.fft.
From my personal viewpoint as an user, numpy.fft must have some mysterious issue inside! Since matlab fft and scipy fft have no trouble in my code at all.