I'm trying to solve the system:
dP/dt = AP
where,
A is the matrix [ [1,2], [3,4]]
P is the the matrix [ [P1, P2], [P3, P4] ]
with initial condition P0 = [P10, P20, P30, P40] = [1,2,3,5]
I've implemented two different methods in python but I'm getting different answers.
The first is to split the system into 2 pairs of coupled ODEs and then find the eigenvalues and eigenvectors of A. These can then be substituted into a general solution:
Below is some of the code focusing on obtaining a solution for P1.
A = np.array([[1,2],[3,4]])
# extract eigenvalues
lambda1, lambda2, = np.linalg.eig(A)[0]
# extract eigenvectors
x1, x2 = np.linalg.eig(A)[1]
# combine eigenvectors and solve system at t=0 with initial condition to get constants
x1 = x1.reshape(2,1)
x2 = x2.reshape(2,1)
X = np.concatenate((x1,x2),axis=1)
P10 = 1
P30 = 3
Pinitial = np.array([P10, P30])
C1, C2 = np.linalg.solve(X, Pinitial)
# combine everything to get the solution to P1
t = np.linspace(0,3)
P1true = C1 * np.exp(lambda1 * t) * x1[0] + C2 * np.exp(lambda2 * t) * x2[0]
print(P1true)
My output to this implementation is the following:
[ 1.00000000e+00 4.90610966e-01 -1.96909212e-01 -1.13239194e+00
-2.41285490e+00 -4.17309019e+00 -6.60037624e+00 -9.95491931e+00
-1.45982562e+01 -2.10327188e+01 -2.99562678e+01 -4.23386834e+01
-5.95274276e+01 -8.33947363e+01 -1.16541999e+02 -1.62583733e+02
-2.26542159e+02 -3.15395444e+02 -4.38839463e+02 -6.10346242e+02
-8.48634612e+02 -1.17971363e+03 -1.63972185e+03 -2.27887232e+03
-3.16693407e+03 -4.40084834e+03 -6.11531116e+03 -8.49747721e+03
-1.18073904e+04 -1.64063715e+04 -2.27964611e+04 -3.16752246e+04
-4.40119010e+04 -6.11532094e+04 -8.49703621e+04 -1.18063333e+05
-1.64044684e+05 -2.27933923e+05 -3.16705455e+05 -4.40049940e+05
-6.11432160e+05 -8.49560890e+05 -1.18043123e+06 -1.64016232e+06
-2.27894028e+06 -3.16649669e+06 -4.39972081e+06 -6.11323640e+06
-8.49409776e+06 -1.18022094e+07]
My second implementation is to use scipy's odeint:
# function that defines dP/dt
def model(P,t):
A = np.array([[1,2], [3,4]])
P = np.array([[P[0], P[1]],[P[2],P[3]]])
RHS = np.matmul(A,P)
RHS = RHS.reshape(1,4)
dPdt = RHS.tolist()
return dPdt[0]
# initial condition
P0 = [1,2,3,5]
#time points
t = np.linspace(0,3)
#solve model
P = odeint(model,P0,t)
# print P1 solution
print(P[:,0])
For which I get the following output:
[1.00000000e+00 1.50619854e+00 2.20691051e+00 3.17795032e+00
4.52465783e+00 6.39339726e+00 8.98753443e+00 1.25896370e+01
1.75923199e+01 2.45411052e+01 3.41939721e+01 4.76041020e+01
6.62348466e+01 9.21194739e+01 1.28083127e+02 1.78051229e+02
2.47477996e+02 3.43941844e+02 4.77972676e+02 6.64201371e+02
9.22956950e+02 1.28248577e+03 1.78203503e+03 2.47613711e+03
3.44056260e+03 4.78059170e+03 6.64250701e+03 9.22956240e+03
1.28241709e+04 1.78187343e+04 2.47584791e+04 3.44009754e+04
4.77988372e+04 6.64146291e+04 9.22805262e+04 1.28220154e+05
1.78156830e+05 2.47541841e+05 3.43949539e+05 4.77904179e+05
6.64028793e+05 9.22641500e+05 1.28197351e+06 1.78125097e+06
2.47497704e+06 3.43888166e+06 4.77818858e+06 6.63910199e+06
9.22476676e+06 1.28174445e+07]
Which isn't the same as my first implementation? Can anyone see where I've gone wrong? I feel like the problem might be with my second implementation?
Related
I am trying to use scipy's solve_bvp in python to solve differential equations that depend on a known parameter that changes over time. I have this parameter saved in a numpy array. However, when I try to use this array in the derivatives function, I get the following error ValueError: operands could not be broadcast together with shapes (10,) (11,).
Below is a simplified version of my code. I want the variable d2 to take certain values at different times according to an array, d2_set_values. The differential equations for some of the 12 variables then depend on d2. I hope it's clear from this code what I'm trying to achieve.
import numpy as np
from scipy.integrate import solve_bvp
t = np.linspace(0, 10, 11)
# Known parameter that changes over time
d2_set_values = np.zeros(t.size)
d2_set_values[:4] = 0.1
d2_set_values[4:8] = 0.2
d2_set_values[8:] = 0.1
# Initialise y vector
y = np.zeros((12, t.size))
# ODEs
def fun(x, y):
S1, I1, R1, S2, I2, R2, lamS1, lamI1, lamR1, lamS2, lamI2, lamR2 = y
d1 = 0.5*(I1 + 0.1*I2)*(lamS1 - lamI1)
d2 = d2_set_values
dS1dt = -0.5*S1*(1-d1)*(I1 + 0.1*I2)
dS2dt = -0.5*S2*(1-d2)*(I2 + 0.1*I1)
dI1dt = 0.5*S1*(1-d1)*(I1 + 0.1*I2) - 0.2*I1
dI2dt = 0.5*S2*(1-d2)*(I2 + 0.1*I1) - 0.2*I2
dR1dt = 0.2*I1
dR2dt = 0.2*I2
dlamS1dt = 0.5*(1-d1)*S1*lamS1
dlamS2dt = 0.5*(1-d2)*S2*lamS2
dlamI1dt = 0.5*(1-d1)*I1*lamI1
dlamI2dt = 0.5*(1-d2)*I2*lamI2
dlamR1dt = lamR1
dlamR2dt = lamR2
return np.vstack((dS1dt, dI1dt, dR1dt, dS2dt, dI2dt, dR2dt, dlamS1dt, dlamI1dt, dlamR1dt, dlamS2dt, dlamI2dt, dlamR2dt))
# Boundary conditions
def bc(ya, yb):
return np.array([ya[0]-0.99, ya[1]-0.01, ya[2]-0., ya[3]-1.0, ya[4]-0., ya[5]-0.,
yb[6]-0., yb[7]-1., yb[8]-0., yb[9]-0, yb[10]-0, yb[11]-0])
# Run the solver
sol = solve_bvp(fun, bc, t, y)
I have even tried reducing the size of d2_set_values by one, but that doesn't solve the issue.
Any help I can get would be much appreciated!
When running the ODR algorithm on some experiment data, I've been asked to run it with the following model:
It is clear that this fitting function is containing a redundant degree of freedom.
When I run the fitting on my experiment data I get enormous relative errors of beta, starting from 8000% relative error.
When I try to run the fitting again but with a fitting function that doesn't have a redundant degree of freedom, such as:
I don't get this kind of problem.
Why is this happening? Why the ODR algorithm is so sensitive for redundant degrees of freedom? I wasn't able to answer these questions to my supervisors. An answer will be much appreciated.
Reproducing code example:
from scipy.odr import RealData, Model, ODR
def func1(a, x):
return a[0] * (x + a[1]) / (a[3] * (x + a[1]) + a[1] * x) + a[2]
def func2(a, x):
return a[0] / (x + a[1]) + a[2]
# fmt: off
zx = [
1911.125, 2216.95, 2707.71, 3010.225, 3410.612, 3906.015, 4575.105, 5517.548,
6918.481,
]
dx = [
0.291112577, 0.321695254, 0.370771197, 0.401026507, 0.441068641, 0.490601621,
0.557573268, 0.651755155, 0.79184836,
]
zy = [
0.000998056, 0.000905647, 0.000800098, 0.000751041, 0.000699982, 0.000650532,
0.000600444, 0.000550005, 0.000500201,
]
dy = [
5.49029e-07, 5.02824e-07, 4.5005e-07, 4.25532e-07, 3.99991e-07, 3.75266e-07,
3.50222e-07, 3.25003e-07, 3.00101e-07,
]
# fmt: on
data = RealData(x=zx, y=zy, sx=dx, sy=dy)
print("Func 1")
print("======")
beta01 = [
1.46,
4775.4,
0.01,
1000,
]
model1 = Model(func1)
odr1 = ODR(data, model1, beta0=beta01)
result1 = odr1.run()
print("beta", result1.beta)
print("sd beta", result1.sd_beta)
print("relative", result1.sd_beta / result1.beta * 100)
print()
print()
print("Func 2")
print("======")
beta02 = [
1,
1,
1,
]
model2 = Model(func2)
odr2 = ODR(data, model2, beta0=beta02)
result2 = odr2.run()
print("beta", result2.beta)
print("sd beta", result2.sd_beta)
print("relative", result2.sd_beta / result2.beta * 100)
This prints out:
Func 1
======
beta [ 1.30884537e+00 -2.82585952e+03 7.79755196e-04 9.47943376e+01]
sd beta [1.16144608e+02 3.73765816e+06 6.12613738e-01 4.20775596e+03]
relative [ 8873.82193523 -132266.24068473 78564.88054498 4438.82627453]
Func 2
======
beta [1.40128121e+00 9.80844274e+01 3.00511669e-04]
sd beta [2.73990552e-03 3.22344713e+00 3.74538794e-07]
relative [0.1955286 3.28640051 0.12463369]
Scipy/Numpy/Python version information:
Versions are:
Scipy - 1.4.1
Numpy - 1.18.2
Python - 3.7.2
The problem is not with the degrees of freedom.
The degrees of freedom is the difference between the number of data points and the number of fitting parameters.
The problem has the same number of degrees of freedom for the two formulae, as they have the same number of parameters.
It also looks like that you do not have free degrees of freedom, which is good news, it means that it can potentially be fitted.
However, you are right that first expression has some problem: the parameters you are trying to fit are not independent.
This is probably better understood with some simpler example.
Consider the following expression:
y = x + b + c
which you try to fit, given n data for x and y with n >> 2.
The question is: what are the optimal value for b and c? This cannot be answered. All you can say from x and y data is about the combination. Therefore, if b + c is 0, the fit cannot tell us if b = 1000, c = -1000 or b = 1, c= -1, but at least we can say that given b we can determine c.
What is the error on a given b? Potentially infinite. That is the reason for the fitting to give you that large relative error.
My question is that how to write theese equations in array and solve?
from scipy import linalg
import numpy as np
import matplotlib.pyplot as plt
x = np.array[-23,1100,2300],[2300,1500,550],[550,1600,]
I tried to write in the array above, but I couldn't figure out how to replace 'In' and 'Vs2' in the question. Can you help me solve the question?
You want to solve these equations for several voltages, which suggests the use of a for-loop. For clarity, it's usually better to use identifiers for values, thus for instance, R1 rather than 1100. Put the R1 in formulae and let the computer do the simple arithmetic for you.
You may be thinking of using the linalg solve function since you need to solve a square matrix of order three. The unknowns are the currents. Therefore, do the algebra so that you have expressions for the coefficients of the matrix, and for the right side of the equation, in terms of resistances and voltages.
For the matrix (as indicated in the documentation at https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.solve.html#scipy.linalg.solve),
a = np.array([[f1(Rs, Vs), f2(Rs, Vs), f3(Rs, Vs)], [...], [...]])
For the vector on the right side,
b = np.array([f4(Rs, Vs), f5(Rs,Vs), f6(Rs, Vs)])
Then currents = solve(a, b)
Notice that f1, f2, etc are those functions that you have to calculate algebraically.
Now put this code in a loop, more or less like this:
for vs2 in [10,15,20,25]:
currents = solve(a, b)
Because you've got the resistances and vs2's in your algebraic expressions you'll get the corresponding currents. You'll need to collect the currents corresponding to voltages for plotting.
Addition: Partial result of algebraic manipulation:
More: How I would avoid most of the pesky algebra using the sympy library:
>>> R1, R2, R3, R4, R5, Vs1 = 1100, 2300, 1500, 550, 1600, 23
>>> from sympy import *
>>> var('I1,I2,I3,Vs2')
(I1, I2, I3, Vs2)
>>> eq1 = -Vs1 + R1*I1 + R2 * (I1-I2)
>>> eq1
3400*I1 - 2300*I2 - 23
>>> eq2 = R2*(I2-I1)+R3*I2+R4*(I2-I3)
>>> eq2
-2300*I1 + 4350*I2 - 550*I3
>>> eq3 = R4*(I3-I2)+R5*I3 + Vs2
>>> eq3
-550*I2 + 2150*I3 + Vs2
>>> from scipy import linalg
>>> import numpy as np
>>> for Vs2 in [10,15,20,25]:
... ls = np.array([[3400,-2300,0],[-2300,4350,-550],[0,-550,2150]])
... rs = np.array([23, 0, -Vs2])
... I = linalg.solve(ls, rs)
... Vs2, I
...
(10, array([ 0.01007914, 0.0048996 , -0.00339778]))
(15, array([ 0.00975305, 0.00441755, -0.00584667]))
(20, array([ 0.00942696, 0.0039355 , -0.00829557]))
(25, array([ 0.00910087, 0.00345346, -0.01074446]))
In order to solve a linear system of equations for unknown vector x=In which is classically written as Ax=b, you need to specify a coefficient matrix A and right hand side vector b to linalg.solve function. Based on your question, you just have to re-write in matrix form your three equations in terms of unknown currents to get A and b which was done with sympy but it is pretty overkill here imo. Here follows an easier to read solution with analytic A:
from scipy.linalg import lu_factor, lu_solve
import numpy as np
import matplotlib.pyplot as plt
# your data
R1 = 1100
R2 = 2300
R3 = 1500
R4 = 550
R5 = 1600
Vs1 = 23
# Vs2 range of interest as a list
Vs2_range = [10,15,20,25]
# construct A: the coefficient matrix of the left-hand side in terms of In = [I1, I2, I3]
A = np.array([[ R1+R2, -R2, 0],
[ -R2, R2+R3+R4, -R4],
[ 0, -R4, R4+R5]])
# pre-compute pivoted LU decomposition of A to solve Ax=b (because only b is changing here)
A_LU = lu_factor(A)
# initialize results
res = np.empty((len(Vs2_range),3))
# loop over Vs2 values
for i,Vs2 in enumerate(Vs2_range):
# construct b: the right hand side vector for each Vs2
b = np.array([Vs1,0,-Vs2])
# then solve the linear system Ax=b
In = lu_solve(A_LU,b)
# stock results as rows of the res array
res[i,:] = In
# plot each current In (column of res) vs Vs2_range
for i in range(3):
plt.plot(Vs2_range,res[:,i],'-+',label='I'+str(i+1))
plt.xlabel('Vs2 [V]')
plt.ylabel('I [A]')
plt.legend()
which gives:
Hope this helps.
In order to use PyStruct to perform image segmentation (by means of inference [1]), I first need to build a graph whose nodes correspond to pixels and edges are the link between these pixels.
I have thus written a function, which works, to do so:
def create_graph_for_pystruct(mrf, betas, nb_labels):
M, N = mrf.shape
b1, b2, b3, b4 = betas
edges = []
pairwise = np.zeros((nb_labels, nb_labels))
# loop over rows
for i in range(M):
# loop over columns
for j in range(N):
# get rid of pixels belonging to image's borders
if i!=0 and i!=M-1 and j!=0 and j!=N-1:
# get the current linear index
current_linear_ind = i * N + j
# retrieve its neighborhood (yield a list of tuple (row, col))
neigh = np.array(getNeighborhood(i, j, M, N))
# convert neighbors indices to linear ones
neigh_linear_ind = neigh[:, 0] * N + neigh[:, 1]
# add edges
[edges.append((current_linear_ind, n)) for n in neigh_linear_ind]
mat1 = b1 * np.eye(nb_labels)
mat2 = b2 * np.eye(nb_labels)
mat3 = b3 * np.eye(nb_labels)
mat4 = b4 * np.eye(nb_labels)
pairwise = np.ma.dstack((pairwise, mat1, mat1, mat2, mat2, mat3, mat3, mat4, mat4))
return np.array(edges), pairwise[:, :, 1:]
However, it is slow and I am wondering where I can improve my function in order to speed it up.
[1] https://pystruct.github.io/generated/pystruct.inference.inference_dispatch.html
Here is a code suggestion that should run much faster (in numpy one should focus on using vectorisation against for-loops). I try to build the whole output in a single pass using vectorisation, I used the helpfull np.ogrid to generate xy coordinates.
def new(mrf, betas, nb_labels):
M, N = mrf.shape
b1, b2, b3, b4 = betas
mat1,mat2,mat3,mat4 = np.array([b1,b2,b3,b4])[:,None,None]*np.eye(nb_labels)[None,:,:]
pairwise = np.array([mat1, mat1, mat2, mat2, mat3, mat3, mat4, mat4]*((M-2)*(N-2))).transpose()
m,n=np.ogrid[0:M,0:N]
a,b,c= m[0:-2]*N+n[:,0:-2],m[1:-1]*N+n[:,0:-2],m[2: ]*N+n[:,0:-2]
d,e,f= m[0:-2]*N+n[:,1:-1],m[1:-1]*N+n[:,1:-1],m[2: ]*N+n[:,1:-1]
g,h,i= m[0:-2]*N+n[:,2: ],m[1:-1]*N+n[:,2: ],m[2: ]*N+n[:,2: ]
center_index = e
edges_index = np.stack([a,b,c,d,f,g,h,i])
edges=np.empty(list(edges_index.shape)+[2])
edges[:,:,:,0]= center_index[None,:,:]
edges[:,:,:,1]= edges_index
edges=edges.reshape(-1,2)
return edges,pairwise
Timing and comparison test :
import timeit
args=(np.empty((40,50)), [1,2,3,4], 10)
f1=lambda : new(*args)
f2=lambda : create_graph_for_pystruct(*args)
edges1, pairwise1 = f1()
edges2, pairwise2 = f2()
#outputs are not exactly indentical: the order isn't the the same
#I sort both to compare the results
edges1 = edges1[np.lexsort(np.fliplr(edges1).T)]
edges2 = edges2[np.lexsort(np.fliplr(edges2).T)]
print("edges identical ?",(edges1 == edges2).all())
print("pairwise identical ?",(pairwise1 == pairwise2).all())
print("new : ",timeit.timeit(f1,number=1))
print("old : ",timeit.timeit(f2,number=1))
Output :
edges identical ? True
pairwise identical ? True
new : 0.015270026000507642
old : 4.611805051001284
Note: I had to guess what was in the getNeighborhood function
After running a multiple linear regression using numpy.linalg.lstsq I get 4 arrays as described in the documentation, however it is not clear to me how do I get the intercept value. Does anyone know this? I'm new to statistical analysis.
Here is my model:
X1 = np.array(a)
X2 = np.array(b)
X3 = np.array(c)
X4 = np.array(d)
X5 = np.array(e)
X6 = np.array(f)
X1l = np.log(X1)
X2l = np.log(X2)
X3l = np.log(X3)
X6l = np.log(X6)
Y = np.array(g)
A = np.column_stack([X1l, X2l, X3l, X4, X5, X6l, np.ones(len(a), float)])
result = np.linalg.lstsq(A, Y)
This is a sample of what my model is generating:
(array([ 654.12744154, -623.28893569, 276.50269246, 11.52493817,
49.92528734, -375.43282832, 3852.95023087]), array([ 4.80339071e+11]),
7, array([ 1060.38693842, 494.69470547, 243.14700033, 164.97697748,
58.58072929, 19.30593045, 13.35948642]))
I believe the intercept is the second array, still I'm not sure about that, as its value is just too high.
The intersect is the coefficient that corresponds to the column of ones, which in this case is:
result[0][6]
To make it clearer to see, consider your regression, which is something like:
y = c1*x1 + c2*x2 + c3*x3 + c4*x4 + m
written in matrix form as:
[[y1], [[x1_1, x2_1, x3_1, x4_1, 1], [[c1],
[y2], [x1_2, x2_2, x3_2, x4_2, 1], [c2],
[y3], = [x1_3, x2_3, x3_3, x4_3, 1], * [c3],
... ... [c4],
[yn]] [x1_n, x2_n, x3_n, x4_n, 1]] [m]]
or:
Y = A * C
where A is the so called "Coefficient' matrix and C the vector containing the solution for your regression. Note that m corresponds to the column of ones.