Develop Reference

scipy.integrate.odeint update initial conditions every timestep

I´m using scipy.integrate.odeint to solve the equations of motion of a given system with a script from where I selected the most relevant part to this specific problem:
# Equations of Motion function to integrate
def solveEquationsofMotion(y0, t, nRigidBodies, nCoordinates, nConstraintsByType, dataConst, Phi, dPhidq, niu, gamma, massMatrix, gVector, alpha, beta, sda_Parameters):
...
Some calculations
matA = numpy.array
...
dydt = np.hstack((qp,qpp))
return dydt
#Integrator results
solution = odeint(solveEquationsofMotion, y0, time_span,args=(nRigidBodies, nCoordinates, nConstraintsByType, dataConst, Phi, dPhidq, niu, gamma, massMatrix, gVector, alpha, beta), full_output=0)
and it works fine.
However now I need to multiply part of the integration result (solution variable) by matA variable in each timestep to use again as the initial conditions for the next timestep.
I've looked in the scipy.integrate.odeint documentation but I haven't seen any relevant information.
Any help would be very much appreciated.
Kind Regards
Ivo

If you have to change the solution at every step, it is more logical to use the step-by-step integrator ode. It is supposed to be used in a loop anyway, so one may as well change the conditions meanwhile. Here is an example of solving y' = -sqrt(t)*y (vector valued) where y is multiplied by matA after every step.
The steps in t variable are determined by the array t. The main step is y[k, :] = r.integrate(t[k]) which gets the next value of solution, and then the initial condition is changed by r.set_initial_value(matA.dot(y[k, :]), t[k]).
import numpy as np
from scipy.integrate import ode
def f(t, y):
return -np.sqrt(t)*y
matA = np.array([[0, 1], [-1, 0]])
t = np.linspace(0, 10, 20)
y_0 = [2, 3]
y = np.zeros((len(t), len(y_0)))
y[0, :] = y_0
r = ode(f)
r.set_initial_value(y[0], t[0])
for k in range(1, len(t)):
y[k, :] = r.integrate(t[k])
r.set_initial_value(matA.dot(y[k, :]), t[k])
The values of y thus obtained are neither monotone nor positive, as the actual solution of the ODE would be - this shows that the multiplication by matA had an effect.
[[ 2.00000000e+00 3.00000000e+00]
[ 1.55052494e+00 2.32578740e+00]
[ 1.46027833e+00 -9.73518889e-01]
[-5.32831945e-01 -7.99247918e-01]
[-3.91483887e-01 2.60989258e-01]
[ 1.16154133e-01 1.74231200e-01]
[ 7.11807536e-02 -4.74538357e-02]
[-1.79307961e-02 -2.68961942e-02]
[-9.45453427e-03 6.30302285e-03]
[ 2.07088441e-03 3.10632661e-03]
[ 9.57623940e-04 -6.38415960e-04]
[-1.85274552e-04 -2.77911827e-04]
[-7.61389508e-05 5.07593005e-05]
[ 1.31604315e-05 1.97406472e-05]
[ 4.85413044e-06 -3.23608696e-06]
[-7.56142819e-07 -1.13421423e-06]
[-2.52269779e-07 1.68179853e-07]
[ 3.56625306e-08 5.34937959e-08]
[ 1.08295735e-08 -7.21971567e-09]
[-1.39690370e-09 -2.09535555e-09]]

Wrong optimization with scipy (maximize)

I'm trying to use scipy.optimize to identify the optimal values for 3 parameters(variable). I am starting with a very simple optimization function that sums the analyzed parameters together with some predefined (past) values. The values are bound using some fixed values. I set the value of the sign parameter to -1 as I am dealing with a maximization problem. However, scipy returns [0, 0, 0] as optimal values (same as setting sign=1), while the correct solution is [2, 2, 2]. Am I setting something wrong? What am I missing?
import scipy.optimize as optimize
import numpy as np
old = [1,1,1]
def f(params,sign=-1.0):
first, second, third = params
return sum(old+[first, second, third])
initial_guess = [2,2,2]
in1 = 1
in2 = 2
in3 = 1
bnds = ((0, in1+2), (0, in2+2), (0, in3+2))
result = optimize.minimize(f, initial_guess, bounds=bnds)
print result.x

In general when performing nonlinear optimization, libraries like your function to take only a single parameter vector. A good idea, generally, if you want to maximize a function is to minimize its inverse. If you simply want to maximize the value of x1+x2+x3, I would write things out this way:
from scipy.optimize import minimize
def f(x):
return 1/sum(x)
guess = [2,2,2]
x1bnds = (0, 3)
x2bnds = (0, 4)
x3bnds = (0, 5)
bnds = (x1bnds, x2bnds, x3bnds)
result = minimize(f, guess, bounds=bnds)
print(result.x) will give you [3,4,5] because the optimizer hit the bounds.
If you want to operate on the distance between your input parameters and some other values, I would modify the setup as so:
from functools import partial
from scipy.optimize import minimize
import numpy as np
other_values = np.asarray([3,4,5])
def f(x, other_pts):
x_lcl = np.asarray(x)
difference = x_lcl-other_pts
return 1/difference.sum()
guess = [2,2,2]
x1bnds = (0, 3)
x2bnds = (0, 4)
x3bnds = (0, 5)
bnds = (x1bnds, x2bnds, x3bnds)
f_opt = partial(f, other_values)
result = minimize(f_opt, guess, bounds=bnds)
print(result.x) will give you [0,0,0] because the optimizer hit the bounds.
It is a good idea to make the function you optimize not depend on external data (globals) -- using a partial will make everything a little nicer.
If you don't want to use numpy, you could use a list comprehension to do the elementwise subtraction of x and the other parameter vector, but this way things are a little nicer.

defining a fuction to be used in solving equation from a data file

I am completely new to python and in fact any fundamental programming language, I use Mathematica for my all my symbolic and numeric calculations. I am learning to work with python and finding it really awesome! Here is a problem I am trying to solve but stuck without a clue!
I have a data file for example
0. 1.
0.01 0.9998000066665778
0.02 0.9992001066609779
... ..
Which just the {t, Cos[2t]}.
I want to define a function out of this data and use it in solving an equation in python. My Mathematica intuition tells me that I should define the function like:
iFunc[x_] = Interpolation[iData, x]
and rest of the job is easy. for instance
NDSolve[{y''[x] + iFunc[x] y[x] == 0, y[0] == 1, y[1] == 0}, y, {x, 0, 1}]
Solves the equation easily. (I have not tried with more complicated cases though).
Now how to do the job in python and also accuracy is an important issue for me. So, now I would like to ask two questions.
1. Is this the most accurate method in Mathematica?
2. And what is the equivalent of more accurate way to do the problem in python?
Here is my humble attempt to solve the problem (with a lot of input from StackOverflow) where the definition with cos(2t) works:
from scipy.integrate import odeint
import numpy as np
import matplotlib.pyplot as plt
from math import cos
from scipy import interpolate
data = np.genfromtxt('cos2t.dat')
T = data[:,0] #first column
phi = data[:,1] #second column
f = interpolate.interp1d(T, phi)
tmin = 0.0# There should be a better way to define from the data
dt = 0.01
tmax = 2*np.pi
t = np.arange(tmin, tmax, dt)
phinew = f(t) # use interpolation function returned by `interp1d`
"""
def fun(z, t):
x, y = z
return np.array([y, -(cos(2*t))*x ])
"""
def fun(z, t):
x, y = z
return np.array([y, -(phinew(t))*x ])
sol1 = odeint(fun, [1, 0], t)[..., 0]
# for checking the plots
plt.plot(t, sol1, label='sol')
plt.show()
*When I run the code with interpolated function from cos(2t) data, is not working...the error message tell
Traceback (most recent call last): File "testde.py", line 30,
in <module> sol1 = odeint(fun, [1, 0], t)[..., 0]
File "/home/archimedes/anaconda3/lib/python3.6/site-packages/scip‌​y/integrate/odepack.‌​py",
line 215, in odeint ixpr, mxstep, mxhnil, mxordn, mxords)
File "testde.py",
line 28, in fun return np.array([y, -(phinew(t))*x ])
TypeError: 'numpy.ndarray' object is not callable.
I really can't decipher them. Please help...

In Mathematica, the usual way is simply
iFunc = Interpolation[iData]
Interpolation[iData] already returns a function.

To sub-question 2
With
t = np.arange(tmin, tmax, dt)
phinew = f(t) # use interpolation function returned by `interp1d`
equivalent to
phinew = np.array([ f(s) for s in t])
you construct phinew not as callable function but as array of values, closing the circle array to interpolation function to array. Use f which is a scalar function directly in the derivatives function,
def fun(z, t):
x, y = z
return np.array([y, -f(t)*x ])

How to real-time filter with scipy and lfilter?

Disclaimer: I am probably not as good at DSP as I should be and therefore have more issues than I should have getting this code to work.
I need to filter incoming signals as they happen. I tried to make this code to work, but I have not been able to so far.
Referencing scipy.signal.lfilter doc
import numpy as np
import scipy.signal
import matplotlib.pyplot as plt
from lib import fnlib
samples = 100
x = np.linspace(0, 7, samples)
y = [] # Unfiltered output
y_filt1 = [] # Real-time filtered
nyq = 0.5 * samples
f1_norm = 0.1 / nyq
f2_norm = 2 / nyq
b, a = scipy.signal.butter(2, [f1_norm, f2_norm], 'band', analog=False)
zi = scipy.signal.lfilter_zi(b,a)
zi = zi*(np.sin(0) + 0.1*np.sin(15*0))
This sets zi as zi*y[0 ] initially, which in this case is 0. I have got it from the example code in the lfilter documentation, but I am not sure if this is correct at all.
Then it comes to the point where I am not sure what to do with the few initial samples.
The coefficients a and b are len(a) = 5 here.
As lfilter takes input values from now to n-4, do I pad it with zeroes, or do I need to wait until 5 samples have gone by and take them as a single bloc, then continuously sample each next step in the same way?
for i in range(0, len(a)-1): # Append 0 as initial values, wrong?
y.append(0)
step = 0
for i in xrange(0, samples): #x:
tmp = np.sin(x[i]) + 0.1*np.sin(15*x[i])
y.append(tmp)
# What to do with the inital filterings until len(y) == len(a) ?
if (step> len(a)):
y_filt, zi = scipy.signal.lfilter(b, a, y[-len(a):], axis=-1, zi=zi)
y_filt1.append(y_filt[4])
print(len(y))
y = y[4:]
print(len(y))
y_filt2 = scipy.signal.lfilter(b, a, y) # Offline filtered
plt.plot(x, y, x, y_filt1, x, y_filt2)
plt.show()

I think I had the same problem, and found a solution on https://github.com/scipy/scipy/issues/5116:
from scipy import zeros, signal, random
def filter_sbs():
data = random.random(2000)
b = signal.firwin(150, 0.004)
z = signal.lfilter_zi(b, 1) * data[0]
result = zeros(data.size)
for i, x in enumerate(data):
result[i], z = signal.lfilter(b, 1, [x], zi=z)
return result
if __name__ == '__main__':
result = filter_sbs()
The idea is to pass the filter state z in each subsequent call to lfilter. For the first few samples the filter may give strange results, but later (depending on the filter length) it starts to behave correctly.

The problem is not how you are buffering the input. The problem is that in the 'offline' version, the state of the filter is initialized using lfilter_zi which computes the internal state of an LTI so that the output will already be in steady-state when new samples arrive at the input. In the 'real-time' version, you skip this so that the filter's initial state is 0. You can either initialize both versions to using lfilter_zi or else initialize both to 0. Then, it doesn't matter how many samples you filter at a time.
Note, if you initialize to 0, the filter will 'ring' for a certain amount of time before reaching a steady state. In the case of FIR filters, there is an analytic solution for determining this time. For many IIR filters, there is not.
This following is correct. For simplicity's sake I initialize to 0 and feed the input on sample at a time. However, any non-zero block size will produce equivalent output.
from scipy import signal, random
from numpy import zeros
def filter_sbs(data, b):
z = zeros(b.size-1)
result = zeros(data.size)
for i, x in enumerate(data):
result[i], z = signal.lfilter(b, 1, [x], zi=z)
return result
def filter(data, b):
result = signal.lfilter(b,1,data)
return result
if __name__ == '__main__':
data = random.random(20000)
b = signal.firwin(150, 0.004)
result1 = filter_sbs(data, b)
result2 = filter(data, b)
print(result1 - result2)
Output:
[ 0.00000000e+00 0.00000000e+00 0.00000000e+00 ... -5.55111512e-17
0.00000000e+00 1.66533454e-16]

Fitting negative binomial in python

In scipy there is no support for fitting a negative binomial distribution using data
(maybe due to the fact that the negative binomial in scipy is only discrete).
For a normal distribution I would just do:
from scipy.stats import norm
param = norm.fit(samp)
Is there something similar 'ready to use' function in any other library?

Statsmodels has discrete.discrete_model.NegativeBinomial.fit(), see here:
https://www.statsmodels.org/dev/generated/statsmodels.discrete.discrete_model.NegativeBinomial.fit.html#statsmodels.discrete.discrete_model.NegativeBinomial.fit

Not only because it is discrete, also because maximum likelihood fit to negative binomial can be quite involving, especially with an additional location parameter. That would be the reason why .fit() method is not provided for it (and other discrete distributions in Scipy), here is an example:
In [163]:
import scipy.stats as ss
import scipy.optimize as so
In [164]:
#define a likelihood function
def likelihood_f(P, x, neg=1):
n=np.round(P[0]) #by definition, it should be an integer
p=P[1]
loc=np.round(P[2])
return neg*(np.log(ss.nbinom.pmf(x, n, p, loc))).sum()
In [165]:
#generate a random variable
X=ss.nbinom.rvs(n=100, p=0.4, loc=0, size=1000)
In [166]:
#The likelihood
likelihood_f([100,0.4,0], X)
Out[166]:
-4400.3696690513316
In [167]:
#A simple fit, the fit is not good and the parameter estimate is way off
result=so.fmin(likelihood_f, [50, 1, 1], args=(X,-1), full_output=True, disp=False)
P1=result[0]
(result[1], result[0])
Out[167]:
(4418.599495886474, array([ 59.61196161, 0.28650831, 1.15141838]))
In [168]:
#Try a different set of start paramters, the fit is still not good and the parameter estimate is still way off
result=so.fmin(likelihood_f, [50, 0.5, 0], args=(X,-1), full_output=True, disp=False)
P1=result[0]
(result[1], result[0])
Out[168]:
(4417.1495981801972,
array([ 6.24809397e+01, 2.91877405e-01, 6.63343536e-04]))
In [169]:
#In this case we need a loop to get it right
result=[]
for i in range(40, 120): #in fact (80, 120) should probably be enough
_=so.fmin(likelihood_f, [i, 0.5, 0], args=(X,-1), full_output=True, disp=False)
result.append((_[1], _[0]))
In [170]:
#get the MLE
P2=sorted(result, key=lambda x: x[0])[0][1]
sorted(result, key=lambda x: x[0])[0]
Out[170]:
(4399.780263084549,
array([ 9.37289361e+01, 3.84587087e-01, 3.36856705e-04]))
In [171]:
#Which one is visually better?
plt.hist(X, bins=20, normed=True)
plt.plot(range(260), ss.nbinom.pmf(range(260), np.round(P1[0]), P1[1], np.round(P1[2])), 'g-')
plt.plot(range(260), ss.nbinom.pmf(range(260), np.round(P2[0]), P2[1], np.round(P2[2])), 'r-')
Out[171]:
[<matplotlib.lines.Line2D at 0x109776c10>]

I know this thread is quite old, but current readers may want to look at this repo which is made for this purpose: https://github.com/gokceneraslan/fit_nbinom
There's also an implementation here, though part of a larger package: https://github.com/ernstlab/ChromTime/blob/master/optimize.py

I stumbled across this thread, and found an answer for anyone else wondering.
If you simply need the n, p parameterisation used by scipy.stats.nbinom you can convert the mean and variance estimates:
mu = np.mean(sample)
sigma_sqr = np.var(sample)
n = mu**2 / (sigma_sqr - mu)
p = mu / sigma_sqr
If you the dispersionparameter you can use a negative binomial regression model from statsmodels with just an interaction term. This will find the dispersionparameter alpha using MLE.
# Data processing
import pandas as pd
import numpy as np
# Analysis models
import statsmodels.formula.api as smf
from scipy.stats import nbinom
def convert_params(mu, alpha):
"""
Convert mean/dispersion parameterization of a negative binomial to the ones scipy supports
Parameters
----------
mu : float
Mean of NB distribution.
alpha : float
Overdispersion parameter used for variance calculation.
See https://en.wikipedia.org/wiki/Negative_binomial_distribution#Alternative_formulations
"""
var = mu + alpha * mu ** 2
p = mu / var
r = mu ** 2 / (var - mu)
return r, p
# Generate sample data
n = 2
p = 0.9
sample = nbinom.rvs(n=n, p=p, size=10000)
# Estimate parameters
## Mean estimates expectation parameter for negative binomial distribution
mu = np.mean(sample)
## Dispersion parameter from nb model with only interaction term
nbfit = smf.negativebinomial("nbdata ~ 1", data=pd.DataFrame({"nbdata": sample})).fit()
alpha = nbfit.params[1] # Dispersion parameter
# Convert parameters to n, p parameterization
n_est, p_est = convert_params(mu, alpha)
# Check that estimates are close to the true values:
print("""
{:<3} {:<3}
True parameters: {:<3} {:<3}
Estimates : {:<3} {:<3}""".format('n', 'p', n, p,
np.round(n_est, 2), np.round(p_est, 2)))