I have a set of data showing radition not being absorbed as a function of velocity. The data shows a very clear dip or if we plot the inverse of the data the absorbtion, we get a clear peak instead. I have no reason not to belive this peak to be a gaussian and would like to make a fit to get the variance of this peak. So I've tried to use scipy.optimize.curve_fit, to achieve this. Both using scipy.stats.norm.pdf and a self written version of the function. No matter initial guesses. The resulting fit is way of.
I attached the code and a picture of the resulting graph.
What am I doing wrong? Are there any general tricks for these kind of tasks?
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
import numpy as np
cts = []
vel = []
file = open("Heisenberg/Mössbauer/Final.lst", "r")
linesArr = file.readlines()
for i in range(210, 260):
lineList1 = linesArr[i].split()
cts.append(int(lineList1[1]))
chn = (int(lineList1[0]))
tempVel = -0.04 * chn + 9.3
vel.append(tempVel)
def func (x, mu,sigma):
return (1 / (np.sqrt(np.pi * 2) *sigma)) * np.exp(-0.5*((x-mu)/sigma)**2)
data = np.array(cts)
cts_norm = (data - data.min())/ (data.max() - data.min())
cts_inv = 1 - cts_norm
fit, error = curve_fit(func, vel, cts_inv, p0=[0.2, 0.2])
print(fit)
plt.plot(vel, cts_inv, 'bo')
plt.plot(vel, func(vel, fit[0],fit[1]),"r")
The issue is that you are trying to fit a normal distribution with data that is not a probability distribution! Probability distributions have an integral equal to 1, but that is not the case for your data, which can have any amplitude. It would be hard to normalize your data to satisfy this. Instead, you can simply add a new parameter which controls the "amplitude" of the normal distribution, like below:
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
import numpy as np
cts = [0, 0, 0, 0, -1, -2, -5, -10, -5, -2, -1, 0, 0, 0, 0]
vel = np.linspace(-0.75, 1.25, 15)
def func(x, mu, sigma, a):
return a * np.exp(-0.5 * ((x - mu) / sigma) ** 2) # << here
data = np.array(cts)
cts_norm = (data - data.min()) / (data.max() - data.min())
cts_inv = 1 - cts_norm
fit, error = curve_fit(func, vel, cts_inv, p0=[0.2, 0.2, 1]) # << here
print(fit)
plt.plot(vel, cts_inv, 'bo')
plt.plot(vel, func(vel, fit[0], fit[1], fit[2]), "r") # << and here
plt.show()
(I used some dummy data as I don't have access to your file, but it doesn't really matter)
I would add a little more flexibility to your model as follows. I retrieved your data by taking a screenshot of the image and using this free web service.
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import curve_fit
from scipy.stats import norm
data = np.loadtxt(r"C:\Users\Cristiano\Desktop\data.txt", delimiter=",")
x = data[:, 0]
y = data[:, 1]
def f(x, a, b, mu, sigma):
return a + b * np.exp(-(x - mu) ** 2 / (2 * sigma ** 2))
popt, pcov = curve_fit(f, x, y)
mean, std = norm.fit(data)
plt.scatter(x, y)
xx = np.linspace(-0.75, 1.25, 1000)
plt.plot(xx, f(xx, *popt))
plt.show()
Related
I need to find the centre of the bright spot in sequent images like this:cross correlation
My idea is to fit the image with a 2d gassian function in order to find the maximum. I use scipy.optimize.curve_fit for this. I created a simulated data for fitting and here the fit works well.
import scipy.optimize as opt
import numpy as np
import pylab as plt
import cv2
#define 2d gaussian function
def gauss2dFunc(xy, xo, yo, sigma):
x = xy[0]
y = xy[1]
return np.exp( -((x-xo)*(x-xo) + (y-yo)*(y-yo)) / (2.*sigma*sigma) ).ravel()
xvec = np.array(range(200))
yvec = np.array(range(200))
X,Y = np.meshgrid(xvec,yvec)
initial_guess = [100,100,10] #xo,yo,sigma
#create data with gaussian distribution
def gauss2d(shape=(200,200),sigma=20):
m,n = [(ss-1.)/2. for ss in shape]
y,x = np.ogrid[-m:m+1,-n:n+1]
h = np.exp( -(x*x + y*y) / (2.*sigma*sigma) )
h[ h < np.finfo(h.dtype).eps*h.max() ] = 0
h = h / h.max()
return h[:200, :200]
#add noise
gauss2d_noise = gauss2d() + 0.2*np.random.normal(size=gauss2d().shape)
#fit with 2d gaussian function
popt, pcov = opt.curve_fit(gauss2dFunc, (X, Y), gauss2d_noise.ravel() , p0=initial_guess)
fit = gauss2dFunc((X,Y), *popt)
f, axarr = plt.subplots(1,2)
axarr[0].imshow(gauss2d_noise, cmap='gray')
axarr[1].imshow(fit.reshape(200,200), cmap='gray')
left:simulated data - right:fit
But as soon I try to fit real image it fails.
import scipy.optimize as opt
import numpy as np
import pylab as plt
import cv2
#define 2d gaussian function
def gauss2dFunc(xy, xo, yo, sigma):
x = xy[0]
y = xy[1]
return np.exp( -((x-xo)*(x-xo) + (y-yo)*(y-yo)) / (2.*sigma*sigma) ).ravel()
xvec = np.array(range(200))
yvec = np.array(range(200))
X,Y = np.meshgrid(xvec,yvec)
initial_guess = [100,100,10] #xo,yo,sigma
#read iamge
image = cv2.imread('cor.png', 0)
print(np.shape(image))
#fit with 2d gaussian function
popt, pcov = opt.curve_fit(gauss2dFunc, (X, Y), image.ravel() , p0=initial_guess)
fit = gauss2dFunc((X,Y), *popt)
f, axarr = plt.subplots(1,2)
axarr[0].imshow(image, cmap='gray')
axarr[1].imshow(fit.reshape(200,200), cmap='gray')
left:real image - right:fit
I am new in Python and I have no idea why it doesn't work. It would be great if someone would help me.
I try to display a gyroid structure (triply periodic minimal surface) in python. I have tried several packages but pyvista seems the more appropriate for my study. The gyroid is a periodic minimal surface that can be modelled according to the following equation:
sin(a(i - 1)) * cos(a(j - 1)) + sin(a(j - 1)) * cos(a(k - 1)) + sin(a(k - 1)) * cos(a(i - 1)) < b
where a is a scaling parameter equal to 2pi/l and l is the unit cell length. The value b controls the volume fraction and typically varies between 1.32 and 0.15.
It seems to work well nevertheless from a certain value of b, discontinuities appear in my mesh which is not correct. I tried to use pymesh to repair it or scypy.interpolate but it changes nothing. Do you have an idea? Thanks in advance.
import pyvista as pv
import numpy as np
from numpy import cos, sin, pi
from random import uniform
from scipy.interpolate import RegularGridInterpolator
import pymeshfix
lattice_par = 1.0 # Unit cell length
a = (2*pi)/lattice_par
def gyroid(x, y, z, b=1.2):
return sin(a*(x - 1))*cos(a*(y - 1)) + sin(a*(y - 1))*cos(a*(z - 1)) + sin(a*(z - 1))*cos(a*(x - 1)) - b
# %% Data
x, y, z = np.mgrid[-2:2:31j, -3:3:31j, 0:4:31j]
vol = gyroid(x, y, z)
grid = pv.StructuredGrid(x, y, z)
grid["vol"] = vol.flatten()
contours = grid.contour([0]) # Isosurface = 0
# %% Visualization
pv.set_plot_theme('document')
p = pv.Plotter()
p.add_mesh(contours, scalars=contours.points[:, 2], show_scalar_bar=False, interpolate_before_map=True,
show_edges=False, smooth_shading=False, render=True)
p.show_axes_all()
p.add_floor()
p.show_grid()
p.add_title('Gyroid')
p.show(window_size=[2040, 1500])
I am trying to fit a gaussian to a discrete potential using the astropy.modeling package. Although I assign a negative amplitude to the gaussian, it returns a null gaussian, i.e, with zero amplitude everywhere:
plot(c[0], pot_x, label='Discrete potential')
plot(c[0], g(c[0]), label='Gaussian fit')
legend()
I have the following code lines to perform the fitting:
g_init = models.Gaussian1D(amplitude=-1., mean=0, stddev=1.)
fit_g = fitting.LevMarLSQFitter()
g = fit_g(g_init, c[0], pot_x)
Where
c[0] = array([13.31381488, 13.31944489, 13.32507491, 13.33070493, 13.33633494,
13.34196496, 13.34759498, 13.35322499, 13.35885501, 13.36448503,
13.37011504, 13.37574506, 13.38137507, 13.38700509, 13.39263511,
13.39826512, 13.40389514, 13.40952516, 13.41515517, 13.42078519])
pot_x = array([ -1.72620157, -3.71811187, -6.01282809, -6.98874144,
-8.36645166, -14.31787771, -23.3688849 , -26.14679496,
-18.85970983, -10.73888697, -7.10763373, -5.81176637,
-5.44146953, -5.37165105, -4.6454408 , -2.90307138,
-1.66250349, -1.66096343, -1.8188269 , -1.41980552])
Does anyone have an ideia what the problem might be?
Solved: I just had to assign a mean that is in the range of the domain, like 13.35.
As I am not familiar with Astropy, I used scipy. The code below provides the following outpt:
import numpy as np
from matplotlib import pyplot as plt
from scipy.optimize import curve_fit
x = np.asarray([13.31381488, 13.31944489, 13.32507491, 13.33070493, 13.33633494,
13.34196496, 13.34759498, 13.35322499, 13.35885501, 13.36448503,
13.37011504, 13.37574506, 13.38137507, 13.38700509, 13.39263511,
13.39826512, 13.40389514, 13.40952516, 13.41515517, 13.42078519])
y = -np.asarray([ -1.72620157, -3.71811187, -6.01282809, -6.98874144,
-8.36645166, -14.31787771, -23.3688849 , -26.14679496,
-18.85970983, -10.73888697, -7.10763373, -5.81176637,
-5.44146953, -5.37165105, -4.6454408 , -2.90307138,
-1.66250349, -1.66096343, -1.8188269 , -1.41980552])
mean = sum(x * y) / sum(y)
sigma = np.sqrt(sum(y * (x - mean)**2) / sum(y))
def Gauss(x, a, x0, sigma):
return a * np.exp(-(x - x0)**2 / (2 * sigma**2))
popt,pcov = curve_fit(Gauss, x, y, p0=[max(y), mean, sigma])
plt.plot(x, y, 'b+:', label='data')
plt.plot(x, Gauss(x, *popt), 'r-', label='fit')
plt.legend()
By simplicity, I reused this answer. I am not entirely certain about the mean and sigma definition, as I am not used to fitting a Gaussian on a 2D dataset. However, it doesn't really matter as it is simply used to define an approximatation used to start the curve_fit algorithm.
I've been trying to implement the skewed generalized t distribution in Python to model some financial returns. I based my code on formulas found on Wikipedia, and I used the Beta distribution from scipy.
from scipy.special import beta
import numpy as np
from math import sqrt
def sgt(x, params):
# This function accepts an array of 5 parameters [mu, sigma, lambda, p, q]
mu, sigma, lam, p, q = params
v = (q**(-1/p)) / (sqrt((3*lam*lam + 1)*beta(3/p, q-2/p)/beta(1/p, q) - 4*lam*lam*(beta(2/p, q-1/p)/(beta(1/p, q)))**2))
m = 2*v*sigma*lam*q**(1/p)*beta(2/p, q - 1/p) / beta(1/p, q)
fx = p / (2*v*sigma*(q**(1/p))*beta(1/p, q)*((abs(x-mu+m)**p/(q*(v*sigma)**p*(lam*np.sign(x-mu+m)+1)**p + 1)+1)**(1/p + q)))
return fx
Now, the function seems to work perfectly fine for some sets of parameters, but terribly for other sets of parameters.
For example:
dx = 0.001
x_axis = np.arange(-10, 10, dx)
ok_parameters = [0, 2, 0, 3, 8]
bad_parameters = [0, 2, 0, 1.05, 2.1]
ok_distribution = sgt(x_axis, ok_parameters)
bad_distribution = sgt(x_axis, bad_parameters)
If I try to compute the integrals of those two numbers:
a = np.sum(ok_distribution*dx)
b = np.sum(bad_distribution*dx)
I obtain the results a = 1.0013233154393804 and b = 2.2799746093533346.
Now, in theory both of these should be 1, but I assume since I approximated the integral the value won't always be exactly 1. In the second case however I don't understand why the value is so high.
Does anyone know what the issue is?
These are the graphs of the ok distribution (blue) and bad distribution (orange)
I believe there was just a typo (though I couldn't exactly find where) in your definition sgt. Here is an implementation that works.
%matplotlib inline
import matplotlib.pyplot as plt
from scipy.special import beta
import numpy as np
from math import sqrt
from typing import Union
from scipy import integrate
# Generalised Student T probability Distribution
def generalized_student_t(x:Union[float, np.ndarray], mu:float, sigma:float,
lam:float, p:float, q:float) \
-> Union[float, np.ndarray]:
v = q**(-1/p) * ((3*lam**2 + 1)*(beta(3/p, q - 2/p)/beta(1/p,q)) - 4*lam**2*(beta(2/p, q - 1/p)/beta(1/p,q))**2)**(-1/2)
m = 2*v*sigma*lam*q**(1/p)*beta(2/p,q - 1/p)/beta(1/p,q)
fx = p / (2*v*sigma*q**(1/p)*beta(1/p,q)*(abs(x-mu+m)**p/(q*(v*sigma)**p)*(lam*np.sign(x-mu+m)+1)**p + 1)**(1/p + q))
return fx
def plot_cdf_pdf(x_axis:np.ndarray, pmf:np.ndarray) -> None:
"""
Plot the PDF and CDF of the array returned from the function.
"""
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 6))
ax1.plot(x_axis, pmf)
ax1.set_title('PDF')
ax2.plot(x_axis, integrate.cumtrapz(x=x_axis, y=pmf, initial = 0))
ax2.set_title('CDF')
pass
dx = 0.0001
x_axis = np.arange(-10, 10, dx)
# Create the Two
distribution1 = generalized_student_t(x=x_axis, mu=0, sigma=1, lam=0, p=2, q=100)
distribution2 = generalized_student_t(x=x_axis, mu=0, sigma=2, lam=0, p=1.05, q=2.1)
plot_cdf_pdf(x_axis=x_axis, pmf=distribution1)
plot_cdf_pdf(x_axis=x_axis, pmf=distribution2)
We can also check that the integral of the PDFs are 1
integrate.simps(x=x_axis, y = distribution1)
integrate.simps(x=x_axis, y = distribution2)
We can see the results of the integral are 0.99999999999999978 and 0.99752026308335162. The reason they are not exactly 1 is due the CDF being defined as integral from -infinity to infinity of the PDF.
How do I numerically solve an ODE in Python?
Consider
\ddot{u}(\phi) = -u + \sqrt{u}
with the following conditions
u(0) = 1.49907
and
\dot{u}(0) = 0
with the constraint
0 <= \phi <= 7\pi.
Then finally, I want to produce a parametric plot where the x and y coordinates are generated as a function of u.
The problem is, I need to run odeint twice since this is a second order differential equation.
I tried having it run again after the first time but it comes back with a Jacobian error. There must be a way to run it twice all at once.
Here is the error:
odepack.error: The function and its Jacobian must be callable functions
which the code below generates. The line in question is the sol = odeint.
import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
from numpy import linspace
def f(u, t):
return -u + np.sqrt(u)
times = linspace(0.0001, 7 * np.pi, 1000)
y0 = 1.49907
yprime0 = 0
yvals = odeint(f, yprime0, times)
sol = odeint(yvals, y0, times)
x = 1 / sol * np.cos(times)
y = 1 / sol * np.sin(times)
plot(x,y)
plt.show()
Edit
I am trying to construct the plot on page 9
Classical Mechanics Taylor
Here is the plot with Mathematica
In[27]:= sol =
NDSolve[{y''[t] == -y[t] + Sqrt[y[t]], y[0] == 1/.66707928,
y'[0] == 0}, y, {t, 0, 10*\[Pi]}];
In[28]:= ysol = y[t] /. sol[[1]];
In[30]:= ParametricPlot[{1/ysol*Cos[t], 1/ysol*Sin[t]}, {t, 0,
7 \[Pi]}, PlotRange -> {{-2, 2}, {-2.5, 2.5}}]
import scipy.integrate as integrate
import matplotlib.pyplot as plt
import numpy as np
pi = np.pi
sqrt = np.sqrt
cos = np.cos
sin = np.sin
def deriv_z(z, phi):
u, udot = z
return [udot, -u + sqrt(u)]
phi = np.linspace(0, 7.0*pi, 2000)
zinit = [1.49907, 0]
z = integrate.odeint(deriv_z, zinit, phi)
u, udot = z.T
# plt.plot(phi, u)
fig, ax = plt.subplots()
ax.plot(1/u*cos(phi), 1/u*sin(phi))
ax.set_aspect('equal')
plt.grid(True)
plt.show()
The code from your other question is really close to what you want. Two changes are needed:
You were solving a different ODE (because you changed two signs inside function deriv)
The y component of your desired plot comes from the solution values, not from the values of the first derivative of the solution, so you need to replace u[:,0] (function values) for u[:, 1] (derivatives).
This is the end result:
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import odeint
def deriv(u, t):
return np.array([u[1], -u[0] + np.sqrt(u[0])])
time = np.arange(0.01, 7 * np.pi, 0.0001)
uinit = np.array([1.49907, 0])
u = odeint(deriv, uinit, time)
x = 1 / u[:, 0] * np.cos(time)
y = 1 / u[:, 0] * np.sin(time)
plt.plot(x, y)
plt.show()
However, I suggest that you use the code from unutbu's answer because it's self documenting (u, udot = z) and uses np.linspace instead of np.arange. Then, run this to get your desired figure:
x = 1 / u * np.cos(phi)
y = 1 / u * np.sin(phi)
plt.plot(x, y)
plt.show()
You can use scipy.integrate.ode. To solve dy/dt = f(t,y), with initial condition y(t0)=y0, at time=t1 with 4th order Runge-Kutta you could do something like this:
from scipy.integrate import ode
solver = ode(f).set_integrator('dopri5')
solver.set_initial_value(y0, t0)
dt = 0.1
while t < t1:
y = solver.integrate(t+dt)
t += dt
Edit: You have to get your derivative to first order to use numerical integration. This you can achieve by setting e.g. z1=u and z2=du/dt, after which you have dz1/dt = z2 and dz2/dt = d^2u/dt^2. Substitute these into your original equation, and simply iterate over the vector dZ/dt, which is first order.
Edit 2: Here's an example code for the whole thing:
import numpy as np
import matplotlib.pyplot as plt
from numpy import sqrt, pi, sin, cos
from scipy.integrate import ode
# use z = [z1, z2] = [u, u']
# and then f = z' = [u', u''] = [z2, -z1+sqrt(z1)]
def f(phi, z):
return [z[1], -z[0]+sqrt(z[0])]
# initialize the 4th order Runge-Kutta solver
solver = ode(f).set_integrator('dopri5')
# initial value
z0 = [1.49907, 0.]
solver.set_initial_value(z0)
values = 1000
phi = np.linspace(0.0001, 7.*pi, values)
u = np.zeros(values)
for ii in range(values):
u[ii] = solver.integrate(phi[ii])[0] #z[0]=u
x = 1. / u * cos(phi)
y = 1. / u * sin(phi)
plt.figure()
plt.plot(x,y)
plt.grid()
plt.show()
scipy.integrate() does ODE integration. Is that what you are looking for?