I need to find the centre of the bright spot in sequent images like this:cross correlation
My idea is to fit the image with a 2d gassian function in order to find the maximum. I use scipy.optimize.curve_fit for this. I created a simulated data for fitting and here the fit works well.
import scipy.optimize as opt
import numpy as np
import pylab as plt
import cv2
#define 2d gaussian function
def gauss2dFunc(xy, xo, yo, sigma):
x = xy[0]
y = xy[1]
return np.exp( -((x-xo)*(x-xo) + (y-yo)*(y-yo)) / (2.*sigma*sigma) ).ravel()
xvec = np.array(range(200))
yvec = np.array(range(200))
X,Y = np.meshgrid(xvec,yvec)
initial_guess = [100,100,10] #xo,yo,sigma
#create data with gaussian distribution
def gauss2d(shape=(200,200),sigma=20):
m,n = [(ss-1.)/2. for ss in shape]
y,x = np.ogrid[-m:m+1,-n:n+1]
h = np.exp( -(x*x + y*y) / (2.*sigma*sigma) )
h[ h < np.finfo(h.dtype).eps*h.max() ] = 0
h = h / h.max()
return h[:200, :200]
#add noise
gauss2d_noise = gauss2d() + 0.2*np.random.normal(size=gauss2d().shape)
#fit with 2d gaussian function
popt, pcov = opt.curve_fit(gauss2dFunc, (X, Y), gauss2d_noise.ravel() , p0=initial_guess)
fit = gauss2dFunc((X,Y), *popt)
f, axarr = plt.subplots(1,2)
axarr[0].imshow(gauss2d_noise, cmap='gray')
axarr[1].imshow(fit.reshape(200,200), cmap='gray')
left:simulated data - right:fit
But as soon I try to fit real image it fails.
import scipy.optimize as opt
import numpy as np
import pylab as plt
import cv2
#define 2d gaussian function
def gauss2dFunc(xy, xo, yo, sigma):
x = xy[0]
y = xy[1]
return np.exp( -((x-xo)*(x-xo) + (y-yo)*(y-yo)) / (2.*sigma*sigma) ).ravel()
xvec = np.array(range(200))
yvec = np.array(range(200))
X,Y = np.meshgrid(xvec,yvec)
initial_guess = [100,100,10] #xo,yo,sigma
#read iamge
image = cv2.imread('cor.png', 0)
print(np.shape(image))
#fit with 2d gaussian function
popt, pcov = opt.curve_fit(gauss2dFunc, (X, Y), image.ravel() , p0=initial_guess)
fit = gauss2dFunc((X,Y), *popt)
f, axarr = plt.subplots(1,2)
axarr[0].imshow(image, cmap='gray')
axarr[1].imshow(fit.reshape(200,200), cmap='gray')
left:real image - right:fit
I am new in Python and I have no idea why it doesn't work. It would be great if someone would help me.
Related
I have a set of data showing radition not being absorbed as a function of velocity. The data shows a very clear dip or if we plot the inverse of the data the absorbtion, we get a clear peak instead. I have no reason not to belive this peak to be a gaussian and would like to make a fit to get the variance of this peak. So I've tried to use scipy.optimize.curve_fit, to achieve this. Both using scipy.stats.norm.pdf and a self written version of the function. No matter initial guesses. The resulting fit is way of.
I attached the code and a picture of the resulting graph.
What am I doing wrong? Are there any general tricks for these kind of tasks?
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
import numpy as np
cts = []
vel = []
file = open("Heisenberg/Mössbauer/Final.lst", "r")
linesArr = file.readlines()
for i in range(210, 260):
lineList1 = linesArr[i].split()
cts.append(int(lineList1[1]))
chn = (int(lineList1[0]))
tempVel = -0.04 * chn + 9.3
vel.append(tempVel)
def func (x, mu,sigma):
return (1 / (np.sqrt(np.pi * 2) *sigma)) * np.exp(-0.5*((x-mu)/sigma)**2)
data = np.array(cts)
cts_norm = (data - data.min())/ (data.max() - data.min())
cts_inv = 1 - cts_norm
fit, error = curve_fit(func, vel, cts_inv, p0=[0.2, 0.2])
print(fit)
plt.plot(vel, cts_inv, 'bo')
plt.plot(vel, func(vel, fit[0],fit[1]),"r")
The issue is that you are trying to fit a normal distribution with data that is not a probability distribution! Probability distributions have an integral equal to 1, but that is not the case for your data, which can have any amplitude. It would be hard to normalize your data to satisfy this. Instead, you can simply add a new parameter which controls the "amplitude" of the normal distribution, like below:
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
import numpy as np
cts = [0, 0, 0, 0, -1, -2, -5, -10, -5, -2, -1, 0, 0, 0, 0]
vel = np.linspace(-0.75, 1.25, 15)
def func(x, mu, sigma, a):
return a * np.exp(-0.5 * ((x - mu) / sigma) ** 2) # << here
data = np.array(cts)
cts_norm = (data - data.min()) / (data.max() - data.min())
cts_inv = 1 - cts_norm
fit, error = curve_fit(func, vel, cts_inv, p0=[0.2, 0.2, 1]) # << here
print(fit)
plt.plot(vel, cts_inv, 'bo')
plt.plot(vel, func(vel, fit[0], fit[1], fit[2]), "r") # << and here
plt.show()
(I used some dummy data as I don't have access to your file, but it doesn't really matter)
I would add a little more flexibility to your model as follows. I retrieved your data by taking a screenshot of the image and using this free web service.
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import curve_fit
from scipy.stats import norm
data = np.loadtxt(r"C:\Users\Cristiano\Desktop\data.txt", delimiter=",")
x = data[:, 0]
y = data[:, 1]
def f(x, a, b, mu, sigma):
return a + b * np.exp(-(x - mu) ** 2 / (2 * sigma ** 2))
popt, pcov = curve_fit(f, x, y)
mean, std = norm.fit(data)
plt.scatter(x, y)
xx = np.linspace(-0.75, 1.25, 1000)
plt.plot(xx, f(xx, *popt))
plt.show()
Suppose I have a 2D Gaussian with pdf
I want to draw an ellipse corresponding to the level-set (contour)
Following here I know that I can replace the precision matrix with its eigendecomposition to obtain
where gamma is
Then to find coordinates of the points on the ellipse I would have to do
I tried plotting this but it is not working.
Plotting the Contours
from scipy.stats import multivariate_normal
import numpy as np
from numpy.linalg import eigh
import math
import matplotlib.pyplot as plt
# Target distribution
sx2 = 1.0
sy2 = 2.0
rho = 0.6
Sigma = np.array([[sx2, rho*math.sqrt(sx2)*math.sqrt(sy2)], [rho*math.sqrt(sx2)*math.sqrt(sy2), sy2]])
target = multivariate_normal(mean=np.zeros(2), cov=Sigma)
# Two different contours
xy = target.rvs()
xy2 = target.rvs()
# Values where to plot the density
x, y = np.mgrid[-2:2:0.1, -2:2:0.1]
zz = target.pdf(np.dstack((x, y)))
fig, ax = plt.subplots()
ax.contour(x,y, zz, levels=np.sort([target.pdf(xy), target.pdf(xy2)]))
ax.set_aspect("equal")
plt.show()
The code above shows the contour
Plotting the Ellipse
# Find gamma and perform eigendecomposition
gamma = math.log(1 / (4*(np.pi**2)*sx2*sy2*(1 - rho**2)*(target.pdf(xy)**2)))
eigenvalues, P = eigh(np.linalg.inv(Sigma))
# Compute u and v as per link using thetas from 0 to 2pi
thetas = np.linspace(0, 2*np.pi, 100)
uv = (gamma / np.sqrt(eigenvalues)) * np.hstack((np.cos(thetas).reshape(-1,1), np.sin(thetas).reshape(-1, 1)))
# Plot
plt.scatter(uv[:, 0], uv[:, 1])
However this clearly doesn't work.
You should square sx2 and sy2 in gamma.
gamma should be square rooted.
Multiply the resulting ellipse by P^-1 to get points in the original coordinate system. That's mentioned in the linked post. You have to convert back to the original coordinate system. I don't know actually how to code this, or if it actually works, so I leave the coding to you.
gamma = math.log(1 / (4*(np.pi**2)*(sx2**2)*(sy2**2)*(1 - rho**2)*(target.pdf(xy)**2)))
eigenvalues, P = eigh(np.linalg.inv(Sigma))
# Compute u and v as per link using thetas from 0 to 2pi
thetas = np.linspace(0, 2*np.pi, 100)
uv = (np.sqrt(gamma) / np.sqrt(eigenvalues)) * np.hstack((np.cos(thetas).reshape(-1,1), np.sin(thetas).reshape(-1, 1)))
orig_coord=np.linalg.inv(P) * uv #I don't how to code this in python
plt.scatter(orig_coord[:,0], orig_coord[:,1])
plt.show()
My attempt at coding it:
gamma = math.log(1 / (4*(np.pi**2)*(sx2**2)*(sy2**2)*(1 - rho**2)*(target.pdf(xy)**2)))
eigenvalues, P = eigh(np.linalg.inv(Sigma))
# Compute u and v as per link using thetas from 0 to 2pi
thetas = np.linspace(0, 2*np.pi, 100)
uv = (np.sqrt(gamma) / np.sqrt(eigenvalues)) * np.hstack((np.cos(thetas).reshape(-1,1), np.sin(thetas).reshape(-1, 1)))
orig_coord=np.zeros((100,2))
for i in range(len(uv)):
orig_coord[i,0]=np.matmul(np.linalg.inv(P), uv[i,:])[0]
orig_coord[i,1]=np.matmul(np.linalg.inv(P), uv[i,:])[1]
# Plot
plt.scatter(orig_coord[:, 0], orig_coord[:, 1])
gamma1 = math.log(1 / (4*(np.pi**2)*(sx2**2)*(sy2**2)*(1 - rho**2)*(target.pdf(xy2)**2)))
uv1 = (np.sqrt(gamma1) / np.sqrt(eigenvalues)) * np.hstack((np.cos(thetas).reshape(-1,1), np.sin(thetas).reshape(-1, 1)))
orig_coord1=np.zeros((100,2))
for i in range(len(uv)):
orig_coord1[i,0]=np.matmul(np.linalg.inv(P), uv1[i,:])[0]
orig_coord1[i,1]=np.matmul(np.linalg.inv(P), uv1[i,:])[1]
plt.scatter(orig_coord1[:, 0], orig_coord1[:, 1])
plt.axis([-2,2,-2,2])
plt.show()
Sometimes the plots don't work and you get the error invalid sqrt, but when it works it looks fine.
]2
I want to plot the regression line and calculate the regression coefficient. Along with that the R square/Goodness of fit.
I have determined the regression line on the curve and calculated the coefficient of that regression line but not be able to calculate the goodness of fit for those.
Kindly help me to solve that problem.
The code calculated the regression coefficient and draw the regression line on the plot. But i am not be able to calculate the R squared/Goodness of fit.
Code of goodness of fit is also written but giving an error "TypeError: can't convert type 'ndarray' to numerator/denominator"
The error is related with the "def coefficient of determination part"
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
from statistics import mean
from matplotlib import style
style.use('ggplot')
#loading of imageries
im1 = Image.open('D:\\code\\test\\subset\\image1.tif')
im2 = Image.open('D:\\code\\test\\subset\\image2.tif')
#im.show()
#Conversion into array
img1 = np.array(im1)
img2 = np.array(im2)
def estimate_coef(x, y):
# number of observations/points
n = np.size(x)
# mean of x and y vector
m_x, m_y = np.mean(x), np.mean(y)
# calculating cross-deviation and deviation about x
SS_xy = np.sum(y*x) - n*m_y*m_x
SS_xx = np.sum(x*x) - n*m_x*m_x
# calculating regression coefficients
b_1 = SS_xy / SS_xx
b_0 = m_y - b_1*m_x
return(b_0, b_1)
def squared_error(ys_orig,ys_line):
return sum((ys_line - ys_orig) * (ys_line - ys_orig))
def coefficient_of_determination(ys_orig,ys_line):
y_mean_line = [mean(ys_orig) for y in ys_orig]
squared_error_regr = squared_error(ys_orig, ys_line)
squared_error_y_mean = squared_error(ys_orig, ys_mean_line)
return 1 - (squared_error_regr/squared_error_y_mean)
def plot_regression_line(x, y, b):
# plotting the actual points as scatter plot
plt.scatter(x, y, color = "m",
marker = "o", s = 30)
# predicted response vector
y_pred = b[0] + b[1]*x
# plotting the regression line
plt.plot(x, y_pred, color = "g")
# putting labels
plt.xlabel('x')
plt.ylabel('y')
# function to show plot
plt.show()
b_0, b_1 = estimate_coef(img1, img2)
regression_line = [(b_0*x)+b_1 for x in img1]
r_squared = coefficient_of_determination(img2,regression_line)
print(r_squared)
def main():
# observations
x = img1
y = img2
# estimating coefficients
b = estimate_coef(x, y)
print("Estimated coefficients:\nb_0 = {} \ \nb_1 = {}".format(b[0], b[1]))
# plotting regression line
plot_regression_line(x, y, b)
if __name__ == "__main__":
main()
I have a bunch of x, y points that represent a sigmoidal function:
x=[ 1.00094909 1.08787635 1.17481363 1.2617564 1.34867881 1.43562284
1.52259341 1.609522 1.69631283 1.78276102 1.86426648 1.92896789
1.9464453 1.94941586 2.00062852 2.073691 2.14982808 2.22808316
2.30634034 2.38456905 2.46280126 2.54106611 2.6193345 2.69748825]
y=[-0.10057627 -0.10172142 -0.10320428 -0.10378959 -0.10348456 -0.10312503
-0.10276956 -0.10170055 -0.09778279 -0.08608644 -0.05797392 0.00063599
0.08732999 0.16429878 0.2223306 0.25368884 0.26830932 0.27313931
0.27308756 0.27048902 0.26626313 0.26139534 0.25634544 0.2509893 ]
I use scipy.interpolate.UnivariateSpline() to fit to some cubic spline as follows:
from scipy.interpolate import UnivariateSpline
s = UnivariateSpline(x, y, k=3, s=0)
xfit = np.linspace(x.min(), x.max(), 200)
plt.scatter(x,y)
plt.plot(xfit, s(xfit))
plt.show()
This is what I get:
Since I specify s=0, the spline adheres completely to the data, but there are too many wiggles. Using a higher k value leads to even more wiggles.
So my questions are --
How should I correctly use scipy.interpolate.UnivariateSpline() to fit my data? More precisely, how do I make the spline minimise its wiggling?
Is this even the correct choice for this kind of a sigmoidal function? Should I be using something like scipy.optimize.curve_fit() with a trial tanh(x) function instead?
There are several options, I list a few below. The last one seems to give the best output. Whether you should use a spline or an actual function depends on what you want to do with the output; I list two analytical functions below that could be used but I don't know in which context the data were derived so it is hard to find the best one for you.
You can play with s, e.g. for s=0.005, the plot looks like this (still not extremely pretty but you could further adjust):
But I would indeed use a "proper" function and fit using e.g. curve_fit. The function below is still not ideal as it is monotonically increasing, so we miss the decrease at the end; the plot looks as follows:
This is the entire code, for both the spline and the actual fit:
from scipy.interpolate import UnivariateSpline
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import curve_fit
def func(x, ymax, n, k, c):
return ymax * x ** n / (k ** n + x ** n) + c
x=np.array([ 1.00094909, 1.08787635, 1.17481363, 1.2617564, 1.34867881, 1.43562284,
1.52259341, 1.609522, 1.69631283, 1.78276102, 1.86426648, 1.92896789,
1.9464453, 1.94941586, 2.00062852, 2.073691, 2.14982808, 2.22808316,
2.30634034, 2.38456905, 2.46280126, 2.54106611, 2.6193345, 2.69748825])
y=np.array([-0.10057627, -0.10172142, -0.10320428, -0.10378959, -0.10348456, -0.10312503,
-0.10276956, -0.10170055, -0.09778279, -0.08608644, -0.05797392, 0.00063599,
0.08732999, 0.16429878, 0.2223306, 0.25368884, 0.26830932, 0.27313931,
0.27308756, 0.27048902, 0.26626313, 0.26139534, 0.25634544, 0.2509893 ])
popt, pcov = curve_fit(func, x, y, p0=[y.max(), 2, 2, -0.1], bounds=([0, 0, 0, -0.2], [0.4, 45, 2000, 10]))
xfit = np.linspace(x.min(), x.max(), 200)
plt.scatter(x, y)
plt.plot(xfit, func(xfit, *popt))
plt.show()
s = UnivariateSpline(x, y, k=3, s=0.005)
xfit = np.linspace(x.min(), x.max(), 200)
plt.scatter(x, y)
plt.plot(xfit, s(xfit))
plt.show()
A third option is to use a more advanced function that can also reproduce the decrease at the end and differential_evolution for the fit; that seems to give the best fit:
The code is as follows (using the same data as above):
from scipy.optimize import curve_fit, differential_evolution
def sigmoid_with_decay(x, a, b, c, d, e, f):
return a * (1. / (1. + np.exp(-b * (x - c)))) * (1. / (1. + np.exp(d * (x - e)))) + f
def error_sigmoid_with_decay(parameters, x_data, y_data):
return np.sum((y_data - sigmoid_with_decay(x_data, *parameters)) ** 2)
res = differential_evolution(error_sigmoid_with_decay,
bounds=[(0, 10), (0, 25), (0, 10), (0, 10), (0, 10), (-1, 0.1)],
args=(x, y),
seed=42)
xfit = np.linspace(x.min(), x.max(), 200)
plt.scatter(x, y)
plt.plot(xfit, sigmoid_with_decay(xfit, *res.x))
plt.show()
The fit is quite sensitive regarding the bounds, so be careful when you play with that...
This illustrates the result of fitting two halves of the data to different functions, the lower half to all data with X < 2.0 and the upper half to all data with X >= 1.9, so that there is overlap in the data for the fitted curves. The code switches from one equation to another at the center of the overlap region, X = 1.95.
import numpy, matplotlib
import matplotlib.pyplot as plt
xData=numpy.array([ 1.00094909, 1.08787635, 1.17481363, 1.2617564, 1.34867881, 1.43562284,
1.52259341, 1.609522, 1.69631283, 1.78276102, 1.86426648, 1.92896789,
1.9464453, 1.94941586, 2.00062852, 2.073691, 2.14982808, 2.22808316,
2.30634034, 2.38456905, 2.46280126, 2.54106611, 2.6193345, 2.69748825])
yData=numpy.array([-0.10057627, -0.10172142, -0.10320428, -0.10378959, -0.10348456, -0.10312503,
-0.10276956, -0.10170055, -0.09778279, -0.08608644, -0.05797392, 0.00063599,
0.08732999, 0.16429878, 0.2223306, 0.25368884, 0.26830932, 0.27313931,
0.27308756, 0.27048902, 0.26626313, 0.26139534, 0.25634544, 0.2509893 ])
# function for x < 1.95 (fitted up to 2.0 for overlap)
def lowerFunc(x_in): # Bleasdale-Nelder Power With Offset
# coefficients
a = -1.1431476643503597E+03
b = 3.3819340844164983E+21
c = -6.3633178925040745E+01
d = 3.1481973843740194E+00
Offset = -1.0300724909782859E-01
temp = numpy.power(a + b * numpy.power(x_in, c), -1.0 / d)
temp += Offset
return temp
# function for x >= 1.95 (fitted down to 1.9 for overlap)
def upperFunc(x_in): # rational equation with Offset
# coefficients
a = -2.5294212380048242E-01
b = 1.4262697377369586E+00
c = -2.6141935706529118E-01
d = -8.8730045918252121E-02
Offset = -4.8283287597672708E-01
temp = (a * numpy.power(x_in, 2) + b * numpy.log(x_in)) # numerator
temp /= (1.0 + c * numpy.power(numpy.log(x_in), -1) + d * numpy.exp(x_in)) # denominator
temp += Offset
return temp
def combinedFunc(x_in):
returnVal = []
for x in x_in:
if x < 1.95:
returnVal.append(lowerFunc(x))
else:
returnVal.append(upperFunc(x))
return returnVal
modelPredictions = combinedFunc(xData)
absError = modelPredictions - yData
SE = numpy.square(absError) # squared errors
MSE = numpy.mean(SE) # mean squared errors
RMSE = numpy.sqrt(MSE) # Root Mean Squared Error, RMSE
Rsquared = 1.0 - (numpy.var(absError) / numpy.var(yData))
print('RMSE:', RMSE)
print('R-squared:', Rsquared)
##########################################################
# graphics output section
def ModelAndScatterPlot(graphWidth, graphHeight):
f = plt.figure(figsize=(graphWidth/100.0, graphHeight/100.0), dpi=100)
axes = f.add_subplot(111)
# first the raw data as a scatter plot
axes.plot(xData, yData, 'D')
# create data for the fitted equation plot
xModel = numpy.linspace(min(xData), max(xData))
yModel = combinedFunc(xModel)
# now the model as a line plot
axes.plot(xModel, yModel)
axes.set_xlabel('X Data') # X axis data label
axes.set_ylabel('Y Data') # Y axis data label
plt.show()
plt.close('all') # clean up after using pyplot
graphWidth = 800
graphHeight = 600
ModelAndScatterPlot(graphWidth, graphHeight)
I have a trajectory formed by a sequence of (x,y) pairs. I would like to interpolate points on this trajectory using splines.
How do I do this? Using scipy.interpolate.UnivariateSpline doesn't work because neither x nor y are monotonic. I could introduce a parametrization (e.g. length d along the trajectory), but then I have two dependent variables x(d) and y(d).
Example:
import numpy as np
import matplotlib.pyplot as plt
import math
error = 0.1
x0 = 1
y0 = 1
r0 = 0.5
alpha = np.linspace(0, 2*math.pi, 40, endpoint=False)
r = r0 + error * np.random.random(len(alpha))
x = x0 + r * np.cos(alpha)
y = x0 + r * np.sin(alpha)
plt.scatter(x, y, color='blue', label='given')
# For this special case, the following code produces the
# desired results. However, I need something that depends
# only on x and y:
from scipy.interpolate import interp1d
alpha_i = np.linspace(alpha[0], alpha[-1], 100)
r_i = interp1d(alpha, r, kind=3)(alpha_i)
x_i = x0 + r_i * np.cos(alpha_i)
y_i = x0 + r_i * np.sin(alpha_i)
plt.plot(x_i, y_i, color='green', label='desired')
plt.legend()
plt.show()
Using splprep you can interpolate over curves of any geometry.
from scipy import interpolate
tck,u=interpolate.splprep([x,y],s=0.0)
x_i,y_i= interpolate.splev(np.linspace(0,1,100),tck)
Which produces a plot like the one given, but only using the x and y points and not the alpha and r paramters.
Sorry about my original answer, I misread the question.