I have this EMG signal and I would like to plot the mean power frequency based on this article. I implement it in Matlab using the following code:
clear all;
close all;
EMG=load('EMG.txt');
N=1000; %my window
z=1;
fs=200 %sampling rate
for i=1:length(EMG)-N
DUM=0;
NUM=0;
FT=fft(EMG(i:i+N-1));
psd=FT.*conj(FT);
NFFT=length(fft2);
f = [1:NFFT/2]*fs/N;
for j=1:NFFT/2
NUM=NUM+f(j)*psd(j);
DUM=DUM+psd(j);
end
MPF(z)=NUM/DUM;
z=z+1;
end
And the plot of MPF is:
Following I am trying to do the same in Python. The code is:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
df = pd.read_csv('EMG.txt', names=['EMG'])
EMG=df['EMG'].tolist()
sampling_rate=200
N=1000 #my window
FT=np.fft.fft(EMG, axis=0)
psd=FT*np.conj(FT)
NFFT=len(FT)
f =(np.arange(0,NFFT/2)*sampling_rate)/N
NUM=0
DUM=0
MPF=[]
for j in np.arange(1,NFFT/2):
NUM=NUM+f[j]*psd[j]
DUM=DUM+psd[i]
MPF.append(NUM/DUM)
plt.plot(MPF)
plt.show()
And the plot of the MPF is:
Why are different?
Update
Following the advice of Dan at the comment section I modify my Python code as follow and the results more or less the same, except that the Matlab code is way faster than Python which in my case was running out of memory:
sampling_rate=200
N=1000
MPF=[]
for i in range(0,len(EMG)-N):
signal=EMG[i:(i+N)]
FT=np.fft.fft(signal, axis=0)
psd=FT*np.conj(FT)
NFFT=len(FT)
f =(np.arange(0,NFFT/2)*sampling_rate)/N
D_1=0
N_1=0
for j in np.arange(1,NFFT/2):
D_1=D_1+f[j]*psd[j]
N_1=N_1+psd[j]
MPF.append(D_1/N_1)
plt.plot(MPF)
plt.show()
Choosing the first 22000 samples the results are:
'detrend' option would cause the difference of the first value (0 Hz or DC).
The default setting is 'constant', which removes DC component from the original data.
f, Pxx_den = signal.periodogram(x, fs, detrend=False)
I am going on with this question and answer.
1) In Matlab, try to use the built-in function meanfreq, and medfreq will also help you to find the median frequency.
2) Until now, I have not found a package to solve the mean/median frequency with Python.
3) I also tried to solve the mean/median frequency with Python.
And I follow the computation steps in Matlab meanfreq. (In Matlab command windows, type edit meanfreq.m, you could see the source code of meanfreq. The source code would help a lot.
4) The following is my python-based implementation of meanfreq
from scipy import signal
def meanfreq(x, fs):
f, Pxx_den = signal.periodogram(x, fs)
Pxx_den = np.reshape( Pxx_den, (1,-1) )
width = np.tile(f[1]-f[0], (1, Pxx_den.shape[1]))
f = np.reshape(f, (1, -1))
P = Pxx_den * width
pwr = np.sum(P)
mnfreq = np.dot(P, f.T)/pwr
return mnfreq
5) You can debug these source code of both mine and Matlab.
6) There is also a small 'bug' in my code, or not mine. When you debug them, you may find the first (only the first value) of Pxx is totally different. The rest of values are identical.
I am also very confused here. Help me. :)
Related
I have an integration equations to calculate key rate and need to convert it into Python.
The equation to calculate key rate is given by:
where R(n) is:
and p(n)dn is:
The key rate should be plotted like this:
I have sucessfully plotted the static model of the graph using following equation:
import numpy as np
import math
from math import pi,e,log
import matplotlib.pyplot as plt
n1=np.arange(10, 55, 1)
n=10**(-n1/10)
Y0=1*(10**-5)
nd=0.25
ed=0.03
nsys=nd*n
QBER=((1/2*Y0)+(ed*nsys))/(Y0+nsys)
H2=-QBER*np.log2(QBER)-(1-QBER)*np.log2(1-QBER)
Rsp=np.log10((Y0+nsys)*(1-(2*H2)))
print (Rsp)
plt.plot(n1,Rsp)
plt.xlabel('Loss (dB)')
plt.ylabel('log10(Rate)')
plt.show()
However, I failed to plot the R^ratewise model. This is my code:
import numpy as np
import matplotlib.pyplot as plt
def h2(x):
return -x*np.log2(x)-(1-x)*np.log2(1-x)
e0=0.5
ed=0.03
Y0=1e-5
nd=0.25
nt=np.linspace(0.1,0.00001,1000)
y=np.zeros(np.size(nt))
Rate=np.zeros(np.size(nt))
eta_0=0.0015
for (i,eta) in enumerate(nt):
nsys=eta*nd
sigma=0.9
y[i]=1/(eta*sigma*np.sqrt(2*np.pi))*np.exp(-(np.log(eta/eta_0)+(1/2*sigma*sigma))**2/(2*sigma*sigma))
Rate[i]=(max(0.0,(Y0+nsys)*(1-2*h2(min(0.5,(e0*Y0+ed*nsys)/(Y0+nsys))))))*y[i]
plt.plot(nt,np.log10(Rate))
plt.xlabel('eta')
plt.ylabel('Rate')
plt.show()
Hopefully that anyone can help me to code the key rate with integration p(n)dn as stated above. This is the paper for referrence:
key rate
Thank you.
I copied & ran your second code block as-is, and it generated a plot. Is that what you wanted?
Using y as the p(n) in the equation, and the Rsp as the R(n), you should be able to use
NumPy's trapz function
to approximate the integral from the sampled p(n) and R(n):
n = np.linspace(0, 1, no_of_samples)
# ...generate y & Rst from n...
R_rate = np.trapz(y * Rst, n)
However, you'll have to change your code to sample y & Rst using the same n, spanning from 0 to 1`.
P.S. there's no need for the loop in your second code block; it can be condensed by removing the i's, swapping eta for nt, and using NumPy's minimum and maximum functions, like so:
nsys=nt*nd
sigma=0.9
y=1/(nt*sigma*np.sqrt(2*np.pi))*np.exp(-(np.log(nt/eta_0)+(1/2*sigma*sigma))**2/(2*sigma*sigma))
Rate=(np.maximum(0.0,(Y0+nsys)*(1-2*h2(np.minimum(0.5,(e0*Y0+ed*nsys)/(Y0+nsys))))))*y
I have a Matlab script to compute the DFT of a signal and plot it:
(data can be found here)
clc; clear; close all;
fid = fopen('s.txt');
txt = textscan(fid,'%f');
s = cell2mat(txt);
nFFT = 100;
fs = 24000;
deltaF = fs/nFFT;
FFFT = [0:nFFT/2-1]*deltaF;
win = hann(length(s));
sw = s.*win;
FFT = fft(sw, nFFT)/length(s);
FFT = [FFT(1); 2*FFT(2:nFFT/2)];
absFFT = 20*log10(abs(FFT));
plot(FFFT, absFFT)
grid on
I am trying to translate it to Python and can't get the same result.
import numpy as np
from matplotlib import pyplot as plt
x = np.genfromtxt("s.txt", delimiter=' ')
nfft = 100
fs = 24000
deltaF = fs/nfft;
ffft = [n * deltaF for n in range(nfft/2-1)]
ffft = np.array(ffft)
window = np.hanning(len(x))
xw = np.multiply(x, window)
fft = np.fft.fft(xw, nfft)/len(x)
fft = fft[0]+ [2*fft[1:nfft/2]]
fftabs = 20*np.log10(np.absolute(fft))
plt.figure()
plt.plot(ffft, np.transpose(fftabs))
plt.grid()
The plots I get (Matlab on the left, Python on the right):
What am I doing wrong?
Both codes are different in one case you concatenate two lists
FFT = [FFT(1); 2*FFT(2:nFFT/2)];
in the matlab code
in the other you add the first value of fft with the rest of the vector
fft = fft[0]+ [2*fft[1:nfft/2]]
'+' do not concatenate here because you have numpy array
In python, it should be:
fft = fft[0:nfft/2]
fft[1:nfft/2] = 2*fft[1:nfft/2]
I am not a Mathlab user so I am not sure but there are few things I'd ask to see if I can help you.
You called np.array after array has been made (ffft). That probably will not change the nature of array as well as you hoped, perhaps it would be better to try to define it inside np.array(n * deltaF for n in range(nfft/2-1)) I am not sure of formatting but you get the idea. The other thing is that the range doesn't seem right to me. You want it to have a value of 49?
Another one is the fft = fft[0]+ [2*fft[1:nfft/2]] compared to FFT = [FFT(1); 2*FFT(2:nFFT/2)]; I am not sure if the comparsion is accurate or not. It just seemed to be a different type of definition to me?
Also, when I do these type of calculations, I 'print' out the intermediate steps so I can compare the numbers to see where it breaks.
Hope this helps.
I found out that using np.fft.rfft instead of np.fft.fft and modifying the code as following does the job :
import numpy as np
from matplotlib import pyplot as pl
x = np.genfromtxt("../Matlab/s.txt", delimiter=' ')
nfft = 100
fs = 24000
deltaF = fs/nfft;
ffft = np.array([n * deltaF for n in range(nfft/2+1)])
window = np.hanning(len(x))
xw = np.multiply(x, window)
fft = np.fft.rfft(xw, nfft)/len(x)
fftabs = 20*np.log10(np.absolute(fft))
pl.figure()
pl.plot(np.transpose(ffft), fftabs)
pl.grid()
The resulting plot :
right result with Python
I can see that the first and the last points, as well as the amplitudes are not the same. It isn't a problem for me (I am more interested in the general shape), but if someone can explain, I'd be happy.
I have a program in MATLAB which I want to port to Python. The problem is that in it I use the built-in spectrogram function and, although the matplotlib specgram function seems identical, I'm getting different results when I run both.
These is the code I've been running.
MATLAB:
data = 1:999; %Dummy data. Just for testing.
Fs = 8000; % All the songs we'll be working on will be sampled at an 8KHz rate
tWindow = 64e-3; % The window must be long enough to get 64ms of the signal
NWindow = Fs*tWindow; % Number of elements the window must have
window = hamming(NWindow); % Window used in the spectrogram
NFFT = 512;
NOverlap = NWindow/2; % We want a 50% overlap
[S, F, T] = spectrogram(data, window, NOverlap, NFFT, Fs);
Python:
import numpy as np
from matplotlib import mlab
data = range(1,1000) #Dummy data. Just for testing
Fs = 8000
tWindow = 64e-3
NWindow = Fs*tWindow
window = np.hamming(NWindow)
NFFT = 512
NOverlap = NWindow/2
[s, f, t] = mlab.specgram(data, NFFT = NFFT, Fs = Fs, window = window, noverlap = NOverlap)
And this is the result I get in both executions:
http://i.imgur.com/QSPvYsC.png
(The F and T variables are exactly the same in both programs)
It's obvious that they're different; in fact, the Python execution even doesn't return complex numbers. What could be the problem? Is there any way to fix it or I should use another spectrogram function?
Thank you so much in advance for your help.
In matplotlib, specgram by default returns the power spectral density (mode='PSD'). In MATLAB, spectrogram by default returns the short-time fourier transform, unless nargout==4, in which case it also computes the PSD. To get the matplotlib behaviour to match the MATLAB behaviour, set mode='complex'
Dear fellow coders and science guys :)
I am using python with numpy and matplotlib to simulate a perceptron, proud to say it works pretty well.
I used python even tough I've never seen it before, cause I heard matplotlib offered amazing graph visualisation capabilities.
Using functions below I get a 2d array that looks like this:
[[aplha_1, 900], [alpha_2], 600, .., [alpha_99, 900]
So I get this 2D array and would love to write a function that would enable me to analyze the convergence.
I am looking for something that will easily and intuitively (don't have time to study a whole new library for 5 hours now) draw a function like this sketch:
def get_convergence_for_alpha(self, _alpha):
epochs = []
for i in range(0, 5):
epochs.append(self.perceptron_algorithm())
self.weights = self.generate_weights()
avg = sum(epochs, 0) / len(epochs)
res = [_alpha, avg]
return res
And this is the whole generation function.
def alpha_convergence_function(self):
res = []
for i in range(1, 100):
res.append(self.get_convergence_for_alpha(i / 100))
return res
Is this easily doable?
You can convert your nested list to a 2d numpy array and then use slicing to get the alphas and epoch counts (just like in matlab).
import numpy as np
import matplotlib.pyplot as plt
# code to simulate the perceptron goes here...
res = your_object.alpha_convergence_function()
res = np.asarray(res)
print('array size:', res.shape)
plt.xkcd() # so you get the sketchy look :)
# first column -> x-axis, second column -> y-axis
plt.plot(res[:,0], res[:,1])
plt.show()
Remove the plt.xkcd() line if you don't actually want the plot to look like a sketch...
In my application, the data data is sampled on a distorted grid, and I would like to resample it to a nondistorted grid. In order to test this, I wrote this program with examplary distortions and a simple function as data:
from __future__ import division
import numpy as np
import scipy.interpolate as intp
import pylab as plt
# Defining some variables:
quadratic = -3/128
linear = 1/16
pn = np.poly1d([quadratic, linear,0])
pixels_x = 50
pixels_y = 30
frame = np.zeros((pixels_x,pixels_y))
x_width= np.concatenate((np.linspace(8,7.8,57) , np.linspace(7.8,8,pixels_y-57)))
def data(x,y):
z = y*(np.exp(-(x-5)**2/3) + np.exp(-(x)**2/5) + np.exp(-(x+5)**2))
return(z)
# Generating grid coordinates
yt = np.arange(380,380+pixels_y*4,4)
xt = np.linspace(-7.8,7.8,pixels_x)
X, Y = np.meshgrid(xt,yt)
Y=Y.T
X=X.T
Y_m = np.zeros((pixels_x,pixels_y))
X_m = np.zeros((pixels_x,pixels_y))
# generating distorted grid coordinates:
for i in range(pixels_y):
Y_m[:,i] = Y[:,i] - pn(xt)
X_m[:,i] = np.linspace(-x_width[i],x_width[i],pixels_x)
# Sample data:
for i in range(pixels_y):
for j in range(pixels_x):
frame[j,i] = data(X_m[j,i],Y_m[j,i])
Y_m = Y_m.flatten()
X_m = X_m.flatten()
frame = frame.flatten()
##
Y = Y.flatten()
X = X.flatten()
ipf = intp.interp2d(X_m,Y_m,frame)
interpolated_frame = ipf(xt,yt)
At this point, I have to questions:
The code works, but I get the the following warning:
Warning: No more knots can be added because the number of B-spline coefficients
already exceeds the number of data points m. Probably causes: either
s or m too small. (fp>s)
kx,ky=1,1 nx,ny=54,31 m=1500 fp=0.000006 s=0.000000
Also, some interpolation artifacts appear, and I assume that they are related to the warning - Do you guys know what I am doing wrong?
For my actual applications, the frames need to be around 500*100, but when doing this, I get a MemoryError - Is there something I can do to help that, apart from splitting the frame into several parts?
Thanks!
This problem is most likely related to the usage of bisplrep and bisplev within interp2d. The docs mention that they use a smooting factor of s=0.0 and that bisplrep and bisplev should be used directly if more control over s is needed. The related docs mention that s should be found between (m-sqrt(2*m),m+sqrt(2*m)) where m is the number of points used to construct the splines. I had a similar problem and found it solved when using bisplrep and bisplev directly, where s is only optional.
For 2d interpolation,
griddata
is solid, local, fast.
Take a look at problem-with-2d-interpolation-in-scipy-non-rectangular-grid on SO.
You might want to look at the following interp method in basemap:
mpl_toolkits.basemap.interp
http://matplotlib.sourceforge.net/basemap/doc/html/api/basemap_api.html
unless you really need spline-based interpolation.