I am look for a way to obtain the frequency from a signal. Here's an example:
signal = [numpy.sin(numpy.pi * x / 2) for x in range(1000)]
This Array will represent the sample of a recorded sound (x = miliseconds)
sin(pi*x/2) => 250 Hrz
How can we go from the signal (list of points), to obtaining the frequencies form this array?
Note:
I have read many Stackoverflow threads and watch many youtube videos. I am yet to find an answer. Please use simple words.
(I am Thankfull for every answer)
What you're looking for is known as the Fourier Transform
A bit of background
Let's start with the formal definition:
The Fourier transform (FT) decomposes a function (often a function of time, or a signal) into its constituent frequencies
This is in essence a mathematical operation that when applied over a signal, gives you an idea of how present each frequency is in the time series. In order to get some intuition behind this, it might be helpful to look at the mathematical definition of the DFT:
Where k here is swept all the way up t N-1 to calculate all the DFT coefficients.
The first thing to notice is that, this definition resembles somewhat that of the correlation of two functions, in this case x(n) and the negative exponential function. While this may seem a little bit abstract, by using Euler's formula and by playing a bit around with the definition, the DFT can be expressed as the correlation with both a sine wave and a cosine wave, which will account for the imaginary and the real parts of the DFT.
So keeping in mind that this is in essence computing a correlation, whenever a corresponding sine or cosine from the decomposition of the complex exponential matches with that of x(n), there will be a peak in X(K), meaning that, such frequency is present in the signal.
How can we do the same with numpy?
So having given a very brief theoretical background, let's consider an example to see how this can be implemented in python. Lets consider the following signal:
import numpy as np
import matplotlib.pyplot as plt
Fs = 150.0; # sampling rate
Ts = 1.0/Fs; # sampling interval
t = np.arange(0,1,Ts) # time vector
ff = 50; # frequency of the signal
y = np.sin(2*np.pi*ff*t)
plt.plot(t, y)
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.show()
Now, the DFT can be computed by using np.fft.fft, which as mentioned, will be telling you which is the contribution of each frequency in the signal now in the transformed domain:
n = len(y) # length of the signal
k = np.arange(n)
T = n/Fs
frq = k/T # two sides frequency range
frq = frq[:len(frq)//2] # one side frequency range
Y = np.fft.fft(y)/n # dft and normalization
Y = Y[:n//2]
Now, if we plot the actual spectrum, you will see that we get a peak at the frequency of 50Hz, which in mathematical terms it will be a delta function centred in the fundamental frequency of 50Hz. This can be checked in the following Table of Fourier Transform Pairs table.
So for the above signal, we would get:
plt.plot(frq,abs(Y)) # plotting the spectrum
plt.xlabel('Freq (Hz)')
plt.ylabel('|Y(freq)|')
plt.show()
Related
I'm trying to do some tests before I proceed analyzing some real dataset via FFT, and I've found the following problem.
First, I create a signal as the sum of two cosines and then use rfft to to the transformation (since it has only real values):
import numpy as np
import matplotlib.pyplot as plt
from scipy.fft import rfft, rfftfreq
# Number of sample points
N = 800
# Sample spacing
T = 1.0 / 800.0
x = np.linspace(0.0, N*T, N)
y = 0.5*np.cos(10*2*np.pi*x) + 0.5*np.cos(200*2*np.pi*x)
# FFT
yf = rfft(y)
xf = rfftfreq(N, T)
fig, ax = plt.subplots(1,2,figsize=(15,5))
ax[0].plot(x,y)
ax[1].plot(xf, 2.0/N*np.abs(yf))
As it can be seen from the definition of the signal, I have two oscillations with amplitude 0.5 and frequency 10 and 200. Now, I would expect the FFT spectrum to be something like two deltas at those points, but apparently increasing the frequency broadens the peaks:
From the first peak it can be infered that the amplitude is 0.5, but not for the second. I've tryied to obtain the area under the peak using np.trapz and use that as an estimate for the amplitude, but as it is close to a dirac delta it's very sensitive to the interval I choose. My problem is that I need to get the amplitude as exact as possible for my data analysis.
EDIT: As it seems to be something related with the number of points, I decided to increment (now that I can) the sample frequency. This seems to solve the problem, as it can be seen in the figure:
However, it still seems strange that for a certain number of points and sample frequency, the high frequency peaks broaden...
It is not strange , you have leakage of the frequency bins. When you discretize the signal (sampling) needed for the Fourier transfrom , frequency bins are created which are frequency intervals where the the amplitude is calculated. And each bin has wide which is given by the sample_rate / num_points . So , the less the number of bins the more difficult is to assign precise amplitudes to every frequency. Other problems in choosing the best sampling rate exist such as the shannon-nyquist theorem to prevent aliasing. https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem . But depending on the problem sometimes there some custom rates used for sampling. E.g. when dealing with audio a sampling rate of 44,100 Hz is widely used , cause is based on the limits of the human hearing. So it depends also on nature of the data you want to perform analysis as you wrote. Anyway , since this question has also theoretical value , you can also check https://dsp.stackexchange.com for some useful info.
I would comment to George's answer, but yet I cannot.
Maybe a starting point for your research are the properties of the Discrete Fourier Transform.
The signal in the time domain is actual the cosines multiplied by a box window which transforms into the frequency domain as the convolution of the deltas with the sinc function. The sinc functions will smear the spectrum.
However, I am not sure we are observing spectral leakage here, since the window fits exactly to the full period of cosines. The discretization of the bins might still play a role here.
I have already tried histogram equalization based on image and it works just fine.
But now I want to implement the same approach using audio frequency instead of image gray scale. Which means I would like to make the spectrum flatter. The sampling rate I use is 44.1kHz and want to make the frequency evenly spread to range 0-22050Hz, but the peak is still the highest.
Here is the spectrum:
And this is what I have tried:
I think the original histogram I plot is already wrong, I can't count the number of occurrences per frequency, or maybe I shouldn't do this at all. Somebody told me I need to use fft() but I have no idea how to do it.
Any help would be appreciated! Thanks
Here is the code for how I plot the spectrum :
import librosa
import numpy as np
import matplotlib.pyplot as plt
import math
file = 'example.wav'
y, sr = librosa.load(file, sr=None)
n_fft = 2048
S = librosa.stft(y, n_fft=n_fft, hop_length=n_fft//2)
S = abs(S)
D_AVG = np.mean(S, axis=1)
plt.figure(figsize=(25, 12))
plt.bar(np.arange(D_AVG.shape[0]), D_AVG)
x_ticks_positions = [n for n in range(0, n_fft // 2, n_fft // 16)]
x_ticks_labels = [str(sr / 2048 * n) + 'Hz' for n in x_ticks_positions]
plt.xticks(x_ticks_positions, x_ticks_labels)
plt.xlabel('Frequency')
plt.ylabel('dB')
plt.savefig('spectrum.png')
"Equalization" in the sense of making a flat frequency spectrum is usually done by a whitening transformation. This post on dsp.stackexchange might also be helpful. As Mark mentioned, this spectral equalization is different from histogram equalization in image processing.
Equalizing/whitening the spectrum of a signal:
Estimate the PSD. Given an array of samples x with sample rate fs, you can compute a robust estimate of the power spectral density (PSD) with scipy.signal.welch:
f, psd = scipy.signal.welch(x, fs=fs)
This function performs Welch's method. Basically, it divides up the signal into several segments, does FFT on each one, and averages the power spectra to get a good estimate of how much power x has at each frequency on average. The point of all this is it gets a more reliable frequency characterization than just taking one FFT of x as a whole.
Compute equalizer gain. Use eq_gain = 1 / (1e-6 + psd)**0.5, or something similar, to determine the gain of the equalizer. The 1e-6 denominator offset is to avoid division by zero. It often happens that the PSD extremely small for some frequencies because, say, x went through an anti-aliasing filter that made some high frequency powers nearly zero.
Apply the equalizer gain. Finally, eq_gain needs to be applied to the signal x to equalize it. There are many ways this could be done, but one way is to use scipy.signal.firwin2 to turn the gains into an FIR filter,
eq_filter = scipy.signal.firwin2(99, f, eq_gain, fs=fs)
and use a convolution or scipy.signal.lfilter to apply the filter to x. You can then use scipy.signal.welch again to check that the PSD is flatter than before.
I have 3d-array of accelerator signal data which sampled in 50 Hz meaning that the time step is 1/50=.02. My goal is to compute the main frequency of this sensor using Numpy or Scipy. My question is that should I compute the frequency of each column separately, using multidimensional fft or computing single Vector and then compute fft.
I used the following function to compute the main frequency.
from scipy import fftpack
import numpy as np
def fourier(signal, timestep):
data = signal - np.mean(signal)
N = len(data) // 2 # we need half of data
freq = fftpack.fftfreq(len(data), d=timestep)[:N]
fft = fftpack.fft(data)[:N]
amp = np.abs(fft) / N
order = np.argsort(amp)[::-1] ## sort based on the importance
return freq[order][0]
A 3D array of accelerometer sensors produces an array of 5 dimensions: the space coordinates, time and the components of the acceleration.
Taking the DFT over the time dimension corresponds to analysing sensors one at a time: each sensor would produce a main frequency, likely slightly different from one sensor to another, as if the sensors were uncoupled.
As an alternative, let's think about taking the DFT over both spacial coordinates and time. It corresponds to writing the compound signal as a sum of sinusoidal plane waves:
where Ǹ is a scaling factor obtained by multiplying the number of points to the number of time samples. In the sequel, I'll drop this global scaling independent from x,y,z,t,k_x,k_y,k_z and w.
At this point, modeling the physics generating this acceleration would be a significant asset. Indeed, using this DFT makes little sense if the phenomenon is dispersive. Nevetheless, the diffusion, elasticity or acoustics in an uniform material are non-dispersive: each frequency lives indepently from the others. Furthermore, knowing the physics is useful as an energy can be defined. For instance, the kinetic energy associated to the wave k_x,k_y,k_z,w writes:
Therefore, the kinetic energy associated to a given frequency w writes:
As a consequence, this reasoning provides a physically-based way to merge the pointwise DFTs over time . Indeed, according to the Parseval's identity:
Regarding practical considerations, substracting the average as you did is indeed a good start. If computing the velocity is considered by multiplying by 1/w^2, the zero frequency (i.e. the average) is to be zeroed, to avoid occurence of infinite or Nan.
Moreover, applying a window prior to computing the time DFT could help limit problems related to spectral leakage. DFT is designed for periodic signals of periods consistent with that of the frame. More specifically, it computes the Fourier transform of a signal built by repeating your frame again and again. As a consequence, artifical discontinuities may appear at the edges, inducing misleading non-existing frequencies. Windows drops near zero close to the edge of the frame, thus reducing the discontinuities and their effect. As a consequence, it could be suggested to apply a window to the space dimensions as well, to keep the consistency with the physical plane wave decomposition. It would result in giving more weight to the accelerators at the center of the 3D array.
The plane wave decomposition also requires that the spacial spacing of the sensor must be about twice smaller than the expected wavelength. Otherwise, another phenomenon called aliasing occurs. Nevertheless, the power spectrum W(w) might be less sensitive to this issue than the plane wave decomposition. On the contrary, if the elastic strain energy is computed starting from the acceleration, aliasing could become a real problem, because computing the strain requires derivative with respect to space coordinates, i.e. multiplication by k_x, k_y or k_z, and space aliasing corresponds to using the wrong k_x.
Once W(w) is computed, the frequencies corresponding to each peak can be estimated by computing the mean frequency over the peak with respect to power density as in Why are frequency values rounded in signal using FFT? .
Here is a sample code generating some plane waves of frequencies not consistent with the size of the frame (both time and space). Hanning windows are applied, the kinetic energy is computed and the frequencies corresponding to each peak are retreived.
import matplotlib.pyplot as plt
import numpy as np
from scipy import signal
import scipy
spacingx=1.
spacingy=1.
spacingz=1.
spacingt=1./50.
Nx=5
Ny=5
Nz=5
Nt=512
frequency1=9.5
frequency2=13.7
frequency3=22.3
#building a signal
acc=np.zeros((Nx,Ny,Nz,Nt,3))
for i in range(Nx):
for j in range(Ny):
for k in range(Nz):
for l in range(Nt):
acc[i,j,k,l,0]=np.sin(i*spacingx+j*spacingy-2*np.pi*frequency1*l*spacingt)
acc[i,j,k,l,1]=np.sin(i*spacingx+1.5*k*spacingz-2*np.pi*frequency2*l*spacingt)
acc[i,j,k,l,2]=np.sin(1.5*i*spacingx+k*spacingz-2*np.pi*frequency3*l*spacingt)
#applying a window both in time and space
hanningx=np.hanning(Nx)
hanningy=np.hanning(Ny)
hanningz=np.hanning(Nz)
hanningt=np.hanning(Nt)
for i in range(Nx):
hx=hanningx[i]
for j in range(Ny):
hy=hanningy[j]
for k in range(Nz):
hz=hanningx[k]
for l in range(Nt):
ht=hanningt[l]
acc[i,j,k,l,0]*=hx*hy*hz*ht
acc[i,j,k,l,1]*=hx*hy*hz*ht
acc[i,j,k,l,2]*=hx*hy*hz*ht
#computing the DFT over time.
acctilde=np.fft.fft(acc,axis=3)
#kinetic energy
print acctilde.shape[3]
kineticW=np.zeros(acctilde.shape[3])
frequencies=np.fft.fftfreq(Nt, spacingt)
for l in range(Nt):
oneonomegasquared=0.
if l>0:
oneonomegasquared=1.0/(frequencies[l]*frequencies[l])
for i in range(Nx):
for j in range(Ny):
for k in range(Nz):
kineticW[l]+= oneonomegasquared*(np.real(np.vdot(acctilde[i,j,k,l,:],acctilde[i,j,k,l,:])))
plt.plot(frequencies[0:acctilde.shape[3]],kineticW,'k-',label=r'$W(f)$')
#plt.plot(xi,np.real(fourier),'k-', lw=3, color='red', label=r'$f$, Hz')
plt.legend()
plt.show()
# see https://stackoverflow.com/questions/54714169/why-are-frequency-values-rounded-in-signal-using-fft/54775867#54775867
peaks, _= signal.find_peaks(kineticW, height=np.max(kineticW)*0.1)
print "potential frequencies index", peaks
#compute the mean frequency of the peak with respect to power density
powerpeak=np.zeros(len(peaks))
powerpeaktimefrequency=np.zeros(len(peaks))
for i in range(len(kineticW)):
dist=1000
jnear=0
for j in range(len(peaks)):
if dist>np.abs(i-peaks[j]):
dist=np.abs(i-peaks[j])
jnear=j
powerpeak[jnear]+=kineticW[i]
powerpeaktimefrequency[jnear]+=kineticW[i]*frequencies[i]
powerpeaktimefrequency=np.divide(powerpeaktimefrequency,powerpeak)
print 'corrected frequencies', powerpeaktimefrequency
Is there a way to generate a quasi periodic signal (a signal with a specific frequency distribution, like a normal distribution)? In addition,
the signal should not have a stationary frequency distribution since the inverse Fourier transform of a Gaussian function is still a Gaussian function, while what I want is an oscillating signal.
I used a discrete series of Normally distributed frequencies to generate the signal, that is
The frequencies distribute like this:
So with initial phases
, I got the signal
However, the signal is like
and its FFT spectrum is like
.
I found that the final spectrum is only similar to a Gaussian function within a short time period since t=0 (corresponding to the left few peaks in figure4 which are extremely high), and the rest of the signal only contributed to the glitches on both sides of the peak in figure5.
I thought the problem may have come from the initial phases. I tried randomly distributed initial phases but it also didn't work.
So, what is the right way to generate such a signal?
Here is my python code:
import numpy as np
from scipy.special import erf, erfinv
def gaussian_frequency(array_length = 10000, central_freq = 100, std = 10):
n = np.arange(array_length)
f = np.sqrt(2)*std*erfinv(2*n/array_length - erf(central_freq/np.sqrt(2)/std)) + central_freq
return f
f = gaussian_frequency()
phi = np.linspace(0,2*np.pi, len(f))
t = np.linspace(0,100,100000)
signal = np.zeros(len(t))
for k in range(len(f)):
signal += np.sin(phi[k] + 2*np.pi*f[k]*t)
def fourierPlt(signal, TIMESTEP = .001):
num_samples = len(signal)
k = np.arange(num_samples)
Fs = 1/TIMESTEP
T = num_samples/Fs
frq = k/T # two sides frequency range
frq = frq[range(int(num_samples/2))] # one side frequency range
fourier = np.fft.fft(signal)/num_samples # fft computing and normalization
fourier = abs(fourier[range(int(num_samples/2))])
fourier = fourier/sum(fourier)
plt.plot(frq, fourier, 'r', linewidth = 1)
plt.title("Fast Fourier Transform")
plt.xlabel('$f$/Hz')
plt.ylabel('Normalized Spectrum')
return(frq, fourier)
fourierPlt(signal)
If you want your signal to be real-valued, you need to mirror the frequency component: you need the positive and negative frequencies to be complex conjugates of each other. I presume you thought of this.
A Gaussian-shaped frequency spectrum (with mean at f=0) yields a Gaussian-shaped signal.
Shifting the frequency spectrum by a frequency f0 causes the time-domain signal to be multiplied by exp(j 2 π f0 t). That is, you only change its phase.
Assuming you still want a real-valued time signal, you'll have to duplicate the frequency spectrum and shift it in both directions. This causes a multiplication by
exp(j 2 π f0 t)+exp(-j 2 π f0 t) = 2 cos(2 π f0 t) .
Thus, your signal is a Gaussian modulating a cosine.
I'm using MATLAB here for the example, I hope you can easily translate this to Python:
t=0:300;
s=exp(-(t-150).^2/30.^2) .* cos(2*pi*0.1*t);
subplot(2,1,1)
plot(t,s)
xlabel('time')
S=abs(fftshift(fft(s)));
f=linspace(-0.5,0.5,length(S));
subplot(2,1,2)
plot(f,S)
xlabel('frequency')
For those interested in image processing: the Gabor filter is exactly this, but with the frequency spectrum shifted only one direction. The resulting filter is complex, the magnitude of the filtering result is used. This leads to a phase-independent filter.
I am trying to use a fast fourier transform to extract the phase shift of a single sinusoidal function. I know that on paper, If we denote the transform of our function as T, then we have the following relations:
However, I am finding that while I am able to accurately capture the frequency of my cosine wave, the phase is inaccurate unless I sample at an extremely high rate. For example:
import numpy as np
import pylab as pl
num_t = 100000
t = np.linspace(0,1,num_t)
dt = 1.0/num_t
w = 2.0*np.pi*30.0
phase = np.pi/2.0
amp = np.fft.rfft(np.cos(w*t+phase))
freqs = np.fft.rfftfreq(t.shape[-1],dt)
print (np.arctan2(amp.imag,amp.real))[30]
pl.subplot(211)
pl.plot(freqs[:60],np.sqrt(amp.real**2+amp.imag**2)[:60])
pl.subplot(212)
pl.plot(freqs[:60],(np.arctan2(amp.imag,amp.real))[:60])
pl.show()
Using num=100000 points I get a phase of 1.57173880459.
Using num=10000 points I get a phase of 1.58022110476.
Using num=1000 points I get a phase of 1.6650441064.
What's going wrong? Even with 1000 points I have 33 points per cycle, which should be enough to resolve it. Is there maybe a way to increase the number of computed frequency points? Is there any way to do this with a "low" number of points?
EDIT: from further experimentation it seems that I need ~1000 points per cycle in order to accurately extract a phase. Why?!
EDIT 2: further experiments indicate that accuracy is related to number of points per cycle, rather than absolute numbers. Increasing the number of sampled points per cycle makes phase more accurate, but if both signal frequency and number of sampled points are increased by the same factor, the accuracy stays the same.
Your points are not distributed equally over the interval, you have the point at the end doubled: 0 is the same point as 1. This gets less important the more points you take, obviusly, but still gives some error. You can avoid it totally, the linspace has a flag for this. Also it has a flag to return you the dt directly along with the array.
Do
t, dt = np.linspace(0, 1, num_t, endpoint=False, retstep=True)
instead of
t = np.linspace(0,1,num_t)
dt = 1.0/num_t
then it works :)
The phase value in the result bin of an unrotated FFT is only correct if the input signal is exactly integer periodic within the FFT length. Your test signal is not, thus the FFT measures something partially related to the phase difference of the signal discontinuity between end-points of the test sinusoid. A higher sample rate will create a slightly different last end-point from the sinusoid, and thus a possibly smaller discontinuity.
If you want to decrease this FFT phase measurement error, create your test signal so the your test phase is referenced to the exact center (sample N/2) of the test vector (not the 1st sample), and then do an fftshift operation (rotate by N/2) so that there will be no signal discontinuity between the 1st and last point in your resulting FFT input vector of length N.
This snippet of code might help:
def reconstruct_ifft(data):
"""
In this function, we take in a signal, find its fft, retain the dominant modes and reconstruct the signal from that
Parameters
----------
data : Signal to do the fft, ifft
Returns
-------
reconstructed_signal : the reconstructed signal
"""
N = data.size
yf = rfft(data)
amp_yf = np.abs(yf) #amplitude
yf = yf*(amp_yf>(THRESHOLD*np.amax(amp_yf)))
reconstructed_signal = irfft(yf)
return reconstructed_signal
The 0.01 is the threshold of amplitudes of the fft that you would want to retain. Making the THRESHOLD greater(more than 1 does not make any sense), will give
fewer modes and cause higher rms error but ensures higher frequency selectivity.
(Please adjust the TABS for the python code)