Is there a way to generate a quasi periodic signal (a signal with a specific frequency distribution, like a normal distribution)? In addition,
the signal should not have a stationary frequency distribution since the inverse Fourier transform of a Gaussian function is still a Gaussian function, while what I want is an oscillating signal.
I used a discrete series of Normally distributed frequencies to generate the signal, that is
The frequencies distribute like this:
So with initial phases
, I got the signal
However, the signal is like
and its FFT spectrum is like
.
I found that the final spectrum is only similar to a Gaussian function within a short time period since t=0 (corresponding to the left few peaks in figure4 which are extremely high), and the rest of the signal only contributed to the glitches on both sides of the peak in figure5.
I thought the problem may have come from the initial phases. I tried randomly distributed initial phases but it also didn't work.
So, what is the right way to generate such a signal?
Here is my python code:
import numpy as np
from scipy.special import erf, erfinv
def gaussian_frequency(array_length = 10000, central_freq = 100, std = 10):
n = np.arange(array_length)
f = np.sqrt(2)*std*erfinv(2*n/array_length - erf(central_freq/np.sqrt(2)/std)) + central_freq
return f
f = gaussian_frequency()
phi = np.linspace(0,2*np.pi, len(f))
t = np.linspace(0,100,100000)
signal = np.zeros(len(t))
for k in range(len(f)):
signal += np.sin(phi[k] + 2*np.pi*f[k]*t)
def fourierPlt(signal, TIMESTEP = .001):
num_samples = len(signal)
k = np.arange(num_samples)
Fs = 1/TIMESTEP
T = num_samples/Fs
frq = k/T # two sides frequency range
frq = frq[range(int(num_samples/2))] # one side frequency range
fourier = np.fft.fft(signal)/num_samples # fft computing and normalization
fourier = abs(fourier[range(int(num_samples/2))])
fourier = fourier/sum(fourier)
plt.plot(frq, fourier, 'r', linewidth = 1)
plt.title("Fast Fourier Transform")
plt.xlabel('$f$/Hz')
plt.ylabel('Normalized Spectrum')
return(frq, fourier)
fourierPlt(signal)
If you want your signal to be real-valued, you need to mirror the frequency component: you need the positive and negative frequencies to be complex conjugates of each other. I presume you thought of this.
A Gaussian-shaped frequency spectrum (with mean at f=0) yields a Gaussian-shaped signal.
Shifting the frequency spectrum by a frequency f0 causes the time-domain signal to be multiplied by exp(j 2 π f0 t). That is, you only change its phase.
Assuming you still want a real-valued time signal, you'll have to duplicate the frequency spectrum and shift it in both directions. This causes a multiplication by
exp(j 2 π f0 t)+exp(-j 2 π f0 t) = 2 cos(2 π f0 t) .
Thus, your signal is a Gaussian modulating a cosine.
I'm using MATLAB here for the example, I hope you can easily translate this to Python:
t=0:300;
s=exp(-(t-150).^2/30.^2) .* cos(2*pi*0.1*t);
subplot(2,1,1)
plot(t,s)
xlabel('time')
S=abs(fftshift(fft(s)));
f=linspace(-0.5,0.5,length(S));
subplot(2,1,2)
plot(f,S)
xlabel('frequency')
For those interested in image processing: the Gabor filter is exactly this, but with the frequency spectrum shifted only one direction. The resulting filter is complex, the magnitude of the filtering result is used. This leads to a phase-independent filter.
Related
Suppose B is a signal of duration T sampled each dt = 1sec.
After employing empirical mode decomposition (EMD) to B,
n number of intrinsic mode functions (IMF) would be extracted.
Using Hilbert transform, instantaneous frequencies (IF) and instantaneous amplitudes (A) were calculated. As a result, the IF is a matrix with the number of rows equal to the length of signal B and the number of columns equal to n (so is A).
import emd
B = np.sin(np.linspace(0,2*np.pi,100000))+0.05*np.random.rand(100000)
dt = 1
time_vect = np.linspace(0, len(B), len(B))
## Get IMF's
imf = emd.sift.sift(B)
## Get instantaneous phase, frequency, and instantaneous amplitude
IP, IF, A = emd.spectra.frequency_transform(imf, sample_rate, 'hilbert')
Now I want to construct a marginal Hilbert spectrum using IF and A. I found the following formula in the literature:
where,
T = len(B)
h = np.zeros(len(A.T))
for i in range(len(A.T)):
h[i] =(1/T)*np.sum(A.T[i]**2)
What confuses me is that the result of integration is a vector of length n (that is the number of IMFs). On the other hand, the values of IF for each IMF differ from others and they are not increasingly sorted (like what we have in wavelet transform). My question is, what is supposed to be the frequency for each IMF. For example, if I wanted to plot h[i] as a function of frequency how should I do it?
I tried to create a spectogram of magnitudes using scipy.signal.spectogram.
Unfortunately I didn't get it working.
My test signal should be a sine with frequency 400 Hz and an amplitude of 1. The result for the magnitude of the spectogram seems to be 0.5 instead of 1.0. I have no idea what the problem could be.
import numpy as np
import matplotlib.pyplot as plt
from scipy import signal
# 2s time range with 44kHz
t = np.arange(0, 2, 1/44000)
# test signal: sine with 400Hz amplitude 1
x = np.sin(t*2*np.pi*440)
# spectogram for spectrum of magnitudes
f, t, Sxx = signal.spectrogram(x,
44000,
"hanning",
nperseg=1000,
noverlap=0,
scaling="spectrum",
return_onesided=True,
mode="magnitude"
)
# plot last frequency plot
plt.plot(f, Sxx[:,-1])
print("highest magnitude is: %f" %np.max(Sxx))
A strictly real time domain signal is conjugate symmetric in the frequency domain. e.g. will appear in both the positive and negative (or upper) half of a complex result FFT.
Thus you need to add together the two "halves" together of an FFT result to get the total energy (Parseval's theorem). Or just double one side, since complex conjugates have equal magnitudes.
I am look for a way to obtain the frequency from a signal. Here's an example:
signal = [numpy.sin(numpy.pi * x / 2) for x in range(1000)]
This Array will represent the sample of a recorded sound (x = miliseconds)
sin(pi*x/2) => 250 Hrz
How can we go from the signal (list of points), to obtaining the frequencies form this array?
Note:
I have read many Stackoverflow threads and watch many youtube videos. I am yet to find an answer. Please use simple words.
(I am Thankfull for every answer)
What you're looking for is known as the Fourier Transform
A bit of background
Let's start with the formal definition:
The Fourier transform (FT) decomposes a function (often a function of time, or a signal) into its constituent frequencies
This is in essence a mathematical operation that when applied over a signal, gives you an idea of how present each frequency is in the time series. In order to get some intuition behind this, it might be helpful to look at the mathematical definition of the DFT:
Where k here is swept all the way up t N-1 to calculate all the DFT coefficients.
The first thing to notice is that, this definition resembles somewhat that of the correlation of two functions, in this case x(n) and the negative exponential function. While this may seem a little bit abstract, by using Euler's formula and by playing a bit around with the definition, the DFT can be expressed as the correlation with both a sine wave and a cosine wave, which will account for the imaginary and the real parts of the DFT.
So keeping in mind that this is in essence computing a correlation, whenever a corresponding sine or cosine from the decomposition of the complex exponential matches with that of x(n), there will be a peak in X(K), meaning that, such frequency is present in the signal.
How can we do the same with numpy?
So having given a very brief theoretical background, let's consider an example to see how this can be implemented in python. Lets consider the following signal:
import numpy as np
import matplotlib.pyplot as plt
Fs = 150.0; # sampling rate
Ts = 1.0/Fs; # sampling interval
t = np.arange(0,1,Ts) # time vector
ff = 50; # frequency of the signal
y = np.sin(2*np.pi*ff*t)
plt.plot(t, y)
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.show()
Now, the DFT can be computed by using np.fft.fft, which as mentioned, will be telling you which is the contribution of each frequency in the signal now in the transformed domain:
n = len(y) # length of the signal
k = np.arange(n)
T = n/Fs
frq = k/T # two sides frequency range
frq = frq[:len(frq)//2] # one side frequency range
Y = np.fft.fft(y)/n # dft and normalization
Y = Y[:n//2]
Now, if we plot the actual spectrum, you will see that we get a peak at the frequency of 50Hz, which in mathematical terms it will be a delta function centred in the fundamental frequency of 50Hz. This can be checked in the following Table of Fourier Transform Pairs table.
So for the above signal, we would get:
plt.plot(frq,abs(Y)) # plotting the spectrum
plt.xlabel('Freq (Hz)')
plt.ylabel('|Y(freq)|')
plt.show()
So, I am trying to figure out how to use DFT in practice to detect prevalent frequencies in a signal. I have been trying to wrap my head around what Fourier transforms are and how DFT algorithms work, but apparently I still have ways to go. I have written some code to generate a signal (since the intent is to work with music, I generated a major C chord, hence the weird frequency values) and then tried to work back to the frequency numbers. Here is the code I have
sr = 44100 # sample rate
x = np.linspace(0, 1, sr) # one second of signal
tpi = 2 * np.pi
data = np.sin(261.63 * tpi * x) + np.sin(329.63 * tpi * x) + np.sin(392.00 * tpi * x)
freqs = np.fft.fftfreq(sr)
fft = np.fft.fft(data)
idx = np.argsort(np.abs(fft))
fft = fft[idx]
freqs = freqs[idx]
print(freqs[-6:] * sr)
This gives me [-262. 262. -330. 330. -392. 392.]
which is different from the frequencies I encoded (261.63, 329.63 and 392.0). What am I doing wrong and how do I fix it?
Indeed, if the frame lasts T seconds, the frequencies of the DFT are k/T Hz, where k is an integer. As a consequence, oversampling does not improve the accuracy of the estimated frequency, as long as these frequencies are identifed as maxima of the magnitude of the DFT. On the contrary, considering longer frames lasting 100s would induce a spacing between the DFT frequencies of 0.01Hz, which might be good enough to produce the expected frequency. It is possible to due much better, by estimating the frequency of a peak as its mean frequency wih respect to power density.
Figure 1: even after applying a Tuckey window, the DFT of the windowed signal is not a sum of Dirac: there is still some spectral leakage at the bottom of the peaks. This power must be accounted for as the frequencies are estimated.
Another issue is that the length of the frame is not a multiple of the period of the signal, which may not be periodic anyway. Nevertheless, the DFT is computed as if the signal were periodic but discontinuous at the edge of the frame. It induce spurous frequencies described as spectral leakage. Windowing is the reference method to deal with such problems and mitigate the problem related to the artificial discontinuity. Indeed, the value of a window continuously decrease to zero near the edges of the frame. There is a list of window functions and a lot of window functions are available in scipy.signal. A window is applied as:
tuckey_window=signal.tukey(len(data),0.5,True)
data=data*tuckey_window
At that point, the frequencies exibiting the largest magnitude still are 262, 330 and 392. Applying a window only makes the peaks more visible: the DFT of the windowed signal features three distinguished peaks, each featuring a central lobe and side lobes, depending on the DFT of the window. The lobes of these windows are symmetric: the central frequency can therefore be computed as the mean frequency of the peak, with respect to power density.
import numpy as np
from scipy import signal
import scipy
sr = 44100 # sample rate
x = np.linspace(0, 1, sr) # one second of signal
tpi = 2 * np.pi
data = np.sin(261.63 * tpi * x) + np.sin(329.63 * tpi * x) + np.sin(392.00 * tpi * x)
#a window...
tuckey_window=signal.tukey(len(data),0.5,True)
data=data*tuckey_window
data -= np.mean(data)
fft = np.fft.rfft(data, norm="ortho")
def abs2(x):
return x.real**2 + x.imag**2
fftmag=abs2(fft)[:1000]
peaks, _= signal.find_peaks(fftmag, height=np.max(fftmag)*0.1)
print "potential frequencies ", peaks
#compute the mean frequency of the peak with respect to power density
powerpeak=np.zeros(len(peaks))
powerpeaktimefrequency=np.zeros(len(peaks))
for i in range(1000):
dist=1000
jnear=0
for j in range(len(peaks)):
if dist>np.abs(i-peaks[j]):
dist=np.abs(i-peaks[j])
jnear=j
powerpeak[jnear]+=fftmag[i]
powerpeaktimefrequency[jnear]+=fftmag[i]*i
powerpeaktimefrequency=np.divide(powerpeaktimefrequency,powerpeak)
print 'corrected frequencies', powerpeaktimefrequency
The resulting estimated frequencies are 261.6359 Hz, 329.637Hz and 392.0088 Hz: it much better than 262, 330 and 392Hz and it satisfies the required 0.01Hz accuracy for such a pure noiseless input signal.
DFT result bins are separated by Fs/N in frequency, where N is the length of the FFT. Thus, the duration of your DFT window limits the resolution in terms of DFT result bin frequency center spacings.
But, for well separated frequency peaks in low noise (high S/N), instead of increasing the duration of the data, you can instead estimate the frequency peak locations to a higher resolution by interpolating the DFT result between the DFT result bins. You can try parabolic interpolation for a coarse frequency peak location estimate, but windowed Sinc interpolation (essentially Shannon-Whittaker reconstruction) would provide far better frequency estimation accuracy and resolution (given a low enough noise floor around the frequency peak(s) of interest, e.g. no nearby sinusoids in your artificial waveform case).
Since you want to get a resolution of 0.01 Hz, you will need to sample at least 100 sec worth of data. You will be able to resolve frequencies up to about 22.05 kHz.
I'm plotting sine waves (left column) and their respective frequency domain representations (right column):
The first wave (amplitude: 10; frequency: 0.5) has a somewhat messed up fft representation
The second wave (amplitude: 15; frequency: 5.0) looks absolutely as expected.
The third wave is just the first and the second wave summed up and inherits the problems
The second frequency plot has exactly one peak at x=5 (frequency), y=15 (amplitude).
Why does the first frequency plot have multiple peaks when there's only one frequency?
import numpy as np
import matplotlib.pyplot as plt
def sine(freq, time_interval, rate, amp=1):
w = 2. * np.pi * freq
t = np.linspace(0, time_interval, time_interval*rate)
y = amp*np.sin(w * t)
return y
def buildData():
secs = 3
Fs = 44100
# frequency, duration, sampling rate, amplitude
y1 = sine(0.5, secs, Fs, 10)
y2 = sine(5, secs, Fs, 15)
y3 = y1 + y2
signals = [y1, y2, y3]
showSignals(signals, Fs, secs)
def showSignals(signals, fs, secs):
nrSigs = len(signals)
fig = plt.figure()
fig.subplots_adjust(hspace=.5)
for i in range(len(signals)):
cols=2
pltIdc = []
for col in range(1,cols+1):
pltIdc.append(i*cols+col)
s = signals[i]
t = np.arange(0, secs, 1.0/fs)
ax1 = plt.subplot(nrSigs, cols, pltIdc[0])
ax1.set_title('signal')
ax1.set_xlabel('time')
ax1.set_ylabel('amplitude')
ax1.plot(t, s)
amps = 2*abs(np.fft.fft(s))/len(s) # scaled power spectrum
amps = amps[0:len(amps)/2] # because of the symmetry
amps = amps[0:50] # only the first 50 frequencies, arbitrarily chosen
# this should be close to the amplitude:
print 'magnitude of amplitudes: ' + str(sum(amps*amps)**0.5)
freqs=np.arange(0, len(amps), 1)/secs
ax2 = plt.subplot(nrSigs, cols, pltIdc[1])
ax2.grid(True)
ax2.set_title(r"$\frac{2 \cdot fft(s)}{len(s)}$")
ax2.set_xlabel('freq')
ax2.set_ylabel('amplitude')
ax2.stem(freqs, amps)
plt.show()
buildData()
The FFT routine performs a (fast implementation) discrete Fourier transform, which decomposes a time-series signal into a N-length orthonormal basis consisting of the Fourier "roots of unity".
You will get a discrete, single value of the FFT output if and only if you input a signal that is one of the Fourier basis functions (or a phase-rotated version thereof) because it will have a non-zero inner product with one and only one member of the basis set (by definition).
Your first example has 1.5 cycles within the analysis window, so it cannot be a root of unity (one property of the Fourier basis functions is that they have integral cycle counts within the analysis window). Consequently, there is a non-zero "DC offset" (the average over the analysis window is not exactly zero), which will always yield a "DC" term (nonzero Fourier contribution at index 0 corresponding to a DC offset). Because it's a non-integral cycle count within the analysis window, you also get contributions from other frequencies out of the FFT in addition to the dominant contribution from the frequency nearest that of your sinusoid. This is as expected - any sinusoid that is not itself a fourier basis function will have non-zero inner product with multiple fourier basis functions (and hence multiple spectral contributions in the FFT output).
Your 3rd example is just the sum of the two others, so by linearity of the Fourier transform the output of the FFT is simply the sum of the FFTs of the two individual signals. This is also expected: FFT(a+b) = FFT(a) + FFT(b).
A DFT or FFT will only produce a single point result (spike in the graph) from a sinusoid if the frequency is periodic in exactly an integer number of periods within the FFTs length. Otherwise the energy will get spread out among all the other FFT result bins (but mostly in nearby result frequency bins). This is not "messed up" but normal expected behavior for finite length DFTs.