I am comparing FFT vs. CWT for a specific signal.
It is not clear to me how to read of the respective frequencies and amplitudes from the corresponding scaleogram of the CWT. Furthermore, I have the impression that the CWT is quite imprecise?
The spectrogram seems to be quite good in predicting the precise frequencies, but for the CWT, I tried many different wavelets and the result is the same.
Did I oversee something? Is this just not the appropriate tool for this problem?
Below, you find my example sourcecode and the corresponding plot.
import matplotlib.pyplot as plt
import numpy as np
from numpy import pi as π
from scipy.signal import spectrogram
import pywt
f_s = 200 # Sampling rate = number of measurements per second in [Hz]
t = np.arange(-10,10, 1 / f_s) # Time between [-10s,10s].
T1 = np.tanh(t)/2 + 1.0 # Period in [s]
T2 = 0.125 # Period in [s]
f1 = 1 / T1 # Frequency in [Hz]
f2 = 1 / T2 # Frequency in [Hz]
N = len(t)
x = 13 * np.sin(2 * π * f1 * t) + 42 * np.sin(2 * π * f2 * t)
fig, (ax1, ax2, ax3, ax4) = plt.subplots(4,1, sharex = True, figsize = (10,8))
# Signal
ax1.plot(t, x)
ax1.grid(True)
ax1.set_ylabel("$x(t)$")
ax1.set_title("Signal x(t)")
# Frequency change
ax2.plot(t, f1)
ax2.grid(True)
ax2.set_ylabel("$f_1$ in [Hz]")
ax2.set_title("Change of frequency $f_1(t)$")
# Moving fourier transform, i.e. spectrogram
Δt = 4 # window length in [s]
Nw = np.int(2**np.round(np.log2(Δt * f_s))) # Number of datapoints within window
f, t_, Sxx = spectrogram(x, f_s, window='hanning', nperseg=Nw, noverlap = Nw - 100, detrend=False, scaling='spectrum')
Δf = f[1] - f[0]
Δt_ = t_[1] - t_[0]
t2 = t_ + t[0] - Δt_
im = ax3.pcolormesh(t2, f - Δf/2, np.sqrt(2*Sxx), cmap = "inferno_r")#, alpha = 0.5)
ax3.grid(True)
ax3.set_ylabel("Frequency in [Hz]")
ax3.set_ylim(0, 10)
ax3.set_xlim(np.min(t2),np.max(t2))
ax3.set_title("Spectrogram using FFT and Hanning window")
# Wavelet transform, i.e. scaleogram
cwtmatr, freqs = pywt.cwt(x, np.arange(1, 512), "gaus1", sampling_period = 1 / f_s)
im2 = ax4.pcolormesh(t, freqs, cwtmatr, vmin=0, cmap = "inferno" )
ax4.set_ylim(0,10)
ax4.set_ylabel("Frequency in [Hz]")
ax4.set_xlabel("Time in [s]")
ax4.set_title("Scaleogram using wavelet GAUS1")
# plt.savefig("./fourplot.pdf")
plt.show()
Your example waveform is composed of only a few pure tones at any given time, so the spectrogram plot is going to look very clean and readable.
The wavelet plot is going to look "messy" in comparison because you have to sum Gaussian wavelets of may different scales (and thus frequencies) to approximate each component pure tone in your original signal.
Both the short-time FFT and wavelet transforms are in the category time-frequency transforms, but have a different kernel. The kernel of the FFT is a pure tone, but the kernel of the wavelet transform you provided is a gaussian wavelet. The FFT pure tone kernel is going to cleanly correspond to the type of waveform you showed in your question, but the wavelet kernel will not.
I doubt that the result you got for different wavelets was numerically exactly the same. It probably just looks the same to the naked eye the way you've plotted it. It appears that you are using wavelets for the wrong purpose. Wavelets are more useful in analyzing the signal than plotting it. Wavelet analysis is unique as each datapoint in a wavelet composition encodes frequency, phase, and windowing information simultanesouly. This allows for designing algorithms that lie on a continuum between timeseries and frequency analysis, and can be very powerful.
As far as your claim about different wavelets giving the same results, this is clearly not true for all wavelets: I adapted your code, and it produces the image following it. Sure, GAUS2 and MEXH seem to produce similar plots (zoom in real close and you will find they are subtly different), but that's because the second-order gauss wavelet looks similar to the Mexican hat wavelet.
import matplotlib.pyplot as plt
import numpy as np
from numpy import pi as π
from scipy.signal import spectrogram, wavelets
import pywt
import random
f_s = 200 # Sampling rate = number of measurements per second in [Hz]
t = np.arange(-10,10, 1 / f_s) # Time between [-10s,10s].
T1 = np.tanh(t)/2 + 1.0 # Period in [s]
T2 = 0.125 # Period in [s]
f1 = 1 / T1 # Frequency in [Hz]
f2 = 1 / T2 # Frequency in [Hz]
N = len(t)
x = 13 * np.sin(2 * π * f1 * t) + 42 * np.sin(2 * π * f2 * t)
fig, (ax1, ax2, ax3, ax4) = plt.subplots(4,1, sharex = True, figsize = (10,8))
wvoptions=iter(['gaus1','gaus2','mexh','morl'])
axes=[ax1,ax2,ax3,ax4]
for ax in axes:
# Wavelet transform, i.e. scaleogram
try:
choice=next(wvoptions)
cwtmatr, freqs = pywt.cwt(x, np.arange(1, 512), choice, sampling_period = 1 / f_s)
im = ax.pcolormesh(t, freqs, cwtmatr, vmin=0, cmap = "inferno" )
ax.set_ylim(0,10)
ax.set_ylabel("Frequency in [Hz]")
ax.set_xlabel("Time in [s]")
ax.set_title(f"Scaleogram using wavelet {choice.upper()}")
except:
pass
# plt.savefig("./fourplot.pdf")
plt.tight_layout()
plt.show()
Related
i'm trying to get the frequency of a signal via fourier transform but it's not able to recognize it (sets the peak to f=0). Maybe something is wrong in my code (FULL reprudible code at the end of the page):
PF = fft.fft(Y[0,:])/Npoints #/Npoints to get the true amplitudes
ZF = fft.fft(Y[1,:])/Npoints
freq = fft.fftfreq(Npoints,deltaT)
PF = fft.fftshift(PF) #change of ordering so that the frequencies are increasing
ZF = fft.fftshift(ZF)
freq = fft.fftshift(freq)
plt.plot(freq, np.abs(PF))
plt.show()
plt.plot(T,Y[0,:])
plt.show()
where Npoints is the number of intervals (points) and deltaT is the time spacing of the intervals. You can see that the peak is at f=0
I show also a plot of Y[0,:] (my signal) over time where it's clear that the signal has a characteristic frequency
FULL REPRUDICIBLE CODE
import numpy as np
import matplotlib.pyplot as plt
#numerical integration
from scipy.integrate import solve_ivp
import scipy.fft as fft
r=0.5
g=0.4
e=0.6
H=0.6
m=0.15
#define a vector of K between 0 and 4 with 50 componets
K=np.arange(0.1,4,0.4)
tsteps=np.arange(7200,10000,5)
Npoints=len(tsteps)
deltaT=2800/Npoints #sample spacing
for k in K :
i=0
def RmAmodel(t,y):
return [r*y[0]*(1-y[0]/k)-g*y[0]/(y[0]+H)*y[1], e*g*y[0]/(y[1]+H)*y[1]-m*y[1]]
sol = solve_ivp(RmAmodel, [0,10000], [3,3], t_eval=tsteps) #t_eval specify the points where the solution is desired
T=sol.t
Y=sol.y
vk=[]
for i in range(Npoints):
vk.append(k)
XYZ=[vk,Y[0,:],Y[1,:]]
#check periodicity over P and Z with fourier transform
#try Fourier analysis just for the last value of K
PF = fft.fft(Y[0,:])/Npoints #/Npoints to get the true amplitudes
ZF = fft.fft(Y[1,:])/Npoints
freq = fft.fftfreq(Npoints,deltaT)
PF = fft.fftshift(PF) #change of ordering so that the frequencies are increasing
ZF = fft.fftshift(ZF)
freq = fft.fftshift(freq)
plt.plot(T,Y[0,:])
plt.show()
plt.plot(freq, np.abs(PF))
plt.show()
I can't pinpoint where the problem is. It looks like there is some problem in the fft code. Anyway, I have little time so I will just put a sample code I made before. You can use it as reference or copy-paste it. It should work.
import numpy as np
import matplotlib.pyplot as plt
from scipy.fft import fft, fftfreq
fs = 1000 #sampling frequency
T = 1/fs #sampling period
N = int((1 / T) + 1) #number of sample points for 1 second
t = np.linspace(0, 1, N) #time array
pi = np.pi
sig1 = 1 * np.sin(2*pi*10*t)
sig2 = 2 * np.sin(2*pi*30*t)
sig3 = 3 * np.sin(2*pi*50*t)
#generate signal
signal = sig1 + sig2 + sig3
#plot signal
plt.plot(t, signal)
plt.show()
signal_fft = fft(signal) #getting fft
f2 = np.abs(signal_fft / N) #full spectrum
f1 = f2[:N//2] #half spectrum
f1[1:] = 2*f1[1:] #actual amplitude
freq = fs * np.linspace(0,N/2,int(N/2)) / N #frequency array
#plot fft result
plt.plot(freq, f1)
plt.xlim(0,100)
plt.show()
i have two time series discretized on log10 timestamps which i want to convolve with one another, using python. For linearly spaced time series i use scipy.signal.convole, which works just fine. Is there something similar for log10 discretization? The cut-off frequency is defined in terms of the standard deviation of the Gauss Kernel (sigma). The goal is to filter the log with the same cut-off frequency.
import numpy as np
import matplotlib.pyplot as plt
import scipy.signal as sg
f = lambda t : np.sin(2 * t) + np.sin(0.5 * t)
sigma = 1
kernel = lambda x : np.exp(-x**2 / (2 * sigma**2)) / (np.sqrt(2 * np.pi) * sigma)
# LINEAR DISCRETIZATION
t_linear = np.arange(0, 100, 0.01)
signal_linearly_discretized = f(t_linear)
t_kernel_linear = np.arange(-10, 10, 0.01)
kernel_linearly_discretized = kernel(t_kernel_linear)
# Filter by convolution of kernel and signal
signal_filtered_linear = np.convolve(kernel_linearly_discretized, signal_linearly_discretized,
mode="same") / np.sum(kernel_linearly_discretized)
# LOG DISCRETIZATION
t_log = np.logspace(0, 100, 1000)
signal_log_discretized = f(t_log)
"""
Filtering signal_log_discretized is unclear
"""
# Filter by convolution of kernel and signal
signal_filtered_linear = np.convolve(kernel_linearly_discretized, signal_linearly_discretized,
mode="same") / np.sum(kernel_linearly_discretized)
# Plotting of linear signal and filtered signal
fig, axes = plt.subplots(1, 2)
ax = axes[0]
ax.plot(t_linear, signal_linearly_discretized)
ax.set_title("signal")
ax = axes[1]
ax.plot(t_linear, signal_filtered_linear)
ax.set_title("signal filtered")
fig.tight_layout()
Above you can see the code that i would use to apply a Gauss filter to a linearly discretized signal. I want to perform the same (or a comparable) operation on the signal signal_log_discretized.
Thanks
I've constructed a figure containing 3 different plots representing a position of 3 different masses over some time range. I want to find the period of each. I'm not familiar with the FFT function that I've come across while searching for ways to find the period online. How do I go about this?
Below is the kernel for the plot and the figure; I won't include the code used to build all these variables as it would be quite extensive.
I know I code just start making vertical lines and then estimate by eye, but I'd much rather do it through coding than with that method.
#using the times in days
T9 = 500
dt9 = 0.5
num9 = T9/dt9
times9 = np.linspace(0, T9, num9)
xpos_q9_m1_AU_new = xpos_q9_m1_AU[:-1]
xpos_q9_m2_AU_new = xpos_q9_m2_AU[:-1]
xpos_q9_m3_AU_new = xpos_q9_m3_AU[:-1]
plt.plot(times9, xpos_q9_m1_AU_new)
plt.plot(times9, xpos_q9_m2_AU_new)
plt.plot(times9, xpos_q9_m3_AU_new)
plt.xlabel('Time (days)')
plt.ylabel('X Positions (AU)')
plt.title('X Position of the Kepler 16 System over Time')
plt.legend(['Body 1', 'Body 2', 'Body 3'])
plt.savefig('q9_plot.png');
What you're looking for is a Fourier Transform. This function determines what frequencies make up a wave. Scipy has a module that does this for you nicely:
from scipy.fft import fft
# Number of sample points
N = 600
# sample spacing
T = 1.0 / 800.0
x = np.linspace(0.0, N * T, N)
y = np.sin(50.0 * 2.0 * np.pi * x)
yf = fft(y)
xf = np.linspace(0.0, 1.0 / (2.0 * T), N // 2)
import matplotlib.pyplot as plt
plt.plot(xf, 2.0 / N * np.abs(yf[0:N // 2]))
plt.grid()
plt.show()
This gives you a graph showing the predicted frequency.
Here is my code, But it doesn't work
Load the dataset
dataset = np.loadtxt("filename", delimiter=",")
X = dataset[:,0:5]
Y = dataset[:,5]
compute spectrogram
N = 1e5
amp = 2 * np.sqrt(2)
noise_power = 0.001 * X / 2
time = np.arange(dataset) / X
freq = np.linspace(1e3, 2e3, N)
x = amp * np.sin(2*np.pi*freq*time)
x += np.random.normal(scale=np.sqrt(noise_power), size=time.shape)
Compute and plot the spectrogram.
f, t, Sxx = signal.spectrogram(x, X)
plt.pcolormesh(t, f, Sxx)
plt.ylabel('Frequency [Hz]')
plt.xlabel('Time [sec]')
plt.show()
this is a part of my dataset
33,4.9106E+13,-0.6946377,12.680544,0.50395286
33,4.91061E+13,5.012288,11.264028,0.95342433
33,4.91061E+13,4.903325,10.882658,-0.08172209
33,4.91062E+13,-0.61291564,18.496431,3.0237172
Your posted data does not have headers, so I'm guessing that your last 3 columns is actual acceleration, the first column is of no interest and the second column is symbolizing time. However if that is the timesignal you are sampling at what frequency?
You still have not really explained what is not as you expect, however it seems like you try to make a plot with several different channels. As far as I know one makes a spectrogram on one signal only.
Here is how I would treat similar data as you have, although I had to fake all data so that it becomes somewhat meaningful plots.
import numpy as np
import matplotlib.pyplot as plt
# faked data with noise & trends, with assumption of which column is what
fake_size = int(1e4)
time = np.arange(fake_size)/1000 # 1kHz
# your dataset[:,1] ?
base_freq = 2 * np.pi * 100
x = np.sin(4*base_freq*time) + 0.2 * np.random.random(fake_size)
# your dataset[:,2] ?
y = np.sin(2*base_freq*time) + 0.1 * np.random.random(fake_size) - 0.05 + time
# your dataset[:,3] ?
z = np.sin(3*base_freq*time) + np.sin(1.5*base_freq*time)+ 0.1 * np.random.random(fake_size) - 0.05
# your dataset[:,4] ?
xyz_magnitude = x**2 + y**2 + z**2
to_plot = [('x', x), ('y', y), ('z', z), ('xyz', xyz_magnitude)]
for chl, data in to_plot:
plt.figure(); plt.title(chl)
plt.specgram(data, Fs=1000)
plt.xlabel('Time [s]'); plt.ylabel('Frequency [Hz]')
This will quickly give a picture of what the frequencies are in these spectras, however if you are truly chasing relative magnitudes and or phases, you will have to do additional math as well as plots. Please note that your visual impression of the frequencies are dependent upon magnitude, so if you for example do np.sqrt on the xyz_magnitude the plot will not look exactly the same.
I think, what amplitude in the noised regions on the FFT-plot of the filtered signal should be lower, then now. Probably, small numeric deviations/errors presented in the scipy.fftpack.lfilter().
I tried to add noise to the existing signal, but no result.
Why FFT-amplitude of the filtered signal (green) in the noise regions are so high?
Update:
300 dB of an FFT-amplitude are non-physical - it's clear, it's due to 64bit float in the Python-environment.
FFT of the filtered signal (green) has so low dB (as ~67 dB) due to ~4000 samples at all signal (not 'per second' as on the picture, little mistake, not critical), sample rate = 200 samples/sec. 1 frequency bin=200/4000/2=0.025Hz, 2000 bins shown.
If we take a longer signal, we get a more frequency resolution per bin (i.e. 40 000samples, 1 freq bin = 200/40 000/2 = 0.0025 Hz). And also we get FFT of a filtered signal ~87 dB.
(Numbers 67 dB and 87 dB seems to be non-physical, because of an initial signal SNR 300dB, but I try to add some noise to the existing signal and get the same numbers)
If you want to get a picture of a non-depend FFT to the number of samples in a signal, you should use a windowed FFT and slide window to compute a whole-signal-FFT.
'''
Created on 13.02.2013, python 2.7.3
#author:
'''
from numpy.random import normal
from numpy import sin, pi, absolute, arange, round
#from numpy.fft import fft
from scipy.fftpack import fft, ifft
from scipy.signal import kaiserord, firwin, lfilter, freqz
from pylab import figure, clf, plot, xlabel, ylabel, xlim, ylim, title, grid, axis, show, log10,\
subplots_adjust, subplot
def filter_apply(filename):
pass
def sin_generator(freq_hz = 1000, sample_rate = 8000, amplitude = 1.0, time_s = 1):
nsamples = round(sample_rate * time_s)
t = arange(nsamples) / float(sample_rate.__float__())
signal = amplitude * sin(2*pi*freq_hz.__float__()*t)
return signal, nsamples
def do_fir(signal, sample_rate):
return signal
#-----------------make a signal---------------
freq_hz = 10.0
sample_rate = 400
amplitude = 1.0
time_s = 10
a1, nsamples = sin_generator(freq_hz, sample_rate, amplitude, time_s)
a2, nsamples = sin_generator(50.0, sample_rate, 0.5*amplitude, time_s)
a3, nsamples = sin_generator(150.0, sample_rate, 0.5*amplitude, time_s)
mu, sigma = 0, 0.1 # mean and standard deviation
noise = normal(mu, sigma, nsamples)
signal = a1 + a2 + a3 # + noise
#----------------create low-pass FIR----
# The Nyquist rate of the signal.
nyq_rate = sample_rate / 2.0
# The desired width of the transition from pass to stop,
# relative to the Nyquist rate. We'll design the filter
# with a 5 Hz transition width.
width = 5.0/nyq_rate
# The desired attenuation in the stop band, in dB.
ripple_db = 60.0
# Compute the order and Kaiser parameter for the FIR filter.
N, beta = kaiserord(ripple_db, width)
print 'N = ',N, 'beta = kaiser param = ', beta
# The cutoff frequency of the filter.
cutoff_hz = 30.0
# Use firwin with a Kaiser window to create a lowpass FIR filter.
# Length of the filter (number of coefficients, i.e. the filter order + 1)
taps = firwin(N, cutoff_hz/nyq_rate, window=('kaiser', beta))
# Use lfilter to filter x with the FIR filter.
filtered_signal = lfilter(taps, 1.0, signal)
#----------------plot signal----------------------
hh,ww=2,2
figure(figsize=(12,9))
subplots_adjust(hspace=.5)
#figure(1)
subplot(hh,ww,1)
# existing signal
x = arange(nsamples) / float(sample_rate)
# The phase delay of the filtered signal.
delay = 0.5 * (N-1) / sample_rate
# original signal
plot(x, signal, '-bo' , linewidth=2)
# filtered signal shifted to compensate for
# the phase delay.
plot(x-delay, filtered_signal, 'r-' , linewidth=1)
# Plot just the "good" part of the filtered signal.
# The first N-1 samples are "corrupted" by the
# initial conditions.
plot(x[N-1:]-delay, filtered_signal[N-1:], 'g', linewidth=2)
xlabel('time (s)')
ylabel('amplitude')
axis([0, 1.0/freq_hz*2, -(amplitude*1.5),amplitude*1.5]) # two periods of freq_hz
title('Signal (%d samples)' % nsamples)
grid(True)
#-------------- FFT of the signal
subplot(hh,ww,2)
signal_fft=fft(signal)
filtered_fft =fft(filtered_signal[N-1:])
# existing signal
y = 20*log10( ( abs( signal_fft/nsamples )*2.0)/max( abs( signal_fft/nsamples )*2.0) )# dB Amplitude
x = arange(nsamples)/float(nsamples)*float(sample_rate)
# filtered signal
y_filtered = 20*log10( (abs(filtered_fft/ (nsamples - N + 1) )*2.0)/max(abs(signal_fft/ (nsamples - N + 1) )*2.0) )# dB Amplitude
x_filtered = arange(nsamples - N + 1)/float(nsamples - N + 1)*float(sample_rate)
yy = fft(ifft(filtered_fft))
plot(x,y, linewidth=1)
plot(x_filtered, y_filtered, 'g', linewidth=2)
xlim(0, sample_rate/2) # compensation of mirror (FFT imaginary part)
xlabel('freq (Hz)')
ylabel('amplitude, (dB)')
title('Signal (%d samples)' % nsamples)
grid(True)
#--------------FIR ampitude response
subplot(hh,ww,3)
w, h = freqz(taps, worN=8000)
#plot((w/pi)*nyq_rate, absolute(h), linewidth=2)
plot((w/pi)*nyq_rate, 20*log10(absolute(h)/1.0),'r', linewidth=1)
xlabel('Frequency (Hz)')
#ylabel('Gain -blue')
ylabel('Gain (dB)')
title('Frequency Response')
#ylim(-0.05, 1.05)
grid(True)
#--------------FIR coeffs
subplot(hh,ww,4)
plot(taps, 'bo-', linewidth=2)
title('Filter Coefficients (%d taps)' % N)
grid(True)
show()
I think you -300dB of noise is pretty low. Keep in mind the this is a log scale, so we are talking about a couple of digits at 64bit resolution.
Using double precision float (64bit) you won't get any lower.