Develop Reference

Multi-taper Spectral Analysis with spectrum in python

I'm using the multi-taper analysis using the spectrum library on python (https://pyspectrum.readthedocs.io/en/latest/install.html), but I can't understand fully the amplitude of the output.
Here a piece of code for illustration:
from spectrum import *
N=500
dt=2*10**-3
# Creating a signal with 2 sinus waves.
x = np.linspace(0.0, N*dt, N)
y = np.sin(50.0 * 2.0*np.pi*x) + 0.5*np.sin(80.0 * 2.0*np.pi*x)
# classical FFT
yf = fft.fft(y)
xf = np.linspace(0.0, 1.0/(2.0*dt), N//2)
# The multitapered method
NW=2.5
k=4
[tapers, eigen] = dpss(N, NW, k)
Sk_complex, weights, eigenvalues=pmtm(y, e=eigen, v=tapers, NFFT=500, show=False)
Sk = abs(Sk_complex)
Sk = np.mean(Sk * np.transpose(weights), axis=0)
# ploting both the results
plt.plot(xf,abs(yf[0:N//2])*dt*2)
plt.plot(xf,Sk[0:N//2])
Both the results are similar and find frequency peak at 50 and 80 Hz.
The classical FFT finds as well the good amplitude (1 and 0.5)
But the multi taper method do not find the proper amplitude. In this example it is around 5 times to important.
Do anyone knows actually how to properly display the results ?
thanks

From my understanding, there is a couple of factors that are at play here.
First, to get the multitaper estimate of the power spectrum density, you should compute like this:
Sk = abs(Sk_complex)**2
Sk = np.mean(Sk * np.transpose(weights), axis=0) * dt
I.e., you need to average the power spectrum, not the Fourier components.
Second, to get the power spectrum, you just need to divide the energy spectrum by N of your estimation using fft and multiply by dt as you did (and you need the **2 to get the power from the fourier components):
plt.plot(xf,abs(yf[0:N//2])**2 / N * dt)
plt.plot(xf,Sk[0:N//2])
Finally, what should be directly comparable, is not so much the amplitude in the power spectrum density, but the total power. You can look at:
print(np.sum(abs(yf[0:N//2])**2/N * dt), np.sum(Sk[0:N//2]))
Which match very closely.
So your whole code becomes:
from spectrum import *
N=500
dt=2*10**-3
# Creating a signal with 2 sinus waves.
x = np.linspace(0.0, N*dt, N)
y = np.sin(50.0 * 2.0*np.pi*x) + 0.5*np.sin(80.0 * 2.0*np.pi*x)
# classical FFT
yf = fft.fft(y)
xf = np.linspace(0.0, 1.0/(2.0*dt), N//2)
# The multitapered method
NW=2.5
k=4
[tapers, eigen] = dpss(N, NW, k)
Sk_complex, weights, eigenvalues=pmtm(y, e=eigen, v=tapers, NFFT=N, show=False)
Sk = abs(Sk_complex)**2
Sk = np.mean(Sk * np.transpose(weights), axis=0) * dt
# ploting both results
plt.figure()
plt.plot(xf,abs(yf[0:N//2])**2 / N * dt)
plt.plot(xf,Sk[0:N//2])
# ploting both results in log scale
plt.semilogy(xf, abs(yf[0:N // 2]) ** 2 / N * dt)
plt.semilogy(xf, Sk[0:N // 2])
# comparing total power
print(np.sum(abs(yf[0:N//2])**2 / N * dt), np.sum(Sk[0:N//2]))

Spectrogram vs. Scaleogram for Time-Varying Frequency

I am comparing FFT vs. CWT for a specific signal.
It is not clear to me how to read of the respective frequencies and amplitudes from the corresponding scaleogram of the CWT. Furthermore, I have the impression that the CWT is quite imprecise?
The spectrogram seems to be quite good in predicting the precise frequencies, but for the CWT, I tried many different wavelets and the result is the same.
Did I oversee something? Is this just not the appropriate tool for this problem?
Below, you find my example sourcecode and the corresponding plot.
import matplotlib.pyplot as plt
import numpy as np
from numpy import pi as π
from scipy.signal import spectrogram
import pywt
f_s = 200 # Sampling rate = number of measurements per second in [Hz]
t = np.arange(-10,10, 1 / f_s) # Time between [-10s,10s].
T1 = np.tanh(t)/2 + 1.0 # Period in [s]
T2 = 0.125 # Period in [s]
f1 = 1 / T1 # Frequency in [Hz]
f2 = 1 / T2 # Frequency in [Hz]
N = len(t)
x = 13 * np.sin(2 * π * f1 * t) + 42 * np.sin(2 * π * f2 * t)
fig, (ax1, ax2, ax3, ax4) = plt.subplots(4,1, sharex = True, figsize = (10,8))
# Signal
ax1.plot(t, x)
ax1.grid(True)
ax1.set_ylabel("$x(t)$")
ax1.set_title("Signal x(t)")
# Frequency change
ax2.plot(t, f1)
ax2.grid(True)
ax2.set_ylabel("$f_1$ in [Hz]")
ax2.set_title("Change of frequency $f_1(t)$")
# Moving fourier transform, i.e. spectrogram
Δt = 4 # window length in [s]
Nw = np.int(2**np.round(np.log2(Δt * f_s))) # Number of datapoints within window
f, t_, Sxx = spectrogram(x, f_s, window='hanning', nperseg=Nw, noverlap = Nw - 100, detrend=False, scaling='spectrum')
Δf = f[1] - f[0]
Δt_ = t_[1] - t_[0]
t2 = t_ + t[0] - Δt_
im = ax3.pcolormesh(t2, f - Δf/2, np.sqrt(2*Sxx), cmap = "inferno_r")#, alpha = 0.5)
ax3.grid(True)
ax3.set_ylabel("Frequency in [Hz]")
ax3.set_ylim(0, 10)
ax3.set_xlim(np.min(t2),np.max(t2))
ax3.set_title("Spectrogram using FFT and Hanning window")
# Wavelet transform, i.e. scaleogram
cwtmatr, freqs = pywt.cwt(x, np.arange(1, 512), "gaus1", sampling_period = 1 / f_s)
im2 = ax4.pcolormesh(t, freqs, cwtmatr, vmin=0, cmap = "inferno" )
ax4.set_ylim(0,10)
ax4.set_ylabel("Frequency in [Hz]")
ax4.set_xlabel("Time in [s]")
ax4.set_title("Scaleogram using wavelet GAUS1")
# plt.savefig("./fourplot.pdf")
plt.show()

Your example waveform is composed of only a few pure tones at any given time, so the spectrogram plot is going to look very clean and readable.
The wavelet plot is going to look "messy" in comparison because you have to sum Gaussian wavelets of may different scales (and thus frequencies) to approximate each component pure tone in your original signal.
Both the short-time FFT and wavelet transforms are in the category time-frequency transforms, but have a different kernel. The kernel of the FFT is a pure tone, but the kernel of the wavelet transform you provided is a gaussian wavelet. The FFT pure tone kernel is going to cleanly correspond to the type of waveform you showed in your question, but the wavelet kernel will not.
I doubt that the result you got for different wavelets was numerically exactly the same. It probably just looks the same to the naked eye the way you've plotted it. It appears that you are using wavelets for the wrong purpose. Wavelets are more useful in analyzing the signal than plotting it. Wavelet analysis is unique as each datapoint in a wavelet composition encodes frequency, phase, and windowing information simultanesouly. This allows for designing algorithms that lie on a continuum between timeseries and frequency analysis, and can be very powerful.
As far as your claim about different wavelets giving the same results, this is clearly not true for all wavelets: I adapted your code, and it produces the image following it. Sure, GAUS2 and MEXH seem to produce similar plots (zoom in real close and you will find they are subtly different), but that's because the second-order gauss wavelet looks similar to the Mexican hat wavelet.
import matplotlib.pyplot as plt
import numpy as np
from numpy import pi as π
from scipy.signal import spectrogram, wavelets
import pywt
import random
f_s = 200 # Sampling rate = number of measurements per second in [Hz]
t = np.arange(-10,10, 1 / f_s) # Time between [-10s,10s].
T1 = np.tanh(t)/2 + 1.0 # Period in [s]
T2 = 0.125 # Period in [s]
f1 = 1 / T1 # Frequency in [Hz]
f2 = 1 / T2 # Frequency in [Hz]
N = len(t)
x = 13 * np.sin(2 * π * f1 * t) + 42 * np.sin(2 * π * f2 * t)
fig, (ax1, ax2, ax3, ax4) = plt.subplots(4,1, sharex = True, figsize = (10,8))
wvoptions=iter(['gaus1','gaus2','mexh','morl'])
axes=[ax1,ax2,ax3,ax4]
for ax in axes:
# Wavelet transform, i.e. scaleogram
try:
choice=next(wvoptions)
cwtmatr, freqs = pywt.cwt(x, np.arange(1, 512), choice, sampling_period = 1 / f_s)
im = ax.pcolormesh(t, freqs, cwtmatr, vmin=0, cmap = "inferno" )
ax.set_ylim(0,10)
ax.set_ylabel("Frequency in [Hz]")
ax.set_xlabel("Time in [s]")
ax.set_title(f"Scaleogram using wavelet {choice.upper()}")
except:
pass
# plt.savefig("./fourplot.pdf")
plt.tight_layout()
plt.show()

How to obtain spectrogram of a 3D accelerometer file in python?

Here is my code, But it doesn't work
Load the dataset
dataset = np.loadtxt("filename", delimiter=",")
X = dataset[:,0:5]
Y = dataset[:,5]
compute spectrogram
N = 1e5
amp = 2 * np.sqrt(2)
noise_power = 0.001 * X / 2
time = np.arange(dataset) / X
freq = np.linspace(1e3, 2e3, N)
x = amp * np.sin(2*np.pi*freq*time)
x += np.random.normal(scale=np.sqrt(noise_power), size=time.shape)
Compute and plot the spectrogram.
f, t, Sxx = signal.spectrogram(x, X)
plt.pcolormesh(t, f, Sxx)
plt.ylabel('Frequency [Hz]')
plt.xlabel('Time [sec]')
plt.show()
this is a part of my dataset
33,4.9106E+13,-0.6946377,12.680544,0.50395286
33,4.91061E+13,5.012288,11.264028,0.95342433
33,4.91061E+13,4.903325,10.882658,-0.08172209
33,4.91062E+13,-0.61291564,18.496431,3.0237172

Your posted data does not have headers, so I'm guessing that your last 3 columns is actual acceleration, the first column is of no interest and the second column is symbolizing time. However if that is the timesignal you are sampling at what frequency?
You still have not really explained what is not as you expect, however it seems like you try to make a plot with several different channels. As far as I know one makes a spectrogram on one signal only.
Here is how I would treat similar data as you have, although I had to fake all data so that it becomes somewhat meaningful plots.
import numpy as np
import matplotlib.pyplot as plt
# faked data with noise & trends, with assumption of which column is what
fake_size = int(1e4)
time = np.arange(fake_size)/1000 # 1kHz
# your dataset[:,1] ?
base_freq = 2 * np.pi * 100
x = np.sin(4*base_freq*time) + 0.2 * np.random.random(fake_size)
# your dataset[:,2] ?
y = np.sin(2*base_freq*time) + 0.1 * np.random.random(fake_size) - 0.05 + time
# your dataset[:,3] ?
z = np.sin(3*base_freq*time) + np.sin(1.5*base_freq*time)+ 0.1 * np.random.random(fake_size) - 0.05
# your dataset[:,4] ?
xyz_magnitude = x**2 + y**2 + z**2
to_plot = [('x', x), ('y', y), ('z', z), ('xyz', xyz_magnitude)]
for chl, data in to_plot:
plt.figure(); plt.title(chl)
plt.specgram(data, Fs=1000)
plt.xlabel('Time [s]'); plt.ylabel('Frequency [Hz]')
This will quickly give a picture of what the frequencies are in these spectras, however if you are truly chasing relative magnitudes and or phases, you will have to do additional math as well as plots. Please note that your visual impression of the frequencies are dependent upon magnitude, so if you for example do np.sqrt on the xyz_magnitude the plot will not look exactly the same.

Strange FFT output python

I am trying to take the FFT and plot it. Problem is, my code works for small frequencies (like 50) but doesn't work for the bigger frequencies I need. What is going on with my code?! I expect to see a spike at the frequency of the sine wave I input, but the spike is at different frequencies depending on the sample spacing I use.
bins = 600
ss = 2048
freq = 44100
centerfreq = freq*bins/ss
# Number of samplepoints
N = ss
# sample spacing
T = 1 / 800.
x = np.linspace(0.0, N*T, N)
y = sin(2*np.pi*centerfreq*x)
yf = fft(y)
xf = np.linspace(0.0, 1.0/(2.0*T), N/2)
plt.plot(xf, 2.0/N * np.abs(yf[0:N/2]), 'r')

The code is right, you need to brush up your Fourier Theory and Nyquist Sampling Theorem and make sure the numbers make sense. The problem is with your x-axis scale. The plot function plots the first item in x with the first item in y, if x is not scaled up to your expectations, you are in for a surprise. You also see this if you plot a sinusoidal signal (sine wave) and expect 'degrees' and you get radians for instance. Its your duty to scale it up well so that it lines up to your expectation.
Refer to this SO answer https://stackoverflow.com/a/25735436/2061422.
from scipy import *
from numpy import *
from pylab import * # imports for me to get going
bins = 600
ss = 2048
freq = 44100
centerfreq = freq*bins/ss
print centerfreq
# Number of samplepoints
N = ss
# sample spacing
T = 1. / freq # i have decreased the spacing considerably
x = np.linspace(0.0, N*T, N)
sample_spacing = x[1] - x[0] # but this is the real sample spacing
y = sin(2*np.pi*centerfreq*x)
yf = fft(y)
xf = np.linspace(0.0, 1.0/(2.0*T), N/2)
freqs = np.fft.fftfreq(len(y), sample_spacing) # read the manual on this fella.
plt.plot(freqs[:N/2], 1.0/N * np.abs(yf[0:N/2]), 'r')
plt.grid()
plt.show()

How do I generate a spectrogram of a 1D signal in python?

I'm not sure how to do this and I was given an example, spectrogram e.g. but this is in 2D.
I have code here that generates a mix of frequencies and I can pick these out in the fft, how may I
see these in a spectrogram? I appreciate that the frequencies in my example don't change over time; so does this mean I'll see a straight line across the spectrogram?
my code and the output image:
# create a wave with 1Mhz and 0.5Mhz frequencies
dt = 2e-9
t = np.arange(0, 10e-6, dt)
y = np.cos(2 * pi * 1e6 * t) + (np.cos(2 * pi * 2e6 *t) * np.cos(2 * pi * 2e6 * t))
y *= np.hanning(len(y))
yy = np.concatenate((y, ([0] * 10 * len(y))))
# FFT of this
Fs = 1 / dt # sampling rate, Fs = 500MHz = 1/2ns
n = len(yy) # length of the signal
k = np.arange(n)
T = n / Fs
frq = k / T # two sides frequency range
frq = frq[range(n / 2)] # one side frequency range
Y = fft(yy) / n # fft computing and normalization
Y = Y[range(n / 2)] / max(Y[range(n / 2)])
# plotting the data
subplot(3, 1, 1)
plot(t * 1e3, y, 'r')
xlabel('Time (micro seconds)')
ylabel('Amplitude')
grid()
# plotting the spectrum
subplot(3, 1, 2)
plot(frq[0:600], abs(Y[0:600]), 'k')
xlabel('Freq (Hz)')
ylabel('|Y(freq)|')
grid()
# plotting the specgram
subplot(3, 1, 3)
Pxx, freqs, bins, im = specgram(y, NFFT=512, Fs=Fs, noverlap=10)
show()

What you have is technically correct, but you just need to look at a signal with an interesting spectrogram. For that, you need the frequency to vary with time. (And for that to happen, you need many oscillations, since it takes a few oscillations to establish a frequency, and then you need many of these to have the frequency change with time in an interesting way.)
Below I've modified you code as little as possible to get a frequency that does something interesting (fscale just ramps the frequency over time). I'm posting all the code to get this to work, but I only change three of the top four lines.
# create a wave with 1Mhz and 0.5Mhz frequencies
dt = 40e-9
t = np.arange(0, 1000e-6, dt)
fscale = t/max(t)
y = np.cos(2 * pi * 1e6 * t*fscale) + (np.cos(2 * pi * 2e6 *t*fscale) * np.cos(2 * pi * 2e6 * t*fscale))
y *= np.hanning(len(y))
yy = np.concatenate((y, ([0] * 10 * len(y))))
# FFT of this
Fs = 1 / dt # sampling rate, Fs = 500MHz = 1/2ns
n = len(yy) # length of the signal
k = np.arange(n)
T = n / Fs
frq = k / T # two sides frequency range
frq = frq[range(n / 2)] # one side frequency range
Y = fft(yy) / n # fft computing and normalization
Y = Y[range(n / 2)] / max(Y[range(n / 2)])
# plotting the data
subplot(3, 1, 1)
plot(t * 1e3, y, 'r')
xlabel('Time (micro seconds)')
ylabel('Amplitude')
grid()
# plotting the spectrum
subplot(3, 1, 2)
plot(frq[0:600], abs(Y[0:600]), 'k')
xlabel('Freq (Hz)')
ylabel('|Y(freq)|')
grid()
# plotting the specgram
subplot(3, 1, 3)
Pxx, freqs, bins, im = specgram(y, NFFT=512, Fs=Fs, noverlap=10)
show()
Also, note here that only the spectrogram is useful. If you can see a good waveform or spectra, the spectrogram probably won't be interesting: 1) if the waveform is clear you probably don't have enough data and time over which the frequency is both well defined and changes enough to be interesting; 2) if the full spectra is clear, you probably don't have enough variation in frequency for the spectrogram, since the spectrum is basically just an average of what you see changing with time in the spectrogram.
If you really want to see the spectrogram of your original signal, you just need to zoom on the y-axis to see the peaks you are expecting (note that the spectrogram y-axis is 2.5e8, must larger than in your spectrum):

To get what you're after:
1) sample the 1d waveform at high frequency (at least 5 times the frequency of its highest frequency component)
2) use blocks of samples (powers of 2 like 1024,16384,etc) to compute an FFT
3) for each spectrum plot a vertical line of pixels whose color represents the amplitude of each frequency.
4) repeat steps 2 and 3 for each block of samples.
In your case, the plot has a whole rainbow of colors which should not be present with only a couple very distinct frequencies. Your spectral plot has rather wide bands around the peaks but that could be due to a low sampling rate and smooth plotting.

I am just starting on Python 3.6
Thank you for the Spectrogram sample code!
However with Python 3.6 I struggled a bit to make this sample spectrogram code to work (functions calls and float division
I have edited code so it now works on python 3.6 for my python newbies pals out there.
Enjoy
'''
Original Script for Python 2.7
https://stackoverflow.com/questions/19052324/how-do-i-generate-a-spectrogram-of-a-1d-signal-in-python
Modified in August 2017 for Python 3.6
Python 2.7 two integers / Division generate Integer
Python 3.6 two integers / Division generate Float
Python 3.6 two integers // Division generate integer
'''
import numpy as np
from scipy import fftpack
import matplotlib.pyplot as plt
dt = 40e-9
t = np.arange(0, 1000e-6, dt)
fscale = t/max(t)
y = np.cos(2 * np.pi * 1e6 * t*fscale) + (np.cos(2 * np.pi * 2e6 *t*fscale) * np.cos(2 * np.pi * 2e6 * t*fscale))
y *= np.hanning(len(y))
yy = np.concatenate((y, ([0] * 10 * len(y))))
# FFT of this
Fs = 1 / dt # sampling rate, Fs = 500MHz = 1/2ns
n = len(yy) # length of the signal
k = np.arange(n)
T = n / Fs
frq = k / T # two sides frequency range
frq = frq[range(n // 2)] # one side frequency range
Y = fftpack.fft(yy) / n # fft computing and normalization
Y = Y[range(n // 2)] / max(Y[range(n // 2)])
# plotting the data
plt.subplot(3, 1, 1)
plt.plot(t * 1e3, y, 'r')
plt.xlabel('Time (micro seconds)')
plt.ylabel('Amplitude')
plt.grid()
# plotting the spectrum
plt.subplot(3, 1, 2)
plt.plot(frq[0:600], abs(Y[0:600]), 'k')
plt.xlabel('Freq (Hz)')
plt.ylabel('|Y(freq)|')
plt.grid()
# plotting the specgram
plt.subplot(3, 1, 3)
Pxx, freqs, bins, im = plt.specgram(y, NFFT=512, Fs=Fs, noverlap=10)
plt.show()