I'm not sure how to do this and I was given an example, spectrogram e.g. but this is in 2D.
I have code here that generates a mix of frequencies and I can pick these out in the fft, how may I
see these in a spectrogram? I appreciate that the frequencies in my example don't change over time; so does this mean I'll see a straight line across the spectrogram?
my code and the output image:
# create a wave with 1Mhz and 0.5Mhz frequencies
dt = 2e-9
t = np.arange(0, 10e-6, dt)
y = np.cos(2 * pi * 1e6 * t) + (np.cos(2 * pi * 2e6 *t) * np.cos(2 * pi * 2e6 * t))
y *= np.hanning(len(y))
yy = np.concatenate((y, ([0] * 10 * len(y))))
# FFT of this
Fs = 1 / dt # sampling rate, Fs = 500MHz = 1/2ns
n = len(yy) # length of the signal
k = np.arange(n)
T = n / Fs
frq = k / T # two sides frequency range
frq = frq[range(n / 2)] # one side frequency range
Y = fft(yy) / n # fft computing and normalization
Y = Y[range(n / 2)] / max(Y[range(n / 2)])
# plotting the data
subplot(3, 1, 1)
plot(t * 1e3, y, 'r')
xlabel('Time (micro seconds)')
ylabel('Amplitude')
grid()
# plotting the spectrum
subplot(3, 1, 2)
plot(frq[0:600], abs(Y[0:600]), 'k')
xlabel('Freq (Hz)')
ylabel('|Y(freq)|')
grid()
# plotting the specgram
subplot(3, 1, 3)
Pxx, freqs, bins, im = specgram(y, NFFT=512, Fs=Fs, noverlap=10)
show()
What you have is technically correct, but you just need to look at a signal with an interesting spectrogram. For that, you need the frequency to vary with time. (And for that to happen, you need many oscillations, since it takes a few oscillations to establish a frequency, and then you need many of these to have the frequency change with time in an interesting way.)
Below I've modified you code as little as possible to get a frequency that does something interesting (fscale just ramps the frequency over time). I'm posting all the code to get this to work, but I only change three of the top four lines.
# create a wave with 1Mhz and 0.5Mhz frequencies
dt = 40e-9
t = np.arange(0, 1000e-6, dt)
fscale = t/max(t)
y = np.cos(2 * pi * 1e6 * t*fscale) + (np.cos(2 * pi * 2e6 *t*fscale) * np.cos(2 * pi * 2e6 * t*fscale))
y *= np.hanning(len(y))
yy = np.concatenate((y, ([0] * 10 * len(y))))
# FFT of this
Fs = 1 / dt # sampling rate, Fs = 500MHz = 1/2ns
n = len(yy) # length of the signal
k = np.arange(n)
T = n / Fs
frq = k / T # two sides frequency range
frq = frq[range(n / 2)] # one side frequency range
Y = fft(yy) / n # fft computing and normalization
Y = Y[range(n / 2)] / max(Y[range(n / 2)])
# plotting the data
subplot(3, 1, 1)
plot(t * 1e3, y, 'r')
xlabel('Time (micro seconds)')
ylabel('Amplitude')
grid()
# plotting the spectrum
subplot(3, 1, 2)
plot(frq[0:600], abs(Y[0:600]), 'k')
xlabel('Freq (Hz)')
ylabel('|Y(freq)|')
grid()
# plotting the specgram
subplot(3, 1, 3)
Pxx, freqs, bins, im = specgram(y, NFFT=512, Fs=Fs, noverlap=10)
show()
Also, note here that only the spectrogram is useful. If you can see a good waveform or spectra, the spectrogram probably won't be interesting: 1) if the waveform is clear you probably don't have enough data and time over which the frequency is both well defined and changes enough to be interesting; 2) if the full spectra is clear, you probably don't have enough variation in frequency for the spectrogram, since the spectrum is basically just an average of what you see changing with time in the spectrogram.
If you really want to see the spectrogram of your original signal, you just need to zoom on the y-axis to see the peaks you are expecting (note that the spectrogram y-axis is 2.5e8, must larger than in your spectrum):
To get what you're after:
1) sample the 1d waveform at high frequency (at least 5 times the frequency of its highest frequency component)
2) use blocks of samples (powers of 2 like 1024,16384,etc) to compute an FFT
3) for each spectrum plot a vertical line of pixels whose color represents the amplitude of each frequency.
4) repeat steps 2 and 3 for each block of samples.
In your case, the plot has a whole rainbow of colors which should not be present with only a couple very distinct frequencies. Your spectral plot has rather wide bands around the peaks but that could be due to a low sampling rate and smooth plotting.
I am just starting on Python 3.6
Thank you for the Spectrogram sample code!
However with Python 3.6 I struggled a bit to make this sample spectrogram code to work (functions calls and float division
I have edited code so it now works on python 3.6 for my python newbies pals out there.
Enjoy
'''
Original Script for Python 2.7
https://stackoverflow.com/questions/19052324/how-do-i-generate-a-spectrogram-of-a-1d-signal-in-python
Modified in August 2017 for Python 3.6
Python 2.7 two integers / Division generate Integer
Python 3.6 two integers / Division generate Float
Python 3.6 two integers // Division generate integer
'''
import numpy as np
from scipy import fftpack
import matplotlib.pyplot as plt
dt = 40e-9
t = np.arange(0, 1000e-6, dt)
fscale = t/max(t)
y = np.cos(2 * np.pi * 1e6 * t*fscale) + (np.cos(2 * np.pi * 2e6 *t*fscale) * np.cos(2 * np.pi * 2e6 * t*fscale))
y *= np.hanning(len(y))
yy = np.concatenate((y, ([0] * 10 * len(y))))
# FFT of this
Fs = 1 / dt # sampling rate, Fs = 500MHz = 1/2ns
n = len(yy) # length of the signal
k = np.arange(n)
T = n / Fs
frq = k / T # two sides frequency range
frq = frq[range(n // 2)] # one side frequency range
Y = fftpack.fft(yy) / n # fft computing and normalization
Y = Y[range(n // 2)] / max(Y[range(n // 2)])
# plotting the data
plt.subplot(3, 1, 1)
plt.plot(t * 1e3, y, 'r')
plt.xlabel('Time (micro seconds)')
plt.ylabel('Amplitude')
plt.grid()
# plotting the spectrum
plt.subplot(3, 1, 2)
plt.plot(frq[0:600], abs(Y[0:600]), 'k')
plt.xlabel('Freq (Hz)')
plt.ylabel('|Y(freq)|')
plt.grid()
# plotting the specgram
plt.subplot(3, 1, 3)
Pxx, freqs, bins, im = plt.specgram(y, NFFT=512, Fs=Fs, noverlap=10)
plt.show()
Related
I'm looking for a clarification of Fourier transform principles.
I try to do something quite simple: Create a signal (sine wave with a given frequency and phase shift) and recreate its params with Fourier transform. Frequency estimations work fine, but when it comes to phase, it looks like I get systematic shift (-pi/2).
import numpy as np
import matplotlib.pyplot as plt
duration = 4.0 # lenght of window (in sec)
ticks_per_sec = 400.0 # sampling interval
samples = int(ticks_per_sec*duration)
phase_shift = np.pi / 2 # sin wave shift in angle
freq = 1 # sine wave freq in Hz
phase_shift = round(ticks_per_sec/freq * phase_shift/(2*np.pi)) # angle value translated to no of ticks
t = np.arange(phase_shift, samples+phase_shift) / ticks_per_sec
s = 1 * np.sin(2.0 * np.pi * freq * t)
N = s.size
fig, axs = plt.subplots(1, 3, figsize=(18, 6))
axs[0].grid(True)
axs[0].set_ylabel("Amplitude")
axs[0].set_xlabel("Time [s]")
axs[0].set_title(f"F: {freq}, Phase shift: {phase_shift} ticks.")
axs[0].plot(np.arange(samples)/ticks_per_sec, s)
f = np.linspace(0, ticks_per_sec, N)
fft = np.fft.fft(s)
peak_pos = np.argmax(np.abs(fft[:N//2]))
axs[1].set_ylabel("Amplitude")
axs[1].set_xlabel("Frequency [Hz]")
axs[1].set_title(f"Peak bar: {peak_pos}")
barlist = axs[1].bar(f[:N // 2], np.abs(fft)[:N // 2] * (1 / (N//2)), width=1.5) # 1 / N is a normalization factor
barlist[peak_pos].set_color('r')
axs[2].set_ylabel("Angle")
axs[2].set_xlabel("Frequency [Hz]")
axs[2].set_title(f"Peak angle: {np.angle(fft[peak_pos])}")
barlist = axs[2].bar(f[:N // 2], np.angle(fft)[:N // 2], width=1.5)
barlist[peak_pos].set_color('r')
fig.show()
Plotted Results of the code above
Please help me if there's a bug in my code that I can't notice, or I misunderstand something.
Thank you in advance.
Your code is just fine, this is not a programming issue.
Let's recall that a sine wave can be expressed as a cosine wave with a phase shift (or vice versa), now remember that sine function as an inherent phase shift of -pi/2 in real Fourier basis relatively to cosine.
This means that your code should output a pi/2 phase angle when replacing np.sin by np.cos, i.e. returns input phase_shift, or equivalently, returns a phase angle of zero when specifying phase_shift = np.pi / 2, i.e. phase shift and sine phase compensate each other.
I'm using the multi-taper analysis using the spectrum library on python (https://pyspectrum.readthedocs.io/en/latest/install.html), but I can't understand fully the amplitude of the output.
Here a piece of code for illustration:
from spectrum import *
N=500
dt=2*10**-3
# Creating a signal with 2 sinus waves.
x = np.linspace(0.0, N*dt, N)
y = np.sin(50.0 * 2.0*np.pi*x) + 0.5*np.sin(80.0 * 2.0*np.pi*x)
# classical FFT
yf = fft.fft(y)
xf = np.linspace(0.0, 1.0/(2.0*dt), N//2)
# The multitapered method
NW=2.5
k=4
[tapers, eigen] = dpss(N, NW, k)
Sk_complex, weights, eigenvalues=pmtm(y, e=eigen, v=tapers, NFFT=500, show=False)
Sk = abs(Sk_complex)
Sk = np.mean(Sk * np.transpose(weights), axis=0)
# ploting both the results
plt.plot(xf,abs(yf[0:N//2])*dt*2)
plt.plot(xf,Sk[0:N//2])
Both the results are similar and find frequency peak at 50 and 80 Hz.
The classical FFT finds as well the good amplitude (1 and 0.5)
But the multi taper method do not find the proper amplitude. In this example it is around 5 times to important.
Do anyone knows actually how to properly display the results ?
thanks
From my understanding, there is a couple of factors that are at play here.
First, to get the multitaper estimate of the power spectrum density, you should compute like this:
Sk = abs(Sk_complex)**2
Sk = np.mean(Sk * np.transpose(weights), axis=0) * dt
I.e., you need to average the power spectrum, not the Fourier components.
Second, to get the power spectrum, you just need to divide the energy spectrum by N of your estimation using fft and multiply by dt as you did (and you need the **2 to get the power from the fourier components):
plt.plot(xf,abs(yf[0:N//2])**2 / N * dt)
plt.plot(xf,Sk[0:N//2])
Finally, what should be directly comparable, is not so much the amplitude in the power spectrum density, but the total power. You can look at:
print(np.sum(abs(yf[0:N//2])**2/N * dt), np.sum(Sk[0:N//2]))
Which match very closely.
So your whole code becomes:
from spectrum import *
N=500
dt=2*10**-3
# Creating a signal with 2 sinus waves.
x = np.linspace(0.0, N*dt, N)
y = np.sin(50.0 * 2.0*np.pi*x) + 0.5*np.sin(80.0 * 2.0*np.pi*x)
# classical FFT
yf = fft.fft(y)
xf = np.linspace(0.0, 1.0/(2.0*dt), N//2)
# The multitapered method
NW=2.5
k=4
[tapers, eigen] = dpss(N, NW, k)
Sk_complex, weights, eigenvalues=pmtm(y, e=eigen, v=tapers, NFFT=N, show=False)
Sk = abs(Sk_complex)**2
Sk = np.mean(Sk * np.transpose(weights), axis=0) * dt
# ploting both results
plt.figure()
plt.plot(xf,abs(yf[0:N//2])**2 / N * dt)
plt.plot(xf,Sk[0:N//2])
# ploting both results in log scale
plt.semilogy(xf, abs(yf[0:N // 2]) ** 2 / N * dt)
plt.semilogy(xf, Sk[0:N // 2])
# comparing total power
print(np.sum(abs(yf[0:N//2])**2 / N * dt), np.sum(Sk[0:N//2]))
I've constructed a figure containing 3 different plots representing a position of 3 different masses over some time range. I want to find the period of each. I'm not familiar with the FFT function that I've come across while searching for ways to find the period online. How do I go about this?
Below is the kernel for the plot and the figure; I won't include the code used to build all these variables as it would be quite extensive.
I know I code just start making vertical lines and then estimate by eye, but I'd much rather do it through coding than with that method.
#using the times in days
T9 = 500
dt9 = 0.5
num9 = T9/dt9
times9 = np.linspace(0, T9, num9)
xpos_q9_m1_AU_new = xpos_q9_m1_AU[:-1]
xpos_q9_m2_AU_new = xpos_q9_m2_AU[:-1]
xpos_q9_m3_AU_new = xpos_q9_m3_AU[:-1]
plt.plot(times9, xpos_q9_m1_AU_new)
plt.plot(times9, xpos_q9_m2_AU_new)
plt.plot(times9, xpos_q9_m3_AU_new)
plt.xlabel('Time (days)')
plt.ylabel('X Positions (AU)')
plt.title('X Position of the Kepler 16 System over Time')
plt.legend(['Body 1', 'Body 2', 'Body 3'])
plt.savefig('q9_plot.png');
What you're looking for is a Fourier Transform. This function determines what frequencies make up a wave. Scipy has a module that does this for you nicely:
from scipy.fft import fft
# Number of sample points
N = 600
# sample spacing
T = 1.0 / 800.0
x = np.linspace(0.0, N * T, N)
y = np.sin(50.0 * 2.0 * np.pi * x)
yf = fft(y)
xf = np.linspace(0.0, 1.0 / (2.0 * T), N // 2)
import matplotlib.pyplot as plt
plt.plot(xf, 2.0 / N * np.abs(yf[0:N // 2]))
plt.grid()
plt.show()
This gives you a graph showing the predicted frequency.
Below is my code
where I am wanting to sweep a 400MHz tone between -50Mhz and 50Mhz. Meaning the
sweep should span between 350MHz and 450MHz. But that is no the case. Don't
understand what the reason for this is. It has been suggested that this is because frequency is the derivative of the phase. I have given that a shot too in the commented lines in the 'sweep_sine' function but that does not seem to help either. Any help would be appreciated.
Image of unexpected program output added. As you can see I am able to shift my 400MHz tone to 300MHz. When I try to sweep in a fashion similar to shift; the output is incorrect, as in it does not sweep from 350MHz to 450MHz with the 400MHz tone in the middle.
I am able to sweep the signal correctly now as can be seen in image 2. The time domain signal also looks fine when the signal is of the e^i2*pift form. But when I use a real signal of the form sin(2*pift) the time domain version looks corrupted (image 3). What could be the reason for that? Thanks.
https://i.stack.imgur.com/9H1Dk.png
https://i.stack.imgur.com/Ey3tQ.png
https://i.stack.imgur.com/FzmDS.png
import numpy as np
import matplotlib.pyplot as plt
def gen_sig(freq=380e6, fs=6080e6, n_samp=6400):
ts = 1/fs
t_arr = np.arange(0, n_samp)*ts
sig = np.exp(2 * 1j * np.pi * freq * t_arr)
#sig = np.sin(2 * np.pi * freq * t_arr)
return sig,ts
def freq_shift_sine(sine, ts, shift_freq = 50e6):
tx_sig_t = np.linspace(0, ts*sine.shape[-1], num=sine.shape[-1])
#tx_sig_sqrd = np.square(tx_sig_t)
#hift the sine
freq_shftd_sig = sine * np.exp(1.0j * 2 * np.pi * (shift_freq * tx_sig_t))
#freq_shftd_sig = sine * np.exp(1.0j * np.pi * (shift_freq * tx_sig_sqrd))
return freq_shftd_sig
def sweep_sine(sine, ts, up_lim = 50e6, low_lim = -50e6):
tx_sig_t = np.arange(0, sine.shape[-1])*ts
tx_sig_sqrd = np.square(tx_sig_t)
phi = low_lim*tx_sig_t + (up_lim-low_lim)*(tx_sig_sqrd/(2*ts*sine.shape[-1]))
dopp_shftd_sig = sine * np.exp(1.0j* 2 *np.pi * phi)
return dopp_shftd_sig
if __name__=='__main__':
#generate a sine wave 16 times over sampled
tx_sig, t_samp = gen_sig(freq=400e6, fs=6400e6, n_samp=6400)
#do an fft
tx_sig_fft = np.fft.fft(tx_sig)
#generate freqency axis for fft
freq_arr = np.fft.fftfreq(tx_sig.shape[-1], t_samp)
#shift sine wave
tx_sig_shifted = freq_shift_sine(tx_sig, t_samp, shift_freq = -100e6)
#fft the shifted sine
tx_sig_shftd_fft = np.fft.fft(tx_sig_shifted)
#sweep sine wave by up_lim+low_lim Hz
tx_sig_swept = sweep_sine(tx_sig, t_samp, up_lim = 50e6, low_lim = -50e6)
#fft the swept sine
tx_sig_swept_fft = np.fft.fft(tx_sig_swept)
plt.figure()
plt.plot(freq_arr, abs(tx_sig_fft))
plt.plot(freq_arr, abs(tx_sig_shftd_fft))
plt.plot(freq_arr, abs(tx_sig_swept_fft))
plt.axis([0,1e9, 0, 2e3])
plt.figure()
plt.plot(tx_sig)
plt.plot(tx_sig_shifted)
plt.plot(tx_sig_swept)
plt.axis([0,100, -1.2, 1.2])
I may be wrong but I think the problem is in your signal. You have only a real part of it, and shifting the phase in the complex plane doesn't help so much in the real part.
The possible treatment for this problem is to make the signal a complex one. The best way is to make a Hilbert transform and use it as a signal.
Your code may look like this
import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import hilbert
def gen_sine(freq=380e6, fs=6080e6, n_samp=6400):
ts = 1/fs
t_arr = np.arange(0, n_samp)*ts
#sine_sig = np.exp(2 * 1j * np.pi * freq * t_arr)
sine_sig = np.sin(2 * np.pi * freq * t_arr)
return sine_sig,ts
def freq_shift_sine(sine, ts, shift_freq = 50e6):
tx_sig_t = np.linspace(0, ts*sine.shape[-1], num=sine.shape[-1])
#tx_sig_sqrd = np.square(tx_sig_t)
#hift the sine
freq_shftd_sig = hilbert(tx_sig) * np.exp(1.0j * 2 * np.pi * (shift_freq * tx_sig_t))
#freq_shftd_sig = sine * np.exp(1.0j * np.pi * (shift_freq * tx_sig_sqrd))
return freq_shftd_sig
def sweep_sine(sine, ts, up_lim = 50e6, low_lim = -50e6):
#tx_sig_t = np.arange(0, sine.shape[-1])*ts
tx_sig_t = np.linspace(0, ts*sine.shape[-1], num=sine.shape[-1])
#tx_sig_sqrd = np.square(tx_sig_t)
freq_step_arr = np.linspace(low_lim, up_lim, sine.shape[-1])
dopp_shftd_sig = hilbert(tx_sig) * np.exp(1.0j * 2 * np.pi * (freq_step_arr * tx_sig_t))
#dopp_shftd_sig = sine * np.exp(1.0j * np.pi * (freq_step_arr * tx_sig_sqrd))
return dopp_shftd_sig
if __name__=='__main__':
#generate a sine wave 16 times over sampled
tx_sig, t_samp = gen_sine(freq=400e6, fs=6400e6, n_samp=6400)
#do an fft
tx_sig_fft = np.fft.fft(tx_sig)
#generate freqency axis for fft
freq_arr = np.fft.fftfreq(tx_sig.shape[-1], t_samp)
#shift sine wave
tx_sig_shifted = freq_shift_sine(tx_sig, t_samp, shift_freq = -100e6)
#fft the shifted sine
tx_sig_shftd_fft = np.fft.fft(tx_sig_shifted)
#sweep sine wave by up_lim+low_lim Hz
tx_sig_swept = sweep_sine(tx_sig, t_samp, up_lim = 50e6, low_lim = -50e6)
#fft the swept sine
tx_sig_swept_fft = np.fft.fft(tx_sig_swept)
#plt.figure()
#plt.plot(freq_arr, abs(tx_sig_swept_fft))
#plot sine wave fft
#plt.figure()
plt.figure(1)
plt.plot(freq_arr, abs(tx_sig_fft))
plt.plot(freq_arr, abs(tx_sig_shftd_fft))
plt.plot(freq_arr, abs(tx_sig_swept_fft))
plt.axis([0,1e9, 0, 2e3])
plt.figure(2)
plt.specgram(tx_sig_swept, NFFT=80, Fs=6400e6, noverlap=16)
#plt.axis([0,0.000001, 0, 5e6])
plt.figure(3)
plt.subplot(311)
t_time = np.arange(0, tx_sig.shape[-1])*t_samp
plt.plot(t_time, tx_sig)
plt.plot(t_time, np.imag(hilbert(tx_sig)) )
plt.subplot(312)
plt.plot(t_time, tx_sig_shifted)
plt.subplot(313)
plt.plot(t_time, tx_sig_swept )
plt.show()
It produces more or less ok spectrogram and doesn't corrupt the resulted signal. Hope it helps.
You have made a very common mistake. Incrementing both the frequency and the time (when you multiply them to create a phase parameter input for the exp or sine function) at each time step, increments the frequency too much. Instead figure out how much the phase should change at each new time step for the desired frequency at that time step, and add that phase delta instead of doing a multiplication to get a new phase input to your sine or exp function.
Here is my code, But it doesn't work
Load the dataset
dataset = np.loadtxt("filename", delimiter=",")
X = dataset[:,0:5]
Y = dataset[:,5]
compute spectrogram
N = 1e5
amp = 2 * np.sqrt(2)
noise_power = 0.001 * X / 2
time = np.arange(dataset) / X
freq = np.linspace(1e3, 2e3, N)
x = amp * np.sin(2*np.pi*freq*time)
x += np.random.normal(scale=np.sqrt(noise_power), size=time.shape)
Compute and plot the spectrogram.
f, t, Sxx = signal.spectrogram(x, X)
plt.pcolormesh(t, f, Sxx)
plt.ylabel('Frequency [Hz]')
plt.xlabel('Time [sec]')
plt.show()
this is a part of my dataset
33,4.9106E+13,-0.6946377,12.680544,0.50395286
33,4.91061E+13,5.012288,11.264028,0.95342433
33,4.91061E+13,4.903325,10.882658,-0.08172209
33,4.91062E+13,-0.61291564,18.496431,3.0237172
Your posted data does not have headers, so I'm guessing that your last 3 columns is actual acceleration, the first column is of no interest and the second column is symbolizing time. However if that is the timesignal you are sampling at what frequency?
You still have not really explained what is not as you expect, however it seems like you try to make a plot with several different channels. As far as I know one makes a spectrogram on one signal only.
Here is how I would treat similar data as you have, although I had to fake all data so that it becomes somewhat meaningful plots.
import numpy as np
import matplotlib.pyplot as plt
# faked data with noise & trends, with assumption of which column is what
fake_size = int(1e4)
time = np.arange(fake_size)/1000 # 1kHz
# your dataset[:,1] ?
base_freq = 2 * np.pi * 100
x = np.sin(4*base_freq*time) + 0.2 * np.random.random(fake_size)
# your dataset[:,2] ?
y = np.sin(2*base_freq*time) + 0.1 * np.random.random(fake_size) - 0.05 + time
# your dataset[:,3] ?
z = np.sin(3*base_freq*time) + np.sin(1.5*base_freq*time)+ 0.1 * np.random.random(fake_size) - 0.05
# your dataset[:,4] ?
xyz_magnitude = x**2 + y**2 + z**2
to_plot = [('x', x), ('y', y), ('z', z), ('xyz', xyz_magnitude)]
for chl, data in to_plot:
plt.figure(); plt.title(chl)
plt.specgram(data, Fs=1000)
plt.xlabel('Time [s]'); plt.ylabel('Frequency [Hz]')
This will quickly give a picture of what the frequencies are in these spectras, however if you are truly chasing relative magnitudes and or phases, you will have to do additional math as well as plots. Please note that your visual impression of the frequencies are dependent upon magnitude, so if you for example do np.sqrt on the xyz_magnitude the plot will not look exactly the same.