Develop Reference

Breaking apart a sin wave but phase doesn't seem to match up sometimes

I originally posted this in physics stack exchange but they requested it be posted here as well....
I am trying to create a known signal with a known wavelength, amplitude, and phase. I then want to break this signal apart into all of its frequencies, find amplitudes, phases, and wavelengths for each frequency, then create equations for each frequency based on these new wavelengths, amplitudes, and phases. In theory, the equations should be identical to the individual signals. However, they are not. I am almost positive it is an issue with phase but I cannot figure out how to resolve it. I will post the exact code to reproduce this below. Please help as my phase, wavelength, and amplitudes will vary once I get more complicated signals so it need to work for any combination of these.
import numpy as np
from matplotlib import pyplot as plt
from scipy import fftpack
# create signal
time_vec = np.arange(1, 11, 1)
wavelength = 1/.1
phase = 0
amp = 10
created_signal = amp * np.sin((2 * np.pi / wavelength * time_vec) + phase)
# plot it
fig, axs = plt.subplots(2, 1, figsize=(10,6))
axs[0].plot(time_vec, created_signal, label='exact_data')
# get fft and freq array
sig_fft = fftpack.fft(created_signal)
sample_freq = fftpack.fftfreq(created_signal.size, d=1)
# do inverse fft and verify same curve as original signal. This is fine!
filtered_signal = fftpack.ifft(sig_fft)
filtered_signal += np.mean(created_signal)
# create individual signals for each frequency
filtered_signals = []
for i in range(len(sample_freq)):
high_freq_fft = sig_fft.copy()
high_freq_fft[np.abs(sample_freq) < np.nanmin(sample_freq[i])] = 0
high_freq_fft[np.abs(sample_freq) > np.nanmax(sample_freq[i])] = 0
filtered_sig = fftpack.ifft(high_freq_fft)
filtered_sig += np.mean(created_signal)
filtered_signals.append(filtered_sig)
# get phase, amplitude, and wavelength for each individual frequency
sig_size = len(created_signal)
wavelength = []
ph = []
amp = []
indices = []
for j in range(len(sample_freq)):
wavelength.append(1 / sample_freq[j])
indices.append(int(sig_size * sample_freq[j]))
for j in indices:
phase = np.arctan2(sig_fft[j].imag, sig_fft[j].real)
ph.append([phase])
amp.append([np.sqrt((sig_fft[j].real * sig_fft[j].real) + (sig_fft[j].imag * sig_fft[j].imag)) / (sig_size / 2)])
# create an equation for each frequency based on each phase, amp, and wavelength found from above.
def eqn(filtered_si, wavelength, time_vec, phase, amp):
return amp * np.sin((2 * np.pi / wavelength * time_vec) + phase)
def find_equations(filtered_signals_mean, high_freq_fft, wavelength, filtered_signals, time_vec, ph, amp):
equations = []
for i in range(len(wavelength)):
temp = eqn(filtered_signals[i], wavelength[i], time_vec, ph[i], amp[i])
equations.append(temp + filtered_signals_mean)
return equations
filtered_signals_mean = np.abs(np.mean(filtered_signals))
equations = find_equations(filtered_signals_mean, sig_fft, wavelength,
filtered_signals, time_vec, ph, amp)
# at this point each equation, for each frequency should match identically each signal from each frequency,
# however, the phase seems wrong and they do not match!!??
axs[0].plot(time_vec, filtered_signal, '--', linewidth=3, label='filtered_sig_combined')
axs[1].plot(time_vec, filtered_signals[1], label='filtered_sig[-1]')
axs[1].plot(time_vec, equations[1], label='equations[-1]')
axs[0].legend()
axs[1].legend()
fig.tight_layout()
plt.show()

These are issues with your code:
filtered_signal = fftpack.ifft(sig_fft)
filtered_signal += np.mean(created_signal)
This only works because np.mean(created_signal) is approximately zero. The IFFT already takes the DC component into account, the zero frequency describes the mean of the signal.
filtered_signals = []
for i in range(len(sample_freq)):
high_freq_fft = sig_fft.copy()
high_freq_fft[np.abs(sample_freq) < np.nanmin(sample_freq[i])] = 0
high_freq_fft[np.abs(sample_freq) > np.nanmax(sample_freq[i])] = 0
filtered_sig = fftpack.ifft(high_freq_fft)
filtered_sig += np.mean(created_signal)
filtered_signals.append(filtered_sig)
Here you are, in the first half of the iterations, going through all the frequencies, taking both the negative and positive frequencies into account. For example, when i=1, you take both the -0.1 and the 0.1 frequencies. The second half of the iterations you are applying the IFFT to a zero signal, none of the np.abs(sample_freq) are smaller than zero by definition.
So the filtered_signals[1] contains a sine wave constructed by both the -0.1 and the 0.1 frequency components. This is good. Otherwise it would be a complex-valued function.
for j in range(len(sample_freq)):
wavelength.append(1 / sample_freq[j])
indices.append(int(sig_size * sample_freq[j]))
Here the second half of the indices array contains negative values. Not sure what you were planning with this, but it causes subsequent code to index from the end of the array.
for j in indices:
phase = np.arctan2(sig_fft[j].imag, sig_fft[j].real)
ph.append([phase])
amp.append([np.sqrt((sig_fft[j].real * sig_fft[j].real) + (sig_fft[j].imag * sig_fft[j].imag)) / (sig_size / 2)])
Here, because the indices are not the same as the j in the previous loop, phase[j] doesn't always correspond to wavelength[j], they refer to values from different frequency components in about half the cases. But those cases we shouldn't be evaluating any way. The code assumes a real-valued input, for which the magnitude and phase of only the positive frequencies is sufficient to reconstruct the signal. You should skip all the negative frequencies here.
Next, you build sine waves using the collected information, but using a time_vec that starts at 1, not at 0 as the FFT assumes. And therefore the signal is shifted with respect to the expected value. Furthermore, when phase==0, you should create an even signal (i.e. a cosine, not a sine).
Thus, changing the following two lines of code will create the correct output:
time_vec = np.arange(0, 10, 1)
and
def eqn(filtered_si, wavelength, time_vec, phase, amp):
return amp * np.cos((2 * np.pi / wavelength * time_vec) + phase)
# ^^^
Note that these two changes corrects the plotted graph, but doesn't correct all the issues in the code discussed above.

I solved this finally after 2 days of frustration. I still have no idea why this is the way it is so any insight would be great. The solution is to use the phase produced by arctan2(Im, Re) and modify it according to this equation.
phase = np.arctan2(sig_fft[j].imag, sig_fft[j].real)
formula = ((((wavelength[j]) / 2) - 2) * np.pi) / wavelength[j]
ph.append([phase + formula])
I had to derive this equation from data but I still do not know why this works. Please let me know. Finally!!

How do I calculate PSD, Median Frequency and Mean Frequency in python?

I am trying to extract frequency features from EMG Data on python with a sliding window. I do not have too much knowledge on frequency analysis, so I apologize in advance if I've got some wrong concepts.
I am trying to follow the definitions from this website:
MDF
more information
My questions are whether I am performing the calculations right, and if not how can I proceed to calculate them?
Windows_Size = 0.125
Overlap = 0.5
Bins = 256
START, END = df_filtered['Time'].min(), df_filtered['Time'].max()
Fs = 2000 #EMG Sampling Frequency
P = 1.0 / Fs
Windows = np.arange(START + Windows_Size, END, Windows_Size * (1 - Overlap))
FREQ = []
for w in WINDOWS:
win_start, win_end = w - Windows_Size, w
for var in ['Biceps_Femoris_Sq_Correct']:
value = df_filtered.loc[(win_start <= df_filtered['Time']) & (df_filtered['Time'] < win_end), var].values
fft = np.fft.fft(value * np.hamming(value.shape[0]), n=Bins)[1:Bins//2]
freq = np.fft.fftfreq(Bins, P)[1:Bins//2]
amp = np.abs(fft)
energy = amp ** 2 # Is this the right way to calculate power spectrum?
median_freq = energy/2
#mean_freq = EMG power spectrum is divided into two regions with equal
#amplitude... is this right?
mean_freq = np.sum(energy * freq) / np.sum(energy)
Average frequency which is calculated as the sum of product of the EMG power
spectrum, and the frequency divided by the total sum of the power spectrum. Is this the right way to calculate mean frequency?

the MDF should be something like
energy_cumsum = np.cumsum(energy)
MDF = freq[np.where(energy_cumsum>np.max(energy_cumsum)/2)[0][0]]

Numpy FFT error - Comb filter with envelope

I am currently having some issues understanding some discrepancies between the frequency response function calculated through the Z-transform and numpy's FFT algorithm. It is of a simple echo represented by the impulse response:
h[n] = δ[n] + αδ[n-L]
Where α is an attenuation factor for the echo and L is the delay amount in samples. The corresponding transfer function is given by:
H(f) = ( e^(j2πfΔL) + α ) / e^(j2πfΔL)
Where Δ is the sampling period.
I seem to be getting different results using the same number of frequency bins when directly plotting the transfer function magnitude above and when using numpy's fft algorithm.
In particular, the FFT magnitude seems to form an envelope around the overall spectrum - I believe that I should be getting a simple comb filter as that of the transfer function method: imgur
Could anyone clarify why this may be happening and whether I have potentially overlooked anything? Is this due to errors in the DFT algorithms employed?
Appreciate your time, cheers!
import matplotlib.pyplot as pyplt
import numpy as np
fs = 48000 # Sample rate
T = 1/fs # Sample period
L = 3000 # Delay
a = 0.5 # Attenuation factor
# h[n] = dirac[n] + a * dirac[n-L]
h = np.zeros(L)
h[0] = 1
h[L-1] = a
# Transfer function H
freqs = np.arange(0, fs, fs/(2*L))
e = np.exp(1j*freqs*2*np.pi*L*T)
H = (e + a)/(e)
# Transfer function H via FFT - same # of bins
H_FFT = np.fft.fft(h, 2*L)
pyplt.figure()
# Correct comb filter
pyplt.plot(np.abs(H))
# Runing FFT gives a form of strange envelope error
pyplt.plot(np.abs(H_FFT))
pyplt.legend()

Your code is almost OK. What you need to change:
As far as I understand there is no reason the length of Fourier transform vector will be proportional to L. It needs to be the size of your sampling frequency, i.e. fs.
Your L is too high. The result oscillates too quickly. Try with lower L.
Here is a modified code to show how it works, plotted in two different figures for clarity:
import matplotlib.pyplot as plt
import numpy as np
fs = 48000 # Sample rate
T = 1/fs # Sample period
L = 3 # Delay
a = 0.5 # Attenuation factor
# h[n] = dirac[n] + a * dirac[n-L]
h = np.zeros(fs)
h[0] = 1
h[L] = a
# Transfer function H
freqs = np.arange(0, fs)
e = np.exp(1j*freqs*2*np.pi*L*T)
H = (e + a)/(e)
# Transfer function H via FFT - same # of bins
H_FFT = np.fft.fft(h)
# Correct comb filter
plt.figure()
plt.plot(np.abs(H))
# Runing FFT gives a form of strange envelope error
plt.figure()
plt.plot(np.abs(H_FFT))

python - how can I generate a WAV file with beeps?

is there a way in python to generate a continuous series of beeps in increasing amplitude and export it into a WAV file?

I've based this on the answer to the previous question and added a lot of comments. Hopefully this makes it clear. You'll probably want to introduce a for loop to control the number of beeps and the increasing volume.
#!/usr/bin/python
# based on : www.daniweb.com/code/snippet263775.html
import math
import wave
import struct
# Audio will contain a long list of samples (i.e. floating point numbers describing the
# waveform). If you were working with a very long sound you'd want to stream this to
# disk instead of buffering it all in memory list this. But most sounds will fit in
# memory.
audio = []
sample_rate = 44100.0
def append_silence(duration_milliseconds=500):
"""
Adding silence is easy - we add zeros to the end of our array
"""
num_samples = duration_milliseconds * (sample_rate / 1000.0)
for x in range(int(num_samples)):
audio.append(0.0)
return
def append_sinewave(
freq=440.0,
duration_milliseconds=500,
volume=1.0):
"""
The sine wave generated here is the standard beep. If you want something
more aggresive you could try a square or saw tooth waveform. Though there
are some rather complicated issues with making high quality square and
sawtooth waves... which we won't address here :)
"""
global audio # using global variables isn't cool.
num_samples = duration_milliseconds * (sample_rate / 1000.0)
for x in range(int(num_samples)):
audio.append(volume * math.sin(2 * math.pi * freq * ( x / sample_rate )))
return
def save_wav(file_name):
# Open up a wav file
wav_file=wave.open(file_name,"w")
# wav params
nchannels = 1
sampwidth = 2
# 44100 is the industry standard sample rate - CD quality. If you need to
# save on file size you can adjust it downwards. The stanard for low quality
# is 8000 or 8kHz.
nframes = len(audio)
comptype = "NONE"
compname = "not compressed"
wav_file.setparams((nchannels, sampwidth, sample_rate, nframes, comptype, compname))
# WAV files here are using short, 16 bit, signed integers for the
# sample size. So we multiply the floating point data we have by 32767, the
# maximum value for a short integer. NOTE: It is theortically possible to
# use the floating point -1.0 to 1.0 data directly in a WAV file but not
# obvious how to do that using the wave module in python.
for sample in audio:
wav_file.writeframes(struct.pack('h', int( sample * 32767.0 )))
wav_file.close()
return
append_sinewave(volume=0.25)
append_silence()
append_sinewave(volume=0.5)
append_silence()
append_sinewave()
save_wav("output.wav")

I added minor improvements to the JCx code above. As author said, its not cool to use global variables. So I wrapped his solution into class, and it works just fine:
import math
import wave
import struct
class BeepGenerator:
def __init__(self):
# Audio will contain a long list of samples (i.e. floating point numbers describing the
# waveform). If you were working with a very long sound you'd want to stream this to
# disk instead of buffering it all in memory list this. But most sounds will fit in
# memory.
self.audio = []
self.sample_rate = 44100.0
def append_silence(self, duration_milliseconds=500):
"""
Adding silence is easy - we add zeros to the end of our array
"""
num_samples = duration_milliseconds * (self.sample_rate / 1000.0)
for x in range(int(num_samples)):
self.audio.append(0.0)
return
def append_sinewave(
self,
freq=440.0,
duration_milliseconds=500,
volume=1.0):
"""
The sine wave generated here is the standard beep. If you want something
more aggresive you could try a square or saw tooth waveform. Though there
are some rather complicated issues with making high quality square and
sawtooth waves... which we won't address here :)
"""
num_samples = duration_milliseconds * (self.sample_rate / 1000.0)
for x in range(int(num_samples)):
self.audio.append(volume * math.sin(2 * math.pi * freq * ( x / self.sample_rate )))
return
def save_wav(self, file_name):
# Open up a wav file
wav_file=wave.open(file_name,"w")
# wav params
nchannels = 1
sampwidth = 2
# 44100 is the industry standard sample rate - CD quality. If you need to
# save on file size you can adjust it downwards. The stanard for low quality
# is 8000 or 8kHz.
nframes = len(self.audio)
comptype = "NONE"
compname = "not compressed"
wav_file.setparams((nchannels, sampwidth, self.sample_rate, nframes, comptype, compname))
# WAV files here are using short, 16 bit, signed integers for the
# sample size. So we multiply the floating point data we have by 32767, the
# maximum value for a short integer. NOTE: It is theortically possible to
# use the floating point -1.0 to 1.0 data directly in a WAV file but not
# obvious how to do that using the wave module in python.
for sample in self.audio:
wav_file.writeframes(struct.pack('h', int( sample * 32767.0 )))
wav_file.close()
return
if __name__ == "__main__":
bg = BeepGenerator()
bg.append_sinewave(volume=0.25, duration_milliseconds=100)
bg.append_silence()
bg.append_sinewave(volume=0.5, duration_milliseconds=700)
bg.append_silence()
bg.save_wav("output.wav")

I adjusted it a bit further, now it should be a lot faster, and I added a function for playing multiple tones at the same time.
import numpy as np
import scipy.io.wavfile
class BeepGenerator:
def __init__(self):
# Audio will contain a long list of samples (i.e. floating point numbers describing the
# waveform). If you were working with a very long sound you'd want to stream this to
# disk instead of buffering it all in memory list this. But most sounds will fit in
# memory.
self.audio = []
self.sample_rate = 44100.0
def append_silence(self, duration_milliseconds=500):
"""
Adding silence is easy - we add zeros to the end of our array
"""
num_samples = duration_milliseconds * (self.sample_rate / 1000.0)
for x in range(int(num_samples)):
self.audio.append(0.0)
return
def append_sinewave(
self,
freq=440.0,
duration_milliseconds=500,
volume=1.0):
"""
The sine wave generated here is the standard beep. If you want something
more aggressive you could try a square or saw tooth waveform. Though there
are some rather complicated issues with making high quality square and
sawtooth waves... which we won't address here :)
"""
num_samples = duration_milliseconds * (self.sample_rate / 1000.0)
x = np.array([i for i in range(int(num_samples))])
sine_wave = volume * np.sin(2 * np.pi * freq * (x / self.sample_rate))
self.audio.extend(list(sine_wave))
return
def append_sinewaves(
self,
freqs=[440.0],
duration_milliseconds=500,
volumes=[1.0]):
"""
The sine wave generated here is the standard beep. If you want something
more aggressive you could try a square or saw tooth waveform. Though there
are some rather complicated issues with making high quality square and
sawtooth waves... which we won't address here :)
len(freqs) must be the same as len(volumes)
"""
volumes = list(np.array(volumes)/sum(volumes))
num_samples = duration_milliseconds * (self.sample_rate / 1000.0)
x = np.array([i for i in range(int(num_samples))])
first_it = True
for volume, freq in zip(volumes, freqs):
print(freq)
if first_it:
sine_wave = volume * np.sin(2 * np.pi * freq * (x / self.sample_rate))
first_it = False
else:
sine_wave += volume * np.sin(2 * np.pi * freq * (x / self.sample_rate))
self.audio.extend(list(sine_wave))
return
def save_wav(self, file_name):
# Open up a wav file
# wav params
# 44100 is the industry standard sample rate - CD quality. If you need to
# save on file size you can adjust it downwards. The standard for low quality
# is 8000 or 8kHz.
# WAV files here are using short, 16 bit, signed integers for the
# sample size. So we multiply the floating point data we have by 32767, the
# maximum value for a short integer. NOTE: It is theoretically possible to
# use the floating point -1.0 to 1.0 data directly in a WAV file but not
# obvious how to do that using the wave module in python.
self.audio = np.array(self.audio).astype(np.float32)
scipy.io.wavfile.write(file_name, int(self.sample_rate), np.array(self.audio))
return
if __name__ == "__main__":
bg = BeepGenerator()
bg.append_sinewave(volume=1, duration_milliseconds=100)
bg.append_silence()
bg.append_sinewave(volume=0.5, duration_milliseconds=700)
bg.append_silence()
bg.append_sinewaves(volumes=[1, 1], duration_milliseconds=700, freqs=[880, 660])
bg.append_silence()
bg.save_wav("output.wav")

Python Code: Geometric Brownian Motion - what's wrong?

I'm pretty new to Python, but for a paper in University I need to apply some models, using preferably Python. I spent a couple of days with the code I attached, but I can't really help, what's wrong, it's not creating a random process which looks like standard brownian motions with drift. My parameters like mu and sigma (expected return or drift and volatility) tend to change nothing but the slope of the noise process. That's my problem, it all looks like noise. Hope my problem is specific enough, here is my coode:
import math
from matplotlib.pyplot import *
from numpy import *
from numpy.random import standard_normal
'''
geometric brownian motion with drift!
Spezifikationen:
mu=drift factor [Annahme von Risikoneutralitaet]
sigma: volatility in %
T: time span
dt: lenght of steps
S0: Stock Price in t=0
W: Brownian Motion with Drift N[0,1]
'''
T=1
mu=0.025
sigma=0.1
S0=20
dt=0.01
Steps=round(T/dt)
t=(arange(0, Steps))
x=arange(0, Steps)
W=(standard_normal(size=Steps)+mu*t)### standard brownian motion###
X=(mu-0.5*sigma**2)*dt+(sigma*sqrt(dt)*W) ###geometric brownian motion####
y=S0*math.e**(X)
plot(t,y)
show()

According to Wikipedia,
So it appears that
X=(mu-0.5*sigma**2)*t+(sigma*W) ###geometric brownian motion####
rather than
X=(mu-0.5*sigma**2)*dt+(sigma*sqrt(dt)*W)
Since T represents the time horizon, I think t should be
t = np.linspace(0, T, N)
Now, according to these Matlab examples (here and here), it appears
W = np.random.standard_normal(size = N)
W = np.cumsum(W)*np.sqrt(dt) ### standard brownian motion ###
not,
W=(standard_normal(size=Steps)+mu*t)
Please check the math, however, I could be wrong.
So, putting it all together:
import matplotlib.pyplot as plt
import numpy as np
T = 2
mu = 0.1
sigma = 0.01
S0 = 20
dt = 0.01
N = round(T/dt)
t = np.linspace(0, T, N)
W = np.random.standard_normal(size = N)
W = np.cumsum(W)*np.sqrt(dt) ### standard brownian motion ###
X = (mu-0.5*sigma**2)*t + sigma*W
S = S0*np.exp(X) ### geometric brownian motion ###
plt.plot(t, S)
plt.show()
yields

An additional implementation using the parametrization of the gaussian law though the normal fonction (instead of standard_normal), a bit shorter.
import numpy as np
T = 2
mu = 0.1
sigma = 0.01
S0 = 20
dt = 0.01
N = round(T/dt)
# reversely you can specify N and then compute dt, which is more common in financial litterature
X = np.random.normal(mu * dt, sigma* np.sqrt(dt), N)
X = np.cumsum(X)
S = S0 * np.exp(X)