Develop Reference

Audio Frequency Separation In Python

I'm working on a nerd project for fun.
The project is analog video recorded onto an audio cassette.
The challenge lies in the very limited bandwidth.
I have a method to record color video at 24fps along with mono audio.
I got the video stuff working but need some help with the audio.
Here is the signal I have to work with:
Note: using YUV color space
Left channel:
Sync Pulses &
Y (luma) data
Right channel:
U & V (chroma) data
mixed with
Mono audio (Amplitude Modulated at 14kHz)
I'm not sure how to separate the color data from the audio.
I've looked into FFT with numpy a bit but am not fully understanding it.
Basically what I need is a band filter to separate 13990Hz - 14010Hz (to account for wow and flutter)

Ok here is a little test code that shows how this works.
import matplotlib.pyplot as plt
import numpy as np
import math as mt
from scipy.signal import butter, sosfilt, lfilter
def butter_bandpass(lowcut, highcut, fs, order=5):
nyq = 0.5 * fs
low = lowcut / nyq
high = highcut / nyq
sos = butter(order, [low, high], analog=False, btype='band', output='sos')
return sos
def bandpass(data, lowcut, highcut, fs, order=5):
sos = butter_bandpass(lowcut, highcut, fs, order=order)
y = sosfilt(sos, data)
return y
def bandstop(data, lowcut, highcut, fs, order):
nyq = 0.5 * fs
low = lowcut / nyq
high = highcut / nyq
i, u = butter(order, [low, high], btype='bandstop')
y = lfilter(i, u, data)
return y
# Modulation & Bandpass Test
# note that the data is just test data and isn't actual audio or color data
Fs = 192000 # rate
f = 14000 # carrier frequency (Hz)
sample = 192000 # total length
x = np.arange(sample)
signal = (np.sin(2 * np.pi * 1000 * x / Fs)) # audio
noise = 0.5*(np.sin(2 * np.pi * 10000 * x / Fs)) # color data
y = [] # combined AM audio and color data
for t in range(0, sample, 1):
amp = (signal[t] + 1) / 2
sine = amp * (mt.sin(2*mt.pi * f * t / Fs))
y.append((sine+noise[t])/2)
# y is raw signal
w = bandpass(y, 1600, 1800, 24000, order=1) # only AM audio signal
v = bandstop(y, 1450, 1950, 24000, order=6) # Rest of the signal
# Note: lowered the sample rate input for the filters (and frequencies accordingly)
# since it didn't like 192khz
# The color data does get impacted by the bandstop filter but this is
# mostly not noticable as the YUV color space is used meaning
# the color resolution can be reduced drastically without noticable effect
plt.plot(y)
plt.plot(w)
plt.plot(v)
plt.xlabel('sample')
plt.ylabel('amplitude')
plt.show()
If you want to check out the full code along with a wav of the signal and an example video of the output here's a link:
https://drive.google.com/drive/folders/18ogpK4n43d_Q0tjdmlm2uIRZBIrRu01y?usp=sharing

how do I remove large spike at the beginning of a signal after applying Butterworth Bandpass filter in python?

I want to detrend a signal using a bandpass filter. I used a Butterworth filter with FL=0.1 Hz and FH=20Hz in python but after applying this bandpass filter I observed a large spike at the beginning of the detrended signal.
what is this spike for? and how do I remove this spike in python?
you can download "data1.csv" using this link.
from scipy.signal import butter, lfilter
from numpy import genfromtxt
import numpy as np
import matplotlib.pyplot as plt
def butter_bandpass(lowcut, highcut, fs, order=5):
nyq = 0.5 * fs
low = lowcut / nyq
high = highcut / nyq
b, a = butter(order, [low, high], btype='band')
return b, a
def butter_bandpass_filter(data, lowcut, highcut, fs, order=5):
b, a = butter_bandpass(lowcut, highcut, fs, order=order)
y = lfilter(b, a, data)
return y
BP_without_NaN = genfromtxt('data1.csv', delimiter=',')
framerate=1024
# detrending [0.1Hz 20Hz]
OMW = butter_bandpass_filter(data = BP_without_NaN, lowcut = 0.1, highcut = 20 , fs = framerate, order = 3)
# plot OMW
time = np.linspace(0, len(OMW)/framerate ,len(OMW))
plt.plot(time, OMW)
plt.show()

When applying a Butterworth (or any IIR) filter, each output sample is computed based on previous output samples,
y[n] = b0 * x[n] + b1 * x[n-1] + ... + bk * x[n-k]
- a1 * y[n-1] + ... + ak * y[n-k]
This makes a problem for how start the filter at the beginning of the signal, since no output samples have been computed yet. A typical way handling this is to assume that all outputs at negative times n < 0 are zero, or equivalently, to conceptually extrapolate the input as zero. This zero initialization approach is what scipy.signal.lfilter does by default.
However, this approach is not always a good fit. If the input signal does not begin close to zero, then extrapolating zeros introduces an artificial step or jump discontinuity in the signal, and the output signal will begin with an unwanted transient, the filter's response to this artificial step.
Looking at your linked data, the input begins with a few hundred samples equal to 154. This explains the large spike at the beginning of your plot. You can mostly remove the spike by changing the filter initialization. Use scipy.signal.lfilter_zi to compute initial conditions for lfilter for step response steady-state of value 154:
zi = lfilter_zi(b, a) * data[0]
Then use it with lfilter as y, _ = lfilter(b, a, data, zi=zi). Here is what I get with this change:

Why are margin-values of butterworth bandpass filter so unrealistically big?

I use a butterworth bandpass:
def butter_bandpass_prep(lowcut, highcut, fs, order=5):
"""Butterworth bandpass auxilliary function."""
nyq = 0.5 * fs # Minimal frequency (nyquist criterion)
low = lowcut / nyq
high = highcut / nyq
b, a = sp.signal.butter(order, [low, high], btype='band')
return b, a
def butter_bandpass(src, lowcut, highcut, fs, order=5):
"""Butterworth bandpass filter."""
b, a = butter_bandpass_prep(lowcut, highcut, fs, order=order)
dst = sp.signal.filtfilt(b, a, src)
return dst
I general it seems to work well, but at the margins I always get some unrealistic values. It seems as if the filtered margin-values drop to zero...
Is there a way to prevent that from happening?
Thank you a lot!

Python High Pass Filter

I implemented an high pass filter in python using this code:
from scipy.signal import butter, filtfilt
import numpy as np
def butter_highpass(cutoff, fs, order=5):
nyq = 0.5 * fs
normal_cutoff = cutoff / nyq
b, a = butter(order, normal_cutoff, btype='high', analog=False)
return b, a
def butter_highpass_filter(data, cutoff, fs, order=5):
b, a = butter_highpass(cutoff, fs, order=order)
y = filtfilt(b, a, data)
return y
rawdata = np.loadtxt('sampleSignal.txt', skiprows=0)
signal = rawdata
fs = 100000.0
cutoff = 100
order = 6
conditioned_signal = butter_highpass_filter(signal, cutoff, fs, order)
I am applying this filter on a 100 kHz voltage signal and it works fine for cutoff frequencies >= 60 Hz. But it doesn't work below. I would like to cut off all the frequencies below 10 Hz. Any hints where my mistake is? What I observed is the lower the order of the filter the lower the cutoff frequency could be.
The sample Signal can be found here.

I hope this can help you:
import numpy as np
import pandas as pd
from scipy import signal
import matplotlib.pyplot as plt
def sine_generator(fs, sinefreq, duration):
T = duration
nsamples = fs * T
w = 2. * np.pi * sinefreq
t_sine = np.linspace(0, T, nsamples, endpoint=False)
y_sine = np.sin(w * t_sine)
result = pd.DataFrame({
'data' : y_sine} ,index=t_sine)
return result
def butter_highpass(cutoff, fs, order=5):
nyq = 0.5 * fs
normal_cutoff = cutoff / nyq
b, a = signal.butter(order, normal_cutoff, btype='high', analog=False)
return b, a
def butter_highpass_filter(data, cutoff, fs, order=5):
b, a = butter_highpass(cutoff, fs, order=order)
y = signal.filtfilt(b, a, data)
return y
fps = 30
sine_fq = 10 #Hz
duration = 10 #seconds
sine_5Hz = sine_generator(fps,sine_fq,duration)
sine_fq = 1 #Hz
duration = 10 #seconds
sine_1Hz = sine_generator(fps,sine_fq,duration)
sine = sine_5Hz + sine_1Hz
filtered_sine = butter_highpass_filter(sine.data,10,fps)
plt.figure(figsize=(20,10))
plt.subplot(211)
plt.plot(range(len(sine)),sine)
plt.title('generated signal')
plt.subplot(212)
plt.plot(range(len(filtered_sine)),filtered_sine)
plt.title('filtered signal')
plt.show()

Since my reputation is low I am unable to comment on your question - "What is the relationship between cutoff and filter order?" This is not an answer to your original question.
For an FIR filter, for a given cutoff frequency, the slope of the impulse response plot (|H(f)| vs f) is steeper for a higher order filter. So, to achieve higher attenuation for the undesired frequency range, you increase the filter order. But what happens when the filter order is so high that the impulse response is an ideal box function? You will see an inter-symbol interference (ISI in digital communications) like effect. The intensity of this effect increases when the ratio of cutoff frequency to the sampling frequency gets smaller (think of the relation between the width of box function in frequency domain and the width of the main lobe of the sinc function).
I first observed this when I tried to implement a very narrow band low-pass IIR filter on a TI DSP microcontroller. The TI library, implemented the filter as a cascaded bi-quad structure to handle the well known truncation effects. This still did not solve the problem because the problem is not due to truncation alone. The way I solved this problem was to use an anti-aliasing filter followed by down-sampling the input signal, followed my desired low-pass IIR filter.
I understand that you are implementing a HPF, which is an LPF translated in frequency domain. Hope this answers some of your questions. Let me know if down-sampling works for you.

I add function to verify above Konstantin Purtov's code, so you could see relationship between order and cut off frequency. The code is mostly from Warren Weckesser.
import scipy
import sys
from scipy import signal
from scipy import pi
from scipy.io.wavfile import write
import matplotlib.pyplot as plt
import numpy as np
from scipy.signal import butter, lfilter, freqz
# ----- ----- ----- -----
def butter_highpass(cutoff, fs, order=5):
nyq = 0.5 * fs
normal_cutoff = cutoff / nyq
b, a = signal.butter(order, normal_cutoff, btype='high', analog=False)
return b, a
def butter_highpass_filter(data, cutoff, fs, order=5):
b, a = butter_highpass(cutoff, fs, order=order)
y = signal.filtfilt(b, a, data)
return y
# ----- -----
# (1)
def foo(sel):
if (sel == 1):
# Filter requirements.
order = 6
fs = 300.0 # sample rate, Hz
cutoff = 10 # desired cutoff frequency of the filter, Hz
# Get the filter coefficients so we can check its frequency response.
b, a = butter_highpass(cutoff, fs, order)
# Plot the frequency response.
w, h = freqz(b, a, worN=8000)
plt.subplot(2, 1, 1)
plt.plot(0.5 * fs * w / np.pi, np.abs(h), 'b')
plt.plot(cutoff, 0.5 * np.sqrt(2), 'ko')
plt.axvline(cutoff, color='k')
plt.xlim(0, 0.5 * fs)
plt.title("High Filter Frequency Response")
plt.xlabel('Frequency [Hz]')
plt.grid()
# Demonstrate the use of the filter.
# First make some data to be filtered.
T = 0.5 # seconds
n = int(T * fs) # total number of samples
t = np.linspace(0, T, n, endpoint=False)
# "Noisy" data. We want to recover the 20 Hz signal from this.
data = np.sin(1.2 * 2 * np.pi * t) + 1.5 * np.cos(5 * 2 * np.pi * t) + 0.5 * np.sin(20.0 * 2 * np.pi * t)
# Filter the data, and plot both the original and filtered signals.
y = butter_highpass_filter(data, cutoff, fs, order)
plt.subplot(2, 1, 2)
plt.plot(t, data, 'b-', label='data')
plt.plot(t, y, 'g-', linewidth=2, label='filtered data')
plt.xlabel('Time [sec]')
plt.grid()
plt.legend()
plt.subplots_adjust(hspace=0.35)
plt.show()
else:
print ('Please, choose among choices, thanks.')
# ----- -----
def main():
sel = int (sys.argv[1])
foo(sel)
# ----- ----- ----- ----- ----- -----
if __name__ == '__main__':
main()

Calculate energy for each frequency band around frequency F of interest in Python

I am newbie in signal processing, in this question, i want to ask how to obtain energy for each frequency band around interested frequency F. I have found a formula, but I dont know how to implement it in Python. This is the formula and my Fourier transform plot:
x = np.linspace(0,5,100)
y = np.sin(2*np.pi*x)
## fourier transform
f = np.fft.fft(y)
## sample frequencies
freq = np.fft.fftfreq(len(y), d=x[1]-x[0])
plt.plot(freq, abs(f)**2) ## will show a peak at a frequency of 1 as it should.

You are almost there as Mike pointed but here is a different approach which is simpler to understand.you can set a variable that holds the filtered signal and return a 1d array of Af, then apply the above formula which is quite simple (squared sum of these amplitudes)
Filter out the signals like this
from scipy.signal import butter, lfilter
def butter_bandpass(lowcut, highcut, fs, order=5):
nyq = 0.5 * fs
low = lowcut / nyq
high = highcut / nyq
b, a = butter(order, [low, high], btype='band')
return b, a
def butter_bandpass_filter(data, lowcut, highcut, fs, order=5):
b, a = butter_bandpass(lowcut, highcut, fs, order=order)
y = lfilter(b, a, data)
return y
now assuming y is your original signal and you need energy of 5Hz component in signal ,
#let fs = 250
#let order = 5
oneD_array_of_amps_of_fiveHz_component = butter_bandpass_filter(y, 4, 6, 250, 5)
#calculate energy like this
energy_of_fiveHz_comp = sum([x*2 for x in oneD_array_of_amps_of_fiveHz_component])

Using the signal processing module from Finding local maxima/minima with Numpy in a 1D numpy array you would do the following:
from scipy.signal import argrelextrema
import numpy as np
delta = 0.1
F = 1
X_delta = abs(freq - F) < delta
A_f_delta = abs(f[X_delta])**2
maximum_ix = argrelextrema(A_f, np.greater)
E_F_delta = np.sum(A_f[maximum_ix])/2
E_F_delta
2453.8235776144866