Develop Reference

Realtime step detection from wav file

My goal is to take a real time audio stream, and find the steps in it to signal my lights to flash to it.
Right now I have this code:
import pyaudio
import numpy as np
import matplotlib.pyplot as plt
CHUNK = 2**5
RATE = 44100
LEN = 3
p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paInt16, channels=1, rate=RATE, input=True, frames_per_buffer=CHUNK)
print(1)
frames = []
n = 0
for i in range(int(LEN*RATE/CHUNK)): #go for a LEN seconds
n += 1
data = np.fromstring(stream.read(CHUNK),dtype=np.int16)
num = 0
for ii in data:
num += abs(ii)
print(num)
frames.append(data)
stream.stop_stream()
stream.close()
p.terminate()
plt.figure(1)
plt.title("Signal Wave...")
plt.plot(frames)
open("frames.txt", "w").write(str(frames))
It takes the live audio steam created by pyaudio in this format
[[0,0,-1,0,0,0,0,-1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,-1,0,0,0,0,0,0,0,0,0]),[1,0,-1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,-1,0,0,0,0,0,0,0,0,0]]
(this is depicting silence)
and adds all of the number together after they have gone through the abs() function (absolute value)
This gives an accurate(ish) representation of what a graph like this looks like
I see the numbers getting larger and the big jumps should be easy to calculate, but the smaller jumps are almost indistinguishable from silence.
I found this answer that seems right, but i dont know how to use it.
Any help would be appreciated
Thanks!

calculating FFT in frames and writing to a file

I'm new to python,I'm trying get a FFT value of a uploaded wav file and return the FFT of each frame in each line of a text file (using GCP)
using scipy or librosa
Frame rate i require is 30fps
wave file will be of 48k sample rate
so my questions are
how do i divide the samples for the whole wav file into samples of each frame
How do add empty samples to make the length of the frame samples power of 2 (as 48000/30 = 1600 add 448 empty samples to make it 2048)
how do i normalize the resulting FFT array to [-1,1]?

You can use pyaudio with callback to acheive whatever you are doing.
import pyaudio
import wave
import time
import struct
import sys
import numpy as np
if len(sys.argv) < 2:
print("Plays a wave file.\n\nUsage: %s filename.wav" % sys.argv[0])
sys.exit(-1)
wf = wave.open(sys.argv[1], 'rb')
# instantiate PyAudio (1)
p = pyaudio.PyAudio()
def callback_test(data, frame_count, time_info, status):
frame_count =1024
elm = wf.readframes(frame_count) # read n frames
da_i = np.frombuffer(elm, dtype='<i2') # convert to little endian int pairs
da_fft = np.fft.rfft(da_i) # fast fourier transform for real values
da_ifft = np.fft.irfft(da_fft) # inverse fast fourier transform for real values
da_i = da_ifft.astype('<i2') # convert to little endian int pairs
da_m = da_i.tobytes() # convert to bytes
return (da_m, pyaudio.paContinue)
# open stream using callback (3)
stream = p.open(format=p.get_format_from_width(wf.getsampwidth()),
channels=wf.getnchannels(),
rate=wf.getframerate(),# sampling frequency
output=True,
stream_callback=callback_test)
# # start the stream (4)
stream.start_stream()
# # wait for stream to finish (5)
while stream.is_active():
time.sleep(0.1)
# # stop stream (6)
stream.stop_stream()
stream.close()
wf.close()
# close PyAudio (7)
p.terminate()
Please refer these links for further study:
https://people.csail.mit.edu/hubert/pyaudio/docs/#example-callback-mode-audio-i-o
and
Python change pitch of wav file

Changing sound waves in-place: frequency and amplitude

I'm trying to make some changes to a sound frequency and amplitude in place. I am currently getting the sound data, but whenever I try multiplying it by a value, for example to change amplitude, I get a lot of noise. I was wondering how to do that in a clean way. I need to loop through the data because the changes in frequency and amplitude rely on user input (I'll later make changes according to hand position using a webcam). With this current code, my input file had a single channel. I am not sure why, but in "getNewWave", if I change it to np.int16, the audio gets noise as well. Thanks!!
import pyaudio
import wave
import sys
import numpy as np
def getNewWave(data):
newdata = np.frombuffer(data, np.int8)
#make some changes to amplitude and frequency
return newdata
def main():
CHUNK = 1024
if len(sys.argv) < 2:
print("Missing input wav file. File must have single channel")
sys.exit(-1)
wf = wave.open(sys.argv[1], 'rb')
p = pyaudio.PyAudio()
stream = p.open(format=pyaudio.paInt16,
channels=1,
rate=wf.getframerate(),
output=True, frames_per_buffer=CHUNK)
data = wf.readframes(CHUNK)
while data != '':
stream.write(getNewWave(data))
data = wf.readframes(CHUNK)
stream.stop_stream()
stream.close()
p.terminate()
main()

Pyaudio recording wav file with soundcard gives an empty recording

I've been trying to work on a project to detect time shift between two streaming audio signals. I worked with python3, Pyaudio and I'm using a Motux828 sound card with a Neumann KU-100 microphone which takes a stereo input. So when i check my input_device_index I am the correct one which is the 4th one connnected to MOTU soundcard.
However when i record with:
import time
import pyaudio
import wave
CHUNK = 1024 * 3 # Chunk is the bytes which are currently processed
FORMAT = pyaudio.paInt16
RATE = 44100
RECORD_SECONDS = 2
WAVE_OUTPUT = "temp.wav"
p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,channels=2,rate=RATE,input=True,frames_per_buffer=CHUNK,input_device_index=4)
frames = [] # np array storing all the data
for i in range(0, int(RATE / CHUNK * RECORD_SECONDS)):
data = stream1.read(CHUNK)
frames.append(data1)
stream.stop_stream()
stream.close()
p.terminate()
wavef = wave.open(WAVE_OUTPUT, 'wb') # opening the file
wavef.setnchannels(1)
wavef.setsampwidth(p.get_sample_size(FORMAT))
wavef.setframerate(RATE)
wavef.writeframes(b''.join(frames1)) # writing the data to be saved
wavef.close()
I record a wave file with no sound, with almost no noise(naturally)
Also I can record with 3rd party softwares with the specific microphone.
It works completely, fine.
NOTE:
Sound card is 24-bit depth normally, I also tried paInt24 that records a wave file with pure noise

I think u mentioned wrong variable names as i seen your code. The wrong variables are :
data = stream1.read(CHUNK)
frames.append(data1)
wavef.writeframes(b''.join(frames1))
the correct values are :
data = stream.read(CHUNK)
frames.append(data)
wavef.writeframes(b''.join(frames))

How to handle in_data in Pyaudio callback mode?

I'm doing a project on Signal Processing in python. So far I've had a little succes with the nonblocking mode, but it gave a considerable amount of delay and clipping to the output.
I want to implement a simple real-time audio filter using Pyaudio and Scipy.Signal, but in the callback function provided in the pyaudio example when I want to read the in_data I can't process it. Tried converting it in various ways but with no success.
Here's a code I want to achieve(read data from mic, filter, and output ASAP):
import pyaudio
import time
import numpy as np
import scipy.signal as signal
WIDTH = 2
CHANNELS = 2
RATE = 44100
p = pyaudio.PyAudio()
b,a=signal.iirdesign(0.03,0.07,5,40)
fulldata = np.array([])
def callback(in_data, frame_count, time_info, status):
data=signal.lfilter(b,a,in_data)
return (data, pyaudio.paContinue)
stream = p.open(format=pyaudio.paFloat32,
channels=CHANNELS,
rate=RATE,
output=True,
input=True,
stream_callback=callback)
stream.start_stream()
while stream.is_active():
time.sleep(5)
stream.stop_stream()
stream.close()
p.terminate()
What is the right way to do this?

Found the answer to my question in the meantime, the callback looks like this:
def callback(in_data, frame_count, time_info, flag):
global b,a,fulldata #global variables for filter coefficients and array
audio_data = np.fromstring(in_data, dtype=np.float32)
#do whatever with data, in my case I want to hear my data filtered in realtime
audio_data = signal.filtfilt(b,a,audio_data,padlen=200).astype(np.float32).tostring()
fulldata = np.append(fulldata,audio_data) #saves filtered data in an array
return (audio_data, pyaudio.paContinue)

I had a similar issue trying to work with the PyAudio callback mode, but my requirements where:
Working with stereo output (2 channels).
Processing in real time.
Processing the input signal using an arbitrary impulse response, that could change in the middle of the process.
I succeeded after a few tries, and here are fragments of my code (based on the PyAudio example found here):
import pyaudio
import scipy.signal as ss
import numpy as np
import librosa
track1_data, track1_rate = librosa.load('path/to/wav/track1', sr=44.1e3, dtype=np.float64)
track2_data, track2_rate = librosa.load('path/to/wav/track2', sr=44.1e3, dtype=np.float64)
track3_data, track3_rate = librosa.load('path/to/wav/track3', sr=44.1e3, dtype=np.float64)
# instantiate PyAudio (1)
p = pyaudio.PyAudio()
count = 0
IR_left = first_IR_left # Replace for actual IR
IR_right = first_IR_right # Replace for actual IR
# define callback (2)
def callback(in_data, frame_count, time_info, status):
global count
track1_frame = track1_data[frame_count*count : frame_count*(count+1)]
track2_frame = track2_data[frame_count*count : frame_count*(count+1)]
track3_frame = track3_data[frame_count*count : frame_count*(count+1)]
track1_left = ss.fftconvolve(track1_frame, IR_left)
track1_right = ss.fftconvolve(track1_frame, IR_right)
track2_left = ss.fftconvolve(track2_frame, IR_left)
track2_right = ss.fftconvolve(track2_frame, IR_right)
track3_left = ss.fftconvolve(track3_frame, IR_left)
track3_right = ss.fftconvolve(track3_frame, IR_right)
track_left = 1/3 * track1_left + 1/3 * track2_left + 1/3 * track3_left
track_right = 1/3 * track1_right + 1/3 * track2_right + 1/3 * track3_right
ret_data = np.empty((track_left.size + track_right.size), dtype=track1_left.dtype)
ret_data[1::2] = br_left
ret_data[0::2] = br_right
ret_data = ret_data.astype(np.float32).tostring()
count += 1
return (ret_data, pyaudio.paContinue)
# open stream using callback (3)
stream = p.open(format=pyaudio.paFloat32,
channels=2,
rate=int(track1_rate),
output=True,
stream_callback=callback,
frames_per_buffer=2**16)
# start the stream (4)
stream.start_stream()
# wait for stream to finish (5)
while_count = 0
while stream.is_active():
while_count += 1
if while_count % 3 == 0:
IR_left = first_IR_left # Replace for actual IR
IR_right = first_IR_right # Replace for actual IR
elif while_count % 3 == 1:
IR_left = second_IR_left # Replace for actual IR
IR_right = second_IR_right # Replace for actual IR
elif while_count % 3 == 2:
IR_left = third_IR_left # Replace for actual IR
IR_right = third_IR_right # Replace for actual IR
time.sleep(10)
# stop stream (6)
stream.stop_stream()
stream.close()
# close PyAudio (7)
p.terminate()
Here are some important reflections about the code above:
Working with librosa instead of wave allows me to use numpy arrays for processing which is much better than the chunks of data from wave.readframes.
The data type you set in p.open(format= must match the format of the ret_data bytes. And PyAudio works with float32 at most.
Even index bytes in ret_data go to the right headphone, and odd index bytes go to the left one.
Just to clarify, this code sends the mix of three tracks to the output audio in stereo, and every 10 seconds it changes the impulse response and thus the filter being applied.
I used this for testing a 3d audio app I'm developing, and so the impulse responses where Head Related Impulse Responses (HRIRs), that changed the position of the sound every 10 seconds.
EDIT:
This code had a problem: the output had a noise of a frequency corresponding to the size of the frames (higher frequency when size of frames was smaller). I fixed that by manually doing an overlap and add of the frames. Basically, the ss.oaconvolve returned an array of size track_frame.size + IR.size - 1, so I separated that array into the first track_frame.size elements (which was then used for ret_data), and then the last IR.size - 1 elements I saved for later. Those saved elements would then be added to the first IR.size - 1 elements of the next frame. The first frame adds zeros.