I have two arrays that are identical (by design because I obtained the second one by doing an FFT and then inverse FFT of the first one). However, when I write the first one to a .wav file, I get sound-producing file as opposed to not when I do the same with the second one. I get no sound. Here is my code:
fs, data = wavfile.read(filename)
a = data.T[0]
c = fft(a)
y2 = fftp.ifft(c)
y2 = np.array([int(round(i)) for i in y2.real])
Now when I try:
sum(y2==a)==len(a)
I get True, which means the two arrays are identical. The only difference is that one has "dtype=int16":
In [322]: a
Out[322]: array([ 1, 1, 1, ..., 21, 20, 21], dtype=int16)
In [321]: y2
Out[321]: array([ 1, 1, 1, ..., 21, 20, 21])
How do I convert the second array to a format where it produces a valid .wav file as well?
That "only difference" is a huge difference.
The WAV format, by default, stores samples as signed little-endian 16-bit integers. So, when you write an array of int16 values as raw data, you get a playable WAV file (on a little-endian system, at least).
But when you write an array of int32 values, you get nonsense—each number turns into 2 samples, one of which is the high word of your data, the next of which is the low word. So, you've got your original audio samples at half speed, and interleaves with what's effectively random noise.
Or, alternatively, you can use a non-default WAV format. You didn't show enough of your code to show how you're handling this, but you can write WAV files in a variety of different formats, from 8-bit unsigned int to 32-bit float, and 32-bit signed ints are a valid format. WAV files can even handle compression (including MP3).
But less-common formats may not actually play with every tool; a lot of programs assume a WAV is 16-bit integers, and won't know what to do with anything else.
So, you're probably better off writing 16-bit integers.
Or, maybe you're already doing that—writing 32-bit int values with the right header—and maybe your player is handling them properly.
But you're writing 32-bit int values between -32768 and 32767. Which means you're only using 1/65536th of the dynamic range, so everything will be incredibly quiet. If you want to write 32-bit int values, you want to normalize them to the 32-bit int range, not the 16-bit int range.
The simplest solution to all of these problems is: convert the values back to int16 before writing them:
y3 = y2.astype(np.int16)
Related
I have an array of numbers
a = [440, 320, 650]
I am trying to write a .wav file that writes those frequencies in order. But I don't know if scipy.io.wavfile is able to write an array of frequencies to a wav file. All I can do right now is something like
wavfile.write('Sine.wav', rate, a[0])
I am thinking to do something like this though.
for x in range(0, len(a)):
#wavfile.addFrequency('Sine.wav', rate, a[x])
"In order" doesn't sound very clear to me. I guess you would like to mix those sine waves. For that you must make a single wave with the 3 sines by summation of their amplitude values. For each sample, three values must be added (one for each sine) taking care that the result never overflows -1.0 or 1.0 (for float values, for 16 bit integers the range would be -32768 to 32767 instead).
Now if you plane to render the three waves successively instead, you have to determine the duration of each segment and take care that the junction between two waves is done at amplitude zero to prevent numeric clicks.
I am trying to convert a c_byte array into different datatypes in Python, e.g. converting a eight-entry c_byte array into a int64 or double.
In my project, I read a long c_byte array (n>500) containing multiple sensor values with different datatypes. So maybe the first entry is a bool, the second and third entry represent a int8 and entries 4-11 store a double. I am looking for a convenient way of casting those array-entries into the required datatypes.
At the moment, I am transcribing the byte-array into strings containing the binary number. I was thinking about manually writing functions to convert those strings into floats and ints, but I hope there is a more elegant way of doing so. Also, i run into problems converting signed ints...
def convert_byte_to_binary(array):
binary = ''
for i in array:
binary += format(i, '#010b')[2:]
return binary
def convert_binary_to_uint(binary):
return int(binary, 2)
array = read_cbyte_array(address, length) # reads an array of size length, starting at address
array
[15, 30, 110, 7, 65]
convert_byte_to_binary(array)
'0000111100011110011011100000011101000001'
I found the bitstring library, which does something very similar to what I want. Unfortunately, I did not find any support for 64bit integers or double floats.
Ideally, I would have a set of function that can convert the ctypes.c_byte-array into the corresponding ctypes-types.
The struct library is intended for this.
Here's a short example. The value '<?2b8d' represents:
< little-endian
? bool
2b two 1-byte signed int.
8d eight doubles.
import ctypes
import struct
# Build the raw data
org_data = struct.pack('<?2b8d',True,-1,2,1.1,2.2,3.3,4.4,5.5,6.6,7.7,8.8)
print(org_data)
# Just to demo using a c_byte array...
byte_data = (ctypes.c_byte * len(org_data))(*org_data)
print(byte_data)
# convert to the original data
data = struct.unpack('<?2b8d',byte_data)
print(data)
Output:
b'\x01\xff\x02\x9a\x99\x99\x99\x99\x99\xf1?\x9a\x99\x99\x99\x99\x99\x01#ffffff\n#\x9a\x99\x99\x99\x99\x99\x11#\x00\x00\x00\x00\x00\x00\x16#ffffff\x1a#\xcd\xcc\xcc\xcc\xcc\xcc\x1e#\x9a\x99\x99\x99\x99\x99!#'
<__main__.c_byte_Array_67 object at 0x0000025D4066B0C8>
(True, -1, 2, 1.1, 2.2, 3.3, 4.4, 5.5, 6.6, 7.7, 8.8)
Why is python inserting strange negative numbers into my array?
I am reading some numerical data from a plain text file like below:
fp=np.genfromtxt("mytextfile.txt", dtype=('int32'), delimiter='\r\n')
The information contained in the file are all positive numbers, file is formatted like below and there are 300000 of these numbers:
12345
45678
1056789
232323
6789010001
1023242556
When I print out the fp read in array, the first half of the array is correctly read in but the last half is strange negative numbers that aren't in my file at all.
How can I get it to read correctly what is in the file?
You told it the numbers are int32s, but at least some of your numbers, e.g. 6789010001 are larger than a signed 32 bit quantity can represent (6789010001 is larger than an unsigned 32 bit quantity can represent).
If all the numbers are positive, I'd suggested using uint64 as your data type (you should check that all numbers in the file are in fact less than 2**64 though).
You have data larger then int32 can save, this causes overflow and makes it negative.
Numpy's Integers doesn't act like python's integer it's like a C integer
Try change dtype from int32 to int64 or object may help
I need to store a massive numpy vector to disk. Right now the vector that I am trying to store is ~2.4 billion elements long and the data is float64. This takes about 18GB of space when serialized out to disk.
If I use struct.pack() and use float32 (4 bytes) I can reduce it to ~9GB. I don't need anywhere near this amount of precision disk space is going to quickly becomes an issue as I expect the number of values I need to store could grow by an order of magnitude or two.
I was thinking that if I could access the first 4 significant digits I could store those values in an int and only use 1 or 2 bytes of space. However, I have no idea how to do this efficiently. Does anyone have any idea or suggestions?
If your data is between 0 and 1, and 16bit is enough you can save the data as uint16:
data16 = (65535 * data).round().astype(uint16)
and expand the data with
data = data16 / 65535.0
Generally speaking, I'd recommend against using float16, but for what it's worth, it's quite easy to do.
However, the struct module can't convert to/from 16-bit floats.
Therefore, you'll need to do something similar to:
import numpy as np
x = np.linspace(0, 1, 1000)
x = x.astype(np.float16)
with open('outfile.dat', 'w') as outfile:
x.tofile(outfile)
Note that "outfile.dat" is exactly 2000 bytes - two bytes per item. tofile just writes the raw, "packed" binary data to disk. There's no header, etc, and no difference in the output between using it and the struct module.
Use struct.pack() with the f type code to get them into 4-byte packets.
I am trying to port a portion of a code written in a different language (an obscure one called Igor Pro by Wavemetrics for those of you have heard of it) to Python.
In this code, there is a conversion of a data type from a 16-bit integer (read in from a 16-bit, big endian binary file) to single-precision (32-bit) floating-point. In this program, the conversion is as follows:
Signed 16-bit integer:
print tmp
tmp[0]={-24160,18597,-24160,18597,-24160}
converted to 32-bit floating-point:
Redimension/S/E=1 tmp
print tmp
tmp[0]={339213,339213,5.79801e-41,0,0}
The /S flag/option indicates that the data type of tmp should be float32 instead of int16. However, I believe the important flag/option is /E=1, which is said to "Force reshape without converting or moving data."
In Python, the conversion is as follows:
>>> tmp[:5]
array([-24160, 18597, -24160, 18597, -24160], dtype=int16)
>>> tmp.astype('float32')
array([-24160., 18597., -24160., ..., 18597., -24160., 18597.], dtype=float32)
Which is what I expect, but I need to find a function/operation that emulates the /E=1 option in the original code above. Is there an obvious way in which -24160 and 18597 would both be converted to 339213? Does this have anything to do with byteswap or newbyteorder or something else?
import numpy
tmp=numpy.array([-24160,18597,-24160,18597,-24160, 0], numpy.int16)
tmp.dtype = numpy.float32
print tmp
Result:
[ 3.39213000e+05 3.39213000e+05 5.79801253e-41]
I had to add a zero to the list of value because there are an odd number of values. It cannot interpret those as 32 bit floats since there 5 16 bit values.
Use view instead of astype:
In [9]: tmp=np.array([-24160, 18597, -24160, 18597, -24160, 18597], dtype=int16)
In [10]: tmp.view('float32')
Out[10]: array([ 339213., 339213., 339213.], dtype=float32)
.astype creates a copy of the array cast to the new dtype
.view returns a view of the array (with the same underlying data),
with the data interpreted according to the new dtype.
Is there an obvious way in which -24160 and 18597 would both be converted to 339213?
No, but neither is there any obvious way in which -24160 would convert to 339213 and 5.79801e-41 and 0.
It looks more like the conversion takes two input numbers to create one output (probably by concatenating the raw 2×16 bits to 32 bits and calling the result a float). In that case the pair -24160,18597 consistently becomes 339213, and 5.79801e-41 probably results from -24160,0 where the 0 is invented because we run out of inputs. Since 5.79801e-41 looks like it might be a single-precision denormal, this implies that the two 16-bit blocks are probably concatenated in little-endian order.
It remains to see whether you need to byte-swap each of the 16-bit inputs, but you can check that for yourself.