I read a 24-bit mono audio .wav file into an array of type <i4 (<i3 doesn't exist)
data = numpy.fromfile(fid, dtype=`<i4`, count=size//3)
What should I do in order to get the audio samples properly ? Should I swap bytes order of something like this, how ?
Here is the solution for reading 24 bits files (thanks to Warren Weckesser's gist https://gist.github.com/WarrenWeckesser/7461781) :
data = numpy.fromfile(fid, dtype='u1', count=size) # first read byte per byte
a = numpy.empty((len(data)/3, 4), dtype=`u1`)
a[:, :3] = data.reshape((-1, 3))
a[:, 3:] = (a[:, 3 - 1:3] >> 7) * 255
data = a.view('<i4').reshape(a.shape[:-1])
This can be directly inserted in def _read_data_chunk(fid, noc, bits): (scipy\io\wavfile.py).
You can convert the data into a numpy array of uint8, then add the 0 to each sample by using reshape and hstack;
In [1]: import numpy as np
I'm using a generated sequence here as an example.
In [2]: a = np.array([1,2,3]*10, dtype=np.uint8)
In [3]: a
Out[3]:
array([1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2,
3, 1, 2, 3, 1, 2, 3], dtype=uint8)
In [4]: a = a.reshape((-1,3))
Reshape allows you to group the samples:
In [5]: a
Out[5]:
array([[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3],
[1, 2, 3]], dtype=uint8)
Make the zeros that have to be added.
In [6]: b = np.zeros(10, dtype=np.uint8).reshape((-1,1))
In [7]: b
Out[7]:
array([[0],
[0],
[0],
[0],
[0],
[0],
[0],
[0],
[0],
[0]], dtype=uint8)
Now we add the zeroes. Assuming you're using a little-endian system, the added zero goes at the front, to scale the data.
(I hope I got this endianness stuff right. If the sample now sounds very faint, I got it wrong and you need to use (a,b) instead of (b,a))
In [8]: c = np.hstack((b, a))
In [9]: c
Out[9]:
array([[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3]], dtype=uint8)
Reshape it back.
In [10]: c.reshape((1,-1))
Out[10]:
array([[0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1,
2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]], dtype=uint8)
Convert to bytes:
In [11]: bytearray(c.reshape((1,-1)))
bytearray(b'\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03\x00\x01\x02\x03')
Now you have 4-byte samples.
Related
I have an array like this
Nbank = np.array([[2, 3, 1],
[1, 2, 2],
[3, 2, 1],
[3, 2, 1],
[2, 3, 2],
[2, 2, 3],
[1, 1, 3],
[2, 1, 1],
[2, 2, 3],
[1, 1, 1],
[2, 1, 1],
[2, 3, 1],
[1, 2, 1]])
I want to return an array with only one column. The condition is to return the most common value in each row; if multiple values have the same number of occurrences, just return the maximum of them.
I used this code
most_f = np.array([np.bincount(row).argmax() for row in Nbank])
if multiple values have the same number of occurrences, it returns the first item instead of the maximum. how can I work this around?
You could use a Counter after sorting in descending order by row. There's a most_common that will return what you want. Since it's sorted already, the first element is always either the largest or the most frequent.
import numpy as np
from collections import Counter
Nbank = np.array([[2, 3, 1],
[1, 2, 2],
[3, 2, 1],
[3, 2, 1],
[2, 3, 2],
[2, 2, 3],
[1, 1, 3],
[2, 1, 1],
[2, 2, 3],
[1, 1, 1],
[2, 1, 1],
[2, 3, 1],
[1, 2, 1]])
np.array([Counter(sorted(row, reverse=True)).most_common(1)[0][0] for row in Nbank])
Output
array([3, 2, 3, 3, 2, 2, 1, 1, 2, 1, 1, 3, 1])
I believe this will solve the problem. You could probable make it into a one liner with some fancy list comprehension, but I don't think that would be worth while.
most_f = []
for n in Nbank: #iterate over elements
counts = np.bincount(n) #count the number of elements of each value
most_f.append(np.argwhere(counts == np.max(counts))[-1][0]) #append the last and highest
You can cheat a little bit and reverse each row in order to make np.argmax return indice of the rightmost occurence which corresponds to the largest item:
N = np.max(arr)
>>> [N - np.argmax(np.bincount(row, minlength=N+1)[::-1]) for row in Nbank]
[3, 2, 3, 3, 2, 2, 1, 1, 2, 1, 1, 3, 1]
You might also like to avoid loops which is definitely adviseable if you want to take full advantages of numpy. Unfortunately np.bincount is not supported for 2D arrays but you can do it manually:
N, M = arr.shape[0], np.max(arr)+1
bincount_2D = np.zeros(shape=(N, M), dtype=int)
advanced_indexing = np.repeat(np.arange(N), arr.shape[1]), arr.ravel()
np.add.at(bincount_2D, advanced_indexing, 1)
>>> bincount_2D
array([[0, 1, 1, 1],
[0, 1, 2, 0],
[0, 1, 1, 1],
[0, 1, 1, 1],
[0, 0, 2, 1],
[0, 0, 2, 1],
[0, 2, 0, 1],
[0, 2, 1, 0],
[0, 0, 2, 1],
[0, 3, 0, 0],
[0, 2, 1, 0],
[0, 1, 1, 1],
[0, 2, 1, 0]])
And then repeat the process for all the rows simultaneously:
>>> M -1 - np.argmax(bincount_2D[:,::-1], axis=1)
array([3, 2, 3, 3, 2, 2, 1, 1, 2, 1, 1, 3, 1], dtype=int64)
I have a quick question about the numpy unique function. I want to return the unique column values for each row
import numpy as np
a = np.array([[3, 2, 3, 2, 1, 3, 1, 2, 1, 3, 1, 2, 2, 2, 3, 3],
[3, 2, 3, 2, 3, 3, 3, 3, 2, 2, 3, 1, 2, 1, 2, 1],
[3, 3, 3, 2, 3, 3, 3, 2, 2, 2, 3, 2, 2, 3, 1, 1]]) # a.shape is (3,16)
np.unique(a)
array([1, 2, 3]) # not what I want
np.unique(a,axis=1)
array([[1, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3],
[2, 3, 1, 1, 2, 2, 3, 1, 2, 2, 3],
[2, 3, 2, 3, 2, 3, 2, 1, 1, 2, 3]]) # also not what I want, and I'm not even sure what its doing
np.apply_along_axis(np.unique,1,a)
array([[1, 2, 3],
[1, 2, 3],
[1, 2, 3]]) # this is what I want
The problem is that I also want to use other features of np.unqiue, like returning index values. Can anyone help me to get np.unique to work by itself?
You can loop over rows and collect unique values:
import numpy as np
a = np.array([[3, 2, 3, 2, 1, 3, 1, 2, 1, 3, 1, 2, 2, 2, 3, 3],
[3, 2, 3, 2, 3, 3, 3, 3, 2, 2, 3, 1, 2, 1, 2, 1],
[3, 3, 3, 2, 3, 3, 3, 2, 2, 2, 3, 2, 2, 3, 1, 1]])
arr = np.empty((0,3), int)
for row in a:
arr = np.append(arr, np.array([np.unique(a)]), axis=0)
Output:
[[1 2 3]
[1 2 3]
[1 2 3]]
numpy will not be able to return a matrix with rows of different sizes. your example has exactly 3 distinct values per row which makes np.apply_along_axis work but if you had a value of 4 in one of the rows or only 1s and 2s on a row it would fail.
To obtain what you are looking for you will need to use a normal Python list as the result. You can build it using a list comprehension:
import numpy as np
a = np.array([[1, 2, 2, 2, 1, 1, 1, 2, 1, 2, 1, 2, 2, 2, 1, 1],
[3, 2, 3, 2, 3, 3, 3, 3, 2, 2, 3, 1, 2, 1, 2, 1],
[3, 3, 3, 2, 3, 3, 4, 2, 2, 2, 3, 2, 2, 3, 1, 1]])
r = [ np.unique(row) for row in a ]
print(r)
# [array([1, 2]), array([1, 2, 3]), array([1, 2, 3, 4])]
r = [ np.unique(row,return_index=True)for row in a ]
print(r)
# [(array([1, 2]), array([0, 1])),
# (array([1, 2, 3]), array([11, 1, 0])),
# (array([1, 2, 3, 4]), array([14, 3, 0, 6]))]
One thing you could do is build a mask of the values that are the first of their kind on each row. This can be done using numpy.
Here's one way to do it (hopefully, numpy experts could suggest something less convoluted):
np.sum(np.cumsum(np.cumsum(a==np.unique(a)[:,None,None],axis=2),axis=2)==1,axis=0)
array([[1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0],
[1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0]])
Such a mask offers many processing options such as finding indices of the first occurrence on each line (using np.argwhere), erasing/assigning first or subsequent occurrences, and more.
I have the given dataset:
data = np.array([
[1, 2, 1, 3, 1, 2, 1],
[3, 4, 1, 5, 2, 7, 2],
[2, 1, 2, 1, 1, 4, 5],
[6, 1, 2 ,3, 1, 3, 1]])
cols_idx = np.array([0, 0, 1, 0, 1, 0, 0])
I want to return columns from data where cols_idx == 1. For that I used:
data[:, np.nonzero(cols_idx)]
But it returns a 3D instead a 2D array:
data[:, np.nonzero(cols_idx)]
array([[[1, 1]],
[[1, 2]],
[[2, 1]],
[[2, 1]]])
data[:, np.nonzero(cols_idx)].shape
(4, 1, 2)
I would like the output to be:
data[:, np.nonzero(cols_idx)]
array([[1, 1],
[1, 2],
[2, 1],
[2, 1]])
data[:, np.nonzero(cols_idx)].shape
(4, 2)
How can I achieve that?
print(np.nonzero(cols_idx)) gives (array([2, 4]),) (a tuple rather than just an array)
So you should use np.nonzero(cols_idx)[0] # gives [2 4] to get what you want:
Full code:
import numpy as np
data = np.array([
[1, 2, 1, 3, 1, 2, 1],
[3, 4, 1, 5, 2, 7, 2],
[2, 1, 2, 1, 1, 4, 5],
[6, 1, 2 ,3, 1, 3, 1]])
cols_idx = np.array([0, 0, 1, 0, 1, 0, 0])
new_data = data[:, np.nonzero(cols_idx)[0]]
print(new_data)
'''[[1 1]
[1 2]
[2 1]
[2 1]]'''
print(new_data.shape) # (4,2)
From numpy documentation:
While the nonzero values can be obtained with a[nonzero(a)], it is recommended to use x[x.astype(bool)] or x[x != 0] instead, which will correctly handle 0-d arrays.
So it's better to use:
data[:, cols_idx.astype(bool)]
or
data[:, cols_idx != 0]
I have an
n = np.array([[1, 12, 1, 3],
[1, 1, 12, 0]])
and would like to duplicate it such that if I have a double-digit number in the array, it breaks the array into two identical arrays where the first array has the first digit and the second array has the second digit. In the above example, I would have 4 copies of the matrix. The assumptions are that there are either single digit or double digit numbers in the array.
n1 = [1, 1, 1, 3], [1, 1, 1, 0]
n2 = [1, 1, 1, 3], [1, 1, 2, 0]
n3 = [1, 2, 1, 3], [1, 1, 1, 0]
n4 = [1, 2, 1, 3], [1, 1, 2, 0]
Approach 1: itertools.product
>>> import numpy as np
>>> from itertools import product
>>> from pprint import pprint
>>>
>>> n = np.array([[1, 12, 1, 3],
... [1, 1, 12, 0]])
>>>
>>> pprint([np.reshape(nn, n.shape).astype(int) for nn in product(*map(str, n.ravel()))])
[array([[1, 1, 1, 3],
[1, 1, 1, 0]]),
array([[1, 1, 1, 3],
[1, 1, 2, 0]]),
array([[1, 2, 1, 3],
[1, 1, 1, 0]]),
array([[1, 2, 1, 3],
[1, 1, 2, 0]])]
Note that this happens to work also for longer numbers.
>>> n = np.array([462, 3, 15, 1, 0])
>>> pprint([np.reshape(nn, n.shape).astype(int) for nn in product(*map(str, n.ravel()))])
[array([4, 3, 1, 1, 0]),
array([4, 3, 5, 1, 0]),
array([6, 3, 1, 1, 0]),
array([6, 3, 5, 1, 0]),
array([2, 3, 1, 1, 0]),
array([2, 3, 5, 1, 0])]
Approach 2: np.meshgrid
>>> import numpy as np
>>>
>>> n = np.array([[1, 12, 1, 3],
... [1, 1, 12, 0]])
>>>
>>> te = np.where(n>=10)
>>> dims = tuple(np.log10(n[te]).astype(int) + 1)
>>>
>>> out = np.empty(dims + n.shape, dtype=n.dtype)
>>> out[...] = n
>>> out[(Ellipsis,) + te] = np.moveaxis(np.meshgrid(*(s//10**np.arange(i)[::-1]%10 for i, s in zip(dims, n[te])), indexing='ij'), 0, -1)
>>>
>>> out
array([[[[1, 1, 1, 3],
[1, 1, 1, 0]],
[[1, 1, 1, 3],
[1, 1, 2, 0]]],
[[[1, 2, 1, 3],
[1, 1, 1, 0]],
[[1, 2, 1, 3],
[1, 1, 2, 0]]]])
Is there a way to loop-through this tuple(?) where the left array are positions in an array and the right array is the value I would like to insert into the given positions:
(array([ 0, 4, 6, ..., 9992, 9996, 9997]), array([3, 3, 3, ..., 3, 3, 3]))
The output above is generated from the following piece of code:
np.where(h2 == h2[i,:].max())[1]
I would like the result to be like this:
array[0] = 3
array[4] = 3
...
array[9997] = 3
Just use a simple indexing:
indices, values = my_tuple
array[indices] = values
If you don't have the final array yet you can create it using a desire function like np.zeros, np.ones, etc. with a size as the size of maximum index.
I think you want the transpose of the where tuple:
In [204]: x=np.arange(1,13).reshape(3,4)
In [205]: x
Out[205]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12]])
In [206]: idx=np.where(x)
In [207]: idx
Out[207]:
(array([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2], dtype=int32),
array([0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3], dtype=int32))
In [208]: ij=np.transpose(idx)
In [209]: ij
Out[209]:
array([[0, 0],
[0, 1],
[0, 2],
[0, 3],
[1, 0],
[1, 1],
[1, 2],
[1, 3],
[2, 0],
[2, 1],
[2, 2],
[2, 3]], dtype=int32)
In fact there's a function that does just that:
np.argwhere(x)
Iterating on ij, I can print:
In [213]: for i,j in ij:
...: print('array[{}]={}'.format(i,j))
...:
array[0]=0
array[0]=1
array[0]=2
zip(*) is a list version of transpose:
for i,j in zip(*idx):
print(i,j)