Given a list of numpy 2d-arrays of size mxn, what is the best way to get an array of size n (number of columns of each matrix in the list) where the i-the value of the array is the maximum of column i, across all matrices in the list?
>>> import numpy as np
>>> a = np.array([[1,11,5,2], [3,9,1,12], [5,7,7,1]])
>>> a
array([[ 1, 11, 5, 2],
[ 3, 9, 1, 12],
[ 5, 7, 7, 1]])
Max by column
>>> a.max(axis=0)
array([ 5, 11, 7, 12])
Max by row
>>> a.max(axis=1)
array([11, 12, 7])
If you have a list of 2D numpy arrays:
>>> a = np.array([[1,11,5,2], [3,9,1,12], [5,7,7,1]])
>>> b = np.array([[2,4,6,8],[1,3,2,1],[5,6,7,8]])
>>> l = [a,b]
You can use a list comprehension
>>> [i.max(axis=0) for i in l]
[array([ 5, 11, 7, 12]),
array([5, 6, 7, 8])]
>>> [i.max(axis=1) for i in l]
[array([11, 12, 7]),
array([8, 3, 8])]
You can first stack the arrays vertically and then take the maximum of each column:
np.vstack(list_of_arrays).max(axis=0)
Related
i would like to get an multidimentional array in arr1.shape = (x,y)
which would be filled with values like from np.arange(z), where z is number of spaces in arr1.
it is known that, that i could make
arr2 = np.random.randn(x,y)
but then the values would be random...
Is there any straight way, which allows me not to iterate through the array?
You could use numpy.reshape to take the result of numpy.arange and reshape into (x,y) dimensions
>>> import numpy as np
>>> x = 5
>>> y = 3
>>> np.reshape(np.arange(x*y), (x,y))
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14]])
Lets say I have a Python Numpy array a.
a = numpy.array([1,2,3,4,5,6,7,8,9,10,11])
I want to create a matrix of sub sequences from this array of length 5 with stride 3. The results matrix hence will look as follows:
numpy.array([[1,2,3,4,5],[4,5,6,7,8],[7,8,9,10,11]])
One possible way of implementing this would be using a for-loop.
result_matrix = np.zeros((3, 5))
for i in range(0, len(a), 3):
result_matrix[i] = a[i:i+5]
Is there a cleaner way to implement this in Numpy?
Approach #1 : Using broadcasting -
def broadcasting_app(a, L, S ): # Window len = L, Stride len/stepsize = S
nrows = ((a.size-L)//S)+1
return a[S*np.arange(nrows)[:,None] + np.arange(L)]
Approach #2 : Using more efficient NumPy strides -
def strided_app(a, L, S ): # Window len = L, Stride len/stepsize = S
nrows = ((a.size-L)//S)+1
n = a.strides[0]
return np.lib.stride_tricks.as_strided(a, shape=(nrows,L), strides=(S*n,n))
Sample run -
In [143]: a
Out[143]: array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
In [144]: broadcasting_app(a, L = 5, S = 3)
Out[144]:
array([[ 1, 2, 3, 4, 5],
[ 4, 5, 6, 7, 8],
[ 7, 8, 9, 10, 11]])
In [145]: strided_app(a, L = 5, S = 3)
Out[145]:
array([[ 1, 2, 3, 4, 5],
[ 4, 5, 6, 7, 8],
[ 7, 8, 9, 10, 11]])
Starting in Numpy 1.20, we can make use of the new sliding_window_view to slide/roll over windows of elements.
And coupled with a stepping [::3], it simply becomes:
from numpy.lib.stride_tricks import sliding_window_view
# values = np.array([1,2,3,4,5,6,7,8,9,10,11])
sliding_window_view(values, window_shape = 5)[::3]
# array([[ 1, 2, 3, 4, 5],
# [ 4, 5, 6, 7, 8],
# [ 7, 8, 9, 10, 11]])
where the intermediate result of the sliding is:
sliding_window_view(values, window_shape = 5)
# array([[ 1, 2, 3, 4, 5],
# [ 2, 3, 4, 5, 6],
# [ 3, 4, 5, 6, 7],
# [ 4, 5, 6, 7, 8],
# [ 5, 6, 7, 8, 9],
# [ 6, 7, 8, 9, 10],
# [ 7, 8, 9, 10, 11]])
Modified version of #Divakar's code with checking to ensure that memory is contiguous and that the returned array cannot be modified. (Variable names changed for my DSP application).
def frame(a, framelen, frameadv):
"""frame - Frame a 1D array
a - 1D array
framelen - Samples per frame
frameadv - Samples between starts of consecutive frames
Set to framelen for non-overlaping consecutive frames
Modified from Divakar's 10/17/16 11:20 solution:
https://stackoverflow.com/questions/40084931/taking-subarrays-from-numpy-array-with-given-stride-stepsize
CAVEATS:
Assumes array is contiguous
Output is not writable as there are multiple views on the same memory
"""
if not isinstance(a, np.ndarray) or \
not (a.flags['C_CONTIGUOUS'] or a.flags['F_CONTIGUOUS']):
raise ValueError("Input array a must be a contiguous numpy array")
# Output
nrows = ((a.size-framelen)//frameadv)+1
oshape = (nrows, framelen)
# Size of each element in a
n = a.strides[0]
# Indexing in the new object will advance by frameadv * element size
ostrides = (frameadv*n, n)
return np.lib.stride_tricks.as_strided(a, shape=oshape,
strides=ostrides, writeable=False)
Consider this simple problem: in a list of integers I need multiply all even number by 10.
I can certainly do element-wise operation such as:
[if x%2==0: x=x*10 for x in arr]
But what if I the operation has to be operated on the array level? The trouble I am having is after the operation on the filtered array, how do I nicely put them back to the original array?
Code example:
arr=np.arange(1,10) # the original array array([1, 2, 3, 4, 5, 6, 7, 8, 9])
filter1 = arr%2==0 # the filter
arr1=arr[filter1] # the filtered array array([2, 4, 6, 8])
arr1=arr1*10 # the 'array'-wise operation array([20, 40, 60, 80])
# this is the part I am trying to improve
i=0
j=0
arr2=[]
for f in filter1:
if f:
arr2.append(arr1[i])
i=i+1
else:
arr2.append(arr[j])
j=j+1
# output arr2: [1, 20, 3, 40, 5, 60, 7, 80, 9]
>>> a = np.arange(10)
>>> a
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>> a[a%2 == 0] *= 10
>>> a
array([ 0, 1, 20, 3, 40, 5, 60, 7, 80, 9])
This is how you can do element-wise operation on array based on a condition:
arr[arr%2 == 0] *= 10
Note this does not create new array but modifies the array.
If you want a new array as well, you can copy the array after the operation:
arr2 = arr.copy()
This question already has answers here:
Selecting Random Windows from Multidimensional Numpy Array Rows
(2 answers)
Closed 3 years ago.
I have an m x n numpy array arr, and for each column of arr, I have a given range of rows that I want to access.
I have an n x 1 array vec that describes when this range starts.
The range has some constant duration d.
How can I extract this d x n array of interest efficiently?
Can this be done by clever slicing?
My initial thought was to try something like:
arr = np.tile(np.arange(10),(4,1)).T
vec = np.array([3,4,5,4])
d = 3
vec_2 = vec+d
out = arr[vec:vec2,np.arange(n)]
But this gives the following error:
TypeError: only integer scalar arrays can be converted to a scalar index
The desired output would be the following array:
array([[3, 4, 5, 4],
[4, 5, 6, 5],
[5, 6, 7, 6],
[6, 7, 8, 7])
I could loop over d, but performance is important for this piece of code so I would prefer to vectorize it.
In [489]: arr=np.arange(24).reshape(6,4)
In [490]: vec=np.array([0,2,1,3])
Taking advantage of the recent expansion of linspace to generate several arrays:
In [493]: x = np.linspace(vec,vec+2,3).astype(int)
In [494]: x
Out[494]:
array([[0, 2, 1, 3],
[1, 3, 2, 4],
[2, 4, 3, 5]])
In [495]: arr[x, np.arange(4)]
Out[495]:
array([[ 0, 9, 6, 15],
[ 4, 13, 10, 19],
[ 8, 17, 14, 23]])
the column iteration approach:
In [498]: np.stack([arr[i:j,k] for k,(i,j) in enumerate(zip(vec,vec+3))],1)
Out[498]:
array([[ 0, 9, 6, 15],
[ 4, 13, 10, 19],
[ 8, 17, 14, 23]])
I have a very huge numpy array like this:
np.array([1, 2, 3, 4, 5, 6, 7 , ... , 12345])
I need to create subgroups of n elements (in the example n = 3) in another array like this:
np.array([[1, 2, 3],[4, 5, 6], [6, 7, 8], [...], [12340, 12341, 12342], [12343, 12344, 12345]])
I did accomplish that using normal python lists, just appending the subgroups to another list. But, I'm having a hard time trying to do that in numpy.
Any ideas how can I do that?
Thanks!
You can use np.reshape(-1, 3), where the -1 means "whatever's left".
>>> array = np.arange(1, 12346)
>>> array
array([ 1, 2, 3, ..., 12343, 12344, 12345])
>>> array.reshape(-1, 3)
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
...,
[12337, 12338, 12339],
[12340, 12341, 12342],
[12343, 12344, 12345]])
You can use np.reshape():
From the documentation (link in title):
numpy.reshape(a, newshape, order='C')
Gives a new shape to an array without changing its data.
Here is an example of how you can apply it to your situation:
>>> import numpy as np
>>> a = np.array([1, 2, 3, 4, 5, 6, 7, 8, 12345])
>>> a.reshape((int(len(a)/3), 3))
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 12345]], dtype=object)
Note that obviously, the length of the array (len(a)) has to be a multiple of 3 to be able to reshape it into a 2-dimensional numpy array, because they must be rectangular.