I think this is straightforward but I can't quite get it. I have a large 3d array and I want to reduce the 3rd dim by some factor and then sum the values to get to that reduced size. An example that works to get what I want is:
import numpy as np
arr=np.ones((10,10,16))
processed_data=np.zeros((arr.shape[0], arr.shape[1]), dtype='object')
factor=2
for i in range(arr.shape[0]):
for j in range(arr.shape[1]):
processed_data[i][j]=arr[i][j].reshape(int(arr.shape[2]/factor),-1).sum(axis=1)
So we take the last dimension, reshape it to an extra dimension and then sum along that dimension. In the example above the data is a 10x10x16 array of all 1s so with a factor=2 we get a 10x10x8 array out with the data all being 2s. I hope this illustrates what I am trying to achieve. If the factor would change to 4 we would get a 10x10x4 array out.
This method is not ideal as it involves creating a separate processed_data 'object' array where I would rather leave it as a 3D array, just with a reduced third dimension. It also involves iterating over every element in the 2D array which I don't think is neccessary. And it's really slow.
Any help appreciated - I suspect it is a combination of reshaping and transposing but cannot get my head around it.
Thanks.
I think you can reshape on the whole data and sum:
arr.reshape(*arr.shape[:2], -1, 2).sum(axis=-1)
Related
I have a (square) 2 dimensional numpy array where I would like to compare (subtract) all of the values within each row to each other but not to other rows so the output should be a 3D array.
matrix = np.array([[10,1,32],[32,4,15],[6,3,1]])
Output should be a 3x3x3 array which looks like:
output = [[[0,-9,22],[0,-28,-17],[0,-3,-5]], [[9,0,31],[28,0,11],[3,0,-2]], [[-22,-31,0],[17,-11,0],[5,2,0]]]
I.e. for output[0], for each of the 3 rows of matrix, subtract that row's zeroth element from every other, for output[1] subtract each row's first element etc.
This seems to me like a reduced version of numpy's ufunc.outer functionality which should be possible with
tryouter = np.subtract(matrix, matrix)
and then taking some clever slice and/or transposition.
Indeed, if you do this, one finds that: output[i,j] = tryouter[i,j,i]
This looks like it should be solvable by using np.transpose to switch the 1 and 2 axes and then taking the arrays on the new 0,1 diagonal but I can't work out how to do this with numpy diagonal or any slicing method.
Is there a way to do this or is there a simpler approach to this whole problem built into numpy?
Thanks :)
You're close, you can do it with broadcasting:
out = matrix[None, :, :] - matrix.T[:, :, None]
Here .T is the same as np.transpose, and using None as an index introduces a new dummy dimension of size 1.
Suppose I have a numpy array A with shape (j,d,d) and I want to obtain an array with shape j, in which each entry corresponds to the determinant of each (d,d) array.
I tried using np.apply_along_axis(np.linalg.det(A), axis=0), but np.apply_along_axis only seems to work for 1D slices.
Is there an efficient way of doing that using only numpy?
np.linalg.det can already do this for an array of arbitrary shape as long as the last two dimensions are square. You can see the documentation here.
My goal is to to turn a row vector into a column vector and vice versa. The documentation for numpy.ndarray.transpose says:
For a 1-D array, this has no effect. (To change between column and row vectors, first cast the 1-D array into a matrix object.)
However, when I try this:
my_array = np.array([1,2,3])
my_array_T = np.transpose(np.matrix(myArray))
I do get the wanted result, albeit in matrix form (matrix([[66],[640],[44]])), but I also get this warning:
PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
my_array_T = np.transpose(np.matrix(my_array))
How can I properly transpose an ndarray then?
A 1D array is itself once transposed, contrary to Matlab where a 1D array doesn't exist and is at least 2D.
What you want is to reshape it:
my_array.reshape(-1, 1)
Or:
my_array.reshape(1, -1)
Depending on what kind of vector you want (column or row vector).
The -1 is a broadcast-like, using all possible elements, and the 1 creates the second required dimension.
If your array is my_array and you want to convert it to a column vector you can do:
my_array.reshape(-1, 1)
For a row vector you can use
my_array.reshape(1, -1)
Both of these can also be transposed and that would work as expected.
IIUC, use reshape
my_array.reshape(my_array.size, -1)
I want to average a slice of a numpy array (its an image).
Currently i'm iterating over each pixel as follows but its dreadfully slow. I know there is a better way but I cant work it out. Its probably the numpy fancy indexing but i'm stuck.
I've used openCV to read the image into a numpy array with the shape 640,480,3 and I want to change the each of the last bit i.e [123,121,234] to the average of that slice for each of the 640x480.
You don't have to give me the answer but a shove in the right direction would be helpful.
This is whats slow for me:
def bw_image_arr(self):
for x in self.images:
for y in x:
for z in y:
z = z.mean()
Use axis argument to do mean-reduction along last axis and then broadcast to the original shape with np.broadcast_to -
np.broadcast_to(images.mean(axis=-1,keepdims=True),images.shape)
That np.broadcast_to helps us on achieving memory efficiency by giving us original shaped view into the averaged array. If you need the final output with its own memory space, append with .copy() -
np.broadcast_to(images.mean(axis=-1,keepdims=True),images.shape).copy()
Alternatively, we can use np.repeat -
images.mean(axis=-1,keepdims=True).repeat(images.shape[-1],axis=-1)
The posted solutions work for ndarrays of generic dimensions. Hence, will work on one image or a set of images with the desired result of average along the last axis being broadcasted/replicated/repeated along the same.
Also, note that the final output would be of float dtype. So, we might want to convert or/and round to int for usual image-dtype of unsigned-int dtype output.
You need to average over the x and y axes. In your case the axes 1 and 2 (you can input it in numpy.mean as a tuple). Then if you have 50 images in the first dimension example you will get (50, 3) shaped array.
I'm still getting the hang of working with numpy and array-wise operations.
I'm looking for the way of getting the row-wise average of a list of 2D arrays.
E.g I have a 4x3x25 array and I'm looking to get a 3x25 array of the row-wise averages.
If everything’s in one 3D array already, you can just do:
A.mean(axis=0)
…which will operate along the first dimension.
If it’s actually just a list of 2D arrays, you’ll have to convert it to a 3D array first. I would do:
A = np.dstack(list_of_arrays) # Combine the 2D arrays along a new 3rd dimension
A.mean(axis=2) # Calculate the means along that new dimension