Related
I have a numpy array of shape (294, 62, 350). Along the third dimension (the 350), I need to combine every two columns into one longer one which would result in an array of shape (294, 124, 175). For example if I have this array:
a_3d_array = np.array([[[1, 2, 3, 6, 1, 2], [3, 4, 3, 6, 1, 4]],
[[5, 2, 2, 1, 4, 2], [2, 9, 4, 3, 2, 7]]])
The expected output would be:
expected_output = np.array([[[5, 2, 4], [2, 4, 2], [ 2, 1, 2], [9, 3, 7]],
[[1, 3, 1], [3, 3, 1], [2, 6, 2], [4, 6, 4]]])
Sorry as I'm new to python and I don't have a clue how to approach this and thus I don't have a "my own attempt" to include here.
a_3d_array = np.array([[[1, 2, 3, 6, 1, 2], [3, 4, 3, 6, 1, 4]],
[[5, 2, 2, 1, 4, 2], [2, 9, 4, 3, 2, 7]]])
output = np.hstack([a_3d_array[:, :, ::2], a_3d_array[:, :, 1::2]])
To combine every N-th column:
N = 3
output = np.hstack([an_array[:, :, idx::N] for idx in range(N)])
You can reshape and reverse the first dimension:
a_3d_array.reshape((2,4,3), order='F')[::-1]
If you don't know the shape:
x,y,z = a_3d_array.shape
a_3d_array.reshape((x,y*2,-1), order='F')[::-1]
output:
array([[[5, 2, 4],
[2, 4, 2],
[2, 1, 2],
[9, 3, 7]],
[[1, 3, 1],
[3, 3, 1],
[2, 6, 2],
[4, 6, 4]]])
I think I'm missing something obvious. I want to find a cartesian product of arr1 (a 1d numpy array), and the ROWS of arr2 (a 2d numpy array). So, if arr1 has 4 elements and arr2 has shape (5,2), the output should have shape (20,3). (see below)
import numpy as np
arr1 = np.array([1, 4, 7, 3])
arr2 = np.array([[0, 1],
[2, 3],
[4, 5],
[4, 0],
[9, 9]])
The desired output is:
arr3 = np.array([[1, 0, 1],
[1, 2, 3],
[1, 4, 5],
[1, 4, 0],
[1, 9, 9],
[4, 0, 1],
[4, 2, 3],
[4, 4, 5],
[4, 4, 0],
[4, 9, 9],
[7, 0, 1],
[7, 2, 3],
[7, 4, 5],
[7, 4, 0],
[7, 9, 9],
[3, 0, 1],
[3, 2, 3],
[3, 4, 5],
[3, 4, 0],
[3, 9, 9]])
I've been trying to use transpose and reshape with code like np.array(np.meshgrid(arr1,arr2)), but no success yet.
I'm hoping the solution can be generalized because I also need to deal with situations like this: Get all combinations of the ROWS of a 2d (10,2) array and the ROWS of a 2d array (20, 5) to get an output array (200,7).
Here is a vectorized solution that works for your general case as well:
arr1 = np.array([[1, 4],
[7, 3]])
arr2 = np.array([[0, 1],
[2, 3],
[4, 5],
[4, 0],
[9, 9]])
np.hstack((np.repeat(arr1,len(arr2),0),np.stack((arr2,)*len(arr1)).reshape(-1,arr2.shape[1])))
output of shape (2,2)*(5,2)->(10,4):
[[1 4 0 1]
[1 4 2 3]
[1 4 4 5]
[1 4 4 0]
[1 4 9 9]
[7 3 0 1]
[7 3 2 3]
[7 3 4 5]
[7 3 4 0]
[7 3 9 9]]
You can use hstack to add columns to arr2, and vstack to get the final array.
np.vstack(np.apply_along_axis(lambda x: np.hstack([np.repeat(x[0], arr2.shape[0]).reshape(-1, 1),
arr2]),
1,
arr1[:, None]))
I think this should do it:
import numpy as np
arr0 = np.array([1, 4, 7, 3])
arr1 = np.reshape(arr0, (len(arr0),1))
arr2 = np.array([[0, 1],
[2, 3],
[4, 5],
[4, 0],
[9, 9]])
r1,c1 = arr1.shape
r2,c2 = arr2.shape
arrOut = np.zeros((r1,r2,c1+c2), dtype=arr1.dtype)
arrOut[:,:,:c1] = arr1[:,None,:]
arrOut[:,:,c1:] = arr2
arrOut.reshape(-1,c1+c2)
The output is:
array([[1, 0, 1],
[1, 2, 3],
[1, 4, 5],
[1, 4, 0],
[1, 9, 9],
[4, 0, 1],
[4, 2, 3],
[4, 4, 5],
[4, 4, 0],
[4, 9, 9],
[7, 0, 1],
[7, 2, 3],
[7, 4, 5],
[7, 4, 0],
[7, 9, 9],
[3, 0, 1],
[3, 2, 3],
[3, 4, 5],
[3, 4, 0],
[3, 9, 9]])
I want to compute the mean for axis=0 for a tf.SparseTensor. I want something like tf.sparse_reduce_sum. TensorFlow doesn't provide a similar function for the mean calculation. Is there any way to count the values in each row in order to divide them with the sum?
indices = np.array([[0, 0], [0, 1], [0, 2], [0, 3], [0, 4], [0, 5],
[1, 0], [1, 1], [1, 3], [1, 4], [1, 5],
[2, 1], [2, 2], [2, 3], [2, 4],
[3, 0], [3, 1], [3, 2], [3, 3], [3, 4], [3, 5],
[4, 0], [4, 2], [4, 3], [4, 4], [4, 5]], dtype=np.int64)
values = np.array([7, 6, 7, 4, 5, 4,
6, 7, 4, 3, 4,
3, 3, 1, 1,
1, 2, 2, 3, 3, 4,
1, 1, 2, 3, 3], dtype=np.float64)
dense_shape = np.array([5, 6], dtype=np.int64)
tRatings = tf.SparseTensor(indices, values, dense_shape)
You can compute the reduced mean from the reduced sum by dividing by the size of the 0th dimension:
tRatings = tf.SparseTensor(indices, values, dense_shape)
reduced_sum = tf.sparse_reduce_sum(tRatings, 0) # Sum of each row
reduced_mean = reduced_sum / tf.cast(tRatings.dense_shape[0], tf.float64) # Mean of each row
Try to use get_shape() and then multiply shape[0] * shape[1] this is the total number of elements
I'd like to turn an open mesh returned by the numpy ix_ routine to a list of coordinates
eg, for:
In[1]: m = np.ix_([0, 2, 4], [1, 3])
In[2]: m
Out[2]:
(array([[0],
[2],
[4]]), array([[1, 3]]))
What I would like is:
([0, 1], [0, 3], [2, 1], [2, 3], [4, 1], [4, 3])
I'm pretty sure I could hack it together with some iterating, unpacking and zipping, but I'm sure there must be a smart numpy way of achieving this...
Approach #1 Use np.meshgrid and then stack -
r,c = np.meshgrid(*m)
out = np.column_stack((r.ravel('F'), c.ravel('F') ))
Approach #2 Alternatively, with np.array() and then transposing, reshaping -
np.array(np.meshgrid(*m)).T.reshape(-1,len(m))
For a generic case with for generic number of arrays used within np.ix_, here are the modifications needed -
p = np.r_[2:0:-1,3:len(m)+1,0]
out = np.array(np.meshgrid(*m)).transpose(p).reshape(-1,len(m))
Sample runs -
Two arrays case :
In [376]: m = np.ix_([0, 2, 4], [1, 3])
In [377]: p = np.r_[2:0:-1,3:len(m)+1,0]
In [378]: np.array(np.meshgrid(*m)).transpose(p).reshape(-1,len(m))
Out[378]:
array([[0, 1],
[0, 3],
[2, 1],
[2, 3],
[4, 1],
[4, 3]])
Three arrays case :
In [379]: m = np.ix_([0, 2, 4], [1, 3],[6,5,9])
In [380]: p = np.r_[2:0:-1,3:len(m)+1,0]
In [381]: np.array(np.meshgrid(*m)).transpose(p).reshape(-1,len(m))
Out[381]:
array([[0, 1, 6],
[0, 1, 5],
[0, 1, 9],
[0, 3, 6],
[0, 3, 5],
[0, 3, 9],
[2, 1, 6],
[2, 1, 5],
[2, 1, 9],
[2, 3, 6],
[2, 3, 5],
[2, 3, 9],
[4, 1, 6],
[4, 1, 5],
[4, 1, 9],
[4, 3, 6],
[4, 3, 5],
[4, 3, 9]])
I'd like to obtain a 1D array of indexes from a 3D matrix.
For instance given x = np.random.randint(10, size=(10,3,3)), I'd like to do something like np.argmax(x, axis=(1,2)) just like you can do with np.max, that is, obtain a 1D array of length 10 containing the indexes (0 to 8) of the maximums of each submatrix of size (3,3).
I have not found anything helpful so far and I want to avoid looping on the first dimension (and use np.argmax(x)) as it is quite big.
Cheers!
Reshape to merge those last two axes and then use np.argmax -
idx = x.reshape(x.shape[0],-1).argmax(-1)
out = np.unravel_index(idx, x.shape[-2:])
Sample run -
In [263]: x = np.random.randint(10, size=(4,3,3))
In [264]: x
Out[264]:
array([[[0, 9, 2],
[7, 7, 8],
[2, 5, 9]],
[[1, 7, 2],
[8, 9, 0],
[2, 8, 3]],
[[7, 5, 0],
[7, 1, 6],
[5, 1, 1]],
[[0, 7, 3],
[5, 4, 1],
[9, 8, 9]]])
In [265]: idx = x.reshape(x.shape[0],-1).argmax(-1)
In [266]: np.unravel_index(idx, x.shape[-2:])
Out[266]: (array([0, 1, 0, 2]), array([1, 1, 0, 0]))
If you meant getting the merged index, then its simpler -
x.reshape(x.shape[0],-1).argmax(1)
Sample run -
In [283]: x
Out[283]:
array([[[2, 3, 7],
[8, 1, 0],
[3, 6, 9]],
[[8, 0, 5],
[2, 2, 9],
[9, 0, 9]],
[[1, 9, 2],
[5, 0, 3],
[7, 2, 1]],
[[1, 6, 5],
[2, 3, 7],
[7, 4, 6]]])
In [284]: x.reshape(x.shape[0],-1).argmax(1)
Out[284]: array([8, 5, 1, 5])