I want to override numpy.matmul() function, such that instead of dot product of two matrix vectors, calculate how many elements are element-wise equal between those two vectors.
Example:
In [1]: import numpy as np
In [2]: m1 = np.matrix([[1, 2], [3, 4]])
In [3]: m2 = np.matrix([[1, 3], [2, 4]])
In [4]: np.matmul(m1, m2)
Out[4]:
matrix([[ 5, 11],
[11, 25]])
Instead of the result above, I want to have following:
In [4]: myfunction(m1, m2)
Out[4]:
matrix([[2, 0],
[0, 2]])
[1, 2] and [1, 2] dot product is 5, but I want np.sum([1,2] == [1,2]), which is 2.
My question is how can I change in np.matmul() function np.dot() product of two v1 and v2 vectors with np.sum(v1 == v2)
In [93]: m1,m2=np.array([[1,2],[3,4]]),np.array([[1,3],[2,4]])
In [94]: np.matmul(m1,m2)
Out[94]:
array([[ 5, 11],
[11, 25]])
The dot product can be calculated with:
In [95]: (m1[:,:,None]*m2[None,:,:]).sum(axis=1)
Out[95]:
array([[ 5, 11],
[11, 25]])
This doesn't have the speed of matmul because it doesn't pass anything to BLAS-like functions, but it is still reasonably fast, making full use of numpy array operations.
I think your desired version would be:
In [96]: (m1[:,:,None]==m2[None,:,:]).sum(axis=1)
Out[96]:
array([[2, 0],
[0, 2]])
Related
Suppose I have an array a of shape (2, 2, 2):
a = np.array([[[7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
and an array b that is the max of a: b=a.max(-1) (row-wise):
b = np.array([[9, 19],
[24, 18]])
I'd like to obtain the index of elements in b using index in flattened a, i.e. a.reshape(-1):
array([ 7, 9, 19, 18, 24, 5, 18, 11])
The result should be an array that is the same shape with b with indices of b in flattened a:
array([[1, 2],
[4, 6]])
Basically this is the result of maxpool2d when return_indices= True in pytorch, but I'm looking for an implementation in numpy. I've used where but it seems doesn't work, also is it possible to combine finding max and indices in one go, to be more efficient? Thanks for any help!
I have a solution similar to that of Andras based on np.argmax and np.arange. Instead of "indexing the index" I propose to add a piecewise offset to the result of np.argmax:
import numpy as np
a = np.array([[[7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
off = np.arange(0, a.size, a.shape[2]).reshape(a.shape[0], a.shape[1])
>>> off
array([[0, 2],
[4, 6]])
This results in:
>>> a.argmax(-1) + off
array([[1, 2],
[4, 6]])
Or as a one-liner:
>>> a.argmax(-1) + np.arange(0, a.size, a.shape[2]).reshape(a.shape[0], a.shape[1])
array([[1, 2],
[4, 6]])
The only solution I could think of right now is generating a 2d (or 3d, see below) range that indexes your flat array, and indexing into that with the maximum indices that define b (i.e. a.argmax(-1)):
import numpy as np
a = np.array([[[ 7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
multi_inds = a.argmax(-1)
b_shape = a.shape[:-1]
b_size = np.prod(b_shape)
flat_inds = np.arange(a.size).reshape(b_size, -1)
flat_max_inds = flat_inds[range(b_size), multi_inds.ravel()]
max_inds = flat_max_inds.reshape(b_shape)
I separated the steps with some meaningful variable names, which should hopefully explain what's going on.
multi_inds tells you which "column" to choose in each "row" in a to get the maximum:
>>> multi_inds
array([[1, 0],
[0, 0]])
flat_inds is a list of indices, from which one value is to be chosen in each row:
>>> flat_inds
array([[0, 1],
[2, 3],
[4, 5],
[6, 7]])
This is indexed into exactly according to the maximum indices in each row. flat_max_inds are the values you're looking for, but in a flat array:
>>> flat_max_inds
array([1, 2, 4, 6])
So we need to reshape that back to match b.shape:
>>> max_inds
array([[1, 2],
[4, 6]])
A slightly more obscure but also more elegant solution is to use a 3d index array and use broadcasted indexing into it:
import numpy as np
a = np.array([[[ 7, 9],
[19, 18]],
[[24, 5],
[18, 11]]])
multi_inds = a.argmax(-1)
i, j = np.indices(a.shape[:-1])
max_inds = np.arange(a.size).reshape(a.shape)[i, j, multi_inds]
This does the same thing without an intermediate flattening into 2d.
The last part is also how you can get b from multi_inds, i.e. without having to call a *max function a second time:
b = a[i, j, multi_inds]
This is a long one-liner
new = np.array([np.where(a.reshape(-1)==x)[0][0] for x in a.max(-1).reshape(-1)]).reshape(2,2)
print(new)
array([[1, 2],
[4, 3]])
However number = 18 is repeated twice; So which index is the target.
I am new to using numpy so sorry if this sounds obvious, I did try to search through stackoverflow before I post this though..
I have two "list of lists" numpy arrays of length n (n = 3 in the example below)
a = np.array([[1, 2], [3, 4], [5, 6]])
b = np.array([[2, 2], [3, 3], [4, 4]])
I want to get a 1d array with the dot product of the lists at each corresponding index, i.e.
[(1*2 + 2*2), (3*3 + 4*3), (5*4 + 6*4)]
[6, 21, 44]
how should I go about doing it? thanks in advance!
You can do this
np.sum(a*b,axis=1)
The sum method in the other answer is the most straight forward method:
In [19]: a = np.array([[1, 2], [3, 4], [5, 6]])
...: b = np.array([[2, 2], [3, 3], [4, 4]])
In [20]: a*b
Out[20]:
array([[ 2, 4],
[ 9, 12],
[20, 24]])
In [21]: _.sum(1)
Out[21]: array([ 6, 21, 44])
With dot we have think a bit outside the box. einsum is easiest way of specifying a dot like action with less-than-obvious dimension combinations:
In [22]: np.einsum('ij,ij->i',a,b)
Out[22]: array([ 6, 21, 44])
Note that the i dimension is carried through. dot does ij,jk->ik, which would require extracting the diagonal (throwing away extra terms). In matmul/# terms, the i dimension is a 'batch' one, that doesn't actually participate in the sum-of-products. To use that:
In [23]: a[:,None,:]#b[:,:,None]
Out[23]:
array([[[ 6]],
[[21]],
[[44]]])
and then remove the extra size 1 dimensions:
In [24]: _.squeeze()
Out[24]: array([ 6, 21, 44])
In einsum terms this is i1j,ij1->i11
I have the following arrays:
a = np.arange(12).reshape((2, 2, 3))
and
b = np.zeros((2, 2))
Now I want to use b to access a, s.t. at each for index i,j we take the z-th element of a, if b[i, j] = z.
Meaning for the above example the answer should be [[0, 3], [6, 9]].
I feel this is very related to np.choose, but yet somehow cannot quite manage it.
Can you help me?
Two approaches could be suggested.
With explicit range arrays for advanced-indexing -
m,n = b.shape
out = a[np.arange(m)[:,None],np.arange(n),b.astype(int)]
With np.take_along_axis -
np.take_along_axis(a,b.astype(int)[...,None],axis=2)[...,0]
Sample run -
In [44]: a
Out[44]:
array([[[ 0, 1, 2],
[ 3, 4, 5]],
[[ 6, 7, 8],
[ 9, 10, 11]]])
In [45]: b
Out[45]:
array([[0., 0.],
[0., 0.]])
In [46]: m,n = b.shape
In [47]: a[np.arange(m)[:,None],np.arange(n),b.astype(int)]
Out[47]:
array([[0, 3],
[6, 9]])
In [48]: np.take_along_axis(a,b.astype(int)[...,None],axis=2)[...,0]
Out[48]:
array([[0, 3],
[6, 9]])
I have a 2D array:
>>> in_arr = np.array([[1,2],[4,3]])
array([[1, 2],
[4, 3]])
and I find the sorted indices by columns to yield another 2D array:
>>> col_sort = np.argsort(in_arr, axis=1)
array([[0, 1],
[1, 0]])
I would like to know the efficient numpy slice to index the first by the second:
>>> redordered_in_arr = np.*SOME_SLICE_METHOD*(in_arr, col_sort, axis=1)
array([[1, 2],
[3, 4]])
The intention is to then perform a (more complicated) function on the array by column, e.g.:
>>> arr_with_function = reordered_in_arr ** np.array([1,2])
array([[1, 4],
[3, 16]])
and return the elements to their original position in the array
>>> return_order = np.argsort(col_sort, axis=1)
>>> redordered_in_arr = np.*SOME_SLICE_METHOD*(arr_with_function, return_order, axis=1)
array([[1, 4],
[16, 3]])
Ok so thinking about it as I type I might just use apply_over_axis, but I would still like know how to the above efficiently in case it is of value later..
If you want to do all those operations in-place then you don't need argsort(). Numpy supports in-place operations in such situations:
In [12]: in_arr = np.array([[1,2],[4,3]])
In [13]: in_arr.sort(axis=1)
In [14]: in_arr **= [1, 2]
In [15]: in_arr
Out[15]:
array([[ 1, 4],
[ 3, 16]])
But if you need the indices of the sorted items you can get the expected result with a simple indexing.
In [18]: in_arr[np.arange(2)[:,None], col_sort]
Out[18]:
array([[1, 2],
[3, 4]])
I want to implement the following calculation in python :
With
and
.
Here are matrices themselves.
Thus python should compute a matrix-times-vector multiplication, where the vector-elements are matrices.Is this possible without a for-loop?
Using numpy:
In [1]: import numpy as np
In [2]: M1 = np.array([[1, 2], [3, 4]])
In [3]: M2 = np.array([[10, 20], [30, 40]])
In [4]: E = np.array([[1, 1], [1, -1]])
In [5]: v = np.array([M1, M2])
In [6]: np.tensordot(E, v, 1)
Out[6]:
array([[[ 11, 22],
[ 33, 44]],
[[ -9, -18],
[-27, -36]]])
if you're familiar with einstein summation notation, you can imagine E subscripted as E_ij, and v as v_jkl, and more generally write the np.tensordot in #steve-kern's example as:
In [6]: np.einsum('ij,jkl',E,v)