Related
I have two matrices (numpy arrays), mu and nu. From these I would like to create a third array as follows:
new_array_{j, k, l} = mu_{l, k} nu_{j, k}
I can do it naively using list comprehensions:
[[[mu[l, k] * nu[j, k] for k in np.arange(N)] for l in np.arange(N)] for j in np.arange(N)]
but it quickly becomes slow.
How can I create new_array using numpy functions which should be faster?
Two quick solutions (without my usual proofs and explanations):
res = np.einsum('lk,jk->jkl', mu, nu)
res = mu.T[None,:,:] * nu[:,:,None] # axes in same order as result
#!/usr/bin/env python
import numpy as np
# example data
mu = np.arange(10).reshape(2,5)
nu = np.arange(15).reshape(3,5) + 20
# get array sizes
nl, nk = mu.shape
nj, nk_ = nu.shape
assert(nk == nk_)
# get arrays with dimensions (nj, nk, nl)
# in the case of mu3d, we need to add a slowest varying dimension
# so (after transposing) this can be done by cycling through the data
# nj times along the slowest existing axis and then reshaping
mu3d = np.concatenate((mu.transpose(),) * nj).reshape(nj, nk, nl)
# in the case of nu3d, we need to add a new fastest varying dimension
# so this can be done by repeating each element nl times, and again it
# needs reshaping
nu3d = nu.repeat(nl).reshape(nj, nk, nl)
# now just multiple element by element
new_array = mu3d * nu3d
print(new_array)
Gives:
>>> mu
array([[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]])
>>> nu
array([[20, 21, 22, 23, 24],
[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34]])
>>> nj, nk, nl
(3, 5, 2)
>>> mu3d
array([[[0, 5],
[1, 6],
[2, 7],
[3, 8],
[4, 9]],
[[0, 5],
[1, 6],
[2, 7],
[3, 8],
[4, 9]],
[[0, 5],
[1, 6],
[2, 7],
[3, 8],
[4, 9]]])
>>> nu3d
array([[[20, 20],
[21, 21],
[22, 22],
[23, 23],
[24, 24]],
[[25, 25],
[26, 26],
[27, 27],
[28, 28],
[29, 29]],
[[30, 30],
[31, 31],
[32, 32],
[33, 33],
[34, 34]]])
>>> new_array
array([[[ 0, 100],
[ 21, 126],
[ 44, 154],
[ 69, 184],
[ 96, 216]],
[[ 0, 125],
[ 26, 156],
[ 54, 189],
[ 84, 224],
[116, 261]],
[[ 0, 150],
[ 31, 186],
[ 64, 224],
[ 99, 264],
[136, 306]]])
I am generating multidimensional array of different sizes, though they'll all have an even number of columns.
>> import numpy as np
>> x = np.arange(24).reshape((3,8))
Which results in:
array([[ 0, 1, 2, 3, 4, 5, 6, 7],
[ 8, 9, 10, 11, 12, 13, 14, 15],
[16, 17, 18, 19, 20, 21, 22, 23]])
I am able to slice with numpy and get the first column in an array:
>> newarr = x[0:,0:2]
array([[ 0, 1],
[ 8, 9],
[16, 17]])
However, I want to have one array that is just a list of the columns where column 1 and 2 are together, 3 and 4 are together, and so on.. For example:
array([[[ 0, 1],
[ 8, 9],
[16, 17]],
[[ 2, 3],
[10, 11],
[18, 19]],
etc....]
)
This code below works but it's clunky and my arrays are not all the same. Some arrays have 16 columns, some have 34, some have 50, etc.
>> newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]])]
There's got to be a better way to do this than
newarr = [x[0:,0:2]]+[x[0:,2:4]]+[x[0:,4:6]]+...+[x[0:,n:n+2]]
Help!
My idea is adding a for loop:
slice_len = 2
x_list = [x[0:, slice_len*i:slice_len*(i+1)] for i in range(x.shape[1] // slice_len)]
Output:
[array([[ 0, 1],
[ 8, 9],
[16, 17]]), array([[ 2, 3],
[10, 11],
[18, 19]]), array([[ 4, 5],
[12, 13],
[20, 21]]), array([[ 6, 7],
[14, 15],
[22, 23]])]
I have many 3*2 matrices(A1,A2,A3..), and each of the 3*2 is a draw. In the case two draws, we have a 3*4 ( we horizontally stack each draw of A1,A2). Clearly, it is easier for me to draw the 3*4 matrix (A) as a larger matrices once instead of draw a 3*2 over and over again.
But I need to perform a matrix multiplication for each draw(each A1,A2...) to a matrix B. Say A1*B, and A2*B ...AN*B
#each draw of the 3*2 matrix
A1 = np.array([[ 0, 1],
[ 4, 5],
[ 8, 9]])
A2 = np.array([[ 2, 3],
[ 6, 7],
[ 10, 11]])
# A is [A1,A2]
# Easier to draw A once for all (the larger matrix)
A = np.array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
b = np.array([[ 0, 1],
[ 4, 5]
])
desired output
array([[ 4, 5, 12, 17],
[20, 29, 28, 41],
[36, 53, 44, 65]])
You can reshape matrix A to 2 columns so that it is conformable to b, do the matrix multiplication, and then reshape it back:
np.dot(A.reshape(-1, 2), b).reshape(3, -1)
#array([[ 4, 5, 12, 17],
# [20, 29, 28, 41],
# [36, 53, 44, 65]])
If you are unsure about how to store/stack the incoming arrays, one way would be stacking those as a 3D array, such that the each of those incoming arrays are index-able by its first axis -
a = np.array((A1,A2))
Sample run -
In [143]: a = np.array((A1,A2))
In [144]: a.shape
Out[144]: (2, 3, 2)
|-----------------> axis of stacking
Then, to get the equivalent output of matrix-multiplications of each incoming array with b, we could use np.tensordot on the 3D stacked array a with b, thus losing the last axis from a and first from b in the sum-reduction, like so -
out = np.tensordot(a,b,axes=((2),(0)))
Let's have a look at the output values and compare against each matrix-multiplication with A1, A2, etc. -
In [138]: out[0]
Out[138]:
array([[ 4, 5],
[20, 29],
[36, 53]])
In [139]: out[1]
Out[139]:
array([[12, 17],
[28, 41],
[44, 65]])
In [140]: A1.dot(b)
Out[140]:
array([[ 4, 5],
[20, 29],
[36, 53]])
In [141]: A2.dot(b)
Out[141]:
array([[12, 17],
[28, 41],
[44, 65]])
Thus, essentially with this stacking operation and later on tensordot we have :
out[0], out[1], .... = A1.dot(b), A2.dot(b), ....
Alternative to np.tensordot -
We could use a simpler version with np.matmul, to get the same output as with tensordot -
out = np.matmul(a,b)
On Python 3.5, there's an even simpler version that replaces np.matmul, the # operator -
out = a # b
Even if not needed for the calculation einsum can help us think through the problem:
In [584]: np.einsum('ij,jk->ik', A1,b)
Out[584]:
array([[ 4, 5],
[20, 29],
[36, 53]])
In [585]: np.einsum('ij,jk->ik', A2,b)
Out[585]:
array([[12, 17],
[28, 41],
[44, 65]])
A is (3,4), which won't work with the (2,2) b. Think of it as trying work with a doubled j dimension: 'i(2j),jk->i?k'. But what if we inserted an axis? 'imk,jk->imk'? Or added the extra dimension to i?
In [587]: np.einsum('imj,jk->imk', A.reshape(3,2,2),b)
Out[587]:
array([[[ 4, 5],
[12, 17]],
[[20, 29],
[28, 41]],
[[36, 53],
[44, 65]]])
The numbers are there, just the shape is (3,2,2).
In [590]: np.einsum('imj,jk->imk', A.reshape(3,2,2),b).reshape(3,4)
Out[590]:
array([[ 4, 5, 12, 17],
[20, 29, 28, 41],
[36, 53, 44, 65]])
Or you could build A from the start so that mij,jk->mik works (#Divaker)
#Psidom:
np.einsum('ij,jk->ik', A.reshape(3,2,2).reshape(-1,2) ,b).reshape(3,-1)
`#piRSquared':
'kj,jI->kI`
Shift you perspective. You are locking yourself into 3 x 2 unnecessarily.
You can think of A1 and A2 as 2x3 instead, then A would be
array([[ 0, 4, 8, 2, 6, 10],
[ 1, 5, 9, 3, 7, 11]])
Then take the transpose of b = b.T
array([[0, 4],
[1, 5]])
So that you can do you operation
b # A
array([[ 4, 20, 36, 12, 28, 44],
[ 5, 29, 53, 17, 41, 65]])
Let your "draws" look like this
A = np.random.randint(10, size=(2, 9))
A
array([[7, 2, 1, 0, 9, 9, 1, 0, 2],
[8, 6, 1, 6, 6, 2, 4, 2, 9]])
b # A
array([[32, 24, 4, 24, 24, 8, 16, 8, 36],
[47, 32, 6, 30, 39, 19, 21, 10, 47]])
I have the following array:
x = np.arange(24).reshape((2,3,2,2))
array([[[[ 0, 1],
[ 2, 3]],
[[ 4, 5],
[ 6, 7]],
[[ 8, 9],
[10, 11]]],
[[[12, 13],
[14, 15]],
[[16, 17],
[18, 19]],
[[20, 21],
[22, 23]]]])
I would like to reshape it to a (3,4,2) array like below:
array([[[ 0, 1],
[ 2, 3],
[12, 13],
[14, 15]],
[[ 4, 5],
[ 6, 7],
[16, 17],
[18, 19]],
[[ 8, 9],
[10, 11],
[20, 21],
[22, 23]]])
I've tried to use reshape but it gave me the following which is not what I want.
array([[[ 0, 1],
[ 2, 3],
[ 4, 5],
[ 6, 7]],
[[ 8, 9],
[10, 11],
[12, 13],
[14, 15]],
[[16, 17],
[18, 19],
[20, 21],
[22, 23]]])
Can someone please help?
Use transpose and then reshape like so -
shp = x.shape
out = x.transpose(1,0,2,3).reshape(shp[1],-1,shp[-1])
x = np.arange(24).reshape((2,3,2,2))
y = np.dstack(zip(x))[0]
print y
result:
[[[ 0 1]
[ 2 3]
[12 13]
[14 15]]
[[ 4 5]
[ 6 7]
[16 17]
[18 19]]
[[ 8 9]
[10 11]
[20 21]
[22 23]]]
You can also use concatenate like so-
out=np.concatenate((x),axis=1)
I will note those since you mentioned this is for performance, this doesn't seem faster than Divakar suggestion:
shp = x.shape
out = x.transpose(1,0,2,3).reshape(shp[1],-1,shp[-1])
If anyone does a bench mark or finds something faster I would love to know.
I have an 2D-array a with shape (k,n) and I want to 'multiply' it with an 1D-array b of shape (m,):
a = np.array([[2, 8],
[4, 7],
[1, 2],
[5, 2],
[7, 4]])
b = np.array([3, 5, 5])
As a result of the 'multiplication' I'm looking for:
array([[[2*3,2*5,2*5],[8*3,8*5,8*5]],
[[4*3,4*5,4*5],[7*3,7*5,7*5]],
[[1*3,1*5,1*5], ..... ]],
................. ]]])
= array([[[ 6, 10, 10],
[24, 40, 40]],
[[12, 20, 20],
[21, 35, 35]],
[[ 3, 5, 5],
[ ........ ]],
....... ]]])
I could solve it with a loop of course, but I'm looking for a fast vectorized way of doing it.
Extend a to a 3D array case by adding a new axis at the end with np.newaxis/None and then do elementwise multiplication with b, bringing in broadcasting for a vectorized solution, like so -
b*a[...,None]
Sample run -
In [19]: a
Out[19]:
array([[2, 8],
[4, 7],
[1, 2],
[5, 2],
[7, 4]])
In [20]: b
Out[20]: array([3, 5, 5])
In [21]: b*a[...,None]
Out[21]:
array([[[ 6, 10, 10],
[24, 40, 40]],
[[12, 20, 20],
[21, 35, 35]],
[[ 3, 5, 5],
[ 6, 10, 10]],
[[15, 25, 25],
[ 6, 10, 10]],
[[21, 35, 35],
[12, 20, 20]]])