I was wondering if there's a way to compute multiple outer products and stack the results in a single operation.
Say I have an Nx1 vector and take the outer product with a 1xM vector, the result will be an NxM matrix.
What if I had an NxR matrix A, and an RxM matrix B. Is it possible to construct an NxMxR matrix where each layer of the output matrix is the outer product of the corresponding column of A and row of B?
I know it's really easy to do this in a single for loop over R, but I wanted to know if there's a faster way using numpy builtins (as there usually is when numpy is concerned).
I haven't been able to figure out a set of indices that work with einsum (and I'm not even sure if einsum is the right approach, since there is no summation involved here)
Yes, of course, using broadcasting or Einsum (the fact that there is no summation does not matter)
N, M, R = 8, 9, 16
A = numpy.random.rand(N)
B = numpy.random.rand(M)
C = A[:, None] * B[None, :]
D = numpy.einsum('a,b->ab', A, B)
numpy.allclose(C, D)
# True
C.shape
# (8, 9)
A = numpy.random.rand(N, R)
B = numpy.random.rand(M, R)
C = A[:, None, :] * B[None, :, :]
D = numpy.einsum('ar,br->abr', A, B)
numpy.allclose(C, D)
# True
C.shape
# (8, 9, 16)
Related
I want to use tensordot to compute the dot product of a specific dim of two tensors. Like:
A is a tensor, whose shape is (3, 4, 5)
B is a tensor, whose shape is (3, 5)
I want to do a dot use A's third dim and B's second dim, and get a output whose dims is (3, 4)
Like below:
for i in range(3):
C[i] = dot(A[i], B[i])
How to do it by tensordot?
Well, do you want this in numpy or in Theano?
In the case, where, as you state, you would like to contract axis 3 of A against axis 2 of B, both are straightforward:
import numpy as np
a = np.arange(3 * 4 * 5).reshape(3, 4, 5).astype('float32')
b = np.arange(3 * 5).reshape(3, 5).astype('float32')
result = a.dot(b.T)
in Theano this writes as
import theano.tensor as T
A = T.ftensor3()
B = T.fmatrix()
out = A.dot(B.T)
out.eval({A: a, B: b})
however, the output then is of shape (3, 4, 3). Since you seem to want an output of shape (3, 4), the numpy alternative uses einsum, like so
einsum_out = np.einsum('ijk, ik -> ij', a, b)
However, einsum does not exist in Theano. So the specific case here can be emulated as follows
out = (a * b[:, np.newaxis]).sum(2)
which can also be written in Theano
out = (A * B.dimshuffle(0, 'x', 1)).sum(2)
out.eval({A: a, B: b})
In this specific case, einsum is probably easier to understand than tensordot. For example:
c = np.einsum('ijk,ik->ij', a, b)
I'm going to over-simplify the explanation a bit to make things more immediately understandable. We have two input arrays (separated by the comma) and this yields our output array (to the right of the ->).
a has shape 3, 4, 5 and we'll refer to it as ijk
b has shape 3, 5 (ik)
We want the output c to have shape 3, 4 (ij)
Seems a bit magical, right? Let's break that down a bit.
The letters we "lose" as we cross the -> are axes that will be summed over. That's what dot is doing, as well.
We want output with shape 3, 4, so we're eliminating k
Therefore, the output c should be ij
This means we'll refer to b as ik.
As a full example:
import numpy as np
a = np.random.random((3, 4, 5))
b = np.random.random((3, 5))
# Looping through things
c1 = []
for i in range(3):
c1.append(a[i].dot(b[i]))
c1 = np.array(c1)
# Using einsum instead
c2 = np.einsum('ijk,ik->ij', a, b)
assert np.allclose(c1, c2)
You can do this with tensordot as well. I'll add an example of that as soon as I have a bit more time. (Of course, if anyone else would like to add a tensordot example as another answer in the meantime, feel free!)
I have arrays e, (shape q by l) f (shape n by l), and w (shape n by l), and I want to create an array M where M[s,i,j] = np.sum(w[s, :] * e[i, :] * e[j, :]), and an array F, where F[s,j] = np.sum(w[s, :] * f[s, :] * e[j, :]).
Both are easy enough to do by, for instance, looping through elements of M, but I want to be more efficient (my real data has something like 1M entries of length 5k). For F, I can use F = np.inner(w * f, e) (which I verified produces the same answer as the loop). M is more difficult, so the first step is to loop through dimension zero of with a list comprehension, saying that M = np.stack([np.inner(r[:] * e, e) for r in w]) (I have verified this also works the same as the loop). np.inner() doesn't take any axes arguments, so it's not clear to me how to tell the arrays to just broadcast over all rows of w.
Finally, I need to use elements of M and F to create a matrix A, where A[s,i] = np.sum(np.linalg.inv(M[s, :, :])[i, :] * F[i, :]). This also looks inner-product-ish, but taking lots of individual inverses is time-consuming, so is there a way to compute inverses of slices, short of looping?
Some test values in my arrays are as follows:
e = array([[-0.9840087 , -0.17812043],
[ 0.17812043, -0.9840087 ]])
w = array([[ 1.12545297e+01, 1.48690140e+02],
[ 7.30718244e+00, 4.07840612e+02],
[ 2.62753065e+02, 2.27085711e+02],
[ 1.53045364e+01, 5.63025281e+02],
[ 8.00555079e+00, 2.16207407e+02],
[ 1.54070190e+01, 1.87213209e+06],
[ 2.71802081e+01, 1.06392902e+02],
[ 3.46300255e+01, 1.29404438e+03],
[ 7.77638140e+00, 4.18759293e+03],
[ 1.12874849e+01, 5.75023379e+02]])
f = array([[ 0.48907404, 0.06111084],
[-0.21899297, -0.02207311],
[ 0.58688524, 0.05156326],
[ 0.57407751, 0.10004592],
[ 0.94172351, 0.03895357],
[-0.7489003 , -0.08911183],
[-0.7043736 , -0.19014227],
[ 0.58950925, 0.16587887],
[-0.35557142, -0.14530267],
[ 0.24548714, 0.03221844]])
M[s,i,j] = np.sum(w[s, :] * e[i, :] * e[j, :])
translates to
M = np.einsum('sk,ik,jk->sij',w,e,e)
and
F[s,j] = np.sum(w[s, :] * f[s, :] * e[j, :])
F = np.einsum('sk,sk,jk->sj', w, f, e)
I haven't tested these with your samples, but the translation is simple enough.
With real large arrays you may have to break the expressions up into pieces. With 4 iteration variables the overall iteration space can be very big. But first see if these expressions work with modest sized arrays.
As for
A[s,i] = np.sum(np.linalg.inv(M[s, :, :])[i, :] * F[i, :])
I looks like np.linalg.inv(M) works, performing the s i x i inverses
If so then
IM = np.linalg.inv(M)
A = np.einsum('skm,ik,im->si', IM, F)
I'm guessing more here.
Again, dimension might get too large, but try it small first.
Typically linear equation solutions are recommended over direct inverses, something like
A = F/M
A = np.linalg.solve(M, F)
since you probably want A such that M#A=F (# matrix product). But I'm kind of rusty on these matters. Also check tensorsolve and tensorinv.
This is pretty much the same question as here Matrix/Tensor Triple Product? , but for theano.
So I have three matrices A, B, C of sizes n*r, m*r, l*r, and I want to compute the 3D tensor of shape (n,m,l) resulting from the triple (trilinear) product:
X[i,j,k] = \sum_a A[i,a] B[j,a] C[k,a]
A, B and C are shared variables:
A = theano.shared(numpy.random.randn(n,r))
B = theano.shared(numpy.random.randn(m,r))
C = theano.shared(numpy.random.randn(l,r))
I'd like to write it with a single theano expression, is there a way to do so?
If there are many, which one is the fastest?
np.einsum('nr,mr,lr->nml', A, B, C)
is equivalent to
np.dot(A[:, None, :] * B[None, :, :], C.T)
which can be implemented in Theano as
theano.dot(A[:, None, :] * B[None, :, :], C.T)
I'm trying to do an integration with numpy:
A = n.trapz(B,C)
but I have some issues with B and C shapes
B is a filled array inizialized with numpy zeros function
B=np.zeros((N,1))
C is a column extracted from a matrix, always inizialized with numpy:
C = D[:,0]
D = np.zeros((N,2))
the problem is that:
n.shape(B) # (N,1)
n.shape(C) # (N,)
how can I manage this?
Try
B = np.zeros(N)
np.trapz(B, C)
Also, you np.trapz accepts multi-dimensional arrays, so arrays of shape (N, 1) are ok; you just need to specify an axis to handle it properly.
B = np.zeros((N, 1))
C = D[:, 0]
np.trapz(B, C.reshape(N, 1), axis=1)
I have a matrix
A = [[ 1. 1.]
[ 1. 1.]]
and two arrays (a and b), every array contains 20 float numbers How can I multiply the using formula:
( x' = A * ( x )
y' ) y
Is this correct? m = A * [a, b]
Matrix multiplication with NumPy arrays can be done with np.dot.
If X has shape (i,j) and Y has shape (j,k) then np.dot(X,Y) will be the matrix product and have shape (i,k). The last axis of X and the second-to-last axis of Y is multiplied and summed over.
Now, if a and b have shape (20,), then np.vstack([a,b]) has shape (2, 20):
In [66]: np.vstack([a,b]).shape
Out[66]: (2, 20)
You can think of np.vstack([a, b]) as a 2x20 matrix with the values of a on the first row, and the values of b on the second row.
Since A has shape (2,2), we can perform the matrix multiplication
m = np.dot(A, np.vstack([a,b]))
to arrive at an array of shape (2, 20).
The first row of m contains the x' values, the second row contains the y' values.
NumPy also has a matrix subclass of ndarray (a special kind of NumPy array) which has convenient syntax for doing matrix multiplication with 2D arrays. If we define A to be a matrix (rather than a plain ndarray which is what np.array(...) creates), then matrix multiplication can be done with the * operator.
I show both ways (with A being a plain ndarray and A2 being a matrix) below:
import numpy as np
A = np.array([[1.,1.],[1.,1.]])
A2 = np.matrix([[1.,1.],[1.,1.]])
a = np.random.random(20)
b = np.random.random(20)
c = np.vstack([a,b])
m = np.dot(A, c)
m2 = A2 * c
assert np.allclose(m, m2)