This is pretty much the same question as here Matrix/Tensor Triple Product? , but for theano.
So I have three matrices A, B, C of sizes n*r, m*r, l*r, and I want to compute the 3D tensor of shape (n,m,l) resulting from the triple (trilinear) product:
X[i,j,k] = \sum_a A[i,a] B[j,a] C[k,a]
A, B and C are shared variables:
A = theano.shared(numpy.random.randn(n,r))
B = theano.shared(numpy.random.randn(m,r))
C = theano.shared(numpy.random.randn(l,r))
I'd like to write it with a single theano expression, is there a way to do so?
If there are many, which one is the fastest?
np.einsum('nr,mr,lr->nml', A, B, C)
is equivalent to
np.dot(A[:, None, :] * B[None, :, :], C.T)
which can be implemented in Theano as
theano.dot(A[:, None, :] * B[None, :, :], C.T)
Related
I've two tensors of shape a(16,8,8,64) and b(64,64). Suppose, I extract last dimension of ainto another column vector c, I want to compute matmul(matmul(c.T, b), c). I want this to be done in each of the first 3 dimensions of a. That is the final product should be of shape (16,8,8,1). How can I achieve this in pytorch?
Can be done as follows:
row_vec = a[:, :, :, None, :].float()
col_vec = a[:, :, :, :, None].float()
b = (b[None, None, None, :, :]).float()
prod = torch.matmul(torch.matmul(row_vec, b), col_vec)
I was wondering if there's a way to compute multiple outer products and stack the results in a single operation.
Say I have an Nx1 vector and take the outer product with a 1xM vector, the result will be an NxM matrix.
What if I had an NxR matrix A, and an RxM matrix B. Is it possible to construct an NxMxR matrix where each layer of the output matrix is the outer product of the corresponding column of A and row of B?
I know it's really easy to do this in a single for loop over R, but I wanted to know if there's a faster way using numpy builtins (as there usually is when numpy is concerned).
I haven't been able to figure out a set of indices that work with einsum (and I'm not even sure if einsum is the right approach, since there is no summation involved here)
Yes, of course, using broadcasting or Einsum (the fact that there is no summation does not matter)
N, M, R = 8, 9, 16
A = numpy.random.rand(N)
B = numpy.random.rand(M)
C = A[:, None] * B[None, :]
D = numpy.einsum('a,b->ab', A, B)
numpy.allclose(C, D)
# True
C.shape
# (8, 9)
A = numpy.random.rand(N, R)
B = numpy.random.rand(M, R)
C = A[:, None, :] * B[None, :, :]
D = numpy.einsum('ar,br->abr', A, B)
numpy.allclose(C, D)
# True
C.shape
# (8, 9, 16)
I have arrays e, (shape q by l) f (shape n by l), and w (shape n by l), and I want to create an array M where M[s,i,j] = np.sum(w[s, :] * e[i, :] * e[j, :]), and an array F, where F[s,j] = np.sum(w[s, :] * f[s, :] * e[j, :]).
Both are easy enough to do by, for instance, looping through elements of M, but I want to be more efficient (my real data has something like 1M entries of length 5k). For F, I can use F = np.inner(w * f, e) (which I verified produces the same answer as the loop). M is more difficult, so the first step is to loop through dimension zero of with a list comprehension, saying that M = np.stack([np.inner(r[:] * e, e) for r in w]) (I have verified this also works the same as the loop). np.inner() doesn't take any axes arguments, so it's not clear to me how to tell the arrays to just broadcast over all rows of w.
Finally, I need to use elements of M and F to create a matrix A, where A[s,i] = np.sum(np.linalg.inv(M[s, :, :])[i, :] * F[i, :]). This also looks inner-product-ish, but taking lots of individual inverses is time-consuming, so is there a way to compute inverses of slices, short of looping?
Some test values in my arrays are as follows:
e = array([[-0.9840087 , -0.17812043],
[ 0.17812043, -0.9840087 ]])
w = array([[ 1.12545297e+01, 1.48690140e+02],
[ 7.30718244e+00, 4.07840612e+02],
[ 2.62753065e+02, 2.27085711e+02],
[ 1.53045364e+01, 5.63025281e+02],
[ 8.00555079e+00, 2.16207407e+02],
[ 1.54070190e+01, 1.87213209e+06],
[ 2.71802081e+01, 1.06392902e+02],
[ 3.46300255e+01, 1.29404438e+03],
[ 7.77638140e+00, 4.18759293e+03],
[ 1.12874849e+01, 5.75023379e+02]])
f = array([[ 0.48907404, 0.06111084],
[-0.21899297, -0.02207311],
[ 0.58688524, 0.05156326],
[ 0.57407751, 0.10004592],
[ 0.94172351, 0.03895357],
[-0.7489003 , -0.08911183],
[-0.7043736 , -0.19014227],
[ 0.58950925, 0.16587887],
[-0.35557142, -0.14530267],
[ 0.24548714, 0.03221844]])
M[s,i,j] = np.sum(w[s, :] * e[i, :] * e[j, :])
translates to
M = np.einsum('sk,ik,jk->sij',w,e,e)
and
F[s,j] = np.sum(w[s, :] * f[s, :] * e[j, :])
F = np.einsum('sk,sk,jk->sj', w, f, e)
I haven't tested these with your samples, but the translation is simple enough.
With real large arrays you may have to break the expressions up into pieces. With 4 iteration variables the overall iteration space can be very big. But first see if these expressions work with modest sized arrays.
As for
A[s,i] = np.sum(np.linalg.inv(M[s, :, :])[i, :] * F[i, :])
I looks like np.linalg.inv(M) works, performing the s i x i inverses
If so then
IM = np.linalg.inv(M)
A = np.einsum('skm,ik,im->si', IM, F)
I'm guessing more here.
Again, dimension might get too large, but try it small first.
Typically linear equation solutions are recommended over direct inverses, something like
A = F/M
A = np.linalg.solve(M, F)
since you probably want A such that M#A=F (# matrix product). But I'm kind of rusty on these matters. Also check tensorsolve and tensorinv.
Let tensor T has shape [B, N, N, 6] and I want to multiply matrices [b, N, N, 0:3] by [b, N, N, 5] element-wise for each b in range(B). Note, that [N, N, 4] should not be changed. What is the best way to do this using tensorflow?
My attempts:
result = tf.empty([B, N, N, 5])
for b in range(B):
for i in range(4)
result[b, :, :, i] = tf.mul(T[b, :, :, i], T[b, :, :, 5])
result[b, :, :, 4] = T[b, :, :, 4]
In TensorFlow, it's not generally possible to build a tensor value by assigning to slices. The programming model tends to be more functional than imperative. One way of implementing your calculation is as follows:
result = tf.concat(3, [tf.mul(T[:, :, :, 0:4], T[:, :, :, 5:6]), T[:, :, :, 4:5]])
Note that you don't need multiple multiplications, because (i) the original computation is already element-wise on the 0th dimension (for b in range(B)), and (ii) TensorFlow will broadcast the second argument to the multiplication in the 3rd dimension.
I'm trying to do an integration with numpy:
A = n.trapz(B,C)
but I have some issues with B and C shapes
B is a filled array inizialized with numpy zeros function
B=np.zeros((N,1))
C is a column extracted from a matrix, always inizialized with numpy:
C = D[:,0]
D = np.zeros((N,2))
the problem is that:
n.shape(B) # (N,1)
n.shape(C) # (N,)
how can I manage this?
Try
B = np.zeros(N)
np.trapz(B, C)
Also, you np.trapz accepts multi-dimensional arrays, so arrays of shape (N, 1) are ok; you just need to specify an axis to handle it properly.
B = np.zeros((N, 1))
C = D[:, 0]
np.trapz(B, C.reshape(N, 1), axis=1)