I need to do the following two operations:
solve Ax=b by inverting the n-by-n matrix A, and
solve r=Ar using power iteration (i.e. by repeated multiplying current vector r by A) such as one would do for the PageRank algorithm.
My question is: When computing the matrix-vector product A^{-1}b or the matrix-vector product Ar, is it better to use numpy.dot or numpy.matmul? (I understand there might be differences in higher dimensions, but my question is only for the case where A is a 2D array and b, r are vectors.)
From the numpy doc for np.dot:
Dot product of two arrays. Specifically, If both a and b are 2-D arrays, it is matrix multiplication, but using matmul or a # b is
preferred.
So basically for your case, it does not matter, although matmul is preferred according to the doc.
Also since one of your arrays is 1-D, from docs for np.matmul:
If the second argument is 1-D, it is promoted to a matrix by appending
a 1 to its dimensions. After matrix multiplication the appended 1 is
removed.
And:
matmul differs from dot in two important ways:
Multiplication by scalars is not allowed, use * instead. Stacks of matrices are
broadcast together as if the matrices were elements, respecting the
signature
Therefore, they would work the same in your case, but I would go with numpy doc's recommendation on using matmul.
Related
I have a matrix A with shape=(N, N) and a matrix B with the same shape=(N, N).
I am constructing a matrix M using the following einsum (using the opt_einsum library):
M = oe.contract('nm,in,jm,pn,qm->ijpq', A, B, B, B, B)
This is evaluating the following sum:
This yeilds matrix M with shape (N, N, N, N). I then reshape this to a 2D array of shape (N**2, N**2)
M = M.reshape((N**2, N**2))
This must be 2D as it is treated as a linear operator.
I want to use the sparse library, as M is sparse, and becomes too large to store for large N. I can make A and B sparse, and insert them into the oe.contract.
The problem is, sparse only supports 2D arrays and so fails to produce the 4D output of shape (N, N, N. N). Is there a way to combine the einsum and reshape steps to allow sparse to be used in this way, as the final shape of M is 2D?
This may not help with your use of opt_einsum, but with a bit of reorganizing I can speed up np.einsum quite a bit, at least for small arrays.
Do a partial product of two B:
c1 = np.einsum('in,jm->ijnm',B,B).reshape(N*N,N,N)
The pq pair is the same, so we don't need to recalculate it:
c2 = np.einsum('nm,onm,pnm->op',A,c1,c1)
I verified that this works for two (3,3) arrays, and the speed up is about 10x.
We can even reshape the nm to 1d, though this doesn't improve speed:
c1 = np.einsum('in,jm->ijnm',B,B).reshape(N*N,N*N)
c3 = np.einsum('n,on,pn->op',A.reshape(N*N),c1,c1)
I did not correctly interpret the error given by opt_einsum.
The problem is not that sparse does not support ND sparse arrays (it does!), but that I was not using a true einsum, as the indices summed over appear more than twice (n and m). As stated in the opt_einsum documentation this will result in the use of the sparse.einsum function, of which none exists. Using only 1 or 2 of each index works. Using a differerent path, one suggested for example by hpaulj can be used to solve the problem.
For example, I have an equation for projection matrix which works for 1 dimensional vectors:
where P is projection matrix and T is transpose.
We know that we can't simplify this fraction more (by cancelling terms) since denominator is a dot product (thus 0 dimensional scalar, number) and numerator is a matrix (column multiplied by row is a matrix).
I'm not sure how could I define function for this equation in numpy, considering that the current function that I'm using does not differentiate between these terms, multiplication is treated as it has commutative property. I'm using numpy.multiply method:
>>> import numpy as np
>>> a = np.array(a)
>>> a*a.T
array([1, 4, 9])
>>> a.T*a
array([1, 4, 9])
As you see, both of them output vectors.
I've also tried using numpy.matmul method:
>>> np.matmul(a, a.T)
14
>>> np.matmul(a.T, a)
14
which gives dot product for both of the function calls.
I also did try numpy.dot but it obviously doesn't work for numerator terms.
From my understanding, the first function call should output matrix (since column is multiplied by row) and the second function call should output a scalar in a proper case.
Am I mistaken? Is there any method that differentiates between a multiplied by a transpose and a transpose multiplied by a?
Thank you!
Note that 1-dimensional numpy arrays are not column vectors (and operations such as transposition do not make sense). If you want to obtain a column vector you should define your array as a 2-dimensional array (with the second dimension size equal to 1).
However, you don't need to define a column vector, as numpy offers functions to do what you want by manipulating an 1D array as follows
P = np.outer(a,a)/np.inner(a,a)
Stelios' answer is the best, no doubt but for completeness you can use the # operator with 2-d arrays:
a = np.array([1,4,9])[np.newaxis]
P = (a.T # a) / (a # a.T)
I have a 4x4x256 tensor and a 128x256 matrix. I need to multiply each 256-d depth-wise vector of the tensor by the matrix, such that I get a 4x4x128 tensor as a result.
Working in Numpy it's not clear to me how to do this. In their current shape it doesn't look like any variant of np.dot exists to do this. Manipulating the shapes to take advantage of broadcasting rules doesn't seem to provide any help. np.tensordot and np.einsum may be useful but looking at the documentation is going right over my head.
Is there an efficient way to do this?
You can use np.einsum to do this operation. An example with random values:
a = np.arange(4096.).reshape(4,4,256)
b = np.arange(32768.).reshape(128,256)
c = np.einsum('ijk,lk->ijl',a,b)
print(c.shape)
Here, the subscripts argument is: ijk,lk->ijl
From your requirement, i=4, j=4, k=256, l=128
The comma separates the subscripts for two operands, and the subscripts state that the multiplication should be performed over the last subscript in each tensor (the subscript k which is common to both the tensors).
The tensor subscript after the -> states that the resultant tensor should have the shape (i,j,l). Now depending on the type of operation you are performing, you might have to retain this subscript or change this subscript to jil, but the rest of the subscripts remains the same.
In python I have 2 three dimensional arrays:
T with size (n,n,n)
U with size (k,n,n)
T and U can be seen as many 2-D arrays one next to the other. I need to multiply all those matrices, ie I have to perform the following operation:
for i in range(n):
H[:,:,i] = U[:,:,i].dot(T[:,:,i]).dot(U[:,:,i].T)
As n might be very big I am wondering if this operation could be in some way speed up with numpy.
Carefully looking into the iterators and how they are involved in those dot product reductions, we could translate all of those into one np.einsum implementation like so -
H = np.einsum('ijk,jlk,mlk->imk',U,T,U)
I read something about NumPy and it's Matrix class. In the documentation the authors write, that we can create only a 2 dimensional Matrix. So I think they mean you can only write something like this:
input = numpy.matrix( ((1,2), (3,4))
Is this right?
But when I write code like this:
input = numpy.matrix( ((1,2), (3,4), (4,5)) )
it also works ...
Normally I would say ok, why not, I'm not intrrested why it works. But I must write an exam for my univerity and so I must know if I've understood it right or do they mean something else with 2D Matrix?
Thanks for your help
They both are 2D matrixes. The first one is 2x2 2D matrix and the second one is 3x2 2D matrix. It is very similar to 2D arrays in programming. The second matrix is defined as int matrix[3][2] in C for example.
Then, a 3D matrix means that it has the following definition: int 3d_array[3][2][3].
In numpy, if i try this with a 3d matrix:
>>> input = numpy.matrix((((2, 3), (4, 5)), ((6, 7), (8, 9))))
ValueError: matrix must be 2-dimensional
emre.'s answer is correct, but I would still like to address the use of numpy matrices, which might be the root of your confusion.
When in doubt about using numpy.matrix, go for ndarrays :
Matrix is actually a ndarray subclass : Everything a matrix can do, ndarray can do it (reverse is not exactly true).
Matrix overrides * and ** operators, and any operation between a Matrix and a ndarray will return a matrix, which is problematic for some algorithms.
More on the ndarray vs matrix debate on this SO post, and specifically this short answer
From Numpy documentation
matrix objects inherit from the ndarray and therefore, they have the same attributes and methods of ndarrays. There are six important differences of matrix objects, however, that may lead to unexpected results when you use matrices but expect them to act like arrays:
Matrix objects can be created using a string notation to allow Matlab-style syntax where spaces separate columns and semicolons (‘;’) separate rows.
Matrix objects are always two-dimensional. This has far-reaching implications, in that m.ravel() is still two-dimensional (with a 1 in the first dimension) and item selection returns two-dimensional objects so that sequence behavior is fundamentally different than arrays.
Matrix objects over-ride multiplication to be matrix-multiplication. Make sure you understand this for functions that you may want to receive matrices. Especially in light of the fact that asanyarray(m) returns a matrix when m is a matrix.
Matrix objects over-ride power to be matrix raised to a power. The same warning about using power inside a function that uses asanyarray(...) to get an array object holds for this fact.
The default __array_priority__ of matrix objects is 10.0, and therefore mixed operations with ndarrays always produce matrices.
Matrices have special attributes which make calculations easier. [...]