I have a matrix called vec with two columns, vec[:,0] and vec[:,1]. P contains two matrices, P[0,:,:] and P[1,:,:]. I want to mulitiply P[0,:,:] with the first column of vec and multiply P[1,:,:] with the second column of vec. However, the operation P#vec also gives me the matrix product of P[0,:,:] with the second column of vec and the matrix product of P[1,:,:] with the first column of vec, which slows my code.
Is it possible to directly compute the pairs column 1 to matrix 1 and column 2 to matrix 2 without the "off" products?
import numpy as np
P=np.arange(50).reshape(2, 5, 5)
vec=np.arange(10).reshape(5,2)
have=P#vec
want=np.column_stack((have[0,:,0],have[1,:,1]))
have,want
There is a very powerful function in numpy called np.einsum. It can perform all kind of tensor contractions, axis reordering and matrix multiplication. For your example you could use
res = np.einsum('nij,jn->in', P, vec)
after which res is exactly like want.
How does this work:
You give the np.einsum function both your arrays as well as a signature (that 'nij,jn->in' string) that tells the function how to multiply the arrays. In short, you want the third axis of the P tensor to be contracted with the first axis of vec. Therefore you choose the same index j in the signature string and leave it out in the part after the ->. A mere broadcast is done if indices appear on the left and right hand side of the ->, which is done here for the n and i indices.
A more complete explanation of this very powerful function with many examples of how to use it can be found at the corresponding numpy documentation.
#/matmul handles batches nicely, but the rules are that for 3d arrays, the first dimension is the batch, and dot is done on the last 2 dimensions, with the usual "last of A with the second to the last of B" pairing.
It took a bit of reading to decipher you description but it appears that you want the first of p to the batch, and last of vec to be the batch. That means vec needs to transformed to a (2,5,1) to work with the (2,5,5) p.
In [176]: P#vec.T[:,:,None]
Out[176]:
array([[[ 60],
[ 160],
[ 260],
[ 360],
[ 460]],
[[ 695],
[ 820],
[ 945],
[1070],
[1195]]])
The result is (2,5,1). We can squeeze out the the last to get (2,5), but apparently you want a (5,2)
In [179]: (P#vec.T[:,:,None])[...,0].T
Out[179]:
array([[ 60, 695],
[ 160, 820],
[ 260, 945],
[ 360, 1070],
[ 460, 1195]])
np.einsum('nij,jn->in', P, vec) does effectively the same, with the n as the batch dimension that is 'carried through' to the result, and sum-of-products on the shared j dimension.
Related
2d array, consists of 2 axes, axis=0 which represents the rows and the axis=1 represents the columns
aa = np.random.randn(10, 2) # Here is 2d array, first axis has 10 rows and second axis has 2 columns
array([[ 0.6999521 , -0.17597954],
[ 1.70622947, -0.85919459],
[-0.90019284, 0.80774052],
[-1.42953238, 0.19727917],
[-0.03416532, 0.49584749],
[-0.28981586, -0.77484498],
[-1.31129122, 0.423833 ],
[-0.43920016, -1.93541758],
[-0.06667634, 2.09925218],
[ 1.24633485, -0.04153847]])
why when I want to scatter the points I only consider the first column and the second column dimension from axis=1? do dimensions mean columns when plotting and at other times they mean axes? can you please explain more the reasons to do it like this? and if there are good references I could benefit myself on dimensions relating to this
plt.scatter(x[:,0], x[:,1]) # this also means dimensions or columns?
x[:,0], x[:,1] why not do x[0,:], x[:,1}
It can be difficult to visualize this, especially in multiple dimensions.
The parameters to the [] operator represent the dimensions. Your first dimension is the rows. The first row is array[0]. Your second dimension is the columns. The entire second column is called array[:,1] -- the ":" is a numpy notation that means "take all of this dimension". array[2,1] refers to the second column in the third row.
plt.scatter expects the x coordinate values as its first parameter, and the y coordinate values as its second parameter. plt.scatter(x[:,0], x[:,1]) means "take all of column 0" and "take all of column 1", which is the way scatter wants them.
With this randn call you make a 2d array with the specified shape. The dimensions, 10 and 2, don't represent anything - that's an abstract (10,2) array. Meaning comes from how you use it.
In [50]: aa = np.random.randn(10, 2)
In [51]: aa
Out[51]:
array([[-0.26769106, 0.09882999],
[-1.5605514 , -1.38614473],
[ 1.23312852, 0.86838848],
[ 1.2603898 , 2.19895989],
[-1.66937976, 0.79666952],
[-0.15596669, 1.47848784],
[ 1.74964902, 0.39280584],
[-1.0982447 , 0.46888408],
[ 0.84396231, -0.34809148],
[-0.83489678, -1.8093045 ]])
That's a display - with rows and columns.
Rather than pass the slices directly to scatter lets assign them to variables:
In [52]: x = aa[:,0]; y = aa[:,1]; x,y
Out[52]:
(array([-0.26769106, -1.5605514 , 1.23312852, 1.2603898 , -1.66937976,
-0.15596669, 1.74964902, -1.0982447 , 0.84396231, -0.83489678]),
array([ 0.09882999, -1.38614473, 0.86838848, 2.19895989, 0.79666952,
1.47848784, 0.39280584, 0.46888408, -0.34809148, -1.8093045 ]))
We now have two 1d arrays with shape (10,) (that's a 1 element tuple). We can then plot them with:
In [53]: plt.scatter(x,y)
I could just as well used
x = np.arange(10); y = np.random.randn(10)
to make two 1d arrays.
The dimensions of the aa array have nothing to do with the axes of a scatter plot.
I could select a 'row' of aa, but will only get a (2,) shape array. That can't be plotted against a (10,) array:
In [53]: aa[0,:]
Out[53]: array([-0.26769106, 0.09882999])
As for meaning of dimensions in sum/mean, why not experiement?
Sum all values:
In [54]: aa.sum()
Out[54]: 2.2598841819604134
sum down the columns, resulting in one value per column:
In [55]: aa.sum(axis=0)
Out[55]: array([-0.49960074, 2.75948492])
It can help to keepdims, producing a (1,2) array:
In [56]: aa.sum(axis=0, keepdims=True)
Out[56]: array([[-0.49960074, 2.75948492]])
or a (10,1) array:
In [57]: aa.sum(axis=1, keepdims=True)
Out[57]:
array([[-0.16886107],
[-2.94669614],
[ 2.101517 ],
[ 3.45934969],
[-0.87271024],
[ 1.32252115],
[ 2.14245486],
[-0.62936062],
[ 0.49587083],
[-2.64420128]])
There's some ambiguity when talking about summing along rows or columns when dealing with 2d arrays. It becomes clearer when we apply sum to 1d arrays (sum the only one), or 3d.
For example, note which dimension is missing when I do:
In [58]: np.arange(24).reshape(2,3,4).sum(axis=1).shape
Out[58]: (2, 4)
or
In [59]: np.arange(24).reshape(2,3,4).sum(axis=2)
Out[59]:
array([[ 6, 22, 38],
[54, 70, 86]])
Again - dimensions of numpy arrays are abstract things. An array can have 0, 1, 2 or more (up to 32) dimensions. Most of linear algebra deals with 2d arrays, matrices and "vectors". You can do LA with numpy, but numpy is used for much more.
edit
You could think of your aa as 10 2-element points. Then aa[:,0] are all the x coordinates. A mean with axis=0 would be the "center of mass" of those points.
In [60]: np.mean(aa, axis=0)
Out[60]: array([-0.04996007, 0.27594849])
Mean on axis=1 may not make sense, though you could calculate the norm of the points (sqrt(x^2+y^2)), or the length of the vectors represented by the points.
In [61]: np.linalg.norm(aa, axis=1)
Out[61]:
array([0.28535218, 2.08727523, 1.50821235, 2.53456249, 1.84973271,
1.48669159, 1.79320052, 1.19414978, 0.91292938, 1.99264533])
For direction of these points I'd use:
np.arctan2(aa[:,0], aa[:,1])
(or maybe switch the 0 and 1).
So I have a 3D data-set (x,y,z), and i want to sum over one of the axes (x) with a set of weights, w = w(x). The start and end index i am summing over is different for every (y,z), I have solved this by masking the 3D-array. The weights are constant with regard to the two variables i am not summing over. Both answers regarding implementation and mathematics are appreciated (is there a smart linalg. way of doing this?).
I have a 3D masked array (A) of shape (x,y,z) and a 1D array (t) of shape (x,). Is there a good way to multiply every (y,z) element in A with the corresponding number in t without expanding t to a 3D array? My current solution is using np.tensordot to make a 3D array of the same shape as A, that holds all the t-values, but it feels very unsatisfactory to spend runtime building the "new_t" array, which is essensially just y*z copies of t.
Example of current solution:
a1 = np.array([[1,2,3,4],
[5,6,7,8],
[9,10,11,12]])
a2 = np.array([[0,1,2,3],
[4,5,6,7],
[8,9,10,11]])
#note: A is a masked array, mask is a 3D array of bools
A = np.ma.masked_array([a1,a2],mask)
t = np.array([10,11])
new_t = np.tensordot(t, np.ones(A[0].shape), axes = 0)
return np.sum(A*new_t, axis=0)
In essence i want to perform t*A[:,i,j] for all i,j with the shortest possible runtime, preferably without using too many other libraries than numpy and scipy.
Another way of producing desired output (again, with far too high run time):
B = [[t*A[:,i,j] for j in range(A.shape[2])] for i in range(A.shape[1])]
return np.sum(B,axis=2)
inspired by #phipsgabler comment
arr1 = np.tensordot(A.T,t,axes=1).T
arr1
array([[ 10, 31, 52, 73],
[ 94, 115, 136, 157],
[178, 199, 220, 241]])
Thanks for good answers! Using tensordot like #alyhosny proposed worked, but replacing masked values with zeros using
A = np.ma.MaskedArray.filled(A,0)
before summing with einsum (thanks #phipsgabler) gave half the run time. Final code:
A = np.ma.MaskedArray(A,mask)
A = np.ma.MaskedArray.filled(A,0)
return np.einsum('ijk,i->jk',A,t)
I sometimes work with 1D arrays:
A = np.array([1, 2, 3, 4])
or 2D arrays (mono or stereo signals read with scipy.io.wavfile):
A = np.array([[1, 2], [3, 4], [5, 6], [7,8]])
Without having to distinguish these 2 cases with an if A.ndim == 2:..., is there a simple one-line solution to multiply this array A by an 1D array B = np.linspace(0., 1., 4)?
If A is 1D, then it's just A * B, and if A is 2D, what I mean is multiply each line of A by each element of B.
Note: this question arises naturally when working with both mono and stereo sounds read with scipy.io.wavfile.
Approach #1
We can use einsum to cover generic ndarrays -
np.einsum('i...,i->i...',A,B)
The trick that works here is the ellipsis that broadcasts for the trailing dimensions after the first axis as they are in A and keeps them in the output, while keeping the first axes for the two inputs aligned, which is the intended multiplication here. With A as 1D array, there's no broadcasting and that essentially reduces to : np.einsum('i,i->i',A,B) under the hoods.
Schematically put :
A : i x ....
B : i
out : i x ....
Hence, covers for A with any number of dimensions.
More info on the use of ellipsis from the docs :
To enable and control broadcasting, use an ellipsis. Default
NumPy-style broadcasting is done by adding an ellipsis to the left of
each term, like np.einsum('...ii->...i', a). To take the trace along
the first and last axes, you can do np.einsum('i...i', a), or to do a
matrix-matrix product with the left-most indices instead of rightmost,
you can do np.einsum('ij...,jk...->ik...', a, b).
Approach #2
Using the fact that we are trying to align the first axis of A with the only axis of 1D array B, we can simply transpose A, multiply with B and finally transpose back -
(A.T*B).T
I was trying to do matrix dot product and transpose with Numpy, and I found array can do many things matrix can do, such as dot product, point wise product, and transpose.
When I have to create a matrix, I have to create an array first.
example:
import numpy as np
array = np.ones([3,1])
matrix = np.matrix(array)
Since I can do matrix transpose and dot product in array type, I don't have to convert array into matrix to do matrix operations.
For example, the following line is valid, where A is an ndarray :
dot_product = np.dot(A.T, A )
The previous matrix operation can be expressed with matrix class variable A
dot_product = A.T * A
The operator * is exactly the same as point-wise product for ndarray. Therefore, it makes ndarray and matrix almost indistinguishable and causes confusions.
The confusion is a serious problem, as said in REP465
Writing code using numpy.matrix also works fine. But trouble begins as
soon as we try to integrate these two pieces of code together. Code
that expects an ndarray and gets a matrix, or vice-versa, may crash or
return incorrect results. Keeping track of which functions expect
which types as inputs, and return which types as outputs, and then
converting back and forth all the time, is incredibly cumbersome and
impossible to get right at any scale.
It will be very tempting if we stick to ndarray and deprecate matrix and support ndarray with matrix operation methods such as .inverse(), .hermitian(), outerproduct(), etc, in the future.
The major reason I still have to use matrix class is that it handles 1d array as 2d array, so I can transpose it.
It is very inconvenient so far how I transpose 1d array, since 1d array of size n has shape (n,) instead of (1,n). For example, if I have to do the inner product of two arrays :
A = [[1,1,1],[2,2,2].[3,3,3]]
B = [[1,2,3],[1,2,3],[1,2,3]]
np.dot(A,B) works fine, but if
B = [1,1,1]
,its transpose is still a row vector.
I have to handle this exception when the dimensions of input variable is unknown.
I hope this help some people with the same trouble, and hope to know if there is any better way to handle matrix operation like in Matlab, especially with 1d array. Thanks.
Your first example is a column vector:
In [258]: x = np.arange(3).reshape(3,1)
In [259]: x
Out[259]:
array([[0],
[1],
[2]])
In [260]: xm = np.matrix(x)
dot produces the inner product, and dimensions operate as: (1,2),(2,1)=>(1,1)
In [261]: np.dot(x.T, x)
Out[261]: array([[5]])
the matrix product does the same thing:
In [262]: xm.T * xm
Out[262]: matrix([[5]])
(The same thing with 1d arrays produces a scalar value, np.dot([0,1,2],[0,1,2]) # 5)
element multiplication of the arrays produces the outer product (so does np.outer(x, x) and np.dot(x,x.T))
In [263]: x.T * x
Out[263]:
array([[0, 0, 0],
[0, 1, 2],
[0, 2, 4]])
For ndarray, * IS element wise multiplication (the .* of MATLAB, but with broadcasting added). For element multiplication of matrix use np.multiply(xm,xm). (scipy sparse matrices have a multiply method, X.multiply(other))
You quote from the PEP that added the # operator (matmul). This, as well as np.tensordot and np.einsum can handle larger dimensional arrays, and other mixes of products. Those don't make sense with np.matrix since that's restricted to 2d.
With your 3x3 A and B
In [273]: np.dot(A,B)
Out[273]:
array([[ 3, 6, 9],
[ 6, 12, 18],
[ 9, 18, 27]])
In [274]: C=np.array([1,1,1])
In [281]: np.dot(A,np.array([1,1,1]))
Out[281]: array([3, 6, 9])
Effectively this sums each row. np.dot(A,np.array([1,1,1])[:,None]) does the same thing, but returns a (3,1) array.
np.matrix was created years ago to make numpy (actually one of its predecessors) feel more like MATLAB. A key feature is that it is restricted to 2d. That's what MATLAB was like back in the 1990s. np.matrix and MATLAB don't have 1d arrays; instead they have single column or single row matrices.
If the fact that ndarrays can be 1d (or even 0d) is a problem there are many ways of adding that 2nd dimension. I prefer the [None,:] kind of syntax, but reshape is also useful. ndmin=2, np.atleast_2d, np.expand_dims also work.
np.sum and other operations that reduced dimensions have a keepdims=True parameter to counter that. The new # gives an operator syntax for matrix multiplication. As far as I know, np.matrix class does not have any compiled code of its own.
============
The method that implements * for np.matrix uses np.dot:
def __mul__(self, other):
if isinstance(other, (N.ndarray, list, tuple)) :
# This promotes 1-D vectors to row vectors
return N.dot(self, asmatrix(other))
if isscalar(other) or not hasattr(other, '__rmul__') :
return N.dot(self, other)
return NotImplemented
I have an N*M matrix A and a N-length vector V. I want to do A + V where each element in a row i of A is summed with the element i in V. How to do that?
e.g.:
A = np.random.rand(3,2)
V = np.array([1,2,3])
A + V
ValueError: operands could not be broadcast together with shapes (3,2) (3)
I want to do the same with multiplication and division.
Solution:
V = V.reshape(-1,1)
A + V
now this works
The concept to look up here is broadcasting. It enables you to have pointwise interaction between any two matrices as long as their shapes correspond or, on the axes without correspondence, at least one of the sides is degenerate, i.e. of size 1. In your case, you need to add an axis to V. One can do this elegantly as follows
A + V[:, np.newaxis]
You need to tell numpy it has to extend the dimensions of the vector V by one, and you can do it using the special index np.newaxis. It would look like this:
import numpy as np
A = np.array([[10,20],[100,200],[1000,2000]])
V = np.array([1,2,3])
A + V[:,np.newaxis]
array([[ 11, 21],
[ 102, 202],
[1003, 2003]])
From the slicing docs:
Each newaxis object in the selection tuple serves to expand the dimensions of the resulting selection by one unit-length dimension. The added dimension is the position of the newaxis object in the selection tuple.