I try to multiply 2 matrix x,y with shape (41) and (41,6)
as it is supposed to broadcast the single matrix to every arrow in the multi-dimensions
I want to do it as :
x*y
but i get this error
ValueError: operands could not be broadcast together with shapes (41,6) (41,)
Is there anything I miss here to make that possible ?
Broadcasting involves 2 steps
give all arrays the same number of dimensions
expand the 1 dimensions to match the other arrays
With your inputs
(41,6) (41,)
one is 2d, the other 1d; broadcasting can change the 1d to (1, 41), but it does not automatically expand in the other direction (41,1).
(41,6) (1,41)
Neither (41,41) or (6,41) matches the other.
So you need to change your y to (41,1) or the x to (6,41)
x.T*y
x*y[:,None]
I'm assuming, of course, that you want element by element multiplication, not the np.dot matrix product.
You can try out this, it will works!
>>> import numpy as np
>>> x = np.array([[1, 2], [1, 2], [1, 2]])
>>> y = np.array([1, 2, 3])
>>> np.dot(y,x)
array([ 6, 12])
Not exactly sure, what you are trying to achieve. Maybe you could give an example of your input and your expected output. One possibility is:
import numpy as np
x = np.array([[1, 2], [1, 2], [1, 2]])
y = np.array([1, 2, 3])
res = x * np.transpose(np.array([y,]*2))
This will multiply each column of x with y, so the result of the above example is:
array([[1, 2],
[2, 4],
[3, 6]])
The multiplication of a ND array (say A) with a 1D one (B) is performed on the last axis by default, which means that the multiplication A * B is only valid if
A.shape[-1] == len(B)
A manipulation on A and B is needed to multiply A with B on another axis than -1:
Method 1: swapaxes
Swap the axes of A so that the axis to multiply with B appear on last postion
C = (A.swapaxes(axis, -1) * B).swapaxes(axis, -1)
example
A = np.arange(2 * 3 * 4).reshape((2, 3, 4))
B = np.array([0., 1., 2.])
print(A)
print(B)
pormpts :
(A)
[[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
[[12 13 14 15]
[16 17 18 19]
[20 21 22 23]]]
(B)
[0. 1. 2.]
A * B returns :
ValueError: operands could not be broadcast together with shapes (2,3,4) (3,)
now multiply A with B on axis 1
axis = 1
C = (A.swapaxes(axis, -1) * B).swapaxes(axis, -1)
returns C :
array([[[ 0., 0., 0., 0.],
[ 4., 5., 6., 7.],
[16., 18., 20., 22.]],
[[ 0., 0., 0., 0.],
[16., 17., 18., 19.],
[40., 42., 44., 46.]]])
Note that first raws of A have been multiplied by 0
last raws have been multiplied by 2
Method 2: reshape B
make B have the same number of dimensions than A, place the items of B on the dimension to be multiplied with A
A * B.reshape((1, len(B), 1))
or equivalently using the convenient 'numpy.newaxis' syntax :
A * B[np.newaxis, :, np.newaxis]
Depends on what you're expecting. One simple solution would be:
y*x
That should give you a matrix of dimensions (1,6).
If you wish to multiply X of dimension (n) to Y of dimension(n,m), you may consider the answers from this post
Tips can be found in the Wikipedia as well:
In Python with the numpy numerical library or the sympy symbolic library, multiplication of array objects as a1*a2 produces the Hadamard product, but with otherwise matrix objects m1*m2 will produce a matrix product.
Simply speaking, slice it up to arrays and perform x*y, or use other routes to fit the requirement.
So, if x has shape (41,6) and y (41,), I'd use np.expand_dims() to add an empty, second dimension (index 1) to y, i.e.,
x * np.expand_dims(y, 1)
This will automatically yield a result with shape (41,6).
Related
I have to input matrices of shape
m1: (n,3)
m2: (n,3)
I want to multiply each row (each n of size 3) with its correspondence of the other matrix, such that i get a (3,3) matrix for each row.
When im trying to just use e.g. m1[0]#m2.T[0] the operation doesnt work, as m[0] delivers a (3,) list instead of a (3,1) matrix, on which i could use matrix operations.
Is there a relatively easy or elegant way to get the desired (3,1) matrix for the matrix multiplication?
By default, numpy gets rid of the singleton dimension, as you have noticed.
You can use np.newaxis (or equivalently None. That is an implementation detail, but also works in pytorch) for the second axis to tell numpy to "invent" a new one.
import numpy as np
a = np.ones((3,3))
a[1].shape # this is (3,)
a[1,:].shape # this is (3,)
a[1][...,np.newaxis].shape # this is (3,1)
However, you can also use dot or outer directly:
>>> a = np.eye(3)
>>> np.outer(a[1], a[1])
array([[0., 0., 0.],
[0., 1., 0.],
[0., 0., 0.]])
>>> np.dot(a[1], a[1])
1.0
Generally, I would recommend using np.einsum for most matrix operations as it very elegant.
To obtain a the row-wise outer product of the vectors contained in m1 and m2 of shape (n, 3) you could do the following:
import numpy as np
m1 = np.array([1, 2, 3]).reshape(1, 3)
m2 = np.array([1, 2, 3]).reshape(1, 3)
result = np.einsum("ni, nj -> nij", m1, m2)
print(result)
>>>array([[[1, 2, 3],
[2, 4, 6],
[3, 6, 9]]])
I have a list of arrays and I want to rank the numbers against the similar positioned numbers in the other arrays in the list.
x = [[12,7,3],
[4 ,5,6],
[7 ,8,9]]
I tried the following and it ranked each numbers against all the numbers and also
the smallest number is ranked 1
scipy.stats.rankdata(x)
array([ 9.,5.5,1.,2., 3.,4.,5.5,7.,8. ])
I want to rank with largest number ranked 1 and each number only ranked against the number that is in the same position in each array of the list.
This is the output that I need.
[[1,2,3]
[3,3,2],
[2,1,1]]
You can also use magic numpy.argsort.
import numpy as np
x = np.array([[12,7,3],
[4 ,5,6],
[7 ,8,9]])
y = x.shape[0] - np.argsort(np.argsort(x, axis = 0), axis = 0)
Output:
In [111]: y
Out[111]:
array([[1, 2, 3],
[3, 3, 2],
[2, 1, 1]])
You have to specify the axis along which to rank. By default it gives the rank according to the ascending order. You can use your knowledge of the shape of your data to turn it into what you want. Example with your data
x = [[12,7,3],
[4 ,5,6],
[7 ,8,9]]
3-scipy.stats.mstats.rankdata(x,axis=0)+1
# you will get
array([[ 1., 2., 3.],
[ 3., 3., 2.],
[ 2., 1., 1.]])
I'm trying to do some basic classification of numpy arrays...
I want to compare a 2d array against a 3d array, along the 3rd dimension, and make a classification based on the corresponding z-axis values.
so given 3 arrays that are stacked into a 3d array:
import numpy as np
a1 = np.array([[1,1,1],[1,1,1],[1,1,1]])
a2 = np.array([[3,3,3],[3,3,3],[3,3,3]])
a3 = np.array([[5,5,5],[5,5,5],[5,5,5]])
a3d = dstack((a1,a2,a3))
and another 2d array
a2d = np.array([[1,2,4],[5,5,2],[2,3,3]])
I want to be able to compare a2d against a3d, and return a 2d array of which level of a3d is closest. (or I suppose any custom function that can compare each value along the z-axis, and return a value base on that comparison.)
EDIT
I modified my arrays to more closely match my data. a1 would be the minimum values, a2 the average values, and a3 the maximum values. So I want to output if each a2d value is closer to a1 (classed "1") a2 (classed "2") or a3 (classed "3"). I'm doing as a 3d array because in the real data, it won't be a simple 3-array choice, but for SO purposes, it helps to keep it simple. We can assume that in the case of a tie, we'll take the lower, so 2 would be classed as level "1", 4 as level "2".
You can use the following list comprehension :
>>> [sum(sum(abs(i-j)) for i,j in z) for z in [zip(i,a2d) for i in a3d]]
[30.0, 22.5, 30.0]
In preceding code i create the following list with zip,that is the zip of each sub array of your 3d list then all you need is calculate the sum of the elemets of subtract of those pairs then sum of them again :
>>> [zip(i,a2d) for i in a3d]
[[(array([ 1., 3., 1.]), array([1, 2, 1])), (array([ 2., 2., 1.]), array([5, 5, 4])), (array([ 3., 1., 1.]), array([9, 8, 8]))], [(array([ 4., 6., 4.]), array([1, 2, 1])), (array([ 5. , 6.5, 4. ]), array([5, 5, 4])), (array([ 6., 4., 4.]), array([9, 8, 8]))], [(array([ 7., 9., 7.]), array([1, 2, 1])), (array([ 8., 8., 7.]), array([5, 5, 4])), (array([ 9., 7., 7.]), array([9, 8, 8]))]]
then for all of your sub arrays you'll have the following list:
[30.0, 22.5, 30.0]
that for each sub-list show a the level of difference with 2d array!and then you can get the relative sub-array from a3d like following :
>>> a3d[l.index(min(l))]
array([[ 4. , 6. , 4. ],
[ 5. , 6.5, 4. ],
[ 6. , 4. , 4. ]])
Also you can put it in a function:
>>> def find_nearest(sub,main):
... l=[sum(sum(abs(i-j)) for i,j in z) for z in [zip(i,sub) for i in main]]
... return main[l.index(min(l))]
...
>>> find_nearest(a2d,a3d)
array([[ 4. , 6. , 4. ],
[ 5. , 6.5, 4. ],
[ 6. , 4. , 4. ]])
You might consider a different approach using numpy.vectorize which lets you efficiently apply a python function to each element of your array.
In this case, your python function could just classify each pixel with whatever breaks you define:
import numpy as np
a2d = np.array([[1,2,4],[5,5,2],[2,3,3]])
def classify(x):
if x >= 4:
return 3
elif x >= 2:
return 2
elif x > 0:
return 1
else:
return 0
vclassify = np.vectorize(classify)
result = vclassify(a2d)
Thanks to #perrygeo and #Kasra - they got me thinking in a good direction.
Since I want a classification of the closest 3d array's z value, I couldn't do simple math - I needed the (z)index of the closest value.
I did it by enumerating both axes of the 2d array, and doing a proximity compare against the corresponding (z)index of the 3d array.
There might be a way to do this without iterating the 2d array, but at least I'm avoiding iterating the 3d.
import numpy as np
a1 = np.array([[1,1,1],[1,1,1],[1,1,1]])
a2 = np.array([[3,3,3],[3,3,3],[3,3,3]])
a3 = np.array([[5,5,5],[5,5,5],[5,5,5]])
a3d = np.dstack((a1,a2,a3))
a2d = np.array([[1,2,4],[5,5,2],[2,3,3]])
classOut = np.empty_like(a2d)
def find_nearest_idx(array,value):
idx = (np.abs(array-value)).argmin()
return idx
# enumerate to get indices
for i,a in enumerate(a2d):
for ii,v in enumerate(a):
valStack = a3d[i,ii]
nearest = find_nearest_idx(valStack,v)
classOut[i,ii] = nearest
print classOut
which gets me
[[0 0 1]
[2 2 0]
[0 1 1]]
This tells me that (for example) a2d[0,0] is closest to the 0-index of a3d[0,0], which in my case means it is closest to the min value for that 2d position. a2d[1,1] is closest to the 2-index, which in my case means closer to the max value for that 2d position.
I've a little issue while working on same big data. But for now, let's assume I've got an NumPy array filled with zeros
>>> x = np.zeros((3,3))
>>> x
array([[ 0., 0., 0.],
[ 0., 0., 0.],
[ 0., 0., 0.]])
Now I want to change some of these zeros with specific values. I've given the index of the cells I want to change.
>>> y = np.array([[0,0],[1,1],[2,2]])
>>> y
array([[0, 0],
[1, 1],
[2, 2]])
And I've got an array with the desired (for now random) numbers, as follow
>>> z = np.array(np.random.rand(3))
>>> z
array([ 0.04988558, 0.87512891, 0.4288157 ])
So now I thought I can do the following:
>>> x[y] = z
But than it's filling the whole array like this
>>> x
array([[ 0.04988558, 0.87512891, 0.4288157 ],
[ 0.04988558, 0.87512891, 0.4288157 ],
[ 0.04988558, 0.87512891, 0.4288157 ]])
But I was hoping to get
>>> x
array([[ 0.04988558, 0, 0 ],
[ 0, 0.87512891, 0 ],
[ 0, 0, 0.4288157 ]])
EDIT
Now I've used a diagonal index, but what in the case my index is not just diagonal. I was hoping following works:
>>> y = np.array([[0,1],[1,2],[2,0]])
>>> x[y] = z
>>> x
>>> x
array([[ 0, 0.04988558, 0 ],
[ 0, 0, 0.87512891 ],
0.4288157, 0, 0 ]])
But it's filling whole array just like above
Array indexing works a bit differently on multidimensional arrays
If you have a vector, you can access the first three elements by using
x[np.array([0,1,2])]
but when you're using this on a matrix, it will return the first few rows. Upon first sight, using
x[np.array([0,0],[1,1],[2,2]])]
sounds reasonable. However, NumPy array indexing works differently: It still treats all those indices in a 1D fashion, but returns the values from the vector in the same shape as your index vector.
To properly access 2D matrices you have to split both components into two separate arrays:
x[np.array([0,1,2]), np.array([0,1,2])]
This will fetch all elements on the main diagonal of your matrix. Assignments using this method is possible, too:
x[np.array([0,1,2]), np.array([0,1,2])] = 1
So to access the elements you've mentioned in your edit, you have to do the following:
x[np.array([0,1,2]), np.array([1,2,0])]
So numpy has some convenience functions for combining several arrays into one, e.g. hstack and vstack. I'm wondering if there's something similar but for stacking the component arrays diagonally?
Say I have N arrays of shape (n_i, m_i), and I want to combine them into a single array of size (sum_{1,N}n_i, sum_{1,N}m_i) such that the component arrays form blocks on the diagonal of the result array.
And yes, I know how to solve it manually, e.g. with the approach described in How to "embed" a small numpy array into a predefined block of a large numpy array? . Just wondering if there's an easier way.
Ah, How can I transform blocks into a blockdiagonal matrix (NumPy) mentions that scipy.linalg.block_diag() is the solution, except that the version of scipy installed on my workstation is so old it doesn't have it. Any other ideas?
It does seem block_diag does exactly what you want. So if for some reason you can't update scipy, then here is the source from v0.8.0 if you wish to simply define it!
import numpy as np
def block_diag(*arrs):
"""Create a block diagonal matrix from the provided arrays.
Given the inputs `A`, `B` and `C`, the output will have these
arrays arranged on the diagonal::
[[A, 0, 0],
[0, B, 0],
[0, 0, C]]
If all the input arrays are square, the output is known as a
block diagonal matrix.
Parameters
----------
A, B, C, ... : array-like, up to 2D
Input arrays. A 1D array or array-like sequence with length n is
treated as a 2D array with shape (1,n).
Returns
-------
D : ndarray
Array with `A`, `B`, `C`, ... on the diagonal. `D` has the
same dtype as `A`.
References
----------
.. [1] Wikipedia, "Block matrix",
http://en.wikipedia.org/wiki/Block_diagonal_matrix
Examples
--------
>>> A = [[1, 0],
... [0, 1]]
>>> B = [[3, 4, 5],
... [6, 7, 8]]
>>> C = [[7]]
>>> print(block_diag(A, B, C))
[[1 0 0 0 0 0]
[0 1 0 0 0 0]
[0 0 3 4 5 0]
[0 0 6 7 8 0]
[0 0 0 0 0 7]]
>>> block_diag(1.0, [2, 3], [[4, 5], [6, 7]])
array([[ 1., 0., 0., 0., 0.],
[ 0., 2., 3., 0., 0.],
[ 0., 0., 0., 4., 5.],
[ 0., 0., 0., 6., 7.]])
"""
if arrs == ():
arrs = ([],)
arrs = [np.atleast_2d(a) for a in arrs]
bad_args = [k for k in range(len(arrs)) if arrs[k].ndim > 2]
if bad_args:
raise ValueError("arguments in the following positions have dimension "
"greater than 2: %s" % bad_args)
shapes = np.array([a.shape for a in arrs])
out = np.zeros(np.sum(shapes, axis=0), dtype=arrs[0].dtype)
r, c = 0, 0
for i, (rr, cc) in enumerate(shapes):
out[r:r + rr, c:c + cc] = arrs[i]
r += rr
c += cc
return out