I'm trying to convert a piece of MATLAB code, and this is a line I'm struggling with:
f = 0
wlab = reshape(bsxfun(#times,cat(3,1-f,f/2,f/2),lab),[],3)
I've come up with
wlab = lab*(np.concatenate((3,1-f,f/2,f/2)))
How do I reshape it now?
Not going to do it for your code, but more as a general knowledge:
bsxfun is a function that fills a gap in MATLAB that python doesn't need to fill: broadcasting.
Broadcasting is a thing where if a matrix that is being multiplied/added/whatever similar is not the same size as the other one being used, the matrix will be repeated.
So in python, if you have a 3D matrix A and you want to multiply every 2D slice of it with a matrix B that is 2D, you dont need anything else, python will broadcast B for you, it will repeat the matrix again and again. A*B will suffice. However, in MATLAB that will raise an error Matrix dimension mismatch. To overcome that, you'd use bsxfun as bsxfun(#times,A,B) and this will broadcast (repeat) B over the 3rd dimension of A.
This means that converting bsxfun to python generally requires nothing.
MATLAB
reshape(x,[],3)
is the equivalent of numpy
np.reshape(x,(-1,3))
the [] and -1 are place holders for 'fill in the correct shape here'.
===============
I just tried the MATLAB expression is Octave - it's on a different machine, so I'll just summarize the action.
For lab=1:6 (6 elements) the bsxfun produces a (1,6,3) matrix; the reshape turns it into (6,3), i.e. just removes the first dimension. The cat produces a (1,1,3) matrix.
np.reshape(np.array([1-f,f/2,f/2])[None,None,:]*lab[None,:,None],(-1,3))
For lab with shape (n,m), the bsxfun produces a (n,m,3) matrix; the reshape would make it (n*m,3)
So for a 2d lab, the numpy needs to be
np.array([1-f,f/2,f/2])[None,None,:]*lab[:,:,None]
(In MATLAB the lab will always be 2d (or larger), so this 2nd case it closer to its action even if n is 1).
=======================
np.array([1-f,f/2,f/2])*lab[...,None]
would handle any shaped lab
If I make the Octave lab (4,2,3), the `bsxfun is also (4,2,3)
The matching numpy expression would be
In [94]: (np.array([1-f,f/2,f/2])*lab).shape
Out[94]: (4, 2, 3)
numpy adds dimensions to the start of the (3,) array to match the dimensions of lab, effectively
(np.array([1-f,f/2,f/2])[None,None,:]*lab) # for 3d lab
If f=0, then the array is [1,0,0], so this has the effect of zeroing values on the last dimension of lab. In effect, changing the 'color'.
It is equivalent to
import numpy as np
wlab = np.kron([1-f,f/2,f/2],lab.reshape(-1,1))
In Python, if you use numpy you do not need to do any broadcasting, as this is done automatically for you.
For instance, looking at the following code should make it clearer:
>>> import numpy as np
>>> a = np.array([[1, 2, 3], [3, 4, 5], [6, 7, 8], [9, 10, 100]])
>>> b = np.array([1, 2, 3])
>>>
>>> a
array([[ 1, 2, 3],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 100]])
>>> b
array([1, 2, 3])
>>>
>>> a - b
array([[ 0, 0, 0],
[ 2, 2, 2],
[ 5, 5, 5],
[ 8, 8, 97]])
>>>
Related
I have following numpy arrays:
whole = np.array(
[1, 0, 3, 0, 6]
)
sparse = np.array(
[9, 8]
)
Now I want to replace every zero in the whole array in chronological order with the items in the sparse array. In the example my desired array would look like:
merged = np.array(
[1, 9, 3, 8, 6]
)
I could write a small algorithm by myself to fix this but if someone knows a time efficient way to solve this I would be very grateful for you help!
Do you assume that sparse has the same length as there is zeros in whole ?
If so, you can do:
import numpy as np
from copy import copy
whole = np.array([1, 0, 3, 0, 6])
sparse = np.array([9, 8])
merge = copy(whole)
merge[whole == 0] = sparse
if the lengths mismatch, you have to restrict to the correct length using len(...) and slicing.
I encountered a problem in python programming.
I was manipulating the feature extraction in deep learning. I would like to add several 2D arrays into a 3D array in a for loop. I could achieve the purpose by using this easy way shown below. This method is not realistic in a large sample.
But in my situation, the data returned in the function of one sample is a 2D array (i.e. shape is (41,4)), the data itself is in the loop (i.e. 30 samples), the results I would like to obtain is a 3D array (i.e. shape is (30,41,4)).
I didn't find any related information, I really stuck here, hope someone could help me.
import numpy as np
a = np.array([[1,2,3],[4,5,6]])
b = np.array([[2,2,3],[4,5,6]])
c = np.array([[3,2,3],[4,5,6]])
print(a)
print(a.shape)
com = np.array([a,b,c])
print(com)
print(com.shape)
You can use np.stack
>>> arr = np.stack((a,b,c))
>>> arr
array([[[1, 2, 3],
[4, 5, 6]],
[[2, 2, 3],
[4, 5, 6]],
[[3, 2, 3],
[4, 5, 6]]])
>>> arr.shape
(3, 2, 3)
I'm working on some linear algebra stuff, and simply can't figure out why numpy gives the following:
The result I got from mathematica, and by hand is
Edit: If you need the matrices:
test = [[19722145, -21016468, 51417377],
[-185674670, 298847128, -428429486],
[289326728, -516012704, 691212936]]
A = [[9, 4, 1], [2, 0, 8], [-8, 8, -8]]
As noted by #PaulPanzer, you need to use np.int64 dtype arrays. NumPy uses np.int32 for your input arrays on your platform / system configuration1 and does not check for integer overflow.
However, the result of your matrix multiplication includes integers which are too large to be stored in np.int32.
Since NumPy doesn't automatically upcast the input arrays to np.int64, you need to specify np.int64 explicitly, either when you define the array or via upcasting:
import numpy as np
test = np.array([[19722145, -21016468, 51417377],
[-185674670, 298847128, -428429486],
[289326728, -516012704, 691212936]],
dtype=np.int64)
A = np.array([[9, 4, 1],
[2, 0, 8],
[-8, 8, -8]],
dtype=np.int64)
res = np.dot(test, A)
print(res)
[[ -275872647 490227596 -559748615]
[ 2354058114 -4170134568 5632538242]
[-3957788344 6687010400 -9368478392]]
1 Here's another example. There's also been some discussion on platform-specific issues.
I'm looking for something similar to ARRAYFUN in MATLAB, but for Python. What I need to do is to compute a matrix whose components are exp(j*dot([kx,ky], [x,y])), where [kx,ky] is a fixed known vector, and [x,y] is an element from a meshgrid.
What I was trying to do is to define
RX, RY = np.meshgrid(np.arange(N), np.arange(M))
R = np.dstack((RX,RY))
and then iterate over the R indices, filling a matrix with the same shape as R, in which each component would be exp(j*dot([kx,ky], [x,y])), with [x,y] being in R. This doesn't look efficient nor elegant.
Thanks for your help.
You could do what we used to do in MATLAB before they added ARRAYFUN - change the calculation so it works with arrays. That could be tricky in the days when everything in MATLAB was 2d; allowing more dimensions made it easier. numpy allows more than 2 dimensions.
Anyways, here a quick attempt:
In [497]: rx,ry=np.meshgrid(np.arange(3),np.arange(4))
In [498]: R=np.dstack((rx,ry))
In [499]: R.shape
Out[499]: (4, 3, 2)
In [500]: kx,ky=1,2
In [501]: np.einsum('i,jki->jk',[kx,ky],R)
Out[501]:
array([[0, 1, 2],
[2, 3, 4],
[4, 5, 6],
[6, 7, 8]])
There are other versions of dot, matmul and tensordot, but einsum is the one I like to use. I've worked with it enough to quickly set up a multidimensional dot.
Now just apply the 1j and exp to each element:
In [502]: np.exp(np.einsum('i,jki->jk',[kx,ky],R)*1j)
Out[502]:
array([[ 1.00000000+0.j , 0.54030231+0.84147098j,
-0.41614684+0.90929743j],
[-0.41614684+0.90929743j, -0.98999250+0.14112001j,
-0.65364362-0.7568025j ],
[-0.65364362-0.7568025j , 0.28366219-0.95892427j,
0.96017029-0.2794155j ],
[ 0.96017029-0.2794155j , 0.75390225+0.6569866j ,
-0.14550003+0.98935825j]])
numpy.square seems to give incorrect output when scipy.sparse matrices are passed to it:
import numpy as np
import scipy.sparse as S
a = np.array([np.arange(5), np.arange(5), np.arange(5), np.arange(5), np.arange(5)])
a
# array([[0, 1, 2, 3, 4],
# [0, 1, 2, 3, 4],
# [0, 1, 2, 3, 4],
# [0, 1, 2, 3, 4],
# [0, 1, 2, 3, 4]])
np.square(a)
# array([[ 0, 1, 4, 9, 16],
# [ 0, 1, 4, 9, 16],
# [ 0, 1, 4, 9, 16],
# [ 0, 1, 4, 9, 16],
# [ 0, 1, 4, 9, 16]])
b = S.lil_matrix(a)
c = np.square(b)
c
# <5x5 sparse matrix of type '<class 'numpy.int64'>'
# with 20 stored elements in Compressed Sparse Row format>
c[2,2]
# 20
# Expected output is 4, as in np.square(a) output above.
Is this a bug?
In general, passing in scipy.sparse matrices into numpy functions that take arrays ("array_like") as input, results to undefined/unintended behavior.
There is no automatic sparse -> dense cast.
Numpy does not know anything about Scipy's sparse matrices.
Sparse matrices are not "array_like" in the sense understood by Numpy.
What the numpy functions then do is to treat the sparse matrices as just some Python objects of an unknown type --- in general resulting to putting them to 1-element object arrays, and working on from there. For returning scalar results, the temporary object array is discarded and just the object contained inside it is returned, so it's easy to miss that something strange was actually done.
Object arrays have some fallbacks for performing arithmetic etc operations on their elements (unknown Python objects), including calling operator.mul of the element if * needs to be performed and so on. This then combined with the above results to the behavior you see.
Update: As pointed out by hpaulj, the reason is probably a bit more involved. np.square is able to detect np.matrix and is able to square the elements. However, it falters on sp.sparse.*matrix.
This is not a bug; this is the subtle difference between how numpy and scipy implement the __mul__ operator. By default, * for numpy.ndarray performs element-wise multiplication whereas for numpy.matrix (and consequently, for scipy.sparse.*matrix), it performs matrix multiplication (from PEP 465):
numpy provides two different types with different __mul__ methods. For
numpy.ndarray objects, * performs elementwise multiplication, and
matrix multiplication must use a function call (numpy.dot). For
numpy.matrix objects, * performs matrix multiplication, and
elementwise multiplication requires function syntax.
Internally, numpy.square uses the provided argument's __mul__ method, which is different for ndarrays and matrixes.