I'm trying to populate an array in python more efficiently. I have a 5x3 matrix A that I am transforming into a 3x3 matrix (Z) by calculating z11, z12, ..., z33 independently. The code below works, but it's clunky and I'm hoping to automate this into a loop so that it will take an A matrix of any size (n x m) and transform it into a Z matrix of size (m x m). If someone could help me out I would greatly appreciate it!
import numpy as np
A = np.array([[1,0,0],
[0,1,0],
[0,1,1],
[0,0,-1],
[0,0,1]])
A1=A[:,0]
A2=A[:,1]
A3=A[:,2]
C = np.array([-2,-2, -9,-6,-4])
X = np.array([-4,-4,-8])
z11 = (sum(A1*A1))*(C[0]/X[0])
z12 = (sum(A1*A2))*(C[0]/X[1])
z13 = (sum(A1*A3))*(C[0]/X[2])
z21 = (sum(A2*A1))*(C[1]/X[0])
z22 = (sum(A2*A2))*(C[1]/X[1])
z23 = (sum(A2*A3))*(C[1]/X[2])
z31 = (sum(A3*A1))*(C[2]/X[0])
z32 = (sum(A3*A2))*(C[2]/X[1])
z33 = (sum(A3*A3))*(C[2]/X[2])
Z = np.array([[z11,z12,z13],
[z21,z22,z23],
[z31,z32,z33]])
We can use the broadcasting to achieve the same. First let's increase A by one dimension using A[:, None] and then multiply it with A. Since shape of A[:, None] is (3, 1, 5) and shape of A is (3, 5), numpy first repeats(intuitively) the array corresponding to dimension where both array don't match and then does the multiplication. This way each column of A gets multiplied with every other column(to makes sure that columns are multiplied, I have used transpose) Then we can take sum along the last axis and multiply with C[:, None] to achieve the desired output.
Use:
m = A.shape[1]
B = A[:, None].T * A.T
Z = np.sum(B, axis = -1).astype(float)*C[:m, None]/X
Output:
>>> Z
array([[0.5 , 0. , 0. ],
[0. , 1. , 0.25 ],
[0. , 2.25 , 3.375]])
Related
Given two arrays, a and b, with shapes; (3, 3) and (1000,). How do I multiply them to get an array with shape (3, 3, 1000)?
a = np.array([[1,2,3],[4,5,6],[7,8,9]])
b = np.linspace(1, 1000, 1000)
c = a * b # does not work
c = np.outer(a, b) # does not work
c = np.outer(a, b[None,] # nope
I have tried a lot of things, too many to remember them all.
I have also googled (and searched on SO) but to no avail.
IIUC, use numpy.einsum:
c = np.einsum("ij,k->ijk", a, b)
Output:
c.shape
# (3, 3, 1000)
You can do it with multiplication by reshaping your arrays:
M,N = a.shape
B = b.size
c = a.reshape(M,N,1) * b.reshape(1,1,B)
print(c.shape)
print(c[:,:,0])
print(c[:,:,B-1])
Output:
% python3 script.py
(3, 3, 1000)
[[1. 2. 3.]
[4. 5. 6.]
[7. 8. 9.]]
[[1000. 2000. 3000.]
[4000. 5000. 6000.]
[7000. 8000. 9000.]]
You need to understand the broadcasting rules. The bottomline is:
both arrays need to have the same number of axes and
the sizes along each axis must be the same or one of them must be 1.
You can multiply and array with shape (3,3) only by another with shape (3,3), (3,1) or (1,3). There are other broadcasting rules. Read them.
Your shapes are (3,3) and (1000,). As you said, you need the final shape to be 3-dimensional. The 3 needs to match an axis with length 1. Same with the 1000. So you can add axes to each to end up with shapes (3, 3, 1) and (1, 1, 1000):
c = a[:, :, np.newaxis]* b[np.newaxis,np.newaxis]
I tried understanding numpy broadcasting with 3d arrays but I think the OP there is asking something slightly different.
I have a 3D numpy array like so -
IQ = np.array([
[[1,2],
[3,4]],
[[5,6],
[7,8]]
], dtype = 'float64')
The shape of this array is (2,2,2). I want to apply a function to each 1x2 array in this 3D matrix like so -
def func(IQ):
I = IQ[0]
Q = IQ[1]
amp = np.power((np.power(I,2) + np.power(Q, 2)),1/2)
phase = math.atan(Q/I)
return [amp, phase]
As you can see, I want to apply my function to each 1x2 array and replace it with the return value of my function. The output is a 3D array with the same dimensions. Is there a way to broadcast this function to each 1x2 array in my original 3D array? Currently I am using loops which becomes very slow as the 3D array increases in dimensions.
Currently I am doing this -
#IQ is defined from above
for i in range(IQ.shape[0]):
for j in range(IQ.shape[1]):
I = IQ[i,j,0]
Q = IQ[i,j,1]
amp = np.power((np.power(I,2) + np.power(Q, 2)),1/2)
phase = math.atan(Q/I)
IQ[i,j,0] = amp
IQ[i,j,1] = phase
And the returned 3D array is -
[[[ 2.23606798 1.10714872]
[ 5. 0.92729522]]
[[ 7.81024968 0.87605805]
[10.63014581 0.85196633]]]
One way is to slice the arrays to extract the I and Q values, perform the computations using normal broadcasting, and then stick the values back together:
>>> Is, Qs = IQ[...,0], IQ[...,1]
>>> np.stack(((Is**2 + Qs**2) ** 0.5, np.arctan2(Qs, Is)), axis=-1)
array([[[ 2.23606798, 1.10714872],
[ 5. , 0.92729522]],
[[ 7.81024968, 0.87605805],
[10.63014581, 0.85196633]]])
It can be done using arrays:
# sort of sum of squares along axis 2, ie (IQ[..., 0]**2 + IQ[..., 1]**2 + ...)**0.5
amp = np.sqrt(np.square(IQ).sum(axis=2))
amp
>>> array([[ 2.23606798, 5. ],
[ 7.81024968, 10.63014581]])
# and phase is arctan for each component in each matrix
phase = np.arctan2(IQ[..., 1], IQ[..., 0])
phase
>>> array([[1.10714872, 0.92729522],
[0.87605805, 0.85196633]])
# then combine the arrays to 3d
np.stack([amp, phase], axis=2)
>>> array([[[ 2.23606798, 1.10714872],
[ 5. , 0.92729522]],
[[ 7.81024968, 0.87605805],
[10.63014581, 0.85196633]]])
I = IQ[..., 0]
Q = IQ[..., 1]
amp = np.linalg.norm(IQ, axis= 2)
phase = np.arctan(Q/I)
IQ[..., 0] = amp
IQ[..., 1] = phase
IQ
>> [[[ 2.23606798, 1.10714872],
[ 5. , 0.92729522]],
[[ 7.81024968, 0.87605805],
[10.63014581, 0.85196633]]]
I was wondering if there is a more straight forward, more efficient way of generating a distance matrix given the H x W of the matrix, and the starting index location.
For simplicity lets take a 3x3 matrix where the starting point is (0,0). Thus, the distance matrix to be generated is:
[[ 0. 1. 2. ]
[ 1. 1.41421356 2.23606798]
[ 2. 2.23606798 2.82842712]]
Index (0,1) is 1 distance away, while index (2,2) is 2.828 distance away.
The code I have so far is below:
def get_distances(start, height, width):
matrix = np.zeros((height, width), dtype=np.float16)
indexes = [(y, x) for y, row in enumerate(matrix) for x, val in enumerate(row)]
to_points = np.array(indexes)
start_point = np.array(start)
distances = np.linalg.norm(to_points - start_point, ord=2, axis=1.)
return distances.reshape((height, width))
height = 3
width = 3
start = [0,0]
distance_matrix = get_distances(start, height, width)
This is pretty efficient already, I think. But numpy always surprise me with some tricks that I usually never think of, so I was wondering if there exist one in this scenario. Thanks
You can use hypot() and broadcast:
import numpy as np
x = np.arange(3)
np.hypot(x[None, :], x[:, None])
or the outer method:
np.hypot.outer(x, x)
the result:
array([[ 0. , 1. , 2. ],
[ 1. , 1.41421356, 2.23606798],
[ 2. , 2.23606798, 2.82842712]])
to calculate the distance between every point on a grid to a fixed point (x, y):
x, y = np.ogrid[0:3, 0:3]
np.hypot(x - 2, y - 2)
I'm having some array operation issues. Here's an example:
A = np.ones((5,2))
B = np.ones((5,2)) * 2
X = np.zeros((5,1))
C = A[:,0] + B[:,0]
D = C + X
The shapes I'm getting are:
shape(A[:,0]) = (5,)
shape(B[:,0]) = (5,)
shape(X) = (5,1)
shape(C) = (5,)
shape(D) = (5,5)
When I extract a column from an array, the output is from shape (5,), not (5,1). Is there any way to correct that without having to reshape arrays all the time?
When I add D = C + X, the result is an (5,5) array, but should be (5,1).
Solution 1
D = X + C.reshape(shape(X))
shape(D)
#(5, 1)
print(D)
#[[ 3.]
# [ 3.]
# [ 3.]
# [ 3.]
# [ 3.]]
Solution 2 (better) numpy-convert-row-vector-to-column-vector
C = A[:,0:1] + B[:,0:1]
Why,
C and X have different shapes, and you sum row with number, geting a matrix with shape (5,5)
print(C)
#[ 3. 3. 3. 3. 3.]
print(X)
#[[ 0.]
# [ 0.]
# [ 0.]
# [ 0.]
# [ 0.]]
When broadcasting an array like C with (5,) with a 2d array, numpy adds dimensions at the start as needed, (1,5). So a (1,5) + (5,1) => (5,5).
To get a (5,1) result, you need, in one way or other, make C a (5,1) array.
C[:,None] + X # None or np.newaxis is an easy way
C.reshape(5,1) + X # equivalent
or index A with a list or slice
C = A[:,[0]] + B[:,[0]]
A[:,0] removes a dimension, producing a (5,) array.
Note, MATLAB adds the default dimensions to the end; numpy because it has a default C order, does so at the start. Adding dimensions like that requires minimal change, just changing the shape.
Functions like np.sum have a keepdimensions parameter to avoid this sort of dimension reduction.
I have two arrays:
a = [[a11,a12],
[a21,a22]]
b = [[b11,b12],
[b21,b22]]
What I would like to do is build up a matrix as follows:
xx = np.mean(a[:,0]*b[:,0])
xy = np.mean(a[:,0]*b[:,1])
yx = np.mean(a[:,1]*b[:,0])
yy = np.mean(a[:,1]*b[:,1])
and return an array c such that
c = [[xx,xy],
yx,yy]]
Is there a nice pythonic way to do this in numpy? Because at the moment I have done it by hand, exactly as above, so the dimensions of the output array are coded in by hand, rather than determined as according to the size of the input arrays a and b.
Is there an error in your third element? If, as seems reasonable, you want yx = np.mean(a[:,1]*b[:,0]) instead of yx = np.mean(b[:,1]*a[:,0]), then you can try the following:
a = np.random.rand(2, 2)
b = np.random.rand(2, 2)
>>> c
array([[ 0.26951488, 0.19019219],
[ 0.31008754, 0.1793523 ]])
>>> np.mean(a.T[:, None, :]*b.T, axis=-1)
array([[ 0.26951488, 0.19019219],
[ 0.31008754, 0.1793523 ]])
It will actually be faster to avoid the intermediate array and express your result as a matrix multiplication:
>>> np.dot(a.T, b) / a.shape[0]
array([[ 0.26951488, 0.19019219],
[ 0.31008754, 0.1793523 ]])