I have a 2D array x of shape (48, 7), and a 4D array T of shape (48, 7, 48, 7). When I multiply x * T, python broadcasts the dimensions, but not in the way I expected (actually, I don´t understand how it is broadcasting). The following loop would achieve what I want:
for i in range(48):
for j in range(7):
Tx[i, j, :, :] = x[i, j] * T[i, j, :, :]
Where Tx is an array of shape (48, 7, 48, 7). My question is, is there a way to achieve the same result using broadcasting?
Broadcasting aligns trailing dimensions. In other words, x * Tx is doing this:
for i in range(48):
for j in range(7):
Tx[:, :, i, j] = x[i, j] * T[:, :, i, j]
To get the leading dimensions to line up, add unit dimensions to x:
Tx = x[..., None, None] * T
Alternatively, you can use np.einsum to specify the dimensions explicitly:
Tx = np.einsum('ij,ij...->ij...', x, T)
I found the solution.
Python broadcasts from the rightmost dimension and works its way to the left (source).
By transposing the first two dimensions and the last two dimensions:
T = np.transpose(T, (2,3,0,1))
It will then broadcast the way I expected. After that, the resulting array can be transposed again to recover the original shape:
Tx = x*T
Tx = np.transpose(Tx, (2,3,0,1))
Related
I have a matrix A of the shape (N, N, T). Then I have a vector of V shape (N,). I want to perform the following operation A[i, j, ...] = A[i, j, ...]*V[i]/V[j]. I'm doing this with the following loop, but sure there is a way to do it with broadcast.
A = np.random.randint(0, 5, (2, 2, 3))
V = np.array([2, 3])
for i in range(2):
for j in range(2):
A[i, j, ...] *= V[i]
A[i, j, ...] /= V[j]
I've thought about doing it with element-wise multiplication and broadcast of numpy, and I try approaches like A * V[:, None, None] but always got an error.
Is there a more efficient way to do it?
Here's one way to do it -
(A*V[:,None,None])/V[:,None]
Alternatively, in two steps -
A *= V[:,None,None]
A /= V[:,None]
Leverage multi-cores with numexpr -
import numexpr as ne
ne.evaluate('A*V3D/V2D',{'V3D':V[:,None,None],'V2D':V[:,None]})
Note that you might be getting error because you might be doing edits into an int array with float results. So, either convert to float array at the start or write to a new array with the one-step approaches.
I have an issue in using python with matrix multiplication and reshape. for example, I have a column S of size (16,1) and another matrix H of size (4,4), I need to reshape the column S into (4,4) in order to multiply it with H and then reshape it again into (16,1), I did that in matlab as below:
clear all; clc; clear
H = randn(4,4,16) + 1j.*randn(4,4,16);
S = randn(16,1) + 1j.*randn(16,1);
for ij = 1 : 16
y(:,:,ij) = reshape(H(:,:,ij)*reshape(S,4,[]),[],1);
end
y = mean(y,3);
Coming to python :
import numpy as np
H = np.random.randn(4,4,16) + 1j * np.random.randn(4,4,16)
S = np.random.randn(16,) + 1j * np.random.randn(16,)
y = np.zeros((4,4,16),dtype=complex)
for ij in range(16):
y[:,:,ij] = np.reshape(h[:,:,ij]#S.reshape(4,4),16,1)
But I get an error here that we can't reshape the matrix y of size 256 into 16x1.
Does anyone have an idea about how to solve this problem?
Simply do this:
S.shape = (4,4)
for ij in range(16):
y[:,:,ij] = H[:,:,ij] # S
S.shape = -1 # equivalent to 16
np.dot operates over the last and second-to-last axis of the two operands if they have two or more axes. You can move your axes around to use this.
Keep in mind that reshape(S, 4, 4) in Matlab is likely equivalent to S.reshape(4, 4).T in Python.
So given H of shape (4, 4, 16) and S of shape (16,), you can multiply each channel of H by a reshaped S using
np.moveaxis(np.dot(np.moveaxis(H, -1, 0), S.reshape(4, 4).T), 0, -1)
The inner moveaxis call makes H into (16, 4, 4) for easy multiplication. The outer one reverses the effect.
Alternatively, you could use the fact that S will be transposed to write
np.transpose(S.reshape(4, 4), np.transpose(H))
There are two issues in your solution
1) reshape method takes a shape in the form of a single tuple argument, but not multiple arguments.
2) The shape of your y-array should be 16x1x16, not 4x4x16. In Matlab, there is no issue since it automatically reshapes y as you update it.
The correct version would be the following:
import numpy as np
H = np.random.randn(4,4,16) + 1j * np.random.randn(4,4,16)
S = np.random.randn(16,) + 1j * np.random.randn(16,)
y = np.zeros((16,1,16),dtype=complex)
for ij in range(16):
y[:,:,ij] = np.reshape(H[:,:,ij]#S.reshape((4,4)),(16,1))
I have two arrays: A such that A.shape = (N, M, K) and index such that index.shape = (N, M). As a result I want to get such array B that B[i, j] = A[i, j, index[i, j]]. How to achieve that in the simplest way?
One solution that comes to my mind is that we can do it like this:
B = A.reshape(-1, K)[np.arange(N * M), index.reshape(-1)].reshape(N, M). I wonder if there are any other solutions?
You could use 2D extended range array for the first axis that aligns with the shape of the indexing array index and hence use advanced-indexing, like so -
B = A[np.arange(N)[:,None], np.arange(M), index]
I have a tensor with the shape (5,48,15). How can I access an element along the 0th axis and still maintain 3 dimensions without needing to reshape. For example:
x.shape # this is (5,48,15)
m = x[0,:,:]
m.shape # This is (48,15)
m_new = m.reshape(1,48,15)
m_new.shape # This is now (1,48,15)
Is this possible without needing to reshape?
When you index an axis with a single integer, as with x[0, :, :], the dimensionality of the returned array drops by one.
To keep three dimensions, you can either...
insert a new axis at the same time as indexing:
>>> x[None, 0, :, :].shape
(1, 48, 15)
or use slicing:
>>> x[:1, :, :].shape
(1, 48, 15)
or use fancy indexing:
>>> x[[0], :, :].shape
(1, 48, 15)
The selection index needs to be a slice or list (or array):
m = x[[0],:,:]
m = x[:1,:,:]
m = x[0:1,:,:]
Is there an easy way to index a numpy multidimensional array along the last dimension, using an array of indices? For example, take an array a of shape (10, 10, 20). Let's assume I have an array of indices b, of shape (10, 10) so that the result would be c[i, j] = a[i, j, b[i, j]].
I've tried the following example:
a = np.ones((10, 10, 20))
b = np.tile(np.arange(10) + 10, (10, 1))
c = a[b]
However, this doesn't work because it then tries to index like a[b[i, j], b[i, j]], which is not the same as a[i, j, b[i, j]]. And so on. Is there an easy way to do this without resorting to a loop?
There are several ways to do this. Let's first generate some test data:
In [1]: a = np.random.rand(10, 10, 20)
In [2]: b = np.random.randint(20, size=(10,10)) # random integers in range 0..19
One way to solve the question would be to create two index vectors, where one is a row vector and the other a column vector of 0..9 using meshgrid:
In [3]: i1, i0 = np.meshgrid(range(10), range(10), sparse=True)
In [4]: c = a[i0, i1, b]
This works because i0, i1 and b will all be broadcasted to 10x10 matrices. Quick test for correctness:
In [5]: all(c[i, j] == a[i, j, b[i, j]] for i in range(10) for j in range(10))
Out[5]: True
Another way would be to use choose and rollaxis:
# choose needs a sequence of length 20, so move last axis to front
In [22]: aa = np.rollaxis(a, -1)
In [23]: c = np.choose(b, aa)
In [24]: all(c[i, j] == a[i, j, b[i, j]] for i in range(10) for j in range(10))
Out[24]: True