I have a vector with the following values: dv = array([0., 0., 1.]).
How do I diagonalize this vector into a 3D array with each element in the vector have its own diagonal:
array([[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[1., 0., 0.],
[0., 1., 0.],
[0., 0., 1.]]])
So far i have tried:
import numpy as np
a = np.zeros((3,3,3))
di = np.diag_indices(3,3)
a[di] = dv
This is almost correct, but it does not change all of the elements on the diagonal.
Try this -
a = np.zeros((3,3,3))
dv = np.array([0, 0, 1])
i,j = np.diag_indices(3) #diagonal indices for a 2D matrix (3,3)
a[:,i,j] = dv[:,None]
a
array([[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[1., 0., 0.],
[0., 1., 0.],
[0., 0., 1.]]])
You basically get 2D matrix diagonals and use those to get a view of the 3 diagonals from the original matrix. You then repeat the diagonal values you want to update to the same shape by broadcasting and map it to the diagonals.
The above approach is based on fetching a view from the original numpy array and then using assignment to update it. If you are not trying to do an assignment task, you can simply use arr.diagonal with axis1 and axis2 parameters to get a copy of the diagonal values for those 2 axes. Note, the shape of those axes must be equal (square matrix)
a = np.arange(0,2*3*3).reshape(2,3,3)
# array([[[ 0, 1, 2],
# [ 3, 4, 5],
# [ 6, 7, 8]],
# [[ 9, 10, 11],
# [12, 13, 14],
# [15, 16, 17]]])
a.diagonal(axis1=1, axis2=2)
# array([[ 0, 4, 8],
# [ 9, 13, 17]])
Related
Given the below tensor that has vectors of all zeros and vectors with ones and zeros:
tensor([[0., 0., 0., 0.],
[0., 1., 1., 0.],
[0., 0., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 0.],
[0., 0., 1., 0.],
[1., 0., 0., 1.],
[0., 0., 0., 0.],...])
How can I have an array of indices of the vectors with ones and zeros so the output is like this:
indices = tensor([ 1, 3, 5, 6,...])
Update
A way to do it is:
indices = torch.unique(torch.nonzero(y>0,as_tuple=True)[0])
But I'm not sure if there's a better way to do it.
An alternative way is to use torch.Tensor.any coupled with torch.Tensor.nonzero:
>>> x.any(1).nonzero()[:,0]
tensor([1, 3, 5, 6])
Otherwise, since the tensor contains only positive value, you can sum the columns and mask:
>>> x.sum(1).nonzero()[:,0]
tensor([1, 3, 5, 6])
Suppose i have a 2d index Array of shape [B,1,N,2] i.e N points holding indexes on a target tensor of size [B,1,H,W].
what is the best way to assign value to all the indices in the tensor?
for example:
for b in batchsize:
for i in N:
target[b,0,ind[i,0],ind[i,1]] = 1
but not in loop form
thanks
If we look at this setup, you have a tensor target shaped (b, 1, h, w) and a tensor containing indices ind, shaped (b, 1, N, 2). You want to assign 1 to the N points per batch given by the two coordinates in ind.
The way I see it you could use torch.scatter_. We will stick with a 3D tensor since axis=1 is unused. Given a 3D tensor and argument value=1 and dim=1, .scatter_ operates on the input tensor as so:
input[i][index[i][j]] = 1
This does not exactly fit your setting since what would wish for is rather
input[i][index1[i][j]][index2[i][j]] = 1
In order to use scatter you could flatten target and unfold the values in ind accordingly.
Let's take an example:
>>> target = torch.zeros(2, 1, 10, 10)
>>> ind = torch.tensor([[[[0, 0], [1, 1], [2, 2]]],
[[[1, 2], [3, 4], [7, 8]]]])
tensor([[[[0, 0],
[1, 1],
[2, 2]]],
[[[1, 2],
[3, 0],
[4, 2]]]])
We will start by splitting ind into xs and ys coordinates:
>>> x, y = ind[..., 0], ind[..., 1]
Unfold and reshape them:
>>> z = x*target.size(-1) + y
tensor([[[ 0, 4, 8]],
[[ 5, 9, 14]]])
Flatten target:
>>> t = target.flatten(2)
tensor([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
Then scatter the 1s:
>>> t.scatter_(dim=2, index=z, value=1)
tensor([[[1., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0.]],
[[0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 1.]]])
And finally reshape back to the original shape:
>>> t.reshape_as(target)
tensor([[[[1., 0., 0.],
[0., 1., 0.],
[0., 0., 1.],
[0., 0., 0.],
[0., 0., 0.]]],
[[[0., 0., 0.],
[0., 0., 1.],
[0., 0., 0.],
[1., 0., 0.],
[0., 0., 1.]]]])
In summary:
>>> x, y = ind[..., 0], ind[..., 1]
>>> z = x*target.size(-1) + y
>>> target.flatten(2).scatter_(dim=2, index=z, value=1).reshape_as(target)
This last line will mutate target.
It's a bit hard to tell what you mean by "a 2D array of shape [B,1,N,2]" (that looks like a 4D array in commonly used notation). But it looks like across all elements in the batch, you want to assign the same values on dimensions 2 and 3 to each element indexed by N. You can use list indexing notation:
# assuming ind is a list of tuples of (dim2 index, dim3 index)
# unzip ind such that each resulting list contains indices along a single dimension
dim2_indices = [item[0] for item in ind]
dim3_indices = [item[1] for item in ind]
target[:,0,dim2,dim3] = 1
I have a piece of numpy code that I know works. I know this because I have tested it in my generic case successfully. However, I arrived at the solution after two hours of back and forth referencing the docs and trial and error. I can't grasp how I would know to do this intuitively.
The setup:
a = np.zeros((5,5,3))
The goal: Set to 1 indices 0,1 of axis 1, 0,1 of axis 2, all of axis 3 and indices 3,4 of axis 1, 3,4 of axis 2, all of axis 3
Clearer goal: Set block 1 and 2's first two rows to 1 and block 3 and 4's last two rows to 1
The result:
ax1 =np.array([np.array([0,1]),np.array([3,4])])
ax1 =np.array([x[:,np.newaxis] for x in ax1])
ax2 = np.array([[[0,1]],[[3,4]]])
a[ax1,ax2,:] = 1
a
Output:
array([[[1., 1., 1.],
[1., 1., 1.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[1., 1., 1.],
[1., 1., 1.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[1., 1., 1.],
[1., 1., 1.]],
[[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
[1., 1., 1.],
[1., 1., 1.]]])
I'm inclined to believe I should be able to look at the shape of the matrix in question, the shape of the indices, and the index operation to intuitively know the output. However, I can't put the story together in my head. Like, what's the final shape of the subspace it is altering? How would you explain how this works?
The shapes:
input: (5, 5, 3)
ind1: (2, 2, 1)
ind2: (2, 1, 2)
final_op: input[ind1, ind2, :]
With shapes
ind1: (2, 2, 1)
ind2: (2, 1, 2)
they broadcast together to select a (2,2,2) space
In [4]: ax1
Out[4]:
array([[[0],
[1]],
[[3],
[4]]])
In [5]: ax2
Out[5]:
array([[[0, 1]],
[[3, 4]]])
So for the 1st dimension (blocks) it is selecting blocks 0,1,3,and 4. In the second dimension it is also selecting these rows.
Together that's the first 2 rows of the first 2 blocks, and the last 2 rows of the last 2 blocks. That's where the 1s appear in your result.
A simpler way of creating the index arrays:
In [7]: np.array([[0,1],[3,4]])[:,:,None] # (2,2) expanded to (2,2,1)
In [8]: np.array([[0,1],[3,4]])[:,None,:] # expand to (2,1,2)
This is how broadcasting expands them:
In [10]: np.broadcast_arrays(ax1,ax2)
Out[10]:
[array([[[0, 0], # block indices
[1, 1]],
[[3, 3],
[4, 4]]]),
array([[[0, 1], # row indices
[0, 1]],
[[3, 4],
[3, 4]]])]
This may make the pattern clearer:
In [15]: a[ax1,ax2,:] = np.arange(1,5).reshape(2,2,1)
In [16]: a[:,:,0]
Out[16]:
array([[1., 2., 0., 0., 0.],
[3., 4., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 1., 2.],
[0., 0., 0., 3., 4.]])
I have a 60000 by 200 numpy array. I want to make it 60000 by 201 by adding a column of 1's to the right (so every row is [prev, 1]).
Concatenate with axis = 1 doesn't work because it seems like concatenate requires all input arrays to have the same dimension.
How should I do this?
Let me just throw in a very simple example with much smaller size. The principle should be the same.
a = np.zeros((6,2))
array([[ 0., 0.],
[ 0., 0.],
[ 0., 0.],
[ 0., 0.],
[ 0., 0.],
[ 0., 0.]])
b = np.ones((6,1))
array([[ 1.],
[ 1.],
[ 1.],
[ 1.],
[ 1.],
[ 1.]])
np.hstack((a,b))
array([[ 0., 0., 1.],
[ 0., 0., 1.],
[ 0., 0., 1.],
[ 0., 0., 1.],
[ 0., 0., 1.],
[ 0., 0., 1.]])
Using numpy index trick to append a 1D vector to a 2D array
a = np.zeros((6,2))
# array([[ 0., 0.],
# [ 0., 0.],
# [ 0., 0.],
# [ 0., 0.],
# [ 0., 0.],
# [ 0., 0.]])
b = np.ones(6) # or np.ones((6,1))
#array([1., 1., 1., 1., 1., 1.])
np.c_[a,b]
# array([[0., 0., 1.],
# [0., 0., 1.],
# [0., 0., 1.],
# [0., 0., 1.],
# [0., 0., 1.],
# [0., 0., 1.]])
Under cover all the stack variants (including append and insert) end up doing a concatenate. They just precede it with some sort of array reshape.
In [60]: A = np.arange(12).reshape(3,4)
In [61]: np.concatenate([A, np.ones((A.shape[0],1),dtype=A.dtype)], axis=1)
Out[61]:
array([[ 0, 1, 2, 3, 1],
[ 4, 5, 6, 7, 1],
[ 8, 9, 10, 11, 1]])
Here I made a (3,1) array of 1s, to match the (3,4) array. If I wanted to add a new row, I'd make a (1,4) array.
While the variations are handy, if you are learning, you should become familiar with concatenate and the various ways of constructing arrays that match in number of dimensions and necessary shapes.
The first thing to think about is that numpy arrays are really not meant to change size. So you should ask yourself, can you create your original matrix as 60k x 201 and then fill the last column afterwards. This is usually best.
If you really must do this, see
How to add column to numpy array
I think the numpy method column_stack is more interesting because you do not need to create a column numpy array to stack it in the matrix of interest. With the column_stack you just need to create a normal numpy array.
I'm trying to access the corner values of a numpy ndarray. I'm absolutely stumped as for methodology. Any help would be greatly appreciated.
For example, from the below array I'd like a return value of array([1,0,0,5]) or array([[1,0],[0,5]]).
array([[ 1., 0., 0., 0.],
[ 0., 1., 0., 0.],
[ 0., 0., 1., 5.],
[ 0., 0., 5., 5.]])
To add variety to the answers, you can get a view (not a copy) of the corner items doing:
corners = a[::a.shape[0]-1, ::a.shape[1]-1]
Or, for a generic n-dimensional array:
corners = a[tuple(slice(None, None, j-1) for j in a.shape)]
Doing this, you can modify the original array by modifying the view:
>>> a = np.arange(9).reshape(3, 3)
>>> a
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
>>> corners = a[tuple(slice(None, None, j-1) for j in a.shape)]
>>> corners
array([[0, 2],
[6, 8]])
>>> corners += 1
>>> a
array([[1, 1, 3],
[3, 4, 5],
[7, 7, 9]])
EDIT Ah, you want a flat list of corner values... That cannot in general be achieved with a view, so #IanH's answer is what you are looking for.
How about
A[[0,0,-1,-1],[0,-1,0,-1]]
where A is the array.
Use np.ix_ to construct the indices.
>>> a
array([[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 5.],
[0., 0., 5., 5.]])
>>> corners = np.ix_((0,-1),(0,-1))
>>> a[corners]
array([[1., 0.],
[0., 5.]])
You can manually specify the corners (using negative indexes):
a = numpy.array([[ 1., 0., 0., 0.],
[ 0., 1., 0., 0.],
[ 0., 0., 1., 5.],
[ 0., 0., 5., 5.]])
result = numpy.array([a[0][0],a[0][-1],a[-1][0],a[-1][-1]])
# result will contain array([ 1., 0., 0., 5.])
result = numpy.array([a[0][0],a[0][-1],a[-1][0],a[-1][-1]])
# result will contain array([[ 1., 0.],
# [ 0., 5.]])