I have multiple one-hot encoded vectors per sample across batch - shape (3, 5, 10), where 3 is batch size, 5 is number of one-hot vectors and 10 is number of classes.
I want to randomly pick one one-hot encoded vector per sample, so during training these are going to be different every epoch. But there are also some padding vectors that I had to add to be able to work with data loader, and I don't want to pick them. There are not 3 actual vectors per sample, I just generated it like that
Example input (there are padding vectors at the end):
tensor([[[0, 1, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 0, 1, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[1, 0, 0, 0, 0],
[0, 1, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[1, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 0, 1, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]]]))
Expected output:
tensor([[0, 0, 0, 1, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 0, 1]])
Can I achieve it using only PyTorch? I care about not moving data between devices
I tried using random_indices = torch.randint(low=0, high=3, size=(3,)) and then applying it on my tensor, but I wasnt able to pass multiple high values
Related
i have defined a matrix m , i wish to return TRUE if there is any column which has all its elements as 1, for example :
m = [[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0]]
i have tried various approaches but they all seem to return True in all cases(since there are multiple columns with all elements as 0)
I agree with ansev's response. You gave us a list of lists. I prefer numpy for these kind of exercises.
import numpy as np
m = [[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0]]
np.array(m).all(axis=0).any()
Output
True
I have a numpy array of shape (X,Y,Z). I want to check each of the Z dimension and delete the non-zero dimension really fast.
Detailed explanation:
I would like to check array[:,:,0] if any entry is non-zero.
If yes, ignore and check array[:,:,1].
Else if No, delete dimension array[:,:,0]
Also not 100% sure what your after but I think you want
np.squeeze(array, axis=2)
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.squeeze.html
I'm not certain what you want but this hopefully points in the right direction.
Edit 1st Jan:
Inspired by #J.Warren's use of np.squeeze I think np.compress may be more appropriate.
This does the compression in one line
np.compress((a!=0).sum(axis=(0,1)), a, axis=2) #
To explain the first parameter in np.compress
(a!=0).sum(axis=(0, 1)) # sum across both the 0th and 1st axes.
Out[37]: array([1, 1, 0, 0, 2]) # Keep the slices where the array !=0
My first answer which may no longer be relevant
import numpy as np
a=np.random.randint(2, size=(3,4,5))*np.random.randint(2, size=(3,4,5))*np.random.randint(2, size=(3,4,5))
# Make a an array of mainly zeroes.
a
Out[31]:
array([[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 1, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 0, 0, 0],
[1, 0, 0, 0, 0]]])
res=np.zeros(a.shape[2], dtype=np.bool)
for ix in range(a.shape[2]):
res[ix] = (a[...,ix]!=0).any()
res
Out[34]: array([ True, True, False, False, True], dtype=bool)
# res is a boolean array of which slices of 'a' contain nonzero data
a[...,res]
# use this array to index a
# The output contains the nonzero slices
Out[35]:
array([[[0, 0, 0],
[0, 0, 1],
[0, 0, 0],
[0, 0, 0]],
[[0, 1, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 1],
[0, 0, 0],
[1, 0, 0]]])
I am working with a 2-d numpy array which looks like this:
array([[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
...,
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
...,
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]],
[[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0],
...,
[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]])
So I have numpy inside numpy which has a list of four values (pixels in RGBA to be specific). I want to set all values to 0. What is the most pythonic way to do it?
Thanks in advance!
m[:] = 0
would set all values of your array m to zero.
If you need an array of zeros with the same shape and datatype of m, use:
np.zeros_like(m)
import numpy as np
a = np.random.randn(10, 10)
b = np.zeros_like(a)
b will be an ndarray of exactly the same shape as the original, filled with 0.
a = np.diag(np.array([2,3,4,5,6]),k=-1)
For the above code, I want to know how to change it for shaping the 6*6 matrix into 6*5 matrix with the first line is filled with 0 and the following lines with 2,3,4,5,6 to be diagonal? Thank you very much
I don't understand what you want to know.
In your code if k>0
then the resultant matrix will have k extra columns,if k=2 then,
output will be :
array([[0, 0, 2, 0, 0, 0, 0],
[0, 0, 0, 3, 0, 0, 0],
[0, 0, 0, 0, 4, 0, 0],
[0, 0, 0, 0, 0, 5, 0],
[0, 0, 0, 0, 0, 0, 6],
[0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0]])
And if k<0 then it will have the k extra rows , for example if k=-1
then:
array([[0, 0, 0, 0, 0, 0],
[2, 0, 0, 0, 0, 0],
[0, 3, 0, 0, 0, 0],
[0, 0, 4, 0, 0, 0],
[0, 0, 0, 5, 0, 0],
[0, 0, 0, 0, 6, 0]])
and if k=0 then :
array([[2, 0, 0, 0, 0],
[0, 3, 0, 0, 0],
[0, 0, 4, 0, 0],
[0, 0, 0, 5, 0],
[0, 0, 0, 0, 6]])
I think you want to create a matrix of 5*5 and then want too add a row. Then you can do it using this
a=a.tolist()
Now a is 2d list and you can insert the row wherever you want.
Do this for your result.
a.insert(0,[0,0,0,0,0])
I have a list of ground truth labels:
yTrue = ['class2','classC','class3','class3','classA','classB','class2']
and a list of the possible classes (distinct, in custom order):
orderedClasses = ['classA','class2','classB','class3','classC']
I want to code the list in One-Vs-The-Rest for all possible classes.
Desired output:
[[0,1,0,0,0],[0,0,0,0,1],[0,0,0,1,0],[0,0,0,1,0],[1,0,0,0,0],[0,0,1,0,0],[0,1,0,0,0]]
I tried to use sklearn.preprocessing.label_binarize (doc) but the problem is it doesn't maintain my custom order for classes:
[[0,0,1,0,0],[0,0,0,0,1],[1,0,0,0,0],[1,0,0,0,0],[0,0,0,1,0],[0,1,0,0,0],[0,0,1,0,0]]
Looking for an Pythonic and efficient way to get the desired output
Simply pass orderedClasses as classes parameter
In [15]: label_binarize(yTrue, orderedClasses)
Out[15]:
array([[0, 1, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 0, 1, 0],
[0, 0, 0, 1, 0],
[1, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 1, 0, 0, 0]])