I am new to torch7, and I can't find a way to get the some non contiguous indices of a tensor based on another tensor. In numpy, what I do is the following:
array = np.zeros(5) # array = [0 0 0 0 0]
indices = np.array([0, 2, 4])
array[indices] = np.array([1, 2, 3]) # array = [1 0 2 0 3]
Is there a way to do something similar in torch7? Something like:
array = torch.zeros(5) -- array = [0 0 0 0 0]
indices = torch.Tensor({1, 3, 5})
array[indices] = torch.Tensor({1, 2, 3}) -- array = [1 0 2 0 3]
Thanks!
Ok, looking arround, I couldn't find an exact solution, but I found an approximation of what I wanted to do, I share it in case someone else finds it useful:
array = torch.zeros(5) -- array = [0 0 0 0 0]
indices = torch.LongTensor({1, 3, 5}) -- Is important that this is a LongTensor
array:indexAdd(1, indices, torch.Tensor({1, 2, 3})) -- array = [1 0 2 0 3]
torch.IndexCopy does exactly what you need:
array:indexCopy(1, indices, torch.Tensor({1, 2, 3}))
If you are a python user, maybe you could also try https://github.com/imodpasteur/lutorpy . For example, you can process your array in python and then convert it to torch tensor. If you want to convert back to numpy array, the conversion is instant because it only pass the pointer, and two object in python and lua are sharing the same memory.
array = np.zeros(5) # array = [0 0 0 0 0]
indices = np.array([0, 2, 4])
array[indices] = np.array([1, 2, 3]) # array = [1 0 2 0 3]
require("torch")
# convert numpy array to torch tensor
tensor = torch.fromNumpyArray(array)
# put your torch code here
#convert back to numpy array
array = tensor.asNumpyArray()
# now array is sharing memory with tensor
Related
I have a numpy array, I want to replace whole values to zeros except some range of index.
1
2
3
4
5
I tried
Import numpy as np
data=np.loadtxt('data.txt')
print(data)
expected output
0
0
3
0
0
You can traverse the array with a for loop and check if the traversed element is in a list of desired selected values:
import numpy as np
a = np.array([1, 2, 3, 4, 5])
nums = [3]
for i in range(len(a)):
if a[i] in nums:
pass
else:
a[i] = 0
print(a)
Output:
[0 0 3 0 0]
As you're working with a numpy array, use vectorial methods.
Here isin to form a boolean mask for replacement:
data = np.array([1, 2, 3, 4, 5])
l = [3]
data[~np.isin(data, l)] = 0
data
# array([0, 0, 3, 0, 0])
Given a 2D array, I would like to permute this array row-wise.
Currently, I will create a for loop to permute the 2D array row by row as below:
for i in range(npart):
pr=np.random.permutation(range(m))
# arr_rand3 is the same as arr, but with each row permuted
arr_rand3[i,:]=arr[i,pr]
But, I wonder whether there is some setting within Numpy that can perform this in single line (without the for-loop).
The full code is
import numpy as np
arr=np.array([[0,0,0,0,0],[0,4,1,1,1],[0,1,1,2,2],[0,3,2,2,2]])
npart=len(arr[:,0])
m=len(arr[0,:])
# Permuted version of arr
arr_rand3=np.zeros(shape=np.shape(arr),dtype=np.int)
# Nodal association matrix for C
X=np.zeros(shape=(m,m),dtype=np.double)
# Random nodal association matrix for C_rand3
X_rand3=np.zeros(shape=(m,m),dtype=np.double)
for i in range(npart):
pr=np.random.permutation(range(m))
# arr_rand3 is the same as arr, but with each row permuted
arr_rand3[i,:]=arr[i,pr]
In Numpy 1.19+ you should be able to do:
import numpy as np
arr = np.array([[0, 0, 0, 0, 0], [0, 4, 1, 1, 1], [0, 1, 1, 2, 2], [0, 3, 2, 2, 2]])
rng = np.random.default_rng()
arr_rand3 = rng.permutation(arr, axis=1)
print(arr_rand3)
Output
[[0 0 0 0 0]
[4 0 1 1 1]
[1 0 1 2 2]
[3 0 2 2 2]]
According to the documentation, the method random.Generator.permutation receives a new parameter axis:
axis int, optional
The axis which x is shuffled along. Default is 0.
1.
i try to make a numpy array with shape:(6962341, 268148), type: np.uint8
2.
i have the data consist of [x1,x2,x3,x4], [x2,x1], [x4,x5,x3]...
3.
i want to assign array[x1,x2] += 1, array[x1,x3] += 1, array[x1,x4] += 1, array[x2,x3] += 1, ...
4.
so i have tried a function of the following structure.
import numpy as np
from itertools import combinations
base_array = np.zeros((row_size, col_size), dtype=np.uint8))
for each_list in data:
for (x,y) in list(combinations(each_list,2)):
if x>y:
base_array[y,x] += 1
else:
base_array[x,y] += 1
it basically compute the upper triangle of a matrix and i will use the upper triangle value. also you can think this is similar to make the base matrix A for co-occurrence matrix. but this function is too slow and i think it is possible to make faster.
What should i do?
Assuming your data is integers (since they represent rows and columns) or you can hash your data x1, x2, ... into 1, 2, ... integers, here is a fast solution:
#list of pairwise combinations in your data
comb_list = []
for each_list in data:
comb_list += list(combinations(each_list,2))
#convert combination int to index (numpy is 0 based indexing)
comb_list = np.array(comb_list) - 1
#make array with flat indices
flat = np.ravel_multi_index((comb_list[:,0],comb_list[:,1]),(row_size,col_size))
#count number of duplicates for each index using np.bincount
base_array = np.bincount(flat,None,row_size*col_size).reshape((row_size,col_size)).astype(np.uint8)
sample data:
[[1, 2, 3, 4], [2, 1], [4, 5, 3, 4]]
Corresponding output:
[[0 1 1 1 0]
[1 0 1 1 0]
[0 0 0 2 0]
[0 0 1 1 1]
[0 0 1 1 0]]
EDIT: corresponding to explanation in comments:
data=[[1, 2, 3, 4], [2, 1], [4, 5, 3, 4]]
base_array = np.zeros((len(data), np.max(np.amax(data))), dtype=np.uint8)
for i, each_list in enumerate(data):
for j in each_list:
base_array[i, j-1] = 1
Output:
[[1 1 1 1 0]
[1 1 0 0 0]
[0 0 1 1 1]]
I have a list in python and the first numbers are [[29.046875, 1], [33.65625, 1], [18.359375, 1], [11.296875, 1], [36.671875, 1], [23.578125, 1],.........,[34.5625, 1]]
The above list is given an id of listNumber. I'm trying to use numpy.argsort to sort it based on the float elements:
listNumber = np.array(listNumber)
print(np.argsort(listNumber))
But this gives me the following but not sure why:
[[1 0]
[1 0]
[1 0]
...
[1 0]
[1 0]
[1 0]]
Why is this returning this? and is there another way to approach this?
Ok so i think there's two things going on here:
1- Your list is a list of lists
2- The 'argsort' function:
returns the indices that would sort an array.
According to the documentation.
So what is happening is the function reads through each item of the list, which in itself is a list, say index 0 is:
[29.046875, 1]
Then it is saying, okay this is another list so let me sort it and then return a number based on where it would go if it was the new index:
[29.046875, 1] -> [1, 0]
Because 1 would come before 29 if it was sorted in ascending order.
It does this for every nested list then gives you a final list containing all these 1's and 0's.
This answers the first question. Another user was able to answer the second :)
You must set axis like:
import numpy as np
l = [[29.046875, 1], [33.65625, 1], [18.359375, 1], [11.296875, 1], [36.671875, 1], [23.578125, 1],[34.5625, 1]]
l = np.argsort(l, axis=0) # sorts along first axis (down)
print(l)
output:
[[3 0]
[2 1]
[5 2]
[0 3]
[1 4]
[6 5]
[4 6]]
Try this;
sortedList = listNumber[listNumber[:,0].argsort(axis=0)]
print(sortedList)
I don't know why people like using predone functions instead of using their own algorithm. Anyway, you are using argsort in a bad way. argsort returns an array containing the INDEXES of your elements, thos are 2 examples :
Code 1:
import numpy as geek
# input array
in_arr = geek.array([ 2, 0, 1, 5, 4, 1, 9])
print ("Input unsorted array : ", in_arr)
out_arr = geek.argsort(in_arr)
print ("Output sorted array indices : ", out_arr)
print("Output sorted array : ", in_arr[out_arr])
Output :
Input unsorted array : [2 0 1 5 4 1 9]
Output sorted array indices : [1 2 5 0 4 3 6]
Output sorted array : [0 1 1 2 4 5 9]
Code 2:
# Python program explaining
# argpartition() function
import numpy as geek
# input 2d array
in_arr = geek.array([[ 2, 0, 1], [ 5, 4, 3]])
print ("Input array : ", in_arr)
# output sorted array indices
out_arr1 = geek.argsort(in_arr, kind ='mergesort', axis = 0)
print ("Output sorteded array indices along axis 0: ", out_arr1)
out_arr2 = geek.argsort(in_arr, kind ='heapsort', axis = 1)
print ("Output sorteded array indices along axis 1: ", out_arr2)
Output:
Input array : [[2 0 1]
[5 4 3]]
Output sorteded array indices along axis 0: [[0 0 0]
[1 1 1]]
Output sorteded array indices along axis 1: [[1 2 0]
[2 1 0]]
I am supposing that your data is stored in listnumber
import numpy as np
new_listnumber = listnumber[:, 0]
index_array = np.argsort(new_listnumber , axis=0)
New_val = listnumber[index_array]
print(New_val)
I have a sparse matrix in which I want to increment all the values of non-zero elements by one. However, I cannot figure it out. Is there a way to do it using standard packages in python? Any help will be appreciated.
I cannot comment on it's performance but you can do (Scipy 1.1.0);
>>> from scipy.sparse import csr_matrix
>>> a = csr_matrix([[0, 2, 0], [1, 0, 0]])
>>> print(a)
(0, 1) 2
(1, 0) 1
>>> a[a.nonzero()] = a[a.nonzero()] + 1
>>> print(a)
(0, 1) 3
(1, 0) 2
If your matrix have 2 dimensions, you can do the following:
sparse_matrix = [[element if element==0 else element+1 for element in row ]for row in sparse_matrix]
It will iterate over every element of your matrix and return the element without any change if it is equals to zero, else it add 1 to the element and return it.
More about conditionals in list comprehension in the answer for this question.
You can use the package numpy which has efficient functions for dealing with n-dimensional arrays. What you need is:
array[array>0] += 1
where array is the numpy array of your matrix. Example here:
`
import numpy as np
my_matrix = [[2,0,0,0,7],[0,0,0,4,0]]
array = np.array(my_matrix);
print("Matrix before incrementing values: \n", array)
array[array>0] += 1
print("Matrix after incrementing values: \n", array)`
Outputs:
Matrix before incrementing values:
[[2 0 0 0 7]
[0 0 0 4 0]]
Matrix after incrementing values:
[[3 0 0 0 8]
[0 0 0 5 0]]
Hope this helps!
When you have a scipy sparse matrix (scipy.sparse) is:
import scipy.sparse as sp
my_matrix = [[2,0,0,0,7],[0,0,0,4,0]]
my_matrix = sp.csc_matrix(my_matrix)
my_matrix.data += 1
my_matrix.todense()
Returns:
[[3, 0, 0, 0, 8], [0, 0, 0, 5, 0]]