Say I have a square 2d array (mat) and a 1d array (arr) which is the same length as the flattened 2d array (the values inside are different). Given the row and column index of the 2d array, how can I map it into the 1d array index to give the value of the same position as the 2d array? Here's a small (4 element) example of what I mean:
mat = np.array([[6, 7], [8, 9]])
arr = np.array([1, 2, 3, 4])
The mapping I'm looking for is:
mat[0,0] -> arr[0]
mat[0,1] -> arr[1]
mat[1,0] -> arr[2]
mat[1,1] -> arr[3]
Note that I can't match with values as they aren't the same so the mapping would be on the index itself.
> arr = np.random.randint(0, 10, size=(9,))
> arr
array([5, 7, 6, 5, 9, 6, 2, 0, 1])
> mat = np.random.randint(0, 10, size=(3,3))
> mat
array([[2, 8, 3],
[1, 4, 6],
[9, 2, 3]])
> idx = np.arange(len(arr)).reshape(mat.shape[0],mat.shape[1])
> idx
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
> mat[0, 0], arr[idx[0, 0]]
2, 5
Related
Say I have some time-series data in the form of a simple array.
X1 = np.array[(1, 2, 3, 4]
The Hankel matrix can be obtained by using scipy.linalg.hankel, which would look something like this:
hankel(X1)
array([[1, 2, 3, 4],
[2, 3, 4, 0],
[3, 4, 0, 0],
[4, 0, 0, 0]])
Now assume I had a larger array in the form of
X2 = np.array([1, 2, 3, 4, 5, 6, 7])
What I want to do is fill in the zeros in this matrix with the numbers that are next in the index (specific to each row). Taking the same Hankel matrix earlier by using the first four values in the array X2, I'd like to see the following output:
hankel(X2[:4])
array([[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 6],
[4, 5, 6, 7]])
How would I do this? I'd ideally like to use this for larger data.
Appreciate any tips or pointers given. Thanks!
If you have a matrix with the appropriate index values into your dataset, you can use integer array indexing directly into your dataset.
To create the index matrix, you can simply use the upper-left quadrant of a double-sized Hankel array. There are likely simpler ways to create the index matrix, but this does the trick.
>>> X = np.array([9, 8, 7, 6, 5, 4, 3])
>>> N = 4 # the size of the "window"
>>> indices = scipy.linalg.hankel(np.arange(N*2))[:N, :N]
>>> indices
array([[0, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 6]])
>>> X[indices]
array([[9, 8, 7, 6],
[8, 7, 6, 5],
[7, 6, 5, 4],
[6, 5, 4, 3]])
I'm working with numpy and I hit a roadblock, I think it's an easy question and can be done using indexing, but I still haven't figure it out. So I have 2D array and from each row I get the index of the minimum value, what I want is to use this index to add values to the 2D array, here is an example
a = np.array([[9, 4, 9, 9],
[1, 6, 4, 6],
[8, 7, 1, 5],
[8, 9, 2, 7]])
values = np.array([1, 2, 3, 4])
minimum = np.argmin(a, axis = 1) #To find the index of the minimum values
print(minimum) #minimum = [1, 0, 2, 2]
So in the first row of the 2D array and the first index of the minimum array I want to add the value of 1, in the second row of the 2D array using the second index, I want to add the value of 2 and so on, my desired output should be something like this.
output = [[9, 5, 9, 9],
[3, 6, 4, 6],
[8, 7, 4, 5],
[8, 9, 6, 7]]
I try this, but I failed:
newarray = a[:, minimum] + values
newarray = [[ 5, 11, 12, 13],
[ 7, 3, 7, 8],
[ 8, 10, 4, 5],
[10, 10, 5, 6]]
Thank you for all your help!
You were close. Try this:
newarray = a.copy()
newarray[np.arange(len(a)), minimum] += values
Let's say I have the following NumPy arrays:
i = array([2, 4, 5])
j = array([0, 1, 2])
I would like to have a very efficient method (built-in if possible) to sum those vectors and have an output that looks like this:
[[2 4 5]
[3 5 6]
[4 6 7]]
So basically each column is the array j to which the k th element of i has been added (k = 0, 1, 2 in this case)
Use numpy.add.outer.
>>> import numpy as np
>>> i = np.array([2, 4, 5])
>>> j = np.array([0, 1, 2])
>>>
>>> np.add.outer(j, i)
array([[2, 4, 5],
[3, 5, 6],
[4, 6, 7]])
Or with broadcasting:
In [272]: i[:,None]+j
Out[272]:
array([[2, 3, 4],
[4, 5, 6],
[5, 6, 7]])
i[:,None] makes a (3,1) array, which broadcasts with a (3,) (or (1,3)) to make a (3,3).
(short version of my question: In numpy, is there an elegant way of emulating tf.sequence_mask from tensorflow?)
I have a 2d array a (each row represents a sequence of different length). Next, there is a 1d array b (representing sequence lengths). Is there an elegant way to get a (flattened) array that would contain only such elements of a that belong to the sequences as specified by their length b:
a = np.array([
[1, 2, 3, 2, 1], # I want just [:3] from this row
[4, 5, 5, 5, 1], # [:2] from this row
[6, 7, 8, 9, 0] # [:4] from this row
])
b = np.array([3,2,4]) # 3 elements from the 1st row, 2 from the 2nd, 4 from the 4th row
the desired result:
[1, 2, 3, 4, 5, 6, 7, 8, 9]
By elegant way I mean something that avoids loops.
Use broadcasting to create a mask of the same shape as the 2D array and then simply mask and extract valid elements -
a[b[:,None] > np.arange(a.shape[1])]
Sample run -
In [360]: a
Out[360]:
array([[1, 2, 3, 2, 1],
[4, 5, 5, 5, 1],
[6, 7, 8, 9, 0]])
In [361]: b
Out[361]: array([3, 2, 4])
In [362]: a[b[:,None] > np.arange(a.shape[1])]
Out[362]: array([1, 2, 3, 4, 5, 6, 7, 8, 9])
I'm having a bit of a difficulty. I'm trying to vectorize some code in python in order to make it faster. I have an array which I sort (A) and get the index list (Ind). I have another array (B) which I would like to sort by the index list, without using loops which I think bottlenecks the computation.
A = array([[2, 1, 9],
[1, 1, 5],
[7, 4, 1]])
Ind = np.argsort(A)
This is the result of Ind:
Ind = array([[1, 0, 2],
[0, 1, 2],
[2, 1, 0]], dtype=int64)
B is the array i would like to sort by Ind:
B = array([[ 6, 3, 9],
[ 1, 5, 3],
[ 2, 7, 13]])
I would like to use Ind to rearrange my elements in B as such (B rows sorted by A rows indexes):
B = array([[ 3, 6, 9],
[ 1, 5, 3],
[13, 7, 2]])
Any Ideas? I would be glad to get any good suggestion. I want to mention I am using millions of values, I mean arrays of 30000*5000.
Cheers,
Robert
I would do something like this:
import numpy as np
from numpy import array
A = array([[2, 1, 9],
[1, 1, 5],
[7, 4, 1]])
Ind = np.argsort(A)
B = array([[ 3, 6, 9],
[ 1, 5, 3],
[13, 7, 2]])
# an array of the same shape as A and B with row numbers for each element
rownums = np.tile(np.arange(3), (3, 1)).T
new_B = np.take(B, rownums * 3 + Ind)
print(new_B)
# [[ 6 3 9]
# [ 1 5 3]
# [ 2 7 13]]
You can replace the magic number 3 with the array shape.