Suppose I have the array:
[[2,1,5,2],
[1,4,2,1],
[4,5,5,7],
[1,5,9,3]]
I am trying to transpose the array to shape (16, 3) where the first two elements in the resulting array are the index numbers, and the last is the value. eg:
[[0, 0, 2], [1, 0, 1], [2, 0, 5], [3, 0, 2], [0, 1, 4], ....]
Is this possible with a numpy function or similar? Or I have to do this with my own function?
Working example code:
import numpy as np
src = np.array([[2,1,5,2],
[1,4,2,1],
[4,5,5,7],
[1,5,9,3]])
dst = np.array([])
for x in range(src.shape[0]):
for y in range(src.shape[1]):
dst = np.append(dst, [[y, x, src[x][y]]])
print(dst.reshape(16,3))
I don't know if there is a function in numpy for that, but you can use list comprehension to easily build that array:
import numpy as np
src = np.array([[2,1,5,2],
[1,4,2,1],
[4,5,5,7],
[1,5,9,3]])
dst = np.array([ [y, x, src[x][y]]
for x in range(src.shape[0])
for y in range(src.shape[1])])
print(dst.reshape(16,3))
Hope this can help.
Update: There is a numpy function for that:
You can use numpy.ndenumerate:
dst = np.array([[*reversed(x), y] for x, y in np.ndenumerate(src)])
print(dst)
#[[0 0 2]
# [1 0 1]
# [2 0 5]
# [3 0 2]
# [0 1 1]
# [1 1 4]
# [2 1 2]
# [3 1 1]
# [0 2 4]
# [1 2 5]
# [2 2 5]
# [3 2 7]
# [0 3 1]
# [1 3 5]
# [2 3 9]
# [3 3 3]]
ndenumerate will return an iterator yielding pairs of array coordinates and values. You will first need to reverse the coordinates for your desired output. Next unpack the coordinates into a list1 with the value and use a list comprehension to consume the iterator.
Original Answer
You can try:
dst = np.column_stack(zip(*[*reversed(np.indices(src.shape)), src])).T
print(dst)
#[[0 0 2]
# [1 0 1]
# [2 0 5]
# [3 0 2]
# [0 1 1]
# [1 1 4]
# [2 1 2]
# [3 1 1]
# [0 2 4]
# [1 2 5]
# [2 2 5]
# [3 2 7]
# [0 3 1]
# [1 3 5]
# [2 3 9]
# [3 3 3]]
Explanation
First, use numpy.indices to get an array representing the indices of a grid with the shape of src.
print(np.indices(src.shape))
#[[[0 0 0 0]
# [1 1 1 1]
# [2 2 2 2]
# [3 3 3 3]]
#
# [[0 1 2 3]
# [0 1 2 3]
# [0 1 2 3]
# [0 1 2 3]]]
We can reverse these (since that's the order you want in your final output), and unpack into a list1 that also contains src.
Then zip all of the elements of this list to get the (col, row, val) triples. We can stack these together using numpy.column_stack.
list(zip(*[*reversed(np.indices(src.shape)), src]))
#[(array([0, 1, 2, 3]), array([0, 0, 0, 0]), array([2, 1, 5, 2])),
# (array([0, 1, 2, 3]), array([1, 1, 1, 1]), array([1, 4, 2, 1])),
# (array([0, 1, 2, 3]), array([2, 2, 2, 2]), array([4, 5, 5, 7])),
# (array([0, 1, 2, 3]), array([3, 3, 3, 3]), array([1, 5, 9, 3]))]
Finally transpose (numpy.ndarray.T) to get the final output.
Notes:
Unpacking into a list is only available in python 3.5+
Related
I’m looking for an efficient way to make index matrix to Boolean matrix, without a loop (with NumPy).
The index matrix build from tuples which represents indices. I need to build a Boolean matrix (in different and known size) from it, which going to “1” on all of the indices that in the index matrix, and “0” in all the other positions. As an example if x array with shape (5, 3, 2) be as:
x = np.array([[[0, 0], [0, 1], [0, 3]],
[[1, 0], [1, 3], [1, 4]],
[[2, 2], [2, 3], [2, 4]],
[[3, 1], [3, 3], [3, 4]],
[[4, 2], [4, 3], [4, 4]]])
the desired output be in shape (5, 5) as:
[[1 1 0 1 0]
[1 0 0 1 1]
[0 0 1 1 1]
[0 1 0 1 1]
[0 0 1 1 1]]
In the first line the indices given are (0,0) (0,1) (0,3) ,
so the first line of the Boolean matrix is 11010 (1 where
index exists and 0 otherwise)
In the next line the indices are (1,0) (1,3) (1,4), so the line in the Boolean matrix is 10011
And so on…
I wrote a function that do it with loops, it’s attached. But it performs too slow! I’m looking for much efficiency way, with NumPy.
Thanks for all helpers!!
Row and column ids arrays can be taken from x using indexing. Then we can create a zero NumPy array with the desired shape; where, maximum column numbers can be taken from the x values and row numbers will be as for x:
row_ids = x[:, :, 0]
# [[0 0 0]
# [1 1 1]
# [2 2 2]
# [3 3 3]
# [4 4 4]]
cols_ids = x[:, :, 1]
# [[0 1 3]
# [0 3 4]
# [2 3 4]
# [1 3 4]
# [2 3 4]]
B = np.zeros((x.shape[0], x.max() + 1), dtype=np.int64)
# [[0 0 0 0 0]
# [0 0 0 0 0]
# [0 0 0 0 0]
# [0 0 0 0 0]
# [0 0 0 0 0]]
Now, we can fill the B array by 1 using indexing as:
B[row_ids, cols_ids] = 1
# [[1 1 0 1 0]
# [1 0 0 1 1]
# [0 0 1 1 1]
# [0 1 0 1 1]
# [0 0 1 1 1]]
I have 2 2d arrays and I would like to return all values that are differing in the second array while keeping the existing dimensions.
I've done something like diff = arr2[np.nonzero(arr2-arr1)] works to give me the differing elements but how do I keep the dimensions and relative position of the elements?
Example Input:
arr1 = [[0 1 2] arr2 = [[0 1 2]
[3 4 5] [3 5 5]
[6 7 8]] [6 7 8]]
Expected output:
diff = [[0 0 0]
[0 5 0]
[0 0 0]]
How about the following:
import numpy as np
arr1 = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
arr2 = np.array([[0, 1, 2], [3, 5, 5], [6, 7, 8]])
diff = arr2 * ((arr2 - arr1) != 0)
print(diff)
# [[0 0 0]
# [0 5 0]
# [0 0 0]]
EDIT: Surprisingly to me, the following first version of my answer (corrected by OP) might be faster:
diff = arr2 * np.abs(np.sign(arr2 - arr1))
If they are numpy arrays, you could do
ans = ar1 * 0
ans[ar1 != ar2] = ar2[ar1 != ar2]
ans
# array([[0, 0, 0],
# [0, 5, 0],
# [0, 0, 0]])
Without numpy, you can use map
list(map(lambda a, b: list(map(lambda x, y: y if x != y else 0, a, b)), arr1, arr2))
# [[0, 0, 0], [0, 5, 0], [0, 0, 0]]
Data
import numpy as np
arr1 = [[0, 1, 2], [3, 4, 5], [6, 7, 8]]
arr2 = [[0, 1, 2], [3, 5, 5], [6, 7, 8]]
ar1 = np.array(arr1)
ar2 = np.array(arr2)
I am surprised no one proposed the numpy.where method:
diff = np.where(arr1!=arr2, arr2, 0)
Literally, where arr1 and arr2 are different take the values of arr2, else take 0.
Output:
array([[0, 0, 0],
[0, 5, 0],
[0, 0, 0]])
np.copyto
You can check for inequality between the two arrays then use np.copyto with np.zeros/ np.zeros_like.
out = np.zeros(arr2.shape) # or np.zeros_like(arr2)
np.copyto(out, arr2, where=arr1!=arr2)
print(out)
# [[0 0 0]
# [0 5 0]
# [0 0 0]]
np.where
You can use np.where and specify x, y args.
out = np.where(arr1!=arr2, arr2, 0)
# [[0 0 0]
# [0 5 0]
# [0 0 0]]
In a single line, how can I get the product of the arrays of an array?
I need it to be done for multi columns cases
2 columns example:
X = [[1 4]
[2 3]
[0 2]
[1 5]
[3 1]
[3 6]]
sol = [4 6 0 5 3 18]
4 columns example:
X = [[1 4 2 3]
[2 3 1 5]
[0 2 3 4]
[1 5 2 2]
[3 1 1 6]
[3 6 3 1]]
sol = [24 30 0 20 18 54]
This is a row-wise multiplication. You can perform this with:
X.prod(axis=1)
for example:
>>> X
array([[1, 4],
[2, 3],
[0, 2],
[1, 5],
[3, 1],
[3, 6]])
>>> a.prod(axis=1)
array([ 4, 6, 0, 5, 3, 18])
You can also use numpy.multiply.reduce
np.multiply.reduce(x, axis=1)
In Tensorflow, say I have two matrices M and N, how can I get a tensor whose (i, j) element is the element-wise product of the i-th row of M and j-th row of N?
Here's a trick: expand both matrices to 3D and do elemet-wise multiply (a.k.a. Hadamard product).
# Let `a` and `b` be the rank 2 tensors, with the same 2nd dimension
lhs = tf.expand_dims(a, axis=1)
rhs = tf.expand_dims(b, axis=0)
products = lhs * rhs
Let's check that it works:
tf.InteractiveSession()
# 2 x 3
a = tf.constant([
[1, 2, 3],
[3, 2, 1],
])
# 3 x 3
b = tf.constant([
[2, 1, 1],
[2, 2, 0],
[1, 2, 1],
])
lhs = tf.expand_dims(a, axis=1)
rhs = tf.expand_dims(b, axis=0)
products = lhs * rhs
print(products.eval())
# [[[2 2 3]
# [2 4 0]
# [1 4 3]]
#
# [[6 2 1]
# [6 4 0]
# [3 4 1]]]
The same trick actually works in numpy as well and with any element-wise binary operation (sum, product, division, ...). Here's an example of row-by-row element-wise sum tensor:
# 2 x 3
a = np.array([
[1, 2, 3],
[3, 2, 1],
])
# 3 x 3
b = np.array([
[2, 1, 1],
[2, 2, 0],
[1, 2, 1],
])
lhs = np.expand_dims(a, axis=1)
rhs = np.expand_dims(b, axis=0)
sums = lhs + rhs
# [[[2 2 3]
# [2 4 0]
# [1 4 3]]
#
# [[6 2 1]
# [6 4 0]
# [3 4 1]]]
Given base array X of shape (2, 3, 4) which can be interpreted as two sets of 3 elements each, where every element is 4-dimensional, I want to sample from this array X in the following way.
From each of 2 sets I want to pick 2 subsets each defined by the binary array of length 3, other subsets would be set to 0. So the sampling process is defined by the array of shape (2, 2, 3). The result of this sampling should have shape (2, 2, 3, 4).
Here's the code that does what I need but I wonder if it could be rewritten more efficiently using numpy indexing.
import numpy as np
np.random.seed(3)
sets = np.random.randint(0, 10, [2, 3, 4])
subset_masks = np.random.randint(0, 2, [2, 2, 3])
print('Base set\n', sets, '\n')
print('Subset masks\n', subset_masks, '\n')
result = np.empty([2, 2, 3, 4])
for set_index in range(sets.shape[0]):
for subset_index, subset in enumerate(subset_masks[set_index]):
print('----')
picked_subset = subset.reshape(3, 1) * sets[set_index]
result[set_index][subset_index] = picked_subset
print('Picking subset ', subset, 'from set #', set_index)
print(picked_subset, '\n')
Output
Base set
[[[8 9 3 8]
[8 0 5 3]
[9 9 5 7]]
[[6 0 4 7]
[8 1 6 2]
[2 1 3 5]]]
Subset masks
[[[0 0 1]
[1 0 0]]
[[1 0 1]
[0 1 1]]]
----
Picking subset [0 0 1] from set # 0
[[0 0 0 0]
[0 0 0 0]
[9 9 5 7]]
----
Picking subset [1 0 0] from set # 0
[[8 9 3 8]
[0 0 0 0]
[0 0 0 0]]
----
Picking subset [1 0 1] from set # 1
[[6 0 4 7]
[0 0 0 0]
[2 1 3 5]]
----
Picking subset [0 1 1] from set # 1
[[0 0 0 0]
[8 1 6 2]
[2 1 3 5]]
Extend each of them to 4D by adding new axis for subset_masks along the last one and for sets as the second axis. For adding those new axes, we can use None/np.newaxis. Then, leverage NumPy broadcasting to perform the element-wise multiplication, like so -
subset_masks[...,None]*sets[:,None]
Just for the kicks probably, we can also use np.einsum -
np.einsum('ijk,ilj->iljk',sets,subset_masks)