I have an 4 dimensional array (named colors) which assigns a color (ie 3 values R, G, B) to each 3d point (x, y, z) -> (r, g, b)
I have another 2 dimensional array (named visible) which tells me which z pane I can see when I look down on it (x, y)->z
I want to create 3 dimensional array (view) which tells me what I see. (x, y) -> (r, g, b)
How can I do that with numpy smart indexing?
I tried
colors=np.array([
[
[[0, 0, 0], [1, 0, 0]],
[[0, 1, 0], [0, 0, 1]]],
[
[[1, 0, 1], [1, 1, 0]],
[[0, 1, 1], [1, 1, 1]]]])
visible=np.array(
[[0, 1],
[1, 0]])
view=colors[:, :,visible[:, :]]
expected=np.array(
[[[0, 0, 0], [1, 1, 0]],
[[0, 1, 1], [0, 0, 1]]])
But that gives me 5 dimensional array.
You can use this:
x = np.array([[0,1],[0,1]])
y = np.array([[0,0],[1,1]])
colors[(visible, y, x)]
It gives:
array([[[0, 0, 0],
[1, 1, 0]],
[[0, 1, 1],
[0, 0, 1]]])
x and y select which pixels you want, while visible is your z plane selector. They can actually be 1D and they will broadcast to fill the other dimension. You can construct arbitrary-size x and y like this:
x = np.arange(colors.shape[2])
y = np.arange(colors.shape[1]).reshape(-1,1) # transpose
Your problem was quite interesting and challenging.
Numpy's advanced indexing works a bit other like you tried to use it intuitively.
There are more options to achieve what you want:
1. You can use advanced indexing as follows using with a bit help of numpy.indices():
import numpy as np
colors=np.array([
[
[[0, 0, 0], [1, 0, 0]],
[[0, 1, 0], [0, 0, 1]]],
[
[[1, 0, 1], [1, 1, 0]],
[[0, 1, 1], [1, 1, 1]]]])
visible = np.array(
[[0, 1],
[1, 0]])
x_ind, y_ind = np.indices(visible.shape)
view = colors[visible, x_ind, y_ind]
print(view)
Out:
[[[0 0 0]
[1 1 0]]
[[0 1 1]
[0 0 1]]]
2. Alternatively you can use numpy.choose() which is very intuitive way in this case:
import numpy as np
colors=np.array([
[
[[0, 0, 0], [1, 0, 0]],
[[0, 1, 0], [0, 0, 1]]],
[
[[1, 0, 1], [1, 1, 0]],
[[0, 1, 1], [1, 1, 1]]]])
visible = np.array(
[[0, 1],
[1, 0]])
visible = visible.reshape(2,2,1)
view = np.choose(visible, colors)
print(view)
Out:
[[[0 0 0]
[1 1 0]]
[[0 1 1]
[0 0 1]]]
Related
I am trying to manipulate array2 so that the row and column is dependent on the len of the strings as for array1
str1 = "Hi"
str2 = "Bye"
array1 = [[[0, 0] for y in range(len(str2)+1)] for x in range(len(str1)+1)]
print(array1)
#output: [[[0, 0], [0, 0], [0, 0], [0, 0]], [[0, 0], [0, 0], [0, 0], [0, 0]], [[0, 0], [0, 0], [0, 0], [0, 0]]]
array2 = [[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]
#want array2 to implement same format as array1 where the row and column is determined by the len of str1 and str2
temp = [[[array2[i], array2[j]] for y in range(len(str2)+1)] for x in range(len(str1)+1)] #does not work
I tried to remove some brackets from temp however, did not work.
I tried to manipulate the method I used for array1, but did not work. I was expecting the rows and columns to be dependent on the len of the strings as for array2.
The current code has no idea what to do with array2[i], array2[j] cause neither i nor j have been defined.
This code is working as expected (I've organized the output for better readability):
str1 = "Hi"
str2 = "Bye"
array1 = [[[0, 0] for y in range(len(str2)+1)] for x in range(len(str1)+1)]
print(array1)
#output is 4x3: [
#[[0, 0], [0, 0], [0, 0], [0, 0]],
#[[0, 0], [0, 0], [0, 0], [0, 0]],
#[[0, 0], [0, 0], [0, 0], [0, 0]]]
array2 = [[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]
temp = [[[0, 0] for y in range(len(str2)+1)] for x in range(len(str1)+1)]
#output is 4x3: [
#[[0, 0], [0, 0], [0, 0], [0, 0]],
#[[0, 0], [0, 0], [0, 0], [0, 0]],
#[[0, 0], [0, 0], [0, 0], [0, 0]]]
If you want to use a certain set of numbers from the array, you need to change the code to:
i, o = 0, 1
temp = [[[array2[i], array2[j]] for y in range(len(str2)+1)] for x in range(len(str1)+1)]
If this doesn't solve your problem, please provide further explanation
I have a tensor with shape (7, 2, 3)
I want to select one of the two row vectors from each of the 7 2x3 matrices, i.e.
[
[[0, 0, 0],
[1, 1, 1]],
[[0, 0, 0],
[1, 1, 1]],
...x7
]
to
a = [
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]
...x7
]
b = [
[1, 1, 1],
[1, 1, 1],
[1, 1, 1]
...x7
]
each with shape (7, 3).
How can I do this without reshape? (I find reshape to be kind of confusing when some dimensions are the same).
I also know of
np.array(map(lambda item: item[0], x)))
but I would like a more concise way if there is one.
just use looped indexing: data[:, i, :] where i loops from 0 through 1
import numpy as np
a = np.array([
[[0, 0, 0],
[1, 1, 1]],
[[0, 0, 0],
[1, 1, 1]]
])
print(a[:, 1, :])
will produce
[[1 1 1]
[1 1 1]]
I want to create a Cartesian product of two numpy so that the first numpy will be the rows and the second will be the columns.
For example get these two numpys:
a = np.array([0,0])
b = np.array([0,1,2,3])
The expected result should be 2d numpy like this:
[[0 0 0]
[0 0 1]
[0 0 2]
[0 0 3]]
The following code does not produce the requested result:
a = np.array([0,0])
b = np.array([0,1,2,3])
_new_ = []
for idx in range(len(a)):
for i in a:
newArr = np.append(a[idx], b)
_new_.append(newArr)
print(np.stack(_new_))
What needs to be changed to produce the desired result?
You can use np.tile with np.column_stack
np.column_stack([np.tile(a, (len(b), 1)), b])
array([[0, 0, 0],
[0, 0, 1],
[0, 0, 2],
[0, 0, 3]])
If you have a a as 2D array
a = np.array([[0, 0], [1, 1]])
b = np.array([0,1,2,3])
np.c_[np.tile(a, (len(b), 1)), np.repeat(b, len(a), axis=0)]
array([[0, 0, 0],
[1, 1, 0],
[0, 0, 1],
[1, 1, 1],
[0, 0, 2],
[1, 1, 2],
[0, 0, 3],
[1, 1, 3]])
I have an array of numbers between 0 and 3 and I want to create a 2D array of their binary digits.
in the future may be I need to have array of numbers between 0 and 7 or 0 to 15.
Currently my array is defined like this:
a = np.array([[0], [1], [2], [3]], dtype=np.uint8)
I used numpy unpackbits function:
b = np.unpackbits(a, axis=1)
and the result is this :
array([[0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 1, 0],
[0, 0, 0, 0, 0, 0, 1, 1]], dtype=uint8)
As you can see it created a 2d array with 8 items in column while I'm looking for 2 columns 2d array.
here is my desired array:
array([[0, 0],
[0, 1],
[1, 0],
[1, 1]])
Is this related to data type uint8 ?
what is your idea?
One way of approaching the problem is to just adapt your b to match your desired output via a simple slicing, similarly to what suggested in #GrzegorzSkibinski answer:
import numpy as np
def gen_bits_by_val(values):
n = int(max(values)).bit_length()
return np.unpackbits(values, axis=1)[:, -n:].copy()
print(gen_bits_by_val(a))
# [[0 0]
# [0 1]
# [1 0]
# [1 1]]
Alternatively, you could create a look-up table, similarly to what suggested in #WarrenWeckesser answer, using the following:
import numpy as np
def gen_bits_by_num(n):
values = np.arange(2 ** n, dtype=np.uint8).reshape(-1, 1)
return np.unpackbits(values, axis=1)[:, -n:].copy()
bits2 = gen_bits_by_num(2)
print(bits2)
# [[0 0]
# [0 1]
# [1 0]
# [1 1]]
which allows for all kind of uses thereby indicated, e.g.:
bits4 = gen_bits_by_num(4)
print(bits4[[1, 3, 12]])
# [[0 0 0 1]
# [0 0 1 1]
# [1 1 0 0]]
EDIT
Considering #PaulPanzer answer the line:
return np.unpackbits(values, axis=1)[:, -n:]
has been replaced with:
return np.unpackbits(values, axis=1)[:, -n:].copy()
which is more memory efficient.
It could have been replaced with:
return np.unpackbits(values << (8 - n), axis=1, count=n)
with similar effects.
You can use the count keyword. It cuts from the right so you also have to shift bits before applying unpackbits.
b = np.unpackbits(a<<6, axis=1, count=2)
b
# array([[0, 0],
# [0, 1],
# [1, 0],
# [1, 1]], dtype=uint8)
This produces a "clean" array:
b.flags
# C_CONTIGUOUS : True
# F_CONTIGUOUS : False
# OWNDATA : True
# WRITEABLE : True
# ALIGNED : True
# WRITEBACKIFCOPY : False
# UPDATEIFCOPY : False
In contrast, slicing the full 8-column output of unpackbits is in a sense a memory leak because the discarded columns will stay in memory as long as the slice lives.
You can truncate b to keep just the columns since the first column with 1:
b=b[:, int(np.argwhere(b.max(axis=0)==1)[0]):]
For such a small number of bits, you can use a lookup table.
For example, here bits2 is an array with shape (4, 2) that holds the bits of the integers 0, 1, 2, and 3. Index bits2 with the values from a to get the bits:
In [43]: bits2 = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
In [44]: a = np.array([[0], [1], [2], [3]], dtype=np.uint8)
In [45]: bits2[a[:, 0]]
Out[45]:
array([[0, 0],
[0, 1],
[1, 0],
[1, 1]])
This works fine for 3 or 4 bits, too:
In [46]: bits4 = np.array([[0, 0, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 0, 1, 1], [0, 1, 0, 0], [
...: 0, 1, 0, 1], [0, 1, 1, 0], [0, 1, 1, 1], [1, 0, 0, 0], [1, 0, 0, 1], [1, 0, 1, 0], [1, 0,
...: 1, 1], [1, 1, 0, 0], [1, 1, 0, 1], [1, 1, 1, 0], [1, 1, 1, 1]])
In [47]: bits4
Out[47]:
array([[0, 0, 0, 0],
[0, 0, 0, 1],
[0, 0, 1, 0],
[0, 0, 1, 1],
[0, 1, 0, 0],
[0, 1, 0, 1],
[0, 1, 1, 0],
[0, 1, 1, 1],
[1, 0, 0, 0],
[1, 0, 0, 1],
[1, 0, 1, 0],
[1, 0, 1, 1],
[1, 1, 0, 0],
[1, 1, 0, 1],
[1, 1, 1, 0],
[1, 1, 1, 1]])
In [48]: x = np.array([0, 1, 5, 14, 9, 8, 15])
In [49]: bits4[x]
Out[49]:
array([[0, 0, 0, 0],
[0, 0, 0, 1],
[0, 1, 0, 1],
[1, 1, 1, 0],
[1, 0, 0, 1],
[1, 0, 0, 0],
[1, 1, 1, 1]])
I have some data that I want to "one-hot encode" and it is represented as a 1-dimensional vector of positions.
Is there any function in NumPy that can expand my x into my x_ohe?
I'm trying to avoid using for-loops in Python at all costs for operations like this after watching Jake Vanderplas's talk
x = np.asarray([0,0,1,0,2])
x_ohe = np.zeros((len(x), 3), dtype=int)
for i, pos in enumerate(x):
x_ohe[i,pos] = 1
x_ohe
# array([[1, 0, 0],
# [1, 0, 0],
# [0, 1, 0],
# [1, 0, 0],
# [0, 0, 1]])
If x only contains non negative integers, you can compare x with a sequence use numpy broadcasting and convert the result to ints:
(x[:,None] == np.arange(x.max()+1)).astype(int)
#array([[1, 0, 0],
# [1, 0, 0],
# [0, 1, 0],
# [1, 0, 0],
# [0, 0, 1]])
Or initialize first, then assign ones use advanced indexing:
x_ohe = np.zeros((len(x), 3), dtype=int)
x_ohe[np.arange(len(x)), x] = 1
x_ohe
#array([[1, 0, 0],
# [1, 0, 0],
# [0, 1, 0],
# [1, 0, 0],
# [0, 0, 1]])
A one liner :
np.equal.outer(x,range(3)).astype(int)
array([[1, 0, 0],
[1, 0, 0],
[0, 1, 0],
[1, 0, 0],
[0, 0, 1]])
np.equal.outer(x,np.unique(x)).astype(int) works also here.