I have a 2D array which describes index ranges for a 1D array like
z = np.array([[0,4],[4,9]])
The 1D array
a = np.array([1,1,1,1,0,0,0,0,0,1,1,1,1])
I want to have a view on the 1D array with the index range defined by z. So, for only the first range
a[z[0][0]:z[0][1]]
How to get it for all ranges? Is it possible to use as_strided with unequal lengths defined by z as shape? I want to avoid to copy data, actually I only want a different view on a for further computation.
In [66]: a = np.array([1,1,1,1,0,0,0,0,0,1,1,1,1])
In [67]: z = np.array([[0,4],[4,9]])
So generating the slices from the rows of z we get 2 arrays:
In [68]: [a[x[0]:x[1]] for x in z]
Out[68]: [array([1, 1, 1, 1]), array([0, 0, 0, 0, 0])]
Individually those arrays are views. But together they aren't an array. The lengths diff, so they can't be vstacked into a (2,?) array. They can be hstacked but that won't be a view.
The calculation core of np.array_split is:
sub_arys = []
sary = _nx.swapaxes(ary, axis, 0)
for i in range(Nsections):
st = div_points[i]
end = div_points[i + 1]
sub_arys.append(_nx.swapaxes(sary[st:end], axis, 0))
Ignoring the swapaxes bit, this is doing the same thing as my list comprehension.
for x, y in z:
array_view = a[x:y]
# do something with array_view
Related
It is similar to some questions around SO, but I don't quite understand the trick to get what I want.
I have two arrays,
arr of shape (x, y, z)
indexes of shape (x, y) which hold indexes of interest for z.
For each value of indexes I want to get the actual value in arr where:
arr.x == indexes.x
arr.y == indexes.y
arr.z == indexes[x,y]
This would give an array of shape(x,y) similar to indexes' shape.
For example:
arr = np.arange(99)
arr = arr.reshape(3,3,11)
indexes = np.asarray([
[0,2,2],
[1,2,3],
[3,2,10]])
# indexes.shape == (3,3)
# Example for the first element to be computed
first_element = arr[0,0,indexes[0,0]]
With the above indexes, the expected arrays would look like:
expected_result = np.asarray([
[0,13,24],
[34,46,58],
[69,79,98]])
I tried elements = np.take(arr, indexes, axis=z)
but it gives an array of shape (x, y, x, y)
I also tried things like elements = arr[indexes, indexes,:] but I don't get what I wish.
I saw a few answers involving transposing indexes and transforming it into tuples but I don't understand how it would help.
Note: I'm a bit new to numpy so I don't fully understand indexing yet.
How would you solve this numpy style ?
This can be done using np.take_along_axis
import numpy as np
#sample data
np.random.seed(0)
arr = np.arange(3*4*2).reshape(3, 4, 2) # 3d array
idx = np.random.randint(0, 2, (3, 4)) # array of indices
out = np.squeeze(np.take_along_axis(arr, idx[..., np.newaxis], axis=-1))
In this code, the array of indices gets added one more axis, so it can be broadcasted to the shape of the array arr from which we are making the selection. Then, since the return value of np.take_along_axis has the same shape as the array of indices, we need to remove this extra dimension using np.squeeze.
Another option is to use np.choose, but in this case the axis along which you are making selections must be moved to be the first axis of the array:
out = np.choose(idx, np.moveaxis(arr, -1, 0))
The solution here should work for you: Indexing 3d numpy array with 2d array
Adapted to your code:
ax_0 = np.arange(arr.shape[0])[:,None]
ax_1 = np.arange(arr.shape[1])[None,:]
new_array = arr[ax_0, ax_1, indexes]
You can perform such an operation with np.take_along_axis, the operation can only be applied along one dimension so you will need to reshape your input and indices.
The operation you are looking to perform is:
out[i, j] = arr[i, j, indices[i, j]]
However, we are forced to reshape both arr and indices, i.e. map (i, j) to k, such that we can apply np.take_along_axis. The following operation will take place:
out[k] = arr[k, indices[k]] # indexing along axis=1
The actual usage here comes down to:
>>> put = np.take_along_axis(arr.reshape(9, 11), indices.reshape(9, 1), axis=1)
array([[ 0],
[13],
[24],
[34],
[46],
[58],
[69],
[79],
[91]])
Then reshape back to the shape of indices:
>>> put.reshape(indices.shape)
array([[ 0, 13, 24],
[34, 46, 58],
[69, 79, 91]])
Say I have, as an arbitrary example, two 2d numpy arrays, X and Y where
X = np.array([[0,1,2,3],
[4,5,6,7]])
Y = np.array([[1,2,3,4],
[1,1,7,3]])
I want to create a new 2d numpy array, Z, that is the argmax of X,Y element-wise so Z would be, in this example:
Z = np.array([[1,2,3,4],
[4,5,7,7]])
I've tried variations of the following none return the intended result
np.array([(np.argmax(X,Y))]) --> error
I know I can do this simply by using a nested for loop but that isn't very efficient for very large datasets. Is there an efficient, numpy-specific, way to create a new 2d array (Z, in the example above) composed of the argmax by element from two 2d arrays (X and Y, in the example above)?
You're looking for np.maximum:
>>> np.maximum(X, Y)
array([[1, 2, 3, 4],
[4, 5, 7, 7]])
which compares the arrays element-wise and returns the maximum for each of them.
Use numpy.where:
Z = np.where(X > Y, X, Y)
Here, the first argument X > Y compares X and Y element by element, and returns a boolean array of the comparison. Then we use the boolean array to build Z: if the element at an index is True, it uses the value from X, and if it is False it uses the value from Y.
y = np.array([1, 1, 0, 0])
print("Y 1:", y)
y = y.T
print("Y 2:", y)
Both print the same:
[1,1,0,0]
The numpy array a = [1,1,0,0] shape is [4,] and b= [[1,1,0,0]] is [1,4]. Can I say that a is a vector and b is a matrix. So the transpose operation doesn't work for vector so the two prints are the same.
As you can see from the documentation here, transposing a 1-D array returns an unchanged view of the original array. In your case, y is a 1-D array, hence the transposed output will be unchanged.
numpy distinguishes 1d "vectors" from 2d "row vectors" and "column vectors". it's common to switch between them using something like:
# create 1d vector
v = np.array([1, 1, 0, 0])
# row vector
rv = v[np.newaxis, :]
# column vector
cv = v[:, np.newaxis]
note that np.newaxis is defined to be None, so you'll often see this written as, e.g. v[:,None] and because needing a column vector is such a common operation there are obscure shorthands like np.c_[v]
now that rv and cv are 2d they can be transposed as you were expecting
switching back to a 1d vector is similar:
# from row vector
v = rv[0, :]
# from column vector
v = cv[:, 0]
I want to use numpy.ix_ to generate an multi-dimensional index for a 2D space of values. However, I need to use a subindex to look up the indices for one dimension. For example,
assert subindex.shape == (ny, nx)
data = np.random.random(size=(ny,nx))
# Generator returning the index tuples
def get_idx(ny,nx,subindex):
for y in range(ny):
for x in range(nx):
yi = y # This is easy
xi = subindex[y,x] # Get the second index value from the subindex
yield (yi,xi)
# Generator returning the data values
def get_data_vals(ny,nx,data,subindex):
for y in range(ny):
for x in range(nx):
yi = y # This is easy
xi = subindex[y,x] # Get the second index value from the subindex
yield data[y,subindex[y,x]]
So instead of the for loops above, I'd like to use a multi-dimensional index to index data Using numpy.ix_, I guess I would have something like:
idx = numpy.ix_([np.arange(ny), ?])
data[idx]
but I don't know what the second dimension argument should be. I'm guessing it should be something involving numpy.choose?
What you actually seem to want is:
y_idx = np.arange(ny)[:,np.newaxis]
data[y_idx, subindex]
BTW, you could achieve the same thing with y_idx = np.arange(ny).reshape((-1, 1)).
Let's look at a small example:
import numpy as np
ny, nx = 3, 5
data = np.random.rand(ny, nx)
subindex = np.random.randint(nx, size=(ny, nx))
Now
np.arange(ny)
# array([0, 1, 2])
are just the indices for the "y-axis", the first dimension of data. And
y_idx = np.arange(ny)[:,np.newaxis]
# array([[0],
# [1],
# [2]])
adds a new axis to this array (after the existing axis) and effectively transposes it. When you now use this array in an indexing expression together with the subindex array, the former gets broadcasted to the shape of the latter. So y_idx becomes effectively:
# array([[0, 0, 0, 0, 0],
# [1, 1, 1, 1, 1],
# [2, 2, 2, 2, 2]])
And now for each pair of y_idx and subindex you look up an element in the data array.
Here you can find out more about "fancy indexing"
It sounds like you need to do two things:
Find all indices into the data array and
Translate the column indices according to some other array, subindex.
The code below therefore generates indices for all array positions (using np.indices), and reshapes it to (..., 2) -- a 2-D list of coordinates representing each position in the array. For each coordinate, (i, j), we then translate the column coordinate j using the subindex array provided, and then use that translated index as the new column index.
With numpy, it is not necessary to do that in a for-loop--we can simply pass in all the indices at once:
i, j = np.indices(data.shape).reshape((-1, 2)).T
data[i, subindex[i, j]]
I have 2 arrays [nx1] that store xpixel (sample) and ypixel (line) coordinates, respectively. I have another array [nxn] storing an image. What I would like to do is create a third array which stores the pixel values from the image array at the given coordinates. I have this working with the following, but wonder if a built-in numpy function would be more efficient.
#Create an empty array to store the values from the image.
newarr = numpy.zeros(len(xsam))
#Iterate by index and pull the value from the image.
#xsam and ylin are the line and sample numbers.
for x in range(len(newarr)):
newarr[x] = image[ylin[x]][xsam[x]]
print newarr
A random generator determines the length of xsam and ylin along with the direction of travel through the image. It is therefore totally different with each iteration.
You can use advanced indexing:
In [1]: import numpy as np
In [2]: image = np.arange(16).reshape(4, 4)
In [3]: ylin = np.array([0, 3, 2, 2])
In [4]: xsam = np.array([2, 3, 0, 1])
In [5]: newarr = image[ylin, xsam]
In [6]: newarr
array([ 2, 15, 8, 9])
If image is a numpy array and ylin, xsam are one dimensional:
newarr = image[ylin, xsam]
If ylin, xsam are two-dimensional with the second dimension being 1 e.g., ylin.shape == (n, 1) then convert them to one-dimensional form first:
newarr = image[ylin.reshape(-1), xsam.reshape(-1)]