Develop Reference

Numpy: Replace every n element in the first half of an array - python

If I have a numpy array and want to replace every nth element to 0 in the first half of the array( no change in the second half), how can I do this efficiently? Now my code is not efficient enough:
for i in xrange(1,half,n):
s[i] = 0

Just use a[:a.size//2:n] = 0. e.g.:
a = np.ones(10)
a[:a.size//2:2] = 0
a
array([ 0., 1., 0., 1., 0., 1., 1., 1., 1., 1.])
Another example:
a = np.ones(20)
n = 3
a[:a.size//2:n] = 0
a
array([ 0., 1., 1., 0., 1., 1., 0., 1., 1., 0., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1.])

You could slice the array by doing something like:
import numpy as np
# make an array of 11 elements filled with zeros
my_arr = np.zeros(11)
# get indexes to change in the array. range is: range(start, stop[, step])
a = range(0, 5, 2)
# print the original array
print my_arr
# Change the array
my_arr[a] = 1
# print the changes
print my_arr
Outputs:
array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
array([ 0., 1., 0., 1., 0., 0., 0., 0., 0., 0., 0.])

Related

I have a 2-dimentional tensor arr with 0 as all the entries. I have a second tensor idx. I want to make all entries in arr with the indices in idx into 1.
arr = torch.zeros(size = (2,10))
idx = torch.Tensor([
[0,2],
[4,5]
])
arr[idx] = 1 #This doesn't work
print(arr)
The output should look like this:
tensor([[1., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 1., 1., 0., 0., 0., 0.]])
I had high confidence that I would definitely find someone else ask this in SO, however I couldn't find one. I hope it isn't duplicate.

Use scatter() along dim=1 or the innermost dimension in this case i.e. dim=-1. Note that in place of src tensor, I just passed the constant value 1.
In [31]: arr = torch.zeros(size=(2, 10))
In [32]: idx = torch.tensor([
...: [0, 2],
...: [4, 5]
...: ])
In [33]: torch.scatter(arr, 1, idx, 1)
Out[33]:
tensor([[1., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 1., 1., 0., 0., 0., 0.]])
In [34]: torch.scatter(arr, -1, idx, 1)
Out[34]:
tensor([[1., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 1., 1., 0., 0., 0., 0.]])

Using DataArray objects in xarray what is the best way to find all cells that have values != 0.
For example in pandas I would do
df.loc[df.col1 > 0]
My specific example I'm trying to look at 3 dimensional brain imaging data.
first_image_xarray.shape
(140, 140, 96)
dims = ['x','y','z']
Looking at the documentation for xarray.DataArray.where it seems I want something like this:
first_image_xarray.where(first_image_xarray.y + first_image_xarray.x > 0,drop = True)[:,0,0]
But I still get arrays with zeros.
<xarray.DataArray (x: 140)>
array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., -0., 0., -0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
Dimensions without coordinates: x
Also - a side question - why are there some negative zeros? Are these values rounded and -0. is actually equal to something like -0.009876 or something?

(Answer to main question)
You are almost there. However, a slight syntax difference makes a big difference here. On one hand, here is the solution to filter >0 values using a "value-based" mask.
# if you want to DROP values which do not suffice a mask condition
first_image_xarray[:,0,0].where(first_image_xarray[:,0,0] > 0, drop=True)
or
# if you want to KEEP values which do not suffice a mask condition as nan
first_image_xarray[:,0,0].where(first_image_xarray[:,0,0] > 0, np.nan)
On the other hand, the reason why your attempt did not work as you hoped is because with first_image_xarray.x, it is referring to the index of elements in the array (in x direction) rather than referring to the value of the elements. Thus only the 1st element of your output should be nan instead of 0 because it only does not suffice the mask condition in slice [:,0,0]. Yes, you were creating an "index-based" mask.
The following small experiment (hopefully) articulates this critical difference.
Suppose we have DataArray which consists of only 0 and 1 (dimension is aligned with the original post (OP) of the question (140,140,96)). Firstly let's mask it based on index as OP did:
import numpy as np
import xarray as xr
np.random.seed(0)
# create a DataArray which randomly contains 0 or 1 values
a = xr.DataArray(np.random.randint(0, 2, 140*140*96).reshape((140, 140, 96)), dims=('x', 'y', 'z'))
# with this "index-based" mask, only elements where index of both x and y are 0 are replaced by nan
a.where(a.x + a.y > 0, drop=True)[:,0,0]
Out:
<xarray.DataArray (x: 140)>
array([ nan, 0., 1., 1., 0., 0., 0., 1., 0., 0., 0., 0.,
0., 1., 0., 1., 0., 1., 0., 0., 0., 1., 0., 0.,
1., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 0., 1., 1., 1., 1., 1., 1., 1., 0., 1.,
1., 0., 0., 0., 1., 1., 1., 0., 0., 1., 0., 0.,
1., 0., 1., 1., 0., 0., 1., 0., 0., 1., 1., 1.,
0., 0., 0., 1., 1., 0., 1., 0., 1., 1., 0., 0.,
0., 0., 1., 1., 0., 1., 1., 1., 1., 0., 1., 0.,
0., 0., 0., 0., 0., 0., 1., 0., 1., 1., 0., 0.,
0., 0., 1., 0., 1., 0., 0., 0., 0., 1., 0., 1.,
0., 0., 1., 0., 0., 0., 0., 0., 1., 1., 0., 0.,
0., 1., 0., 0., 1., 0., 0., 1.])
Dimensions without coordinates: x
With the mask above, only the element where index of both x and y are 0 turns in to nan and the rest has not been changed or dropped at all.
In contrast, the proposed solution masks the DataArray based on the values of DataArray elements.
# with this "value-based" mask, all the values which do not suffice the mask condition are dropped
a[:,0,0].where(a[:,0,0] > 0, drop=True)
Out:
<xarray.DataArray (x: 65)>
array([ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1.])
Dimensions without coordinates: x
This successfully dropped all the values which do not suffice a mask condition based on the values of DataArray elements.
(Answer to side question)
As for the origin of -0 and 0 in DataArray, rounded values from negative or positive side towards 0 would be the possibility: A related discussion was done here How to eliminate the extra minus sign when rounding negative numbers towards zero in numpy? The below is a tiny example of this case.
import numpy as np
import xarray as xr
xr_array = xr.DataArray([-0.1, 0.1])
# you can use either xr.DataArray.round() or np.round() for rounding values of DataArray
xr.DataArray.round(xr_array)
Out:
<xarray.DataArray (dim_0: 2)>
array([-0., 0.])
Dimensions without coordinates: dim_0
np.round(xr_array)
Out:
<xarray.DataArray (dim_0: 2)>
array([-0., 0.])
Dimensions without coordinates: dim_0
As a side note, the other possibility for getting -0 in NumPy array can be numpy.set_printoptions(precision=0), which hides below decimal point like below (but I know this is not the case this time since you are using DataArray):
import numpy as np
# default value is precision=8 in ver1.15
np.set_printoptions(precision=0)
np.array([-0.1, 0.1])
Out:
array([-0., 0.])
Anyway, My best guess is that the conversion to -0 should be manual and intentional rather than automatic in data preparation & pre-processing phase.
Hope this helps.

I have an array of zeros
arr = np.zeros([5,5])
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
I want to assign values based on index so I did this .
out = np.array([[nan,2.,4.,1.,1.],[nan,3.,4.,4.,4.]])
arr[out[0].astype(int),np.arange(len(out[0]))] = 1
arr[out[1].astype(int),np.arange(len(out[1]))] = 1
Assignment works fine if there is 0 instead of nan.
How can I skip assignment in case of nan? and Is it possible to assign values at once from a multidimensional index array rather than using for loop ?

Mask it -
mask = ~np.isnan(out)
arr[out[0,mask[0]].astype(int),np.flatnonzero(mask[0])] = 1
arr[out[1,mask[1]].astype(int),np.flatnonzero(mask[1])] = 1
Sample run -
In [171]: out
Out[171]:
array([[ nan, 2., 4., 1., 1.],
[ nan, 3., 4., 4., 4.]])
In [172]: mask = ~np.isnan(out)
...: arr[out[0,mask[0]].astype(int),np.flatnonzero(mask[0])] = 1
...: arr[out[1,mask[1]].astype(int),np.flatnonzero(mask[1])] = 1
...:
In [173]: arr
Out[173]:
array([[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 1., 1.],
[ 0., 1., 0., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 1., 1., 1.]])
Alternative, replace the flatnonzero calls with range-masking -
r = np.arange(arr.shape[1])
arr[out[0,mask[0]].astype(int),r[mask[0]]] = 1
arr[out[1,mask[1]].astype(int),r[mask[1]]] = 1
If you are working with a lot many rows than just 2 and you want to assign them in a vectorized manner, here's one method, using linear-indexing -
n = arr.shape[1]
linear_idx = (out*n + np.arange(n))
np.put(arr, linear_idx[~np.isnan(linear_idx)].astype(int), 1)

I want to fill a matrix from an array of indices :
import numpy as np
indx = [[0,1,2],[1,2,4],[0,1,3],[2,3,4],[0,3,4]]
x = np.zeros((5,5))
for i in range(5):
x[i,indx[i]] = 1.
The result is :
array([[ 1., 1., 1., 0., 0.],
[ 0., 1., 1., 0., 1.],
[ 1., 1., 0., 1., 0.],
[ 0., 0., 1., 1., 1.],
[ 1., 0., 0., 1., 1.]])
As desired.
Question
Is there a way to do this in pure python/numpy without looping ?

Use advanced-indexing after intialization -
x[np.arange(len(indx))[:,None], indx] = 1

Given a numpy array of zeros, say
arr = np.zeros((5, 5))
and an array of indices that represent vertices of a polygon, say
verts = np.array([[0, 2], [2, 0], [2, 4]])
1) What is the elegant way of doing
for v in verts:
arr[v[0], v[1]] = 1
such that the resulting array is
In [108]: arr
Out[108]:
array([[ 0., 0., 1., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 1., 0., 0., 0., 1.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])
2) How can I fill the array with ones such that the output array is
In [158]: arr
Out[158]:
array([[ 0., 0., 1., 0., 0.],
[ 0., 1., 1., 1., 0.],
[ 1., 1., 1., 1., 1.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]])

To answer the first part of your question: arr[tuple(verts.T)] = 1
verts.T transposes your indices to a (2, n) array, where the two rows correspond to the row and column dimensions of arr. These are then unpacked into a tuple of (row_indices, col_indices), which we then use to index into arr.
We could write this a bit more verbosely as:
row_indices = verts[:, 0]
col_indices = verts[:, 1]
arr[row_indices, col_indices] = 1
For the second part, one method that will work for arbitrary polygons would be to use matplotlib.Path.contains_points, as described here:
from matplotlib.path import Path
points = np.indices(arr.shape).reshape(2, -1).T
path = Path(verts)
mask = path.contains_points(points, radius=1e-9)
mask = mask.reshape(arr.shape).astype(arr.dtype)
print(repr(mask))
# array([[ 0., 0., 1., 0., 0.],
# [ 0., 1., 1., 1., 0.],
# [ 1., 1., 1., 1., 1.],
# [ 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0.]])