Is It possible to switch rows or columns between numpy arrays? - python

I have the following numpy arrays
[[[0 0 1 0 0]
[1 0 0 0 0]
[0 0 1 0 0]]
[[1 0 0 0 0]
[0 0 1 0 0]
[0 0 0 1 0]]]
am trying to switch rows between them, 1 row 2 rows it doesn't matter am trying to see if it's possible.
The output can be for the 1st row or 2nd row or 2 first rows respectively:
[[[0 0 1 0 0] [[[0 0 1 0 0] [[[1 0 0 0 0]x
[1 0 0 0 0] [0 0 1 0 0]x [0 0 1 0 0]x
[0 0 0 1 0]]x [0 0 1 0 0]] [0 0 1 0 0]]
[[1 0 0 0 0] [[1 0 0 0 0] [[0 0 1 0 0]x
[0 0 1 0 0] [1 0 0 0 0]x [1 0 0 0 0]x
[0 0 1 0 0]]]x [0 0 0 1 0]]] [0 0 0 1 0]]]
Is it possible? If so How?


You can switch values like rows on NumPy arrays with Python variable swap operator:
import numpy as np
m = np.array([[0, 0, 1, 0, 0],
[1, 0 ,0, 0, 0],
[0, 0, 1, 0, 0]])
n = np.array([[1, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[0, 0, 0, 1, 0]])
#m[:, 0], n[:, 0] = n[:, 0].copy(), m[:, 0].copy() #Only for columns
m[0], n[0] = n[0].copy(), m[0].copy() #For rows
print(m, n)
Output:
[[1 0 0 0 0]
[1 0 0 0 0]
[0 0 1 0 0]]
[[0 0 1 0 0]
[0 0 1 0 0]
[0 0 0 1 0]]

Related

Use Numpy arrays for large data file

I have a numpy array which is of the following type:-
[[[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
...
[[0 0 0 ... 0 0 0]
[0 0 1 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 1 1]
[0 0 0 ... 0 1 0]]
[[1 1 1 ... 0 0 0]
[1 1 0 ... 0 0 0]
[1 1 1 ... 0 0 0]
...
[0 1 1 ... 1 1 1]
[1 0 0 ... 0 0 0]
[1 0 0 ... 0 0 0]]]
[[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
...
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 1 1]
[0 0 0 ... 0 1 0]]
[[0 0 0 ... 1 1 0]
[0 0 0 ... 1 0 0]
[0 0 0 ... 1 0 0]
...
[0 0 0 ... 0 0 0]
[1 0 1 ... 0 0 0]
[0 0 0 ... 0 0 0]]]
[[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
...
[[0 0 0 ... 0 0 0]
[0 0 1 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 1 1]
[0 0 0 ... 0 0 0]]
[[1 1 1 ... 0 0 0]
[1 1 0 ... 1 1 1]
[0 1 1 ... 1 1 1]
...
[1 0 0 ... 0 0 0]
[1 0 0 ... 1 0 0]
[1 0 0 ... 0 1 0]]]
...
[[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 1 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 1 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
...
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 1 0 ... 0 0 0]
[1 1 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 1 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 1 0 ... 1 1 0]
...
[0 1 0 ... 0 0 0]
[0 1 0 ... 0 0 0]
[1 0 1 ... 1 1 1]]]
[[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 1 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 1 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
...
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[1 1 1 ... 1 1 0]
[1 1 0 ... 0 1 1]
[0 0 0 ... 1 0 1]
...
[0 0 1 ... 0 0 0]
[0 1 0 ... 0 0 0]
[1 0 0 ... 0 0 0]]
[[0 0 0 ... 1 0 0]
[1 1 1 ... 0 1 1]
[0 0 1 ... 1 0 1]
...
[1 0 0 ... 1 0 1]
[0 1 0 ... 1 0 0]
[0 0 0 ... 1 0 0]]]
[[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 1 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 1 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
...
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 1 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]
[[1 1 0 ... 0 0 0]
[1 1 0 ... 0 0 0]
[1 0 0 ... 0 0 0]
...
[1 0 0 ... 0 1 0]
[1 0 0 ... 1 1 0]
[0 1 1 ... 1 1 1]]
[[0 0 0 ... 0 0 0]
[0 1 0 ... 1 1 0]
[1 0 0 ... 0 1 0]
...
[0 0 0 ... 0 0 1]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]]]]
I want to extract only 1/4th of the array but the file is too big to be loaded in RAM. I have tried to use mmap but couldn't understand how to do it because it is giving completely different values than that of my numpy array. Even when I loaded the code using the mem map it gives wrong shape and values. I have noticed that on changing dtype the values also change and cannot understand it. My Method for mem map:-
load = np.memmap("dataset/b.npy", mode='r', shape=(14,8,8))

Numpy way to integer-mask an array

I have a multi-class segmentation mask
eg.
[1 1 1 2 2 2 2 3 3 3 3 3 3 2 2 2 2 4 4 4 4 4 4]
And going to need to get binary segmentation masks for each value
i.e.
[1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1]
Any elegant numpy way to do this?
Ideally an example, where I can set 0 and 1 to other values, if I have to.

Just do "==" as this
import numpy as np
a = np.array([1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 4, 4, 4, 4, 4, 4])
mask1 = (a==1)*5
mask2 = (a==2)*5
mask3 = (a==3)*5
mask4 = (a==4)*5
for mask in [mask1,mask2,mask3,mask4]:
print(mask)
This gives
[5 5 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 5 5 5 5 0 0 0 0 0 0 5 5 5 5 0 0 0 0 0 0]
[0 0 0 0 0 0 0 5 5 5 5 5 5 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 5 5]
You can manipulate the masks further in the same manner, i. e.
mask1[mask1==0] = 3

Native python approach:
You can use comprehension and get the equality values for each unique value using set(<sequence>), then convert the boolean to int to get 0,1 values.
>>> ls =[1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 4, 4, 4, 4, 4, 4]
>>> {v:[int(v==i) for i in ls] for v in set(ls)}
{1: [1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
2: [0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0],
3: [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
4: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1]}
Numpy approach:
Get the unique values for the list using np.unique then expand the axis and transpose the array, then expand the axis for the list also and repeat it n times where n is the number of unique values, finally do the equality comparison and convert it to integer type:
import numpy as np
ls = [1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 4, 4, 4, 4, 4, 4]
uniques = np.expand_dims(np.unique(ls), 0).T
result = (np.repeat(np.expand_dims(ls, 0), uniques.shape[0], 0)==uniques).astype(int)
OUTPUT:
print(result)
[[1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0]
[0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1]]

You can build the mask using np.arange and .repeat() and then use broadcasting and the == operator to generate the arrays:
a = np.array([1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 4, 4, 4, 4, 4, 4])
mask = np.arange(a.min(), a.max()+1).repeat(a.size).reshape(-1, a.size)
a_masked = (a == m).astype(int)
print(a_masked.shape) # (4, 23)
print(a_masked)
# [[1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
# [0 0 0 1 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0]
# [0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0]
# [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1]]
Setting 0 and 1 to other values can be done via normal indexing:
a_masked[a_masked == 0] = 7
a_masked[a_masked == 1] = 42

Is it possible to turn a 3D array to coordinate system?

Is it possible to take a 3D array and and turn it into a coordinate system? My array consists of 0s and 1s. If the value is 1 I want to take the xyz coordinate. In the end I want to output all coordinates to a csv file.
import nibabel as nib
coord = []
img = nib.load('test.nii').get_fdata().astype(int)
test.nii array:
[[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 1 ... 1 1 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 1 1 1]
[0 1 0 ... 0 0 0]]
[[1 0 0 ... 0 0 0]
[0 0 1 ... 0 0 0]
[0 1 0 ... 0 0 0]
...
[0 1 0 ... 0 0 0]
[0 1 0 ... 0 0 0]
[0 0 0 ... 1 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 1 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 1 0 ... 0 1 1]]
...
[[0 0 0 ... 1 0 0]
[0 0 1 ... 0 0 0]
[0 0 1 ... 0 0 0]
...
[0 0 0 ... 1 0 0]
[0 0 0 ... 1 0 0]
[0 0 0 ... 1 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 1]
...
[0 1 0 ... 0 0 0]
[1 0 0 ... 0 0 0]
[1 0 0 ... 0 0 0]]
[[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 0 0]
...
[0 0 0 ... 0 0 0]
[0 0 0 ... 0 1 0]
[0 1 0 ... 0 0 0]]]

That might not necessarily be the best solution, but let's keep it simple (would be great if framework did that for us, but...well):
data = [[[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 1, 1],
[0, 1, 0, 0, 0, 0]],
[[1, 0, 0, 0, 0, 0],
[0, 0, 1, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0]]]
for x in range(len(data)):
for y in range(len(data[x])):
for z in range(len(data[x][y])):
if data[x][y][z] == 1:
print(f"{x} {y} {z}")
yields:
0 2 2
0 2 3
0 2 4
0 4 3
0 4 4
0 4 5
0 5 1
1 0 0
1 1 2
1 2 1
1 3 1
1 4 1
1 5 3

Using np.where() you can get the row, col and depth index of elements that satisfy you condition.
Try this:
row_idx, col_idx, depth_idx = np.where(img==1)

Keep a single element in dataframe of lists

Given the following dataframe:
Movement Distance Speed Delay Loss
0 [1, 1] [1, 1] [25, 25] [0, 0] [0, 0]
1 [1, 1] [1, 1] [25, 25] [0, 0] [0, 0]
2 [1, 1] [1, 1] [25, 25] [0, 0] [0, 0]
3 [1, 1] [1, 1] [25, 25] [0, 0] [0, 0]
4 [1, 1] [1, 1] [25, 25] [0, 0] [0, 0]
How can I remove all but the first element in each column and then unlist so the dataframe becomes like this:
Movement Distance Speed Delay Loss
0 1 1 25 0 0
1 1 1 25 0 0
2 1 1 25 0 0
3 1 1 25 0 0
4 1 1 25 0 0

You can apply with str.slice or indexing equivalently as:
df.apply(lambda x: pd.to_numeric(x.str[0], downcast='integer', errors='ignore'))
Or if the data is already clean, we have convert_dtypes new in pandas 1.0 (thanks cs95):
df.apply(lambda x: x.str[0]).convert_dtypes()
Movement Distance Speed Delay Loss
0 1 1 25 0 0
1 1 1 25 0 0
2 1 1 25 0 0
3 1 1 25 0 0
4 1 1 25 0 0

All possible binary matrices with some properties

I need to generate all possible 4x4 binary matrices that have zeros along the main diagonal, are symmetric, and have six entries equal to 1. Some examples:
[[0,0,0,0],
[0,0,1,1],
[0,1,0,1],
[0,1,1,0]],
[[0,1,1,0],
[1,0,1,0],
[1,1,0,0],
[0,0,0,0]],
[[0,1,0,1],
[1,0,0,1],
[0,0,0,0],
[1,1,0,0]]
How could I do that in Python?

This amounts to choosing which three of the six entries
above the diagonal are 1.
From the list of above-the-diagonal positions in a 4 by 4 matrix:
sage: positions = [(0, 1), (0, 2), (0, 3), (1, 2), (1, 3), (2, 3)]
use Sage's Subsets to get all subsets of size 3 of those positions.
sage: S = Subsets([(0, 1), (0, 2), (0, 3), (1, 2), (1, 3), (2, 3)], 3)
Then build the corresponding matrices.
sage: [matrix(ZZ, 4, lambda i, j: (i, j) in s or (j, i) in s) for s in S]
[
[0 1 1 1] [0 1 1 0] [0 1 1 0] [0 1 1 0] [0 1 0 1] [0 1 0 1]
[1 0 0 0] [1 0 1 0] [1 0 0 1] [1 0 0 0] [1 0 1 0] [1 0 0 1]
[1 0 0 0] [1 1 0 0] [1 0 0 0] [1 0 0 1] [0 1 0 0] [0 0 0 0]
[1 0 0 0], [0 0 0 0], [0 1 0 0], [0 0 1 0], [1 0 0 0], [1 1 0 0],
[0 1 0 1] [0 1 0 0] [0 1 0 0] [0 1 0 0] [0 0 1 1] [0 0 1 1]
[1 0 0 0] [1 0 1 1] [1 0 1 0] [1 0 0 1] [0 0 1 0] [0 0 0 1]
[0 0 0 1] [0 1 0 0] [0 1 0 1] [0 0 0 1] [1 1 0 0] [1 0 0 0]
[1 0 1 0], [0 1 0 0], [0 0 1 0], [0 1 1 0], [1 0 0 0], [1 1 0 0],
[0 0 1 1] [0 0 1 0] [0 0 1 0] [0 0 1 0] [0 0 0 1] [0 0 0 1]
[0 0 0 0] [0 0 1 1] [0 0 1 0] [0 0 0 1] [0 0 1 1] [0 0 1 0]
[1 0 0 1] [1 1 0 0] [1 1 0 1] [1 0 0 1] [0 1 0 0] [0 1 0 1]
[1 0 1 0], [0 1 0 0], [0 0 1 0], [0 1 1 0], [1 1 0 0], [1 0 1 0],
[0 0 0 1] [0 0 0 0]
[0 0 0 1] [0 0 1 1]
[0 0 0 1] [0 1 0 1]
[1 1 1 0], [0 1 1 0]
]
Note that these are the adjacency matrices for all graphs
with three edges on four labeled vertices.
If you want un-labeled vertices, or equivalently the list
of adjacency matrices of equivalence classes of graphs
with three edges on four vertices, you could use Nauty
to enumerate them. Here is how to do that from Sage:
sage: G = graphs.nauty_geng("4 3:3")
sage: G
<generator object nauty_geng at 0x21c89a0f0>
sage: [g.adjacency_matrix() for g in G]
[
[0 0 0 1] [0 0 1 1] [0 0 1 1]
[0 0 0 1] [0 0 0 1] [0 0 0 0]
[0 0 0 1] [1 0 0 0] [1 0 0 1]
[1 1 1 0], [1 1 0 0], [1 0 1 0]
]

Categories

Resources