I have a sequence of N images with some shape (N, x,y). I also have corresponding times for each image, which is just a 1D array of length N.
Some of these times are duplicates, so I want to average the images at the same time steps so that I have a single (x,y) image for each time. I am curious what the best pythonic way for this would be?
Essentially just groupby("time").agg("mean"), but for 2D arrays.
If stick to groupby() paradigm, I would suggest the following:
import numpy as np
import pandas as pd
# the number of 512x512 gray-scale images
N = 200
# the iterator function that generates random images with associated random time points
# the images are numpy 2D arrays
def get_image():
yield [ np.random.randint(24*60*60), np.random.randint(256, size=(512, 512))]
# We generate the list of random images with associated random time points
my_images = [next(get_image()) for _ in range(N)]
df = pd.DataFrame(my_images)
df.columns = ['time_point', 'image']
df = df.sort_values('time_point').reset_index(drop=True)
# making at least some images have same time point as others
df.iloc[range(0,N,5),0] = df.iloc[range(1,N-1,5),0].values
#finally our groupby
result = df.groupby('time_point').mean()
# convert pixels back from floats to integers
result.loc['image'] = result['image'].apply(np.int64)
print(result)
Related
I have a segmentation map with 10 classes (A numpy array of size (m,n,1) which every element is a number from 1~10 specifying a class that the pixel belongs to). I want to convert it to an array of size (m,n,10) where each channel is mask for elements of that specific class. I can do it using a for loop like this:
for i in range(10):
mask[:,:,i] = (seg_map==i)[:,:,0]
but I need a faster way to do this. The for loop takes too much time. Is there any built in function that can outperform the for loop.
Thanks in advance.
One approach:
import numpy as np
np.random.seed(42)
# toy data
data = np.random.randint(0, 10, 20).reshape((5, 4, 1))
# https://stackoverflow.com/a/37323404/4001592
n_values = 10
values = data.flatten()
encoded = np.eye(n_values)[data.ravel()].reshape((5, 4, 10))
match = np.allclose(data.reshape(5, 4), encoded.argmax(-1))
print(match)
One way to verify that the output is correct is to verify that the one-hot encoded value matches back with the index, as below:
match = np.allclose(data.reshape(5, 4), encoded.argmax(-1))
print(match)
Output
True
I have a list of NumPy arrays, I want to apply rot90 and flip function randomly on it. So that in the end, I have a list where some arrays are as it is, and some are modified (with that two fuctions).
I directly pass that list of arrays to numpy.random.choice, it gives me the following error ValueError: a must be 1-dimensional.
thanks in advance.
One approach it to create a population of functions and pick randomly, using random.choice, the one to apply to each image:
import random
import numpy as np
# for reproducibility
random.seed(42)
np.random.seed(42)
# toy data representing the list of images
images = [np.random.randint(255, size=(128, 128)) for _ in range(10)]
functions = [lambda x: x, np.rot90, np.flip]
# pick function at random at apply to image
res = [random.choice(functions)(image) for image in images]
You can just sample indices and apply to the array at the respecting index. So here is an example of the basic idea:
import numpy as np
# generate some random list of arrays
l = [np.random.randint(0,10,(4,4)) for _ in range(10)]
# sample indices and apply rotation and flip
indices = np.random.choice(np.arange(len(l)),int(len(l)/2),replace=False)
new_l = [np.flip(np.rot90(l[i])) if i in indices else l[i] for i in range(len(l))]
Why don't you sample a list of indeces that needs to be modified?
In the following example, I have set:
A list of functions which could be applied transformations
If functions can be applied to the same only once (apply_only_once=True), or multiple applications are permitted (apply_only_once=False)
Number of lines which must be modified is n_lines_to_modify. Clearly, if apply_ony_once=True, n_lines_to_modify must be less or equal to the number of rows in the array; note that, if apply_only_once=False, n_lines_to_modify is not constrained, because multiple transformation can be applied to the same line (corner case: all the transformations applied to one line only!)
arrays is just a test input
In code:
import random
import numpy as np
transformations = [lambda x: x**2, lambda x: x+2]
apply_only_once = True
n_lines_to_modify = 2
arrays = np.array([np.array([1,2,3]), np.array([1,2,3]), np.array([3,4,5])])
if apply_only_once:
to_be_modified = random.sample(range(len(arrays)), n_lines_to_modify)
else:
to_be_modified = [random.choice(range(len(arrays))) for _ in range(n_lines_to_modify)]
for i in to_be_modified:
arrays[i] = random.choice(transformations)(arrays[i])
print(arrays)
I would like to create a NumPy array that all arrays are between -1 to 1. I want to sum all array that becomes zero.
Could you please tell me how can I create this NumPy array?
You can accomplish this by creating two arrays with the two values and concatenating them. To finish it all of you shuffle the final concatenated array.
import numpy as np
size = 10
array_size = int(size / 2)
ones = np.ones(array_size)
minus_ones = np.full(array_size, -1)
sum_zero = np.concatenate((ones, minus_ones))
np.random.shuffle(sum_zero)
print(sum_zero)
Hey guys Ii need help..
I want to use tensorflows data import, where data is loaded by calling the features/labels vectors from a structured numpy array.
https://www.tensorflow.org/programmers_guide/datasets#consuming_numpy_arrays
I want to create such an structured array by adding consecutively the 2 vectors (feature_vec and label_vec) to an numpy structured array.
import numpy as np
# example vectors
feature_vec= np.arange(10)
label_vec = np.arange(10)
# structured array which should get the vectors
struc_array = np.array([feature_vec,label_vec],dtype=([('features',np.float32), ('labels',np.float32)]))
# How can I add now new vectors to struc_array?
struc_array.append(---)
I want later when this array is loaded from file call either the feature vectors (which is a matrix now) by using the fieldname:
with np.load("/var/data/training_data.npy") as data:
features = data["features"] # matrix containing feature vectors as rows
labels = data["labels"] #matrix containing labels vectors as rows
Everything I tried to code was complete crap.. never got a correct output..
Thanks for your help!
Don't create a NumPy array and then append to it. That doesn't really make sense, as NumPy arrays have a fixed size and require a full copy to append a single row or column. Instead, create a list, append to it, then construct the array at the end:
vecs = [feature_vec,label_vec]
dtype = [('features',np.float32), ('labels',np.float32)]
# append as many times as you want:
vecs.append(other_vec)
dtype.append(('other', np.float32))
struc_array = np.array(vecs, dtype=dtype)
Of course, you probably need ot
Unfortunately, this doesn't solve the problem.
i want to get just the labels or the features from structured array by using:
labels = struc_array['labels']
features = struc_array['features']
But when i use the structured array like you did, labels and also features contains all given appended vectors:
import numpy as np
feature_vec= np.arange(10)
label_vec = np.arange(0,5,0.5)
vecs = [feature_vec,label_vec]
dtype = [('features',np.float32), ('labels',np.float32)]
other_vec = np.arange(6,11,0.5)
vecs.append(other_vec)
dtype.append(('other', np.float32))
struc_array = np.array(vecs, dtype=dtype)
# This contains all vectors.. not just the labels vector
labels = struc_array['labels']
# This also contains all vectors.. not just the feature vector
features = struc_array['features']
I am using numpy.random.shuffle to scramble an binary array (code below) but the output does not appear very random. I would expect a random assortment of dots but the resulting array appears to be a semi-regular pattern of dashes. What is going on here?
img = PIL.Image.open(image_path)
array = numpy.array(img)
# threshold image etc
outim=PIL.Image.fromarray(array)
outim.show() # generates left image (below)
numpy.random.shuffle(array)
outim=PIL.Image.fromarray(array)
outim.show() # generates right image (below)
You have shuffled the rows but not the columns. numpy.random.shuffle reorders its input along the first dimension only.
To shuffle the entire thing, try
shape = array.shape
array = array.flatten()
numpy.random.shuffle(array)
array = array.reshape(shape)