I am trying to process some CIFAR10 image data into image tiles for plotting with PIL on a black canvas. I have had success doing this while extracting features from a trained model but I keep getting IndexError: Tuple Index out of range when using Image.fromarray.
My features are shaped to be test data of shape (10000,3072). The data is 32x32x3 images.
I load cifar10 data and then flatten the data but keep getting this error.
Here is my code. Some of it is borrowed from https://medium.com/#pslinge144/representation-learning-cifar-10-23b0d9833c40
import numpy as np
from sklearn.manifold import TSNE
from time import time
from pathlib import Path
from PIL import Image
from time import time
from keras.datasets import cifar10
# Load the raw CIFAR-10 data
_, (X_test, y_test) = cifar10.load_data()
# normalize the xtest data
X_test = X_test.astype('float32')
X_test /= 255.0
features = X_test # this is (10000, 32, 32, 3) numpy array
features = np.reshape(features, (10000, 3072)) # flatten to 2d array
print(features.shape)
perplexities = [5, 30, 50, 100]
for perplexity in perplexities:
print("Starting t-SNE on images now!")
tsne = TSNE(n_components = 2, init = 'random', random_state = 0, perplexity = perplexity, learning_rate = 200).fit_transform(features)
tx, ty = tsne[:,0], tsne[:,1] # grab tsne first and 2nd dimensions
# min max normalize for plotting
tx = (tx-np.min(tx)) / (np.max(tx) - np.min(tx))
ty = (ty-np.min(ty)) / (np.max(ty) - np.min(ty))
width = 4000
height = 3000
max_dim = 100
full_image = Image.new('RGB', (width, height))
for idx, x in enumerate(features):
tile = Image.fromarray(np.uint8(x * 255), 'RGB') # rescale pixel values to [0,255] scale
rs = max(1, tile.width / max_dim, tile.height / max_dim)
tile = tile.resize((int(tile.width / rs),int(tile.height / rs)),Image.ANTIALIAS)
full_image.paste(tile, (int((width-max_dim) * tx[idx]),int((height-max_dim) * ty[idx])))
plots_output_path = Path('../data/processed/tSNE_plots').resolve()
filename = "tsne_perplex%d_plot.png" % (perplexity)
fullpath = plots_output_path.joinpath(filename).resolve()
full_image.save(str(fullpath))
Here is the error:
Traceback (most recent call last):
File "tSNE_image_thumbnail.py", line 80, in <module>
tSNE_image(x_test, 1000, 200, plots_output_path, 2)
File "tSNE_image_thumbnail.py", line 56, in tSNE_image
tile = Image.fromarray(np.uint8(x * 255), 'RGB')
File "/home/zw/src/image_classification_ML/venv/lib/python3.8/site-packages/PIL/Image.py", line 2728, in fromarray
size = shape[1], shape[0]
IndexError: tuple index out of range
Again, this code works fine when extracting my features from my CNN model and using it on a shape (10000, 512) dense layer. Not sure why this is giving me issues. Any ideas? Thanks in advance.
Your are providing arrays with the length of '3072' in your line
tile = Image.fromarray(np.uint8(x * 255), 'RGB')
Simply verify it by calling np.uint8(x * 255).shape for an x, which returns (3072,).
But for an 'RGB' image, you need a dimensionality of 3, not just 1.
Because of that, you get the error tuple index out of range, since an array with three entries and not just one is expected.
That means instead of your (3072,) you need a tuple with three entries, so for example (8,96,4), which would map your one-dimensional array of 3072 values to a matrix of 8 x 96 x 4 (=3072) values.
So you could change the line in your code to
tile = Image.fromarray(np.uint8(x).reshape(8,96,4),'RGB')
But at the end, you should define the shape according to the image dimensions.
Related
I am trying to use the following code for data generator to work with brain vessel defects segmentation. I have generated npy files for nifiti files. each npy files different dimensions [512,512,140] ,[560,560,141]. I am using the following code:
def load_img(img_dir, img_list):
images=[]
for i, image_name in enumerate(img_list):
if (image_name.split('.')[1] == 'npy'):
image = np.load(img_dir+image_name)
images.append(image)
images = np.array(images)
return(images)
def imageLoader(img_dir, img_list, mask_dir, mask_list, batch_size):
L = len(img_list)
#keras needs the generator infinite, so we will use while true
while True:
batch_start = 0
batch_end = batch_size
while batch_start < L:
limit = min(batch_end, L)
X = load_img(img_dir, img_list[batch_start:limit])
Y = load_img(mask_dir, mask_list[batch_start:limit])
yield (X,Y) #a tuple with two numpy arrays with batch_size samples
batch_start += batch_size
batch_end += batch_size
############################################
#Test the generator
from matplotlib import pyplot as plt
import random
train_img_dir = "/content/drive/MyDrive/input_data_512/train/images/"
train_mask_dir = "/content/drive/MyDrive/input_data_512/train/masks/"
train_img_list=os.listdir(train_img_dir)
train_mask_list = os.listdir(train_mask_dir)
batch_size = 2
train_img_datagen = imageLoader(train_img_dir, train_img_list,
train_mask_dir, train_mask_list, batch_size)
The error:
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:13: VisibleDeprecationWarning: Creating an ndarray from ragged nested
sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated.
If you meant to do this, you must specify 'dtype=object' when creating the ndarray
del sys.path[0]
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-6-ac15ea7a12c9> in <module>()
57
58 #Verify generator.... In python 3 next() is renamed as __next__()
---> 59 img, msk = train_img_datagen.__next__()
60
61
1 frames
<ipython-input-6-ac15ea7a12c9> in load_img(img_dir, img_list)
11
12 images.append(image)
---> 13 images = np.array(images)
14
15 return(images)
ValueError: could not broadcast input array from shape (512,512,100) into shape (512,512)
Your problem is with the line images = np.array(images)
The input is a list of differently shaped images. You try to convert it into a single higher-dimensional array. This cannot work. So what do you want to achieve?
From the looks of it, your input has shape (512, 512, 100) and (512, 512). What output shape do you want? Where should the pixels go? What you told numpy to do is create shape (2, 512, 512) but that obviously doesn't work.
What you could do is create (512, 512, 101). If that is desired, replace the offending line with images = np.dstack(images)
that worked thanks. However when I try to plot them I am getting this error: too many indices for array: array is 2-dimensional, but 3 were indexed
img, msk = train_img_datagen.__next__()
img_num = random.randint(0,img.shape[0]-1)
test_img=img[img_num]
test_mask=msk[img_num]
test_mask=np.argmax(test_mask, axis=2)
n_slice=random.randint(0, test_mask.shape[0])
plt.figure(figsize=(12, 8))
plt.subplot(221)
plt.imshow(test_img[:,:,n_slice], cmap='gray')
plt.title('Image flair')
plt.subplot(222)
plt.imshow(test_mask[:,:,n_slice],cmap='gray')
plt.title('Mask')
plt.show()
if I change the axis to 3 I get:
axis 3 is out of bounds for array of dimension 3
I'm struggling in creating a data generator in PyTorch to extract 2D images from many 3D cubes saved in .dat format
There is a total of 200 3D cubes each having a 128*128*128 shape. Now I want to extract 2D images from all of these cubes along length and breadth.
For example, a is a cube having size 128*128*128
So I want to extract all 2D images along length i.e., [:, i, :] which will get me 128 2D images along the length, and similarly i want to extract along width i.e., [:, :, i], which will give me 128 2D images along the width. So therefore i get a total of 256 2D images from 1 3D cube, and i want to repeat this whole process for all 200 cubes, there by giving me 51200 2D images.
So far I've tried a very basic implementation which is working fine but is taking approximately 10 minutes to run. I want you guys to help me create a more optimal implementation keeping in mind time and space complexity. Right now my current approach has a time complexity of O(n2), can we dec it further to reduce the time complexity
I'm providing below the current implementation
from os.path import join as pjoin
import torch
import numpy as np
import os
from tqdm import tqdm
from torch.utils import data
class DataGenerator(data.Dataset):
def __init__(self, is_transform=True, augmentations=None):
self.is_transform = is_transform
self.augmentations = augmentations
self.dim = (128, 128, 128)
seismicSections = [] #Input
faultSections = [] #Ground Truth
for fileName in tqdm(os.listdir(pjoin('train', 'seis')), total = len(os.listdir(pjoin('train', 'seis')))):
unrolledVolSeismic = np.fromfile(pjoin('train', 'seis', fileName), dtype = np.single) #dat file contains unrolled cube, we need to reshape it
reshapedVolSeismic = np.transpose(unrolledVolSeismic.reshape(self.dim)) #need to transpose the axis to get height axis at axis = 0, while length (axis = 1), and width(axis = 2)
unrolledVolFault = np.fromfile(pjoin('train', 'fault', fileName),dtype=np.single)
reshapedVolFault = np.transpose(unrolledVolFault.reshape(self.dim))
for idx in range(reshapedVolSeismic.shape[2]):
seismicSections.append(reshapedVolSeismic[:, :, idx])
faultSections.append(reshapedVolFault[:, :, idx])
for idx in range(reshapedVolSeismic.shape[1]):
seismicSections.append(reshapedVolSeismic[:, idx, :])
faultSections.append(reshapedVolFault[:, idx, :])
self.seismicSections = seismicSections
self.faultSections = faultSections
def __len__(self):
return len(self.seismicSections)
def __getitem__(self, index):
X = self.seismicSections[index]
Y = self.faultSections[index]
return X, Y
Please Help!!!
why not storing only the 3D data in mem, and let the __getitem__ method "slice" it on the fly?
class CachedVolumeDataset(Dataset):
def __init__(self, ...):
super(...)
self._volumes_x = # a list of 200 128x128x128 volumes
self._volumes_y = # a list of 200 128x128x128 volumes
def __len__(self):
return len(self._volumes_x) * (128 + 128)
def __getitem__(self, index):
# extract volume index from general index:
vidx = index // (128 + 128)
# extract slice index
sidx = index % (128 + 128)
if sidx < 128:
# first dim
x = self._volumes_x[vidx][:, :, sidx]
y = self._volumes_y[vidx][:, :, sidx]
else:
sidx -= 128
# second dim
x = self._volumes_x[vidx][:, sidx, :]
y = self._volumes_y[vidx][:, sidx, :]
return torch.squeeze(x), torch.squeeze(y)
I'm using PIL to load images and then transform them to NumPy arrays. Then I've to create a new image based on a list of images, so I append all theearrays to a list and then transform the list back to an array, so the shape for the list of images has 4 dimensions (n_images, height, width, rgb_channels). I'm using this code:
def gallery(array, ncols=4):
nindex, height, width, intensity = array.shape
nrows = nindex // ncols
# want result.shape = (height*nrows, width*ncols, intensity)
result = (array.reshape(nrows, ncols, height, width, intensity)
.swapaxes(1,2)
.reshape(height*nrows, width*ncols, intensity))
return result
def make_array(dim_x):
for i in range(dim_x):
print('series',i)
series = []
for j in range(TIME_STEP-1):
print('photo',j)
aux = np.asarray(Image.open(dirpath+'/images/pre_images /series_{0}_Xquakemap_{1}.jpg'.format(i,j)).convert('RGB'))
print(np.shape(aux))
series.append(aux)
print(np.shape(series))
im = Image.fromarray(gallery(np.array(series)))
im.save(dirpath+'/images/gallery/series_{0}_Xquakemap.jpg'.format(i))
im_shape = (im.size)
make_array(n_photos)
# n_photos is the total of photos in the dirpath
The problem is when the append on the series list happened, the shape of the image (the NumPy array added) gets lost. So when trying to reshape the array in the function gallery it causes a problem. A snippet of the output for the code above is this one:
...
series 2
photo 0
(585, 619, 3)
(1, 585, 619, 3)
photo 1
(587, 621, 3)
(2,)
photo 2
(587, 621, 3)
(3,)
photo 3
(587, 621, 3)
(4,)
...
As you can see, when appending the second photo the list loses a dimension. This is weird because the code works the first two iterations, which use fairly the same images. I tried using np.stack() but the error prevails.
I also find this issue on Github but I think it doesn't apply to this case even if the behavior is similar.
Working on Ubuntu 18, Python 3.7.3 and Numpy 1.16.2.
edit: added what #kwinkunks asked
In the second function, I think you need to move series = [] to before the outer loop.
Here's my reproduction of the problem:
import numpy as np
from PIL import Image
TIME_STEP = 3
def gallery(array, ncols=4):
"""Stitch images together."""
nindex, height, width, intensity = array.shape
nrows = nindex // ncols
result = array.reshape(nrows, ncols, height, width, intensity)
result = result.swapaxes(1,2)
result = result.reshape(height*nrows, width*ncols, intensity)
return result
def make_array(dim_x):
"""Make an image from a list of arrays."""
series = [] # <<<<<<<<<<< This is the line you need to check.
for i in range(dim_x):
for j in range(TIME_STEP - 1):
aux = np.ones((100, 100, 3)) * np.random.randint(0, 256, 3)
series.append(aux.astype(np.uint8))
im = Image.fromarray(gallery(np.array(series)))
return im
make_array(4)
This results in:
I know that I can rotate images in tensorflow using tf.contrib.image.rotate. But suppose I want to apply the rotation randomly at an angle between -0.3 and 0.3 in radians as follows:
images = tf.contrib.image.rotate(images, tf.random_uniform(shape=[batch_size], minval=-0.3, maxval=0.3, seed=mseed), interpolation='BILINEAR')
So far this will work fine. But the problem arises when the batch size changes on the last iteration and I got an error. So how to fix this code and make it work in all case scenarios? Please note that the inputs images are fed using tf.data.Dataset api.
Any help is much appreciated!!
You can't feed tf.contrib.image.rotate with an angles tensor.
But if you inspect the source code you can see it just makes a bunch of argument validations, and then:
image_height = math_ops.cast(array_ops.shape(images)[1],
dtypes.float32)[None]
image_width = math_ops.cast(array_ops.shape(images)[2],
dtypes.float32)[None]
output = transform(
images,
angles_to_projective_transforms(angles, image_height, image_width),
interpolation=interpolation)
tf.contrib.image.transform() receives a projective transform matrix.
tf.contrib.image.angles_to_projective_transforms() generates projective transforms from the rotation angles.
Both accept tensors as arguments, so you can just call the underlying functions.
Here is an example using MNIST
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
# load mnist
from tensorflow.examples.tutorials.mnist
import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot = True)
# Tensorflow random angle rotation
input_size = mnist.train.images.shape[1]
side_size = int(np.sqrt(input_size))
dataset = tf.placeholder(tf.float32, [None, input_size])
images = tf.reshape(dataset, (-1, side_size, side_size, 1))
random_angles = tf.random.uniform(shape = (tf.shape(images)[0], ), minval = -np
.pi / 4, maxval = np.pi / 4)
rotated_images = tf.contrib.image.transform(
images,
tf.contrib.image.angles_to_projective_transforms(
random_angles, tf.cast(tf.shape(images)[1], tf.float32), tf.cast(tf
.shape(images)[2], tf.float32)
))
# Run and Print
sess = tf.Session()
result = sess.run(rotated_images, feed_dict = {
dataset: mnist.train.images,
})
original = np.reshape(mnist.train.images * 255, (-1, side_size, side_size)).astype(
np.uint8)
rotated = np.reshape(result * 255, (-1, side_size, side_size)).astype(np.uint8)
# Print 10 random samples
fig, axes = plt.subplots(2, 10, figsize = (15, 4.5))
choice = np.random.choice(range(len(mnist.test.labels)), 10)
for k in range(10):
axes[0][k].set_axis_off()
axes[0][k].imshow(original[choice[k, ]], interpolation = 'nearest', \
cmap = 'gray')
axes[1][k].set_axis_off()
axes[1][k].imshow(rotated[choice[k, ]], interpolation = 'nearest', \
cmap = 'gray')
I'm using OpenCV to read images into numpy.array, and they have the following shape.
import cv2
def readImages(path):
imgs = []
for file in os.listdir(path):
if file.endswith('.png'):
img = cv2.imread(file)
imgs.append(img)
imgs = numpy.array(imgs)
return (imgs)
imgs = readImages(...)
print imgs.shape # (100, 718, 686, 3)
Each of the image has 718x686 pixels/dimension. There are 100 images.
I don't want to work on 718x686, I'd like to combine the pixels into a single dimension. That is, the shape should look like: (100,492548,3). Is there anyway either in OpenCV (or any other library) or Numpy that allows me to do that?
Without modifying your reading function:
imgs = readImages(...)
print imgs.shape # (100, 718, 686, 3)
# flatten axes -2 and -3, using -1 to autocalculate the size
pixel_lists = imgs.reshape(imgs.shape[:-3] + (-1, 3))
print pixel_lists.shape # (100, 492548, 3)
In case anyone wants it. Here's a general way of doing this
import functools
def combine_dims(a, i=0, n=1):
"""
Combines dimensions of numpy array `a`,
starting at index `i`,
and combining `n` dimensions
"""
s = list(a.shape)
combined = functools.reduce(lambda x,y: x*y, s[i:i+n+1])
return np.reshape(a, s[:i] + [combined] + s[i+n+1:])
With this function you could use it like this:
imgs = combine_dims(imgs, 1) # combines dimension 1 and 2
# imgs.shape = (100, 718*686, 3)
def combine_dims(a, start=0, count=2):
""" Reshapes numpy array a by combining count dimensions,
starting at dimension index start """
s = a.shape
return numpy.reshape(a, s[:start] + (-1,) + s[start+count:])
This function does what you need in a more general way.
imgs = combine_dims(imgs, 1) # combines dimension 1 and 2
# imgs.shape == (100, 718*686, 3)
It works by using numpy.reshape, which turns an array of one shape into an array with the same data but viewed as another shape. The target shape is just the initial shape, but with the dimensions to be combined replaced by -1. numpy uses -1 as a flag to indicate that it should work out itself how big that dimension should be (based on the total number of elements.)
This code is essentially a simplified version of Multihunter's answer, but my edit was rejected and hinted that it should be a separate answer. So there you go.
import cv2
import os
import numpy as np
def readImages(path):
imgs = np.empty((0, 492548, 3))
for file in os.listdir(path):
if file.endswith('.png'):
img = cv2.imread(file)
img = img.reshape((1, 492548, 3))
imgs = np.append(imgs, img, axis=0)
return (imgs)
imgs = readImages(...)
print imgs.shape # (100, 492548, 3)
The trick was to reshape and append to a numpy array. It's not good practice to hardcode the length of the vector (492548) so if I were you I'd also add a line that calculates this number and puts it in a variable, for use in the rest of the script.