I am trying to build a CNN using Keras for an image segmentation task, based on this article. Because my dataset is small, I wanted to use Keras ImageDataGenerator and feed it to fit_generator(). So, I followed the example on the Keras website. But, since zipping the image and mask generators didn't work, I followed this answer and created my own generator.
My input data is of size (701,256,1) and my problem is binary (foreground, background). For each image I have a label of the same shape.
Now, I am facing a dimensionality problem. This was also mentioned in the answer, but I am unsure of how to solve it.
The error:
ValueError: Error when checking target: expected dense_3 to have 2 dimensions, but got array with shape (2, 704, 256, 1)
I am pasting the entire code I have here:
import numpy
import pygpu
import theano
import keras
from keras.models import Model, Sequential
from keras.layers import Input, Dense, Dropout, Activation, Flatten
from keras.layers import Conv2D, MaxPooling2D, Reshape
from keras.layers import BatchNormalization
from keras.preprocessing.image import ImageDataGenerator
from keras.utils import np_utils
from keras import backend as K
def superGenerator(image_gen, label_gen):
while True:
x = image_gen.next()
y = label_gen.next()
yield x[0], y[0]
img_height = 704
img_width = 256
train_data_dir = 'Dataset/Train/Images'
train_label_dir = 'Dataset/Train/Labels'
validation_data_dir = 'Dataset/Validation/Images'
validation_label_dir = 'Dataset/Validation/Labels'
n_train_samples = 1000
n_validation_samples = 500
epochs = 50
batch_size = 2
input_shape = (img_height, img_width,1)
target_shape = (img_height, img_width)
model = Sequential()
model.add(Conv2D(80,(28,28), input_shape=input_shape))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))
model.add(Conv2D(96,(18,18)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))
model.add(Conv2D(128,(13,13)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(160,(8,8)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
model.add(Dense(1024, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(2, activation='softmax'))
model.summary()
model.compile(loss='binary_crossentropy', optimizer='nadam', metrics=['accuracy'])
data_gen_args = dict(
rescale=1./255,
horizontal_flip=True,
vertical_flip=True
)
train_datagen = ImageDataGenerator(**data_gen_args)
train_label_datagen = ImageDataGenerator(**data_gen_args)
test_datagen = ImageDataGenerator(**data_gen_args)
test_label_datagen = ImageDataGenerator(**data_gen_args)
seed = 1
train_image_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=target_shape,
color_mode='grayscale',
batch_size=batch_size,
class_mode = 'binary',
seed=seed)
train_label_generator = train_label_datagen.flow_from_directory(
train_label_dir,
target_size=target_shape,
color_mode='grayscale',
batch_size=batch_size,
class_mode = 'binary',
seed=seed)
validation_image_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=target_shape,
color_mode='grayscale',
batch_size=batch_size,
class_mode = 'binary',
seed=seed)
validation_label_generator = test_label_datagen.flow_from_directory(
validation_label_dir,
target_size=target_shape,
color_mode='grayscale',
batch_size=batch_size,
class_mode = 'binary',
seed=seed)
train_generator = superGenerator(train_image_generator, train_label_generator,batch_size)
test_generator = superGenerator(validation_image_generator, validation_label_generator,batch_size)
model.fit_generator(
train_generator,
steps_per_epoch= n_train_samples // batch_size,
epochs=50,
validation_data=test_generator,
validation_steps=n_validation_samples // batch_size)
model.save_weights('first_try.h5')
I am new to Keras (and CNNs), so any help would be very much appreciated.
Ok. I did some rubberduck-debugging and read a few more articles. Of course the dimensionality was a problem. This simple answer did it for me.
My labels are of shape same as the input image so the output of the model should be of that shape as well. I used Conv2DTranspose to solve this issue.
Related
I am doing image classification with ImageDataGenerator. My data has this structure:
Train
101
102
103
104
Test
101
102
103
104
So, if I understood good, the ImageGenerator automatically does what is needed with labeling.
I train the model, and I get some kind of accuracy. Now I want to do the prediction.
- model.predict
- model.predict_proba
- model.predict_classes
All these give me the same value. Can you quickly explain or refer(I cannot find anything concerning my problem) how I should proceed, or maybe I did something terrible in the code. The biggest problem, I don't understand how the output will differ for 4 different classes. As predict_classes gives me an output [[1]], should not it give me the predicted class?
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, MaxPool2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.regularizers import l1, l2, l1_l2
model = Sequential()
model.add(Conv2D(60, (3, 3), input_shape=(480, 640,3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(60, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(100, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten()) # this converts our 3D feature maps to 1D feature vectors
model.add(Dense(100, activation='relu', activity_regularizer=l1(0.001)))
#model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('softmax'))
model.compile(loss='binary_crossentropy',
optimizer='Adam',
metrics=['accuracy'])
batch_size = 32
# augmentation configuration for train
train_datagen = ImageDataGenerator(
rotation_range=30,
width_shift_range=0.2,
height_shift_range=0.2,
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=False,
vertical_flip=True,
fill_mode = 'nearest')
# augmentation configuration for testing, only rescale
test_datagen = ImageDataGenerator(rescale=1./255)
# reading pictures and generating batches of augmented image data
train_generator = train_datagen.flow_from_directory(
'/media/data/working_dir/categories/readytotest/train',
target_size=(480, 640),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
'/media/data/working_dir/categories/readytotest/test',
target_size=(480, 640),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=800 // batch_size,
epochs=15,
validation_data=validation_generator,
validation_steps=800 // batch_size)
Your model and the generators not for multi class but binary classification. First you need to fix your model last layer to get output with class size. Second you need to fix the generators to use in multi class.
(...)
model.add(Dense(CLS_SZ))
model.add(Activation('softmax'))
(...)
# I am not sure about this read some docs about generator you used.
train_generator = train_datagen.flow_from_directory(
'/media/data/working_dir/categories/readytotest/train',
target_size=(480, 640),
batch_size=batch_size,
class_mode=None)
validation_generator = test_datagen.flow_from_directory(
'/media/data/working_dir/categories/readytotest/test',
target_size=(480, 640),
batch_size=batch_size,
class_mode=None)
from keras import *
import os
import numpy as np
from keras.models import Sequential
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.preprocessing.image import ImageDataGenerator
from keras.layers import Convolution2D, MaxPooling2D, ZeroPadding2D
from keras import optimizers
# Collecting data:
img_width, img_height = 150, 150
training_data_dir = "train"
testing_data_dir = "test"
batch_size = 16
# prepare data augmentation configuration
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
training_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
testing_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
# Building model:
model = Sequential()
model.add(Convolution2D(32, (3, 3), input_shape=(img_width, img_height,3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Convolution2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Convolution2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss="binary_crossentropy",
optimizer="rmsprop",
metrics=["accuracy"])
# Training model:
nb_epoch = 1
nb_train_samples = 2048
nb_validation_samples = 832
model.fit_generator(
train_generator,
samples_per_epoch=nb_train_samples,
epochs=nb_epoch,
validation_data=validation_generator,
validation_steps=nb_validation_samples,
steps_per_epoch=64)
My code here creates a neural network for image classification based on pictures it is trained on, I have searched all over the internet but one thing I can't understand is how to input my own image file to test against the network and for it to print out the output. For example if the net was for classifying dogs and cats, I'm not sure on how to code the bit where I input a jpg/png file for a dog or a cat and the program to output which class it is. Help please?
You call the model's predict method. https://keras.io/models/model/#predict
I trained a binary classifier distinguish clear MNIST images from blurry images. All images are 28*28*1 grayscale digits and I have 40000 for training, 10000 for validating and 8000 for testing. My code looks like:
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
import cv2
import numpy as np
import glob
from PIL import Image
img_width, img_height = 28, 28#all MNIST images are of size (28*28)
train_data_dir = '/Binary Classifier/data/train'#train directory generated by train_cla
validation_data_dir = '/Binary Classifier/data/val'#validation directory generated by val_cla
train_samples = 40000
validation_samples = 10000
epochs = 20
batch_size = 16
if K.image_data_format() == 'channels_first':
input_shape = (1, img_width, img_height)
else:
input_shape = (img_width, img_height, 1)
#build a sequential model to train data
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
train_datagen = ImageDataGenerator(#train data generator
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
val_datagen = ImageDataGenerator(rescale=1. / 255)#validation data generator
train_generator = train_datagen.flow_from_directory(#train generator
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary',color_mode = 'grayscale')
validation_generator = val_datagen.flow_from_directory(#validation generator
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary',color_mode = 'grayscale')
model.fit_generator(#fit the generator to train and validate the model
train_generator,
steps_per_epoch=train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=validation_samples // batch_size)
#model.save_weights('output.h5')#save the output as HDF5 file
filelist = glob.glob('/Binary Classifier/data/image_data/*.png')
x = np.array([np.array(Image.open(fname)) for fname in filelist])
x = np.expand_dims(x, axis=3)
ones=model.predict(x)
But my output prediction in ones[] are all [1.] while the accuracy in training is actually really high(almost perfect). Does anyone know why?
Edit: I think I may get more help if I can show my image data. Basically the MNIST image in the directory is either a (clear) or a (blurry). All are (28*28*1) grayscale images whose format is .png. There are 40000 digits in '/Binary Classifier/data/train' for training, 10000 digits in '/Binary Classifier/data/val' for validation and 58000 digits in '/Binary Classifier/data/image_data/ for testing.
Some suggestions:
Pull data directly from one of your generators and test on that. Treat the generator like you would a list in a for loop to get image/label pairs out. This should sort out any differences in the way you are obtaining data and its formatting (e.g. channel order).
Check how many examples you have in each subdirectory of train/ and val/.
Change your metric to binary_accuracy since you are posing the problem as a binary classification problem (network only has one output).
I've been following the tutorial here to process images of cats, and see if a specific picture contains a cat. The data set I use is here. Is there something missing in the way I read in an image for testing? In my results from model.predict(filePath), I always get the value '[[0.]]' when reading an image containing a cat. The train and validation sets seem to work correctly. I am only having issues reading in an image. (Source code is copied from here)
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
import numpy as np
from keras.preprocessing import image
# dimensions of our images.
img_width, img_height = 150, 150
train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
model.save_weights('first_try.h5')
def _LoadImage(filePath):
test_image = image.load_img(filePath, target_size = (150,150))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis=0)
return test_image
test_this = _LoadImage('test.jpg')
result = model.predict(test_this)
print(result)
Looks like "0" is the label of cat ("The training archive contains 25,000 images of dogs and cats. Train your algorithm on these files and predict the labels for test1.zip (1 = dog, 0 = cat)."), so your model predictions seem to be correct. Remember that the model is predicting (cat and dog) labels and not what class string you might be corresponding with the labels yourself. Try feeding an image of a dog and you should get "1" in return.
I am following this guide as a start to train a model using some cats and dogs images:
https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html
This is the code:
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
# dimensions of our images.
img_width, img_height = 150, 150
train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
epochs = 1
batch_size = 16
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)
# this is a generator that will read pictures found in
# subfolers of 'data/train', and indefinitely generate
# batches of augmented image data
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
# this is a similar generator, for validation data
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
model.save_weights('first_try.h5')
with open('model.json', 'w') as f:
f.write(model.to_json())
So I get two files: first_try.h5 and model.json.
Now I want to try to do a simple image prediction using a sample dog.jpg and a cat.jpg. This is what I tried:
from keras.models import Sequential
from keras.layers import Dense
from keras.models import model_from_json
from PIL import Image
import cv2, numpy as np
# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)
# load weights into new model
loaded_model.load_weights("first_try.h5")
print("Loaded model from disk")
#attempt 1
img = cv2.resize(cv2.imread('cat.jpg'), (150, 150))
mean_pixel = [103.939, 116.779, 123.68]
img = img.astype(np.float32, copy=False)
for c in range(3):
img[:, :, c] = img[:, :, c] - mean_pixel[c]
img = img.transpose((2,0,1))
img = np.expand_dims(img, axis=0)
out1 = loaded_model.predict(img)
print(np.argmax(out1))
#attempt 2
loaded_model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
img = Image.open('dog.jpg')
img = img.convert('RGB')
x = np.asarray(img, dtype='float32')
x = x.transpose(2, 0, 1)
x = np.expand_dims(x, axis=0)
out1 = loaded_model.predict(x)
print(np.argmax(out1))
I get this output:
Using Theano backend.
Loaded model from disk
0
0
Can someone guide me? How to do a model.predict correctly?
I would suggest you use (https://keras.io/getting-started/faq/#how-can-i-save-a-keras-model):
from keras.models import load_model
model.save('model.hdf5')
model = load_model('model.hdf5')
Anyways, what makes you think that this is not the correct output? You do the argmax on 1 value. This is naturally the index 0. If you want the final output of the last layer remove the argmax and then you get a probability.