I created a CNN model for binary classification. I used a balanced database of 300 images. I know it's a small database but I used data augmentation. After fitting the model I got 86% val_accuracy on the validation set, but when I wanted to print the probability for each picture, I got probability 1 for most pictures from the first class and even all probabilities are >0.5, and probability 1 for all images from the second class.
This is my model:
model = keras.Sequential([
layers.InputLayer(input_shape=[128, 128, 3]),
preprocessing.Rescaling(scale=1/255),
preprocessing.RandomContrast(factor=0.10),
preprocessing.RandomFlip(mode='horizontal'),
preprocessing.RandomRotation(factor=0.10),
layers.BatchNormalization(renorm=True),
layers.Conv2D(filters=64, kernel_size=3, activation='relu', padding='same'),
layers.MaxPool2D(),
layers.BatchNormalization(renorm=True),
layers.Conv2D(filters=128, kernel_size=3, activation='relu', padding='same'),
layers.MaxPool2D(),
layers.BatchNormalization(renorm=True),
layers.Conv2D(filters=256, kernel_size=3, activation='relu', padding='same'),
layers.Conv2D(filters=256, kernel_size=3, activation='relu', padding='same'),
layers.MaxPool2D(),
layers.BatchNormalization(renorm=True),
layers.Flatten(),
layers.Dense(8, activation='relu'),
layers.Dense(1, activation='sigmoid'),])
Edit:
model.compile(
optimizer=tf.keras.optimizers.Adam(),
loss='binary_crossentropy',
metrics=['binary_accuracy'],
)
history = model.fit(
ds_train,
validation_data=ds_valid,
epochs=50,
)
Thank you.
A pre-trained model like vgg16 does all the work pretty much well, there is no need to complicate very much the model. So try the following code:
base_model = keras.applications.VGG16(
weights='imagenet',
input_shape=(128, 128, 3),
include_top=False)
base_model.trainable = True
inputs = keras.Input(shape=(128, 128, 3))
x = base_model(inputs, training=False)
x = keras.layers.GlobalAveragePooling2D()(x)
outputs = keras.layers.Dense(1)(x)
model = keras.Model(inputs, outputs)
Set base_model.trainable to False if you want the model to train fast and True for
more accurate results.
Notice that I used the GlobalAveragePooling2D layer, instead of Flatten, to reduce the number of parameters and to unstack the features.
I am trying to create an image classifier using Keras and TensorFlow 2.0.0 backend.
I'm training this model on my local machine on a custom dataset containing a total of 17~ thousand images. The images vary in size and are located in three different folders (training, validation, and test), each containing two subfolders (one for each class).
I tried an architecture similar to VGG16, which yielded more than decent results on this dataset in the past. Note, there is a minor class imbalance in the data (52:48)
When I call fit_generator(), the model doesn't train well; although the training loss lowers slightly throughout the first epoch, it does not change much afterward. Using this architecture with higher regulation, I achieved 85% accuracy after 55~ epochs in the past.
Imports and hyperparameters
import tensorflow as tf
from tensorflow import keras
from keras import backend as k
from keras.layers import Dense, Dropout, Conv2D, MaxPooling2D, Flatten, Input, UpSampling2D
from keras.models import Sequential, Model, load_model
from keras.utils import to_categorical
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import ModelCheckpoint
TRAIN_PATH = 'data/train/'
VALID_PATH = 'data/validation/'
TEST_PATH = 'data/test/'
TARGET_SIZE = (256, 256)
RESCALE = 1.0 / 255
COLOR_MODE = 'grayscale'
EPOCHS = 2
BATCH_SIZE = 16
CLASSES = ['Damselflies', 'Dragonflies']
CLASS_MODE = 'categorical'
CHECKPOINT = "checkpoints/weights.hdf5"
Model
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu',
input_shape=(256, 256, 1), padding='same'))
model.add(Conv2D(32, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.1))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.1))
model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.1))
model.add(Flatten())
model.add(Dense(516, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(2, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='Adam', metrics=['accuracy'])
In the past, I created a custom pipeline to reshape, grayscale, flip, and normalize the images; then, I trained the model using my CPU on batches of processed images.
I tried repeating the process using ImageDataGenerator, flow_from_directory, and GPU support.
# randomly flip images, and scale pixel values
trainGenerator = ImageDataGenerator(rescale=RESCALE,
horizontal_flip=True,
vertical_flip=True)
# only scale the pixel values validation images
validatioinGenerator = ImageDataGenerator(rescale=RESCALE)
# only scale the pixel values test images
testGenerator = ImageDataGenerator(rescale=RESCALE)
# instanciate train flow
trainFlow = trainGenerator.flow_from_directory(
TRAIN_PATH,
target_size = TARGET_SIZE,
batch_size = BATCH_SIZE,
classes = CLASSES,
color_mode = COLOR_MODE,
class_mode = CLASS_MODE,
shuffle=True
)
# instanciate validation flow
validationFlow = validatioinGenerator.flow_from_directory(
VALID_PATH,
target_size = TARGET_SIZE,
batch_size = BATCH_SIZE,
classes = CLASSES,
color_mode = COLOR_MODE,
class_mode= CLASS_MODE,
shuffle=True
)
Then, fitting the model using fit_generator.
checkpoints = ModelCheckpoint(CHECKPOINT, monitor='val_accuracy', verbose=1, save_best_only=True, mode='max')
with tf.device('/GPU:0'):
model.fit_generator(
trainFlow,
validation_data=validationFlow,
callbacks=[checkpoints],
epochs=EPOCHS
)
I tried training it for 40 epochs.
The classifier achieves 52% after the first epoch and does not improve as time goes by.
Testing the classifier
testFlow = testGenerator.flow_from_directory(
TEST_PATH,
target_size = TARGET_SIZE,
batch_size = BATCH_SIZE,
classes = CLASSES,
color_mode = COLOR_MODE,
class_mode= CLASS_MODE,
)
ans = model.predict_generator(testFlow)
When I look at the predictions, the model predicts all the test images as the majority class with the same confidence [0.48498476, 0.51501524].
Have I made sure the data is correct?
Yes. I tested whether the generators yield processed images and their corresponding labels correctly.
Have I tried changing the loss function, activation function, and optimizer?
Yes. I tried changing the class mode to binary, the loss to binary_crossentropy, and changing the last layer to produce a single output with sigmoid activation. No, I did not change the optimizer. However, I did try to increase the learning rate.
Have I tried changing the model's architecture?
Yes. I tried increasing and decreasing model complexity.
Both more layers with less regularization and fewer layers with more regularization produced similar results.
Are the layers trainable?
Yes.
Is the GPU support implemented correctly?
I hope so.
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))
Num GPUs Available: 1
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
config = tf.compat.v1.ConfigProto(log_device_placement=True)
config.gpu_options.allow_growth = True
sess = tf.compat.v1.Session(config=config)
print(sess)
Device mapping:
/job:localhost/replica:0/task:0/device:GPU:0 -> device: 0, name: NVIDIA GeForce GTX 1050 with Max-Q Design, pci bus id: 0000:03:00.0, compute capability: 6.1
<tensorflow.python.client.session.Session object at 0x000001F9443E2CC0>
Have I tried transfer learning?
Not yet.
I found a similar unanswered question from 2017 keras-doesnt-train-using-fit-generator.
Thoughts?
The problem is with your model. I copied your code and ran it on a data set I have used before (which gets high accuracy) and got results similar to yours. I then substituted the simple model below
model = tf.keras.Sequential([
Conv2D(16, 3, padding='same', activation='relu', input_shape=(256 , 256,1)),
MaxPooling2D(),
Conv2D(32, 3, padding='same', activation='relu' ),
MaxPooling2D(),
Conv2D(64, 3, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(128, 3, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(256, 3, padding='same', activation='relu'),
MaxPooling2D(),
Flatten(),
Dense(128, activation='relu'),
Dropout(.3),
Dense(64, activation='relu'),
Dropout(.3),
Dense(2, activation='softmax')
])
model.compile(loss='categorical_crossentropy',
optimizer='Adam', metrics=['accuracy'])
The model trained properly. By the way model.fit_generator is depreciated. You can now just use model.fit which can now handle generators. I then took your model and removed all the dropout layers except for the last one and your model trained properly. Code is:
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu',
input_shape=(256, 256, 1), padding='same'))
model.add(Conv2D(32, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
#model.add(Dropout(0.1))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
#model.add(Dropout(0.1))
model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
model.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2, 2)))
#model.add(Dropout(0.1))
model.add(Flatten())
model.add(Dense(516, activation='relu'))
#model.add(Dropout(0.1))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(2, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='Adam', metrics=['accuracy'])
#Gerry P,
By accident, I found what's causing the error.
Removing from Keras import backend as k resolved the model's inability to learn.
That's not all. I also identified that the model you defined, not calling ModelCheckpoint, and not customizing class names affected the fitting process.
model = Sequential([
Conv2D(16, 3, padding='same', activation='relu', input_shape=(256 , 256, 1)),
MaxPooling2D(),
Conv2D(32, 3, padding='same', activation='relu' ),
MaxPooling2D(),
Conv2D(64, 3, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(128, 3, padding='same', activation='relu'),
MaxPooling2D(),
Conv2D(256, 3, padding='same', activation='relu'),
MaxPooling2D(),
Flatten(),
Dense(128, activation='relu'),
Dropout(.3),
Dense(64, activation='relu'),
Dropout(.3),
Dense(2, activation='softmax')
])
I commented that import to try and resolve an error that occurred when I copy-pasted your sequential model. Then, I forgot to uncomment it when I tested it beautiful or average dataset. I achieved over 80% accuracy after the third epoch. Then, I reverted the changes and tried it on my dataset, and it failed again.
As a bonus, not importing Keras's backend decreased the time it takes to train the model!
Lately, I had to re-install Keras and TensorFlow because they couldn't detect my GPU anymore. I probably made a mistake and installed an incompatible version of Keras.
CUDA==10.0
tensorflow-gpu==2.0.0
keras==2.3.1
Note, it's still not a 100% solution, and the problems arise every so often.
EDIT:
Whenever it doesn't work, simplify the model.
Changed batch size and stopped learning? Simplify the model.
Augmented the images further and stopped learning? Simplify the model.
I am new to neural networks. I am trying to build CON2D + LSTM to recognize gestures from images. I have written below code:
input_shape = (120, 120, 3) // Having 120x120 images in RGB 3 channel
num_classes = 5 //Have to detect 5 gestures
model = Sequential()
# First conv, 32
model.add(TimeDistributed(Conv2D((32), (3,3), input_shape = input_shape)))
model.add(Activation('relu'))
model.add(TimeDistributed(Conv2D((32), (3,3))))
model.add(Activation('relu'))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
# Second conv, 64
model.add(TimeDistributed(Conv2D((64), (3, 3))))
model.add(Activation('relu'))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(TimeDistributed(Flatten()))
model.add(TimeDistributed(Dense(20)))
#configure LSTM
model.add(LSTM(20, activation='relu', return_sequences=False))
model.add(TimeDistributed(Dense(num_classes)))
model.add(Activation('softmax'))
model.compile(optimizer="adam", loss='categorical_crossentropy', metrics=['categorical_accuracy'])
print (model.summary())
I am getting below error after calling model.summary()
ValueError: This model has never been called, thus its weights have not yet been created, so no summary can be displayed. Build the model first (e.g. by calling it on some test data).
That's funny... What versions of TF and Keras are you using?
This would be common in a subclassed model, but sounds new to me in a Sequential model.
Just follow the instructions in the error:
test_predictions = model.predict(numpy.zeros((1,) + input_shape))
model.summary()
I am beginning to build CNN models using Keras.
I have built a CNN with a fairly accurate results using the following architecture.
classifier = Sequential()
classifier.add(Convolution2D(32, (3,3), input_shape = (64, 64, 3), activation='relu'))
classifier.add(MaxPool2D(pool_size = (2,2)))
classifier.add(Convolution2D(32, (3,3), activation='relu'))
classifier.add(MaxPool2D(pool_size = (2,2)))
classifier.add(Convolution2D(32, (3,3), activation='relu'))
classifier.add(MaxPool2D(pool_size = (2,2)))
classifier.add(Convolution2D(32, (3,3), activation='relu'))
classifier.add(MaxPool2D(pool_size = (2,2)))
classifier.add(Flatten())
classifier.add(Dense(units=128, activation='relu'))
classifier.add(Dropout(rate = 0.25))
classifier.add(Dense(units=128, activation='relu'))
classifier.add(Dropout(rate = 0.25))
classifier.add(Dense(units=1, activation='sigmoid'))
classifier.compile(optimizer = 'sgd', loss = 'binary_crossentropy', metrics=['accuracy'])
What I want to do is to run my images through the model, but only the convolutional steps. I am interested in the output of the Flattening process (i.e. get the features from the convolutional steps).
Can someone help me how I can get it in Keras?
Thanks in advance
Here is one solution. If you are interested in the output of layer 'max_pooling2d_4' (You can get the layer name by classifier.summary(), but I suggest you to put names for each layer by e.g. classifier.add(MaxPool2D(pool_size=(2,2), name='pool1'))):
layer_dict = dict([(layer.name, layer) for layer in classifier.layers])
# input tensor
input_tensor = classifier.input
# output tensor of the given layer
layer_output = layer_dict['max_pooling2d_4'].output
# get the output with respect to the input
func = K.function([input_tensor], [layer_output])
# test image: [64, 64, 3]
image = np.ones((64,64,3))
# get activation for the test image
activation = func([image[np.newaxis, :, :, :]])
Say we have a convolutional neural network M. I can extract features from images by using
extractor = Model(M.inputs, M.get_layer('last_conv').output)
features = extractor.predict(X)
How can I get the model that will predict classes using features?
I can't use the following lines because it requires the input of the model to be a placeholder.
predictor = Model([M.get_layer('next_layer').input], M.outputs)
pred = predictor.predict(features)
I also can't use K.function because later I want to use it as part of another model, so I will be appliyng predictor to tf.tensor, not np.array.
This is not the nicest solution, but it works:
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Dense, Dropout, Flatten
def cnn():
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=(28, 28, 1), name='l_01'))
model.add(Conv2D(64, (3, 3), activation='relu', name='l_02'))
model.add(MaxPooling2D(pool_size=(2, 2), name='l_03'))
model.add(Dropout(0.25, name='l_04'))
model.add(Flatten(name='l_05'))
model.add(Dense(128, activation='relu', name='l_06'))
model.add(Dropout(0.5, name='l_07'))
model.add(Dense(10, activation='softmax', name='l_08'))
return model
def predictor(input_shape):
model = Sequential()
model.add(Flatten(name='l_05', input_shape=(12, 12, 64)))
model.add(Dense(128, activation='relu', name='l_06'))
model.add(Dropout(0.5, name='l_07'))
model.add(Dense(10, activation='softmax', name='l_08'))
return model
cnn_model = cnn()
cnn_model.save('/tmp/cnn_model.h5')
predictor_model = predictor(cnn_model.output.shape)
predictor_model.load_weights('/tmp/cnn_model.h5', by_name=True)
Every layer in the model is indexed. So if you know which layers you need, you could loop through them, copying them into a new model. This operation should copy the weights inside the layer as well.
Here's a model (from Oli Blum's answer):
model = Sequential()
# add some layers
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=(28, 28, 1), name='l_01'))
model.add(Conv2D(64, (3, 3), activation='relu', name='l_02'))
model.add(MaxPooling2D(pool_size=(2, 2), name='l_03'))
model.add(Dropout(0.25, name='l_04'))
model.add(Flatten(name='l_05'))
model.add(Dense(128, activation='relu', name='l_06'))
model.add(Dropout(0.5, name='l_07'))
model.add(Dense(10, activation='softmax', name='l_08'))
Say you wanted the last three layers:
def extract_layers(main_model, starting_layer_ix, ending_layer_ix):
# create an empty model
new_model = Sequential()
for ix in range(starting_layer_ix, ending_layer_ix + 1):
curr_layer = main_model.get_layer(index=ix)
# copy this layer over to the new model
new_model.add(curr_layer)
return new_model
It depends on what you want to do.
If you are going to throw away the feature extractor afterwards
If you plan on training the feature extractor later
If you are going to use the extracted features but you don't intend on training the model used to generate them, you could use the predict method to get the features as you did:
features = extractor.predict(X)
then save its output to a file (np.save or cPickle or whatever).
After that you could use that new dataset as the input to a new model.
If you plan on training the feature extractor later you'll need to stack the two networks as seen here with vgg as feature extractor https://github.com/fchollet/keras/issues/4576:
img_width, img_height = 150, 150
vgg16_model = VGG16(include_top=False, weights='imagenet')
input = Input(batch_shape=vgg16_model.output_shape)
x = GlobalAveragePooling2D()(input)
x = Dense(256, activation='relu')(x)
x = Dropout(0.5)(x)
predict = Dense(1, activation='sigmoid')(x)
top_model = Model(input, predict)
top_model.load_weights(os.path.join(data_path, 'VGG16Classifier.hdf5'))
input = Input(shape=(3, img_width, img_height))
x = vgg16_model(input)
predict = top_model(x)
model = Model(input, predict)
PS: This example uses channels first ordering. If you are using tensorflow you should change the shape to shape=(img_width, img_height,3 )