This is my first time posting a question, please pardon me if it wasn't written or structured well.
The dataset consists of images in TIF format. Which means I am running a 3D CNN.
These images are simulated X-Ray images and the dataset has 2 classes; Normal and Anomaly.
Their labels would be '0' for Normal and '1' for Anomaly.
The tree of my folders look like this:
Train
Normal
Anomaly
Validation
Normal
Anomaly
What I have done was initialise 2 arrays;
train and y_train.
I ran a FOR loop which imports and appends the Normal images into train, and append a '0' into y_train for each image appended. So if I have 10 Normal images in train, I will have ten '0' s in y_train as well.
This is repeated for the Anomaly images and they are appended into train and a '1' will be appended into y_train as well. This means train consists of Normal images followed by Anomaly images. And y_train consists of '0' s, followed by '1' s.
Another FOR loop is executed for the validation folder, whereby my arrays are test and y_test.
This is my code for my Neural Network:
def vgg1():
model = Sequential()
model.add(Conv3D(16, (3, 3, 3), activation="relu", padding="same", name="block1_conv1", input_shape=(128, 128, 128, 1), data_format="channels_last")) # 64
model.add(Conv3D(16, (3, 3, 3), activation="relu", padding="same", name="block1_conv2", data_format="channels_last")) # 64
model.add(MaxPooling3D((2,2, 2), strides=(2,2, 2),padding='same', name='block1_pool'))
model.add(Dropout(0.5))
model.add(Conv3D(32, (3, 3, 3), activation="relu", padding="same", name="block2_conv1", data_format="channels_last")) # 128
model.add(Conv3D(32, (3, 3, 3), activation="relu", padding="same", name="block2_conv2", data_format="channels_last")) # 128
model.add(MaxPooling3D((2,2, 2), strides=(2,2, 2),padding='same', name='block2_pool'))
model.add(Dropout(0.5))
model.add(Conv3D(64, (3, 3, 3), activation="relu", padding="same", name="block3_conv1", data_format="channels_last")) # 256
model.add(Conv3D(64, (3, 3, 3), activation="relu", padding="same", name="block3_conv2", data_format="channels_last")) # 256
model.add(Conv3D(64, (3, 3, 3), activation="relu", padding="same", name="block3_conv3", data_format="channels_last")) # 256
model.add(MaxPooling3D((2,2, 2), strides=(2,2, 2),padding='same', name='block3_pool'))
model.add(Dropout(0.5))
model.add(Conv3D(128, (3, 3, 3), activation="relu", padding="same", name="block4_conv1", data_format="channels_last")) # 512
model.add(Conv3D(128, (3, 3, 3), activation="relu", padding="same", name="block4_conv2", data_format="channels_last")) # 512
model.add(Conv3D(128, (3, 3, 3), activation="relu", padding="same", name="block4_conv3", data_format="channels_last")) # 512
model.add(MaxPooling3D((2,2, 2), strides=(2,2, 2),padding='same', name='block4_pool'))
model.add(Dropout(0.5))
model.add(Conv3D(128, (3, 3, 3), activation="relu", padding="same", name="block5_conv1", data_format="channels_last")) # 512
model.add(Conv3D(128, (3, 3, 3), activation="relu", padding="same", name="block5_conv2", data_format="channels_last")) # 512
model.add(Conv3D(128, (3, 3, 3), activation="relu", padding="same", name="block5_conv3", data_format="channels_last")) # 512
model.add(MaxPooling3D((2,2, 2), strides=(2,2, 2),padding='same', name='block5_pool'))
model.add(Dropout(0.5))
model.add(Flatten(name='flatten'))
model.add(Dense(4096, activation='relu',name='fc1'))
model.add(Dense(4096, activation='relu',name='fc2'))
model.add(Dense(2, activation='softmax', name='predictions'))
print(model.summary())
return model
The following code is the initialisation of x_train, y_train, x_test, y_test and model.compile.
I have also converted my labels into one-hot encoding.
from keras.utils import to_categorical
model = vgg1()
model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy']
)
x_train = np.load('/content/drive/My Drive/3D Dataset v2/x_train.npy')
y_train = np.load('/content/drive/My Drive/3D Dataset v2/y_train.npy')
y_train = to_categorical(y_train)
x_test = np.load('/content/drive/My Drive/3D Dataset v2/x_test.npy')
y_test = np.load('/content/drive/My Drive/3D Dataset v2/y_test.npy')
y_test = to_categorical(y_test)
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
This is the problem I want to highlight, which is the constant training accuracy and validation accuracy .
Train on 127 samples, validate on 31 samples
Epoch 1/25
127/127 [==============================] - 1700s 13s/step - loss: 1.0030 - accuracy: 0.7480 - val_loss: 0.5842 - val_accuracy: 0.7419
Epoch 2/25
127/127 [==============================] - 1708s 13s/step - loss: 0.5813 - accuracy: 0.7480 - val_loss: 0.5728 - val_accuracy: 0.7419
Epoch 3/25
127/127 [==============================] - 1693s 13s/step - loss: 0.5758 - accuracy: 0.7480 - val_loss: 0.5720 - val_accuracy: 0.7419
Epoch 4/25
127/127 [==============================] - 1675s 13s/step - loss: 0.5697 - accuracy: 0.7480 - val_loss: 0.5711 - val_accuracy: 0.7419
Epoch 5/25
127/127 [==============================] - 1664s 13s/step - loss: 0.5691 - accuracy: 0.7480 - val_loss: 0.5785 - val_accuracy: 0.7419
Epoch 6/25
127/127 [==============================] - 1666s 13s/step - loss: 0.5716 - accuracy: 0.7480 - val_loss: 0.5710 - val_accuracy: 0.7419
Epoch 7/25
127/127 [==============================] - 1676s 13s/step - loss: 0.5702 - accuracy: 0.7480 - val_loss: 0.5718 - val_accuracy: 0.7419
Epoch 8/25
127/127 [==============================] - 1664s 13s/step - loss: 0.5775 - accuracy: 0.7480 - val_loss: 0.5718 - val_accuracy: 0.7419
Epoch 9/25
127/127 [==============================] - 1660s 13s/step - loss: 0.5753 - accuracy: 0.7480 - val_loss: 0.5711 - val_accuracy: 0.7419
Epoch 10/25
127/127 [==============================] - 1681s 13s/step - loss: 0.5756 - accuracy: 0.7480 - val_loss: 0.5714 - val_accuracy: 0.7419
Epoch 11/25
127/127 [==============================] - 1679s 13s/step - loss: 0.5675 - accuracy: 0.7480 - val_loss: 0.5710 - val_accuracy: 0.7419
Epoch 12/25
127/127 [==============================] - 1681s 13s/step - loss: 0.5779 - accuracy: 0.7480 - val_loss: 0.5741 - val_accuracy: 0.7419
Epoch 13/25
127/127 [==============================] - 1682s 13s/step - loss: 0.5763 - accuracy: 0.7480 - val_loss: 0.5723 - val_accuracy: 0.7419
Epoch 14/25
127/127 [==============================] - 1685s 13s/step - loss: 0.5732 - accuracy: 0.7480 - val_loss: 0.5714 - val_accuracy: 0.7419
Epoch 15/25
127/127 [==============================] - 1685s 13s/step - loss: 0.5701 - accuracy: 0.7480 - val_loss: 0.5710 - val_accuracy: 0.7419
Epoch 16/25
127/127 [==============================] - 1678s 13s/step - loss: 0.5704 - accuracy: 0.7480 - val_loss: 0.5733 - val_accuracy: 0.7419
Epoch 17/25
127/127 [==============================] - 1663s 13s/step - loss: 0.5692 - accuracy: 0.7480 - val_loss: 0.5710 - val_accuracy: 0.7419
Epoch 18/25
127/127 [==============================] - 1657s 13s/step - loss: 0.5731 - accuracy: 0.7480 - val_loss: 0.5717 - val_accuracy: 0.7419
Epoch 19/25
127/127 [==============================] - 1674s 13s/step - loss: 0.5708 - accuracy: 0.7480 - val_loss: 0.5712 - val_accuracy: 0.7419
Epoch 20/25
127/127 [==============================] - 1666s 13s/step - loss: 0.5795 - accuracy: 0.7480 - val_loss: 0.5730 - val_accuracy: 0.7419
Epoch 21/25
127/127 [==============================] - 1671s 13s/step - loss: 0.5635 - accuracy: 0.7480 - val_loss: 0.5753 - val_accuracy: 0.7419
Epoch 22/25
127/127 [==============================] - 1672s 13s/step - loss: 0.5713 - accuracy: 0.7480 - val_loss: 0.5718 - val_accuracy: 0.7419
Epoch 23/25
127/127 [==============================] - 1672s 13s/step - loss: 0.5666 - accuracy: 0.7480 - val_loss: 0.5711 - val_accuracy: 0.7419
Epoch 24/25
127/127 [==============================] - 1669s 13s/step - loss: 0.5695 - accuracy: 0.7480 - val_loss: 0.5724 - val_accuracy: 0.7419
Epoch 25/25
127/127 [==============================] - 1663s 13s/step - loss: 0.5675 - accuracy: 0.7480 - val_loss: 0.5721 - val_accuracy: 0.7419
What is wrong and what can I do to rectify this? I presume that having a constant accuracy is undesirable.
Related
I have an issue with my current model, I am not using any pretrained model and just want to see how I can try to improve the model without any pretrained weights. I am doing an object detection project with around 8000+ images. I created a model as such:
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(300, 300, 3)))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Flatten())
initializer = tf.keras.initializers.HeUniform()
model.add(Dense(256, activation='relu', kernel_initializer=initializer))
model.add(Dropout(0.5))
model.add(Dense(4))
model.compile(loss='mse', optimizer='adam', metrics=[tfr.keras.metrics.MeanAveragePrecisionMetric()])
model.summary()
The issue that I am facing is that I don't get any higher value in the mAP metric, just seems to get stuck at around 80 or less. I just cannot seem to understand why it does not go any higher. The output of the last few epochs:
Epoch 103/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0891 - mean_average_precision_metric_1: 0.8126 - val_loss: 0.0792 - val_mean_average_precision_metric_1: 0.7933
Epoch 104/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0901 - mean_average_precision_metric_1: 0.8029 - val_loss: 0.0830 - val_mean_average_precision_metric_1: 0.7912
Epoch 105/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0909 - mean_average_precision_metric_1: 0.8045 - val_loss: 0.0808 - val_mean_average_precision_metric_1: 0.7931
Epoch 106/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0890 - mean_average_precision_metric_1: 0.8077 - val_loss: 0.0789 - val_mean_average_precision_metric_1: 0.7920
Epoch 107/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0887 - mean_average_precision_metric_1: 0.8042 - val_loss: 0.0885 - val_mean_average_precision_metric_1: 0.7913
Epoch 108/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0887 - mean_average_precision_metric_1: 0.8012 - val_loss: 0.0798 - val_mean_average_precision_metric_1: 0.7922
Epoch 109/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0884 - mean_average_precision_metric_1: 0.8048 - val_loss: 0.0796 - val_mean_average_precision_metric_1: 0.7923
Epoch 110/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0896 - mean_average_precision_metric_1: 0.8009 - val_loss: 0.0838 - val_mean_average_precision_metric_1: 0.7948
Epoch 111/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0883 - mean_average_precision_metric_1: 0.7957 - val_loss: 0.0806 - val_mean_average_precision_metric_1: 0.7930
Epoch 112/150
211/211 [==============================] - 7s 31ms/step - loss: 0.0887 - mean_average_precision_metric_1: 0.8123 - val_loss: 0.0785 - val_mean_average_precision_metric_1: 0.7930
Epoch 00112: early stopping
I am training a CNN model using Keras on Google Colab for binary image classification, the problem is when I use the test dataset divided into 2 classes in the model evaluate the accuracy is fixed at 0.5000. When I use a test dataset without dividing the data into 2 classes this does not happen and I have an accuracy of 0.9167.
My code:
modelo.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(551, 1117, 3)))
modelo.add(MaxPooling2D((2, 2)))
modelo.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
modelo.add(MaxPooling2D((2, 2)))
modelo.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
modelo.add(MaxPooling2D((2, 2)))
modelo.add(Flatten())
modelo.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
modelo.add(Dense(1, activation='sigmoid'))
modelo.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
hist = modelo.fit(traning_generator, validation_data=validation_generator, epochs=10, callbacks=callbacks)
training result:
Epoch 1/10
13/13 [==============================] - 33s 3s/step - loss: 136.5746 - accuracy: 0.5300 - val_loss: 23.4487 - val_accuracy: 0.4068
Epoch 2/10
13/13 [==============================] - 27s 2s/step - loss: 5.3578 - accuracy: 0.4675 - val_loss: 0.7363 - val_accuracy: 0.4068
Epoch 3/10
13/13 [==============================] - 28s 2s/step - loss: 0.6870 - accuracy: 0.5925 - val_loss: 0.7120 - val_accuracy: 0.5932
Epoch 4/10
13/13 [==============================] - 18s 1s/step - loss: 0.5529 - accuracy: 0.7225 - val_loss: 0.8240 - val_accuracy: 0.3898
Epoch 5/10
13/13 [==============================] - 18s 1s/step - loss: 0.4633 - accuracy: 0.7750 - val_loss: 1.1202 - val_accuracy: 0.4322
Epoch 6/10
13/13 [==============================] - 19s 1s/step - loss: 0.5213 - accuracy: 0.7675 - val_loss: 1.4779 - val_accuracy: 0.4407
Epoch 7/10
13/13 [==============================] - 17s 1s/step - loss: 0.1730 - accuracy: 0.9550 - val_loss: 1.8047 - val_accuracy: 0.4492
Epoch 8/10
13/13 [==============================] - 17s 1s/step - loss: 0.0887 - accuracy: 0.9925 - val_loss: 2.4989 - val_accuracy: 0.4831
Epoch 9/10
13/13 [==============================] - 17s 1s/step - loss: 0.0318 - accuracy: 1.0000 - val_loss: 3.7380 - val_accuracy: 0.4407
Epoch 10/10
13/13 [==============================] - 17s 1s/step - loss: 0.0070 - accuracy: 1.0000 - val_loss: 4.7144 - val_accuracy: 0.4492
when testing dataset without class division I get:
test_loss, test_acc = modelo.evaluate(test_dataset)
1/1 [==============================] - 1s 577ms/step - loss: 0.6590 - accuracy: 0.9167
and when testing the test dataset with 2 classes:
2/2 [==============================] - 3s 653ms/step - loss: 1.2334 - accuracy: 0.5000
I am trying to train a CNN model with 2030 preprocessed eye images. the shape of my input data is (2030, 200,200, 1). At first, the shape of the data was 1527. Then I used imblearn.over_sampling.RandomOverSampler to increase the dataset size. I constructed the model with Keras and here is the summary of my model:
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same', activation='relu',
input_shape=(img_cols, img_rows, 1)))
model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (5, 5), padding='same', activation='relu'))
model.add(Conv2D(64, (5, 5), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(3,3)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
optimizer = tf.keras.optimizers.SGD(learning_rate=0.000001)
model.compile(optimizer=optimizer,loss='binary_crossentropy',
metrics=[tf.keras.metrics.SpecificityAtSensitivity(0.5),
tf.keras.metrics.SensitivityAtSpecificity(0.5),'accuracy'])
# Augmentation
train_datagen = ImageDataGenerator(
rescale=1./255,
horizontal_flip=True,
vertical_flip=True,
width_shift_range=0.3,
height_shift_range=0.5,
rotation_range=10,
zoom_range=0.2
)
test_datagen = ImageDataGenerator(rescale=1./255)
train_data = train_datagen.flow(x_train, y_train)
test_data = test_datagen.flow(x_test, y_test)
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss',
factor=0.9,
patience=2, min_lr=0.0000000000000000001)
history=model.fit(train_data, epochs=10, batch_size=32,
validation_data=test_data, callbacks=[reduce_lr])
I trained the model with different parameters (with batch sizes 32, 64, 128, 256, 512, 1024, adding 128 and 256 neuron convolution layers, decreasing and increasing learning rate, using callbacks, varying the dense layer size with 32, 64, ..., 1024) but I always get the following learning process:
*Epoch 1/10
51/51 [==============================] - 14s 238ms/step - loss: 0.6962 - specificity_at_sensitivity_15: 0.4548 - sensitivity_at_specificity_15: 0.4777 - accuracy: 0.4969 - val_loss: 0.6957 - val_specificity_at_sensitivity_15: 0.4112 - val_sensitivity_at_specificity_15: 0.3636 - val_accuracy: 0.4852 - lr: 1.0000e-04
Epoch 2/10
51/51 [==============================] - 12s 226ms/step - loss: 0.6945 - specificity_at_sensitivity_15: 0.4829 - sensitivity_at_specificity_15: 0.4615 - accuracy: 0.5018 - val_loss: 0.6949 - val_specificity_at_sensitivity_15: 0.4467 - val_sensitivity_at_specificity_15: 0.3206 - val_accuracy: 0.4877 - lr: 1.0000e-04
Epoch 3/10
51/51 [==============================] - 12s 227ms/step - loss: 0.6955 - specificity_at_sensitivity_15: 0.4328 - sensitivity_at_specificity_15: 0.4082 - accuracy: 0.5043 - val_loss: 0.6945 - val_specificity_at_sensitivity_15: 0.5584 - val_sensitivity_at_specificity_15: 0.5167 - val_accuracy: 0.4852 - lr: 1.0000e-04
Epoch 4/10
51/51 [==============================] - 12s 226ms/step - loss: 0.6971 - specificity_at_sensitivity_15: 0.4034 - sensitivity_at_specificity_15: 0.4256 - accuracy: 0.5049 - val_loss: 0.6941 - val_specificity_at_sensitivity_15: 0.4010 - val_sensitivity_at_specificity_15: 0.3923 - val_accuracy: 0.4852 - lr: 1.0000e-04
Epoch 5/10
51/51 [==============================] - 12s 226ms/step - loss: 0.6954 - specificity_at_sensitivity_15: 0.4670 - sensitivity_at_specificity_15: 0.4640 - accuracy: 0.4969 - val_loss: 0.6938 - val_specificity_at_sensitivity_15: 0.5584 - val_sensitivity_at_specificity_15: 0.5407 - val_accuracy: 0.4729 - lr: 1.0000e-04
Epoch 6/10
51/51 [==============================] - 12s 227ms/step - loss: 0.6972 - specificity_at_sensitivity_15: 0.4352 - sensitivity_at_specificity_15: 0.3883 - accuracy: 0.4791 - val_loss: 0.6935 - val_specificity_at_sensitivity_15: 0.4772 - val_sensitivity_at_specificity_15: 0.3206 - val_accuracy: 0.4729 - lr: 1.0000e-04
Epoch 7/10
51/51 [==============================] - 12s 227ms/step - loss: 0.6943 - specificity_at_sensitivity_15: 0.4474 - sensitivity_at_specificity_15: 0.4814 - accuracy: 0.5031 - val_loss: 0.6933 - val_specificity_at_sensitivity_15: 0.3604 - val_sensitivity_at_specificity_15: 0.4880 - val_accuracy: 0.4729 - lr: 1.0000e-04
Epoch 8/10
51/51 [==============================] - 12s 225ms/step - loss: 0.6974 - specificity_at_sensitivity_15: 0.4609 - sensitivity_at_specificity_15: 0.4355 - accuracy: 0.4926 - val_loss: 0.6930 - val_specificity_at_sensitivity_15: 0.5279 - val_sensitivity_at_specificity_15: 0.5885 - val_accuracy: 0.4655 - lr: 1.0000e-04
Epoch 9/10
51/51 [==============================] - 12s 226ms/step - loss: 0.6945 - specificity_at_sensitivity_15: 0.4425 - sensitivity_at_specificity_15: 0.4777 - accuracy: 0.5031 - val_loss: 0.6929 - val_specificity_at_sensitivity_15: 0.4619 - val_sensitivity_at_specificity_15: 0.3876 - val_accuracy: 0.4655 - lr: 1.0000e-04
Epoch 10/10
51/51 [==============================] - 12s 226ms/step - loss: 0.6977 - specificity_at_sensitivity_15: 0.4389 - sensitivity_at_specificity_15: 0.4367 - accuracy: 0.4766 - val_loss: 0.6927 - val_specificity_at_sensitivity_15: 0.6091 - val_sensitivity_at_specificity_15: 0.5024 - val_accuracy: 0.4951 - lr: 1.0000e-04*
And evaluation with test data generated from x_test data (2 percent of the 2030 images) resulted in:
13/13 [==============================] - 1s 69ms/step - loss: 0.6927 - specificity_at_sensitivity_15: 0.6091 - sensitivity_at_specificity_15: 0.5024 - accuracy: 0.4951
Accuracy score is : 0.4950738847255707
How can I improve my accuracy score? I tried every possible way, but the maximum I could increase was 53%. Similar codes I saw on the internet reached 76%. It is a medical imaging project, I believe it is better to obtain better accuracy.
If you could try with changing optimizer from SGD to Adam, you will get better accuracy along with doing some other changes like adding more convolution layers, increasing learning rate, removing dropout layers as follows:
model = Sequential()
model.add(Conv2D(16, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (5, 5), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (5, 5), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(128, (5, 5), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(3,3)))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)
model.compile(optimizer=optimizer,loss='binary_crossentropy',
metrics=[tf.keras.metrics.SpecificityAtSensitivity(0.5),
tf.keras.metrics.SensitivityAtSpecificity(0.5),'accuracy'])
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss',factor=0.2,
patience=2, min_lr=0.00001)
history=model.fit(train_dataset, epochs=10, batch_size=32,
validation_data=validation_dataset, callbacks=[reduce_lr])
Output:
Epoch 1/10
63/63 [==============================] - 9s 104ms/step - loss: 0.7631 - specificity_at_sensitivity_2: 0.5760 - sensitivity_at_specificity_2: 0.5700 - accuracy: 0.5445 - val_loss: 0.6751 - val_specificity_at_sensitivity_2: 0.7760 - val_sensitivity_at_specificity_2: 0.7460 - val_accuracy: 0.5050 - lr: 0.0010
Epoch 2/10
63/63 [==============================] - 5s 77ms/step - loss: 0.6570 - specificity_at_sensitivity_2: 0.7260 - sensitivity_at_specificity_2: 0.7030 - accuracy: 0.6030 - val_loss: 0.6652 - val_specificity_at_sensitivity_2: 0.7480 - val_sensitivity_at_specificity_2: 0.6920 - val_accuracy: 0.5990 - lr: 0.0010
Epoch 3/10
63/63 [==============================] - 4s 57ms/step - loss: 0.6277 - specificity_at_sensitivity_2: 0.7920 - sensitivity_at_specificity_2: 0.7650 - accuracy: 0.6565 - val_loss: 0.6696 - val_specificity_at_sensitivity_2: 0.6960 - val_sensitivity_at_specificity_2: 0.6820 - val_accuracy: 0.5930 - lr: 0.0010
Epoch 4/10
63/63 [==============================] - 4s 56ms/step - loss: 0.6163 - specificity_at_sensitivity_2: 0.8080 - sensitivity_at_specificity_2: 0.7830 - accuracy: 0.6570 - val_loss: 0.6330 - val_specificity_at_sensitivity_2: 0.8320 - val_sensitivity_at_specificity_2: 0.7840 - val_accuracy: 0.6520 - lr: 0.0010
Epoch 5/10
63/63 [==============================] - 4s 58ms/step - loss: 0.5710 - specificity_at_sensitivity_2: 0.8710 - sensitivity_at_specificity_2: 0.8420 - accuracy: 0.6995 - val_loss: 0.5940 - val_specificity_at_sensitivity_2: 0.8600 - val_sensitivity_at_specificity_2: 0.8420 - val_accuracy: 0.7030 - lr: 0.0010
Epoch 6/10
63/63 [==============================] - 4s 58ms/step - loss: 0.5426 - specificity_at_sensitivity_2: 0.8930 - sensitivity_at_specificity_2: 0.8790 - accuracy: 0.7250 - val_loss: 0.6158 - val_specificity_at_sensitivity_2: 0.8740 - val_sensitivity_at_specificity_2: 0.8360 - val_accuracy: 0.7060 - lr: 0.0010
Epoch 7/10
63/63 [==============================] - 4s 60ms/step - loss: 0.4991 - specificity_at_sensitivity_2: 0.9260 - sensitivity_at_specificity_2: 0.9100 - accuracy: 0.7550 - val_loss: 0.5927 - val_specificity_at_sensitivity_2: 0.8760 - val_sensitivity_at_specificity_2: 0.8460 - val_accuracy: 0.7280 - lr: 0.0010
Epoch 8/10
63/63 [==============================] - 4s 58ms/step - loss: 0.4597 - specificity_at_sensitivity_2: 0.9480 - sensitivity_at_specificity_2: 0.9300 - accuracy: 0.7885 - val_loss: 0.6473 - val_specificity_at_sensitivity_2: 0.8900 - val_sensitivity_at_specificity_2: 0.8260 - val_accuracy: 0.7320 - lr: 0.0010
Epoch 9/10
63/63 [==============================] - 4s 58ms/step - loss: 0.4682 - specificity_at_sensitivity_2: 0.9500 - sensitivity_at_specificity_2: 0.9310 - accuracy: 0.7900 - val_loss: 0.5569 - val_specificity_at_sensitivity_2: 0.9080 - val_sensitivity_at_specificity_2: 0.8880 - val_accuracy: 0.7330 - lr: 0.0010
Epoch 10/10
63/63 [==============================] - 4s 60ms/step - loss: 0.3974 - specificity_at_sensitivity_2: 0.9740 - sensitivity_at_specificity_2: 0.9600 - accuracy: 0.8155 - val_loss: 0.6180 - val_specificity_at_sensitivity_2: 0.9180 - val_sensitivity_at_specificity_2: 0.8940 - val_accuracy: 0.7540 - lr: 0.0010
The CNN model will classify between two class with training samples = 5974 and validation = 1987.
I am using datagen.flow_from_directory and my model will predict from separate test set. I am running the code For 200 epoch in Google Colab, but after 5 epoch, the training and validation accuracy is not improving.
Accuracy
Epoch 45/200
186/186 [==============================] - 138s 744ms/step - loss: 0.6931 - acc: 0.4983 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 46/200
186/186 [==============================] - 137s 737ms/step - loss: 0.6931 - acc: 0.4990 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 47/200
186/186 [==============================] - 142s 761ms/step - loss: 0.6931 - acc: 0.4987 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 48/200
186/186 [==============================] - 140s 752ms/step - loss: 0.6931 - acc: 0.4993 - val_loss: 0.6931 - val_acc: 0.5005
Epoch 49/200
186/186 [==============================] - 139s 745ms/step - loss: 0.6931 - acc: 0.4976 - val_loss: 0.6931 - val_acc: 0.5010
Epoch 50/200
186/186 [==============================] - 143s 768ms/step - loss: 0.6931 - acc: 0.4992 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 51/200
186/186 [==============================] - 140s 755ms/step - loss: 0.6931 - acc: 0.4980 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 52/200
186/186 [==============================] - 141s 758ms/step - loss: 0.6931 - acc: 0.4990 - val_loss: 0.6931 - val_acc: 0.4995
Epoch 53/200
186/186 [==============================] - 141s 759ms/step - loss: 0.6931 - acc: 0.4985 - val_loss: 0.6931 - val_acc: 0.5000
Epoch 54/200
186/186 [==============================] - 143s 771ms/step - loss: 0.6931 - acc: 0.4987 - val_loss: 0.6931 - val_acc: 0.4995
Epoch 55/200
186/186 [==============================] - 143s 771ms/step - loss: 0.6931 - acc: 0.4992 - val_loss: 0.6931 - val_acc: 0.5005
train_data_path = "/content/drive/My Drive/snk_tod/train"
valid_data_path = "/content/drive/My Drive/snk_tod/valid"
test_data_path = "/content/drive/My Drive/snk_tod/test"
img_rows = 100
img_cols = 100
epochs = 200
print(epochs)
batch_size = 32
num_of_train_samples = 5974
num_of_valid_samples = 1987
#Image Generator
train_datagen = ImageDataGenerator(rescale=1. / 255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
valid_datagen = ImageDataGenerator(rescale=1. / 255)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(train_data_path,
target_size=(img_rows, img_cols),
batch_size=batch_size,
shuffle=True,
class_mode='categorical')
validation_generator = valid_datagen.flow_from_directory(valid_data_path,
target_size=(img_rows, img_cols),
batch_size=batch_size,
shuffle=True,
class_mode='categorical')
test_generator = test_datagen.flow_from_directory(test_data_path,
target_size=(img_rows, img_cols),
batch_size=batch_size,
shuffle=False,
class_mode='categorical')
model = Sequential()
model.add(Conv2D((32), (3, 3), input_shape=(img_rows, img_cols, 3), kernel_initializer="glorot_uniform", bias_initializer="zeros"))
model.add(Activation('relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D((32), (3, 3),kernel_initializer="glorot_uniform", bias_initializer="zeros"))
model.add(Activation('relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D((64), (3, 3),kernel_initializer="glorot_uniform", bias_initializer="zeros"))
model.add(Activation('relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D((64), (3, 3),kernel_initializer="glorot_uniform", bias_initializer="zeros"))
model.add(Activation('relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Flatten()) # this converts our 3D feature maps to 1D feature vectors
model.add(Dropout(0.5))
model.add(Dense(512))
model.add(Dense(2))
model.add(Activation('sigmoid'))
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['acc'])
#Train
history=model.fit_generator(train_generator,
steps_per_epoch=num_of_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=num_of_valid_samples // batch_size)
I've played around with the setup of my architecture a lot, the number of layers, pooling size, dropouts, etc but I always end up in the same ballpark: ~96-98 accuracy and loss between 2-6%.
I'm training from a dataset of 78000 images, 26 classes (letters of the alphabet), and 3000 images per photo. I am satisfied with the accuracy, but are there any suggestions to reduce loss? Also How much loss do you think is too much? Anything else I should add to the model to make it more robust? Code for the model is shown below:
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(200, 200, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(BatchNormalization())
#model.add(Dropout(0.5))
model.add(Conv2D(64, (3, 3), activation='relu', input_shape=(200, 200, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(BatchNormalization())
#model.add(Dropout(0.5))
model.add(Conv2D(64, (3, 3), activation='relu', input_shape=(200, 200, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(BatchNormalization())
#model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(26, activation='softmax'))
Compile and Fit:
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=10, batch_size=64, verbose=1, validation_data=(X_test, y_test))
Output:
Train on 54600 samples, validate on 23400 samples
Epoch 1/10
54600/54600 [==============================] - 68s 1ms/step - loss: 1.1217 - accuracy: 0.6622 - val_loss: 0.9795 - val_accuracy: 0.7394
Epoch 2/10
54600/54600 [==============================] - 67s 1ms/step - loss: 0.2377 - accuracy: 0.9219 - val_loss: 0.2300 - val_accuracy: 0.9277
Epoch 3/10
54600/54600 [==============================] - 67s 1ms/step - loss: 0.1184 - accuracy: 0.9627 - val_loss: 0.2746 - val_accuracy: 0.9286
Epoch 4/10
54600/54600 [==============================] - 67s 1ms/step - loss: 0.0755 - accuracy: 0.9761 - val_loss: 0.1850 - val_accuracy: 0.9517
Epoch 5/10
54600/54600 [==============================] - 69s 1ms/step - loss: 0.0669 - accuracy: 0.9801 - val_loss: 0.2044 - val_accuracy: 0.9450
Epoch 6/10
54600/54600 [==============================] - 69s 1ms/step - loss: 0.0520 - accuracy: 0.9848 - val_loss: 0.2265 - val_accuracy: 0.9485
Epoch 7/10
54600/54600 [==============================] - 72s 1ms/step - loss: 0.0481 - accuracy: 0.9865 - val_loss: 0.1709 - val_accuracy: 0.9559
Epoch 8/10
54600/54600 [==============================] - 66s 1ms/step - loss: 0.0370 - accuracy: 0.9905 - val_loss: 0.1534 - val_accuracy: 0.9659
Epoch 9/10
54600/54600 [==============================] - 66s 1ms/step - loss: 0.0335 - accuracy: 0.9912 - val_loss: 0.1181 - val_accuracy: 0.9703
Epoch 10/10
54600/54600 [==============================] - 66s 1ms/step - loss: 0.0277 - accuracy: 0.9921 - val_loss: 0.1204 - val_accuracy: 0.9704