Develop Reference

Accuracy of same validation dataset differs between last epoch and after fit

The following code gives a log ending with
Epoch 19/20
1/1 [==============================] - 0s 473ms/step - loss: 1.4018 - accuracy: 0.8750 - val_loss: 1.8656 - val_accuracy: 0.8900
Epoch 20/20
1/1 [==============================] - 0s 444ms/step - loss: 0.5904 - accuracy: 0.8750 - val_loss: 2.1255 - val_accuracy: 0.8700
get_dataset: validation
Found 1000 files belonging to 2 classes.
Using 100 files for validation.
4/4 [==============================] - 1s 81ms/step
eval acc: 0.81
My question is:
Why is the val_accuracy after the last epoch (0.87) different from the eval acc (0.81) after the fit?
In my code, I try to use the same dataset for the validation of each epoch during fit and the additional validation afterwards.
[Update 1, 2022-07-19:
Obviously, the two accuracy calculations don't really use the same data. How can I debug which data is actually used?
[Update 3, 2022-07-20: I have followed the data into TensorFlow. The last thing I see is that in Model.evaluate (during fit) and Model.predict the x.filenames are equal. I did not manage to debug much further, because soon in quick_execute the __inference_test_function_248219 resp. the __inference_predict_function_231438 are evaluated outside Python, and the arguments are tensors with dtype=resource, whose contents I cannot see.]
I have deliberately removed my class balancing code to keep my example small. I know that this makes the accuracies less useful, but I don't care about that for now.
Note that get_dataset('validation') is only called once at the beginning of the fit, not at each epoch.
I have now also set max_queue_size=0, use_multiprocessing=False, workers=0 (as seen here, found via this related SO question about TensorFlow 1), but this did not make the accuracies equal.
]
Code:
import tensorflow as tf
from sklearn.metrics import accuracy_score
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.preprocessing import image_dataset_from_directory
inputs = tf.keras.Input(shape=(224, 224, 3))
base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False)
base_output = base_model(inputs)
base_model.trainable = False
out = Flatten(name='flat')(base_output)
out = Dense(1, activation='sigmoid')(out)
model = Model(inputs=inputs, outputs=out)
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
def get_dataset(subset):
print('get_dataset:', subset)
return image_dataset_from_directory(
'data-nodup-1000',
labels="inferred",
label_mode='binary',
color_mode="rgb",
image_size=(224, 224),
shuffle=True,
seed=1,
validation_split=0.1,
subset=subset,
crop_to_aspect_ratio=False,
)
model.fit(
get_dataset('training'),
steps_per_epoch=1,
epochs=20,
validation_data=get_dataset('validation'),
max_queue_size=0,
use_multiprocessing=False,
workers=0,
)
val_dataset = get_dataset('validation')
true_class = tf.concat([y for x, y in val_dataset], axis=0)
pred = model.predict(val_dataset)
pred_class = pred >= .5
print('eval acc:', accuracy_score(true_class, pred_class))
[Update 2, 2022-07-19:
I can also reproduce the behavior with the deprecated ImageDataGenerator, using
from tensorflow.keras.applications.resnet50 import preprocess_input
from keras_preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(
preprocessing_function=preprocess_input,
validation_split=0.1,
)
def get_dataset(subset):
print('get_dataset:', subset)
return datagen.flow_from_directory(
'data-nodup-1000',
class_mode='binary',
target_size=(224, 224),
shuffle=True,
seed=1,
subset=subset,
)
and
true_class = val_dataset.labels
]
[Update 4, 2022-07-21: Note that deactivating shuffling of validation data by setting shuffle=(subset == 'training') makes the two validation accuracies equal. This is not a workaround, however, because the validation set then consists only of class 1, since flow_from_directory doesn't do stratification.
]
My environment:
I am using all up-to-date libraries, like tensorflow 2.9.1 and sklearn 1.1.1 (via pip-compile -U).
The folder data-nodup-1000 contains one subfolder with 113 files of class 0, and one subfolder with 887 files of class 1.

I have now found out that in TensorFlow 2.9.1 model.predict uses the second iteration of the dataset, which is shuffled differently than the first iteration!
It even uses the second iteration when I directly call model.predict(get_dataset('validation'))!
Therefore, the entries of true_class and pred do not match.
Switching to TensorFlow 2.10.0-rc3 and its tf.keras.utils.split_dataset makes the accuracies equal.
Here's the updated code:
import tensorflow as tf
from sklearn.metrics import accuracy_score
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.preprocessing import image_dataset_from_directory
inputs = tf.keras.Input(shape=(224, 224, 3))
base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False)
base_output = base_model(inputs)
base_model.trainable = False
out = Flatten(name='flat')(base_output)
out = Dense(1, activation='sigmoid')(out)
model = Model(inputs=inputs, outputs=out)
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
dataset = image_dataset_from_directory(
'data-synthetic',
labels="inferred",
label_mode='binary',
color_mode="rgb",
image_size=(224, 224),
shuffle=True,
seed=1,
crop_to_aspect_ratio=False,
)
train_dataset, val_dataset = tf.keras.utils.split_dataset(dataset, right_size=0.1)
model.fit(
train_dataset,
steps_per_epoch=1,
epochs=20,
validation_data=val_dataset,
max_queue_size=0,
use_multiprocessing=False,
workers=0,
)
true_class = tf.concat([y for x, y in val_dataset], axis=0)
pred = model.predict(val_dataset)
pred_class = pred >= .5
print('eval acc:', accuracy_score(true_class, pred_class))
which correctly yields:
Epoch 19/20
1/1 [==============================] - 0s 438ms/step - loss: 0.4426 - accuracy: 0.9062 - val_loss: 0.4658 - val_accuracy: 0.8800
Epoch 20/20
1/1 [==============================] - 0s 444ms/step - loss: 2.1619 - accuracy: 0.8438 - val_loss: 0.5886 - val_accuracy: 0.8900
4/4 [==============================] - 1s 87ms/step
eval acc: 0.89

there are a few points about your data which causes this:
First, your data is highly imbalanced (8 to 1 label ratio) which makes the model rather overfit and the CV estimate inaccurate.
Second, in the get_dataset function, the shuffle is set to True so every time you call the get_dataset(), it shuffles your data, and because (1) Your validation set is very small and (2) your train/val split is not stratified over your labels, the validation metrics would vary a lot due to this shuffling.
Suggestions to solve this:
call the get_dataset() only once for train and val dataset before fitting the model and save them as variables. and if there is no sequential order in your data, maybe set shuffle=False.
(optional) If possible make your dataset more balanced by techniques such as data augmentation, over-/under-sampling, etc.
def get_dataset(subset):
return image_dataset_from_directory(
'data-nodup-1000',
labels="inferred",
label_mode='binary',
color_mode="rgb",
image_size=(224, 224),
shuffle=False,
seed=0,
validation_split=0.1,
subset=subset,
crop_to_aspect_ratio=False,
)
train_dataset = get_dataset('training')
val_dataset = get_dataset('validation')
model.fit(
train_dataset,
steps_per_epoch=1,
epochs=20,
validation_data=val_dataset,
)
true_class = tf.concat([y for x, y in val_dataset], axis=0)
pred = model.predict(val_dataset)
pred_class = pred >= .5
print('eval acc:', accuracy_score(true_class, pred_class))

Accuracy is not increasing in images classification when using TensorFlow model

At my job Interview yesterday, I was asked to build a neural network using TesnorFlow in python to classify images from the flowers images dataset.
But even though it should've worked theoretically, for some reason I couldn't increase the accuracy above 20s%.
Python Version: 3.8.13
TensorFlow Versioin: 2.4.1
The data preprocessing methods from the interviewer were given as follows:
# create datase
IMG_SIZE = 160
BATCH_SIZE = 32
AUTOTUNE = tf.data.experimental.AUTOTUNE
def _parse_data(x,y):
image = tf.io.read_file(x)
image = tf.image.decode_jpeg(image, channels=3)
image = tf.cast(image, dtype=tf.float32)
image = tf.math.l2_normalize(image)
image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))
return image,y
def _input_fn(x,y):
ds = tf.data.Dataset.from_tensor_slices((x,y))
ds = ds.map(_parse_data)
ds = ds.shuffle(buffer_size=data_size)
ds = ds.repeat()
ds = ds.batch(BATCH_SIZE)
ds = ds.prefetch(buffer_size=AUTOTUNE)
return ds
train_ds = _input_fn(x_train, y_train)
validation_ds = _input_fn(x_valid, y_valid)
With both training and validation datasets being
<PrefetchDataset shapes: ((None, 160, 160, 3), (None,)), types:
(tf.float32, tf.int32)>
With the network being as follows:
from tensorflow.keras import datasets, layers, models
model_seq = models.Sequential()
model_seq.add(layers.experimental.preprocessing.RandomFlip("horizontal",input_shape=(IMG_SIZE,IMG_SIZE,3)))
model_seq.add(layers.experimental.preprocessing.RandomRotation(0.2))
model_seq.add(layers.experimental.preprocessing.Rescaling(1./255))
model_seq.add(layers.Conv2D(16, 3, padding='same', activation='relu'))
model_seq.add(layers.MaxPooling2D())
model_seq.add(layers.Conv2D(32, 3, padding='same', activation='relu'))
model_seq.add(layers.MaxPooling2D())
model_seq.add(layers.Conv2D(64, 3, padding='same', activation='relu'))
model_seq.add(layers.MaxPooling2D())
model_seq.add(layers.Dropout(0.2))
model_seq.add(layers.Flatten())
model_seq.add(layers.Dense(128, activation='relu'))
model_seq.add(layers.Dense(len(label_names), activation='softmax'))
model_seq.summary()
The output layer being the only thing that isn't allowed to be change.
model_seq.add(layers.Dense(len(label_names), activation='softmax'))
(Please note I was for some reason asked to use model_seq.add(), and even though it could be triggering for some of you, please ignore it this once :) )
For compiling the model, I used the following:
model_seq.compile(optimizer="Adam",
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
And for fitting the model:
history = model_seq.fit(train_ds,epochs=20,
validation_data = validation_ds,
steps_per_epoch=100,validation_steps=100)
The things I've tried:
Using different Augmentation methods (or removing the whole section
from the network).
Changing the Batch and Image sizes.
Using Dropout layers.
Using early stopping as follows:
callback = tf.keras.callbacks.EarlyStopping(
monitor='val_loss',
min_delta=0, patience=3, verbose=0,
mode='auto',baseline=None,
restore_best_weights=True)
history = model_seq.fit(train_ds,epochs=20,
validation_data = validation_ds,
steps_per_epoch=100,validation_steps=100,
callbacks = [callback])
Yet despite all of the above, I couldn't get any results. Since I couldn't find out what I did wrong exactly, I'm hoping someone here could tell me, so I could learn from this experience.
(Please take into consideration that I wasn't allowed to change the preprocessing functions, with the parameters IMG_SIZE and BATCH_SIZE being the only exception).

“TLDR: If you want to use their preprocessing part and don't change anything go to First Approach. If you want to augment images, Go to Second Approach and use ImageDataGenerator”.
First Approach
As you say: I had to use the preprocessing functions and don't change this and because I don't access your data, I use cifar10 dataset and use your preprocessing part. (only line of reading from file changed).
Because you shouldn't change IMG_SIZE=160 in the preprocessing part, I add this layer : tf.keras.layers.Lambda(lambda image: tf.image.resize(image, (32, 32)))) to the network because working with large images causes a crash.
You don't need a very large network, first check with a small network then step by step add parameters then add layers.
We can get a better result like the below: (On cifar10 with your network I get 10% accuracy like you.)
import tensorflow as tf
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()
# create datase
IMG_SIZE = 160
BATCH_SIZE = 32
data_size = 32
AUTOTUNE = tf.data.experimental.AUTOTUNE
def _parse_data(x,y):
# image = tf.io.read_file(x) <- because don't read from path
image = x
# image = tf.image.decode_jpeg(image, channels=3) <- because don't read from path and don't have jpeg
image = tf.cast(image, dtype=tf.float32)
image = tf.math.l2_normalize(image)
image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))
return image,y
def _input_fn(x,y):
ds = tf.data.Dataset.from_tensor_slices((x,y))
ds = ds.map(_parse_data)
ds = ds.shuffle(buffer_size=data_size)
ds = ds.repeat()
ds = ds.batch(BATCH_SIZE)
ds = ds.prefetch(buffer_size=AUTOTUNE)
return ds
train_ds = _input_fn(X_train, y_train)
validation_ds = _input_fn(X_test, y_test)
model = tf.keras.Sequential([
tf.keras.Input(shape=(160, 160, 3)),
tf.keras.layers.Lambda(lambda image: tf.image.resize(image, (32, 32))),
tf.keras.layers.Conv2D(filters=32,kernel_size=(3,3),activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(50,activation='relu'),
tf.keras.layers.Dense(10,activation='softmax')
])
model.compile(optimizer="Adam",
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
metrics=['accuracy'])
history = model.fit(train_ds,epochs=4,
validation_data = validation_ds,
steps_per_epoch=100,validation_steps=100)
Output:
Epoch 1/4
100/100 [==============================] - 8s 38ms/step - loss: 2.2445 - accuracy: 0.1375 - val_loss: 2.1406 - val_accuracy: 0.2138
Epoch 2/4
100/100 [==============================] - 3s 33ms/step - loss: 2.0552 - accuracy: 0.2688 - val_loss: 1.9764 - val_accuracy: 0.3250
Epoch 3/4
100/100 [==============================] - 4s 38ms/step - loss: 1.9468 - accuracy: 0.3022 - val_loss: 1.9014 - val_accuracy: 0.3200
Epoch 4/4
100/100 [==============================] - 4s 36ms/step - loss: 1.8936 - accuracy: 0.3341 - val_loss: 1.8883 - val_accuracy: 0.3419
Second Approach: Augment images with ImageDataGenerator:
import tensorflow as tf
flowers = tf.keras.utils.get_file(
'flower_photos',
'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
untar=True)
data_gen = tf.keras.preprocessing.image.ImageDataGenerator(
rescale=1./255,
rotation_range = 10, # Degree range for random rotations.
horizontal_flip = True, # Randomly flip inputs horizontally.
vertical_flip = True, # Randomly flip inputs vertically.
)
imgs_dataset = data_gen.flow_from_directory(flowers, class_mode='categorical',
target_size=(160, 160), batch_size=32,
shuffle=True)
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(filters=32,kernel_size=(3,3),input_shape=(160, 160, 3),activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(50,activation='relu'),
tf.keras.layers.Dense(5,activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy',metrics=['accuracy'])
model.fit(imgs_dataset,epochs=5)
Output:
Found 3670 images belonging to 5 classes.
Epoch 1/5
115/115 [==============================] - 39s 311ms/step - loss: 1.7904 - accuracy: 0.4161
Epoch 2/5
115/115 [==============================] - 27s 236ms/step - loss: 1.0878 - accuracy: 0.5605
Epoch 3/5
115/115 [==============================] - 28s 244ms/step - loss: 1.0252 - accuracy: 0.6005
Epoch 4/5
115/115 [==============================] - 27s 233ms/step - loss: 0.9735 - accuracy: 0.6196
Epoch 5/5
115/115 [==============================] - 29s 248ms/step - loss: 0.9313 - accuracy: 0.6455

Check the predictions of my Keras model using a test dataset

I want to check if the predictions my model is computing are correct.
I have already trained my model and saved it. My dataset is created using tf.keras.preprocessing.image_dataset_from_directory as following :
ds_train = tf.keras.preprocessing.image_dataset_from_directory(
'data/',
labels = 'inferred',
label_mode = 'categorical',
color_mode = 'rgb',
batch_size = batch_size,
image_size = (img_height, img_width),
shuffle = True,
seed = 123,
validation_split = 0.1,
subset = 'training',
)
ds_validation = tf.keras.preprocessing.image_dataset_from_directory(
'data/',
labels = 'inferred',
label_mode = 'categorical',
color_mode = 'rgb',
batch_size = batch_size,
image_size = (img_height, img_width),
shuffle = True,
seed = 123,
validation_split = 0.1,
subset = 'validation',
)
As my model is working and is evaluated with an accuracy of 95.65% :
Found 921 files belonging to 2 classes.
Using 829 files for training.
Found 921 files belonging to 2 classes.
Using 92 files for validation.
Epoch 1/5
104/104 - 12s - loss: 0.1335 - accuracy: 0.9867
Epoch 2/5
104/104 - 5s - loss: 0.1017 - accuracy: 0.9879
Epoch 3/5
104/104 - 5s - loss: 0.0207 - accuracy: 0.9952
Epoch 4/5
104/104 - 5s - loss: 0.0603 - accuracy: 0.9879
Epoch 5/5
104/104 - 5s - loss: 0.0127 - accuracy: 0.9964
12/12 - 0s - loss: 0.2097 - accuracy: 0.9565
I have created a test_dataset the same way as I created my training and validation datasets.
ds_test = tf.keras.preprocessing.image_dataset_from_directory(
'test/',
labels = 'inferred',
label_mode = 'categorical',
color_mode = 'rgb',
batch_size = batch_size,
image_size = (img_height, img_width),
shuffle = True,
seed = 123,
)
I know want to predict the entire test_dataset using model.predict().
My data is stored in subfolders as followed :
---test/
---label1/
---label2/
I manage to recover all the predictions for all my images but I am not able to check if the predictions are correct. Is there any way to store the label of the image predicted in a list ?
I would like to use something like this :
for i in test_dataset :
list_labels = test_dataset.class_names(i)
predictions = model.predict(test_dataset)
predictions = np.argmax(predictions, axis=1)
So I can check the predictions output and link the predictions with labels.
Futhermore, I don't knwo why but I am not able to get the labels using like (x_train, y_train) = mnist.load_data :
# It is not working
(x_test, y_test) = test_dataset

I get different result for the same keras model

I trained a VGG16 with imagenet-weights to classfiy images with 4 classes.
Train data:3578 images belonging to 4 classes.
Validation data:894 images belonging to 4 classes
Each time i run the code, i get one of this two accuracy value. val_acc: 1.0000 in first run. val_acc: 0.3364 in second run.
Any explication for this? because the difference between the results is to much large.
train_dir = 'C:/Users/ucduq/Desktop/output1/train'
validation_dir = 'C:/Users/ucduq/Desktop/output1/val'
training_data_generator = ImageDataGenerator(
rescale=1./255,
#rotation_range=90,
#horizontal_flip=True,
# vertical_flip=True,
#shear_range=0.9
#zoom_range=0.9
)
validation_data_generator = ImageDataGenerator(rescale=1./255)
IMAGE_WIDTH=150
IMAGE_HEIGHT=150
BATCH_SIZE=32
input_shape=(150,150,3)
training_generator = training_data_generator.flow_from_directory(
train_dir,
target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),
batch_size=BATCH_SIZE,
class_mode="categorical")
validation_generator = validation_data_generator.flow_from_directory(
validation_dir,
target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),
batch_size=BATCH_SIZE,
class_mode="categorical",
shuffle=False)
from keras.applications import VGG16
vgg_conv = VGG16(weights='imagenet',
include_top=False,
input_shape=(150, 150, 3))
model = models.Sequential()
model.add(vgg_conv)
### Add new layers
model.add(layers.Flatten())
model.add(layers.Dense(1024, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(4, activation='softmax'))
model.compile(loss="categorical_crossentropy",optimizer='adam',metrics=["accuracy"])
results = model.fit_generator(training_generator, steps_per_epoch=training_generator.samples/training_generator.batch_size,
epochs=100,
callbacks=callbacks,
validation_data=validation_generator, validation_steps=28)
first run:
Epoch 100/100
111/110 [==============================] - 17s 152ms/step - loss: 1.3593 - acc: 0.3365 - val_loss: 1.3599 - val_acc: 0.3364
second run:
Epoch 100/100
111/110 [==============================] - 18s 158ms/step - loss: 1.9879e-06 - acc: 1.0000 - val_loss: 5.2915e-06 - val_acc: 1.0000

I assume that your data has a class that is 33% of the entire set? If that true, what happen in the first run: is the model didn't learn anything at all(acc: 0.3365).
This might because of incorrect using of data-augmentation, if the commented lines are what you use in the first run then they are the culprits.
The
#shear_range=0.9 and
#zoom_range=0.9
is too much, only one of this means that you discord 90% of each image so the model doesn't learn anything.

Keras image classification validation accuracy higher

I am running an image classification model with images and my problem is that my validation accuracy is higher than my training accuracy.
The data (train/validation) is set up randomly. I am using the InceptionV3 as a pre-trained model. The ratio between accuracy and validation accuracy stays the same over 100 epochs.
I tried a lower learning rate and an additional batch normalization layer.
Does anyone have any ideas on what to look into? I'd appreciate some help, thank you!
base_model = InceptionV3(weights='imagenet', include_top=False)
# add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
# add a fully-connected layer
x = Dense(468, activation='relu')(x)
x = Dropout(0.5)(x)
# and a logistic layer
predictions = Dense(468, activation='softmax')(x)
# this is the model we will train
model = Model(base_model.input,predictions)
# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional InceptionV3 layers
for layer in base_model.layers:
layer.trainable = False
# compile the model (should be done *after* setting layers to non-trainable)
adam = Adam(lr=0.0001, beta_1=0.9)
model.compile(optimizer=adam, loss='categorical_crossentropy', metrics=['accuracy'])
# train the model on the new data for a few epochs
batch_size = 64
epochs = 100
img_height = 224
img_width = 224
train_samples = 127647
val_samples = 27865
train_datagen = ImageDataGenerator(
rescale=1./255,
#shear_range=0.2,
zoom_range=0.2,
zca_whitening=True,
#rotation_range=0.5,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'AD/AutoDetect/',
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='categorical')
validation_generator = test_datagen.flow_from_directory(
'AD/validation/',
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='categorical')
# fine-tune the model
model.fit_generator(
train_generator,
samples_per_epoch=train_samples // batch_size,
nb_epoch=epochs,
validation_data=validation_generator,
nb_val_samples=val_samples // batch_size)
Found 127647 images belonging to 468 classes.
Found 27865 images belonging to 468 classes.
Epoch 1/100
2048/1994 [==============================] - 48s - loss: 6.2839 - acc: 0.0073 - val_loss: 5.8506 - val_acc: 0.0179
Epoch 2/100
2048/1994 [==============================] - 44s - loss: 5.8338 - acc: 0.0430 - val_loss: 5.4865 - val_acc: 0.1004
Epoch 3/100
2048/1994 [==============================] - 45s - loss: 5.5147 - acc: 0.0786 - val_loss: 5.1474 - val_acc: 0.1161
Epoch 4/100
2048/1994 [==============================] - 44s - loss: 5.1921 - acc: 0.1074 - val_loss: 4.8049 - val_acc: 0.1786

see this answer
This occours becauce you add a dropout layer in your model that prevents the accuracy from going to 1.0 during training.