I am running an image classification model with images and my problem is that my validation accuracy is higher than my training accuracy.
The data (train/validation) is set up randomly. I am using the InceptionV3 as a pre-trained model. The ratio between accuracy and validation accuracy stays the same over 100 epochs.
I tried a lower learning rate and an additional batch normalization layer.
Does anyone have any ideas on what to look into? I'd appreciate some help, thank you!
base_model = InceptionV3(weights='imagenet', include_top=False)
# add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
# add a fully-connected layer
x = Dense(468, activation='relu')(x)
x = Dropout(0.5)(x)
# and a logistic layer
predictions = Dense(468, activation='softmax')(x)
# this is the model we will train
model = Model(base_model.input,predictions)
# first: train only the top layers (which were randomly initialized)
# i.e. freeze all convolutional InceptionV3 layers
for layer in base_model.layers:
layer.trainable = False
# compile the model (should be done *after* setting layers to non-trainable)
adam = Adam(lr=0.0001, beta_1=0.9)
model.compile(optimizer=adam, loss='categorical_crossentropy', metrics=['accuracy'])
# train the model on the new data for a few epochs
batch_size = 64
epochs = 100
img_height = 224
img_width = 224
train_samples = 127647
val_samples = 27865
train_datagen = ImageDataGenerator(
rescale=1./255,
#shear_range=0.2,
zoom_range=0.2,
zca_whitening=True,
#rotation_range=0.5,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'AD/AutoDetect/',
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='categorical')
validation_generator = test_datagen.flow_from_directory(
'AD/validation/',
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='categorical')
# fine-tune the model
model.fit_generator(
train_generator,
samples_per_epoch=train_samples // batch_size,
nb_epoch=epochs,
validation_data=validation_generator,
nb_val_samples=val_samples // batch_size)
Found 127647 images belonging to 468 classes.
Found 27865 images belonging to 468 classes.
Epoch 1/100
2048/1994 [==============================] - 48s - loss: 6.2839 - acc: 0.0073 - val_loss: 5.8506 - val_acc: 0.0179
Epoch 2/100
2048/1994 [==============================] - 44s - loss: 5.8338 - acc: 0.0430 - val_loss: 5.4865 - val_acc: 0.1004
Epoch 3/100
2048/1994 [==============================] - 45s - loss: 5.5147 - acc: 0.0786 - val_loss: 5.1474 - val_acc: 0.1161
Epoch 4/100
2048/1994 [==============================] - 44s - loss: 5.1921 - acc: 0.1074 - val_loss: 4.8049 - val_acc: 0.1786
see this answer
This occours becauce you add a dropout layer in your model that prevents the accuracy from going to 1.0 during training.
Related
At my job Interview yesterday, I was asked to build a neural network using TesnorFlow in python to classify images from the flowers images dataset.
But even though it should've worked theoretically, for some reason I couldn't increase the accuracy above 20s%.
Python Version: 3.8.13
TensorFlow Versioin: 2.4.1
The data preprocessing methods from the interviewer were given as follows:
# create datase
IMG_SIZE = 160
BATCH_SIZE = 32
AUTOTUNE = tf.data.experimental.AUTOTUNE
def _parse_data(x,y):
image = tf.io.read_file(x)
image = tf.image.decode_jpeg(image, channels=3)
image = tf.cast(image, dtype=tf.float32)
image = tf.math.l2_normalize(image)
image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))
return image,y
def _input_fn(x,y):
ds = tf.data.Dataset.from_tensor_slices((x,y))
ds = ds.map(_parse_data)
ds = ds.shuffle(buffer_size=data_size)
ds = ds.repeat()
ds = ds.batch(BATCH_SIZE)
ds = ds.prefetch(buffer_size=AUTOTUNE)
return ds
train_ds = _input_fn(x_train, y_train)
validation_ds = _input_fn(x_valid, y_valid)
With both training and validation datasets being
<PrefetchDataset shapes: ((None, 160, 160, 3), (None,)), types:
(tf.float32, tf.int32)>
With the network being as follows:
from tensorflow.keras import datasets, layers, models
model_seq = models.Sequential()
model_seq.add(layers.experimental.preprocessing.RandomFlip("horizontal",input_shape=(IMG_SIZE,IMG_SIZE,3)))
model_seq.add(layers.experimental.preprocessing.RandomRotation(0.2))
model_seq.add(layers.experimental.preprocessing.Rescaling(1./255))
model_seq.add(layers.Conv2D(16, 3, padding='same', activation='relu'))
model_seq.add(layers.MaxPooling2D())
model_seq.add(layers.Conv2D(32, 3, padding='same', activation='relu'))
model_seq.add(layers.MaxPooling2D())
model_seq.add(layers.Conv2D(64, 3, padding='same', activation='relu'))
model_seq.add(layers.MaxPooling2D())
model_seq.add(layers.Dropout(0.2))
model_seq.add(layers.Flatten())
model_seq.add(layers.Dense(128, activation='relu'))
model_seq.add(layers.Dense(len(label_names), activation='softmax'))
model_seq.summary()
The output layer being the only thing that isn't allowed to be change.
model_seq.add(layers.Dense(len(label_names), activation='softmax'))
(Please note I was for some reason asked to use model_seq.add(), and even though it could be triggering for some of you, please ignore it this once :) )
For compiling the model, I used the following:
model_seq.compile(optimizer="Adam",
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
And for fitting the model:
history = model_seq.fit(train_ds,epochs=20,
validation_data = validation_ds,
steps_per_epoch=100,validation_steps=100)
The things I've tried:
Using different Augmentation methods (or removing the whole section
from the network).
Changing the Batch and Image sizes.
Using Dropout layers.
Using early stopping as follows:
callback = tf.keras.callbacks.EarlyStopping(
monitor='val_loss',
min_delta=0, patience=3, verbose=0,
mode='auto',baseline=None,
restore_best_weights=True)
history = model_seq.fit(train_ds,epochs=20,
validation_data = validation_ds,
steps_per_epoch=100,validation_steps=100,
callbacks = [callback])
Yet despite all of the above, I couldn't get any results. Since I couldn't find out what I did wrong exactly, I'm hoping someone here could tell me, so I could learn from this experience.
(Please take into consideration that I wasn't allowed to change the preprocessing functions, with the parameters IMG_SIZE and BATCH_SIZE being the only exception).
“TLDR: If you want to use their preprocessing part and don't change anything go to First Approach. If you want to augment images, Go to Second Approach and use ImageDataGenerator”.
First Approach
As you say: I had to use the preprocessing functions and don't change this and because I don't access your data, I use cifar10 dataset and use your preprocessing part. (only line of reading from file changed).
Because you shouldn't change IMG_SIZE=160 in the preprocessing part, I add this layer : tf.keras.layers.Lambda(lambda image: tf.image.resize(image, (32, 32)))) to the network because working with large images causes a crash.
You don't need a very large network, first check with a small network then step by step add parameters then add layers.
We can get a better result like the below: (On cifar10 with your network I get 10% accuracy like you.)
import tensorflow as tf
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()
# create datase
IMG_SIZE = 160
BATCH_SIZE = 32
data_size = 32
AUTOTUNE = tf.data.experimental.AUTOTUNE
def _parse_data(x,y):
# image = tf.io.read_file(x) <- because don't read from path
image = x
# image = tf.image.decode_jpeg(image, channels=3) <- because don't read from path and don't have jpeg
image = tf.cast(image, dtype=tf.float32)
image = tf.math.l2_normalize(image)
image = tf.image.resize(image, (IMG_SIZE, IMG_SIZE))
return image,y
def _input_fn(x,y):
ds = tf.data.Dataset.from_tensor_slices((x,y))
ds = ds.map(_parse_data)
ds = ds.shuffle(buffer_size=data_size)
ds = ds.repeat()
ds = ds.batch(BATCH_SIZE)
ds = ds.prefetch(buffer_size=AUTOTUNE)
return ds
train_ds = _input_fn(X_train, y_train)
validation_ds = _input_fn(X_test, y_test)
model = tf.keras.Sequential([
tf.keras.Input(shape=(160, 160, 3)),
tf.keras.layers.Lambda(lambda image: tf.image.resize(image, (32, 32))),
tf.keras.layers.Conv2D(filters=32,kernel_size=(3,3),activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(50,activation='relu'),
tf.keras.layers.Dense(10,activation='softmax')
])
model.compile(optimizer="Adam",
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
metrics=['accuracy'])
history = model.fit(train_ds,epochs=4,
validation_data = validation_ds,
steps_per_epoch=100,validation_steps=100)
Output:
Epoch 1/4
100/100 [==============================] - 8s 38ms/step - loss: 2.2445 - accuracy: 0.1375 - val_loss: 2.1406 - val_accuracy: 0.2138
Epoch 2/4
100/100 [==============================] - 3s 33ms/step - loss: 2.0552 - accuracy: 0.2688 - val_loss: 1.9764 - val_accuracy: 0.3250
Epoch 3/4
100/100 [==============================] - 4s 38ms/step - loss: 1.9468 - accuracy: 0.3022 - val_loss: 1.9014 - val_accuracy: 0.3200
Epoch 4/4
100/100 [==============================] - 4s 36ms/step - loss: 1.8936 - accuracy: 0.3341 - val_loss: 1.8883 - val_accuracy: 0.3419
Second Approach: Augment images with ImageDataGenerator:
import tensorflow as tf
flowers = tf.keras.utils.get_file(
'flower_photos',
'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
untar=True)
data_gen = tf.keras.preprocessing.image.ImageDataGenerator(
rescale=1./255,
rotation_range = 10, # Degree range for random rotations.
horizontal_flip = True, # Randomly flip inputs horizontally.
vertical_flip = True, # Randomly flip inputs vertically.
)
imgs_dataset = data_gen.flow_from_directory(flowers, class_mode='categorical',
target_size=(160, 160), batch_size=32,
shuffle=True)
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(filters=32,kernel_size=(3,3),input_shape=(160, 160, 3),activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(50,activation='relu'),
tf.keras.layers.Dense(5,activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy',metrics=['accuracy'])
model.fit(imgs_dataset,epochs=5)
Output:
Found 3670 images belonging to 5 classes.
Epoch 1/5
115/115 [==============================] - 39s 311ms/step - loss: 1.7904 - accuracy: 0.4161
Epoch 2/5
115/115 [==============================] - 27s 236ms/step - loss: 1.0878 - accuracy: 0.5605
Epoch 3/5
115/115 [==============================] - 28s 244ms/step - loss: 1.0252 - accuracy: 0.6005
Epoch 4/5
115/115 [==============================] - 27s 233ms/step - loss: 0.9735 - accuracy: 0.6196
Epoch 5/5
115/115 [==============================] - 29s 248ms/step - loss: 0.9313 - accuracy: 0.6455
I am attempting to training a model based on a frozen Inception_v3 model with 3 classes as an output. When I run the training, training accuracy goes up but not validation accuracy which is more or less exactly at 33.33% i.e. showing completely random prediction. I can't figure where is the bug in my code and/or approach
I tried various form of output after the Inception v3 core with no differences at all.
# Model definition
# InceptionV3 frozen, flatten, dense 1024, dropout 50%, dense 1024, dense 3, lr 0.001 --> does not train
# InceptionV3 frozen, flatten, dense 1024, dense 3, lr 0.001 --> does not train
# InceptionV3 frozen, flatten, dense 1024, dense 3, lr 0.005 --> does not train
# InceptionV3 frozen, GlobalAvgPooling, dense 1024, dense 1024, dense 512, dense 3, lr 0.001 --> does not train
# InceptionV3 frozen, GlobalAvgPooling dropout 0.4 dense 3, lr 0.001, custom pre-process --> does not train
# InceptionV3 frozen, GlobalAvgPooling dropout 0.4 dense 3, lr 0.001, custom pre-process, batch=32 --> does not train
# InceptionV3 frozen, GlobalAvgPooling dropout 0.4 dense 3, lr 0.001, custom pre-process, batch=32, rebalance train/val sets --> does not train
IMAGE_SIZE = 150
BATCH_SIZE = 32
def build_model(image_size):
input_tensor = tf.keras.layers.Input(shape=(image_size, image_size, 3))
inception_base = InceptionV3(include_top=False, weights='imagenet', input_tensor=input_tensor)
for layer in inception_base.layers:
layer.trainable = False
x = inception_base.output
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dropout(0.2)(x)
output_tensor = tf.keras.layers.Dense(3, activation="softmax")(x)
model = tf.keras.Model(inputs=input_tensor, outputs=output_tensor)
return model
model = build_model(IMAGE_SIZE)
model.compile(optimizer=RMSprop(lr=0.002), loss='categorical_crossentropy', metrics=['acc'])
# Data generators with Image augmentations
train_datagen = ImageDataGenerator(
rescale=1./255,
preprocessing_function=tf.keras.applications.inception_v3.preprocess_input,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
# Do not augment validation!
validation_datagen = ImageDataGenerator(
rescale=1./255,
preprocessing_function=tf.keras.applications.inception_v3.preprocess_input)
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=(IMAGE_SIZE, IMAGE_SIZE),
batch_size=BATCH_SIZE,
class_mode='categorical')
validation_generator = validation_datagen.flow_from_directory(
valid_dir,
target_size=(IMAGE_SIZE, IMAGE_SIZE),
batch_size=BATCH_SIZE,
class_mode='categorical')
Output of this cell is:
Found 1697 images belonging to 3 classes.
Found 712 images belonging to 3 classes.
Output of last two epochs of training:
Epoch 19/20
23/23 [==============================] - 6s 257ms/step - loss: 1.1930 - acc: 0.3174
54/54 [==============================] - 20s 363ms/step - loss: 0.7870 - acc: 0.6912 - val_loss: 1.1930 - val_acc: 0.3174
Epoch 20/20
23/23 [==============================] - 6s 255ms/step - loss: 1.1985 - acc: 0.3160
54/54 [==============================] - 20s 362ms/step - loss: 0.7819 - acc: 0.7018 - val_loss: 1.1985 - val_acc: 0.3160
The only big thing that jumps out at me is to ditch the rescale=1./255 ImageDataGenerators, because this is also being handled by tf.keras.applications.inception_v3.preprocess_input, which scales the from -1 to 1; the network's expected input.
I trained a VGG16 with imagenet-weights to classfiy images with 4 classes.
Train data:3578 images belonging to 4 classes.
Validation data:894 images belonging to 4 classes
Each time i run the code, i get one of this two accuracy value. val_acc: 1.0000 in first run. val_acc: 0.3364 in second run.
Any explication for this? because the difference between the results is to much large.
train_dir = 'C:/Users/ucduq/Desktop/output1/train'
validation_dir = 'C:/Users/ucduq/Desktop/output1/val'
training_data_generator = ImageDataGenerator(
rescale=1./255,
#rotation_range=90,
#horizontal_flip=True,
# vertical_flip=True,
#shear_range=0.9
#zoom_range=0.9
)
validation_data_generator = ImageDataGenerator(rescale=1./255)
IMAGE_WIDTH=150
IMAGE_HEIGHT=150
BATCH_SIZE=32
input_shape=(150,150,3)
training_generator = training_data_generator.flow_from_directory(
train_dir,
target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),
batch_size=BATCH_SIZE,
class_mode="categorical")
validation_generator = validation_data_generator.flow_from_directory(
validation_dir,
target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),
batch_size=BATCH_SIZE,
class_mode="categorical",
shuffle=False)
from keras.applications import VGG16
vgg_conv = VGG16(weights='imagenet',
include_top=False,
input_shape=(150, 150, 3))
model = models.Sequential()
model.add(vgg_conv)
### Add new layers
model.add(layers.Flatten())
model.add(layers.Dense(1024, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(4, activation='softmax'))
model.compile(loss="categorical_crossentropy",optimizer='adam',metrics=["accuracy"])
results = model.fit_generator(training_generator, steps_per_epoch=training_generator.samples/training_generator.batch_size,
epochs=100,
callbacks=callbacks,
validation_data=validation_generator, validation_steps=28)
first run:
Epoch 100/100
111/110 [==============================] - 17s 152ms/step - loss: 1.3593 - acc: 0.3365 - val_loss: 1.3599 - val_acc: 0.3364
second run:
Epoch 100/100
111/110 [==============================] - 18s 158ms/step - loss: 1.9879e-06 - acc: 1.0000 - val_loss: 5.2915e-06 - val_acc: 1.0000
I assume that your data has a class that is 33% of the entire set? If that true, what happen in the first run: is the model didn't learn anything at all(acc: 0.3365).
This might because of incorrect using of data-augmentation, if the commented lines are what you use in the first run then they are the culprits.
The
#shear_range=0.9 and
#zoom_range=0.9
is too much, only one of this means that you discord 90% of each image so the model doesn't learn anything.
I'm trying to do image classification with dicom images that have balanced classes using the pre-trained InceptionV3 model.
def convertDCM(PathDCM) :
data = []
for dirName, subdir, files in os.walk(PathDCM):
for filename in sorted(files):
ds = pydicom.dcmread(PathDCM +'/' + filename)
im = fromarray(ds.pixel_array)
im = keras.preprocessing.image.img_to_array(im)
im = cv2.resize(im,(299,299))
data.append(im)
return data
PathDCM = '/home/Desktop/FULL_BALANCED_COLOURED/'
data = convertDCM(PathDCM)
#scale the raw pixel intensities to the range [0,1]
data = np.array(data, dtype="float")/255.0
labels = np.array(labels,dtype ="int")
#splitting data into training and testing
#test_size is percentage to split into test/train data
(trainX, testX, trainY, testY) = train_test_split(
data,labels,
test_size=0.2,
random_state=42)
img_width, img_height = 299, 299 #InceptionV3 size
train_samples = 300
validation_samples = 50
epochs = 25
batch_size = 15
base_model = keras.applications.InceptionV3(
weights ='imagenet',
include_top=False,
input_shape = (img_width,img_height,3))
model_top = keras.models.Sequential()
model_top.add(keras.layers.GlobalAveragePooling2D(input_shape=base_model.output_shape[1:], data_format=None)),
model_top.add(keras.layers.Dense(300,activation='relu'))
model_top.add(keras.layers.Dropout(0.5))
model_top.add(keras.layers.Dense(1, activation = 'sigmoid'))
model = keras.models.Model(inputs = base_model.input, outputs = model_top(base_model.output))
#Compiling model
model.compile(optimizer = keras.optimizers.Adam(
lr=0.0001),
loss='binary_crossentropy',
metrics=['accuracy'])
#Image Processing and Augmentation
train_datagen = keras.preprocessing.image.ImageDataGenerator(
rescale = 1./255,
zoom_range = 0.1,
width_shift_range = 0.2,
height_shift_range = 0.2,
horizontal_flip = True,
fill_mode ='nearest')
val_datagen = keras.preprocessing.image.ImageDataGenerator()
train_generator = train_datagen.flow(
trainX,
trainY,
batch_size=batch_size,
shuffle=True)
validation_generator = train_datagen.flow(
testX,
testY,
batch_size=batch_size,
shuffle=True)
When I train the model, I always get a constant validation accuracy of 0.3889 with the validation loss fluctuating.
#Training the model
history = model.fit_generator(
train_generator,
steps_per_epoch = train_samples//batch_size,
epochs = epochs,
validation_data = validation_generator,
validation_steps = validation_samples//batch_size)
Epoch 1/25
20/20 [==============================]20/20
[==============================] - 195s 49s/step - loss: 0.7677 - acc: 0.4020 - val_loss: 0.7784 - val_acc: 0.3889
Epoch 2/25
20/20 [==============================]20/20
[==============================] - 187s 47s/step - loss: 0.7016 - acc: 0.4848 - val_loss: 0.7531 - val_acc: 0.3889
Epoch 3/25
20/20 [==============================]20/20
[==============================] - 191s 48s/step - loss: 0.6566 - acc: 0.6304 - val_loss: 0.7492 - val_acc: 0.3889
Epoch 4/25
20/20 [==============================]20/20
[==============================] - 175s 44s/step - loss: 0.6533 - acc: 0.5529 - val_loss: 0.7575 - val_acc: 0.3889
predictions= model.predict(testX)
print(predictions)
Predicting the model also only returns an array of one prediction per image:
[[0.457804 ]
[0.45051473]
[0.48343503]
[0.49180537]...
Why is it that the model only predicts one of the two classes? Does this have to do with the constant val accuracy or possibly overfitting?
If you have two classes, every image is in one or the other so probabilities for one class are enough to find everything because the sum of the probabilities for each image is supposed to make 1. So if you have the probabilitie p for 1 class, the probability for the other one is 1-p.
If you want to have the possibility of classifing images not in one of those two class, then you should create a third one.
Also, this line:
model_top.add(keras.layers.Dense(1, activation = 'sigmoid'))
means that the Output is a vector of shape(nb_sample,1) and has the same shape as your training labels
The following is my MLP model,
layers = [10,20,30,40,50]
model = keras.models.Sequential()
#Stacking Layers
model.add(keras.layers.Dense(layers[0], input_dim = input_dim, activation='relu'))
#Defining the shape of input
for layer in layers[1:]:
model.add(keras.layers.Dense(layer, activation='relu'))
#Layer activation function
# Output layer
model.add(keras.layers.Dense(1, activation='sigmoid'))
#Pre-training
model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
#Training
model.fit(train_set, test_set, validation_split = 0.10, epochs = 50, batch_size = 10, shuffle = True, verbose = 2)
# evaluate the network
loss, accuracy = model.evaluate(train_set, test_set)
print("\nLoss: %.2f, Accuracy: %.2f%%" % (loss, accuracy*100))
#predictions
predt = model.predict(final_test)
print(predt)
The problem is that, accuracy is always 0, error log as shown,
Epoch 48/50 - 0s - loss: 1.0578 - acc: 0.0000e+00 - val_loss: 0.4885 - val_acc: 0.0000e+00
Epoch 49/50 - 0s - loss: 1.0578 - acc: 0.0000e+00 - val_loss: 0.4885 - val_acc: 0.0000e+00
Epoch 50/50 - 0s - loss: 1.0578 - acc: 0.0000e+00 - val_loss: 0.4885 - val_acc: 0.0000e+00
2422/2422 [==============================] - 0s 17us/step
Loss: 1.00, Accuracy: 0.00%
As suggested i've changed my learning signal from -1,1 to 0,1 and yet, the following is the error log
Epoch 48/50 - 0s - loss: 8.5879 - acc: 0.4672 - val_loss: 8.2912 - val_acc: 0.4856
Epoch 49/50 - 0s - loss: 8.5879 - acc: 0.4672 - val_loss: 8.2912 - val_acc: 0.4856
Epoch 50/50 - 0s - loss: 8.5879 - acc: 0.4672 - val_loss: 8.2912 - val_acc: 0.4856
2422/2422 [==============================] - 0s 19us/step
You code is very hard to read. This is not the recommended standard to write Keras model. Try this and let us know what you get. Assuming X is a matrix where the rows are the instances and the columns are the features. And Y is the labels
You need to add a channel as the last dimension as explained when using the TensorFlow backend. Furthermore the labels should be split into 2 nodes for better chance of success. A single neuron mapping is often less successful, than using a probabilistic output with 2 nodes.
n = 1000 # Number of instances
m = 4 # Number of features
num_classes = 2 # Number of output classes
... # Your code for loading the data
X = X.reshape(n, m,)
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.33)
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
Build your model. The last layer should use either sigmoid or softmax for a classification task. Try to use the Adadelta optimizer it has been shown to produce better results by traversing the gradient more efficiently, and reducing oscillations. We will also use cross entropy as our loss function as is standard with classification tasks. Binary cross entropy is fine too.
Try to use a standard model configuration. An increasing number of nodes does not really make much sense. The model should look like a prism, small set of input features, many hidden nodes, and a small set of output nodes. You should aim for the least number of hidden layers, make the layers fatter, rather than adding layers.
input_shape = (m,)
model = Sequential()
model.add(Dense(32, activation='relu', input_shape=input_shape))
model.add(Dense(64, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
You can get a summary of your model using
model.summary()
Train your model
epochs = 100
batch_size = 128
# Fit the model weights.
history = model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
To view what happened during training
plt.figure(figsize=(8,10))
plt.subplot(2,1,1)
# summarize history for accuracy
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='lower right')
plt.subplot(2,1,2)
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper right')
plt.show()