Develop Reference

My model keeps predicting the same face, how can i fix it?

I trained a model with a CNN and used it with opencv for real-time face recognition on a webcam. I had some problems with the model, like I got 100% accuracy when it was training and I knew something must have went wrong, and another problem is that when it predicts on webcam images it only gives me one label no matter whose face I show: it gives a label of "me" no matter if I show my face or another person's face.
Note, I only have 250 images in the dataset: 200 images for training and 50 images for testing. For each set, I separated them into 2 classes, "me" and "not_me", where the "me" folder contains images of my face and the "not_me" folder contains images of different peoples' faces.
Is the data not enough for the model or something else?
Please help so that I can learn from my mistake. Thanks before hand.
myface_model.py
# Importing the Keras libraries and packages
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.preprocessing.image import ImageDataGenerator
# Initialising the CNN
classifier = Sequential()
# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Step 3 - Flattening
classifier.add(Flatten())
# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
# output layer
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
# Part 2 - Fitting the CNN to the images
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('dataset/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('dataset/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
classifier.fit_generator(training_set,
steps_per_epoch = 200,
epochs = 25,
validation_data = test_set,
validation_steps = 50)
#save model for later use
fer_json = classifier.to_json()
with open("fer.json", "w") as json_file:
json_file.write(fer_json)
classifier.save_weights("fer.h5")
myface_detector.py
import os
import cv2
import numpy as np
from keras.models import model_from_json
from keras.preprocessing import image
#load model
classifier = model_from_json(open("fer.json", "r").read())
#load weights
classifier.load_weights('fer.h5')
face_haar_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
cap=cv2.VideoCapture(0)
while True:
# captures frame and returns boolean value and captured image
ret,test_img=cap.read()
if not ret:
continue
convt_img= cv2.cvtColor(test_img, cv2.COLOR_BGR2RGB)
faces_detected = face_haar_cascade.detectMultiScale(convt_img, 1.3, 5)
for (x,y,w,h) in faces_detected:
cv2.rectangle(test_img, (x,y), (x+w,y+h), (255,0,0), thickness=4)
roi = convt_img[y:y+h,x:x+w]
roi = cv2.resize(roi,(64,64))
roi = roi.astype("float") / 255.0
img_pixels = image.img_to_array(roi)
img_pixels = np.expand_dims(img_pixels, axis = 0)
predictions = classifier.predict(img_pixels)
#find max indexed array
max_index = np.argmax(predictions[0])
myFace = ('me', 'not_me')
predicted_myFace = myFace[max_index]
cv2.putText(test_img, predicted_myFace, (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 2)
cv2.imshow('my face predictions',test_img)
if cv2.waitKey(10) == ord('q'):#wait until 'q' key is pressed
break
cap.release()
cv2.destroyAllWindows()

Binary classes
So the crux of the issue comes from how we choose to represent our model. Specifically in these lines:
classifier.add(Dense(units = 1, activation = 'sigmoid'))
and
training_set = train_datagen.flow_from_directory('dataset/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary') #<--------
test_set = test_datagen.flow_from_directory('dataset/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary') #<-------------
Here we choose to give our CNN one output node and to have binary class labels. This means that the return value of the predict method will look something like this [[0.12956393]]. This value represents the predicted probability that the test image has a class label 1 (in this case, a label of 'not_me'). To extract the class label, we can simply do:
predictions = classifier.predict(img_pixels)
predicted_label = predictions[0][0] > 0.5
myFace = ('me', 'not_me')
predicted_myFace = myFace[predicted_label]
However, to follow the intent of your original code to choose the class label based on highest probabilities, and to allow us to extend this application to easily predict more classes if needed, we only need to make minor modifications to the model to use categorical class labels.
Categorical classes
Instead of using binary labels, we can use categorical labels. This will allow us to have class labels from 0...n. In this case, we simply have to make the following edits
classifier.add(Dense(units = 2, activation = 'sigmoid')) # note that units = 2 now
and
training_set = train_datagen.flow_from_directory(
'dataset/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'categorical') # note the new class mode
test_set = test_datagen.flow_from_directory(
'dataset/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'categorical') # note the new class mode
By giving our CNN two output nodes and changing the class mode from binary to categorical, our CNN is now able to output a predicted probability for each class label. i.e. the return value of the predict method will look something like:
[[0.012211 0.9917401]]
which means that your code in myface_detector.py will work as is.

How to use a trained model on new inputs?

I have created a CNN model that can be used to differentiate DOGS and CATS. During the training process my model was showing an training accuracy of 99% and testing accuracy of 81% by the end of 4/25 epoch.
Is this normal? or is there any problem that might occur after completion of all the epoch's?
So I need to use this CNN model to my new inputs that do not belong to my training of test set. How do I use my model to predict some new photos?
I have not used classifier.save( ), so after the training can I just use that command so that model gets saved? or do I have to recompile everything with clssifier.save() at the end?
# Part 1 - Building the CNN
# Importing the Keras libraries and packages
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
# Initialising the CNN
classifier = Sequential()
# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Step 3 - Flattening
classifier.add(Flatten())
# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
# Part 2 - Fitting the CNN to the images
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('dataset/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('dataset/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
classifier.fit_generator(training_set,
steps_per_epoch = 8000,
epochs = 25,
validation_data = test_set,
validation_steps = 2000)

The model has a save method that exports the architecture and training configuration of the model to a file which can be later extracted and used. The documentation for the same can be found here.
After importing the model, you can use the model on any data sets that you want to. About the accuracy of the model, it is possible to achieve the same. There is still a huge difference between the train and test accuracy so at the moment it is over-fitting the data. Also, try to randomize the data and train using them to make sure it is not an exceptional case.

Error on number of arguments for keras fit_generator

I am having difficulty debugging, running this training model and then saving the weights.
The code:
#Part 1 - Building the CNN
#Importing the Keras libraries and packages
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
#Initialising the CNN
classifier = Sequential()
#Step 1 - Convolution
#classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
classifier.add(Conv2D(32,3,3,input_shape = (64, 64, 3),activation = 'relu'))
#Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
#Adding a second convolutional layer
classifier.add(Conv2D(32, 3, 3, activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
#Step 3 - Flattening
classifier.add(Flatten())
#Step 4 - Full connection
classifier.add(Dense(128, activation = 'relu'))
classifier.add(Dense(1, activation = 'sigmoid'))
#Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
#Part 2 - Fitting the CNN to the images
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
classifier.fit_generator(training_set,
validation_data = test_set,
validation_steps = 2000,
steps_per_epoch = 8000,
epochs = 25)
classifier.save("weights.h5")
Problem #1: I am getting the following error:
Found 8005 images belonging to 2 classes.
Found 2023 images belonging to 2 classes.
-----------------------------------------------------------------------TypeError Traceback (most recent call last) in ()
58 validation_steps = 2000,
59 steps_per_epoch = 8000,
---> 60 epochs = 25)
61
62
TypeError: fit_generator() takes at least 4 arguments (3 given)
Problem #2: I want to save the trained weights so that I don't need to keep running it over and over again.
I ran classifier.save("weights.h5") separately, and an empty file (since it's unable to train) was created with the below message. How can I save the model weights?
Error! /Users/xx/xx/xx/cnn_050518/model_weights.h5 is not UTF-8 encoded Saving disabled. See Console for more details.
versions of tools used (by entering print tool.version) keras: 1.1.1 tensorflow: 0.11.orc2 python:2.7 Macbook version 10.13.2

How to handle false predictions in a neural network built and trained with Keras?

I am very new to neural networks and I tried a typical first example with help of some Internet-Blogs: Image Classification of cats or dogs. After training the neural network below I tried to identify some random pictures of cats/dogs which I found on Google and which are neither in my training_set nor in my test_set… I found out, that sometimes the network gives a right prediction (recognizing a dog when showing a dog) and unfortunately sometimes a false prediction i.e. I showed a picture of a cat and the network predicts a ‘dog’. How do I handle such mistakes?
Adding all wrong pictures to the training_set or test_set and do the whole training process again? Or is there any other option to tell the network that it has made a false prediction and should adapt its weights?
#Part 1 - Import
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
#Part 2 – Build Network
classifier = Sequential()
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
#Part 3 - Training
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255, shear_range = 0.2, zoom_range = 0.2, horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('C:/…/KNNDaten/training_set', target_size = (64, 64), batch_size = 32, class_mode = 'binary')
test_set = test_datagen.flow_from_directory('C:/…/KNNDaten/test_set', target_size = (64, 64), batch_size = 32, class_mode = 'binary')
classifier.fit_generator(training_set, steps_per_epoch = 8000, epochs = 25, validation_data = test_set, validation_steps = 2000)
#Part 4 – Saving Model and weights
model_json = classifier.to_json()
with open("model1.json", "w") as json_file:
json_file.write(model_json)
classifier.save_weights("model1.h5")
# Part 5 - Making new predictions
import numpy as np
from keras.preprocessing import image
test_image = image.load_img('C:/… /KNNDaten/single_prediction/cat_or_dog_1.jpg', target_size = (64, 64))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = classifier.predict(test_image)
training_set.class_indices
if result[0][0] == 1:
prediction = 'dog'
else:
prediction = 'cat'
print("Image contains: " + prediction);
At the moment my training process looks like:
Results of my training process: accuracy, ...
Thank you very much for your help!

The usual process is to add the incorrectly predicted images to the training data set and retrain the network with random weights or using the weights obtained previously with the new images and the old ones.
When training a network you don't need to initiate with random weitghs, you could use the previous weights, this is sometimes called Transfer Learning. It is important if you try to do this to also include the original images used to train the model, or at least a part of it, if you don't want to overfit the model.
As Dascienz comments using data augmentation techniques can also be very useful to get a better generalization, for example adding the new images and variation of them: rotations, translation, symmetries and rescaling.

cnn feature extraction from an image, python

I need to extract components of advertisement pages; first I need to detect and mark the buttons (little rectangles under the page where it's written 'click') of the web advertisement pages(I use these pages as images for my data.) and I've trained the data in order to detect buttons by CNN and it perfectly knows whether an image is a button or not. However, after I've trained the data, I have some troubles to label these detected buttons by CNN. I will be so glad for any suggestions. Here are the codes;
from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
#Initialising the CNN
classifier = Sequential()
#Step 1: Convolution
classifier.add(Convolution2D(32, 3, 3, input_shape = (64, 64, 3), activation
= 'relu'))
#Step 2: Pooling
classifier.add(MaxPooling2D(pool_size= (2,2)))
#Step 3:Flatten
classifier.add(Flatten())
#Step 4: Full connection
classifier.add(Dense(output_dim = 128, activation = 'relu'))
classifier.add(Dense(output_dim = 1, activation = 'sigmoid'))
#Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics
= ['accuracy'])
#Fitting the CNN to the Images from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1./255)
training_set = train_datagen.flow_from_directory('train_set1',
target_size=(64, 64),
batch_size=32,
class_mode='binary')
test_set = test_datagen.flow_from_directory('test_set1', target_size(64,64),
batch_size=32,class_mode='binary')
classifier.fit_generator(training_set, steps_per_epoch=2754, epochs=25,
validation_data=test_set, nb_val_samples=460)