Tensor model accuracy - machine learning based on images - python

I am trying to build a software that can trade for me based in machine learning.
So I extracted bunch of data, generated charts with some indicators that I use and separate them in BUY, SELL and NOT_DEFINED. and I am using SELL and BUY to train.
Unfortunately the accuracy is not increasing. In certain level I made the accuracy achieve 0.82 but the final model.evaluate(test_set) always gives me a randonmic number around 0.50 . Even the prediction code I build is give me the same result.
Now not even during training the accuracy increase. Can somebody give me ideas how to make some progress?
import tensorflow as tf
from tensorflow import keras
import os
from keras.preprocessing.image import ImageDataGenerator
# Carregando as imagens
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('C:/Users/Adriano/PycharmProjects/machineLearningChatGPT/data/150_reduzido/train',
target_size = (150, 150),
batch_size = 64,
class_mode = 'categorical',
shuffle=False)
test_set = test_datagen.flow_from_directory('C:/Users/Adriano/PycharmProjects/machineLearningChatGPT/data/150_reduzido/validation',
target_size = (150, 150),
batch_size = 64,
class_mode = 'categorical',
shuffle=False)
# Construindo e treinando o modelo
model = keras.Sequential([
keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3)),
keras.layers.MaxPooling2D((2, 2)),
keras.layers.Conv2D(64, (3, 3), activation='relu'),
keras.layers.MaxPooling2D((2, 2)),
# keras.layers.Conv2D(64, (3, 3), activation='relu'),
# keras.layers.MaxPooling2D((2, 2)),
# keras.layers.Conv2D(128, (3, 3), activation='relu'),
# keras.layers.MaxPooling2D((2, 2)),
keras.layers.Flatten(),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dense(2, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(training_set, epochs=20)
# Avaliando o modelo
test_loss, test_acc = model.evaluate(test_set)
# Salvando o modelo e os pesos
model.save("redmodelFalse.h5")
model.save_weights("redweightFalse.h5")
I tried lots of things I saw other people doing. changed the number of epochs, batch_size, add shuffle = false in the training set, add layers, change activation and of course the set of images I am using.

Related

Model val_accuracy higher than test accuracy without dropout regularization

I recently created a machine learning of 810 training and 810 test images (27 classes) in order to identify ASL hand signs. I trained this model using an SGD optimizer with a 0.001 learning rate, 5 epochs, and categorical cross entropy loss. However, my validation accuracy is around 20% higher than my model test accuracy, and I'm not sure why. I've tried adjusting my model structure, optimizers, learning rate, and epochs - this never changes.
Anyone have any ideas? Here's my model code:
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(150, 150, 3)),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(128, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(128, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(27, activation='softmax')
])
model.summary()
from tensorflow.keras.optimizers import SGD
sgd = SGD(learning_rate=0.001, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss = 'categorical_crossentropy',
optimizer = sgd,
metrics = ['accuracy'])
class myCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs={}):
if(logs.get('accuracy')>0.95):
print("\nReached >95% accuracy so cancelling training!")
self.model.stop_training = True
callbacks = myCallback()
train_datagen = ImageDataGenerator(
rescale=1./255,
rotation_range=40,
width_shift_range=0.2, # Shifting image width by 20%
height_shift_range=0.2,# Shifting image height by 20%
shear_range=0.2, # Shearing across X-axis by 20%
zoom_range=0.2, # Image zooming by 20%
horizontal_flip=True,
fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(
"/content/drive/MyDrive/train_asl",
target_size = (150, 150),
class_mode = 'categorical',
batch_size = 5)
validation_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = validation_datagen.flow_from_directory(
"/content/drive/MyDrive/test_asl",
target_size = (150, 150),
class_mode = 'categorical',
batch_size = 5
)
import numpy as np
history = model.fit_generator(
train_generator,
steps_per_epoch = np.ceil(810/5), # 2520 images = batch_size * steps
epochs = 100,
validation_data=validation_generator,
validation_steps = np.ceil(810/5), # 372 images = batch_size * steps
callbacks=[callbacks],
verbose = 2)
Validation and test accuracies could be different if the underlying data distribution of validation and test data is different or unbalanced.
You could ensure that the class distribution of the 27 classes is approximately the same in both validation and test sets, using stratified sampling techniques. You could also check whether the input data distributions are same/different during validation and testing, because 810 images is not a lot especially if you are not using transfer-learning.

Convolutional Model result

I was working with a convolutional model and this is my code
#Import the necessary libraries
import tensorflow as tf
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import numpy as np
import os
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing import image
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(150, 150, 3)),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(128, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Conv2D(128, (3,3), activation='relu'),
tf.keras.layers.MaxPooling2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
model.compile(loss='binary_crossentropy',
optimizer=RMSprop(lr=1e-4),
metrics=['accuracy'])
train_datagen = ImageDataGenerator(
rescale=1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'cats_and_dogs_filtered/train',
target_size=(150, 150),
batch_size=20,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
'cats_and_dogs_filtered/validation',
target_size=(150, 150),
batch_size=20,
class_mode='binary')
history = model.fit(
train_generator,
epochs=15,
validation_data=validation_generator,
)
test_image = image.load_img('single_prediction/cat_or_dog_1.jpg', target_size = (150,150))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = model.predict(test_image)
print(train_generator.class_indices)
print(result)
The class_indices was {'cats': 0, 'dogs': 1}
The output of result was [[5.916551e-06]]
While sigmoid gives us a value between 0 and 1, and this value being near 0 indicates the class it belongs to, I have always gotten a value of 1 or 0 output in the case of binary classification before (Just learning about augmentation). I can not find appropriate resources on google about this answer so I want to know whether I am not doing something wrong here and that I should probably add a threshold as well. Thank you.
Okay, It was my misunderstanding. The value closer to 0 or 1 shows the class it belongs to so adding a threshold of 0.5> or <0.5 was needed.

Getting very poor accuracy on stanford_dogs dataset

I'm trying to train a model on the stanford_dogs dataset to classify 120 dog breeds but my code is acting strange.
I downloaded the image data from http://vision.stanford.edu/aditya86/ImageNetDogs/images.tar
Then ran the following code to split each folder of the breeds into training and testing folders:
dataset_dict = {}
source_path = 'C:/Users/visha/Downloads/stanford_dogs/dataset'
dir_root = os.getcwd()
dataset_folders = [x for x in os.listdir(os.path.join(dir_root, source_path)) if os.path.isdir(os.path.join(dir_root, source_path, x))]
for category in dataset_folders:
dataset_dict[category] = {'source_path': os.path.join(dir_root, source_path, category),
'train_path': create_folder(new_path='C:/Users/visha/Downloads/stanford_dogs/train',
folder_type='train',
data_class=category),
'validation_path': create_folder(new_path='C:/Users/visha/Downloads/stanford_dogs/validation',
folder_type='validation',
data_class=category)}
dataset_folders = [x for x in os.listdir(os.path.join(dir_root, source_path)) if os.path.isdir(os.path.join(dir_root, source_path, x))]
for key in dataset_dict:
print("Splitting Category {} ...".format(key))
split_data(source_path=dataset_dict[key]['source_path'],
train_path=dataset_dict[key]['train_path'],
validation_path=dataset_dict[key]['validation_path'],
split_size=0.7)
I fed the images through the network after some image augmentation and used sigmoid activation in the final layer and categorical_crossentropy loss.
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.optimizers import RMSprop
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(16, (3, 3), activation='relu', input_shape=(150, 150, 3)),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(32, (3, 3), activation='relu'),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Conv2D(64, (3, 3), activation='relu'),
tf.keras.layers.MaxPooling2D(2, 2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512, activation='relu'),
tf.keras.layers.Dense(120, activation='softmax')
])
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
TRAINING_DIR = 'C:/Users/visha/Downloads/stanford_dogs/train'
train_datagen = ImageDataGenerator(rescale=1./255,rotation_range=40,width_shift_range=0.2,height_shift_range=0.2,
shear_range=0.2,zoom_range=0.2,horizontal_flip=True,fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(TRAINING_DIR,
batch_size=10,
class_mode='categorical',
target_size=(150, 150))
VALIDATION_DIR = 'C:/Users/visha/Downloads/stanford_dogs/validation'
validation_datagen = ImageDataGenerator(rescale=1./255, rotation_range=40,width_shift_range=0.2, height_shift_range=0.2,
shear_range=0.2,zoom_range=0.2,horizontal_flip=True,fill_mode='nearest')
validation_generator = validation_datagen.flow_from_directory(VALIDATION_DIR,
batch_size=10,
class_mode='categorical',
target_size=(150, 150))
history = model.fit(train_generator,
epochs=10,
verbose=1,
validation_data=validation_generator)
But the code is not working as intended. The val_accuracy after 10 epochs is something like 4.756.
For validation data you should not do any image augmentation, just do rescale. In validation flow_from_directory set shuffle=False. Be advised that the Stanford Dog Data set is very difficult. To achieve a reasonable degree of accuracy you will need a much more complex model. I recommend you consider transfer learning using the Mobilenet model. The code below shows how to do that.
base_model=tf.keras.applications.mobilenet.MobileNet( include_top=False,
input_shape=(150,150,3) pooling='max', weights='imagenet',dropout=.4)
x=base_model.output
x=keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 )(x)
x = Dense(1024, activation='relu')(x)
x=Dropout(rate=.3, seed=123)(x)
output=Dense(120, activation='softmax')(x)
model=Model(inputs=base_model.input, outputs=output)
model.compile(Adamax(lr=.001),loss='categorical_crossentropy',metrics=
['accuracy'] )
I forgot to mention that Mobilenet was trained on Images with pixel values in the range -1 to +1. So in ImageDataGenerator include the code
preprocessing_function=tf.keras.applications.mobilenet.preprocess_input
this scales the pixels so you do not need the code
rescale=1./255
or alternatively set
rescale=1/157.5-1
which will rescale the values between -1 and +1

Model get 97% accuracy on train and validation but when use custom predict it get wrong

I use a CNN model to train image classification , it got great accuracy at test and validation (98% and 97%), but when use my image to predict it alway go wrong, here is my code:
BATCH_SIZE = 30
IMG_HEIGHT = 256
IMG_WIDTH = 256
STEPS_PER_EPOCH = np.ceil(image_count/BATCH_SIZE)
train_data_gen = image_generator.flow_from_directory(directory=str(data_dir),
batch_size=BATCH_SIZE,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH),
classes = list(CLASS_NAMES))
here is prepare for dataset and data argumentation:
imgDataGen=ImageDataGenerator(
validation_split=0.2,
rescale=1/255,
horizontal_flip=True,
zoom_range=0.3,
rotation_range=15.,
width_shift_range=0.1,
height_shift_range=0.1,
)
prepare data:
train_dataset = imgDataGen.flow_from_directory(
directory=str(data_dir),
target_size = (IMG_HEIGHT, IMG_WIDTH),
classes = list(CLASS_NAMES),
batch_size = BATCH_SIZE,
subset = 'training'
)
val_dataset = imgDataGen.flow_from_directory(
directory=str(data_dir),
target_size = (IMG_HEIGHT, IMG_WIDTH),
classes = list(CLASS_NAMES),
batch_size =BATCH_SIZE,
subset = 'validation'
)
the model
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(256, 256, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(6, activation='sigmoid'))
complie:
model.compile(loss='binary_crossentropy',
optimizer=keras.optimizers.SGD(learning_rate=0.001,momentum=0.9),
metrics=['acc'])
train
history = model.fit_generator(
train_dataset,
validation_data = val_dataset,
workers=10,
epochs=20,
)
It get pretty high accuracy 98% on test and 97% on validation
but when i try with my code to predict
def prepare(filepath):
IMG_SIZE=256
img_array=cv2.imread(filepath)
new_array= cv2.resize(img_array,(IMG_SIZE,IMG_SIZE))
return new_array.reshape(1,IMG_SIZE,IMG_SIZE,3)
model=tf.keras.models.load_model('trained-model.h5',compile=False)
#np.set_printoptions(formatter={'float_kind':'{:f}'.format})
predict=model.predict([prepare('cat.jpg')])
pred_name = CATEGORIES[np.argmax(predict)]
print(pred_name)
it got wrong, with cat image it go for dog and dog for cat, but sometime it go right, just i think 98% is more accurate than this, if i try 5 image of cats it fail 3 or 4 images
so it because dataset or because of code?
please help, thanks
So in your second code-block you have this:
rescale=1/255
This is for normalizing your image into the range [0;1]. So every image gets rescaled (/normalized) before going through the network. But in you las code-block where you test it on an image you didnt add normalization. Try adding that to your "prepare" function:
def prepare(filepath):
IMG_SIZE = 256
img_array = cv2.imread(filepath)
# add this:
img_array = image_array / 255
new_array = cv2.resize(img_array,(IMG_SIZE,IMG_SIZE))
return new_array.reshape(1,IMG_SIZE,IMG_SIZE,3)

How to use a trained model on new inputs?

I have created a CNN model that can be used to differentiate DOGS and CATS. During the training process my model was showing an training accuracy of 99% and testing accuracy of 81% by the end of 4/25 epoch.
Is this normal? or is there any problem that might occur after completion of all the epoch's?
So I need to use this CNN model to my new inputs that do not belong to my training of test set. How do I use my model to predict some new photos?
I have not used classifier.save( ), so after the training can I just use that command so that model gets saved? or do I have to recompile everything with clssifier.save() at the end?
# Part 1 - Building the CNN
# Importing the Keras libraries and packages
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
# Initialising the CNN
classifier = Sequential()
# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))
# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
# Step 3 - Flattening
classifier.add(Flatten())
# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))
# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
# Part 2 - Fitting the CNN to the images
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
training_set = train_datagen.flow_from_directory('dataset/training_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
test_set = test_datagen.flow_from_directory('dataset/test_set',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
classifier.fit_generator(training_set,
steps_per_epoch = 8000,
epochs = 25,
validation_data = test_set,
validation_steps = 2000)
The model has a save method that exports the architecture and training configuration of the model to a file which can be later extracted and used. The documentation for the same can be found here.
After importing the model, you can use the model on any data sets that you want to. About the accuracy of the model, it is possible to achieve the same. There is still a huge difference between the train and test accuracy so at the moment it is over-fitting the data. Also, try to randomize the data and train using them to make sure it is not an exceptional case.

Categories

Resources