How to use a image set in RNN model - python

Hello I am trying to do my first RNN using Keras and Tensorflow, but I am getting stuck on an issue or reshaping my images to fit into the model.
I have looked at this post but could not figure out about the reshaping:
Keras - Input a 3 channel image into LSTM
What I have is a bunch of images that are taken at every frame in a video. I saved all the frames outside of python so I have a very large folder of images.I separated the frames into 21 frames for a segment so 21 images per motion that I want to capture. I want to read in these 21 images as one sequence. I have the same sequence captured from multiple cameras/angles which I want to us in this model. What I want to try is to model a movement and see if a person is doing this movement or not, so it is a binary model yes or no basically. Not the most sophisticated but its a learning process to use this model and keras.
I need help figuring out how to use these images inside the keras model. I have looked at a few tutorials on MINST data set but that didnt help me figure this out.
Any help will be appreciated.
This is the error that is given to me when I try to train the model
ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (2026, 200, 200, 1)
My code is this:
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.layers import LSTM
from tqdm import tqdm
import cv2
import os
import numpy as np
imageSize = 200
#create lables for each image
def labelImage(img):
wordLabel = img.split('.')[-3]
#Conversion to one hot array [lat,not]
if wordLabel == "FWAC":
return[1,0]
else:
return[0,1]
#Process images and add lables
#Convert data into an array and add its lable
def makeTrainingData():
print("Creating Training Data")
trainingData = []
for img in tqdm(os.listdir(trainDir)):
label = labelImage(img)
path = os.path.join(trainDir,img)
img = cv2.imread(path, cv2.IMREAD_GRAYSCALE)
img = cv2.resize(img, (imageSize,imageSize))
trainingData.append([np.array(img),np.array(label)])
#Save the array file to load it into other models if needed
np.save("trainingData.npy", trainingData)
print("Training Data Saved")
return trainingData
#process the testing data in the same manner
def processTestData():
print("Creating Testing Data")
testData = []
for img in tqdm(os.listdir(testDri)):
print("image", img)
path = os.path.join(testDri, img)
imgNum = img.split(".")[0]
img = cv2.imread(path, cv2.IMREAD_GRAYSCALE)
img = cv2.resize(img, (imageSize, imageSize))
testData.append([np.array(img), imgNum])
np.save("testingData.npy", testData)
print("Testing Data Saved")
return testData
rnnSize = 512
model = Sequential()
model.add(LSTM(rnnSize, input_shape=(imageSize, imageSize)))
model.add(Dense(1024))
model.add(Activation('relu'))
model.add(Dense(50))
model.add(Activation('sigmoid'))
model.add(Dense(3))
model.add(Activation('softmax'))
model.compile(loss='mean_squared_error', optimizer='adam',metrics=['accuracy'])
#Data
trainDir = "D:/TrainingDataSets/TrainingSet/"
testDri = "D:/TrainingDataSets/TestingSet/"
#trainData = makeTrainingData()
#testData = processTestData()
trainData = np.load('trainingData.npy')
testData = np.load("testingData.npy")
#resize the image to this See above
train = trainData[:-500]
test = trainData[-200:]
x = []
y = []
for xi in trainData:
x.append(xi[0].reshape((-1, imageSize, imageSize)))
y.append(xi[1])
x_train = np.array([i[0] for i in train]).reshape(-1,imageSize, imageSize,1)
y_train = [i[1] for i in train]
test_x = np.array([i[0] for i in test]).reshape(-1,imageSize , imageSize,1)
test_y = [i[1] for i in test]
epoch = 5
batchSize = 100
model.fit(x_train, y_train, epochs=epoch, batch_size= batchSize, verbose=1, shuffle=False)


For the error before dense layers add this line:
model.add(Flatten())
Previously, you should import:
from keras.layers import Flatten

Related

Run Tensorflow Keras custom model in OpenCV

So I was basically trying to figure out how to import a Tensorflow Keras CNN Model in OpenCV. The Docs I found on Github, weren't helpful and also were not clear about what to do EXACTLY. I have searched whole Youtube for tutorials, but nobody seems to have imported a custom made model before haha. I have basically tried everything...
Saving model as pickle (.p) and reading it in OpenCv (gave me this error: "Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ram://7b832872-99f5-4b67-8675-14f2423877df/variables/variables You may be trying to load on a different device from the computational device. Consider setting the experimental_io_device option in tf.saved_model.LoadOptions to the io_device such as '/job:localhost'."). I can't figure out what this is...
I also tried importing with tf.keras.models.load_model("saved_model.pb"), which also didnt seem to work and throw me the following error: File "h5py\h5f.pyx", line 106, in h5py.h5f.open
OSError: Unable to open file (file signature not found). It seems like I need a .h5 file, which I dont know how to get from my current model.
The next thing I tried was using cv2.dnn.readNetFromTensorflow(). For this to work you need the Tensorflow .pb file, which I have and (i guess its optional) the .pbtxt file. So the first problem was this error, which appeared after pasing my saved_model.pb: Failed to parse GraphDef file: saved_model.pb in function 'cv::dnn::ReadTFNetParamsFromBinaryFileOrDie'. I checked on that and some people wrote you should check the file for corruption and if it the name is written correctly, which I guess it is. Model passed with 0.986 Accuracy in my tests.
No I am on the end with my energy and dont't know what to do. I can't be the only one to have these issues, but certainly, it should be easy to use a tensorflow model in opencv, according to the docs...
I will now share for you the code I am using for creating the model as also the code for reading it in OpenCv. Versions of OpenCV, Python, Tensorflow, cuda, cudnn and pickle I will include below.
Any of your help is greatly appreciated!
This is the Code for creating Tensorflow Model
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import Adam
from keras.utils.np_utils import to_categorical
from keras.layers import Dropout, Flatten
from keras.layers.convolutional import Conv2D, MaxPooling2D
import cv2
from sklearn.model_selection import train_test_split
import tensorflow as tf
import pickle
import os
import pandas as pd
import random
from keras.preprocessing.image import ImageDataGenerator
import time
config = tf.compat.v1.ConfigProto()
config.gpu_options.allow_growth=True
sess = tf.compat.v1.Session(config=config)
# Assume that you have 12GB of GPU memory and want to allocate ~4GB:
gpu_options = tf.compat.v1.GPUOptions(per_process_gpu_memory_fraction=0.333)
sess = tf.compat.v1.Session(config=tf.compat.v1.ConfigProto(gpu_options=gpu_options))
################# Parameters #####################
path = "dataset" # folder with all the class folders
labelFile = 'labels.csv' # file with all names of classes
batch_size_val=10 # how many to process together
steps_per_epoch_val=300
epochs_val=100
imageDimesions = (300,300,3)
testRatio = 0.2
validationRatio = 0.2
###################################################
############################### Importing of the Images
count = 0
images = []
classNo = []
myList = os.listdir(path)
print("Total Classes Detected:",len(myList))
noOfClasses=len(myList)
print("Importing Classes.....")
for x in range (0,len(myList)):
myPicList = os.listdir(path+"/"+str(count))
for y in myPicList:
curImg = cv2.imread(path+"/"+str(count)+"/"+y)
images.append(curImg)
classNo.append(count)
print(count, end =" ")
count +=1
print(" ")
images = np.array(images)
classNo = np.array(classNo)
############################### Split Data
X_train, X_test, y_train, y_test = train_test_split(images, classNo, test_size=testRatio)
X_train, X_validation, y_train, y_validation = train_test_split(X_train, y_train, test_size=validationRatio)
# X_train = ARRAY OF IMAGES TO TRAIN
# y_train = CORRESPONDING CLASS ID
############################### TO CHECK IF NUMBER OF IMAGES MATCHES TO NUMBER OF LABELS FOR EACH DATA SET
print("Data Shapes")
print("Train",end = "");print(X_train.shape,y_train.shape)
print("Validation",end = "");print(X_validation.shape,y_validation.shape)
print("Test",end = "");print(X_test.shape,y_test.shape)
assert(X_train.shape[0]==y_train.shape[0]), "The number of images in not equal to the number of lables in training set"
assert(X_validation.shape[0]==y_validation.shape[0]), "The number of images in not equal to the number of lables in validation set"
assert(X_test.shape[0]==y_test.shape[0]), "The number of images in not equal to the number of lables in test set"
assert(X_train.shape[1:]==(imageDimesions))," The dimesions of the Training images are wrong "
assert(X_validation.shape[1:]==(imageDimesions))," The dimesionas of the Validation images are wrong "
assert(X_test.shape[1:]==(imageDimesions))," The dimesionas of the Test images are wrong"
############################### READ CSV FILE
data=pd.read_csv(labelFile)
print("data shape ",data.shape,type(data))
############################### DISPLAY SOME SAMPLES IMAGES OF ALL THE CLASSES
num_of_samples = []
cols = 5
num_classes = noOfClasses
fig, axs = plt.subplots(nrows=num_classes, ncols=cols, figsize=(5, 300))
fig.tight_layout()
for i in range(cols):
for j,row in data.iterrows():
x_selected = X_train[y_train == j]
axs[j][i].imshow(x_selected[random.randint(0, len(x_selected)- 1), :, :], cmap=plt.get_cmap("gray"))
axs[j][i].axis("off")
if i == 2:
axs[j][i].set_title(str(j)+ "-"+str(row["Name"]))
num_of_samples.append(len(x_selected))
############################### DISPLAY A BAR CHART SHOWING NO OF SAMPLES FOR EACH CATEGORY
print(num_of_samples)
plt.figure(figsize=(12, 4))
plt.bar(range(0, num_classes), num_of_samples)
plt.title("Distribution of the training dataset")
plt.xlabel("Class number")
plt.ylabel("Number of images")
plt.show()
############################### PREPROCESSING THE IMAGES
def grayscale(img):
img = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
return img
def equalize(img):
img =cv2.equalizeHist(img)
return img
def preprocessing(img):
img = grayscale(img) # CONVERT TO GRAYSCALE
img = equalize(img) # STANDARDIZE THE LIGHTING IN AN IMAGE
img = img/255 # TO NORMALIZE VALUES BETWEEN 0 AND 1 INSTEAD OF 0 TO 255
return img
X_train=np.array(list(map(preprocessing,X_train))) # TO IRETATE AND PREPROCESS ALL IMAGES
X_validation=np.array(list(map(preprocessing,X_validation)))
X_test=np.array(list(map(preprocessing,X_test)))
cv2.imshow("GrayScale Images",X_train[random.randint(0,len(X_train)-1)]) # TO CHECK IF THE TRAINING IS DONE PROPERLY
############################### ADD A DEPTH OF 1
X_train=X_train.reshape(X_train.shape[0],X_train.shape[1],X_train.shape[2],1)
X_validation=X_validation.reshape(X_validation.shape[0],X_validation.shape[1],X_validation.shape[2],1)
X_test=X_test.reshape(X_test.shape[0],X_test.shape[1],X_test.shape[2],1)
############################### AUGMENTATAION OF IMAGES: TO MAKEIT MORE GENERIC
dataGen= ImageDataGenerator(width_shift_range=0.1, # 0.1 = 10% IF MORE THAN 1 E.G 10 THEN IT REFFERS TO NO. OF PIXELS EG 10 PIXELS
height_shift_range=0.1,
zoom_range=0.2, # 0.2 MEANS CAN GO FROM 0.8 TO 1.2
shear_range=0.1, # MAGNITUDE OF SHEAR ANGLE
rotation_range=10) # DEGREES
dataGen.fit(X_train)
batches= dataGen.flow(X_train,y_train,batch_size=20) # REQUESTING DATA GENRATOR TO GENERATE IMAGES BATCH SIZE = NO. OF IMAGES CREAED EACH TIME ITS CALLED
X_batch,y_batch = next(batches)
# TO SHOW AGMENTED IMAGE SAMPLES
fig,axs=plt.subplots(1,15,figsize=(20,5))
fig.tight_layout()
for i in range(15):
axs[i].imshow(X_batch[i].reshape(imageDimesions[0],imageDimesions[1]))
axs[i].axis('off')
plt.show()
y_train = to_categorical(y_train,noOfClasses)
y_validation = to_categorical(y_validation,noOfClasses)
y_test = to_categorical(y_test,noOfClasses)
############################### CONVOLUTION NEURAL NETWORK MODEL
def myModel():
no_Of_Filters=60
size_of_Filter=(5,5) # THIS IS THE KERNEL THAT MOVE AROUND THE IMAGE TO GET THE FEATURES.
# THIS WOULD REMOVE 2 PIXELS FROM EACH BORDER WHEN USING 32 32 IMAGE
size_of_Filter2=(3,3)
size_of_pool=(2,2) # SCALE DOWN ALL FEATURE MAP TO GERNALIZE MORE, TO REDUCE OVERFITTING
no_Of_Nodes = 500 # NO. OF NODES IN HIDDEN LAYERS
model= Sequential()
model.add((Conv2D(no_Of_Filters,size_of_Filter,input_shape=(imageDimesions[0],imageDimesions[1],1),activation='relu'))) # ADDING MORE CONVOLUTION LAYERS = LESS FEATURES BUT CAN CAUSE ACCURACY TO INCREASE
model.add((Conv2D(no_Of_Filters, size_of_Filter, activation='relu')))
model.add(MaxPooling2D(pool_size=size_of_pool)) # DOES NOT EFFECT THE DEPTH/NO OF FILTERS
model.add((Conv2D(no_Of_Filters//2, size_of_Filter2,activation='relu')))
model.add((Conv2D(no_Of_Filters // 2, size_of_Filter2, activation='relu')))
model.add(MaxPooling2D(pool_size=size_of_pool))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(no_Of_Nodes,activation='relu'))
model.add(Dropout(0.5)) # INPUTS NODES TO DROP WITH EACH UPDATE 1 ALL 0 NONE
model.add(Dense(noOfClasses,activation='softmax')) # OUTPUT LAYER
# COMPILE MODEL
model.compile(Adam(lr=0.001),loss='categorical_crossentropy',metrics=['accuracy'])
return model
############################### TRAIN
model = myModel()
print(model.summary())
history=model.fit_generator(dataGen.flow(X_train,y_train,batch_size=int(batch_size_val)),steps_per_epoch=int(steps_per_epoch_val),epochs=int(epochs_val),validation_data=(X_validation,y_validation),shuffle=1)
############################### PLOT
plt.figure(1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.legend(['training','validation'])
plt.title('loss')
plt.xlabel('epoch')
plt.figure(2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.legend(['training','validation'])
plt.title('Acurracy')
plt.xlabel('epoch')
plt.show()
score =model.evaluate(X_test,y_test,verbose=0)
print('Test Score:',score[0])
print('Test Accuracy:',score[1])
#model.save(r"C:\Path\To\My\Directory\DetectBrick")
#print("model saved!!")
pickle_out= open(r"C:\Path\To\My\Directory\DetectBrick\model_trained.p","wb") # wb = WRITE BYTE
pickle.dump(model,pickle_out)
pickle_out.close()
cv2.waitKey(0)
This is the code for opening Model in OpenCV (at least trying to :) )
import numpy as np
import cv2
import pickle
from tensorflow import keras
import tensorflow as tf
import h5py
framewidth = 640
frameheight = 480
brightness = 180
threshold = 0.7
font = cv2.FONT_HERSHEY_SIMPLEX
camera = cv2.VideoCapture(0)
camera.set(3, framewidth)
camera.set(4, framewidth)
camera.set(10, framewidth)
#pb="saved_model.pb"
#pbtxt = "" #don't know if I need it .pbtxt
#model = cv2.dnn.readNetFromTensorflow(pb) #pbtxt file would be second parameter, #but dont know if needed
pickle_in = open("model_trained.p", "rb")
model = pickle.load(pickle_in)
def grayscale(img):
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
return img
def equalize(img):
img = cv2.equalizeHist(img)
return img
def preprocessing(img):
img = grayscale(img)
img = equalize(img)
img = img/255
return img
def getClassName(classNo):
if classNo == 0: return '3003'
elif classNo == 1: return '3010'
while camera.IsOpened():
boolean, frameoriginal = camera.read()
img = np.asarray(frameoriginal)
img = cv2.resize(img, (32,32))
img = preprocessing(img)
cv2.imshow("processed image", img)
img = img.reshape(1, 32, 32, 1)
cv2.putText(frameoriginal, "Klasse: ", (20,35), font, 0.75, (0,0,255), 2, cv2.LINE_AA)
cv2.putText(frameoriginal, "Genauigkeit: ", (20,75), font, 0.75, (255,0,0), 2, cv2.LINE_AA)
predictions = model.predict([img])
classIndex = model.predict_classes([img])
probabilityValue = np.amax(predictions)
if probabilityValue > threshold:
cv2.putText(frameoriginal, str(classIndex)+ " " + str(getClassName(classIndex)), (120,35), font, 0.75, (0,0,255),2, cv2.LINE_AA)
cv2.putText(frameoriginal, str(round(probabilityValue*100, 2)) + "%", (180,75), font, 0.75, (0,0,255),2, cv2.LINE_AA)
cv2.imshow("result", frameoriginal)
if cv2.waitKey(2) & OxFF == ord('q'):
break
cv2.destroyAllWindows()
camera.realease()
And here are the Versions I am using
Python: 3.10.5
Tensorflow (GPU): 2.9.1
CUDA: 11.2
Cudnn: 8.1
Pickle: 4.0
System: Windows 11
CPU: AMD Ryzen 5 5600G
GPU: GTX 1660 Super 6GB

pickle.dump(model,pickle_out) | TypeError: can't pickle _thread._local objects

After successfully training traffic signs with tensorflow I would like to pickle the results. I am using tensorflow version 2.0.0, keras 2.3.1 and python 3.7.11.
In the last lines I get the following error:
TypeError Traceback (most recent call last)
/var/folders/t1/39jk5mmd66b2prn30v34_61c0000gn/T/ipykernel_32516/1492349104.py in <module>
224 # STORE THE MODEL AS A PICKLE OBJECT
225 pickle_out= open("model_trained.p","wb") # wb = WRITE BYTE
--> 226 pickle.dump(model,pickle_out)
227 pickle_out.close()
228 cv2.waitKey(0)
TypeError: can't pickle _thread._local objects
Training pictures:
[https://drive.google.com/file/d/1AZeKw90Cb6GgamTBO3mvDdz6PjBwqCCt/view][1]
Labels:
[https://usercontent.one/wp/www.computervision.zone/wp-content/uploads/2020/08/labels.zip?media=1632743877][2]
Source code:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam
from keras.utils.np_utils import to_categorical
from tensorflow.keras.layers import Conv2D, Conv3D, MaxPooling2D, Dropout, Flatten
import cv2
from sklearn.model_selection import train_test_split
import pickle
import os
import pandas as pd
import random
from keras.preprocessing.image import ImageDataGenerator
tf.config.experimental.set_visible_devices([], 'GPU') # deactivates my slow gpu
################# Parameters #####################
path = "myData" # folder with all the class folders
labelFile = 'labels.csv' # file with all names of classes
batch_size_val=50 # how many to process together
steps_per_epoch_val=2000
epochs_val=10
imageDimesions = (32,32,3)
testRatio = 0.2 # if 1000 images split will 200 for testing
validationRatio = 0.2 # if 1000 images 20% of remaining 800 will be 160 for validation
###################################################
############################### Importing of the Images
count = 0
images = []
classNo = []
myList = os.listdir(path)
print("Total Classes Detected:",len(myList))
noOfClasses=len(myList)
print("Importing Classes.....")
for x in range (0,len(myList)-1):
myPicList = os.listdir(path+"/"+str(count))
for y in myPicList:
curImg = cv2.imread(path+"/"+str(count)+"/"+y)
images.append(curImg)
classNo.append(count)
print(count, end =" ")
count +=1
print(" ")
images = np.array(images)
classNo = np.array(classNo)
############################### Split Data
X_train, X_test, y_train, y_test = train_test_split(images, classNo, test_size=testRatio)
X_train, X_validation, y_train, y_validation = train_test_split(X_train, y_train, test_size=validationRatio)
# X_train = ARRAY OF IMAGES TO TRAIN
# y_train = CORRESPONDING CLASS ID
############################### TO CHECK IF NUMBER OF IMAGES MATCHES TO NUMBER OF LABELS FOR EACH DATA SET
print("Data Shapes")
print("Train",end = "");print(X_train.shape,y_train.shape)
print("Validation",end = "");print(X_validation.shape,y_validation.shape)
print("Test",end = "");print(X_test.shape,y_test.shape)
assert(X_train.shape[0]==y_train.shape[0]), "The number of images in not equal to the number of lables in training set"
assert(X_validation.shape[0]==y_validation.shape[0]), "The number of images in not equal to the number of lables in validation set"
assert(X_test.shape[0]==y_test.shape[0]), "The number of images in not equal to the number of lables in test set"
assert(X_train.shape[1:]==(imageDimesions))," The dimesions of the Training images are wrong "
assert(X_validation.shape[1:]==(imageDimesions))," The dimesionas of the Validation images are wrong "
assert(X_test.shape[1:]==(imageDimesions))," The dimesionas of the Test images are wrong"
############################### READ CSV FILE
data=pd.read_csv(labelFile)
print("data shape ",data.shape,type(data))
############################### DISPLAY SOME SAMPLES IMAGES OF ALL THE CLASSES
#num_of_samples = []
#cols = 5
#num_classes = noOfClasses
#fig, axs = plt.subplots(nrows=num_classes, ncols=cols, figsize=(5, 300))
#fig.tight_layout()
#for i in range(cols):
# for j,row in data.iterrows():
# x_selected = X_train[y_train == j]
# axs[j][i].imshow(x_selected[random.randint(0, len(x_selected)- 1), :, :], cmap=plt.get_cmap("gray"))
# axs[j][i].axis("off")
# if i == 2:
# axs[j][i].set_title(str(j)+ "-"+row["Name"])
# num_of_samples.append(len(x_selected))
############################### DISPLAY A BAR CHART SHOWING NO OF SAMPLES FOR EACH CATEGORY
#print(num_of_samples)
#plt.figure(figsize=(12, 4))
#plt.bar(range(0, num_classes), num_of_samples)
#plt.title("Distribution of the training dataset")
#plt.xlabel("Class number")
#plt.ylabel("Number of images")
#plt.show()
############################### PREPROCESSING THE IMAGES
def grayscale(img):
img = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
return img
def equalize(img):
img =cv2.equalizeHist(img)
return img
def preprocessing(img):
img = grayscale(img) # CONVERT TO GRAYSCALE
img = equalize(img) # STANDARDIZE THE LIGHTING IN AN IMAGE
img = img/255 # TO NORMALIZE VALUES BETWEEN 0 AND 1 INSTEAD OF 0 TO 255
return img
X_train=np.array(list(map(preprocessing,X_train))) # TO IRETATE AND PREPROCESS ALL IMAGES
X_validation=np.array(list(map(preprocessing,X_validation)))
X_test=np.array(list(map(preprocessing,X_test)))
cv2.imshow("GrayScale Images",X_train[random.randint(0,len(X_train)-1)]) # TO CHECK IF THE TRAINING IS DONE PROPERLY
############################### ADD A DEPTH OF 1
X_train=X_train.reshape(X_train.shape[0],X_train.shape[1],X_train.shape[2],1)
X_validation=X_validation.reshape(X_validation.shape[0],X_validation.shape[1],X_validation.shape[2],1)
X_test=X_test.reshape(X_test.shape[0],X_test.shape[1],X_test.shape[2],1)
############################### AUGMENTATAION OF IMAGES: TO MAKEIT MORE GENERIC
dataGen= ImageDataGenerator(width_shift_range=0.1, # 0.1 = 10% IF MORE THAN 1 E.G 10 THEN IT REFERS TO NO. OF PIXELS EG 10 PIXELS
height_shift_range=0.1,
zoom_range=0.2, # 0.2 MEANS CAN GO FROM 0.8 TO 1.2
shear_range=0.1, # MAGNITUDE OF SHEAR ANGLE
rotation_range=10) # DEGREES
dataGen.fit(X_train)
batches= dataGen.flow(X_train,y_train,batch_size=20) # REQUESTING DATA GENRATOR TO GENERATE IMAGES BATCH SIZE = NO. OF IMAGES CREAED EACH TIME ITS CALLED
X_batch,y_batch = next(batches)
# TO SHOW AGMENTED IMAGE SAMPLES
fig,axs=plt.subplots(1,15,figsize=(20,5))
fig.tight_layout()
for i in range(15):
axs[i].imshow(X_batch[i].reshape(imageDimesions[0],imageDimesions[1]))
axs[i].axis('off')
plt.show()
y_train = to_categorical(y_train,noOfClasses)
y_validation = to_categorical(y_validation,noOfClasses)
y_test = to_categorical(y_test,noOfClasses)
############################### CONVOLUTION NEURAL NETWORK MODEL
def myModel():
no_Of_Filters=60
size_of_Filter=(5,5) # THIS IS THE KERNEL THAT MOVE AROUND THE IMAGE TO GET THE FEATURES.
# THIS WOULD REMOVE 2 PIXELS FROM EACH BORDER WHEN USING 32 32 IMAGE
size_of_Filter2=(3,3)
size_of_pool=(2,2) # SCALE DOWN ALL FEATURE MAP TO GERNALIZE MORE, TO REDUCE OVERFITTING
no_Of_Nodes = 500 # NO. OF NODES IN HIDDEN LAYERS
model= Sequential()
model.add((Conv2D(no_Of_Filters,size_of_Filter,input_shape=(imageDimesions[0],imageDimesions[1],1),activation='relu'))) # ADDING MORE CONVOLUTION LAYERS = LESS FEATURES BUT CAN CAUSE ACCURACY TO INCREASE
model.add((Conv2D(no_Of_Filters, size_of_Filter, activation='relu')))
model.add(MaxPooling2D(pool_size=size_of_pool)) # DOES NOT EFFECT THE DEPTH/NO OF FILTERS
model.add((Conv2D(no_Of_Filters//2, size_of_Filter2,activation='relu')))
model.add((Conv2D(no_Of_Filters // 2, size_of_Filter2, activation='relu')))
model.add(MaxPooling2D(pool_size=size_of_pool))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(no_Of_Nodes,activation='relu'))
model.add(Dropout(0.5)) # INPUTS NODES TO DROP WITH EACH UPDATE 1 ALL 0 NONE
model.add(Dense(noOfClasses,activation='softmax')) # OUTPUT LAYER
# COMPILE MODEL
model.compile(Adam(lr=0.001),loss='categorical_crossentropy',metrics=['accuracy'])
return model
############################### TRAIN
model = myModel()
#print(model.summary())
history=model.fit_generator(dataGen.flow(X_train,y_train,batch_size=batch_size_val),steps_per_epoch=steps_per_epoch_val,epochs=epochs_val,validation_data=(X_validation,y_validation),shuffle=1)
############################### PLOT
plt.figure(1)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.legend(['training','validation'])
plt.title('loss')
plt.xlabel('epoch')
plt.figure(2)
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.legend(['training','validation'])
plt.title('Acurracy')
plt.xlabel('epoch')
plt.show()
score =model.evaluate(X_test,y_test,verbose=0)
print('Test Score:',score[0])
print('Test Accuracy:',score[1])
# STORE THE MODEL AS A PICKLE OBJECT
pickle_out= open("model_trained.p","wb") # wb = WRITE BYTE
pickle.dump(model,pickle_out)
pickle_out.close()
cv2.waitKey(0)
Thx for Your help! :)
[1]: https://drive.google.com/file/d/1AZeKw90Cb6GgamTBO3mvDdz6PjBwqCCt/view
[2]: https://usercontent.one/wp/www.computervision.zone/wp-content/uploads/2020/08/labels.zip?media=1632743877

As of now, Keras models are pickle-able. But we still recommend using model.save() to save model to disk.

The accuracy problem of hand sign gestures recognition with using CNN in Python

Im working on my senior project in my university and I have only 2 days to fix this problem.I created a hand gesture recognition with using CNN in Python.I used 78000 images with 50x50px values.But I got stuck in the last part of my model.I can not improve my accuracy.When I start to train the data with 100 epochs,the first 15 epochs show 0,039 accuracy and it is horrible,because of that I'm not waiting the end of the train.Maybe it happens because of the values of conv2d or pooling because I don't know how to put the correct values into conv2d,pooling etc.
I'm new and I could not fix the problem.If you help me,I will be grateful for you
The code I wrote is given below;
from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.layers import Dropout
from keras.preprocessing.image import ImageDataGenerator
import tensorflow as tf
import pickle
import cv2
import os
import matplotlib.pyplot as plt
import numpy as np
from tqdm import tqdm
from sklearn.model_selection import train_test_split
from PIL import Image
from numpy import asarray
DATADIR = "asl_alphabet_train"
CATEGORIES = ["A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z"]
X_train = []
y_train = []
X_test=[]
y_test=[]
IMG_SIZE=50
def create_training_data():
for category in CATEGORIES:
path = os.path.join(DATADIR,category) # create path to dogs and cats
class_num = CATEGORIES.index(category) # get the classification (0 or a 1).
for img in tqdm(os.listdir(path)): # iterate over each image per dogs and cats
try:
img_array = cv2.imread(os.path.join(path,img)) # convert to array
#new_array = cv2.resize(img_array, (28, 50 )) # resize to normalize data size
X_train.append(img_array) # add this to our trainingdata
# add this to our X_train
y_train.append(class_num) # add this to our X_train
except Exception as e: # in the interest in keeping the output clean...
pass
create_training_data()
X_train = asarray(X_train)
y_train = asarray(y_train)
"""
nsamples, nx, ny = X_train.shape
X_train = X_train.reshape((nsamples,nx*ny))
"""
X_train, X_test, y_train, y_test = train_test_split(X_train, y_train, test_size=0.2,random_state=0)
N = y_train.size
M = y_train.max()+1
resultArray = np.zeros((N,M),int)
idx = (np.arange(N)*M) + y_train
resultArray.ravel()[idx] = 1
y_train=resultArray
classifier=Sequential()
#convolution step
classifier.add(Convolution2D(filters=96, input_shape=(50,50,3), kernel_size=(11,11), padding='valid',activation="relu"))
#pooling step
classifier.add(MaxPooling2D(pool_size=(2,2)))
#convolution step
classifier.add(Convolution2D(filters=256,kernel_size=(11,11),padding="valid",activation="relu"))
#pooling step
classifier.add(MaxPooling2D(pool_size=(2,2)))
classifier.add(Convolution2D(filters=384,kernel_size=(3,3),padding="valid",activation="relu"))
classifier.add(MaxPooling2D(pool_size=(2,2)))
#flatten step
classifier.add(Flatten())
#Dense(Fully connected step)
classifier.add(Dense(output_dim=128,activation="relu"))
#Dropout to decrease the possibility of overfitting
classifier.add(Dropout(0.5))
#Dense to determine the output
classifier.add(Dense(output_dim=26,activation="softmax"))
#compile step
classifier.compile(optimizer="adam",loss="categorical_crossentropy",metrics=["accuracy"])
enter code here
classifier.fit(X_train,y_train,epochs=100,batch_size=32)
filename="CNN_TEST.sav"
pickle.dump(classifier, open(filename, 'wb'))
y_pred=classifier.predict(X_test)
print(y_pred)

Would recommend the following :
1) Reduce the kernel size in the first two convolutional layers of your model.
2) I believe the MaxPooling layer is not necessary after every convolutional layer. Do verify this.
3) A DropOut of 0.5 could drop out a large number of essential neurons, you might want to lower that.
4) Vary the number of epochs and see how your model performs each time.
Plot "train accuracy vs val accuracy" and "train loss vs val loss" at each attempt and see if your model overfits or underfits.

Model gives great training and testing accuracy but not working well when doing prediction

Hi there I am doing transfer learning with deep learning using VGG16 pre-trained model. I want to extract features from VGG16 to build my own model as I only have access to CPU. Here is my build and train setup.
import os
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from sklearn.metrics import confusion_matrix
from keras.applications import Xception, VGG16, ResNet50
conv_base = VGG16(weights='imagenet',include_top=False,input_shape=(224, 224, 3))
conv_base.summary()
base_dir = 'NewDCDatatset'
train_dir = os.path.join(base_dir, 'Train')
validation_dir = os.path.join(base_dir, 'Validation')
test_dir = os.path.join(base_dir, 'Test')
datagen = ImageDataGenerator(rescale=1./255)
batch_size = 5
def extract_features(directory, sample_count):
features = np.zeros(shape=(sample_count, 7 , 7 , 512))
labels = np.zeros(shape=(sample_count,2))
generator = datagen.flow_from_directory(directory,target_size=(224, 224),batch_size=batch_size,class_mode='categorical')
i = 0
for inputs_batch, labels_batch in generator:
features_batch = conv_base.predict(inputs_batch)
features[i * batch_size : (i + 1) * batch_size] = features_batch
labels[i * batch_size : (i + 1) * batch_size] = labels_batch
i += 1
if i * batch_size >= sample_count:
break
return features, labels
train_features, train_labels = extract_features(train_dir, 2000)
validation_features, validation_labels = extract_features(validation_dir,420 )
test_features, test_labels = extract_features(test_dir, 420)
train_features = np.reshape(train_features, (2000, 7 * 7 * 512))
validation_features = np.reshape(validation_features, (420, 7 * 7 * 512))
test_features = np.reshape(test_features, (420, 7 * 7 * 512))
from keras import models
from keras import layers
from keras import optimizers
model = models.Sequential()
model.add(layers.Dense(64, activation='relu', input_dim=7 * 7 * 512))
model.add(layers.Dense(2, activation='softmax'))
model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['acc'])
history = model.fit(train_features, train_labels,epochs=2,batch_size=5,shuffle=True)
print(model.evaluate(test_features,test_labels))
model.save('TLFACE.h5')
#predictions = model.predict_generator(test_features,steps = 5)
#print(predictions)
And here is the setup for the way I am predicting the new images through my model to classify cat and dog but I am not getting that much accurate results that seldom I am able to correctly classify any image. I don't what mistake I am doing, is it matter of resizing image or what when predicting.
from keras.models import load_model
deep = load_model('TLFACE.h5')
from PIL import Image
import numpy as np
import cv2
file_nam = '4705.jpg'
img = cv2.imread(file_nam)
img = cv2.imshow('frame',img)
cv2.waitKey(1000)
img = Image.open(file_nam).convert("L")
img = img.resize((256,98))
im2arr = np.array(img)
im2arr = im2arr.reshape(1,25088)
# Predicting the Test set r1esults
y_pred = deep.predict(im2arr)
print(y_pred)
print(y_pred[0][0])

According to your code, your model receives a feature vector of size 1 x 7*7*512 = 1 x 25088. This features is the encoding of an image in the conv_base model (implemented in your extract_features method).
However in your example at prediction time you just take an image, reshape it to (256,98), then flatten to (1,25088). You totally skip the extract_features phase. In other words, you try to predict on a totally different data then you use during training (and the automated testing under the fit method). You need to use the exact same preprocessing method for testing, which means first rescale the image to size 224x224x3, then extracting the features using the conv_base model, and finally use your new model to perform the prediction over the extracted features.
Having said that, I want to suggest that you use very "dirty" implementation, this is not organized properly and it easy to get confused that way. You actually want to build a single model that perform classification on an image, and not do all the heavy coding of extracting features etc. to do so you can just use something like:
pretrained_vgg16 = VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))
pretrained_vgg16.trainable = False # This is important if you don't want to modify your base model weights during training.
new_model = models.Sequential([pretrained_vgg16,
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.Dense(2,activation='softmax')])
In addition, please note that it is very importnat to decide if you want to modify the pre-trained model weights during training or not. if not, you need to envoke the pretrained_vgg16.trainable = False line.
Now you can use new_model.fit with a generator that load the images and rescale them to 224x224x3, without explicitly extracting the features. Another important comment is that the original VGG16 model was probably trained on a normalized data (which means pixels values were normalized to be in range [0,1]), you also need to perform the same preprocessing to your images for ideal results.

What what be the best pattern of layers for my Tensorflow/Keras model?

I am currently making an Airbus/Boeing classifier that can determine what aircraft the image is of. I have resized my images to be 1000 by 1000. Do any of you guys mind giving me an example f how I should go about constructing a model with Tensorflow and Keras? Also, what types of layers should I include? I am new to Tensorflow so I am not aware of the types of layers I should use for the maximum accuracy. Thank you. Here is my code:
import numpy as np
import cv2
import os
import tensorflow as tf
from tensorflow import keras
boeing_dir = '#'
airbus_dir = '#'
path = '#'
boeing_data = []
boeing_label = []
airbus_data = []
airbus_label = []
print('begun')
for filename in os.listdir(boeing_dir):
if filename.endswith(".jpg") or filename.endswith(".png") or filename.endswith(".jpeg"):
path_b = os.path.join(boeing_dir, filename)
im = cv2.imread(path_b)
im = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY)
im = cv2.imread(path_b, cv2.IMREAD_GRAYSCALE)
im = cv2.resize(im, (1000, 1000))
boeing_data.append(im)
boeing_label.append(0)
print(im.shape)
for filename in os.listdir(airbus_dir):
if filename.endswith(".jpg") or filename.endswith(".png") or filename.endswith(".jpeg"):
path_b = os.path.join(airbus_dir, filename)
im = cv2.imread(path_b)
im = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY)
im = cv2.imread(path_b, cv2.IMREAD_GRAYSCALE)
im = cv2.resize(im, (1000, 1000))
airbus_data.append(im)
airbus_label.append(1)
print(im.shape)
training_data = boeing_data + airbus_data
training_label = boeing_label + airbus_label
print(training_data)
print(training_label)
training_data = np.array(training_data)#.reshape(-1, 1000, 1000, 1)
training_label = np.asarray(training_label)
I have split the images into images and labels. After this, how do I construct my model?

You should probably experiment first with a prebuilt model. You should do some research before building your own model.
from tensorflow.keras.applications.densenet import DenseNet201 as b #
from tensorflow.keras import layers, models, optimizers, regularizers
from tensorflow.keras.preprocessing import image
from tensorflow.keras.models import Model
The applications module offers a lot more models to choose from.
traingen =image.ImageDataGenerator(rescale=1./255)
validationgen = image.ImageDataGenerator(rescale=1./255)
testgen = image.ImageDataGenerator(rescale=1./255)
train = traingen.flow_from_directory("train",target_size=(1000, 1000), batch_size=batchSize, shuffle=True)
val = validationgen.flow_from_directory("validation",target_size=(1000, 1000), batch_size=size, shuffle=False)
test=testgen.flow_from_directory("test",target_size=(1000, 1000), batch_size=size, shuffle=False)
For this you should split your data in three folders, one for train, validation and test data. inside of these folders you should have two more folders called "boeing" and "airbus" that yontain those respective images. This code reads the images from tose three folders and inferes the class from the folder names
base = b(include_top=False, # include top refers if to include the dense layer that acts as the classifier, this is not needed in this case because we build one later
weights="imagenet", # these are the weights used by the model that were learned from being trained on the imagenet dataset
input_shape=(1000, 1000, 3)) # in case you work with rgb images, if you work with gray images you should use input_shape=(1000, 1000, 1)
x = base.output
predictions = layers.Dense(2, activation='softmax')(x)
model = Model(inputs=base.input, outputs=predictions)
model.compile(optimizer=optimizers.Adam(),
loss='categorical_crossentropy',
metrics=["categorical_accuracy"])
history = model.fit_generator(train,
steps_per_epoch=np.ceil(numImages/batchSize),
epochs=50,
verbose=2,
validation_data= val,
validation_steps=np.ceil(nunImages/size)
)
By using a pretrained network you can take advantage of the knowledge learned from imagenet, those patterns shoud be useful since you are working with natural images.

Categories

Resources