Predicting head poses to get output in the form of roll pitch yawn by using the model in the image, but the result is not accurate. Dataset has a total of 5500 images
What values should be:
Roll: 0.67°
Pitch: -4.89°
Yaw: 22.57°
Values from my model:
Roll: 356.10°
Pitch: 1036.82°
Yaw: 532.35°
`
#opening the dataset
a,b,c,d = pkl.load(open('samples.pkl', 'rb'))
#concatenating the 2 arrays
x = np.concatenate((a,b), axis=0)
y = np.concatenate((c,d), axis=0)
#Assigning degrees to roll pitch and yaw
roll, pitch, yaw = y[:, 0], y[:, 1], y[:, 2]
#test and train division
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=2)
x_val, x_test, y_val, y_test = train_test_split(x_test, y_test, test_size=0.2, random_state=2)
print(x_train.shape, y_train.shape)
print(x_val.shape, y_val.shape)
print(x_test.shape, y_test.shape)
#standazing the test and train sets
std = StandardScaler()
std.fit(x_train)
x_train = std.transform(x_train)
x_val = std.transform(x_val)
x_test = std.transform(x_test)
BATCH_SIZE = 64
EPOCHS = 100
#model of cnn
model = Sequential()
model.add(Dense(units=20, activation='relu', kernel_regularizer='l2', input_dim=x.shape[1]))
model.add(Dense(units=10, activation='relu', kernel_regularizer='l2'))
model.add(Dense(units=3, activation='linear'))
print(model.summary())
#compiling the model
callback_list = [EarlyStopping(monitor='val_loss', patience=25)]
model.compile(optimizer='adam', loss='mean_squared_error',metrics=['accuracy'])
hist = model.fit(x=x_train, y=y_train, validation_data=(x_val, y_val), batch_size=BATCH_SIZE, epochs=EPOCHS, callbacks=callback_list)
model.save('model.h5')
`
First of all predicting head poses from images is not a basic regression problem, it is, in fact, a computer vision problem that needs some computer vision-based approach.
Your model is a dummy MLP that will never perform well with the given problem.
I'm giving you some pointers, you still need to figure out which one works based on experimentation.
Instead of using an MLP, start with a ConvNet. Here's a simple CNN for regression.
model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same',
input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(3))
Get rid of the Flatten, and use Global Pooling if the model overfits.
Try to normalize your output. Instead of blindly regressing, range the roll, pitch to 0 - 1 using min-max normalization. Use sigmoid in the last layer in this case.
Here's a paper to get the basic idea and start with a baseline model: https://aip.scitation.org/doi/pdf/10.1063/1.4982509
Related
I'm new to deep learning. I'm trying to train a model that identifies plant diseases, and I'm using this dataset that has a bunch of images. I'm aware that this is a lot of data, and I'm only using the color subfolder. I want to use all of the data in that subfolder. The thing is, Kaggle only provides 13GB of memory as of now, and my session keeps restarting because my script is trying to use more memory than it has. Here is my code:
### There are some imports here that I removed because there is a lot of them
NUM_CLASSES = 38
IMG_SIZE = 150
x = []
y = []
def train_data_gen(DIR, ID):
for img in os.listdir(DIR):
try:
path = DIR + '/' + img
img = plt.imread(path)
img = cv2.resize(img, (IMG_SIZE,IMG_SIZE))
if img.shape == (IMG_SIZE, IMG_SIZE, 3):
x.append(img)
y.append(ID)
except:
None
#--
for DIR in os.listdir('../input/plantvillage-dataset/color/'):
train_data_gen('../input/plantvillage-dataset/color/' + DIR, DIR)
print(DIR)
#
print('reached label encoder')
le = LabelEncoder()
y = le.fit_transform(y)
x = np.array(x)
y = to_categorical(y, NUM_CLASSES)
print('data split')
x_train,x_test,y_train,y_test = train_test_split(x, y, test_size = 0.15)
x_train,x_val,y_train,y_val = train_test_split(x_train, y_train, test_size = 0.15)
print('datagen')
datagen = ImageDataGenerator(
featurewise_center=False,
samplewise_center=False,
samplewise_std_normalization=False,
rotation_range=60,
zoom_range = 0.1,
width_shift_range=0.1,
height_shift_range=0.1,
shear_range=0.1,
fill_mode = "reflect"
)
print('datagen_fit')
datagen.fit(x_train)
print('model')
model = Sequential()
model.add(Conv2D(64, kernel_size=(3, 3), strides=2, padding='Same', activation='relu', input_shape=(IMG_SIZE, IMG_SIZE, 3)))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(BatchNormalization())
#model.add(Dropout(0.2))
model.add(Conv2D(128, kernel_size=(3, 3), strides=2, padding='Same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(BatchNormalization())
#model.add(Dropout(0.3))
model.add(Conv2D(128, kernel_size=(3, 3), strides=2, padding='Same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(BatchNormalization())
#model.add(Dropout(0.3))
model.add(Conv2D(128, kernel_size=(3, 3), strides=2, padding='Same', activation='relu'))
#model.add(MaxPooling2D(pool_size=(2,2),strides=(2,2)))
model.add(BatchNormalization())
model.add(Dropout(0.3))
model.add(Flatten())
model.add(Dense(1024, activation='relu'))
#model.add(Dropout(0.5))
model.add(BatchNormalization())
model.add(Dense(NUM_CLASSES, activation='softmax'))
print('Model compile')
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
print('Model Fit')
model.fit_generator(datagen.flow(x_train,y_train,batch_size=32), epochs=75, steps_per_epoch=x_train.shape[0]//32, validation_data=(x_val, y_val), verbose=1)
model.save('plantus_model')
I've placed print functions through out my code to see where the actual problem is. The part where it stops is right when I fit datagen. I don't think it's that one thing that uses up so much ram, but all the stuff before it too. How can I reduce RAM usage so I can actually get to training my model?
Thank you in advance for providing answers and constructive feedback.
you are using datagen.fit. This is only necessary if you are setting any of the parameters featurewise_center, samplewise_center, featurewise_std_normalization,
samplewise_std_normalization or zca_whitening to True. Since you are not doing that you do not need to fit the data set. This should avoid you using to much memory.
I use a CNN model to train image classification , it got great accuracy at test and validation (98% and 97%), but when use my image to predict it alway go wrong, here is my code:
BATCH_SIZE = 30
IMG_HEIGHT = 256
IMG_WIDTH = 256
STEPS_PER_EPOCH = np.ceil(image_count/BATCH_SIZE)
train_data_gen = image_generator.flow_from_directory(directory=str(data_dir),
batch_size=BATCH_SIZE,
shuffle=True,
target_size=(IMG_HEIGHT, IMG_WIDTH),
classes = list(CLASS_NAMES))
here is prepare for dataset and data argumentation:
imgDataGen=ImageDataGenerator(
validation_split=0.2,
rescale=1/255,
horizontal_flip=True,
zoom_range=0.3,
rotation_range=15.,
width_shift_range=0.1,
height_shift_range=0.1,
)
prepare data:
train_dataset = imgDataGen.flow_from_directory(
directory=str(data_dir),
target_size = (IMG_HEIGHT, IMG_WIDTH),
classes = list(CLASS_NAMES),
batch_size = BATCH_SIZE,
subset = 'training'
)
val_dataset = imgDataGen.flow_from_directory(
directory=str(data_dir),
target_size = (IMG_HEIGHT, IMG_WIDTH),
classes = list(CLASS_NAMES),
batch_size =BATCH_SIZE,
subset = 'validation'
)
the model
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(256, 256, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(6, activation='sigmoid'))
complie:
model.compile(loss='binary_crossentropy',
optimizer=keras.optimizers.SGD(learning_rate=0.001,momentum=0.9),
metrics=['acc'])
train
history = model.fit_generator(
train_dataset,
validation_data = val_dataset,
workers=10,
epochs=20,
)
It get pretty high accuracy 98% on test and 97% on validation
but when i try with my code to predict
def prepare(filepath):
IMG_SIZE=256
img_array=cv2.imread(filepath)
new_array= cv2.resize(img_array,(IMG_SIZE,IMG_SIZE))
return new_array.reshape(1,IMG_SIZE,IMG_SIZE,3)
model=tf.keras.models.load_model('trained-model.h5',compile=False)
#np.set_printoptions(formatter={'float_kind':'{:f}'.format})
predict=model.predict([prepare('cat.jpg')])
pred_name = CATEGORIES[np.argmax(predict)]
print(pred_name)
it got wrong, with cat image it go for dog and dog for cat, but sometime it go right, just i think 98% is more accurate than this, if i try 5 image of cats it fail 3 or 4 images
so it because dataset or because of code?
please help, thanks
So in your second code-block you have this:
rescale=1/255
This is for normalizing your image into the range [0;1]. So every image gets rescaled (/normalized) before going through the network. But in you las code-block where you test it on an image you didnt add normalization. Try adding that to your "prepare" function:
def prepare(filepath):
IMG_SIZE = 256
img_array = cv2.imread(filepath)
# add this:
img_array = image_array / 255
new_array = cv2.resize(img_array,(IMG_SIZE,IMG_SIZE))
return new_array.reshape(1,IMG_SIZE,IMG_SIZE,3)
I'm trying to create a NN which can recognize the letters of the alphabet (26 classes). I apologize for the lengthy post, but I've included all my relevant code to be as clear as possible. In the end I've explained the issue.
The following block is where I name the paths correctly, and standardize/normalize the image and get it ready for training.
X = [] # Image data
y = [] # Labels
datagen = ImageDataGenerator(samplewise_center=True)
for path in imagepaths:
img = cv2.imread(path)
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img = cv2.resize(img, (200, 200))
img = image.img_to_array(img)
img = datagen.standardize(img)
X.append(img)
# Processing label in image path
category = path.split("\\")[1]
#print(category)
split = (category.split("_"))
if int(split[0]) == 0:
label = int(split[1])
else:
label = int(split[0])
y.append(label)
# Turn X and y into np.array to speed up train_test_split
X = np.array(X, dtype="uint8")
X = X.reshape(len(imagepaths), 200, 200, 1)
y = np.array(y)
Creating the test set.
ts = 0.3
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=ts, random_state=42)
Creating a model. Dense of 26, one output for each letter, and size 200,200,1 to match input:
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(200, 200, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(26, activation='softmax'))
Model compiler and fit:
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=1, batch_size=64, verbose=1, validation_data=(X_test, y_test))
The issue comes here where the output of my model.fit is:
Train on 54600 samples, validate on 23400 samples
Epoch 1/1
54600/54600 [==============================] - 79s 1ms/step - loss: nan - accuracy: 1.8315e-05 - val_loss: nan - val_accuracy: 0.0000e+00
I understand that it may not be high accuracy or anything from the get-go, but why are the losses nan? I posted elsewhere and I was first told to normalize my data (which I fixed for this post). Then I was told that it was possible that my dataset is corrupt or leaking - this is not the case because when I don't do 26 letters at once, it works perfectly fine. (Meaning I tested the code, using letters A-J, dense = 10, etc) and got a high accuracy of about 95%.
Any help is appreciated, I've been scratching my had at this for hours!
In my root folder images I have three folders called 0, 1, 2. In folder 0 there are no smilies. In folder 1 there are happy handwritten smilies and in folder 2 there are sad handwritten smilies.
The images are jpg color images with the dimension 26x26.
This is my code
from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPooling2D, Dropout, Flatten
import numpy as np
import os
import cv2
from sklearn.model_selection import train_test_split
def getImages(path, classes):
folder = os.listdir(path)
classes_counter = 0
images = []
images_classes = []
for x in range (0,len(folder)):
myPicList = os.listdir(path+"/"+ str(classes[classes_counter]))
for pic in myPicList:
img_path = path+"/" + str(classes[classes_counter]) + "/" + pic
img = cv2.imread(img_path)
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
images.append(img)
images_classes.append(classes_counter)
classes_counter +=1
images = np.array(images, dtype="float") / 255
return images, images_classes
def createModel(classes, images_dimension):
classes_amount = len(np.unique(classes))
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), padding='same', activation='relu', input_shape=images_dimension))
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, kernel_size=(3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, kernel_size=(3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(classes_amount, activation='softmax'))
return model
labels = [0,1,2]
images, images_classes = getImages('training-images', labels)
images_dimension=(26,26,3)
X_train, X_test, Y_train, Y_test = train_test_split(images, images_classes, test_size=0.2) # if 1000 images split will 200 for testing
X_train, X_validation, Y_train, Y_validation = train_test_split(X_train, Y_train, test_size=0.2) # if 1000 images 20% of remaining 800 will be 160 for validation
model = createModel(labels, images_dimension)
batch_size = 20
epochs = 100
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit(X_train, Y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(X_validation, Y_validation))
model.evaluate(X_test,Y_test,verbose=0)
In the Line images = np.array(images, dtype="float") / 255 I get the error:
Traceback (most recent call last):
File "train-nn.py", line 54, in <module>
images, images_classes = getImages('training-images', labels)
File "train-nn.py", line 24, in getImages
images = np.array(images, dtype="float") / 255
ValueError: could not broadcast input array from shape (26,26) into shape (26)
I think something is wrong with the datastructure or with the array structure. I have no clue what I did wrong. Maybe someone known this problem and can give me a hint!
Here you can download the whole project as zip file.
http://fileshare.mynotiz.de/cnn-handwritten-smilies.zip
I found the problem. One of my test-data images was not in the format of 26x26 it was 26x23.
I am receiving the following error when I fit the network - ValueError: Error when checking target: expected dense_6 to have shape (2,) but got array with shape (22,)
As far as I can tell the shape should be correct given how the dataset is split? Any help is greatly appreciated, thanks!
The dataset can be found here: https://archive.ics.uci.edu/ml/machine-learning-databases/mushroom/agaricus-lepiota.data
from keras.layers import Dense
from keras.models import Sequential
import keras.utils
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
import numpy as np
import pandas as pd
# seed weights
np.random.seed(3)
# import dataset
data = pd.read_csv('agaricus-lepiota.csv', delimiter=',')
# encode labels as integers so the can be one-hot-encoded which takes int matrix
le = preprocessing.LabelEncoder()
data = data.apply(le.fit_transform)
# one-hot-encode string data (now type int)
ohe = preprocessing.OneHotEncoder(sparse=False)
data = ohe.fit_transform(data)
X = data[:, 1:23]
Y = data[:, 0:1]
# split into test and train set
x_train, y_train, x_test, y_test = train_test_split(X, Y, test_size=.2, random_state=5)
# create model
model = Sequential()
model.add(Dense(500, input_dim=22, activation='relu'))
model.add(Dense(300, activation='relu'))
model.add(Dense(100, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(25, activation='relu'))
model.add(Dense(2, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=1000, batch_size=25)
I found 2 errors in your code.
1)
x_train, y_train, x_test, y_test = train_test_split(X, Y, test_size=.2, random_state=5)
must be
x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=.2, random_state=5)
check this to learn more about the function.
2)
You have only one column in y_train. But the last layer in your model adds two columns. So instead of
model = Sequential()
model.add(Dense(500, input_dim=22, activation='relu'))
model.add(Dense(300, activation='relu'))
model.add(Dense(100, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(25, activation='relu'))
model.add(Dense(2, activation='sigmoid'))
use this:
model = Sequential()
model.add(Dense(500, input_dim=22, activation='relu'))
model.add(Dense(300, activation='relu'))
model.add(Dense(100, activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(25, activation='relu'))
model.add(Dense(1, activation='sigmoid'))