I am trying to do some multiclass image classification with VGG-style, but unfortunately, I cannot reach my goal, and I think I have a stupid mistake in my code.
I have almost 16K images with 4 categories, let's say 1,2,3 and 4. With the following code, I import all the images. I don't want to write for each category in order not to make a huge post. 4 times lines like this:
path_101 = ("/media/data/working_dir/categories/101/")
train_101 = []
for png in os.listdir(path_101):
imageread = img.imread(path_101+png)
imageread = cv2.resize(imageread, (320,240)) #resizing
train_101.append(imageread)
After I concatenate them into one x_data variable.
x_data = np.concatenate((train_101, train_102, train_104, train_105), axis=0)
Following to this, I create my categorical data and do One Hot encoding:
# We create our classify data.
one = np.ones(len(train_101))
four = np.ones(len(train_104)) +3
two = np.ones(len(train_102)) + 1
five = np.ones(len(train_105)) + 4
y = np.concatenate((one, two, four, five), axis= 0).reshape(-1,1)
from sklearn.preprocessing import OneHotEncoder
labels=OneHotEncoder(categories = "auto", handle_unknown = "ignore")
y=labels.fit_transform(y).toarray()
And after these all I do the training part as you can see below:
from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x_data, y, test_size = 0.2)
# import Keras and layers libraries
from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import cross_val_score
from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPool2D , Flatten
model = Sequential()
model.add(Conv2D(input_shape=(240,320,3),filters=64,kernel_size=(3,3),padding="same", activation="relu"))
model.add(Conv2D(filters=64,kernel_size=(3,3),padding="same", activation="relu"))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=128, kernel_size=(3,3), padding="same", activation="relu"))
model.add(Conv2D(filters=128, kernel_size=(3,3), padding="same", activation="relu"))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu"))
model.add(Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu"))
model.add(Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu"))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Flatten())
model.add(Dense(units=512,activation="relu"))
model.add(Dense(units=64, activation="relu"))
model.add(Dense(units=4, activation="softmax"))
model.add(Dense(units=4, activation="softmax"))
from keras.optimizers import Adam
opt = Adam(lr=0.001)
model.compile(optimizer=opt , loss=keras.losses.categorical_crossentropy, metrics=['accuracy'])
model.fit(x_train, y_train, verbose=1, epochs=5, validation_split=0.2)
results = model.evaluate(x_test, y_test, batch_size=64)
After I was having terrible results like 25% of accuracy. And for each epoch it was not changing. I taught that maybe my training output is not going well. I try to predict every 100th image of my initial db with this code:
prs=[]
for k in np.arange(1,15000,100):
imgg=x_data[k]
imgg=imgg[np.newaxis,...]
pr=model.predict_classes(imgg)
prs.append(pr[0])
print(prs)
And I was getting only 0s, or 2s depending on the class that has more input images. So something was wrong with it.
I am new in Neural Networks and maybe I did some amateur stuff.
How can I have class 0, if my y data was one-hot encoded 1,2,4 and 5?
I was thinking about the y variable, if I trained with one-hot encoding, maybe I should decode for predictions or what?
Thanks in advance! Do not hesitate to ask for details
Related
I am trying to run the Keras tutorial mentioned below in python:
#Import Libraries
from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPool2D , Flatten
from keras.optimizers import SGD
#model details
vgg19 = Sequential()
vgg19.add(Conv2D(input_shape=(224,224,3),filters=64,kernel_size=(3,3),padding="same", activation="relu"))
vgg19.add(Conv2D(filters=64,kernel_size=(3,3),padding="same", activation="relu"))
vgg19.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
vgg19.add(Conv2D(filters=128, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(Conv2D(filters=128, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
vgg19.add(Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(Conv2D(filters=256, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
vgg19.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
vgg19.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(Conv2D(filters=512, kernel_size=(3,3), padding="same", activation="relu"))
vgg19.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
vgg19.add(Flatten())
vgg19.add(Dense(units=4096,activation="relu"))
vgg19.add(Dense(units=4096,activation="relu"))
vgg19.add(Dense(units=10, activation="softmax"))
#Preparing Dataset
from keras.datasets import cifar10
from keras.utils import to_categorical
(X, Y), (tsX, tsY) = cifar10.load_data()
# Use a one-hot-encoding
Y = to_categorical(Y)
tsY = to_categorical(tsY)
# Change datatype to float
X = X.astype('float32')
tsX = tsX.astype('float32')
# Scale X and tsX so each entry is between 0 and 1
X = X / 255.0
tsX = tsX / 255.0
#training
optimizer = SGD(lr=0.001, momentum=0.9)
vgg19.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
history = vgg19.fit(X, Y, epochs=100, batch_size=64, validation_data=(tsX, tsY), verbose=0)
Upon training the model, I am getting the below mentioned value error:
ValueError: Input 0 of layer dense_9 is incompatible with the layer: expected axis -1 of input shape to have value 25088 but received input with shape (None, 512)
Please suggest, how to fix the input shape and would be better if someone can provide a brief explanation of the issue.
Thanks in advance!
You can check the shape of X using X.shape.
It clearly shows that the shape of X is (50000,32,32,3)
So your first layer should be like this:
vgg19 = Sequential()
vgg19.add(Conv2D(input_shape=(32,32,3),filters=64,kernel_size=(3,3),padding="same", activation="relu"))
I am trying to use CNN for feature extraction and XGboost for classification of a image data. I researched and found that it could be done by extracting the data after the convolution layers. I found some source code for similar problem and tried on my own.
model = Sequential()
model.add(Conv2D(32, kernel_size=(3,3), strides=(1,1), padding='same', activation="relu", input_shape = data.shape[1:]))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))
model.add(Conv2D(64, kernel_size=(3,3), padding='same', strides=(1,1), activation="relu") )
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2))) #max pool window 2x2
model.add(Conv2D(128, kernel_size=(3,3), padding='same', strides=(1,1), activation="relu"))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2))) #max pool window 2x2
model.add(Conv2D(256, kernel_size=(3,3), padding='same', strides=(1,1), activation="relu"))
model.add(MaxPool2D(pool_size=(2,2), strides=(2,2))) #max pool window 2x2
model.add(Flatten())
model.add(Dense(128, activation="relu", name='firstDenseLayer'))
model.add(Dense(1, activation="sigmoid"))
# model.summary()
# print(model)
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model.fit(data, label, batch_size=16, epochs=10, validation_data=(val_data, val_label))
Below i accessed the dense layer named "firstDenseLayer".
import xgboost as xgb
from keras.models import Model
layerName = 'firstDenseLayer'
intermediate_layer_model = Model(inputs=model.input,
outputs=model.get_layer(layerName).output)
intermediate_output = intermediate_layer_model.predict(data)
from xgboost import XGBClassifier
xgbmodel = XGBClassifier(objective='multi:softmax', num_class= 2)
xgbmodel.fit(intermediate_output, label)
xgbmodel.score(intermediate_output, label)
As i am new in this, i have several confusions.
How the data is being flowed. After i extract the features of the pictures via convolution layers, how do i actually access the data from there?
What is this line of code doing? What data is it extracting?
intermediate_output = intermediate_layer_model.predict(data)
When i omit(keep commented out) the below line,
model.fit(data, label, batch_size=16, epochs=10, validation_data=(val_data, val_label))
from the first snippet and run the XGboost model directly the XGboost gives low accuracy and when i don't it gives higher accuracy. Why is it being like that?
Kindly help me out. I am stuck with this for quite a while. I am just trying to access the extracted features data from the last convolution layer and use that data to do classification using XGboost. As i tried to follow the method that i found from online, i am not sure if it is the the only way of doing it. If there is another way kindly let me know.
The model.fit(...) line does what you would expect, it trains the convnet defined by model on some data and labels. Your classifier yielding lower accuracy when you're using randomly initialized weights (i.e. without running fit) is not surprising.
intermediate_layer_model is constructed as a keras model whose output is the dense layer just before the output of model. Note the name parameter given to the dense layer in the construction of model.
You could just as easily give a name to one of the Conv2D layers and access it the same way. Alternatively, you could store the layer in a python variable, i.e. instead of
model.add(Conv2D(256, kernel_size=(3,3), padding='same', strides=(1,1), activation="relu"))
in the model construction it could say
last_conv_layer = Conv2D(256, kernel_size=(3,3), padding='same', strides=(1,1), activation="relu")
model.add(last_conv_layer)
Then for the intermediate_layer_model you put
intermediate_layer_model = Model(inputs=model.input, outputs=last_conv_layer.output)
I'm building a CNN model to classify images, however, I guess that my model is not learning because of the constant values of accuracy and loss function. See my code below:
Building the images train, test and validation datasets
import pandas as pd
from keras_preprocessing.image import ImageDataGenerator
import numpy as np
#Creating three datasets from the the 3 .txt files.
trainingfile = pd.read_table('data/training.txt', delim_whitespace=True, names=('class', 'image'))
testingfile = pd.read_table('data/testing.txt', delim_whitespace=True, names=('class', 'image'))
validationfile = pd.read_table('data/validation.txt', delim_whitespace=True, names=('class', 'image'))
#Changing target variable type
trainingfile = trainingfile.replace([0, 1, 2], ['class0', 'class1', 'class2'])
testingfile = testingfile.replace([0, 1, 2], ['class0', 'class1', 'class2'])
validationfile = validationfile.replace([0, 1, 2], ['class0', 'class1', 'class2'])
#Data augmentation
datagen=ImageDataGenerator()
train_datagen = ImageDataGenerator(
#Apliquem una mica de rotació no gaire ja que generalment les fotografies estaran centrades
rotation_range=5,
zoom_range=0.1)
#Final datasets containing the images
train=train_datagen.flow_from_dataframe(dataframe=trainingfile, directory="data/", x_col="image", y_col="class", class_mode="categorical", target_size=(256,256),color_mode='rgb',batch_size=32)
test=datagen.flow_from_dataframe(dataframe=testingfile, directory="data/", x_col="image", y_col="class", class_mode="categorical", target_size=(256,256),color_mode='rgb',batch_size=32)
#No data augmentation to the validation dataset.
validation=datagen.flow_from_dataframe(dataframe=validationfile, directory="data/", x_col="image", y_col="class", class_mode="categorical", target_size=(256,256),color_mode='rgb', batch_size=32)
CNN model
from keras.models import Sequential
from keras.layers import Dense, Conv2D, Flatten, Activation, Dropout, MaxPooling2D, BatchNormalization
from keras.constraints import maxnorm
#Creació del model
model = Sequential()
#1r bloc convolució
model.add(Conv2D(32, kernel_size = (3, 3), activation='relu', input_shape=(256, 256,3)))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
#2n bloc convolució
model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
#3r bloc convolució
model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
#4t bloc convolució
model.add(Conv2D(96, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
#5e bloc convolució
model.add(Conv2D(32, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
#Dropout
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
#model.add(Dropout(0.3))
model.add(Dense(3, activation = 'softmax'))
from keras import regularizers, optimizers
from keras.optimizers import RMSprop
from keras.callbacks import EarlyStopping
# Compile model
model.compile(optimizer='adam',loss="categorical_crossentropy",metrics=["accuracy"])
# Early stopping
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1,patience=10)
Training the model
h=model.fit_generator(generator=train,
validation_data=validation,
epochs=50,
callbacks=[es])
Results
It is the first time that I use fit_generator and perhaps I'm not using it properly?
As I can see from the results images, you are training just for 1 epoch. This could be because the EarlyStopping is too strict. Try adding patience=3to the EarlyStopping callback.
es = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=3)
EDIT
Overfitting example:
Check this post to know more about how to deal with overfitting.
I'm working on a binary classification problem. I was getting 69% accuracy at first, but kept running out of memory so I shrunk certain parameters, now it's coming up 0. Any idea whats going on?
model = Sequential()
from keras.layers import Dropout
model.add(Conv2D(96, kernel_size=11, padding="same", input_shape=(300, 300, 1), activation = 'relu'))
model.add(MaxPooling2D(pool_size=(3, 3), strides=(2, 2)))
model.add(Conv2D(128, kernel_size=3, padding="same", activation = 'relu'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
model.add(Conv2D(128, kernel_size=3, padding="same", activation = 'relu'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
from keras.layers.core import Activation
model.add(Flatten())
# model.add(Dense(units=1000, activation='relu' ))
model.add(Dense(units= 300, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1))
model.add(Activation("softmax"))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(
featurewise_center=True,
rotation_range=90,
fill_mode='nearest',
validation_split = 0.2
)
datagen.fit(train)
train_generator = datagen.flow(train, train_labels, batch_size=8)
# # fits the model on batches with real-time data augmentation:
history = model.fit_generator(generator=train_generator,
use_multiprocessing=True,
steps_per_epoch = len(train_generator) / 8,
epochs = 5,
workers=20)
Softmax should only be used if you have a multiclass classification problem. You have a single output from your Dense layer, so you should use sigmoid.
I trained the following model using Keras (Version 2.2.4):
# imports ...
model = Sequential()
model.add(Conv2D(filters=64, kernel_size=5, data_format="channels_last", activation="relu"))
model.add(BatchNormalization())
model.add(MaxPooling2D(data_format="channels_last"))
model.add(Conv2D(filters=32, kernel_size=3, data_format="channels_last", activation="relu"))
model.add(BatchNormalization())
model.add(MaxPooling2D(data_format="channels_last"))
model.add(Flatten(data_format="channels_last"))
model.add(Dense(units=256, activation="relu"))
model.add(Dense(units=128, activation="relu"))
model.add(Dense(units=32, activation="relu"))
model.add(Dense(units=8, activation="softmax"))
# training ...
model.save("model.h5")
The inputs are 28 x 28 grayscale images of shape(28, 28, 1).
I converted the model with tensorflowjs_converter and now I want to load it in my website using TensorFlow.js (Version 1.1.0):
tf.loadLayersModel('./model/model.json')
This produces the following error:
The first layer in a Sequential model must get an `inputShape` or `batchInputShape` argument.
at new e (errors.ts:48)
at e.add (models.ts:440)
at e.fromConfig (models.ts:1020)
at vp (generic_utils.ts:277)
at nd (serialization.ts:31)
at models.ts:299
at common.ts:14
at Object.next (common.ts:14)
at o (common.ts:14)
How can I fix this error without having to retrain the model?
Try to adjust your neural net to this format:
input_img = Input(batch_shape=(None, 28,28,1))
layer1=Conv2D(filters=64, kernel_size=5, data_format="channels_last", activation="relu")(input_img)
layer2=BatchNormalization()(layer1)
.
.
.
final_layer=Dense(units=8, activation="softmax")(previous_layer)
... and so forth. At the end:
model = Model(inputs = input_img, outputs = final_layer)
You have to specify the input shape in the Conv2D layer of your keras model.
# imports ...
model = Sequential()
model.add(Conv2D(input_shape=(28, 28, 1), filters=64, kernel_size=5, data_format="channels_last", activation="relu"))
model.add(BatchNormalization())
model.add(MaxPooling2D(data_format="channels_last"))
model.add(Conv2D(filters=32, kernel_size=3, data_format="channels_last", activation="relu"))
model.add(BatchNormalization())
model.add(MaxPooling2D(data_format="channels_last"))
model.add(Flatten(data_format="channels_last"))
model.add(Dense(units=256, activation="relu"))
model.add(Dense(units=128, activation="relu"))
model.add(Dense(units=32, activation="relu"))
model.add(Dense(units=8, activation="softmax"))
# training ...
model.save("model.h5")
Best approach is to change your keras model and retrain.
Anyway, if you cannot retrain your network, you can manually edit your model.json file.
You need to find the input layer in your model.json file and add
"config": {
...
"batch_input_shape": [
null,
28,
28,
1
]
...
}