I'm following this tutorial from Nabeel Ahmed to create your own emotion detector using Keras (I'm a noob) and I've found a strange behaviour that I'd like to understand. The input data is a bunch of 48x48 images, each one with an integer value between 0 and 6 (each number stands for an emotion label), which represents the emotion present in the image.
train_X.shape -> (28709, 2304) // training-data, 28709 images of 48x48
train_Y.shape -> (28709,) //The emotion present in each image as an integer, 1 = happiness, 2 = sadness, etc.
val_X.shape -> (3589, 2304)
val_Y.shape -> (3589, )
In order to feed the data into the model, train_X and val_X are reshaped (as the tutorial explains)
train_X.shape -> (28709, 48, 48, 1)
val_X.shape -> (3589, 48, 48, 1)
The model, as it is in the tutorial, is this one:
model = Sequential()
input_shape = (48,48,1)
#1st convolution layer
model.add(Conv2D(64, (5, 5), input_shape=input_shape,activation='relu', padding='same'))
model.add(Conv2D(64, (5, 5), activation='relu', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
#2nd convolution layer
model.add(Conv2D(128, (5, 5),activation='relu',padding='same'))
model.add(Conv2D(128, (5, 5),activation='relu',padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
#3rd convolution layer
model.add(Conv2D(256, (3, 3),activation='relu',padding='same'))
model.add(Conv2D(256, (3, 3),activation='relu',padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(128))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dropout(0.2))
################################################################
model.add(Dense(7)) # <- problematic line
################################################################
model.add(Activation('softmax'))
my_optimiser = tf.keras.optimizers.Adam(
learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=False,
name='Adam')
model.compile(loss='categorical_crossentropy', metrics=['accuracy'],optimizer=my_optimiser)
However, when I try to use it, using the tutorial snippet, I get an error in the line of the validation_data like this
history = model.fit(train_X,
train_Y,
batch_size=64,
epochs=80,
verbose=1,
validation_data=(val_X, val_Y),
shuffle=True)
ValueError: Shapes (None, 1) and (None, 7) are incompatible
After reviewing the code and the documentation about the fit method, my only idea was to change the 7 in the last Dense layer of the model to 1, which mysteriously works. I'd like to know what is happening here if anyone could give me a hint.
You seem to be working with sparse integer labels, where each sample belongs to one of seven classes {0, 1, 2, 3, 4, 5, 6}, so I would recommend using SparseCategoricalCrossentropy instead of CategoricalCrossentropy as your loss function. Just change this parameter and your model should work fine. If you want to use CategoricalCrossentropy, you will have to one-hot encode your labels, for example with:
train_Y = tf.keras.utils.to_categorical(train_Y, num_classes=7)
Related
I'm trying to implement transfer learning on my own model but failing. My implementation follows the guides here
https://keras.io/guides/transfer_learning/
How to do transfer-learning on our own models?
https://github.com/anujshah1003/Transfer-Learning-in-keras---custom-data/blob/master/transfer_learning_resnet50_custom_data.py
tensoflow 2.4.1
Keras 2.4.3
Old Model (Works really well):
model = Sequential()
inputShape = (256, 256, 3)
chanDim = -1
# CONV => RELU => POOL
model.add(Conv2D(32, (3, 3), padding="same", input_shape=inputShape))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(MaxPooling2D(pool_size=(3, 3)))
model.add(Dropout(0.25))
# (CONV => RELU) * 2 => POOL
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
# (CONV => RELU) * 2 => POOL
model.add(Conv2D(128, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(Conv2D(128, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
# first (and only) set of FC => RELU layers
model.add(Flatten())
model.add(Dense(1024))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Dropout(0.5))
# softmax classifier
model.add(Dense(classes))
model.add(Activation("softmax"))
Transfer Learning:
old_model = load_model('old.model')
# removes top 2 activation layers
for i in range(2):
old_model.pop()
# mark loaded layers as not trainable
for layer in old_model.layers:
layer.trainable = False
# initialize the new model
in_puts = Input(shape=(256, 256, 3))
count = len(old_model.layers)
ll = old_model.layers[count - 1].output
classes = len(lb.classes_)
ll = Dense(classes)(ll)
ll = Activation("softmax", name="activation3_" + NODE)(ll)
model = Model(inputs=in_puts, outputs=ll) # ERROR
opt = Adam(lr=INIT_LR, decay=INIT_LR / EPOCHS)
model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
# train the network
H = model.fit(x_train, y_train, validation_data=(x_test, y_test), steps_per_epoch=len(x_train) // BS, epochs=EPOCHS, verbose=1)
# save the model to disk
model.save("new.model")
ERROR
ValueError: Graph disconnected: cannot obtain value for tensor
KerasTensor(type_spec=TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32,
name='conv2d_input'), name='conv2d_input', description="created by layer 'conv2d_input'") at
layer "conv2d". The following previous layers were accessed without issue: []
Here a simple way to operate transfer learning with your model
classes = 10
sub_old_model = Model(old_model.input, old_model.layers[-3].output)
sub_old_model.trainable = False
ll = Dense(classes)(sub_old_model.output)
ll = Activation("softmax")(ll)
model = Model(inputs=sub_old_model.input, outputs=ll)
Firstly, create a sub-model with layers from the old model that you want to freeze (trainable = False). In our example, we take all the layers excluding the last Dense and the Softmax activation.
Then pass the sub-model output into the new trainable layers.
At this point, you simply need to create a new model instance to assemble all the pieces
With help from #Marco Cerliani, I was able to solve the problem by changing to LabelEncoder and sparse_categorical_crossentropy
I'm trying to train a 3D CNN model in Keras, but i'm getting this error when I execute the cell:
ValueError: Input 0 of layer conv3d_8 is incompatible with the layer: : expected min_ndim=5, found ndim=4. Full shape received: [None, 4, 150, 150]
My input data is a numpy array with image data. Here are the shapes (I know that 53 is too few, but it's just for studying purposes):
Training data shape: (53, 4, 150, 150)
Training labels shape: (53, 1)
Validation data shape: (14, 4, 150, 150)
Validation labels shape: (14, 1)
The model I'm trying to use is:
# Create the model
model = Sequential()
model.add(Conv3D(32, kernel_size=(3, 3, 3), activation='relu', kernel_initializer='he_uniform', input_shape=(4,150,150)))
model.add(MaxPooling3D(pool_size=(2, 2, 2)))
model.add(BatchNormalization(center=True, scale=True))
model.add(Dropout(0.5))
model.add(Conv3D(64, kernel_size=(3, 3, 3), activation='relu', kernel_initializer='he_uniform'))
model.add(MaxPooling3D(pool_size=(2, 2, 2)))
model.add(BatchNormalization(center=True, scale=True))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(4, activation='softmax'))
# Compile the model
model.compile(loss='categorical_crossentropy',
optimizer=keras.optimizers.Adam(lr=0.001),
metrics=['accuracy'])
model.summary()
# Fit data to model
history = model.fit(treino3d, treino3d_labels,
epochs=40)
Can someone help, please?
Thanks a lot!
It doesn't seem like you need Conv3D layers for this task. Use Conv2D instead, and use only 1 or 2 values in kernel_size and pool_size.
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu',
kernel_initializer='he_uniform',
input_shape=(4,150,150)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(BatchNormalization(center=True, scale=True))
model.add(Dropout(0.5))
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu',
kernel_initializer='he_uniform'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(BatchNormalization(center=True, scale=True))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(256, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(4, activation='softmax'))
Your channel dimension comes first so you will need to tell Keras. Use this line:
tf.keras.backend.set_image_data_format('channels_first')
Or set this parameter in every Conv2D or MaxPooling2D layer:
data_format='channels_first'
Or permute the dimensions of the input tensor to have shape (54, 150, 150, 4):
np.transpose(x, (0, 2, 3, 1))
Full functioning, corrected example:
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'
from tensorflow.keras.layers import *
import numpy as np
from tensorflow.keras.models import Sequential
xtrain = np.random.rand(53, 4, 150, 150)
ytrain = np.random.randint(0, 4, (53, 1))
xtrain = np.transpose(xtrain, (0, 2, 3, 1))
model = Sequential()
model.add(Conv2D(8, kernel_size=(3, 3), activation='relu',
kernel_initializer='he_uniform', input_shape=xtrain.shape[1:]))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(BatchNormalization(center=True, scale=True))
model.add(Dropout(0.5))
model.add(Conv2D(8, kernel_size=(3, 3), activation='relu',
kernel_initializer='he_uniform'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(BatchNormalization(center=True, scale=True))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(32, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(32, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(4, activation='softmax'))
model.compile(loss='sparse_categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.summary()
history = model.fit(xtrain, ytrain, epochs=1)
32/53 [=================>............] - ETA: 2s - loss: 1.8215 - acc: 0.2812
53/53 [==============================] - 5s 91ms/sample - loss: 1.9651 - acc: 0.2264
The first answer is correct. As I can recall (I didn't work with MRI for a couple of years), each slice is represented by 4 channels and each voxel in channel contains an information about the same physical position. Therefore, it would be correct approach to apply 2D-convolution for the image of 4 channels.
For 2D convolution I would recommend to define your input with input_shape=(150,150,4), or even as input_shape=(None, None, 4) - it is more generic. So you don't need to play with channel_first or channel_last configuration. channel_last is default format, as I remember
If you insist to use Conv3d, your input shape should be: input_shape=(150,150,4,1) or (None, None, None, 1)
and trainset should follow shape like: np.random.rand(53, 150, 150, 4, 1)
I have train & test image data for which Shapes are given below.
X_test.shape , y_test.shape , X_train.shape , y_train.shape
((277, 128, 128, 3), (277, 1), (1157, 128, 128, 3), (1157, 1))
I am training a model
def baseline_model():
filters = 100
model = Sequential()
model.add(Conv2D(filters, (3, 3), input_shape=(128, 128, 3), padding='same', activation='relu'))
#model.add(Dropout(0.2))
model.add(BatchNormalization())
model.add(Conv2D(filters, (3, 3), activation='relu', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
#model.add(Flatten())
model.add(Conv2D(filters, (3, 3), activation='relu', padding='same'))
model.add(BatchNormalization())
model.add(Conv2D(filters, (3, 3), activation='relu', padding='same'))
model.add(Activation('linear'))
model.add(BatchNormalization())
model.add(Dense(512, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
# Compile model
lrate = 0.01
epochs = 10
decay = lrate/epochs
sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)
model.compile(loss='sparse_categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
print(model.summary())
return model
But I am getting an error Given below
Error when checking target: expected dense_35 to have 4 dimensions,
but got array with shape (1157, 1)
Please tell me what mistake I am making and how to fix this. I have attached snapshot of model summary
One thing you have probably forgotten to do is adding a Flatten layer right before the first Dense layer:
model.add(BatchNormalization())
model.add(Flatten()) # flatten the output of previous layer before feeding it to Dense layer
model.add(Dense(512, activation='relu'))
You need it because Dense layer does not flatten its input; rather, it is applied on the last dimension.
Although dense_35 needs to feed with 4 dimension data, according to the error, the network feed with 2 dimension data which is the label vector.
i have been studying Keras ConvLSTM2D: ValueError on output layer
i want to use the same code but i want to do regression ( single value ).
I dont know how to do this. And i also dont understand the use of last layers of this post code. Why is averagepolling3d used?
the code from link is
model = Sequential()
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
input_shape=(None, 135, 240, 1),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(AveragePooling3D((1, 135, 240)))
model.add(Reshape((-1, 40)))
model.add(Dense(
units=9,
activation='sigmoid'))
model.compile(
loss='categorical_crossentropy',
optimizer='adadelta'
)
AveragePooling3D is used to reduce each frame in a sequence to a single value + to reduce the #parameters in the Dense Layer. So, the dimension becomes (None, 40 , 1 , 1 ,1 ). Then, using Reshape allows it to use for fully-connected part.
Also, as in Keras ConvLSTM2D: ValueError on output layer, AveragePooling3D is used instead of GlobalMaxPooling2D since data is 5D and Global operations leaves only (batch_size, channels) which is not desirable in your case.
I want to predict numerical Y variable using time-series image data.
My dataset has these dimensions:
X: (7000, 3, 128, 128)
Y: (7000, 1)
7000 is the number of samples, each sample is 128 x 128 image and 3 is the number of channels (RGB). The range of the Y variable is 0-1.
Therefore, I want to apply an LSTM model after a CNN. But if I feed the CNN features to LSTM, so many errors occurs. Please help me.
This is my code:
size = 128
model = Sequential()
model.add(Convolution2D(32, (3, 3), input_shape=(3, size, size)))
model.add(Activation('relu'))
model.add(Convolution2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Convolution2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(Convolution2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
#### LSTM #####
model.add(LSTM(50)) ???????????????????????????
model.add(Flatten())
model.add(Dense(32))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='mse', optimizer= 'adam')
return model