Connecting recurrent layer after cnn, what does tf.expand_dims do? - python

I want to use a model for classifying eight class of images. I think using convolutional layers before recurrent layers can work for my problem. But, there is a problem using recurrent layers immediately after convolutional or dense layer which causes tensorflow give following error.
Input 0 is incompatible with layer simple_rnn_1: expected ndim=3, found ndim=2
I solve this problem using Tensorflow expand_dims() function using inside a Lambda layer. It seems worked properly but, I want to be sure about my model is working right. Despite looking at relevant documentation, I couldn't understand what expand_dims() done to make the model work.
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Dropout, Flatten, Dense, Lambda, SimpleRNN
from tensorflow import expand_dims
def create_model():
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(50, 50, 1), padding='same',
activation='relu'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Lambda(lambda x: expand_dims(model.output, axis=-1)))
model.add(SimpleRNN(64, return_sequences=False))
model.add(Dense(units=8, activation='softmax'))
model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['accuracy'])
return model
I want to interpret CNN output as sequential information using recurrent layers (LSTM, GRU, other recurrent models). Am I doing right using Lambda layer with expand_dims()?

The Flatten layer reduces the dimensions of the images to (Batch Length, Feature dimensions), and the recurrent neural networks expect 3 dimensions for the input instead of 2, as it needs a new dimension for the time-series inputs as in (Batch Length, Time Dimensions, Feature Dimensions)
The expand_dims simply adds an extra dimension that transforms your flattened outputs to (Batch Length, Features Dimensions, 1). This means that the RNN would now work, and it will consider the features dimensions as the time dimension for the data, thus going throw them in an orderly fashion and doing what you intended.

Related

how do I check that CNN model correctly implemented or not?

How do I make it confirm that the dropout model function parameter is correct or not?
Here, I have coded a CNN architecture but I am not able to confirm that architecture correctly implemented or not? how do I confirm it. The input images dimension is 88x128 which has been taken from a research paper (https://ieeexplore.ieee.org/abstract/document/7550060).
I am following architecture and parameters present in attached figure.
Architectural diagram and parametric table
The textual information is present into this figure:
(textual information's)
import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
import numpy as np
import cv2
import os
from keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array
from keras.layers.normalization import BatchNormalization
batch_size = 4
num_classes = 35
epochs = 40
#178, 256, 1
model = Sequential()
model.add(Conv2D(filters=96, input_shape=(128, 88, 1), kernel_size=(18, 18), strides=1, activation='relu', padding='valid'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
model.add(BatchNormalization())
model.add(Conv2D(filters=96, kernel_size=(45, 45), strides=1, activation='relu', padding='valid'))
model.add(MaxPooling2D(pool_size=(3, 3), strides=2))
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(1024))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.summary()
Try to reproduce the paper's result. If the results are not the same (or different by a huge margin), then, you should consider your implementation as incorrect.

How to select an appropriate value for step in predict_generator?

So I have built a convolutional neural network in Keras for classification. I am loading the training, validation and testing images from the flow_from_directory function in ImageDataGenerator package in keras. I am able to train the neural network perfectly and I am using the predict_generator function to predict the class labels for my test images (79,726 images).
So my code looks like this:
network_output=model.predict_generator(test_set,steps=79726 ,verbose=1)
The output of the network is basically probabilities for each of my 10 classes. So the dimension of network_output should be 79726 rows and 10 columns, however, the actual dimensions of network_output is 255074 rows and 10 columns. Which is more outputs than it should be ! I found the number of output changes with steps parameter. How can I select the value of step to get the correct number of outputs.
Please let me know in the comments/answer section if further clarifications/ code is needed to explain the problem better.
Edit: here is the network structure: input_size is32
from keras.models import Sequential
from keras.layers import Flatten,BatchNormalization
from keras.layers import Dense,Dropout
from keras.layers.convolutional import Conv2D
from keras.layers.pooling import MaxPooling2D
from keras.layers.normalization import BatchNormalization
model = Sequential()
model.add(Conv2D(16, 3, input_shape = (input_size, input_size, 3),activation = 'relu'))
model.add(Conv2D(16, 3, activation = 'relu'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2,2)))
model.add(Conv2D(32, 3, activation = 'relu'))
model.add(Conv2D(32, 3, activation = 'relu'))
​model.add(MaxPooling2D(pool_size=(2, 2), strides=(2,2)))
model.add(Flatten())
model.add(Dense(10, activation='softmax'))
model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
​

Adding LSTM layers before the softmax layer

I would like to add an LSTM layer before the softmax layer so that I can keep track of the context of a sequence and use it for prediction. Following is my implementation but I get every time the following error. Please help me to solve this error.
ValueError: Input 0 is incompatible with layer lstm_1: expected ndim=3, found ndim=2
common_model = Sequential()
common_model.add(Conv2D(32, (3, 3), input_shape=self.state_size, padding='same', activation='relu'))
common_model.add(Dropout(0.2))
common_model.add(Conv2D(32, (3, 3), activation='relu', padding='same'))
common_model.add(MaxPooling2D(pool_size=(2, 2)))
common_model.add(Flatten())
common_model.add(Dense(512, activation='relu'))
common_model.add(Dropout(0.5))
common_model.add(Dense(512, activation='relu'))
common_model.add(Dropout(0.5))
common_model.add(Dense(512, activation='relu'))
common_model.add(Dropout(0.5))
agent_model = Sequential()
agent_model.add(common_model)
agent_model.add(LSTM(512, return_sequences=False))
agent_model.add(Dense(self.action_size, activation='softmax'))
agent_model.compile(loss='categorical_crossentropy', optimizer=Adam(lr=self.agent_learning_rate))
critic_model = Sequential()
critic_model.add(common_model)
critic_model.add(Dense(1, activation='linear'))
critic_model.compile(loss="mse", optimizer=Adam(lr=self.critic_learning_rate))
I still don't quite understand the purpose of appending LSTM after Dense, but that error can be explained:
Because in Keras, the LSTM accept the input tensor like (?, m, n) which need to have 3 dims, while the output of Dense is (?, p) which has 2 dims.
You may want to try Embedding or Reshape layer, for example:
model.add(Embedding(512, 64, input_length=512))
or
model.add(Reshape((512, 64)))
Also it is good to check some examples of using LSTM: https://github.com/keras-team/keras/tree/master/examples

How do you use Keras LeakyReLU in Python?

I am trying to produce a CNN using Keras, and wrote the following code:
batch_size = 64
epochs = 20
num_classes = 5
cnn_model = Sequential()
cnn_model.add(Conv2D(32, kernel_size=(3, 3), activation='linear',
input_shape=(380, 380, 1), padding='same'))
cnn_model.add(Activation('relu'))
cnn_model.add(MaxPooling2D((2, 2), padding='same'))
cnn_model.add(Conv2D(64, (3, 3), activation='linear', padding='same'))
cnn_model.add(Activation('relu'))
cnn_model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
cnn_model.add(Conv2D(128, (3, 3), activation='linear', padding='same'))
cnn_model.add(Activation('relu'))
cnn_model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
cnn_model.add(Flatten())
cnn_model.add(Dense(128, activation='linear'))
cnn_model.add(Activation('relu'))
cnn_model.add(Dense(num_classes, activation='softmax'))
cnn_model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adam(), metrics=['accuracy'])
I want to use Keras's LeakyReLU activation layer instead of using Activation('relu'). However, I tried using LeakyReLU(alpha=0.1) in place, but this is an activation layer in Keras, and I get an error about using an activation layer and not an activation function.
How can I use LeakyReLU in this example?
All advanced activations in Keras, including LeakyReLU, are available as layers, and not as activations; therefore, you should use it as such:
from keras.layers import LeakyReLU
# instead of cnn_model.add(Activation('relu'))
# use
cnn_model.add(LeakyReLU(alpha=0.1))
Sometimes you just want a drop-in replacement for a built-in activation layer, and not having to add extra activation layers just for this purpose.
For that, you can use the fact that the activation argument can be a callable object.
lrelu = lambda x: tf.keras.activations.relu(x, alpha=0.1)
model.add(Conv2D(..., activation=lrelu, ...)
Since a Layer is also a callable object, you could also simply use
model.add(Conv2D(..., activation=tf.keras.layers.LeakyReLU(alpha=0.1), ...)
which now works in TF2. This is a better solution as this avoids the need to use a custom_object during loading as #ChristophorusReyhan mentionned.
you can import the function to make the code cleaner and then use it like any other activation.
if you choose not to define alpha, don't forget to add brackets "LeakyReLU()"
from tensorflow.keras.layers import LeakyReLU
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(512, activation=LeakyReLU()))
model.add(tf.keras.layers.Dense(512, activation=LeakyReLU(alpha=0.1)))

Activation function error in a 1D CNN in Keras

I'm creating a model to classify if the input waverform contains rising edge of SDA of I2C line.
My input has 20000 datapoints and 100 training data.
I've initially found an answer regarding the input in here Keras 1D CNN: How to specify dimension correctly?
However, I'm getting an error in the activation function:
ValueError: Error when checking target: expected activation_1 to have 3 dimensions, but got array with shape (100, 1)
My model is:
model.add(Conv1D(filters=n_filter,
kernel_size=input_filter_length,
strides=1,
activation='relu',
input_shape=(20000,1)))
model.add(BatchNormalization())
model.add(MaxPooling1D(pool_size=4, strides=None))
model.add(Dense(1))
model.add(Activation("sigmoid"))
adam = Adam(lr=learning_rate)
model.compile(optimizer= adam, loss='binary_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_label,
nb_epoch=10,
batch_size=batch_size, shuffle=True)
score = np.asarray(model.evaluate(test_new_data, test_label, batch_size=batch_size))*100.0
I can't determine the problem in here. On why the activation function expects a 3D tensor.
The problem lies in the fact that starting from keras 2.0, a Dense layer applied to a sequence will apply the layer to each time step - so given a sequence it will produce a sequence. So your Dense is actually producing a sequence of 1-element vectors and this causes your problem (as your target is not a sequence).
There are several ways on how to reduce a sequence to a vector and then apply a Dense to it:
GlobalPooling:
You may use GlobalPooling layers like GlobalAveragePooling1D or GlobalMaxPooling1D, eg.:
model.add(Conv1D(filters=n_filter,
kernel_size=input_filter_length,
strides=1,
activation='relu',
input_shape=(20000,1)))
model.add(BatchNormalization())
model.add(GlobalMaxPooling1D(pool_size=4, strides=None))
model.add(Dense(1))
model.add(Activation("sigmoid"))
Flattening:
You might colapse the whole sequence to a single vector using Flatten layer:
model.add(Conv1D(filters=n_filter,
kernel_size=input_filter_length,
strides=1,
activation='relu',
input_shape=(20000,1)))
model.add(BatchNormalization())
model.add(MaxPooling1D(pool_size=4, strides=None))
model.add(Flatten())
model.add(Dense(1))
model.add(Activation("sigmoid"))
RNN Postprocessing:
You could also add a recurrent layer on a top of your sequence and make it to return only the last output:
model.add(Conv1D(filters=n_filter,
kernel_size=input_filter_length,
strides=1,
activation='relu',
input_shape=(20000,1)))
model.add(BatchNormalization())
model.add(MaxPooling1D(pool_size=4, strides=None))
model.add(SimpleRNN(10, return_sequences=False))
model.add(Dense(1))
model.add(Activation("sigmoid"))
Conv1D has its output with 3 dimensions (and it will keep like that until the Dense layer).
Conv output: (BatchSize, Length, Filters)
For the Dense layer to output only one result, you need to add a Flatten() or Reshape((shape)) layer, to make it (BatchSize, Lenght) only.
If you call model.summary(), you will see exactly what shape each layer is outputting. You have to adjust the output to be exactly the same shape as the array you pass as the correct results. The None that appears in those shapes is the batch size and may be ignored.
About your model: I think you need more convolution layers, reducing the number of filters gradually, because condensing so much data in a single Dense layer does not usually bring good results.
About dimensions: keras layers toturial and samples

Categories

Resources