Develop Reference

Saving Layer Values of CNN- Keras

I have created the following simple CNN in Keras (borrowed from a DeepLizard tutorial).
model = Sequential([
Conv2D(filters = 10, kernel_size = (3, 3), activation = 'relu', padding = 'same', input_shape = (320, 320, 3)),
MaxPool2D(pool_size = (2, 2), strides = 2),
Conv2D(filters = 10, kernel_size = (3, 3), activation = 'relu', padding = 'same'),
MaxPool2D(pool_size = (2, 2), strides = 2),
Flatten(),
Dense(20, activation = 'softmax'),
])
model.summary()
model.compile(optimizer = Adam(lr = 0.0001), loss = 'categorical_crossentropy', metrics =['accuracy'])
model.fit(x = train_batches, validation_data = valid_batches, epochs = 10, verbose = 2)
predictions = model.predict(x = test_batches, verbose = 0)
As you can see, I am saving the predictions generated by the model to a dataframe named "predictions". But I am also interested in saving the outputs for each of the MaxPool2D layers, the Conv2D layer, and the flatten layer as well. Is there a way that I can save the outputs of those layers to dataframes/lists as well? Is there a functionality for this in Keras?
Thank you!

You can use model.get_layer() function to extract any layer of your model. Visit the documentation here: https://keras.io/api/models/model/#getlayer-method

Thank you for your responses. They led me in the right direction. Here is the solution I ended up utilizing. I recreated the model, but configured the predictions to output the desired layer (in this case, "conv2d", the first convolutional layer). This produces a 4-D array as an output, where the 1st dimension corresponds to the input, the 2nd and 3rd dimensions are the two dimensions of a filter's outputted feature map, and the 4th dimension corresponds to the n-filters being used in that layer (in this case, the 4th dimension would be '10'). My next quest is to find a way to split that 4 dimensional array into separate 3-dimensional arrays, where each new array corresponds to a distinct filter. In this case, I would be looking for 10 3-dimensional arrays, one for each of the filters used in the first convolutional layer.
from keras.models import Model
model = Sequential([
Conv2D(filters = 10, kernel_size = (3, 3), activation = 'relu', padding = 'same', input_shape = (320, 320, 3)),
MaxPool2D(pool_size = (2, 2), strides = 2),
Conv2D(filters = 10, kernel_size = (3, 3), activation = 'relu', padding = 'same'),
MaxPool2D(pool_size = (2, 2), strides = 2),
Flatten(),
Dense(20, activation = 'softmax'),
])
layer_name = 'conv2d'
intermediate_layer_model = Model(inputs=model.input, outputs=model.get_layer(layer_name).output)
intermediate_output = intermediate_layer_model.predict(valid_batches)

Adding convolution layers on top of inception V3 model

I need to train an image classifier using inception V3 model from Keras. The images pass through 5 Conv2D layers and 2 MaxPool2D layers before entering the pre-trained inception V3 model. However my code gives me an error of ValueError: Depth of input (64) is not a multiple of input depth of filter (3) for 'inception_v3_4/conv2d_123/convolution' (op: 'Conv2D') with input shapes: [?,2,2,224], [3,3,3,32]
I reckon my output shape from previous layers is not compatible with the input shape required by Inception. But i am not able to solve it or is it even possible to solve this error. I am a beginner in machine learning and any light in this matter will be greatly appreciated.
My code is as follows:
inception_model = inception_v3.InceptionV3(weights='imagenet', include_top = False)
for layer in inception_model.layers:
layer.trainable = False
input_layer = Input(shape=(224,224,3)) #Image resolution is 224x224 pixels
x = Conv2D(128, (7, 7), padding='same', activation='relu', strides=(2, 2))(input_layer)
x = Conv2D(128, (7, 7), padding='same', activation='relu', strides=(2, 2))(x)
x = Conv2D(64, (7, 7), padding='same', activation='relu', strides=(2, 2))(x)
x = MaxPool2D((3, 3), padding='same',strides=(2, 2))(x)
x = Conv2D(64, (7, 7), padding='same', activation='relu', strides=(2, 2))(x)
x = Conv2D(64, (7, 7), padding='same', activation='relu', strides=(2, 2))(x)
x = MaxPool2D((4, 4), padding='same', strides=(2, 2))(x)
x = inception_model (x) #Error in this line
x = GlobalAveragePooling2D()(x)
predictions = Dense(11, activation='softmax')(x) #I have 11 classes of image to classify
model = Model(inputs = input_layer, outputs=predictions)
model.compile(optimizer=Adam(lr=0.001), loss='categorical_crossentropy', metrics=['acc'])
model.summary()

Like #CAFEBABE said it would be almost useless to do this because the feature map can have almost 3 values but if you still want to try it then you can do this:
x = Conv2D(3, (7, 7), padding='same', activation='relu', strides=(2, 2))(input_layer)
Another thing you will have to remember is that like you used 5 Conv2D and 2 MaxPooling layers above but you can't do that because even in the Inception model there are Conv2D and max-pooling layers which will take the dimensions to negative and give an error. I tried with 2 Conv2D layers and got an error so at max you can use 1.
Also when you are specifying InceptionV3 model specify the input shape.
input_layer = Input(shape=(224,224,3)) #Image resolution is 224x224 pixels
x = Conv2D(128, (7, 7), padding='same', activation='relu', strides=(2, 2))(input_layer)
inception_model = tf.keras.applications.InceptionV3(weights='imagenet', include_top = False, input_shape=x.shape[0])
for layer in inception_model.layers:
layer.trainable = False
x = inception_model (x)
x = GlobalAveragePooling2D()(x)
predictions = Dense(11, activation='softmax')(x) #I have 11 classes of image to classify
model = Model(inputs = input_layer, outputs=predictions)
model.compile(optimizer=Adam(lr=0.001), loss='categorical_crossentropy', metrics=['acc'])
model.summary()
This would work but I doubt it will help the model. Anyways try it who knows what will happen.

Looping and concatenating in Keras Sequential API

Lets say I have a model defined like this:
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import (BatchNormalization, concatenate,
Conv2D, Conv2DTranspose, DepthwiseConv2D,
Dropout, Input, MaxPooling2D,
ReLU, ZeroPadding2D)
input_layer = Input((64, 64, 3))
conv1 = Conv2D(16, (3, 3), padding="same")(input_layer)
conv1 = BatchNormalization()(conv1)
conv1 = ReLU()(conv1)
pool1 = MaxPooling2D((2,2))(conv1)
conv2 = Conv2D(32, (3, 3), padding="same")(pool1)
conv2 = BatchNormalization()(conv2)
conv2 = ReLU()(conv2)
pool2 = MaxPooling2D((2,2))(conv2)
conv3 = Conv2D(64, (3, 3), padding="same")(pool2)
conv3 = BatchNormalization()(conv3)
conv3 = ReLU()(conv3)
pool3 = MaxPooling2D((2,2))(conv3)
mid = Conv2D(128, (3, 3), padding="same")(pool3)
mid = BatchNormalization()(mid)
mid = ReLU()(mid)
dconv3 = Conv2DTranspose(64, (3, 3), strides=(2, 2), padding="same")(mid)
cat3 = concatenate([dconv3, conv3])
dconv2 = Conv2DTranspose(32, (3, 3), strides=(2, 2), padding="same")(dconv3)
cat2 = concatenate([dconv2, conv2])
dconv1 = Conv2DTranspose(16, (3, 3), strides=(2, 2), padding="same")(dconv2)
cat1 = concatenate([dconv1, conv1])
output_layer = Conv2D(1, (1,1), padding="same", activation="sigmoid")(dconv1)
model = Model(input_layer, output_layer)
The model is a very simple UNET which requires that the down sample blocks be concatenated with the upsample blocks. Lets now imagine that I want to define this exact model but with some arbitrary depth aka 2, 3, 4, 5 etc.. downsample and upsample blocks. Instead of having to go in and manually modify the parameters, I would like to automate the model building.
I am very close to accomplishing this, but I fail during concatenation. See below.
class configurable_model():
def __init__(self, csize, channels, start_neurons, depth):
self.csize = csize
self.channels = channels
self.start_neurons = start_neurons
self.depth = depth
def _convblock(self, factor, name):
layer = Sequential(name=name)
layer.add(Conv2D(self.start_neurons * factor, (3, 3), padding="same"))
layer.add(BatchNormalization())
layer.add(ReLU())
return layer
def build_model(self):
model = Sequential()
model.add(Input((self.csize, self.csize, self.channels), name='input'))
factor = 1
for idx in range(self.depth):
model.add(self._convblock(factor, f'downblock{idx}'))
model.add(MaxPooling2D((2,2)))
factor *= 2
model.add(self._convblock(factor, name='middle'))
for idx in reversed(range(self.depth)):
factor //= 2
model.add(Conv2DTranspose(self.start_neurons * factor, (3, 3),
strides=(2, 2), padding="same",
name=f'upblock{idx}'))
#how do I do the concatenation??
model.add(concatenate([model.get_layer(f'upblock{idx}'),
model.get_layer(f'downblock{idx}')]))
model.add(Conv2D(1, (1,1), padding="same",
activation="sigmoid", name='output'))
return model
test = configurable_model(64, 3, 16, 3)
model = test.build_model()
I have tried converting to the functional API, but run into the problem of 'naming' the layers and keeping track of them in the for loops. I tried Concatenate instead of concatenate. I tried model.get_layer('layername').output and model.get_layer('layername').output() in the concatenate statement, etc... nothing is working. The code above gives the error: ValueError: A Concatenate layer should be called on a list of at least 2 inputs.

I was able to get the functional version working by storing the downblocks in a dictionary that I reference later during concatenation. See below:
class configurable_model():
def __init__(self, csize, channels, start_neurons, depth):
self.csize = csize
self.channels = channels
self.start_neurons = start_neurons
self.depth = depth
def _convblock(self, factor, name=None):
block = Sequential(name=name)
block.add(Conv2D(self.start_neurons * factor, (3, 3), padding="same"))
block.add(BatchNormalization())
block.add(ReLU())
block.add(Conv2D(self.start_neurons * factor, (3, 3), padding="same"))
block.add(BatchNormalization())
block.add(ReLU())
return block
def build_model(self):
input_layer = Input((self.csize, self.csize, self.channels), name='input')
x = input_layer
factor = 1
downblocks = {}
for idx in range(self.depth):
x = self._convblock(factor, f'downblock{idx}')(x)
downblocks[f'downblock{idx}'] = x
x = MaxPooling2D((2, 2), name=f'maxpool{idx}')(x)
factor *= 2
x = self._convblock(factor, 'Middle')(x)
for idx in reversed(range(self.depth)):
factor //= 2
x = Conv2DTranspose(self.start_neurons * factor, (3, 3),
strides=(2, 2), padding="same",
name=f'upsample{idx}')(x)
cat = concatenate([x, downblocks[f'downblock{idx}']])
x = self._convblock(factor, f'upblock{idx}')(cat)
output_layer = Conv2D(1, (1, 1), padding="same",
activation="sigmoid", name='output')(x)
return Model(input_layer, output_layer)

Keras: model.fit() getting error for multiple inputs in siamese_model

I am new to Keras and the Siamese network architecture. I have developed a Siamese network with three inputs and one output as follows.
def get_siamese_model(input_shape):
# Define the tensors for the three input phrases
anchor = Input(input_shape, name='anchor')
positive = Input(input_shape, name='positive')
negative = Input(input_shape, name='negative')
# Convolutional Neural Network
model = Sequential()
model.add(Conv2D(64, kernel_size=(2, 2), activation='relu', input_shape=input_shape, padding='same'))
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(16, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(8, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(4, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(2, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(1, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2,1)))
model.add(Flatten())
# Generate the encodings (feature vectors) for the three phrases
anchor_out = model(anchor)
positive_out = model(positive)
negative_out = model(negative)
# Add a customized layer to combine individual output
concat = Lambda(lambda tensors:K.concatenate((tensors[0],tensors[1],tensors[2]),0))
output = concat([anchor_out, positive_out, negative_out])
# Connect the inputs with the outputs
siamese_net = Model(inputs=[anchor,positive,negative],outputs=output)
#plot the model
plot_model(siamese_net, to_file='siamese_net.png',show_shapes=True, show_layer_names=True)
#Error optimization
siamese_net.compile(optimizer=Adam(),
loss=triplet_loss)
# return the model
return siamese_net
while using model.fit() I have written following code:
model = get_siamese_model(input_shape)
X = {
'anchor' : anchor,
'positive' : positive,
'negative' : negative
}
model.fit(np.asarray(X), Y)
I am getting following error message:
ValueError: Error when checking model input:
The list of Numpy arrays that you are passing to your model is not the size the model expected.
Expected to see 3 array(s), but instead got the following list of 1 arrays: [array({'anchor': array([[[[ 4.49218750e-02]...
Any help is appreciated. Thank you in advance.

The following code works for me. Because your names are (anchor, positive, negative), you can use those directly as the keys to your dictionary when passing input. Also, you should make use of the concatenate layer in Keras instead of defining a Lambda. Note that I changed the loss for purposes of this example.
from keras.layers import Input, Conv2D, MaxPooling2D, Flatten, concatenate
from keras.models import Model, Sequential
from keras.optimizers import Adam
from keras.losses import mean_squared_error
import numpy as np
def get_siamese_model(input_shape):
# Define the tensors for the three input phrases
anchor = Input(input_shape, name='anchor')
positive = Input(input_shape, name='positive')
negative = Input(input_shape, name='negative')
# Convolutional Neural Network
model = Sequential()
model.add(Conv2D(64, kernel_size=(2, 2), activation='relu', input_shape=input_shape, padding='same'))
model.add(Conv2D(32, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(16, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(8, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(4, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(2, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(Conv2D(1, kernel_size=(2, 2), activation='relu', padding='same'))
model.add(MaxPooling2D(pool_size=(2,1)))
model.add(Flatten())
# Generate the encodings (feature vectors) for the three phrases
anchor_out = model(anchor)
positive_out = model(positive)
negative_out = model(negative)
# Add a concatenate layer
output = concatenate([anchor_out, positive_out, negative_out])
# Connect the inputs with the outputs
siamese_net = Model(inputs=[anchor,positive,negative],outputs=output)
# Error optimization
siamese_net.compile(optimizer=Adam(), loss=mean_squared_error)
# Summarize model
siamese_net.summary()
# Return the model
return siamese_net
input_shape = (100, 100, 1)
model = get_siamese_model(input_shape)
X = {'anchor': np.ones((5, 100, 100, 1)), # define input as dictionary
'positive': np.ones((5, 100, 100, 1)),
'negative': np.ones((5, 100, 100, 1))}
Y = np.ones((5, 15000))
model.fit(X, Y) # use a dictionary
model.fit([i for i in X.values()], Y) # use a list

how can I wrap function using lambda layer?

I want to do some processes on one of my layer in autoencoder and then send it to next layer but I can not use the predefined functions in Keras like add ,... and I think I should use lambda with function. the output of my autoencoder is a tensor with shape (1,28,28,1)named encoder and I have an input tensor with shape (1,4,4,1) named wtm. now, I want to consider 7x7 blocks in encoder and I add the middle value of each 7x7 block with one value of wtm respectively ( each block of encoder with one value of wtm) I write two functions to do this but it produced this error:
TypeError: 'Tensor' object does not support item assignment
I am a beginner in python and Keras and I search for the reason but unfortunately, I could not understand why it happened and what should I do? please guide me how do I write my lambda layer? I attached the code here. I can do some simple thing like this with lambda
add_const = Kr.layers.Lambda(lambda x: x[0] + x[1])
encoded_merged = add_const([encoded,wtm])
but if wtm has the different shape with encoded or sompe complicated things on layer, I do not know what should I do?
from keras.layers import Input, Concatenate, GaussianNoise,Dropout,BatchNormalization,MaxPool2D,AveragePooling2D
from keras.layers import Conv2D, AtrousConv2D
from keras.models import Model
from keras.datasets import mnist
from keras.callbacks import TensorBoard
from keras import backend as K
from keras import layers
import matplotlib.pyplot as plt
import tensorflow as tf
import keras as Kr
from keras.optimizers import SGD,RMSprop,Adam
from keras.callbacks import ReduceLROnPlateau
from keras.callbacks import EarlyStopping
from keras.callbacks import ModelCheckpoint
import numpy as np
import pylab as pl
import matplotlib.cm as cm
import keract
from matplotlib import pyplot
from keras import optimizers
from keras import regularizers
from tensorflow.python.keras.layers import Lambda;
#-----------------building w train---------------------------------------------
def grid_w(args):
Enc, W = args
# Ex,Ey,Ez=Enc.shape
# Wx,Wy,Wz=W.shape
Enc=tf.reshape(Enc,[28,28])
W=tf.reshape(W,[4,4])
Enc[3::7, 3::7] += W
Enc=tf.reshape(Enc,[1,28,28,1])
W=tf.reshape(W,[1,4,4,1])
# Enc[:, 3::7, 3::7]=K.sum(W,axis=1)
return Enc
def grid_w_output_shape(shapes):
shape1, shape2 = shapes
return (shape1[0], 1)
wt_random=np.random.randint(2, size=(49999,4,4))
w_expand=wt_random.astype(np.float32)
wv_random=np.random.randint(2, size=(9999,4,4))
wv_expand=wv_random.astype(np.float32)
x,y,z=w_expand.shape
w_expand=w_expand.reshape((x,y,z,1))
x,y,z=wv_expand.shape
wv_expand=wv_expand.reshape((x,y,z,1))
#-----------------building w test---------------------------------------------
w_test = np.random.randint(2,size=(1,4,4))
w_test=w_test.astype(np.float32)
w_test=w_test.reshape((1,4,4,1))
#-----------------------encoder------------------------------------------------
#------------------------------------------------------------------------------
wtm=Input((4,4,1))
image = Input((28, 28, 1))
conv1 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl1e')(image)
conv2 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl2e')(conv1)
conv3 = Conv2D(64, (5, 5), activation='relu', padding='same', name='convl3e')(conv2)
#conv3 = Conv2D(8, (3, 3), activation='relu', padding='same', name='convl3e', kernel_initializer='Orthogonal',bias_initializer='glorot_uniform')(conv2)
BN=BatchNormalization()(conv3)
#DrO1=Dropout(0.25,name='Dro1')(BN)
encoded = Conv2D(1, (5, 5), activation='relu', padding='same',name='encoded_I')(BN)
#-----------------------adding w---------------------------------------
encoded_merged=Kr.layers.Lambda(grid_w, output_shape=grid_w_output_shape)([encoded, wtm])
#-----------------------decoder------------------------------------------------
#------------------------------------------------------------------------------
deconv1 = Conv2D(64, (5, 5), activation='elu', padding='same', name='convl1d')(encoded_merged)
deconv2 = Conv2D(64, (5, 5), activation='elu', padding='same', name='convl2d')(deconv1)
deconv3 = Conv2D(64, (5, 5), activation='elu',padding='same', name='convl3d')(deconv2)
deconv4 = Conv2D(64, (5, 5), activation='elu',padding='same', name='convl4d')(deconv3)
BNd=BatchNormalization()(deconv3)
decoded = Conv2D(1, (5, 5), activation='sigmoid', padding='same', name='decoder_output')(BNd)
model=Model(inputs=[image,wtm],outputs=decoded)
decoded_noise = GaussianNoise(0.5)(decoded)
#----------------------w extraction------------------------------------
convw1 = Conv2D(64, (3,3), activation='relu', name='conl1w')(decoded_noise)
convw2 = Conv2D(64, (3, 3), activation='relu', name='convl2w')(convw1)
Avw1=AveragePooling2D(pool_size=(2,2))
convw3 = Conv2D(64, (3, 3), activation='relu', padding='same', name='conl3w')(convw2)
convw4 = Conv2D(64, (3, 3), activation='relu', padding='same', name='conl4w')(convw3)
Avw2=AveragePooling2D(pool_size=(2,2))
convw5 = Conv2D(64, (3, 3), activation='relu', name='conl5w')(convw4)
convw6 = Conv2D(64, (3, 3), activation='relu', padding='same', name='conl6w')(convw5)
BNed=BatchNormalization()(convw6)
#DrO3=Dropout(0.25, name='DrO3')(BNed)
pred_w = Conv2D(1, (1, 1), activation='sigmoid', padding='same', name='reconstructed_W')(BNed)
watermark_extraction=Model(inputs=[image,wtm],outputs=[decoded,pred_w])
watermark_extraction.summary()