I am training an autoencoder constructed using the Sequential API in Keras. I'd like to create separate models that implement the encoding and decoding functions. I know from examples how to do this with the functional API, but I can't find an example of how it's done with the Sequential API. The following sample code is my starting point:
input_dim = 2904
encoding_dim = 4
hidden_dim = 128
# instantiate model
autoencoder = Sequential()
# 1st hidden layer
autoencoder.add(Dense(hidden_dim, input_dim=input_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
autoencoder.add(Dropout(0.5))
# encoding layer
autoencoder.add(Dense(encoding_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
# autoencoder.add(Dropout(0.5))
# 2nd hidden layer
autoencoder.add(Dense(hidden_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
autoencoder.add(Dropout(0.5))
# output layer
autoencoder.add(Dense(input_dim))
I realize I can select individual layers using autoencoder.layer[i], but I don't know how to associate a new model with a range of such layers. I naively tried the following:
encoder = Sequential()
for i in range(0,7):
encoder.add(autoencoder.layers[i])
decoder = Sequential()
for i in range(7,12):
decoder.add(autoencoder.layers[i])
print(encoder.summary())
print(decoder.summary())
which seemingly worked for the encoder part (a valid summary was shown), but the decoder part generated an error:
This model has not yet been built. Build the model first by calling build() or calling fit() with some data. Or specify input_shape or batch_input_shape in the first layer for automatic build.
Since the input shape for a middle layer (i.e. here I am referring to autoencoder.layers[7]) is not explicitly set, when you add it to another model as the first layer, that model would not be built automatically (i.e. building process involves constructing weight tensor for the layers in the model). Therefore, you need to call build method explicitly and set the input shape:
decoder.build(input_shape=(None, encoding_dim)) # note that batch axis must be included
As a side note, there is no need to call print on model.summary(), since it would print the result by itself.
Another way which also works.
input_img = Input(shape=(encoding_dim,))
previous_layer = input_img
for i in range(bottleneck_layer,len(autoencoder.layers)): # bottleneck_layer = index of bottleneck_layer + 1!
next_layer = autoencoder.layers[i](previous_layer)
previous_layer = next_layer
decoder = Model(input_img, next_layer)
Related
I am developing an autoencoder for clustering certain groups of images.
input_images->...->bottleneck->...->output_images
I have calibrated the autoencoder to my satisfaction and saved the model; everything has been developed using keras.tensorflow on python3.
The next step is to apply the autoencoder to a ton of images and cluster them according to cosine distance in the bottleneck layer. Oops, I just realized that I don't know the syntax in keras.tf for running the model on a batch up to a specific layer rather than to the output layer. Thus the question:
How do I run something like Model.predict_on_batch or Model.predict_generator up to the certain "bottleneck" layer and retrieve the values on that layer rather than the values on the output layer?
You need to define a new model (if you didn't define the encoder and decoder as separate models initially, which is usually the easiest option).
If your model was defined without reusing layers, it's just:
inputs = model.input
outputs= model.get_layer('bottleneck').output
encoder = Model(inputs, outputs)
Use the encoder model as any other model.
The full code would be like this,
# ENCODER
encoding_dim = 37310
input_layer = Input(shape=(encoding_dim,))
encoder = Dense(500, activation='tanh')(input_layer)
encoder = Dense(100, activation='tanh')(encoder)
encoder = Dense(50, activation='tanh', name='bottleneck_layer')(encoder)
decoder = Dense(100, activation='tanh')(encoder)
decoder = Dense(500, activation='tanh')(decoder)
decoder = Dense(37310, activation='sigmoid')(decoder)
# full model
model_full = models.Model(input_layer, decoder)
model_full.compile(optimizer='adam', loss='mse')
model_full.fit(x, y, epochs=20, batch_size=16)
# bottleneck model
bottleneck_output = model_full.get_layer('bottleneck_layer').output
model_bottleneck = models.Model(inputs = model_full.input, outputs = bottleneck_output)
bottleneck_predictions = model_bottleneck.predict(X_test)
I am trying to get a fine-tuned MobileNetV3Small running in JavaScript. Unfortunately tfjs does not support the Rescaling layer yet. That shouldn't matter too much though, since I can rescale the image beforehand. Now I would like to get rid of the Rescaling layer in the model, but am failing to do so.
tf.keras' model.layers.pop seems to be not working (see e.g. here).
So I tried to disassemble the layers, like here, skip the rescaling layer and assemble them to a model again. Problem is, that MobileNetV3 has some skip-layers which are concatenated by Add layers with several Inputs, so I end up with:
ValueError: A merge layer should be called on a list of inputs.
Any ideas on how to solve it? Every help would be greatly appreciated!
Here's the code I used for (dis)assembling:
#Creating the Model with the undesired layer
base = tf.keras.applications.MobileNetV3Small(input_shape=(224,224,3), include_top=False, weights='imagenet', minimalistic=True)
model=keras.Model(inputs=base.input, outputs=predictions)
# Dissasemble
layers = [l for l in model.layers]
new_in = keras.Input((224,224,3))
x = new_in
#Assemble again, but Skip Layer-No1, the Rescaling Layer
for idx,l in enumerate(layers[2:]):
l.trainable = False
x = l(x)
results = tf.keras.Model(inputs=new_in, outputs=x)
# Results in mentioned Error
Is it possible to access pre-activation tensors in a Keras Model? For example, given this model:
import tensorflow as tf
image_ = tf.keras.Input(shape=[224, 224, 3], batch_size=1)
vgg19 = tf.keras.applications.VGG19(include_top=False, weights='imagenet', input_tensor=image_, input_shape=image_.shape[1:], pooling=None)
the usual way to access layers is:
intermediate_layer_model = tf.keras.models.Model(inputs=image_, outputs=[vgg19.get_layer('block1_conv2').output])
intermediate_layer_model.summary()
This gives the ReLU outputs for a layer, while I would like the ReLU inputs. I tried doing this:
graph = tf.function(vgg19, [tf.TensorSpec.from_tensor(image_)]).get_concrete_function().graph
outputs = [graph.get_tensor_by_name(tname) for tname in [
'vgg19/block4_conv3/BiasAdd:0',
'vgg19/block4_conv4/BiasAdd:0',
'vgg19/block5_conv1/BiasAdd:0'
]]
intermediate_layer_model = tf.keras.models.Model(inputs=image_, outputs=outputs)
intermediate_layer_model.summary()
but I get the error
ValueError: Unknown graph. Aborting.
The only workaround I've found is to edit the model file to manually expose the intermediates, turning every layer like this:
x = layers.Conv2D(256, (3, 3), activation="relu", padding="same", name="block3_conv1")(x)
into 2 layers where the 1st one can be accessed before activations:
x = layers.Conv2D(256, (3, 3), activation=None, padding="same", name="block3_conv1")(x)
x = layers.ReLU(name="block3_conv1_relu")(x)
Is there a way to acces pre-activation tensors in a Model without essentially editing Tensorflow 2 source code, or reverting to Tensorflow 1 which had full flexibility accessing intermediates?
There is a way to access pre-activation layers for pretrained Keras models using TF version 2.7.0. Here's how to access two intermediate pre-activation outputs from VGG19 in a single forward pass.
Initialize VGG19 model. We can omit top layers to avoid loading unnecessary parameters into memory.
vgg19 = tf.keras.applications.VGG19(
include_top=False,
weights="imagenet"
)
This is the important part: Create a deepcopy of the intermediate layer form which you like to have the features, change the activation of the conv layers to linear (i.e. no activation), rename the layer (otherwise two layers in the model will have the same name which will raise errors) and finally pass the output of the previous through the copied conv layer.
# for more intermediate features wrap a loop around it to avoid copy paste
b5c4_layer = deepcopy(vgg19.get_layer("block5_conv4"))
b5c4_layer.activation = tf.keras.activations.linear
b5c4_layer._name = b5c4_layer.name + str("_preact")
b5c4_preact_output = b5c4_layer(vgg19.get_layer("block5_conv3").output)
b2c2_layer = deepcopy(vgg19.get_layer("block2_conv2"))
b2c2_layer.activation = tf.keras.activations.linear
b2c2_layer._name = b2c2_layer.name + str("_preact")
b2c2_preact_output = b2c2_layer(vgg19.get_layer("block2_conv1").output)
Finally, get the outputs and check if they equal post-activation outputs when we apply ReLU-activation.
vgg19_features = Model(vgg19.input, [b2c2_preact_output, b5c4_preact_output])
vgg19_features_control = Model(vgg19.input, [vgg19.get_layer("block2_conv2").output, vgg19.get_layer("block5_conv4").output])
b2c2_preact, b5c4_preact = vgg19_features(tf.keras.applications.vgg19.preprocess_input(img))
b2c2, b5c4 = vgg19_features_control(tf.keras.applications.vgg19.preprocess_input(img))
print(np.allclose(tf.keras.activations.relu(b2c2_preact).numpy(),b2c2.numpy()))
print(np.allclose(tf.keras.activations.relu(b5c4_preact).numpy(),b5c4.numpy()))
True
True
Here's a visualization similar to Fig. 6 of Wang et al. to see the effect in the feature space.
Input image
To get output of each layer. You have to define a keras function and evaluate it for each layer.
Please refer the code as shown below
from tensorflow.keras import backend as K
inp = model.input # input
outputs = [layer.output for layer in model.layers] # all layer outputs
functors = [K.function([inp], [out]) for out in outputs] # evaluation functions
For more details on this please refer SO Answer.
I am trying to output the previous to last Dense layer in a keras model. I first load the model architecture and the weights:
base_model = applications.ResNet50(weights = None,
include_top = False,
input_shape = (image_size[0], image_size[1], nb_channels))
top_model = Sequential()
top_model.add(Flatten(input_shape=base_model.output_shape[1:]))
top_model.add(Dense(1024, init = 'glorot_uniform', activation='relu', name = 'last_layer_1024'))
top_model.add(Dropout(0.5))
top_model.add(Dense(nb_classes, activation = 'softmax', name = 'softmax_layer'))
top_model_tensor = top_model(base_model.output)
model = Model(inputs = base_model.input, outputs = top_model_tensor)
model.load_weights(weights_path)
Then I remove the last Dense layer by doing this:
model.layers[-1].pop()
#model.outputs = [model.layers[-1].layers[-1].output]
#model.layers[-1].layers[-1].outbound_nodes = []
If I uncomment the commented lines, I get this error: InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'flatten_1_input' with dtype float. If I keep them commented, the last dense layer is NOT effectively removed (by that I mean that when I call predict on model, I still get the output of the last dense layer). How can I solve this issue?
Also, if there is a different method to get the model to output the previous to last dense layer, I can take that as an answer too (instead of trying to fix this way of doing it).
Another solution that does not work is to just cut the long model after you load weights by simply doing this:
short_top_model = Model(top_model.input, top_model.get_layer('last_layer_1024').output)
You get the following error:
RuntimeError: Graph disconnected: cannot obtain value for tensor Tensor("flatten_1_input:0", shape=(?, 1, 1, 2048), dtype=float32, device=/device:GPU:2) at layer "flatten_1_input". The following previous layers were accessed without issue: []
Trying to cut models, change their inputs/outputs etc. sounds not what keras expects from users.
You should just create another model that follows the same path but ends earlier:
#do this "before" creating "top_model_tensor".
short_top = Model(
top_model.input,
top_model.get_layer('last_layer_1024').output
)
top_model_out = top_model(base_model.output)
short_top_out = short_top(base_model.output)
model = Model(base_model.input,top_model_out)
short_model = Model(base_model.input,short_top_out)
Choose which one to use depending on the expected results. Training one updates the other.
A shorter version of the above answer.
#again create connection between two model
feature_vec_model = Model(
top_model.input,
top_model.get_layer('last_layer_1024').output
)
feature_vec_model_output = feature_vec_model(base_model.output)
#Connection created
# Define final connected model & load pretrained weights
complete_feature_vec_model = Model(base_model.input,feature_vec_model_output)
complete_feature_vec_model.load_weights("path_to_model")
The features maps can be obtained using:
from keras import backend as K
# with a Sequential model
get_3rd_layer_output = K.function([model.layers[0].input],
[model.layers[
[model.layers[
[model.layers[3].output])
layer_output = get_3rd_layer_output([X])[
layer_output = get_3rd_layer_output([X])[
layer_output = get_3rd_layer_output([X])[0]
This is good for visualisation of the data. But, I also intend to modify the output for each layer and then fed this output back to the network. Can anyone suggest me how I can do the same?
Thanks
I'm restoring this answer with edits to reflect additional information.
Assuming you have a model similar to this:
model = Sequential()
model.add(Dense(1000, input_dim=1000))
model.add(Dense(1000))
And you want to run a custom modification on the output of the first layer before passing it to the second layer you can use the lambda layer as so:
f = K.function(\* some function *\)
model = Sequential()
model.add(Dense(1000, input_dim=1000))
model.add(Lambda(lambda x: f(x))
model.add(Dense(1000))
If you just want to do this once you can do something like this:
modified_layer_output = your_old_function([X]) * some_modification
get_final_layer_output = K.function([model.layers[3].input],
[model.layers[-1].output])
result = get_final_layer_output(modified_layer_output)
You could also create a new model to learn on your modified layer output.
Edit:
You could do your write your own keras layer to do whatever you want with the input and pass it to the next layer like shown here (https://keras.io/layers/writing-your-own-keras-layers/).