I know that you can reuse Keras layers. For eg I declare two layers for a decoder network:
decoder_layer_1 = Dense(intermediate_dim,activation='relu',name='decoder_layer_1')
decoder_layer_2 = Dense(intermediate_dim,activation='relu',name='decoder_layer_2')
Use in first model:
decoded = decoder_layer_1(z)
decoded = decoder_layer_2(decoded)
Use in second model:
_decoded = decoder_layer_1(decoder_input)
_decoded = decoder_layer_2(_decoded)
The above method is ok if I need to reuse only a couple of layers, cumbersome if I want to reuse a large number of layers (for eg. a decoder network with 10 layers). Is there a more efficient means to do it other than explicitly declaring each layer. Is there a means to implement it as shown below:
decoder_layers = group_of_layers()
Reuse in the first model:
decoded = group_of_layers(z)
Reuse in the second model:
_decoded = group_of_layers(decoder_input)
I struggled with this problem too. What works for me is to wrap shared parts in a model, with its own input definition:
def group_of_layers(intermediate_dim):
shared_model_input = keras.layers.Input(shape=...)
shared_internal_layer = keras.layers.Dense(intermediate_dim, activation='relu', name='shared_internal_layer')(shared_model_input)
shared_model_output = keras.layers.Dense(intermediate_dim, activation='relu', name='shared_model_output')(shared_internal_layer)
return keras.models.Model(shared_model_input, shared_model_output)
In Functional API, you can use the shared model in the same way a single layer as long as the model's input layer matches shape of layers you apply to it:
group = group_of_layers(intermediate_dim)
result1 = group(previous_layer)
result2 = group(different_previous_layer)
The weights are going to be shared then.
This is nicely described in the documentation, see Shared vision model.
Related
I have a Keras-model (let's call it full model), which was already trained and now I would like to create a new submodel using layers m to n of the full model.
E.g. full model has 10 layers and my submodel shall comprise layers 3 to 8
For the case that m=0, the task is trivial as one can use: (assume we want to go to layer 5)
full_model = ... # anything we load from a h5-file
submodel=tf.keras.Model(inputs=full_model.inputs, outputs=full_model.layers[5].output)
# =>
submodel.summary()
tf.keras.utils.plot_model(submodel, to_file = ...)
So, we can use the submodel, get its summary and also get the png-plot of the submodel-architecture.
The concrete problem now is that I don't know how to make this if we want to take the last layers of the model for example. I always get a GraphDisconnected error than.
The only way to get around this, that I found, was to manually loop over the layers (as the function below, "create_submodel", is doing it) - but in my case, I cannot use this because the model is quite complex and the layers are not simply put after each other but they are nested and so on i.e. in the architecture-plot, I do not have a straight series of layers but many different branches in the "tree" of layers.
So: Is there a way to create a submodel (from layer "m" to layer "n") of a "full" model without simple, naive looping through the layers (as demonstrated in the function below)
Thanks very much!
def create_submodel(full_model, start_layer_number=None, end_layer_number=None):
layers = tf.keras.layers
if start_layer_number is None:
start_layer_number = 0
if end_layer_number is None:
end_layer_number = len(full_model.layers)
inp_shape = full_model.layers[start_layer_number].input.shape[1:]
inp = layers.Input(shape=(inp_shape))
x = inp
for i in range(start_layer_number, end_layer_number):
print(i, full_model.layers[i].name)
x = full_model.layers[i](x)
out = x
sub_model = tf.keras.Model(inputs=inp, outputs=out)
sub_model.summary()
return sub_model
I am trying to get a fine-tuned MobileNetV3Small running in JavaScript. Unfortunately tfjs does not support the Rescaling layer yet. That shouldn't matter too much though, since I can rescale the image beforehand. Now I would like to get rid of the Rescaling layer in the model, but am failing to do so.
tf.keras' model.layers.pop seems to be not working (see e.g. here).
So I tried to disassemble the layers, like here, skip the rescaling layer and assemble them to a model again. Problem is, that MobileNetV3 has some skip-layers which are concatenated by Add layers with several Inputs, so I end up with:
ValueError: A merge layer should be called on a list of inputs.
Any ideas on how to solve it? Every help would be greatly appreciated!
Here's the code I used for (dis)assembling:
#Creating the Model with the undesired layer
base = tf.keras.applications.MobileNetV3Small(input_shape=(224,224,3), include_top=False, weights='imagenet', minimalistic=True)
model=keras.Model(inputs=base.input, outputs=predictions)
# Dissasemble
layers = [l for l in model.layers]
new_in = keras.Input((224,224,3))
x = new_in
#Assemble again, but Skip Layer-No1, the Rescaling Layer
for idx,l in enumerate(layers[2:]):
l.trainable = False
x = l(x)
results = tf.keras.Model(inputs=new_in, outputs=x)
# Results in mentioned Error
I am training an autoencoder constructed using the Sequential API in Keras. I'd like to create separate models that implement the encoding and decoding functions. I know from examples how to do this with the functional API, but I can't find an example of how it's done with the Sequential API. The following sample code is my starting point:
input_dim = 2904
encoding_dim = 4
hidden_dim = 128
# instantiate model
autoencoder = Sequential()
# 1st hidden layer
autoencoder.add(Dense(hidden_dim, input_dim=input_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
autoencoder.add(Dropout(0.5))
# encoding layer
autoencoder.add(Dense(encoding_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
# autoencoder.add(Dropout(0.5))
# 2nd hidden layer
autoencoder.add(Dense(hidden_dim, use_bias=False))
autoencoder.add(BatchNormalization())
autoencoder.add(Activation('elu'))
autoencoder.add(Dropout(0.5))
# output layer
autoencoder.add(Dense(input_dim))
I realize I can select individual layers using autoencoder.layer[i], but I don't know how to associate a new model with a range of such layers. I naively tried the following:
encoder = Sequential()
for i in range(0,7):
encoder.add(autoencoder.layers[i])
decoder = Sequential()
for i in range(7,12):
decoder.add(autoencoder.layers[i])
print(encoder.summary())
print(decoder.summary())
which seemingly worked for the encoder part (a valid summary was shown), but the decoder part generated an error:
This model has not yet been built. Build the model first by calling build() or calling fit() with some data. Or specify input_shape or batch_input_shape in the first layer for automatic build.
Since the input shape for a middle layer (i.e. here I am referring to autoencoder.layers[7]) is not explicitly set, when you add it to another model as the first layer, that model would not be built automatically (i.e. building process involves constructing weight tensor for the layers in the model). Therefore, you need to call build method explicitly and set the input shape:
decoder.build(input_shape=(None, encoding_dim)) # note that batch axis must be included
As a side note, there is no need to call print on model.summary(), since it would print the result by itself.
Another way which also works.
input_img = Input(shape=(encoding_dim,))
previous_layer = input_img
for i in range(bottleneck_layer,len(autoencoder.layers)): # bottleneck_layer = index of bottleneck_layer + 1!
next_layer = autoencoder.layers[i](previous_layer)
previous_layer = next_layer
decoder = Model(input_img, next_layer)
I have some cnn, and I want to fetch the value of some intermediate layer corresponding to a some key from the state dict.
How could this be done?
Thanks.
I think you need to create a new class that redefines the forward pass through a given model. However, most probably you will need to create the code regarding the architecture of your model. You can find here an example:
class extract_layers():
def __init__(self, model, target_layer):
self.model = model
self.target_layer = target_layer
def __call__(self, x):
return self.forward(x)
def forward(self, x):
module = self.model._modules[self.target_layer]
# get output of the desired layer
features = module(x)
# get output of the whole model
x = self.model(x)
return x, features
model = models.vgg19(pretrained=True)
target_layer = 'features'
extractor = extract_layers(model, target_layer)
image = Variable(torch.randn(1, 3, 244, 244))
x, features = extractor(image)
In this case, I am using the pre-defined vgg19 network given in the pytorch models zoo. The network has the layers structured in two modules the features for the convolutional part and the classifier for the fully-connected part. In this case, since features wraps all the convolutional layers of the network it is straightforward. If your architecture has several layers with different names, you will need to store their output using something similar to this:
for name, module in self.model._modules.items():
x = module(x) # forward the module individually
if name in self.target_layer:
features = x # store the output of the desired layer
Also, you should keep in mind that you need to reshape the output of the layer that connects the convolutional part to the fully-connected one. It should be easy to do if you know the name of that layer.
The features maps can be obtained using:
from keras import backend as K
# with a Sequential model
get_3rd_layer_output = K.function([model.layers[0].input],
[model.layers[
[model.layers[
[model.layers[3].output])
layer_output = get_3rd_layer_output([X])[
layer_output = get_3rd_layer_output([X])[
layer_output = get_3rd_layer_output([X])[0]
This is good for visualisation of the data. But, I also intend to modify the output for each layer and then fed this output back to the network. Can anyone suggest me how I can do the same?
Thanks
I'm restoring this answer with edits to reflect additional information.
Assuming you have a model similar to this:
model = Sequential()
model.add(Dense(1000, input_dim=1000))
model.add(Dense(1000))
And you want to run a custom modification on the output of the first layer before passing it to the second layer you can use the lambda layer as so:
f = K.function(\* some function *\)
model = Sequential()
model.add(Dense(1000, input_dim=1000))
model.add(Lambda(lambda x: f(x))
model.add(Dense(1000))
If you just want to do this once you can do something like this:
modified_layer_output = your_old_function([X]) * some_modification
get_final_layer_output = K.function([model.layers[3].input],
[model.layers[-1].output])
result = get_final_layer_output(modified_layer_output)
You could also create a new model to learn on your modified layer output.
Edit:
You could do your write your own keras layer to do whatever you want with the input and pass it to the next layer like shown here (https://keras.io/layers/writing-your-own-keras-layers/).