I'm trying to prepend a preprocessing layer to a pre-trained network. This is the code I'm working on:
orig_model = applications.vgg16.VGG16(include_top=True, weights=None, input_tensor=None, input_shape=None, pooling=None, classes=1000)
orig_model.load_weights(weights_path)
preproc_layer = Lambda(preprocess, input_shape=(3,224,224), output_shape=(3,224,224))
model = Sequential()
model.add(preproc_layer)
all_layers = orig_model.layers
for l in all_layers:
config = l.get_config()
copy = layers.deserialize({'class_name':l.__class__.__name__, 'config': config})
weights = l.get_weights()
copy.set_weights(weights)
model.add(copy)
Where preprocess is:
preprocess(x):
x = x[::-1, ...]
x = K.bias_add(x, vgg_mean, data_format='channels_first')
It works for the first InputLayer but throws me an error at copy.set_weights(weights) for the second (Conv2D) layer:
You called `set_weights(weights)` on layer "block1_conv1" with a weight list of length 2, but the layer was expecting 0 weights.
I found something similar on Google: https://github.com/keras-team/keras/issues/4812. Here they suggest setting trainable = True for the layer, but this doesn't work in my case.
Do you have any suggestions? Keras version is 2.1.5, Tensorflow 1.6.0
Related
I have a model that I load this way:
def YOLOv3_pretrained(n_classes=12, n_bbox=3):
yolo3 = tf.keras.models.load_model("yolov3/yolo3.h5")
yolo3.trainable = False
l3 = yolo3.get_layer('leaky_re_lu_71').output
l3_flat = tf.keras.layers.Flatten()(l3)
out3 = tf.keras.layers.Dense(100*(4+1+n_classes))(l3_flat)
out3 = Reshape((100, (4+1+n_classes)), input_shape=(12,))(out3)
yolo3 = Model(inputs=yolo3.input, outputs=[out3])
return yolo3
I want to add a Dense at the end of it but since it takes an input with shape (None, 416,416,3) it doesn't let me do it and it returns an error:
ValueError: The last dimension of the inputs to a Dense layer should be defined. Found None. Full input shape received: (None, None)
I also tried this way with a Sequential (I want to use just the last output of yolo):
def YOLOv3_Dense(n_classes=12):
yolo3 = tf.keras.models.load_model("yolov3/yolo3.h5")
model = Sequential()
model.add(yolo3)
model.add(Flatten())
model.add(Dense(100*(4+1+n_classes)))
model.add(Reshape((100, (4+1+n_classes)), input_shape=(413,413,3)))
return model
But it returns another error:
ValueError: All layers in a Sequential model should have a single output tensor. For multi-output layers, use the functional API.
Is there a way to add the final Dense layer?
The problem is that you are trying to reduce (flatten) an output with multiple None dimensions, which will not work if you want to use the output as input to another layer. You can try using a GlobalAveragePooling2D or GlobalMaxPooling2D instead:
import tensorflow as tf
yolo3 = tf.keras.models.load_model("yolo3.h5")
yolo3.trainable = False
l3 = yolo3.get_layer('leaky_re_lu_71').output
l3_flat = tf.keras.layers.GlobalMaxPooling2D()(l3)
out3 = tf.keras.layers.Dense(100*(4+1+12))(l3_flat)
out3 = tf.keras.layers.Reshape((100, (4+1+12)), input_shape=(12,))(out3)
yolo3 = tf.keras.Model(inputs=yolo3.input, outputs=[out3])
I am working a transfer learning problem. When I create a new model from just the Mobilenet, I set a dropout.
base_model = MobileNet(weights='imagenet', include_top=False, input_shape=(200,200,3), dropout=.15)
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(10, activation='softmax')(x)
I save models as I train using model_checkpoint_callback. As I train I find where overfitting is happening and adjust the amount of frozen layers and the learning rate. Can I also adjust dropout when I save a loaded model again?
I saw this answer but there are no actual dropout layers in Mobilenet, so this
for layer in model.layers:
if hasattr(layer, 'rate'):
print(layer.name)
layer.rate = 0.5
doesn't do anything.
In the past, you had to clone the model for the new dropout to take. I haven't tried it recently.
# This code allows you to change the dropout
# Load model from .json
model.load_weights(filenameToModelWeights) # Load weights
model.layers[-2].rate = 0.04 # layer[-2] is my dropout layer, rate is dropout attribute
model = keras.models.clone(model) # If I do not clone, the new rate is never used. Weights are re-init now.
model.load_weights(filenameToModelWeights) # Load weights
model.predict(x)
credit to
http://www.gergltd.com/home/2018/03/changing-dropout-on-the-fly-during-training-time-test-time-in-keras/
If the model doesn't have dropout layers to even begin with, as with Keras's pretrained mobilenet, you'll have to add them with methods. Here's one way you could do it.
For adding in a single layer
def insert_single_layer_in_keras(model, layer_name, new_layer):
layers = [l for l in model.layers]
x = layers[0].output
for i in range(1, len(layers)):
x = layers[i](x)
# add layer afterward
if layers[i].name == layer_name:
x = new_layer(x)
new_model = Model(inputs=layers[0].input, outputs=x)
return new_model
For systematically adding a layer
def insert_layers_in_model(model, layer_common_name, new_layer):
import re
layers = [l for l in model.layers]
x = layers[0].output
layer_config = new_layer.get_config()
base_name = layer_config['name']
layer_class = type(dropout_layer)
for i in range(1, len(layers)):
x = layers[i](x)
match = re.match(".+" + layer_common_name + "+", layers[i].name)
# add layer afterward
if match:
layer_config['name'] = base_name + "_" + str(i) # no duplicate names, could be done different
layer_copy = layer_class.from_config(layer_config)
x = layer_copy(x)
new_model = Model(inputs=layers[0].input, outputs=x)
return new_model
Run like this
import tensorflow as tf
from tensorflow.keras.applications.mobilenet import MobileNet
from tensorflow.keras.layers import Dropout
from tensorflow.keras.models import Model
base_model = MobileNet(weights='imagenet', include_top=False, input_shape=(192, 192, 3), dropout=.15)
dropout_layer = Dropout(0.5)
# add single layer after last dropout
mobile_net_with_dropout = insert_single_layer_in_model(base_model, "conv_pw_13_bn", dropout_layer)
# systematically add layers after any batchnorm layer
mobile_net_with_multi_dropout = insert_layers_in_model(base_model, "bn", dropout_layer)
By the way, you should absolutely experiment, but it's unlikely you want additional regularization on top of batchnorm for a small net like mobilenet.
I am making a CNN model to use for lane detection. But tensorflow 2 does not have tf.contrib therefore i cannot access the fully_connected layer.
How can I make my own Fully connected layer function?
This is my model so far:
conv2d = tf.nn.conv2d
batch_norm = tf.nn.batch_normalization
dropout = tf.nn.dropout
max_pool = tf.nn.max_pool2d
softmax = tf.nn.softmax
relu = tf.nn.relu
avg_pool = tf.nn.avg_pool2d
checkpoint = tf.train.Checkpoint
def network(x):
model = conv2d(x,filters=[1,5,5,1],strides=[1,2,2,1],padding='SAME')
model = relu(model)
model = batch_norm(model)
model = max_pool(model)
model = conv2d(model,filters=[1,4,4,1],strides=[1,2,2,1],padding='SAME')
model = relu(model)
model = batch_norm(model)
model = max_pool(model)
model = conv2d(model,filters=[1,3,3,1],strides=[1,2,2,1],padding='SAME')
model = relu(model)
model = batch_norm(model)
model = avg_pool(model)
model = dropout(model,0.3)
# i want to add the fully connect layer here then a softmax layer then another fully connected
I think what you might be looking for is the Dense layer in the keras module - https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense
I am trying to use the implementetion of DeepTriage which is a deep learning approach for bug triaging. This website includes dataset, source code and paper. I know that is a very specific area, but I'll try to make it simple.
In the source code they define their approach "DBRNN-A: Deep Bidirectional Recurrent Neural Network with Attention mechanism and with Long Short-Term Memory units (LSTM)" with this code part:
input = Input(shape=(max_sentence_len,), dtype='int32')
sequence_embed = Embedding(vocab_size, embed_size_word2vec, input_length=max_sentence_len)(input)
forwards_1 = LSTM(1024, return_sequences=True, dropout_U=0.2)(sequence_embed)
attention_1 = SoftAttentionConcat()(forwards_1)
after_dp_forward_5 = BatchNormalization()(attention_1)
backwards_1 = LSTM(1024, return_sequences=True, dropout_U=0.2, go_backwards=True)(sequence_embed)
attention_2 = SoftAttentionConcat()(backwards_1)
after_dp_backward_5 = BatchNormalization()(attention_2)
merged = merge([after_dp_forward_5, after_dp_backward_5], mode='concat', concat_axis=-1)
after_merge = Dense(1000, activation='relu')(merged)
after_dp = Dropout(0.4)(after_merge)
output = Dense(len(train_label), activation='softmax')(after_dp)
model = Model(input=input, output=output)
model.compile(loss='categorical_crossentropy', optimizer=Adam(lr=1e-4), metrics=['accuracy'])
SoftAttentionConcat implementation is from here. Rest of the functions are from keras. Also, in the paper they share the structure as:
In the first batch normalization line, it throws this error:
ValueError: Input 0 is incompatible with layer batch_normalization_1: expected ndim=3, found ndim=2
When I use max_sentence_len=50 and max_sentence_len=200 I look at the dimension until the error point, I see these shapes:
Input -> (None, 50)
Embedding -> (None, 50, 200)
LSTM -> (None, None, 1024)
SoftAttentionConcat -> (None, 2048)
So, is there anybody seeing the problem here?
I guess the problem is using TensorFlow code in a Keras structure or some version issues.
By using the question and the answers here, I implemented the attention mechanism in Keras as follows:
attention_1 = Dense(1, activation="tanh")(forwards_1)
attention_1 = Flatten()(attention_1) # squeeze (None,50,1)->(None,50)
attention_1 = Activation("softmax")(attention_1)
attention_1 = RepeatVector(num_rnn_unit)(attention_1)
attention_1 = Permute([2, 1])(attention_1)
attention_1 = multiply([forwards_1, attention_1])
attention_1 = Lambda(lambda xin: K.sum(xin, axis=1), output_shape=(num_rnn_unit,))(attention_1)
last_out_1 = Lambda(lambda xin: xin[:, -1, :])(forwards_1)
sent_representation_1 = concatenate([last_out_1, attention_1])
This works quite well. All the source code that I used for the implementation is available is in GitHub.
I am using VGG16 to finetune it on my dataset.
Here's the model:
def finetune(self, aux_input):
model = applications.VGG16(weights='imagenet', include_top=False)
# return model
drop_5 = Input(shape=(7, 7, 512))
flatten = Flatten()(drop_5)
# aux_input = Input(shape=(1,))
concat = Concatenate(axis=1)([flatten, aux_input])
fc1 = Dense(512, kernel_regularizer=regularizers.l2(self.weight_decay))(concat)
fc1 = Activation('relu')(fc1)
fc1 = BatchNormalization()(fc1)
fc1_drop = Dropout(0.5)(fc1)
fc2 = Dense(self.num_classes)(fc1_drop)
top_model_out = Activation('softmax')(fc2)
top_model = Model(inputs=drop_5, outputs=top_model_out)
output = top_model(model.output)
complete_model = Model(inputs=[model.input, aux_input], outputs=output)
return complete_model
I have two inputs to the model. In the above function, I'm using Concatenate for the flattened array and my aux_input.
I'm not sure if this would work with imagenet weights.
When I run this, I get an error:
ValueError: Graph disconnected: cannot obtain value for tensor
Tensor("aux_input:0", shape=(?, 1), dtype=float32) at layer
"aux_input". The following previous layers were accessed without
issue: ['input_2', 'flatten_1']
Not sure where am I going wrong.
If it matters, this is fit function:
model.fit(x={'input_1': x_train, 'aux_input': y_aux_train}, y=y_train, batch_size=batch_size,
epochs=maxepoches, validation_data=([x_test, y_aux_test], y_test),
callbacks=[reduce_lr, tensorboard], verbose=2)
But, I get an error before this fit function when I call model.summary().
The problem is that you are using aux_input in your top_model but you don't specify it as an input in your definition of top_model. Try replacing your definition of top_model and output with the following:
top_model = Model(inputs=[drop_5, aux_input], outputs=top_model_out)
output = top_model([model.output, aux_input])