So I am building a keras sequential model in which the last output layer is an Upsampling2D layer & I need to feed the input image to that output layer to do a simple operation and return the output, any ideas?
EDIT :
The model mentioned before is the generator of a GAN model in which I need to add the input image to the output of the generator before feeding it to the discriminator
1.You can define a backbone model using inputs of pre-trained model and the outputs of the last layer before the output layer of pre-trained model
2.Base on that backbone model, defined new model have that new skip connection and the output layer as same as pre-trained model
3.Set the weights of output layer in new model to equal to weights of output layer in pre-trained model, using: new_model.layers[-1].set_weights(pre_model.layers[-1].get_weights())
Here is one good article about Adding Layers to the middle of a pre-trained network whithout invalidating the weights
So for the future reference, I solved it by using lambda layers as follow :
# z is the input I needed to use later on with the generator output to perform a certain function
generated_image = self.generator(z)
generated_image_modified=tf.keras.layers.Concatenate()([generated_image,z])
# with x[...,variable_you_need_range] you can access the input we just concatenated in your train loop
lambd = tf.keras.layers.Lambda(lambda x: your_function(x[...,2:5],x[...,:2]))(generated_image_modified)
full_model = self.discriminator(lambd)
self.combined = Model(z,outputs = full_model)
Related
Is it possible to load the weights to the last layer in my new model from trained network by using set_weights and get_weights scheme ?
The point is, i saved the weight of each layer as a mat file (after training) to make some calculation in Matlab and i want just the modified weights of the last layer to be loaded to the last layer in my new model and other layers get the same weights as the trained model. It is a bit trickey, since the saved format is mat.
weights1 = lstm_model1.layers[0].get_weights()[0]
biases1 = lstm_model1.layers[0].get_weights()[1]
weights2 = lstm_model1.layers[2].get_weights()[0]
biases2 = lstm_model1.layers[2].get_weights()[1]
weights3 = lstm_model1.layers[4].get_weights()[0]
biases3 = lstm_model1.layers[4].get_weights()[1]
# Save the weights and biases for adaptation algorithm
savemat("weights1.mat", mdict={'weights1': weights1})
savemat("biases1.mat", mdict={'biases1': biases1})
savemat("weights2.mat", mdict={'weights2': weights2})
savemat("biases2.mat", mdict={'biases2': biases2})
savemat("weights3.mat", mdict={'weights3': weights3})
savemat("biases3.mat", mdict={'biases3': biases3})
How can i load just the old weights of other layers to the new model (without the last layer) and the modified weights of last layer to the last layer in the new one ?
If it was saved as a .h5 file format, this works. However, I’m not sure about .mat:
In simplicity, you just have to callget_weights on the desired layer, and similarly, set_weights on the corresponding layer of the other model:
last_layer_weights = old_model.layers[-1].get_weights()
new_model.layers[-1].set_weights(last_layer_weights)
For a more complete code sample, here you go:
# Create an arbitrary model with some weights, for example
model = Sequential(layers = [
Dense(70, input_shape = (100,)),
Dense(60),
Dense(50),
Dense(5)])
# Save the weights of the model
model.save_weights(“model.h5”)
# Later, load in the model (we only really need the layer in question)
old_model = Sequential(layers = [
Dense(70, input_shape = (100,)),
Dense(60),
Dense(50),
Dense(5)])
old_model.load_weights(“model.h5”)
# Create a new model with slightly different architecture (except for the layer in question, at least)
new_model = Sequential(layers = [
Dense(80, input_shape = (100,)),
Dense(60),
Dense(50),
Dense(5)])
# Set the weights of the final layer of the new model to the weights of the final layer of the old model, but leaving other layers unchanged.
new_model.layers[-1].set_weights(old_model.layers[-1].get_weights())
# Assert that the weights of the final layer is the same, but other are not.
print (np.all(new_model.layers[-1].get_weights()[0] == old_model.layers[-1].get_weights()[0]))
>> True
print (np.all(new_model.layers[-2].get_weights()[0] == old_model.layers[-2].get_weights()[0]))
>> False
I have a CNN model trained using EfficientNetB6.
My task is to extract the features of this trained model by removing the last dense layer and then using those weights to train a boosting model.
i did this using Pytorch earlier and was able to extract the weights from the layers i was interested and predicted on my validation set and then boosted.
I am doing this now in tensorflow but currently stuck.
Below is my model structure and I have tried using the code on the website but did not had any luck.
I want to remove the last dense layer and predict on the validation set using the remaining layers.
I tried using :
layer_name = 'efficientnet-b6'
intermediate_layer_model = tf.keras.Model(inputs = model.input, outputs = model.get_layer(layer_name).output)
but i get an error "
ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 760, 760, 3), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []"
Any way to resolve this?
Sorry my bad.
I simply added a GlobalAveragePooling2D layer after the efficientnet layer and i am able to extract the weights and continue :)
just for reference:
def build_model(dim=CFG['net_size'], ef=0):
inp = tf.keras.layers.Input(shape=(dim,dim,3))
base = EFNS[ef](input_shape=(dim,dim,3),weights='imagenet',include_top=False)
x = base(inp)
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dense(1,activation='sigmoid')(x)
model = tf.keras.Model(inputs=inp,outputs=x)
opt = tf.keras.optimizers.Adam(learning_rate=0.001)
loss = tf.keras.losses.BinaryCrossentropy(label_smoothing=0.05)
model.compile(optimizer=CFG['optimizer'],loss=loss,metrics=[tf.keras.metrics.AUC(name='auc')])
print(model.summary())
return model
history = model.fit(x_spectro_train, y_train_onehot, batch_size=batch_size, epochs=training_epochs, validation_data =(x_spectro_test, y_test_onehot), shuffle=True, callbacks=callbacks_list,class_weight=class_weights, verbose=1)
model=load_model(model_name)
predict_prob_train = model.predict(x_spectro_train,batch_size=batch_size)
inp = model.input # input placeholder
outputs = [layer.output for layer in model.layers] # all layer outputs
functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs] # evaluation functions
layer_outs = [func([x_spectro_train, 0.]) for func in functors] #test mode (0.0), train mode(1.0)
I want to save CNN layer outputs.
I want to train the svm model with CNN layer outputs (not probability)
So I used code from Keras, How to get the output of each layer? and I saw the result.
But my result of CNN layer is different from the result of model.predict.
I monitored the val accuracy, save the best model, and load it.
This is structure of my model. (below image)
enter image description here
I expected that result of layer_outs[13] (last layer) is same with predict_prob_train. However, the results were different. (like below image)
enter image description here
Why the results are different?
You have 7 layers after Conv layer (2 of which are Dense). They also learn stuff and they are 'making the decision' of the model output.
Think about it like this: Conv outputs something, that is the input to Dense1 -> Dense2. All those layers are learning simultaneously. So the goal of Dense1 layer is to learn what Conv layer is 'trying to tell it', how to interpret the results of Conv layer. If you input the image to this Dense1 layer and then to Dense2 layer, you won't get the same result (nor correct one). All of those layers are working together to get the correct prediction.
You cannot isolate 1 layer and expect the correct result.
I am trying to do transfer learning in Keras + Tensorflow on a selected subset of Places-205 dataset, containing only 27 categories. I am using InceptionV3, DenseNet121 and ResNet50, pre-trained on ImageNet, and add a couple of extra layers to adapt to my classes. If the model is ResNet, I add Flatten + Dense for classfication, and if it is DenseNet or Inceptionv3, I add Global Avg Pool + Dense (relu) + Dense (classification).
This is the code snippet:
x = base_model.output
if FLAGS.model in 'resnet50':
x = Flatten(name="flatten")(x)
else:
x = GlobalAveragePooling2D()(x)
# Let's add a fully-connected layer
x = Dense(1024, activation = 'relu')(x)
# And a logistic layer
predictions = Dense(classes, activation = 'softmax')(x)
For DenseNet and Inceptionv3 the training is ok, and the validation accuracy hits 70%, but for ResNet the validation accuracy stays fixed at 0.0369/0.037 (which is 1/27, my number of classes). It seems like it always predicts one class, but it's weird because its training progresses ok and the unspecific model code is exactly the same as for DenseNet and InceptionV3, which do work as expected.
Do you have any idea why it happens?
Thanks a lot!
I had a similar issue as you #Ciprian Andrei Focsaneanu, and what I have found to have worked was to make the previous layers (before the fully connected layers) trainable, as the filters/features of the ResNet50 were not suitable for my application.
Strangely enough, I also trained the VGG16 models, which was initially on the same images (imagenet) but its filters worked for my application, but I digress.
Here's the link to a page that inspired me to do this: https://datascience.stackexchange.com/questions/16840/multi-class-neural-net-always-predicting-1-class-after-optimization
Hope this helps!
I would like to inspect the internal state of an LSTM layer.
In particular I would like to look at the status at prediction time, after feeding a new example to the network.
I have understood this can be done using:
from keras import backend as K
# load pre-trained model somewhere
# select a LSTM layer
for layer in model.layers:
if 'LSTM' in str(layer):
break
# get inputs somewhere
val = np.random.random((...))
x = K.variable(value=val)
initial_state = layer.get_initial_states(???)
[desired_states]=layer.step(inputs=x,states=initial_state)
What is the input shape required by get_initial_states?
Is this a correct way of inspecting an LSTM