Develop Reference

transfer learning in keras when input shape is different

I have a question regarding transfer learning. Let suppose there is a neural network model that takes an input of shape (250,7). I want to initialise the model with the weights of this pre-trained model and then train it on my dataset to update the weights according to my dataset. But my dataset is of shape (251,8). Is there way to initialise the weights using pre-trained model considering my input shape is different? If so how can I do that? Insights will be appreciated.

You could try adding another layer before the transfer learning model. Just like the last layer, this will update its weights on your dataset and should work fine.

How to load the weights of the first few layers of Convolutional Neural Network in Keras and delete the pre-trained model?

I have a pretrained model trained in Keras.
I am trying to use that model in another task, but I don't need all the layers, but only the first 4 conv layers.
I have the model saved in "keras_pretrained_model.h5"
Is it possible to initialize the first 4 conv layers of the new model using the weights of the first 4 conv layers of the pretrained model from the '.h5' file?
Is loading the whole pretrained model first always necessary??
The pretrained model actually takes up a lot of space and I am not sure how to delete the pretrained model after I initializing the new model with the weights. As far as I understand, using tf.keras.backend.clear_session() will clear the new model created along with the old one.
So, my question is
Is there any way to initialize the weights in the new model layers without loading the whole pretrained model?
If I have to load the whole pretrained model, how to delete only the pretrained model without harming the new model in any way?

I have thought of two processes
If the names of the layers of the first 4 layers of the new model is same as the 4 layers in the pre-trained model, then
new_model.load_weights(path_to_old_model_file, by_name = True)
If the names don't mathc, then we can do layer wise weight initialization by taking the weights from the corresponding layers in the old model h5 file and setting the weights using set_weights() method.
I have written a code, which I have uploaded to github, here.
I would be very grateful, if anyone seeing this gives a feedback on this!!

How to use Deep Learning Models from Keras for a problem that does not fit imagenet dataset?

I followed a blog on how to implement a vgg16-model from scratch and want to do the same with the pretrained model from Keras. I looked up some other blogs but can't find a fitting solution I think. My task is to classify integrated circuit images into defect or non defects.
I have seen on a paper that they used pretrained imagenet model of vgg16 for fabric defect detection, where they freezed the first seven layers and fine tuned the last nine for their own problem.
(Source: https://journals.sagepub.com/doi/full/10.1177/1558925019897396)
I have already seen examples on how to freeze all layers except the fully connected layers, but how can I try the example with freezing first x layers and fine tune the others for my problem?
The VGG16 is fairly easy to implement from scratch but for models like resnet or xception it is getting a little trickier.

It is not necessary to implement a model from scratch to freeze a few layers. You can do this on pre-trained models as well. In keras, you'd use trainable = False.
For example, let's say you want to use the pre-trained Xception model from keras and want to freeze the first x layers:
#In your includes
from keras.applications import Xception
#Since you're using the model for a different task, you'd want to remove the top
base_model = Xception(weights='imagenet', include_top=False)
#Freeze layers 0 to x
for layer in base_model.layers[0:x]:
layer.trainable = False
#To see all the layers in detail and to check trainable parameters
base_model.summary()
Ideally you'd want to add another layer on top of this model with the output as your classes. For more details, you can check this keras guide: https://keras.io/guides/transfer_learning/
A lot of times the pre-trained weights can be very useful in other classification tasks but in case you want to train a model from scratch on your dataset, you can load the model without the imagenet weights. Or better, load the weights but don't freeze any layers. This will retrain every layer taking imagenet weights as an initialization.
I hope I've answered your question.

Training softmax layer in tensorflow object detection api

I'm trying to train an inventory-tracking application using tensorflow object detection api and I've used this tutorial.
My image dataset is too small for training all the weights in the neural network and I want to train the few latter layers or even just the softmax layer. But I didn't find any tutorial which tells me how to declare which layers I want to train.
How can I do this?
Can anyone give me a link or github issue about this?

Adding new units to a Keras model layer and changing their weights

I am working on a project that requires me to add new units to the output layer of a neural network to implement a form of transfer learning. I was wondering if I could do this and set the units' weights using either Keras or TensorFlow.
Specifically I would like to append an output neuron to the output layer of the Keras model and set that neuron's initial weights and bias.

Stumbled upon the answer to my own question. Thanks everyone for the answers/comments.
https://keras.io/layers/about-keras-layers/
The first few lines of this source detail how to load and set weights.
Essentially, appending an output neuron to a Keras model can be accomplished by loading the old output layer, appending the new weights, and setting weights for a new layer. Code is below.
# Load weights of previous output layer, set weights for new layer
old_layer_weights = model.layers.pop().get_weights()
new_neuron_weights = np.ndarray(shape=[1,bottleneck_size])
# Set new weights
# Append new weights, add new layer
new_layer = Dense(num_classes).set_weights(np.append(old_layer_weights,new_neuron_weights))
model.add(new_layer)

You could add new units to the output layer of a pre-trained neural network. This form of transfer learning is said to be called using the bottleneck features of a pre-trained network. This could be implemented both in tensorflow as well as in Keras.
Please find the tutorial in Keras below:
https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html
Also, find the tutorial for tensorflow below:
https://github.com/Hvass-Labs/TensorFlow-Tutorials/blob/master/08_Transfer_Learning.ipynb
Hope this helps!