I have used the "microsoft/resnet-50" for a vision task but I want to access the resnet layers inside the base model for Evaluation/Explainability purposes.
Code:
from transformers import AutoFeatureExtractor, TFResNetModel
model = TFResNetModel.from_pretrained("microsoft/resnet-50")
And model.summary() gives:
Since the resnet model is of type "TFResNetMainLayer", how can I access the layers inside this? Is it possible or are there any workarounds?
I have a trained model in ONNX format.
https://github.com/onnx/models/tree/main/vision/object_detection_segmentation/faster-rcnn
I need to convert it to PyTorch to make some modification with outputs like change number of classes, get network to tune on my dataset.
I followed this tutorial to train a pytorch model for instance segmentation:
https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html
I would not like to train a model on entirely different data and classes, totally unrelated to COCO. What changes do I need to make to retrain the model. From my reading I'm guessing besides have the correct number of classes I just need to train this line:
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)
to
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=False)
But I notice there is another parameters: pretrained_backbone=True, trainable_backbone_layers=None should they be changed too?
The function signature is
torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=False, progress=True, num_classes=91, pretrained_backbone=True, trainable_backbone_layers=3, **kwargs)
Setting pretrained=False will tell PyTorch not to download model pre-trained on COCO train2017. You want it as you're interested in training.
Usually, this is enough if you want to train on a different dataset.
When you set pretrained=False, PyTorch will download pretrained ResNet50 on ImageNet. And by default, it'll freeze first two blocks named conv1 and layer1. This is how it was done in Faster R-CNN paper which frooze the initial layers of pretrained backbone.
(Just print model to check its structure).
layers_to_train = ['layer4', 'layer3', 'layer2', 'layer1', 'conv1'][:trainable_layers]
Now, if you don't even want the first two layers to freeze, you can set trainable_backbone_layers=5 (done automatically, when you set pretrained_backbone=False), which will train the entire resnet backbone from scratch.
Check PR#2160.
From maskrcnn_resnet50_fpn document:
pretrained (bool) – If True, returns a model pre-trained on COCO train2017
pretrained_backbone (bool) – If True, returns a model with backbone pre-trained on Imagenet
trainable_backbone_layers (int) – number of trainable (not frozen) resnet layers starting from final block. Valid values are between 0 and 5, with 5 meaning all backbone layers are trainable.
So for training from scratch using:
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=False, pretrained_backbone=False, trainable_backbone_layers=5, num_classes=your_num_classes)
or:
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=False, pretrained_backbone=False, num_classes=your_num_classes)
because in source code of maskrcnn_resnet50_fpn:
if not (pretrained or pretrained_backbone):
trainable_backbone_layers = 5
I am trying to implement a model that uses encoding from multiple pre-trained BERT models on different datasets and gets a combined representation using a fully-connected layer. In this, I want that BERT models should remain fixed and only fully-connected layers should get trained. Is it possible to achieve this in huggingface-transformers? I don't see any flag which allows me to do that.
PS: I don't want to go by the way of dumping the encoding of inputs for each BERT model and use them as inputs.
A simple solution to this is to just exclude the parameters related to the BERT model while passing to the optimizer.
param_optimizer = [x for x in param_optimizer if 'bert' not in x[0]]
optimizer = AdamW(param_optimizer, lr)
I'm messing around with the Keras api in tensorflow, attempting to implement an autoencoder. The sequential model works, but I want to be able to use the encoder (first two layers) and the decoder (last two layers) separately, but using the weights of my already trained model. Is there a way to do this? Do I have to make a custom model?
model = keras.Sequential()
model.add(encoder_1)
model.add(leaky_relu)
model.add(encoder_2)
model.add(leaky_relu2)
model.add(decoder_1)
model.add(leaky_relu3)
model.add(decoder_2)
encoder_model = keras.Sequential()
encoder_model.add(encoder_1)
encoder_model.add(leaky_relu)
encoder_model.add(encoder_2)
encoder_model.add(leaky_relu2)
decoder_model = keras.Sequential()
decoder_model.add(decoder_1)
model.add(leaky_relu3)
decoder_model.add(decoder_2)
I define my models like this but trying to run predict on either the encoder or decoder outputs
'Sequential' object has no attribute '_feed_input_names'
Yes, you should wrap the encoding and decoding layers in separate Model instances that you call separately. The Keras blogporst on autoencoders should contain everything you need to know: https://blog.keras.io/building-autoencoders-in-keras.html