Develop Reference

How to train faster-rcnn in tensorflow using custom dataset in tfrecord format?

I have a data-set in tfrecord format annotated on RoboFlow. Now, I want to train a Faster-Rcnn and SSD from scratch in tensorflow. How will I load the dataset in the program and input to the model? Typical classification models have an input layer with an image shape defined but here I have to deal with image and the annotations as well.

how to confirm if the weights of my pytorch model has been quantized

I was able to successfully quantise a pytorch model for huggingface text classification with intel lpot(neural compressor)
I now have the original fp32 model and quantised int8 models in my machine. For inference I loaded the quantised lpot model with the below code
model = AutoModelForSequenceClassification.from_pretrained('fp32/model/path')
from lpot.utils.pytorch import load
modellpot = load("path/to/lpotmodel/", model)
I am able to see time time improvement of sorts, But I wanted to confirm if the model weights have been actually quantized and use data types such as int8,fp16 etc, which should be ideally the reason of speed up. I iterate over the model weights and print dtypes of the weights, but I see all weights are of type fp32
for param in modellpot.parameters():
print(param.data.dtype)
output
torch.float32
torch.float32
torch.float32
torch.float32
torch.float32
torch.float32
torch.float32
..
...
How do I verify if my pytorch model has been quantised?

Use print(modellpot) to check whether the model is quantized. For example, Linear layer will be converted to QuantizedLinear layer.
Actually, only layers that are supported in PyTorch will be converted into Quantized layer, so not all parameters are int8/uint8.
When the model is printed in the output for each you would be able to see the datatype eg the model output would show dtype as qint8 if int8 quantisation has been performed while printing the model.

How to convert segmentation model model to openvino int8 model?

I am trying convert my tensorflow segmentation model to openvino with quantization. I convert my .pb model to intermediate representation with openvino model optimizer. But how quantize model. In official documentation write that to do it with DL workbench. But in workbench i have only detection and classification dataset.
Can i convert my model to int8 without dataset or can i create dataset to segmentation?

The overall flow for converting a model from FP32 to INT8 is:
Select an FP32 model
Select an appropriate dataset
Run a baseline inference
Configure INT8 calibration settings
Configure inference settings for a calibrated model
View INT8 calibration
View inference results
Compare the calibrated model with the original FP32 model
Only some convolution models in the FP32 format can be quantized to INT8. If your model is incompatible, you will receive an error message.
The second stage of creating a configuration is adding a sample dataset. You can import a dataset, automatically generate a test dataset consisting of Gaussian distributed noise, or select a previously uploaded dataset.
You can find more details in the below link:
http://docs.openvinotoolkit.org/latest/_docs_Workbench_DG_Select_Datasets.html

You can find additional information about low precision inference in OpenVINO here:
General approach: https://docs.openvino.ai/latest/openvino_docs_IE_DG_Int8Inference.html
Post-Training Optimisation Tool (POT) with default algorithm: https://docs.openvino.ai/latest/pot_docs_LowPrecisionOptimizationGuide.html#doxid-pot-docs-low-precision-optimization-guide
Let me know if you still have questions.

Can I convert all the tensorflow slim models to tflite?

I'm training tensorflow slim based models for image classification on a custom dataset. Before I invest a lot of time training such huge a dataset, I wanted to know whether or not can I convert all the models available in the slim model zoo to tflite format.
Also, I know that I can convert my custom slim-model to a frozen graph. It is the step after this which I'm worried about i.e, conversion to .tflite from my custom trained .pb model.
Is this supported ? or is there anyone who is facing conversion problems that has not yet been resolved ?
Thanks.

Many Slim models can be converted to TFLite, but it isn't a guarantee since some models might have ops not supported by TFLite.
What you could do, is try and convert your model to TensorFlow Lite using TFLiteConverter in Python before training. If the conversion succeeds, then you can train your TF model and convert it once again.

Quantization support in Keras

I have a model that is trained in Keras with tensor flow backend. The weights are in .h5 format. I am interested in applying quantization feature part of tensorflow (https://www.tensorflow.org/api_docs/python/tf/quantization). So far, I have managed to convert the weights from .h5 format to tensor flow .pb format using the tool available online (https://github.com/amir-abdi/keras_to_tensorflow/). There are a couple of issues with this and the main concern is I don’t see a reduction in my model size post quantization. Also, I need to re-convert the .pb weights to .h5 format to test it with my infrastructure.
Is there a known best method for performing tensorflow
quantization within Keras?
Is there an easy way to convert weights format from .pb to .h5?
Thanks