I have built an Image Classifier model using Inception V3 and I have saved the model in "SavedModel" format to deploy it to production. I am wondering how I can bundle the pre-processing steps to the final model so that the model ingest data in its natural form.
The pre-processing steps that I have are:
- resizing the image to target_size of 299, 299 using keras load_model
- change the image to numpy array
- expand dimensions
- pre_process input using inception_v3 import preprocess_input call
When a model is deployed, as per my understanding what actually is deployed is the python code for inference utilising the model. In this python code you can write code for all your preprocessing using openCV or any other python libraries and pass the image as an argument to this python code.
eg inferenceFile.py imageToInfer.png
An out of the box thought would be to write a different deep learning model to which as input your non-preprocessed image and output the preprocessed image you feed to the model, Not sure if this could be achievable.
Related
Tl;DR: How could I access the pytorch pre-trained model for Swin-Transformer so that I could extract features from it to train it on segmentation task using DeepLabv3+ head on a custom data set with image sizes of 512
I am testing SwinTransformer backbone with Deeplabv3+ as head for semantic segmentation.
I already have the code for Head and it get the features from backbone and then process the features in different way. It is working fine for ResNet and Xception. The thing is that I want to work with pre-trained SWIN. Now there are many ways but each of those has it's own set of problems.
1. Using timm library for pre-trainned models`
! pip install timm
import timm
import torch
all_swins = timm.list_models('*swin*')
print(all_swins)
model = timm.create_model('swin_large_patch4_window12_384_in22k', in_chans = 3, pretrained = True,)
print(model.default_cfg)
dummy_image = torch.randn(1,3,512,512) # create an image of size 512
model.forward_features(dummy_image) # extract features : Will result in error
new_model = timm.create_model('swin_large_patch4_window12_384_in22k', in_chans = 3, pretrained = True, img_size = 512) # This won't work I'll explain below
The problem is that None of the above code work. Reason being that the model accepts here an image of size 384 but my images are of 512. So you could change the argument img_size for other CNNs but as the author has clarified in this question
It should work with the vit, vit_deit, vit_deit_distilled. Has not been implemented for pit, swin, and tnt yet.
2. Using MMcv / MMSeg library:
Please open this colab notebook. I have commented and documented the part
Problem: The pre-trained weights are for only for a specific method which produced SOTA results i.e ADE dataset using UperNet backbone. I can not use it with DeepLabv3+ on a custom dataset.
3. Segmentations Models Pytorch Library which uses timm encoders
Problem: Again, as it uses timm, so the image resolutions can't be changed.
4. PaddleSeg Library
It has Swin transformer but Deeplabv3+ works only with Resnet50 and 101
Last Resort: In the end, I pulled up the official code from microsoft where I found couple of useful things:
configuration yml file
Code which they use for model building def build_model()
Code inside def main() which parses the arguments and builds the whole model
I don't really know how to build the model changing the image size configurations. If anyone has any idea, please help.
I created a model of darknet53.weights for image classification using my original data in darknet.
(This isn't a YOLO v3 model.)
Is there a way to convert a darknet53.weight to a pytorch pt model?
I tried quoting various codes on github etc., but all of them can convert only YOLOv3 weights file to pytorch's pt model.
I want to compare the accuracy of the darknet53 model created with darknet with other image classification models created with pytorch.
Initially, I tried to make a darknet53 model with pytorch, but that didn't work. Therefore, I created a darknet53 model with darknet.
If anyone knows a good way, please teach me.
Thanks.
I am trying to implement object detection using MobileNetV2 model on Flutter. Since, most of the examples or implementation available online for Flutter app is not using MobileNetV2, so I took a long route to reach to that phase.
The way I achieved this is as follows:
1) Created a python script where I am using MobileNetV2 model (pre-trained on ImageNet for 1000 classes) of Keras (backend Tensorflow) and tested it with images to see if it is returning the correct labels after detecting objects correctly. [Python script provided below for reference]
2) Converted the same MobileNetV2 keras model (MobileNetV2.h5) to Tensorflow Lite model (MobileNetV2.tflite)
3) Followed the existing example of creating Flutter app to use Tensorflow Lite (https://itnext.io/working-with-tensorflow-lite-in-flutter-f00d733a09c3). Replaced the TFLite model shown in the example with the MobileNetV2.tflite model and used the ImageNet classes/labels in https://gist.github.com/aaronpolhamus/964a4411c0906315deb9f4a3723aac57 as the labels.txt.
[GitHub project of the Flutter example is provided here: https://github.com/umair13adil/tensorflow_lite_flutter]
When I now run the Flutter app, it is running without any error, however during classification/predicting the label the output is not correct. For example: It classifies an orange (object id: n07747607) as poncho (object id: n03980874), and classifies pomegranate (object id: n07768694) as banded_gecko (object id: n01675722).
However, if I use the same pictures and test it with my python script, it is returning the correct labels. So, I was wondering if the issue is actually with the label.txt (which holds the labels) used in the Flutter app, where the order of the labels is not matching the inference of the model.
Can anyone mention that how I can resolve the issue to classify the correct objects? How can I get the ImageNet labels that are used by the MobileNetV2 (keras) so that I can use that in the Flutter app?
My Flutter App to detect object using MobileNetv2 can be downloaded from here: https://github.com/somdipdey/Tensorflow_Lite_Object_Detection_Flutter_App
My python script to convert the MobileNetV2 model (keras) to TFLite while testing it on image for classification as follows:
import tensorflow as tf
from tensorflow import keras
from keras.preprocessing import image
from keras.applications.mobilenet_v2 import preprocess_input, decode_predictions
import numpy as np
import PIL
from PIL import Image
import requests
from io import BytesIO
# load the model
model = tf.keras.applications.MobileNetV2(weights='imagenet', include_top=True)
#model = tf.keras.models.load_model('MobileNetV2.h5')
# To save model
model.save('MobileNetV2.h5')
# chose the URL image that you want
URL = "https://images.unsplash.com/photo-1557800636-894a64c1696f?ixlib=rb-1.2.1&w=1000&q=80"
# get the image
response = requests.get(URL)
img = Image.open(BytesIO(response.content))
# resize the image according to each model (see documentation of each model)
img = img.resize((224, 224))
##############################################
# if you want to read the image from your PC
#############################################
# img_path = 'myimage.jpg'
# img = image.load_img(img_path, target_size=(299, 299))
#############################################
# convert to numpy array
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
features = model.predict(x)
# return the top 10 detected objects
num_top = 10
labels = decode_predictions(features, top=num_top)
print(labels)
#load keras model
new_model= tf.keras.models.load_model(filepath="MobileNetV2.h5")
# Create a converter # I could also directly use keras model instead of loading it again
converter = tf.lite.TFLiteConverter.from_keras_model(new_model)
# Convert the model
tflite_model = converter.convert()
# Create the tflite model file
tflite_model_name = "MobileNetV2.tflite"
open(tflite_model_name, "wb").write(tflite_model)
Let me start by sharing the ImageNet labels in two formats, JSON and txt. Given the fact MobileNetV2 is trained on ImageNet, it should be returning results based on these labels.
My initial thought is that there must be an error with the 2nd step of your pipeline. I assume you are trying to convert the trained Keras-based weights to Tensorflow Lite weights (is it the same format with pure Tensorflow?). A good option is to try and find already saved weights in the format of Tensorflow Lite ( but I guess they might not be available and that's why you are doing the conversion). I had similar problems with converting TF weights to Keras so you must be sure whether the conversion was successfully done before even going to step 3, creation of Flutter app to use Tensorflow Lite. A good way to achieve this is by printing all the available classes of your classifier and compare them with the original ImageNet labels given above.
I am fairly new to keras and I am trying transfer learning here:
https://www.tensorflow.org/tutorials/images/transfer_learning
My dataset however is not a binary and I have tfrecord file. I can read the file in tensorflow. I do not want to feed the images as an input to the network as the input comes from the pre-trained model. How can I pass the images and labels in the ImageDataGenerator class in Keras.
For anyone that may have this issue in the future. If the pre-train process is all correct. You can use the tf.data API to read and prepare the images for the training and the (image, label) set, can be fed to to the (.fit) method of your model.
look at this great Post to get familiar how to read the tfrecord file:
https://medium.com/#moritzkrger/speeding-up-keras-with-tfrecord-datasets-5464f9836c36
Directory structure:
Data
-Cats
--<images>.jpg
-Dogs
--<images>.jpg
I'm training a (n-ary) classification model. I want to create an input_fn for serving these images for training.
image dimensions are (200, 200, 3). I have a (keras) generator for them, if they can be used somehow.
I've been looking for a while but haven't found an easy way to do this. I thought this should be a standard use-case? e.g. Keras provides flow_from_directory to serve keras models. I need to use a tf.estimator for AWS Sagemaker so I'm stuck with it.
By using the tf dataset Module you can feed your data directly into your estimator. You basically have 3 ways to integrate this into your api:
1. convert your images into tfrecords and use tfrecorddataset
2 use the tf dataset from generator function to use generators
3 try introducing these decoder functions into your inputpipeline