I developed a Convolutional Neural Network algorithm in python, which classifies images (.jpg) with a specific label, by
1) defining a custom CNN model;
2) setting up an estimator, which locally saves the save_summary and save_checkpoint steps;
3) training the estimator with the estimator.train function.
Now, if I run the estimator.predict function, with a new image, it returns the predicted label.
How can I deploy this trained estimator as RESTful API so that I can call it from a WEB page or an application?
It would help to know if you have used some known framework (Keras, Tensorflow, MXNet...), as most have some recommended ways to serve models over an API.
If you built your solution from scratch, you can "just" use any web framework to deliver your model predictions. To get you started, you may want to take a look to Flask.
Related
I have built a custom Pytorch model, that I would like to deploy as a project. I'm doing so with Airflow on two separate containers. I'm having a problem making predictions on the inference container, and get errors because constants.pkl is not found. More so, when I load the model locally, I have to recreate the object with the parameters used at training.
I tried to save the entire model according to the Pytorch docs, and I can't make predictions on the model.
I am using a simple perceptron based classifier to generate sentiment analysis in Pytorch, complete code here Classifying Yelp Reviews.
The example does sentiment analysis, it outputs if the given input string is positive or negative.
example: this is a pretty small old great book -> positive
The application stores the final model along with the vectorizer.json.
So my question is: What should be the pre-requisites to build a separate application just to test the model, so that it can be used in web-application later-on?
Below is my current understanding and queries for this:
I assume to test, we need to load the model, load model parameters and evaluate for inference, please confirm
model = TheModelClass(*args, **kwargs) # Model class must be defined somewhere
model.load_state_dict(torch.load(PATH))
model.eval() # run if you only want to use it for inference
Once step 1 is done, I hope we can deploy the model using Flask and expose a REST API for model inference.
I found a nice tutorial explaining how to load model for inference, here is the link
https://pytorch.org/tutorials/recipes/recipes/saving_and_loading_models_for_inference.html
Thanks!
i trained keras model on 2 classes and converted it into the .tflite model. now i want to use this model in android studio for simple classification between two classes when i put an image from the gallery. i can't find any help on internet regarding this simple way. On internet there is ways for camera but i don't need that in my simple project.
You will need to use TensorFlow Lite's Java API to run inference on-device. Start with the Android quickstart, if you haven't installed the dependencies etc.
If your model is similar to how standard image classification models (for eg MobileNet) work, you can start with the TFLite Image Classification app source for inspiration. Specifically, you might be interested in the Classifier base-class (and its floating point child class). Classifier demonstrates how you can use TFLite's Java API to instantiate a new Interpreter & run inference for image inputs.
I have a trained machine learning model in python to obtain a regression output, this model is trained with scikit-learn
I want to insert this predictions into firestore, I am going to do it with a cloud function scheduling it every day with cloud scheduler.
My question is where I have to store this trained machine learning model?
Can I store it into google storage and call it in my cloud function to obtain predictions?
Or I should store it into AI platform?
If the answer is into AI platform, why? what advantages I have if I store it into AI platform? can I train the model with new data from there?
I have been reading that this is possible but I don't know why is better and how to it
There is several answer to your question.
Do you want to build a monolith or 2 microservices:
Monolith, I mean the same service (Functions or container) is triggered by the scheduler, load the model, perform the prediction and save it to firestore
Microservice:
1 service is triggered by the scheduler, request a prediction and store the result into Firestore
1 service load the model and answer to prediction query.
In monolith case, AI-Platform is not recommended. In microservice, you can host your prediction service on AI Platform, and the other on Cloud Functions
With tensorflow I also proposed another solution for hosting the model: in Cloud Run. I wrote an article on this. I don't know enough SciKit for telling you is the same thing is possible, but it's a good alternative.
About where to store your trained model? Definitively on Cloud Storage. And even if you build a Cloud Run service with a container like described in my article, where I download the model and load it into the container (and thus the model is not downloaded from Storage at runtime, only at build time), Cloud Storage is the best place for immutable objects.
Finally, your last question about AI Platform. A same name, several services. You can host your model and perform Online Prediction, and you can train your model. It's not the same internal service, not the same usage, not the same API. There is no difference/advantage when you train new model, if you host your online prediction on AI Platform or not
I'm using Google cloud machine learning. I would like to identify different images.
Now I have trained my model with different type of image (using inception model of tensorflow), and I have created a version in Google machine learning with the results.
How can I get prediction about a new image?
Do you have some idea to help me?
Many thanks!
I'm not quite clear on what you're asking. Without more information, I will just point you to the Google blog post and code sample that detail how to train on images.
But back to what I think you're asking...for a model to be deployed to Google Cloud ML a few things have to happen:
It needs to have its inputs and output collections declared in the Tensorflow model before saving the checkpoint.
The model checkpoint needs to be copied to GCS
You must use gcloud to create a new "model" (as far as gcloud is concerned, a model is a namespace for many different tensorflow checkpoints) and then deploy your checkpoint to that gcloud model.
The prediction quickstart has a very similar example here.