I developed a machine learning model locally and wanted to deploy it as web service using Azure Functions.
At first model was save to binary using pickle module and then uploaded into blob storage associated with Azure Function (python language, http-trigger service) instance.
Then, after installing all required modules, following code was used to make a predicition on sample data:
import json
import pickle
import time
postreqdata = json.loads(open(os.environ['req']).read())
with open('D:/home/site/wwwroot/functionName/model_test.pkl', 'rb') as txt:
mod = txt.read()
txt.close()
model = pickle.loads(mod)
scs= [[postreqdata['var_1'],postreqdata['var_2']]]
prediciton = model.predict_proba(scs)[0][1]
response = open(os.environ['res'], 'w')
response.write(str(prediciton))
response.close()
where: predict_proba is a method of a trained model used for training and scs variable is definied for extracting particular values (values of variables for the model) from POST request.
Whole code works fine, predictions are send as a response, values are correct but the execution after sending a request lasts for 150 seconds! (locally it's less than 1s). Moreover, after trying to measure which part of a code takes so long: it's 10th line (pickle.loads(mod)).
Do you have any ideas why it takes such a huge amount of time? Model size is very small (few kB).
Thanks
When a call is made to the AML the first call must warm up the
container. By default a web service has 20 containers. Each container
is cold, and a cold container can cause a large(30 sec) delay.
I suggest you referring to this thread Azure Machine Learning Request Response latency and this article about Azure ML performance.
Hope it helps you.
Related
A bit confused with automatisation of Sagemaker retraining the model.
Currently I have a notebook instance with Sagemaker LinearLerner model making the classification task. So using Estimator I'm making training, then deploying the model creating Endpoint. Afterwards using Lambda function for invoke this endpoint, I add it to the API Gateway receiving the api endpoint which can be used for POST requests and sending back response with class.
Now I'm facing with the problem of retraining. For that I use serverless approach and lambda function getting environment variables for training_jobs. But the problem that Sagemaker not allow to rewrite training job and you can only create new one. My goal is to automatise the part when the new training job and the new endpoint config will apply to the existing endpoint that I don't need to change anything in API gateway. Is that somehow possible to automatically attach new endpoint config with existing endpoint?
Thanks
Yes, use the UpdateEndpoint endpoint. However, if you are using the Python Sagemaker SDK, be aware, there might be some documentation floating around asking you to call
model.deploy(..., update_endpoint=True)
This is apparently now deprecated in v2 of the Sagemaker SDK:
You should instead use the Predictor class to perform this update:
from sagemaker.predictor import Predictor
predictor = Predictor(endpoint_name="YOUR-ENDPOINT-NAME", sagemaker_session=sagemaker_session_object)
predictor.update_endpoint(instance_type="ml.t2.large", initial_instance_count=1)
If I am understanding the question correctly, you should be able to use CreateEndpointConfig near the end of the training job, then use UpdateEndpoint:
Deploys the new EndpointConfig specified in the request, switches to using newly created endpoint, and then deletes resources provisioned for the endpoint using the previous EndpointConfig (there is no availability loss).
If the API Gateway / Lambda is routed via the endpoint ARN, that should not change after using UpdateEndpoint.
I just started using aws sagemaker for running and maintaining models, experiments. just wanted to know is there any persistent layer for the sagemaker from where i can get data of my experiments/models instead of looking into the sagemaker studio. Does sagemaker saves the experiments or its data like s3 location in any table something like modelsdb?
SageMaker Studio is using the SageMaker API to pull all of the data its displaying. Essentially there's no secret API here getting invoked.
Quite a bit of what's being displayed with respect to experiments is from the search results, the rest coming from either List* or Describe* calls. Studio is taking the results from the search request and displaying it in the table format that you're seeing. Search results when searching over resource ExperimentTrialComponent that have a source (such as a training job) will be enhanced with the original sources data ([result]::SourceDetail::TrainingJob) if supported (work is ongoing to add additional source detail resource types).
All of the metadata that is related to resources in SageMaker is available via the APIs; there is no other location (in the cloud) like s3 for that data.
As of this time there is no effort to determine if it's possible to add support into modeldb for SageMaker that I'm aware of. Given that modeldb appears to make some assumptions about it's talking to a relational database it would appear unlikely to be something that would be doable. (I only read the overview very quickly so this might be inaccurate.)
I have built an ML model (using the sklearn module), and I want to serve it predictions via AWS API Gateway + Lambda function.
My problems are:
I can't install sklearn + numpy etc. because of the lambda capacity limitations. (the bundle is greater than 140MB)
Maybe that a silly question, but, do you know if there are better ways to do that task?
I've tried this tutorial, in order to reduce the bundle size. However, it raises an exception because of the --use-wheel flag.
https://serverlesscode.com/post/scikitlearn-with-amazon-linux-container/
bucket = s3.Bucket(os.environ['BUCKET'])
model_stream = bucket.Object(os.environ['MODEL_NAME'])
model = pickle.loads(model_stream)
model.predict(z_features)[0]
Where z_features are my features after using a scalar
Just figure it out!
The solution basically stands on top of the AWS Lambda Layers.
I created a sklearn layer that contains only the relevant compiled libraries.
Then, I run sls package in order to pack a bundle which contains those files together with my own handler.py code.
The last step was to run
sls deploy --package .serverless
Hope it'll be helpful to others.
If you simply want to serve your sklearn model, you could skip the hassle of setting up a lambda function and tinkering with the API Gateway - just upload your model as a pkl file to FlashAI.io which will serve your model automatically for free. It handles high traffic environments and unlimited inference requests. For sklearn models specifically, just check out the user guide and within 5 minutes you'll have your model available as an API.
Disclaimer: I'm the author of this service
I'm training a large-ish model, trying to use for the purpose Azure Machine Learning service in Azure notebooks.
I thus create an Estimator to train locally:
from azureml.train.estimator import Estimator
estimator = Estimator(source_directory='./source_dir',
compute_target='local',
entry_script='train.py')
(my train.py should load and train starting from a large word vector file).
When running with
run = experiment.submit(config=estimator)
I get
TrainingException:
====================================================================
While attempting to take snapshot of
/data/home/username/notebooks/source_dir Your total
snapshot size exceeds the limit of 300.0 MB. Please see
http://aka.ms/aml-largefiles on how to work with large files.
====================================================================
The link provided in the error is likely broken.
Contents in my ./source_dir indeed exceed 300 MB.
How can I solve this?
You can place the training files outside source_dir so that they don't get uploaded as part of submitting the experiment, and then upload them separately to the data store (which is basically using the Azure storage associated with your workspace). All you need to do then is reference the training files from train.py.
See the Train model tutorial for an example of how to upload data to the data store and then access it from the training file.
After I read the GitHub issue Encounter |total Snapshot size 300MB while start logging and the offical document Manage and request quotas for Azure resources for Azure ML service, I think it's an unknown issue which need some time to wait Azure to fix.
Meanwhile, I recommended that you can try to migrate the current work to the other service Azure Databricks, to upload your dataset and codes and then run it in the notebook of Azure Databricks which is host on HDInsight Spark Cluster without any worry about memory or storage limits. You can refer to these samples for Azure ML on Azure Databricks.
I have an R script for a random forest classification model. I connected my R instance to my redshift postgres database with RPostgreSQL and then ran a series of data transformation and model commands, using the caret library.
My model assigns a score to every user. I've been running the model on my own and uploading the data s3 and then transferring it to postgres with \copy
I'd like to automate the process. All the queries stay the same every time, including the R script to build the model, but the underlying data changes and so I want to make sure my model is calculating from the most up to date data and then rewriting the scores table.
I was going to use AWS lambda and set up a cron job, but the model takes more than 300 seconds. Instead, I'm thinking about this architecture:
1- Write a script that triggers the R job to run with a curl request. It would cause the R script to download the new data and create the model with a rest API through plumbR
2- Have the R API return the new scores as a JSON object and then use the Python script to upload the scores to s3 and then\copy them over to redshift postgres.
3- Set up a cron job on AWS elastic beanstalk to trigger this every Monday,
Could that work? Can I use AWS beanstalk to set up a python script to run automatically and, if so, does that mean I need an ec2 instance running 24/7?