Develop Reference

FastAPI loading model.pb - SavedModel file does not exist error

I'm trying to load a trained model on FastAPI and try pinging it from a notebook (to mimic a frontend call). But keep getting error saying the model file doesn't exist. I'm very new to this, any advice welcome...
Training notebook:
model.save('/data/model')
Downloaded the model and put the whole folder in the FastAPI folder.
File structure in FastAPI:
>> API
>> _pycache_
>> model
>> assets
>> variables
keras_metadata.pb
saved_model.pb
>> pyapi-env
api.py
api.py
from fastapi import FastAPI
from tensorflow.keras.models import load_model
...
#app.get("/predict")
def predict(test):
...
model = load_model("./model/saved_model.pb")
...
Testing notebook:
import requests
url = "http://localhost:8000/predict"
params = {
"test": "testing",
}
res = requests.get(url, params=params)
res.json()
Error: OSError: SavedModel file does not exist at: ./model/saved_model.pb\{saved_model.pbtxt|saved_model.pb}

I had the same issue and this worked for me:
model = load_model("./model/")
It seems like your code was treating "saved_model.pb" as a directory and looking for the model file inside it.

How to load an experiment in azureml?

I have many experiment, like:
and now, i want load an experiment
#%% sumonando os pacotes e verificando azureml.core
import azureml.core
import pandas as pd
import numpy as np
import logging
print("AzureML SDK Version: ", azureml.core.VERSION)
#%% Conectando ao azure e crinado o exparimento
from azureml.core import Workspace, Experiment
ws = Workspace.from_config()
print(Experiment.list(ws))
#%%
Experiment = Experiment.from_directory('teste2-Monitor-Runs') `
but
"error": {
"message": "No cache found for current project, try providing resource group and workspace
arguments"
}`
Content: azureml.core.Experiment class - Azure Machine Learning Python

Following AzureML's Experiment class documentation, they define an Experiment as a container of trials or runs. If you wish to access the container or the Experiment as a whole, you can use just access it by name (as long as you have the right workspace configured). See snippet below:
from azureml.core import Experiment, Workspace
ws = Workspace.from_config()
my_exp = Experiment(ws, name="teste2-Monitor-Runs")

I believe it is that way.
from azureml.core import Experiment, Workspace
Experiment = ws.experiments["teste2-Monitor-Runs"]

AWS SageMaker PyTorch: no module named 'sagemaker'

I have deployed a PyTorch model on AWS with SageMaker, and I try to send a request to test the service. However, I got a very vague error message saying "no module named 'sagemaker'". I have tried to search online, but cannot find posts about similar message.
My client code:
import numpy as np
from sagemaker.pytorch.model import PyTorchPredictor
ENDPOINT = '<endpoint name>'
predictor = PyTorchPredictor(ENDPOINT)
predictor.predict(np.random.random_sample([1, 3, 224, 224]).tobytes())
Detailed error message:
Traceback (most recent call last):
File "client.py", line 7, in <module>
predictor.predict(np.random.random_sample([1, 3, 224, 224]).tobytes())
File "/Users/jiashenc/Env/py3/lib/python3.7/site-packages/sagemaker/predictor.py", line 110, in predict
response = self.sagemaker_session.sagemaker_runtime_client.invoke_endpoint(**request_args)
File "/Users/jiashenc/Env/py3/lib/python3.7/site-packages/botocore/client.py", line 276, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/jiashenc/Env/py3/lib/python3.7/site-packages/botocore/client.py", line 586, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from model with message "No module named 'sagemaker'". See https://us-east-2.console.aws.amazon.com/cloudwatch/home?region=us-east-2#logEventViewer:group=/aws/sagemaker/Endpoints/<endpoint name> in account xxxxxxxxxxxxxx for more information.
This bug is because I merge both the serving script and my deploy script together, see below
import os
import torch
import numpy as np
from sagemaker.pytorch.model import PyTorchModel
from torch import cuda
from torchvision.models import resnet50
def model_fn(model_dir):
device = torch.device('cuda' if cuda.is_available() else 'cpu')
model = resnet50()
with open(os.path.join(model_dir, 'model.pth'), 'rb') as f:
model.load_state_dict(torch.load(f, map_location=device))
return model.to(device)
def predict_fn(input_data, model):
device = torch.device('cuda' if cuda.is_available() else 'cpu')
model.eval()
with torch.no_grad():
return model(input_data.to(device))
if __name__ == '__main__':
pytorch_model = PyTorchModel(model_data='s3://<bucket name>/resnet50/model.tar.gz',
entry_point='serve.py', role='jiashenC-sagemaker',
py_version='py3', framework_version='1.3.1')
predictor = pytorch_model.deploy(instance_type='ml.t2.medium', initial_instance_count=1)
print(predictor.predict(np.random.random_sample([1, 3, 224, 224]).astype(np.float32)))
The root cause is the 4th line in my code. It tries to import sagemaker, which is an unavailable library.

(edit 2/9/2020 with extra code snippets)
Your serving code tries to use the sagemaker module internally. The sagemaker module (also called SageMaker Python SDK, one of the numerous orchestration SDKs for SageMaker) is not designed to be used in model containers, but instead out of models, to orchestrate their activity (train, deploy, bayesian tuning, etc). In your specific example, you shouldn't include the deployment and model call code to server code, as those are actually actions that will be conducted from outside the server to orchestrate its lifecyle and interact with it. For model deployment with the Sagemaker Pytorch container, your entry point script just needs to contain the required model_fn function for model deserialization, and optionally an input_fn, predict_fn and output_fn, respectively for pre-processing, inference and post-processing (detailed in the documentation here). This logic is beautiful :) : you don't need anything else to deploy a production-ready deep learning server! (MMS in the case of Pytorch and MXNet, Flask+Gunicorn in the case of sklearn).
In summary, this is how your code should be split:
An entry_point script serve.py that contains model serving code and looks like this:
import os
import numpy as np
import torch
from torch import cuda
from torchvision.models import resnet50
def model_fn(model_dir):
# TODO instantiate a model from its artifact stored in model_dir
return model
def predict_fn(input_data, model):
# TODO apply model to the input_data, return result of interest
return result
and some orchestration code to instantiate a SageMaker Model object, deploy it to a server and query it. This is run from the orchestration runtime of your choice, which could be a SageMaker Notebook, your laptop, an AWS Lambda function, an Apache Airflow operator, etc - and with the SDK for your choice; don't need to use python for this.
import numpy as np
from sagemaker.pytorch.model import PyTorchModel
pytorch_model = PyTorchModel(
model_data='s3://<bucket name>/resnet50/model.tar.gz',
entry_point='serve.py',
role='jiashenC-sagemaker',
py_version='py3',
framework_version='1.3.1')
predictor = pytorch_model.deploy(instance_type='ml.t2.medium', initial_instance_count=1)
print(predictor.predict(np.random.random_sample([1, 3, 224, 224]).astype(np.float32)))

Couldn't create model.tar.gz file while training scikit learn model in AWS Sagemaker

I want to create an endpoint for scikit logistic regression in AWS Sagemaker. I have a train.py file which contains training code for scikit sagemaker.
import subprocess as sb
import pandas as pd
import numpy as np
import pickle,json
import sys
def install(package):
sb.call([sys.executable, "-m", "pip", "install", package])
install('s3fs')
import argparse
import os
if __name__ =='__main__':
parser = argparse.ArgumentParser()
# hyperparameters sent by the client are passed as command-line arguments to the script.
parser.add_argument('--solver', type=str, default='liblinear')
# Data, model, and output directories
parser.add_argument('--output_data_dir', type=str, default=os.environ.get('SM_OUTPUT_DIR'))
parser.add_argument('--model_dir', type=str, default=os.environ.get('SM_MODEL_DIR'))
parser.add_argument('--train', type=str, default=os.environ.get('SM_CHANNEL_TRAIN'))
args, _ = parser.parse_known_args()
# ... load from args.train and args.test, train a model, write model to args.model_dir.
input_files = [ os.path.join(args.train, file) for file in os.listdir(args.train) ]
if len(input_files) == 0:
raise ValueError(('There are no files in {}.\n' +
'This usually indicates that the channel ({}) was incorrectly specified,\n' +
'the data specification in S3 was incorrectly specified or the role specified\n' +
'does not have permission to access the data.').format(args.train, "train"))
raw_data = [ pd.read_csv(file, header=None, engine="python") for file in input_files ]
df = pd.concat(raw_data)
y = df.iloc[:,0]
X = df.iloc[:,1:]
solver = args.solver
from sklearn.linear_model import LogisticRegression
lr = LogisticRegression(solver=solver).fit(X, y)
from sklearn.externals import joblib
def model_fn(model_dir):
lr = joblib.dump(lr, "model.joblib")
return lr
In my sagemaker notebook I ran the following code
import os
import boto3
import re
import copy
import time
from time import gmtime, strftime
from sagemaker import get_execution_role
import sagemaker
role = get_execution_role()
region = boto3.Session().region_name
bucket=<bucket> # Replace with your s3 bucket name
prefix = <prefix>
output_path = 's3://{}/{}/{}'.format(bucket, prefix,'output_data_dir')
train_data = 's3://{}/{}/{}'.format(bucket, prefix, 'train')
train_channel = sagemaker.session.s3_input(train_data, content_type='text/csv')
from sagemaker.sklearn.estimator import SKLearn
sklearn = SKLearn(
entry_point='train.py',
train_instance_type="ml.m4.xlarge",
role=role,output_path = output_path,
sagemaker_session=sagemaker.Session(),
hyperparameters={'solver':'liblinear'})
I'm fitting my model here
sklearn.fit({'train': train_channel})
Now, for creating endpoint,
from sagemaker.predictor import csv_serializer
predictor = sklearn.deploy(1, 'ml.m4.xlarge')
While trying to create endpoint, it is throwing
ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Could not find model data at s3://<bucket>/<prefix>/output_data_dir/sagemaker-scikit-learn-x-y-z-000/output/model.tar.gz.
I checked my S3 bucket. Inside my output_data_dir there is sagemaker-scikit-learn-x-y-z-000 dir which has debug-output\training_job_end.ts file. An additional directory got created outside my <prefix> folder with name sagemaker-scikit-learn-x-y-z-000 that has source\sourcedir.tar.gz file. Generally whenever I trained my models with sagemaker built-in algorithms, output_data_dir\sagemaker-scikit-learn-x-y-z-000\output\model.tar.gz kind of files get created. Can someone please tell me where my scikit model got stored, how to push source\sourcedir.tar.gz inside my prefix code without having doing it manually and how to see contents of sourcedir.tar.gz?
Edit: I elaborated the question regarding prefix. Whenever I run sklearn.fit(), two files with same name sagemaker-scikit-learn-x-y-z-000 are getting created in my S3 bucket. One created inside my <bucket>/<prefix>/output_data_dir/sagemaker-scikit-learn-x-y-z-000/debug-output/training_job_end.ts and other file is created in <bucket>/sagemaker-scikit-learn-x-y-z-000/source/sourcedir.tar.gz. Why is the second file not created inside my <prefix> like the first one? What is contained in sourcedir.tar.gz file?

I am not sure if your model is really stored, if you can't find it in S3. While you define a function with the call of joblib.dump in your entry point script, I am having the call at the end of the main. For example:
# persist model
path = os.path.join(args.model_dir, "model.joblib")
joblib.dump(myestimator, path)
print('model persisted at ' + path)
Then the file can be found in ..\output\model.tar.gz just as in your other cases. In order to double-check that is created you maybe want to have a print statement that can be found in the protocol of the training.

You must dump the model as the last step of your training code. Currently you are doing it in the wrong place, as model_fn goal is to load the model for inference, not for training.
Add the dump after training:
lr = LogisticRegression(solver=solver).fit(X, y)
lr = joblib.dump(lr, args.model_dir)
Change model_fn() to load the model instead of dumping it.
See more here.

This post here explains it well:
https://towardsdatascience.com/deploying-a-pre-trained-sklearn-model-on-amazon-sagemaker-826a2b5ac0b6
In short, the tar.gz gets created by tar-gz-ing the model.joblib binary which was first created joblib.dump. To quote the article:
#Build tar file with model data + inference code
bashCommand = "tar -cvpzf model.tar.gz model.joblib inference.py"
process = subprocess.Popen(bashCommand.split(), stdout=subprocess.PIPE)
output, error = process.communicate()
The inference.py is probably optional.

How to test a RASA model?

I'm trying to write my own chatbot with the RASA framework.
Right now I'm just playing around with it and I have the following piece of code for training purposes.
from rasa.nlu.training_data import load_data
from rasa.nlu.config import RasaNLUModelConfig
from rasa.nlu.model import Trainer
from rasa.nlu import config
training_data = load_data("./data/nlu.md")
trainer = Trainer(config.load("config.yml"))
interpreter = trainer.train(training_data)
model_directory = trainer.persist("./models/nlu",fixed_model_name="current")
Now, I read that if I wanted to test it I should do something like this.
from rasa.nlu.evaluate import run_evaluation
run_evaluation("nlu.md", model_directory)
But this code is not available anymore in rasa.nlu.evaluate nor in rasa.nlu.test!
What's the way, then, of testing a RASA model?

The module was renamed.
Please import
from rasa.nlu.test import run_evaluation
Alternatively you now also do
from rasa.nlu import test
test_result = test(path_to_test_data, unpacked_model)
intent_evaluation_report = test_result["intent_evaluation"]["report"]
print(intent_evaluation_report)