I am doing an Azure Databricks lab 04. Integrating Azure Databricks and Azure Machine Learning -> 2. Deploying Models in Azure Machine Learning. The idea is to 1. train a model 2) deploy that model in an Azure Container Instance (ACI) in AML and 3) make predictions via HTTPS. However, I get an error when deploying the model.
The full code from the notebook is displayed at the bottom or can be found here: https://adb-4934989010098757.17.azuredatabricks.net/?o=4934989010098757#notebook/4364513836468644/command/4364513836468645 .
I run the actual model deployment in the following way:
aci_service_name='nyc-taxi-service'
service = Model.deploy(workspace=ws,
name=aci_service_name,
models=[registered_model],
inference_config=inference_config,
deployment_config= aci_config,
overwrite=True)
service.wait_for_deployment(show_output=True)
print(service.state)
After running the model deployment, the cell runs for over 25 minutes and breaks when checking the status of the inference endpoint. It gives the following error: "
"Service deployment polling reached non-successful terminal state, current service state: Failed
code": "AciDeploymentFailed",
"statusCode": 400,
"message": "Aci Deployment failed with exception: Your container application crashed. This may be caused by errors in your scoring file's init() function.
The scoring script looks like this:
script_dir = 'scripts'
dbutils.fs.mkdirs(script_dir)
script_dir_path = os.path.join('/dbfs', script_dir)
print("Script directory path:", script_dir_path)
%%writefile $script_dir_path/score.py
import json
import numpy as np
import pandas as pd
import sklearn
import joblib
from azureml.core.model import Model
columns = ['passengerCount', 'tripDistance', 'hour_of_day', 'day_of_week',
'month_num', 'normalizeHolidayName', 'isPaidTimeOff', 'snowDepth',
'precipTime', 'precipDepth', 'temperature']
def init():
global model
model_path = Model.get_model_path('nyc-taxi-fare')
model = joblib.load(model_path)
print('model loaded')
def run(input_json):
# Get predictions and explanations for each data point
inputs = json.loads(input_json)
data_df = pd.DataFrame(np.array(inputs).reshape(-1, len(columns)), columns = columns)
# Make prediction
predictions = model.predict(data_df)
# You can return any data type as long as it is JSON-serializable
return {'predictions': predictions.tolist()}
Does someone know how I could fix this potentially? Thanks in advance!
The full code is displayed below:
**Required Libraries**:
* `azureml-sdk[databricks]` via PyPI
* `sklearn-pandas==2.1.0` via PyPI
* `azureml-mlflow` via PyPI
import os
import numpy as np
import pandas as pd
import pickle
import sklearn
import joblib
import math
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn_pandas import DataFrameMapper
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import matplotlib
import matplotlib.pyplot as plt
import azureml
from azureml.core import Workspace, Experiment, Run
from azureml.core.model import Model
print('The azureml.core version is {}'.format(azureml.core.VERSION))
%md
### Connect to the AML workspace
%md
In the following cell, be sure to set the values for `subscription_id`, `resource_group`, and `workspace_name` as directed by the comments. Please note, you can copy the subscription ID and resource group name from the **Overview** page on the blade for the Azure ML workspace in the Azure portal.
#Provide the Subscription ID of your existing Azure subscription
subscription_id = " ..... "
#Replace the name below with the name of your resource group
resource_group = "RG_1"
#Replace the name below with the name of your Azure Machine Learning workspace
workspace_name = "aml-ws"
print("subscription_id:", subscription_id)
print("resource_group:", resource_group)
print("workspace_name:", workspace_name)
%md
**Important Note**: You will be prompted to login in the text that is output below the cell. Be sure to navigate to the URL displayed and enter the code that is provided. Once you have entered the code, return to this notebook and wait for the output to read `Workspace configuration succeeded`.
*Also note that the sign-on link and code only appear the first time in a session. If an authenticated session is already established, you won't be prompted to enter the code and authenticate when creating an instance of the Workspace.*
ws = Workspace(subscription_id, resource_group, workspace_name)
print(ws)
print('Workspace region:', ws.location)
print('Workspace configuration succeeded')
%md
### Load the training data
In this notebook, we will be using a subset of NYC Taxi & Limousine Commission - green taxi trip records available from [Azure Open Datasets]( https://azure.microsoft.com/en-us/services/open-datasets/). The data is enriched with holiday and weather data. Each row of the table represents a taxi ride that includes columns such as number of passengers, trip distance, datetime information, holiday and weather information, and the taxi fare for the trip.
Run the following cell to load the table into a Spark dataframe and reivew the dataframe.
dataset = spark.sql("select * from nyc_taxi_1_csv").toPandas()
display(dataset)
%md
### Use MLflow with Azure Machine Learning for Model Training
In the subsequent cells you will learn to do the following:
- Set up MLflow tracking URI so as to use Azure ML
- Create MLflow experiment – this will create a corresponding experiment in Azure ML Workspace
- Train a model on Azure Databricks cluster while logging metrics and artifacts using MLflow
- Save the trained model to Databricks File System (DBFS)
import mlflow
mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
experiment_name = 'MLflow-AML-Exercise'
mlflow.set_experiment(experiment_name)
print("Training model...")
output_folder = 'outputs'
model_file_name = 'nyc-taxi.pkl'
dbutils.fs.mkdirs(output_folder)
model_file_path = os.path.join('/dbfs', output_folder, model_file_name)
with mlflow.start_run() as run:
df = dataset.dropna(subset=['totalAmount'])
x_df = df.drop(['totalAmount'], axis=1)
y_df = df['totalAmount']
X_train, X_test, y_train, y_test = train_test_split(x_df, y_df, test_size=0.2, random_state=0)
numerical = ['passengerCount', 'tripDistance', 'snowDepth', 'precipTime', 'precipDepth', 'temperature']
categorical = ['hour_of_day', 'day_of_week', 'month_num', 'normalizeHolidayName', 'isPaidTimeOff']
numeric_transformations = [([f], Pipeline(steps=[
('imputer', SimpleImputer(strategy='median')),
('scaler', StandardScaler())])) for f in numerical]
categorical_transformations = [([f], OneHotEncoder(handle_unknown='ignore', sparse=False)) for f in categorical]
transformations = numeric_transformations + categorical_transformations
clf = Pipeline(steps=[('preprocessor', DataFrameMapper(transformations, df_out=True)),
('regressor', GradientBoostingRegressor())])
clf.fit(X_train, y_train)
y_predict = clf.predict(X_test)
y_actual = y_test.values.flatten().tolist()
rmse = math.sqrt(mean_squared_error(y_actual, y_predict))
mlflow.log_metric('rmse', rmse)
mae = mean_absolute_error(y_actual, y_predict)
mlflow.log_metric('mae', mae)
r2 = r2_score(y_actual, y_predict)
mlflow.log_metric('R2 score', r2)
plt.figure(figsize=(10,10))
plt.scatter(y_actual, y_predict, c='crimson')
plt.yscale('log')
plt.xscale('log')
p1 = max(max(y_predict), max(y_actual))
p2 = min(min(y_predict), min(y_actual))
plt.plot([p1, p2], [p1, p2], 'b-')
plt.xlabel('True Values', fontsize=15)
plt.ylabel('Predictions', fontsize=15)
plt.axis('equal')
results_graph = os.path.join('/dbfs', output_folder, 'results.png')
plt.savefig(results_graph)
mlflow.log_artifact(results_graph)
joblib.dump(clf, open(model_file_path,'wb'))
mlflow.log_artifact(model_file_path)
%md
Run the cell below to list the experiment run in Azure Machine Learning Workspace that you just completed.
aml_run = list(ws.experiments[experiment_name].get_runs())[0]
aml_run
%md
## Exercise 1: Register a databricks-trained model in AML
Azure Machine Learning provides a Model Registry that acts like a version controlled repository for each of your trained models. To version a model, you use the SDK as follows. Run the following cell to register the model with Azure Machine Learning.
model_name = 'nyc-taxi-fare'
model_description = 'Model to predict taxi fares in NYC.'
model_tags = {"Type": "GradientBoostingRegressor",
"Run ID": aml_run.id,
"Metrics": aml_run.get_metrics()}
registered_model = Model.register(model_path=model_file_path, #Path to the saved model file
model_name=model_name,
tags=model_tags,
description=model_description,
workspace=ws)
print(registered_model)
%md
## Exercise 2: Deploy a service that uses the model
%md
### Create the scoring script
script_dir = 'scripts'
dbutils.fs.mkdirs(script_dir)
script_dir_path = os.path.join('/dbfs', script_dir)
print("Script directory path:", script_dir_path)
%%writefile $script_dir_path/score.py
import json
import numpy as np
import pandas as pd
import sklearn
import joblib
from azureml.core.model import Model
columns = ['passengerCount', 'tripDistance', 'hour_of_day', 'day_of_week',
'month_num', 'normalizeHolidayName', 'isPaidTimeOff', 'snowDepth',
'precipTime', 'precipDepth', 'temperature']
def init():
global model
model_path = Model.get_model_path('nyc-taxi-fare')
model = joblib.load(model_path)
print('model loaded')
def run(input_json):
# Get predictions and explanations for each data point
inputs = json.loads(input_json)
data_df = pd.DataFrame(np.array(inputs).reshape(-1, len(columns)), columns = columns)
# Make prediction
predictions = model.predict(data_df)
# You can return any data type as long as it is JSON-serializable
return {'predictions': predictions.tolist()}
%md
### Create the deployment environment
from azureml.core import Environment
from azureml.core.environment import CondaDependencies
my_env_name="nyc-taxi-env"
myenv = Environment.get(workspace=ws, name='AzureML-Minimal').clone(my_env_name)
conda_dep = CondaDependencies()
conda_dep.add_pip_package("numpy==1.18.1")
conda_dep.add_pip_package("pandas==1.1.5")
conda_dep.add_pip_package("joblib==0.14.1")
conda_dep.add_pip_package("scikit-learn==0.24.1")
conda_dep.add_pip_package("sklearn-pandas==2.1.0")
conda_dep.add_pip_package("azure-ml-api-sdk")
myenv.python.conda_dependencies=conda_dep
print("Review the deployment environment.")
myenv
%md
### Create the inference configuration
from azureml.core.model import InferenceConfig
inference_config = InferenceConfig(entry_script='score.py', source_directory=script_dir_path, environment=myenv)
print("InferenceConfig created.")
%md
### Create the deployment configuration
In this exercise we will use the Azure Container Instance (ACI) to deploy the model
from azureml.core.webservice import AciWebservice, Webservice
description = 'NYC Taxi Fare Predictor Service'
aci_config = AciWebservice.deploy_configuration(
cpu_cores=3,
memory_gb=15,
location='eastus',
description=description,
auth_enabled=True,
tags = {'name': 'ACI container',
'model_name': registered_model.name,
'model_version': registered_model.version
}
)
print("AciWebservice deployment configuration created.")
%md
### Deploy the model as a scoring webservice
Please note that it can take **10-15 minutes** for the deployment to complete.
aci_service_name='nyc-taxi-service'
service = Model.deploy(workspace=ws,
name=aci_service_name,
models=[registered_model],
inference_config=inference_config,
deployment_config= aci_config,
overwrite=True)
service.wait_for_deployment(show_output=True)
print(service.state)
%md
## Exercise 3: Consume the deployed service
%md
**Review the webservice endpoint URL and API key**
api_key, _ = service.get_keys()
print("Deployed ACI test Webservice: {} \nWebservice Uri: {} \nWebservice API Key: {}".
format(service.name, service.scoring_uri, api_key))
%md
**Prepare test data**
#['passengerCount', 'tripDistance', 'hour_of_day', 'day_of_week', 'month_num',
# 'normalizeHolidayName', 'isPaidTimeOff', 'snowDepth', 'precipTime', 'precipDepth', 'temperature']
data1 = [2, 5, 9, 4, 5, 'Memorial Day', True, 0, 0.0, 0.0, 65]
data2 = [[3, 10, 15, 4, 7, 'None', False, 0, 2.0, 1.0, 80],
[2, 5, 9, 4, 5, 'Memorial Day', True, 0, 0.0, 0.0, 65]]
print("Test data prepared.")
dataset.head()
%md
### Consume the deployed webservice over HTTP
import requests
import json
headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)}
response = requests.post(service.scoring_uri, json.dumps(data1), headers=headers)
print('Predictions for data1')
print(response.text)
print("")
response = requests.post(service.scoring_uri, json.dumps(data2), headers=headers)
print('Predictions for data2')
print(response.text)
%md
### Clean-up
When you are done with the exercise, delete the deployed webservice by running the cell below.
service.delete()
print("Deployed webservice deleted.")
I am trying to generate the XGBoost trainingreport to see feature importances however the following code only generates the profiler report.
from sagemaker import get_execution_role
import numpy as np
import pandas as pd
from sagemaker.predictor import csv_serializer
from sagemaker.debugger import Rule, rule_configs
# Define IAM role
rules=[
Rule.sagemaker(rule_configs.create_xgboost_report())
]
role = get_execution_role()
prefix = 'sagemaker/models'
my_region = boto3.session.Session().region_name
# this line automatically looks for the XGBoost image URI and builds an XGBoost container.
xgboost_container = sagemaker.image_uris.retrieve("xgboost", my_region, "latest")
bucket_name = 'binary-base'
s3 = boto3.resource('s3')
try:
if my_region == 'us-east-1':
s3.create_bucket(Bucket=bucket_name)
else:
s3.create_bucket(Bucket=bucket_name, CreateBucketConfiguration={ 'LocationConstraint': my_region })
print('S3 bucket created successfully')
except Exception as e:
print('S3 error: ',e)
boto3.Session().resource('s3').Bucket(bucket_name).Object(os.path.join(prefix, 'train/train.csv')).upload_file('../Data/Base_Model_Data_No_Labels/train.csv')
boto3.Session().resource('s3').Bucket(bucket_name).Object(os.path.join(prefix, 'validation/val.csv')).upload_file('../Data/Base_Model_Data_No_Labels/val.csv')
boto3.Session().resource('s3').Bucket(bucket_name).Object(os.path.join(prefix, 'test/test.csv')).upload_file('../Data/Base_Model_Data/test.csv'
sess = sagemaker.Session()
xgb = sagemaker.estimator.Estimator(xgboost_container,
role,
volume_size =5,
instance_count=1,
instance_type='ml.m4.xlarge',
output_path='s3://{}/{}/output'.format(bucket_name, prefix, 'xgboost_model'),
sagemaker_session=sess,
rules=rules)
xgb.set_hyperparameters(objective='binary:logistic',
num_round=100,
scale_pos_weight=8.5)
xgb.fit({'train': s3_input_train, "validation": s3_input_val}, wait=True)
When Checking the output path via:
rule_output_path = xgb.output_path + "/" + xgb.latest_training_job.job_name + "/rule-output"
! aws s3 ls {rule_output_path} --recursive
We only see the profiler report generated
What am I doing wrong/missing? I wish to generate the XGboost Training report to see its feature importances.
I have two python files. Here is my ml.py file.
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.linear_model import LinearRegression
import pickle
data = pd.read_csv("C:\\Users\\ryans\\Desktop\\insurance.csv")
le = LabelEncoder()
le.fit(data['sex'])
data['Sex'] = le.transform(data['sex'])
le.fit(data['smoker'])
data['Smoker'] = le.transform(data['smoker'])
le.fit(data['region'])
data['Region'] = le.transform(data['region'])
#independent and dependent columns
x = data[["age", "bmi", "children", "Sex", "Smoker", "Region"]]
y = data['charges']
#split in train and test
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3, random_state=0)
#model training
linreg = LinearRegression()
linreg.fit(x_train, y_train)
#model testing
predictions = linreg.predict(x_test)
linreg.score(x_test,y_test)
#save the model
file = open("C:\\Users\\ryans\\Desktop\\expense_model.pkl", 'wb')
pickle.dump(linreg, file)
Here is my run.py file.
from flask import Flask, render_template, request
import pickle
app = Flask(__name__)
model = pickle.load(open('C:\\Users\\ryans\\Desktop\\expense_model.pkl','rb')) #read mode
#app.route("/")
def home():
return render_template('index.html')
#app.route("/predict", methods=['GET','POST'])
def predict():
if request.method == 'POST':
#access the data from form
## Age
age = int(request.form["age"])
bmi = int(request.form["bmi"])
children = int(request.form["children"])
Sex = int(request.form["Sex"])
Smoker = int(request.form["Smoker"])
Region = int(request.form["Region"])
#get prediction
input_cols = [[age, bmi, children, Sex, Smoker, Region]]
prediction = model.predict(input_cols)
output = round(prediction[0], 2)
return render_template("index.html", prediction_text='Your predicted annual Healthcare Expense is $ {}'.format(output))
if __name__ == "__main__":
app.run(debug=True)
Now, when I try to run run.py in my Anaconda Prompt, I get this error message.
(base) C:\Users\ryans\Desktop\run.py
Traceback (most recent call last):
File "C:\Users\ryans\Desktop\run.py", line 5, in <module>
model = pickle.load(open('C:\\Users\\ryans\\Desktop\\expense_model.pkl','rb')) #read mode
ModuleNotFoundError: No module named 'sklearn'
Sklearn is installed and it works fine when I run it in Spyder. I think, somehow, Flask is not finding the Sklearn correctly. Or, something isn't exposed. Or, rights are not set right. I don't know what's going on with Flask. This is my first time using it. Sklearn is working fine. I know that. Any thoughts on what's wrong here?
You haven't imported sklearn in run.py, so it can't unpickle the sklearn objects in your file.
I am doing a sample pyspark ml exercise where I need to store a model and read it back. I am able to successfully save the model, but when I am trying to read/load it back it is throwing below exception. I am new to spark ml and python, please guide me on this.
Code:
from pyspark.sql import *
from pyspark.ml.feature import RFormula
from pyspark.ml.classification import LogisticRegression
from pyspark.ml import Pipeline
from pyspark.ml.tuning import ParamGridBuilder
from pyspark.ml.evaluation import BinaryClassificationEvaluator
from pyspark.ml.tuning import TrainValidationSplit
from pyspark.ml.tuning import TrainValidationSplitModel
spark = SparkSession.builder.appName("LocalMLSparkSession").master("local").getOrCreate()
df = spark.read.json("/to_data/simpleml.json").orderBy("value2")
df.select(df.color).distinct().show(10, False)
train, test = df.randomSplit([0.7, 0.3])
rForm = RFormula()
ls = LogisticRegression().setLabelCol("label").setFeaturesCol("features")
# setting pipeline
stages = [rForm,ls]
pipeline = Pipeline().setStages(stages)
#setting param grid builder
params = ParamGridBuilder()\
.addGrid(rForm.formula,["lab ~ . + color:value1", "lab ~ . + color:value1 + color:value2"])\
.addGrid(ls.elasticNetParam, [0.0, 0.5, 1.0])\
.addGrid(ls.regParam,[0.1, 0.2])\
.build()
#setting evaluator
evaluator = BinaryClassificationEvaluator()\
.setMetricName("areaUnderROC")\
.setRawPredictionCol("prediction")\
.setLabelCol("label")
#checking hyperparameters to train datasets
tvs = TrainValidationSplit()\
.setTrainRatio(0.75)\
.setEstimatorParamMaps(params)\
.setEstimator(pipeline)\
.setEvaluator(evaluator)
tvsFitted = tvs.fit(train)
evl = evaluator.evaluate(tvsFitted.transform(test))
tvsFitted.transform(test).select("features", "label", "prediction").show(10,False)
print(evl)
pip_model = tvsFitted.bestModel
pip_model.write().overwrite().save("/to_path/sparkml/model")
model = TrainValidationSplitModel().load("/to_path/sparkml/model")
model.transform(test)
Exception:
Traceback (most recent call last):
File "/home/dd/dd/python-workspace/SparkMLPipelineDemo.py", line 59, in <module>
model = TrainValidationSplitModel().load("/to_path/sparkml/model")
TypeError: __init__() missing 1 required positional argument: 'bestModel'
Process finished with exit code 1
You need to remove parenthesis when you load, e.g. replace :
model = TrainValidationSplitModel().load("/to_path/sparkml/model")
with
model = TrainValidationSplitModel.load("/to_path/sparkml/model")
I'm trying to use Tensorflow Serving to make predictions for my model.
This is the client that I'm using:
#!/usr/bin/env python2.7
"""A client that talks to tensorflow_model_server loaded with deepspeech model.
The client queries the service with the given audio and prints a ranked list
of decoded outputs to the standard output, one per line.
Typical usage example:
deepspeech_client.py --server=localhost:9000 --file audio.wav
"""
import os
import sys
local_tf = os.path.join(os.path.dirname(os.path.dirname(os.path.join(os.path.abspath(__file__)))), 'local_tf')
sys.path.append(local_tf)
import threading
from grpc.beta import implementations
import numpy as np
import tensorflow as tf
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2
sys.path.append(os.path.join(os.path.dirname(os.path.realpath(__file__)), '..'))
from util.text import ndarray_to_text
from util.audio import audiofile_to_input_vector
tf.app.flags.DEFINE_string('server', '', 'PredictionService host:port')
tf.app.flags.DEFINE_string('file', '', 'Wave audio file')
# These need to match the constants used when training the deepspeech model
tf.app.flags.DEFINE_string('n_input', 26, 'Number of MFCC features')
tf.app.flags.DEFINE_string('n_context', 9, 'Number of frames of context')
FLAGS = tf.app.flags.FLAGS
def _create_rpc_callback(event):
def _callback(result_future):
exception = result_future.exception()
if exception:
print exception
else:
results = tf.contrib.util.make_ndarray(result_future.result().outputs['outputs'])
for result in results[0]:
print ndarray_to_text(result)
event.set()
return _callback
def do_inference(hostport, audio):
host, port = hostport.split(':')
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = 'deepspeech'
request.inputs['input'].CopyFrom(tf.contrib.util.make_tensor_proto(audio))
event = threading.Event()
result_future = stub.Predict.future(request, 5.0) # 5 seconds
result_future.add_done_callback(_create_rpc_callback(event))
if event.is_set() != True:
event.wait()
def main(_):
if not FLAGS.server:
print 'please specify server host:port'
return
if not FLAGS.file:
print 'pleace specify an audio file'
return
audio_waves = audiofile_to_input_vector(
FLAGS.file, FLAGS.n_input, FLAGS.n_context)
audio = np.array([ audio_waves ])
do_inference(FLAGS.server, audio)
if __name__ == '__main__':
tf.app.run()
It's taken directly from an older commit of the Mozilla DeepSpeech implementation.
I have a trained model (from the same commit) that I'm trying to use to make predictions. I'm using the LDC193S1.wav file to try and make predictions.
When I run the model, I get back a response that is equal to
dtype: DT_INT64 tensor_shape { dim { size: 1 } dim { size: 1 } dim { size: 1 } } int64_val: 0 [[[0]]]
What can I do to fix this?