Develop Reference

How to pass arguments to scoring file when deploying a Model in AzureML

I am deploying a trained model to an ACI endpoint on Azure Machine Learning, using the Python SDK.
I have created my score.py file, but I would like that file to be called with an argument being passed (just like with a training file) that I can interpret using argparse.
However, I don't seem to find how I can pass arguments
This is the code I have to create the InferenceConfig environment and which obviously does not work. Should I fall back on using the extra Docker file steps or so?
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.environment import Environment
from azureml.core.model import InferenceConfig
env = Environment('my_hosted_environment')
env.python.conda_dependencies = CondaDependencies.create(
conda_packages=['scikit-learn'],
pip_packages=['azureml-defaults'])
scoring_script = 'score.py --model_name ' + model_name
inference_config = InferenceConfig(entry_script=scoring_script, environment=env)
Adding the score.py for reference on how I'd love to use the arguments in that script:
#removed imports
import argparse
def init():
global model
parser = argparse.ArgumentParser(description="Load sklearn model")
parser.add_argument('--model_name', dest="model_name", required=True)
args, _ = parser.parse_known_args()
model_path = Model.get_model_path(model_name=args.model_name)
model = joblib.load(model_path)
def run(raw_data):
try:
data = json.loads(raw_data)['data']
data = np.array(data)
result = model.predict(data)
return result.tolist()
except Exception as e:
result = str(e)
return result
Interested to hear your thoughts

This question is a year old. Providing a solution to help those who may still be looking for an answer. My answer to a similar question is here. You may pass native python datatype variables into the inference config and access them as environment variables within the scoring script.

I tackled this problem differently. I could not find a (proper and easy to follow) way to pass arguments for score.py, when it is consumed by InferenceConfig . Instead, what I did was following 4 steps:
Created score_template.py and define variables which should be assigned
Read content of score_template.py and modify it by replacing variables with desired values
Write modified contents into score.py
Finally pass score.py to InferenceConfig
STEP 1 in score_template.py:
import json
from azureml.core.model import Model
import os
import joblib
import pandas as pd
import numpy as np
def init():
global model
#model = joblib.load('recommender.pkl')
model_name="#MODEL_NAME#"
model_saved_file='#MODEL_SAVED_FILE#'
try:
model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), model_saved_file)
model = joblib.load(model_path)
except:
model_path = Model.get_model_path(model_name)
model = joblib.load(model_path)
def run(raw_data):
try:
#data=pd.json_normalize(data)
#data=np.array(data['data'])
data = json.loads(raw_data)["data"]
data = np.array(data)
result = model.predict(data)
# you can return any datatype as long as it is JSON-serializable
return {"result": result.tolist()}
except Exception as e:
error = str(e)
#error= data
return error
STEP 2-4 in deploy_model.py:
#--Modify Entry Script/Pass Model Name--
entry_script="score.py"
entry_script_temp="score_template.py"
# Read in the entry script template
print("Prepare Entry Script")
with open(entry_script_temp, 'r') as file :
entry_script_contents = file.read()
# Replace the target string
entry_script_contents = entry_script_contents.replace('#MODEL_NAME#', model_name)
entry_script_contents = entry_script_contents.replace('#MODEL_SAVED_FILE#', model_file_name)
# Write the file to entry script
with open(entry_script, 'w') as file:
file.write(entry_script_contents)
#--Define configs for the deployment---
print("Get Environtment")
env = Environment.get(workspace=ws, name=env_name)
env.inferencing_stack_version = "latest"
print("Inference Configuration")
inference_config = InferenceConfig(entry_script=entry_script, environment=env, source_directory=base_path)
aci_config = AciWebservice.deploy_configuration(cpu_cores = int(cpu_cores), memory_gb = int(memory_gb),location=location)
#--Deloy the service---
print("Deploy Model")
print("model version:", model_artifact.version)
service = Model.deploy( workspace=ws,
name=service_name,
models=[model_artifact],
inference_config=inference_config,
deployment_config=aci_config,
overwrite=True )
service.wait_for_deployment(show_output=True)

How to deploy using environments can be found here model-register-and-deploy.ipynb . InferenceConfig class accepts source_directory and entry_script parameters, where source_directory is a path to the folder that contains all files(score.py and any other additional files) to create the image.
This multi-model-register-and-deploy.ipynb has code snippets on how to create InferenceConfig with source_directory and entry_script.
from azureml.core.webservice import Webservice
from azureml.core.model import InferenceConfig
from azureml.core.environment import Environment
myenv = Environment.from_conda_specification(name="myenv", file_path="myenv.yml")
inference_config = InferenceConfig(entry_script="score.py", environment=myenv)
service = Model.deploy(workspace=ws,
name='sklearn-mnist-svc',
models=[model],
inference_config=inference_config,
deployment_config=aciconfig)
service.wait_for_deployment(show_output=True)
print(service.scoring_uri)

No suggestions for triggers when using Pytransitions

Trying to use transitions package as per examples provided here https://github.com/pytransitions/transitions
For some reason, neither of the two approaches shown below provide typing suggestions for registered evaporate() trigger (at least in PyCharm 2019.1.2 for Windows x64)
At the same time, these triggers can still be used.
What can be done to have these triggers suggested as I type?
class Matter(Machine):
def say_hello(self): print("hello, new state!")
def say_goodbye(self): print("goodbye, old state!")
def __init__(self):
states = ['solid', 'liquid', 'gas']
Machine.__init__(self, states=states, initial='liquid')
self.add_transition('melt', 'solid', 'liquid')
testmatter= Matter()
testmatter.add_transition('evaporate', 'liquid', 'gas')
testmatter.evaporate()
Out: True
testmatter.get_model_state(testmatter)
Out: <State('gas')#14748976>
class Matter2():
pass
testmatter2 = Matter2()
machine = Machine(model=testmatter2, states=['solid', 'liquid', 'gas', 'plasma'], initial='liquid')
machine.add_transition('evaporate', 'liquid', 'gas')
testmatter2.evaporate()
Out: True

transitions adds triggers at runtime to the model (Matter) instance. This cannot be predicted by IDEs before the initialization code is actually executed. Imho, this is the biggest disadvantage of the way in which transitions works (but again imho, it is also its strength when dealing with dynamic state machines or state machines created/received during runtime but that's another story)
If you use an interactive shell with code completion (ipython), you will see that evaporate (based on __dir__ calls to the model)
will be suggested:
from transitions import Machine
class Model:
pass
model = Model()
>>> model.e # TAB -> nothing
# model will be decorated during machine initialization
machine = Machine(model, states=['A', 'B'],
transitions=[['evaporate', 'A', 'B']], initial='A')
>>> model.e # TAB -> completion!
But I assume that's not how you plan to code. So how can we give hints to the introspection?
Easiest solution: Use a docstring for your model to announce triggers.
from transitions import Machine
class Model:
"""My dynamically extended Model
Attributes:
evaporate(callable): dynamically added method
"""
model = Model()
# [1]
machine = Machine(model, states=['A', 'B'],
transitions=[['evaporate', 'A', 'B']], initial='A')
model.eva # code completion! will also suggest 'evaporate' before it was added at [1]
The problem here is that the IDE will rely on the docstring to be correct. So when the docstring method (masked as attribute) is calles evaparate, it will always suggests that even though you later add evaporate.
Use pyi files and PEP484 (PyCharm workaround)
Unfortunately, PyCharm does not consider attributes in docstrings for code completion as you correctly pointed out (see this discussion for more details). We need to use another approach. We can create so called pyi files to provide hints to PyCharm. Those files are named identically to their .py counterparts but are solely used for IDEs and other tools and must not be imported (see this post). Let's create a file called sandbox.pyi
# sandbox.pyi
class Model:
evaporate = None # type: callable
And now let's create the actual code file sandbox.py (I don't name my playground files 'test' because that always startles pytest...)
# sandbox.py
from transitions import Machine
class Model:
pass
## Having the type hints right here would enable code completion BUT
## would prevent transitions to decorate the model as it does not override
## already defined model attributes and methods.
# class Model:
# evaporate = None # type: callable
model = Model()
# machine initialization
model.ev # code completion
This way you have code completion AND transitions will correctly decorate the model. The downside is that you have another file to worry about which might clutter your project.
If you want to generate pyi files automatically, you could have a look at stubgen or extend Machine to generate event stubs of models for you.
from transitions import Machine
class Model:
pass
class PyiMachine(Machine):
def generate_pyi(self, filename):
with open(f'{filename}.pyi', 'w') as f:
for model in self.models:
f.write(f'class {model.__class__.__name__}:\n')
for event in self.events:
f.write(f' def {event}(self, *args, **kwargs) -> bool: pass\n')
f.write('\n\n')
model = Model()
machine = PyiMachine(model, states=['A', 'B'],
transitions=[['evaporate', 'A', 'B']], initial='A')
machine.generate_pyi('sandbox')
# PyCharm can now correctly infer the type of success
success = model.evaporate()
model.to_A() # A dynamically added method which is now visible thanks to the pyi file
Alternative: Generate machine configurations FROM docstrings
A similar issue has been already discussed in the issue tracker of transitions (see https://github.com/pytransitions/transitions/issues/383). You could also generate the machine configuration from the model's docstring:
import transitions
import inspect
import re
class DocMachine(transitions.Machine):
"""Parses states and transitions from model definitions"""
# checks for 'attribute:value' pairs (including [arrays]) in docstrings
re_pattern = re.compile(r"(\w+):\s*\[?([^\]\n]+)\]?")
def __init__(self, model, *args, **kwargs):
conf = {k: v for k, v in self.re_pattern.findall(model.__doc__, re.MULTILINE)}
if 'states' not in kwargs:
kwargs['states'] = [x.strip() for x in conf.get('states', []).split(',')]
if 'initial' not in kwargs and 'initial' in conf:
kwargs['initial'] = conf['initial'].strip()
super(DocMachine, self).__init__(model, *args, **kwargs)
for name, method in inspect.getmembers(model, predicate=inspect.ismethod):
doc = method.__doc__ if method.__doc__ else ""
conf = {k: v for k, v in self.re_pattern.findall(doc, re.MULTILINE)}
# if docstring contains "source:" we assume it is a trigger definition
if "source" not in conf:
continue
else:
conf['source'] = [s.strip() for s in conf['source'].split(', ')]
conf['source'] = conf['source'][0] if len(conf['source']) == 1 else conf['source']
if "dest" not in conf:
conf['dest'] = None
else:
conf['dest'] = conf['dest'].strip()
self.add_transition(trigger=name, **conf)
# override safeguard which usually prevents accidental overrides
def _checked_assignment(self, model, name, func):
setattr(model, name, func)
class Model:
"""A state machine model
states: [A, B]
initial: A
"""
def go(self):
"""processes information
source: A
dest: B
conditions: always_true
"""
def cycle(self):
"""an internal transition which will not exit the current state
source: *
"""
def always_true(self):
"""returns True... always"""
return True
def on_exit_B(self): # no docstring
raise RuntimeError("We left B. This should not happen!")
m = Model()
machine = DocMachine(m)
assert m.is_A()
m.go()
assert m.is_B()
m.cycle()
try:
m.go() # this will raise a MachineError since go is not defined for state B
assert False
except transitions.MachineError:
pass
This is a very simple docstring-to-machine-configration parser which does not take care of all eventualities that could be part of a docstring. It assumes that every method with a docstring containing ("source: " ) is supposed to be a trigger. It does however also approaches the issue of documentation. Using such a machine would make sure that at least some documentation for the developed machine exists.

How to pickle a class instance if the class has another instance of different class?

So I am trying to build a chatbot using the famous "chatterbot" library in python. I made a class called Trainer which trains my chatbot. So in this class I am initializing the instance for the class "chatterbot" and then training it. So, to avoid retraining it again and again I though to pickle the instance of my Trainer class. So if it already exist, I don't need to retrain it and hence I am trying to pickle the Trainer class instance. I am using dill library to pickle the class instance but as it tries to pickle my model, it shows me the following error:
_pickle.PicklingError: Can't pickle >'sqlalchemy.orm.session.Session'>: it's not the same object as >>sqlalchemy.orm.session.Session
Now, I don't see anywhere in my code that I have created any kind of session. But I believe the chatterbot library that I am using in my Trainer class must be using any kind of session in it. Infact I checked the source code and it is using the logger. So it might be possible that's causing the pain. I have no clue how to proceed with this problem. I tried to change the source code of chatterbot library and removed every occurrence of logger from it but I did nothing but broke the code. Can anyone help me how to fix this issue. I am attaching required code here.
utils:
import logging
from pathlib import Path
import pickle
import dill
import os
from .trainer import Trainer
# Method returns the directories in which model objects are stored/saved.
def model_base_dir():
directory = 'MLModel/pickled_models'
parent_directory = os.pardir
return os.path.abspath(os.path.join(parent_directory,directory))
def picked_model(base_dir=None):
if base_dir == None:
logging.exception("Base Directory does not exist !!!")
raise AssertionError("Base Directory does not exist !!!")
model_path = base_dir+'/version1.pickle'
if Path(model_path).is_file():
with open(model_path,'rb') as handle:
model = dill.load(handle)
return model
else:
return None
def save_model(model_obj):
dir = model_base_dir() # Get the directory where model needs to be saved
with open(dir+'/version1.pickle','wb') as f:
dill.dump(model_obj,f)
f.close()
def train_model():
mod_obj = Trainer()
save_model(mod_obj)
return mod_obj
Trainer
from chatterbot import ChatBot
from chatterbot.trainers import ChatterBotCorpusTrainer
class Trainer():
def __init__(self):
self.chatbot = ChatBot('Dexter')
self.create_chatbot_trainer(language="chatterbot.corpus.english")
def train_chatbot(self,trainer,language):
return trainer.train(language)
def create_chatbot_trainer(self,language):
self.trainer = ChatterBotCorpusTrainer(self.chatbot)
self.trainer = self.train_chatbot(self.trainer,language)
return self.trainer
def __getstate__(self):
d = self.__dict__.copy()
d.pop('_parents',None)
return d
def response(self,text=""):
if text is None:
return "Sorry, your query is empty"
else:
return self.chatbot.get_response(text)
The train_model() gets trigger from my django view.
Any help appreciated.

Why is Django adding a typo when saving a file?

I am writing some test code for a project I am making with Django. The test code is such:
from django.test import TestCase, override_settings
from django.core.files import File
import sys
import os
from .models import Manual
#override_settings(MEDIA_ROOT=os.getcwd()+'/temp/django_test')
# Create your tests here.
class ManualModelTests(TestCase):
def tearDown(self):
try:
os.remove('/temp/django_test/test_image.jpg')
except FileNotFoundError:
pass
def test_normal_manual_upload(self):
in_image = open(os.path.join(os.path.dirname(__file__), 'test_image.jpg'), 'r+b')
in_file = open(os.path.join(os.path.dirname(__file__), 'test_manual.pdf'), 'r+b')
thumbnail = File(in_image)
in_manual = File(in_file)
new_manual = Manual.objects.create(
photo=thumbnail,
manual=in_manual,
make='yoshimura',
model='001',
year_min=2007,
year_max=2010
)
self.assertTrue(os.path.exists('/temp/django_test/photos/test_image.jpg'))
self.assertTrue(os.path.exists
('/temp/django_test/manuals/test_manual.pdf'))
self.tearDown()
The model code is such:
class Manual(models.Model):
photo = models.ImageField(upload_to="photos")
make = models.CharField(max_length=50)
model = models.CharField(max_length=100)
manual = models.FileField(upload_to="manuals")
year_min = models.PositiveIntegerField(default=0)
year_max = models.PositiveIntegerField(default=0)
The test code aims to create a Manual object (just testing to see if the model saves normally). Upon running the code however, after finishing the model constructor call, I get a crazy long error that comes after many calls of python's makdirs() function ends with
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Users\\Jason\\Projects\\motomanuals\\temp\\django_test\\photos\\C:'
A full pastebin of the error can be found here
My guess is the problems stems from the path ending in '\\C:'?
My questions are:
Is my guess right. If not, then why am I getting this error?
Why is Django trying to add '\\C:\' to the file path and how do I fix this?

how to unittest and mock for open funtion

I have read many article over the last 6 hours and i still don't understand mocking and unit-testing. I want to unit test a open function, how can i do this correctly?
i am also concerned as the bulk of my code is using external files for data import and manipulation. I understand that i need to mock them for testing, but I am struggling to understand how to move forward.
Some advice please. Thank you in advance
prototype5.py
import os
import sys
import io
import pandas
pandas.set_option('display.width', None)
def openSetupConfig (a):
"""
SUMMARY
Read setup file
setup file will ONLY hold the file path of the working directory
:param a: str
:return: contents of the file stored as str
"""
try:
setupConfig = open(a, "r")
return setupConfig.read()
except Exception as ve:
ve = (str(ve) + "\n\nPlease ensure setup file " + str(a) + " is available")
sys.exit(ve)
dirPath = openSetupConfig("Setup.dat")
test_prototype5.py
import prototype5
import unittest
class TEST_openSetupConfig (unittest.TestCase):
"""
Test the openSetupConfig function from the prototype 5 library
"""
def test_open_correct_file(self):
result = prototype5.openSetupConfig("Setup.dat")
self.assertTrue(result)
if __name__ == '__main__':
unittest.main()

So the rule of thumb is to mock, stub or fake all external dependencies to the method/function under test. The point is to test the logic in isolation. So in your case you want to test that it can open a file or log an error message if it can't be opened.
import unittest
from mock import patch
from prototype5 import openSetupConfig # you don't want to run the whole file
import __builtin__ # needed to mock open
def test_openSetupConfig_with_valid_file(self):
"""
It should return file contents when passed a valid file.
"""
expect = 'fake_contents'
with patch('__builtin__.open', return_value=expect) as mock_open:
actual = openSetupConfig("Setup.dat")
self.assertEqual(expect, actual)
mock_open.assert_called()
#patch('prototype5.sys.exit')
def test_openSetupConfig_with_invalid_file(self, mock_exit):
"""
It should log an error and exit when passed an invalid file.
"""
with patch('__builtin__.open', side_effect=FileNotFoundError) as mock_open:
openSetupConfig('foo')
mock_exit.assert_called()