I have a classifier working with XGBoost in sagemaker, but despite the training set only having 1s and 0s in the first column (csv file, first column is assumed to be target in sagemaker xgboost), the algorithm returns a decimal.
First 3 records return 1.08, 0.34, and 0.91. I'd assume probabilities but 1.08? If these are rounded to 0 or 1 then they're all correct, but why is it returning non-class values?
Furthermore, the class only contains a predict method - is a predict probability method not possible without using your own model?
The code calling this is:
from flask import Flask
from flask import request
import boto3
from sagemaker.predictor import csv_serializer
import sagemaker
app = Flask(__name__)
#app.route("/")
def hello():
numbers = request.args.get('numbers')
#session
boto_session = boto3.Session(profile_name="profilename",
region_name='regionname')
#sagemaker session
sagemaker_session = sagemaker.Session(boto_session=boto_session)
#endpoint
predictor = sagemaker.predictor.RealTimePredictor(endpoint="modelname",
sagemaker_session=sagemaker_session)
predictor.content_type="text/csv"
predictor.serializer=csv_serializer
predictor.deserializer=None
#result
result=predictor.predict(numbers)
result=result.decode("utf-8")
return f'Output: {result}'
if __name__ == "__main__":
app.run(debug=True, port=5000)
The flask section works fine, I can retrieve predictions at 127.0.0.1:5000.
Sagemaker Version 1.3.0. Version 1.9.0 does not work - it requires fcntrl which is mac/linux only - see this on their repo, apparently it's fixed on pypi but I've tried and the version doesn't change or fix the issue, so I'm stuck on 1.3.0 until they resolve it. Version 1.3.0 does not have a predict_proba method.
Related
This question already has answers here:
How to run a flask application?
(6 answers)
Closed yesterday.
This post was edited and submitted for review yesterday.
I'm trying to follow this online tutorial for a Flask/Tensorflow/React application, but I'm having some trouble at the end now trying to run the Flask server.
Flask version: 2.2.3
Python version: 3.10.0
I've search online for solutions, but nothing I've tried has worked. Here's the ways I've tried to run the application:
Not sure if this could be helpful in coming to a solution, but incase it is, here's my app.py file:
import os
import numpy as np
from flask import Flask, request
from flask_cors import CORS
from keras.models import load_model
from PIL import Image, ImageOps
app = Flask(__name__) # new
CORS(app) # new
#app.route('/upload', methods=['POST'])
def upload():
# Disable scientific notation for clarity
np.set_printoptions(suppress=True)
# Load the model
model = load_model("keras_Model.h5", compile=False)
# Load the labels
class_names = open("labels.txt", "r").readlines()
# Create the array of the right shape to feed into the keras model
# The 'length' or number of images you can put into the array is
# determined by the first position in the shape tuple, in this case 1
data = np.ndarray(shape=(1, 224, 224, 3), dtype=np.float32)
# Replace this with the path to your image
image = Image.open("<IMAGE_PATH>").convert("RGB")
# resizing the image to be at least 224x224 and then cropping from the center
size = (224, 224)
image = ImageOps.fit(image, size, Image.Resampling.LANCZOS)
# turn the image into a numpy array
image_array = np.asarray(image)
# Normalize the image
normalized_image_array = (image_array.astype(np.float32) / 127.5) - 1
# Load the image into the array
data[0] = normalized_image_array
# Predicts the model
prediction = model.predict(data)
index = np.argmax(prediction)
class_name = class_names[index]
confidence_score = prediction[0][index]
# Print prediction and confidence score
print("Class:", class_name[2:], end="")
print("Confidence Score:", confidence_score)
Does anyone know what I'm doing wrong here, is there maybe something obvious I'm missing that's causing the problem? If there's any other info I can add that may be helpful, please let me know, thanks.
Edit:
I've added the execution section at the end of my code:
if __name__ == '__main__':
app.run(host="127.0.0.1", port=5000)
And now 'python -m flask run' does attempt to run the app, so the original post question is answered. There does seem to be a subsequent problem now though, it constantly returns this error. I installed tensorflow using 'pip3 install tensorflow' and it does install successfully, but then it always returns with the module not found error. Tensorflow doesn't appear in pip freeze package list, am now looking to see how/why this is.
Edit2: My question was flagged as already answered here, though I'm struggling to see how, as that post has absolutely no mention at all on why 'flask run' or any way of trying to run a flask app might not work, or what to do when it doesn't, which is what this question is. That post is simply discussing the 'correct' way to run a flask app, not how to run one at all when it's not running.
If you don't want to use the flask command, you need to add code to run the development server instead.
if __name__ == "__main__":
app.run()
You're not using the flask command, but you forgot to add code to run the development server instead.
if __name__ == '__main__':
app.run()
I'm new here and beginner Python user so go easy on me.
I used to successfully forecast GHI and ambient temperature using pvlib-python forecasting module with GFS model using get_data() method for more than two years without problems but a week ago (after updating to 0.8.1 version) I'm encountering the following error as an answer from UNIDATA THREDDS server when querying data:
HTTPError: Error accessing https://thredds.ucar.edu/thredds/ncss/grib/NCEP/GFS/Global_0p5deg/Best?var=u-component_of_wind_isobaric&var=Total_cloud_cover_low_cloud_Mixed_intervals_Average&var=Total_cloud_cover_boundary_layer_cloud_Mixed_intervals_Average&var=Total_cloud_cover_middle_cloud_Mixed_intervals_Average&var=Wind_speed_gust_surface&var=Temperature_surface&var=Total_cloud_cover_entire_atmosphere_Mixed_intervals_Average&var=Total_cloud_cover_convective_cloud&var=Total_cloud_cover_high_cloud_Mixed_intervals_Average&var=Downward_Short-Wave_Radiation_Flux_surface_Mixed_intervals_Average&var=v-component_of_wind_isobaric&time_start=2021-04-27T23%3A00%3A00%2B02%3A00&time_end=2021-04-30T02%3A00%3A00%2B02%3A00&longitude=15.966568&latitude=45.815399&vertCoord=100000&accept=netcdf
Server Error (400: Variable: Total_cloud_cover_low_cloud_Mixed_intervals_Average is not contained in the requested dataset)
Apparently, variable Total_cloud_cover_low_cloud_Mixed_intervals_Average is not available when sending request to UNIDATA THREDDS server. I searched for the problem on the Internet but found nothing similar. I also found in the pvlib source code for forecast module, class GFS that variable Total_cloud_cover_low_cloud_Mixed_intervals_Average is renamed to low_clouds (https://pvlib-python-forecasting.readthedocs.io/en/latest/_modules/pvlib/forecast.html).
Here is the code I'm using to produce the error:
import datetime
from pvlib.forecast import GFS
import pytz
loc_latitude, loc_longitude, tzone = 45.815399, 15.966568, 'Europe/Zagreb'
today_date = datetime.date.today()
naive_time = datetime.time(0, 0, 0)
naive_datetime = datetime.datetime.combine(today_date, naive_time)
timezone = pytz.timezone('Europe/Zagreb')
#timezone aware datetime
aware_datetime = timezone.localize(naive_datetime)
GFS_start = aware_datetime
GFS_end = aware_datetime + datetime.timedelta(days = 2)
GFS_forecast_model = GFS()
GFS_raw_data = GFS_forecast_model.get_data(latitude = loc_latitude, longitude = loc_longitude, start = GFS_start, end = GFS_end)
I also tried to use NAM model instead of GFS and everything works fine but I can't use it because only GFS forecasts globally (I need forecasts for Europe).
I'm using anaconda with Python 3.6.6. and pvlib-python 0.8.1 is installed using conda. I'm available for any other information needed. I would appreciate any help.
After doing some further research I found that this problem has been already reported and solved - github. GFS changed some cloud cover variables. This will be implemented in new version of pvlib.
I have a few machine learning models running via TensorFlow Serving on Kubernetes. I'd like to be able to have one deployment of a particular model, and then load multiple versions.
This seems like it would be easier than having to maintain a separate Kubernetes deployment for each version of each model that we have.
But it's not obvious how to pass the version or model flavor I want to call using the Python gRPC interface to TF Serving. How do I specify the version and pass it in?
For whatever reason, it's not possible to update the model spec in place as you're building the request to pull. Instead, you need to separately build an instance of ModelSpec that includes the version you want, and then pass that in to the constructor for the prediction request.
Also worth pointing out you need to use the Google-specific Int64Value for the version.
from google.protobuf.wrappers_pb2 import Int64Value
from tensorflow_serving.apis.model_pb2 import ModelSpec
from tensorflow_serving.apis import predict_pb2, get_model_metadata_pb2, \
prediction_service_pb2_grpc
from tensorflow import make_tensor_proto
import grpc
model_name = 'mymodel'
input_name = 'model_input'
model_uri = 'mymodel.svc.cluster.local:8500'
X = # something that works
channel = grpc.insecure_channel(model_uri, options=MESSAGE_OPTIONS)
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
version = Int64Value(value=1)
model_spec = ModelSpec(version=version, name=model_name, signature_name='serving_default')
request = predict_pb2.PredictRequest(model_spec=model_spec)
request.inputs[input_name].CopyFrom(make_tensor_proto(X.astype(np.float32), shape=X.shape))
result = stub.Predict(request, 1.0)
channel.close()
I am trying to use the beta Google Custom Prediction Routine in Google's AI Platform to run a live version of my model.
I include in my package predictor.py which contains a Predictor class as such:
import os
import numpy as np
import pickle
import keras
from keras.models import load_model
class Predictor(object):
"""Interface for constructing custom predictors."""
def __init__(self, model, preprocessor):
self._model = model
self._preprocessor = preprocessor
def predict(self, instances, **kwargs):
"""Performs custom prediction.
Instances are the decoded values from the request. They have already
been deserialized from JSON.
Args:
instances: A list of prediction input instances.
**kwargs: A dictionary of keyword args provided as additional
fields on the predict request body.
Returns:
A list of outputs containing the prediction results. This list must
be JSON serializable.
"""
# pre-processing
preprocessed_inputs = self._preprocessor.preprocess(instances[0])
# predict
outputs = self._model.predict(preprocessed_inputs)
# post-processing
outputs = np.array([np.fliplr(x) for x in x_test])
return outputs.tolist()
#classmethod
def from_path(cls, model_dir):
"""Creates an instance of Predictor using the given path.
Loading of the predictor should be done in this method.
Args:
model_dir: The local directory that contains the exported model
file along with any additional files uploaded when creating the
version resource.
Returns:
An instance implementing this Predictor class.
"""
model_path = os.path.join(model_dir, 'keras.model')
model = load_model(model_path, compile=False)
preprocessor_path = os.path.join(model_dir, 'preprocess.pkl')
with open(preprocessor_path, 'rb') as f:
preprocessor = pickle.load(f)
return cls(model, preprocessor)
The full error Create Version failed. Bad model detected with error: "Failed to load model: Unexpected error when loading the model: 'str' object has no attribute 'decode' (Error code: 0)" indicates that the issue is in this script, specifically when loading the model. However, I am able to successfully load the model in my notebook locally with the same code block in predict.py:
from keras.models import load_model
model = load_model('keras.model', compile=False)
I have seen similar posts which suggest to set the version of h5py<3.0.0 but this hasn't helped. I can set versions of modules for my custom prediction routine as such in a setup.py file:
from setuptools import setup
REQUIRED_PACKAGES = ['keras==2.3.1', 'h5py==2.10.0', 'opencv-python', 'pydicom', 'scikit-image']
setup(
name='my_custom_code',
install_requires=REQUIRED_PACKAGES,
include_package_data=True,
version='0.23',
scripts=['predictor.py', 'preprocess.py'])
Unfortunately, I haven't found a good way to debug model deployment in google's AI Platform and the troubleshooting guide is unhelpful. Any pointers would be much appreciated. Thanks!
Edit 1:
The h5py module's version is wrong –– at 3.1.0, despite setting it to 2.10.0 in setup.py. Anyone know why? I confirmed that Keras version and other modules are set properly however. I've tried 'h5py==2.9.0' and 'h5py<3.0.0' to no avail. More on including PyPi package dependencies here.
Edit 2:
So it turns out google currently does not support this capability.
StackOverflow, enzed01
I have encountered the same problem with using AI platform with code that was running fine two months ago, when we last trained our models. Indeed, it is due to the dependency on h5py which fails to load the h5 model out of the blue.
After a while I was able to make it work with runtime 2.2 and python version 3.7. I am also using the custom prediction routine and my model was a simple 2-layer bidirectional LSTM serving classifications.
I had a notebook VM set up with TF == 2.1 and downgraded h5py to <3.0.0 with:
!pip uninstall -y h5py
!pip install 'h5py < 3.0.0'
My setup.py looks like this:
from setuptools import setup
REQUIRED_PACKAGES = ['tensorflow==2.1', 'h5py<3.0.0']
setup(
name="my_package",
version="0.1",
include_package_data=True,
scripts=["preprocess.py", "model_prediction.py"]
)
I added compile=False to my model load code. Without it, I ran into another problem with deployment which was giving following error: Create Version failed. Bad model detected with error: "Failed to load model: Unexpected error when loading the model: 'sample_weight_mode' (Error code: 0)"
The code change from OP:
model = keras.models.load_model(
os.path.join(model_dir,'model.h5'), compile = False)
And this made the model be deployed as before without a problem. I suspect the
compile=False might mean slower prediction serving, but have not noticed anything so far.
Hope this helps anyone stuck and googling these issues!
I am using the RasaNLUHttpInterpreter as stated here to start my server. I give the class all the 4 parameters required (model-name, token, server-name, and project-name). However, I always get the error, that apparently I am handing over 5 arguments (what I don't really do).
The error occurred since I updated my Rasa-Core and NLU to the latest version. However, as in docs, I feel that I use the method correctly. Does anyone have an idea what I am doing wrong or what's happening here?
Here is my run-server.py where I use the RasaNLUHttpInterpreter:
import os
from os import environ as env
from gevent.pywsgi import WSGIServer
from server import create_app
from rasa_core import utils
from rasa_core.interpreter import RasaNLUHttpInterpreter
utils.configure_colored_logging("DEBUG")
user_input_dir = "/app/nlu/" + env["RASA_NLU_PROJECT_NAME"] + "/user_input"
if not os.path.exists(user_input_dir):
os.makedirs(user_input_dir)
nlu_interpreter = RasaNLUHttpInterpreter(
'model_20190702-103405', None, 'http://rasa-nlu:5000', 'test_project')
app = create_app(
model_directory = env["RASA_CORE_MODEL_PATH"],
cors_origins="*",
loglevel = "DEBUG",
logfile = "./logs/rasa_core.log",
interpreter = nlu_interpreter)
http_server = WSGIServer(('0.0.0.0', 5005), app)
http_server.serve_forever()
I am using:
rasa_nlu~=0.15.1
rasa_core==0.14.5
as already mentioned here I have analyzed the problem in detail.
First of all, the method calls and the given link belong to a rasa version that is deprecated. After updating to the latest rasa version which splits up core and nlu, the project was refactored to fit the according documentation.
After rebuilding the bot with the exact same setup, no errors were thrown and the bot worked as expected.
We came to the conclusion that this must have been a particular problem on threxx's workstation.
If someone else might come to this point, he is welcome to post here such that we could help him.