I am currently working on a project where this project requires a server to be connected to the Android project.
I am confused how to make the python script (inference engine script) that gonna be a server
here the following script that i made :
import numpy as np
import pandas as pd
#from sklearn.ensemble import RandomForestClassifier
#from sklearn.neighbors import KNeighborsClassifier
from sklearn import tree
data = pd.read_csv('datawadek.csv')
y = data.KELAS
x = data.drop('KELAS', axis = 1)
#rf = RandomForestClassifier(n_estimators=20)
#nn = KNeighborsClassifier()
cart = tree.DecisionTreeClassifier()
#rf.fit(x,y)
cart = cart.fit(x,y)
print (cart.predict([[1,1,2,3,3,1,1,2]]))
im still working on it and got stuck so if so if there are any suggestions or solutions please let me know.
by the way my project is to create an expert system application with data mining methods.
Try solution if you want a web service for the same, please refer to excellent guide from Miguel simple web service
#!flask/bin/python
from flask import Flask
app = Flask(__name__)
#app.route('/', methods=['GET','POST'])
def index(input):
# input = [[1,1,2,3,3,1,1,2]]
# do all processing here
return cart.predict(input)
if __name__ == '__main__':
app.run(debug=True)
Related
I'm building a python visualization application. I'm using 2 modules for this:
LUX
Pandas Profiling
The above 2 are auto visualization libraries that take in a csv/xlsx file & give descriptive statistics of the data which includes visualizations.
LUX has a customizable function called 'Intent' which analyzes the relationship among specific variables upon selection.
The visualizations get stored in a HTML file on my local system.
What I have tried:
This is my Python code for the visualization:
# Importing libraries
import csv
from mimetypes import init
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import os
import csv
import json
from pandas_profiling import ProfileReport
import pandas_profiling as pp
import lux
# # Pandas profiling
def pand_prof_fun(eda_file, savepath):
profile = ProfileReport(eda_file)
# Activating the widgets
profile.to_widgets()
profile.to_notebook_iframe()
# Saving & generating the report
pp.ProfileReport(eda_file)
profile.to_file(savepath)
profile = ProfileReport(eda_file, title="My Data Profile Report")
return profile
def lux_vis(eda_file, vis_file_path, vis_file_path_intent, intent_list):
eda_file.save_as_html(vis_file_path)
# Visualizing selected variables
eda_file.intent = intent_list
eda_file.save_as_html(vis_file_path_intent)
def run(csv_filepath, int_list):
csv_dataframe = pd.read_csv(csv_filepath)
os.makedirs(r'templates', exist_ok=True)
pand_prof_path = r'templates\pandas_prof.html'
lux_save_path = r'templates\lux_complete.html'
lux_intent_save_path = r'templates\lux_intent.html'
pand_prof_fun(csv_dataframe, pand_prof_path)
lux_vis(csv_dataframe, lux_save_path, lux_intent_save_path, int_list)
if __name__ == '__main__':
csv_filepath = r'C:\Users\projects\Churn_bank.csv'
I now want to build a Flask API such that the user can choose what they want to display, i.e., -->
Lux - displays the pairwise relationship between all variables. (or)
Lux using intent (Customization)- displays the relationship between specified variables. (or)
Pandas Profiling - displays the descriptive statistics of the entire data.
This is my API code for the 2nd module (Lux using intent):
from flask import Flask, render_template
import requests
from flask import Flask, request, jsonify
import json
import traceback
from visualization_function import run
from visualization_function import lux_vis
from visualization_function import pand_prof_fun
app = Flask(__name__)
#app.route('/visualization_with_us', methods = ['POST'])
def home1():
allowed_extensions = ['csv', 'xlsx']
file = request.json.get('filepath')
filename=file.split("/")[-1]
intents = request.json.get('intents')
file_extension = filename.split('.', 1)[1].lower()
try:
if file_extension in allowed_extensions:
run(file, intents)
return "Successful"
else:
return "File not supported"
except:
return "Exception in file"
if __name__ == '__main__':
app.run()
Entering input via 'Postman'
Entering the 'intents' via postman returns "successful" message & gives the right output. The HTML file gets created & saved in my "templates" folder. But when I try to include the other two modules in my API code - 'Module 1' & 'Module 3', my code doesn't work.
Here's how I've included 'Module 1' & 'Module 3' in my API code. My idea is to create a list of the modules & depending on the module chosen, the respective module/function runs -->
from flask import Flask
from flask import Flask, request
from importlib.util import module_from_spec
from flask import Flask, render_template
import requests
from flask import Flask, request, jsonify
import json
import traceback
from visualization_function import run
from visualization_function import lux_vis
from visualization_function import pand_prof_fun
app = Flask(__name__)
#app.route('/visualization_with_apoorva', methods = ['POST'])
def home1():
allowed_extensions = ['csv', 'xlsx']
available_modules=['pandas_profiling', 'lux_overall', 'lux_intent']
file = request.json.get('filepath')
filename=file.split("/")[-1]
file_extension = filename.split('.', 1)[1].lower()
intents = request.json.get('intents')
module_selection= request.json.get('module')
try:
if file_extension in allowed_extensions:
return lux_vis()
if module_selection=='lux_intent':
intents = request.json.get('intents')
run(file, intents)
if module_selection=='lux_overall':
run(file, lux_vis)
if module_selection=='pandas_profiling':
run(file, pand_prof_fun)
return "Successful"
else:
return "File not supported"
except:
return "Exception in file"
if __name__ == '__main__':
app.run()
Error: When I run the route on postman it returns "Exception in file" message.
Postman output
What is the error in my code? How do I include 'module 1' & 'module 3' in the same Flask API code such that the user can choose any one among the 3 modules & get the desired output?
I'm running this code to train a dimple model and pickle the results:
import pandas as pd
df = pd.read_csv('C:\\Users\\ryans\\Desktop\\prices.csv')
df.head()
y = df['Value']
X = df[['Rooms', 'Distance']]
from sklearn import linear_model
lm = linear_model.LinearRegression()
lm.fit(X, y)
lm.predict([[8.2,9]])
import pickle
pickle.dump(lm, open('C:\\Users\\ryans\\Desktop\\model.pkl','wb'))
Now, I open Anaconda Prompt, and run the file: C:\Users\ryans\Desktop\app.py.
Here's the app.py code:
from flask import Flask, request, render_template
import pickle
app = Flask(__name__)
model = pickle.load(open('C:\\Users\\ryans\\Desktop\\model.pkl', 'rb'))
#app.route('/')
def home():
return render_template('index.html')
#app.route('/predict',methods=['POST'])
def predict():
"""Grabs the input values and uses them to make prediction"""
rooms = int(request.form["rooms"])
distance = int(request.form["distance"])
prediction = model.predict([[rooms, distance]]) # this returns a list e.g. [127.20488798], so pick first element [0]
output = round(prediction[0], 2)
return render_template('index.html', prediction_text=f'A house with {rooms} rooms and located {distance} meters from the city center has a value of ${output}')
if __name__ == "__main__":
app.run()
Here is the traceback:
Traceback (most recent call last):
File "C:\Users\ryans\Desktop\app.py", line 2, in <module>
from flask import Flask, request, render_template
ModuleNotFoundError: No module named 'flask'
Flask is already installed, so I think it's some kind of issue like running Flask in a virtual environment, or whatever. This is my first time doing this kind of thing. How can I get Flask to run on Windows?
How did you install flask? If you're running under the conda prompt, try running conda install flask. And then see if it allows you to run your application
I am trying to set up a Flask web app using Elastic Beanstalk on AWS. I have followed the tutorial here and that works fine. I am now looking to expand the Flask webapp, and this works fine, until I import scipy.spatial as spatial, when this is part of my import statements, running eb open just times out. I receive
>>>> HTTP ERROR 504
running the webapp locally works absolutely fine even with the scipy import, it is only when I try and deploy to beanstalk that it doesn't want to work. Below is my code
import os
import requests
from bs4 import BeautifulSoup
import pandas as pd
import scipy.spatial as spatial ##### Removing this and everything works!
from flask import Flask
from flask_cors import CORS
from flask_restful import Resource, Api
from flask_jsonpify import jsonify
# print a nice greeting.
def say_hello(username = "World"):
df = pd.DataFrame({"a":[1,2,3]})
return '<p>Hello %s!</p>\n' % username
# some bits of text for the page.
header_text = '''
<html>\n<head> <title>EB Flask Test</title> </head>\n<body>'''
instructions = '''
<p><em>Hint</em>: This is a RESTful web service! Append a username
to the URL (for example: <code>/Thelonious</code>) to say hello to
someone specific.</p>\n'''
home_link = '<p>Back</p>\n'
footer_text = '</body>\n</html>'
# EB looks for an 'application' callable by default.
application = Flask(__name__)
# add a rule for the index page.
application.add_url_rule('/', 'index', (lambda: header_text +
say_hello() + instructions + footer_text))
# add a rule when the page is accessed with a name appended to the site
# URL.
application.add_url_rule('/<username>', 'hello', (lambda username:
header_text + say_hello(username) + home_link + footer_text))
# run the app.
if __name__ == "__main__":
# Setting debug to True enables debug output. This line should be
# removed before deploying a production app.
application.debug = True
application.run()
I have tried increasing the command timeout for the environment from 600 to 900, although the timeout error occurs well before 600 seconds has elapsed.
Right, I am not sure why this is the case but I updated the version of scipy in my requirements.txt and the app is working as expected!
Originally I had
scipy==1.4.1
Now I have
scipy==1.2.3
I have no idea why this has fixed the deployment issue, especially given 1.4.1 works perfectly locally. If anyone has an idea, or if this a bug I should be reporting it would be good to know!
I'm trying to write my own chatbot with the RASA framework.
Right now I'm just playing around with it and I have the following piece of code for training purposes.
from rasa.nlu.training_data import load_data
from rasa.nlu.config import RasaNLUModelConfig
from rasa.nlu.model import Trainer
from rasa.nlu import config
training_data = load_data("./data/nlu.md")
trainer = Trainer(config.load("config.yml"))
interpreter = trainer.train(training_data)
model_directory = trainer.persist("./models/nlu",fixed_model_name="current")
Now, I read that if I wanted to test it I should do something like this.
from rasa.nlu.evaluate import run_evaluation
run_evaluation("nlu.md", model_directory)
But this code is not available anymore in rasa.nlu.evaluate nor in rasa.nlu.test!
What's the way, then, of testing a RASA model?
The module was renamed.
Please import
from rasa.nlu.test import run_evaluation
Alternatively you now also do
from rasa.nlu import test
test_result = test(path_to_test_data, unpacked_model)
intent_evaluation_report = test_result["intent_evaluation"]["report"]
print(intent_evaluation_report)
We have an ETL data API repo. We do all etl processing inside of it, then spit the data out in API's. These API's are run one at a time from a single command passing the resource class through to the server to build an API. the resource class is in a web directory in an __init__.py.
This is a wonderful convention and quite simple to use, but the problem I am having is coming from trying to get one of the 3 API's available spun up for testing. Our directory stucture is like this (calling the project 'tomato')
tomato
- category_api
- web
- etl
- test
- here is where we are writing some tests (test_category_api.py)
- data
- article_api
- web
- etl
- test
- data
- recommendation_api
- web
- etl
- test
- data
- common
- common shit
Inside this test, I have the following test class. On the seventh line up from the bottom,
you will see a comment on where it breaks. It is the import_module method.
import unittest
import sys
import os
import sys
import json
from importlib import import_module
from flask import Flask
from flask_restful import Api, abort, wraps
from flask_restful.utils import cors
from flask.ext.testing import TestCase
#dir_above_top_level = os.path.join(os.path.abspath(__file__), '../../.. /')
#sys.path.append(os.path.abspath(dir_above_top_level))
_CATEGORY_ENDPOINT = '/v1/category/'
_PACKAGE_NAME = os.environ['RECO_API_PACKAGE_NAME']
_CORS = cors.crossdomain(origin='*',
headers=['Origin', 'X-Requested-With',
'Content-Type', 'Accept'],
methods=['GET', 'HEAD', 'OPTIONS'],
max_age=3600)
class CategoryTests(TestCase):
def __init__(self):
self.app = Flask(__name__)
self._configure_app()
for resource in self.resource_classes:
self.api.add_resource(self.resource,
self.resource.URI_TEMPLATE)
def test_status_code(self):
self.response = self.client.post(_CATEGORY_ENDPOINT,
data=json.dumps(
{'title': 'Enjoy this delicious food'}),
headers=json.dumps(
{'content-type':'application/json'}))
self.assertEqual(self.response.status_code, 200)
def test_version(self):
self.response = self.client.post(_CATEGORY_ENDPOINT,
data=json.dumps(
{"title": "eat some delicious stuff"}),
headers=json.dumps(
{'content-type':'application/json'}))
self.assertEqual(json.dumps(self.response['version']), '1')
def _configure_app(self):
self.app = Flask(__name__)
self.app.config['TESTING'] = True
self.app.debug = True
self.decorators = [_CORS]
self.app.Threaded = True
self.web_package = 'tomato.category.web'
self.package = import_module('.__init__', self.web_package) # WE BREAK HERE
self.resources = package.RESOURCE_NAMES
self.resource_classes = [ getattr(package, resource) for resource in resources ]
self.api = Api(self.app, catch_all_404s=True, decorators=self.decorators)
if __name__ == '__main__':
unittest.main()
we are given an exception when running these tests:
ImportError: No module named tomato.category.web.__init__
yet cd into the main top dir, and ls tomato/category/web gets us __init__.py and its right there with the resource class.
How do I import this class so that I can instantiate the API to run the tests in this class?
Or if I'm completely on the wrong track what should I be doing instead?
You don't need to import __init__, just like you probably wouldn't do from tomato.category.web import __init__. You should be able to import the web package directly.
self.web_package = 'tomato.category.web'
self.package = import_module(self.web_package)
The problem here lies in the directory structure. In the current path, I am not at the top level. It is a module. So What was needed was to uncomment the line two lines at the top, and change the structure to append the path like this.
dir_above_top_level = os.path.join(os.path.abspath(__file__), '../../../..')
sys.path.append(os.path.abspath(dir_above_top_level))
and now, I can import it using
self.web_package = 'tomato.category.web'
self.package = import_module('.__init__', self.web_package)
and now it will import fine and I can grab the resource class to set up the testing API