I have a flask application with a single endpoint, like this (simplified below):
#app.route('/classify', methods=['POST'])
def classify():
p = g.model_loader.get_model()
json = request.get_json()
text = json['text']
return p.classify(text)
def main():
model_loader = ResourceLoader()
with app.app_context():
g.model_loader = model_loader
app.run()
if __name__ == '__main__':
main()
The application needs to load a machine learning model into memory once, in the main method, and then use that model to classify text that is being sent via POST to a flask endpoint. I've tried to do this using an application context, which works locally, but doesn't work on heroku. My understanding currently is that this is because the application context isn't shared across workers. How can I give the classify endpoint access to the model loader?
EDIT: I didn't word the initial question correctly. Each worker should run main() and should thus have access to model_loader in its own application context. However, when I run this on heroku, I get AttributeError: '_AppCtxGlobals' object has no attribute 'model_loader'. Does the application context differ on heroku?
You can't load it into memory once, because Heroku workers might be on completely different machines. You need to run this code in every worker, or store the data somewhere it can be read by any process.
Related
hi i have made a really basic example here to attempt to make this easy to understand, i have a simple flask api that returns a single string, it is fully accessible using localhost, but i want to be able to access it from outside of the local network. i have created a firewall rule that allows TCP traffic in and out on port 5000, but despite this, it does not work. this is currently running in a pycharm community edition IDE, but i have ran it from command line aswell with the same results.
Why can i not access it using http://[IP]:5000/test
my end goal is to be able to access it from any identity given using Tor service using Torrequests module, but to get that far i need to be able to access it externally in the first place
from flask import Flask, request
from flask_restful import Resource, Api
import logging
app = Flask(__name__)
api = Api(app)
class Test(Resource):
def post(self):
return "worked"
def get(self):
return "worked"
api.add_resource(Test, '/test', methods=['GET', 'POST'])
if __name__ == "__main__":
app.run(debug=False ,host='0.0.0.0', port=5000)
Setup :
Language i am using is Python
I am running a flask app with threaded =True and inside the flask app when an endpoint is hit, it starts a thread and returns a thread started successfully with status code 200.
Inside the thread, there is re.Search happening, which takes 30-40 seconds(worst case) and once the thread is completed it hits a callback URL completing one request.
Ideally, the Flask app is able to handle concurrent requests
Issue:
When the re.search is happening inside the thread, the flask app is not accepting concurrent requests. I am assuming some kind of thread locking is happening and unable to figure out.
Question :
is it ok to do threading(using multi processing) inside flask app which has "threaded = True"?
When regex is happening does it do any thread locking?
Code snippet: hit_callback does a post request to another api which is not needed for this issue.
import threading
from flask import *
import re
app = Flask(__name__)
#app.route(/text)
def temp():
text = request.json["text"]
def extract_company(text)
attrib_list = ["llc","ltd"] # realone is 700 in length
entity_attrib = r"((\s|^)" + r"(.)?(\s|$)|(\s|^)".join(attrib_list) + r"(\s|$))"
raw_client = re.search("(.*(?:" + entity_attrib + "))", text, re.I)
hit_callback(raw_client)
extract_thread = threading.Thread(target=extract_company, )
extract_thread.start()
return jsonify({"Response": True}), 200
if __name__ == "__main__":
app.run(debug=True, host='0.0.0.0', port=4557, threaded=True)
Please read up on the GIL - basically, Python can only execute ONE piece of code at the same time - even if in threads. So your re-search runs and blocks all other threads until run to completion.
Solutions: Use Gunicorn etc. pp to run multiple flask processes - do not attempt to do everything in one flask process.
To add something: Also your design is problematic - 40 seconds or so for a http answer is way to long. A better design would have at least two services: a web server and a search service. The first would register a search and give back an id. Your search service would communicate asynchronously with the web service and return resullt +id when the result is ready. Your clients could pull the web service until they get a result.
I'm trying to trigger a python module (market order for Oanda) using web hooks(from trading view).
Similar to this
1) https://www.youtube.com/watch?v=88kRDKvAWMY&feature=youtu.be
and this
2)https://github.com/Robswc/tradingview-webhooks-bot
But my broker is Oanda so I'm using python to place the trade. This link has more information.
https://github.com/hootnot/oanda-api-v20
The method is web hook->ngrok->python. When a web hook is sent, the ngrok (while script is also running) shows a 500 internal service error and that the server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
This is what my script says when its running (see picture);
First says some stuff related the market order then;
running script picture
One thing I noticed is that after Debug it doesn't say Running on... (so maybe my flask is not active?
Here is the python script;
from flask import Flask
import market_orders
# Create Flask object called app.
app = Flask(__name__)
# Create root to easily let us know its on/working.
#app.route('/')
def root():
return 'online'
#app.route('/webhook', methods=['POST'])
def webhook():
if request.method == 'POST':
# Parse the string data from tradingview into a python dict
print(market_orders.myfucn())
else:
print('do nothing')
if __name__ == '__main__':
app.run()
Let me know if there is any other information that would be helpful.
Thanks for your help.
I fixed it!!!! Google FTW
The first thing I learned was how to make my module a FLASK server. I followed these websites to figure this out;
This link helped me set up the flask file in a virtual environment. I also moved my Oanda modules to this new folder. And opened the ngrok app while in this folder via the command window. I also ran the module from within the command window using flask run.
https://topherpedersen.blog/2019/12/28/how-to-setup-a-new-flask-app-on-a-mac/
This link showed me how to set the FLASK_APP and the FLASK_ENV
Flask not displaying http address when I run it
Then I fixed the internal service error by adding return 'okay' after print(do nothing) in my script. This I learned from;
Flask Value error view function did not return a response
I'm trying to set up a Flask API Server from which I can data from local database via an ongoing HTTP Request to another database.
In the local code, I run a thread that is running and updating the local DB every 1 minute.
app = Flask(__name__)
cached_event_log = None
#app.route('/event_log', methods=['GET'])
def get_event_log():
if cached_event_log != None and .get_latest_event_time == cached_event_log[-1]:
return jsonify(cached_event_log)
#MAKE CONNECTION TO DB AND GET THE DATA
return jsonify(event_log)
if(__name__ == '__main__'):
app.run(Debug=True)
I'm struggling to find a "standard" way to set up A request.
Any opinion would be highly appreciated- Thank you,
You could set up a cron job to run a script at a specified interval
Or use something like Advanced Python Scheduler
https://apscheduler.readthedocs.io/en/3.0/
Advanced Python Scheduler support in Flask
https://github.com/viniciuschiele/flask-apscheduler
I got timeout error on my GAE server when it tries to send large files to an EC2 REST server. I found Backends Python API would be a good solution to my example but I had some problems on configuring it.
Following some instructions, I have added a simple backends.yaml in my project folder. But I still received the following error, which seems like I failed to create a backend instance.
File "\Google\google_appengine\google\appengine\api\background_thread\background_thread.py", line 84, in start_new_background_thread
raise ERROR_MAP[error.application_error](error.error_detail)
FrontendsNotSupported
Below is my code, and my question is:
Currently, I got timeout error in OutputPage.py, how do I let this script run on a backend instance?
update
Following Jimmy Kane's suggestions, I created a new script przm_batchmodel_backend.py for the backend instance. After staring my GAE, now it I have two ports (a default and a backend) running my site. Is that correct?
app.yaml
- url: /backend.html
script: przm_batchmodel.py
backends.yaml
backends:
- name: mybackend
class: B1
instances: 1
options: dynamic
OutputPage.py
from przm import przm_batchmodel
from google.appengine.api import background_thread
class OutputPage(webapp.RequestHandler):
def post(self):
form = cgi.FieldStorage()
thefile = form['upfile']
#this is the old way to initiate calculations
#html= przm_batchmodel.loop_html(thefile)
przm_batchoutput_backend.przmBatchOutputPageBackend(thefile)
self.response.out.write(html)
app = webapp.WSGIApplication([('/.*', OutputPage)], debug=True)
przm_batchmodel.py
def loop_html(thefile):
#parses uploaded csv and send its info. to the REST server, the returned value is a html page.
data= csv.reader(thefile.file.read().splitlines())
response = urlfetch.fetch(url=REST_server, payload=data, method=urlfetch.POST, headers=http_headers, deadline=60)
return response
przm_batchmodel_backend.py
class BakendHandler(webapp.RequestHandler):
def post(self):
t = background_thread.BackgroundThread(target=przm_batchmodel.loop_html, args=[thefile])
t.start()
app = webapp.WSGIApplication([('/backend.html', BakendHandler)], debug=True)
You need to create an application 'file'/script for the backend to work. Just like you do with the main.
So something like:
app.yaml
- url: /backend.html
script: przm_batchmodel.py
and on przm_batchmodel.py
class BakendHandler(webapp.RequestHandler):
def post(self):
html = 'test'
self.response.out.write(html)
app = webapp.WSGIApplication([('/backend.html', OutputPage)], debug=True)
May I also suggest using the new feature modules which are easier to setup?
Edit due to comment
Possible the setup was not your problem.
From the docs
Code running on a backend can start a background thread, a thread that
can "outlive" the request that spawns it. They allow backend instances
to perform arbitrary periodic or scheduled tasks or to continue
working in the background after a request has returned to the user.
You can only use backgroundthread on backends.
So edit again. Move the part of the code that is:
t = background_thread.BackgroundThread(target=przm_batchmodel.loop_html, args=[thefile])
t.start()
self.response.out.write(html)
To the backend app