I have a Flask application that is running using gunicorn and nginx. But if I change the value in the db, the application fails to update in the browser under some conditions.
I have a flask script that has the following commands
from msldata import app, db, models
path = os.path.dirname(os.path.abspath(__file__))
manager = Manager(app)
#manager.command
def run_dev():
app.debug = True
if os.environ.get('PROFILE'):
from werkzeug.contrib.profiler import ProfilerMiddleware
app.config['PROFILE'] = True
app.wsgi_app = ProfilerMiddleware(app.wsgi_app, restrictions=[30])
if 'LISTEN_PORT' in app.config:
port = app.config['LISTEN_PORT']
else:
port = 5000
print app.config
app.run('0.0.0.0', port=port)
print app.config
#manager.command
def run_server():
from gunicorn.app.base import Application
from gunicorn.six import iteritems
# workers = multiprocessing.cpu_count() * 2 + 1
workers = 1
options = {
'bind': '0.0.0.0:5000',
}
class GunicornRunner(Application):
def __init__(self, app, options=None):
self.options = options or {}
self.application = app
super(GunicornRunner, self).__init__()
def load_config(self):
config = dict([(key, value) for key, value in iteritems(self.options) if key in self.cfg.settings and value is not None])
for key, value in iteritems(config):
self.cfg.set(key.lower(), value)
def load(self):
return self.application
GunicornRunner(app, options).run()
Now if i run the server run_dev in debug mode db modifications are updated
if run_server is used the modifications are not seen unless the app is restarted
However if i run like gunicorn -c a.py app:app, the db updates are visible.
a.py contents
import multiprocessing
bind = "0.0.0.0:5000"
workers = multiprocessing.cpu_count() * 2 + 1
Any suggestions on where I am missing something..
I also ran into this situation. Running flask in Gunicorn with several workers and the flask-cache won´t work anymore.
Since you are already using
app.config.from_object('default_config') (or similar filename)
just add this to you config:
CACHE_TYPE = "filesystem"
CACHE_THRESHOLD = 1000000 (some number your harddrive can manage)
CACHE_DIR = "/full/path/to/dedicated/cache/directory/"
I bet you used "simplecache" before...
I was/am seeing the same thing, Only when running gunicorn with flask. One workaround is to set Gunicorn max-requests to 1. However thats not a real solution if you have any kind of load due to the resource overhead of restarting the workers after each request. I got around this by having nginx serve the static content and then changing my flask app to render the template and write to static, then return a redirect to the static file.
Flask-Caching SimpleCache doesn't work w. workers > 1 Gunicorn
Had similar issue using version Flask 2.02 and Flask-Caching 1.10.1.
Everything works fine in development mode until you put on gunicorn with more than 1 worker. One probably reason is that on development there is only one process/worker so weirdly under this restrict circumstances SimpleCache works.
My code was:
app.config['CACHE_TYPE'] = 'SimpleCache' # a simple Python dictionary
cache = Cache(app)
Solution to work with Flask-Caching use FileSystemCache, my code now:
app.config['CACHE_TYPE'] = 'FileSystemCache'
app.config['CACHE_DIR'] = 'cache' # path to your server cache folder
app.config['CACHE_THRESHOLD'] = 100000 # number of 'files' before start auto-delete
cache = Cache(app)
Related
I have a FastAPI server configured with Gunicorn, deployed on AWS App Runner. When I try to access the endpoint, it works perfectly, however, after 24 hours, when I try to access the same endpoint, I get a 502 bad gateway error, and nothing is logged on cloudWatch after this point, until I redeploy the application, then it starts working fine again.
I suspect this has to do with my Gunicorn configuration itself which was somehow shutting down my API after some time, and not AWS App Runner, but I have not found any solution. I have also shown my Gunicorn setup below. Any hep will be appreciated.
from fastapi import FastAPI
import uvicorn
from fastapi.middleware.cors import CORSMiddleware
from gunicorn.app.base import BaseApplication
import os
import multiprocessing
api = FastAPI()
def number_of_workers():
print((multiprocessing.cpu_count() * 2) + 1)
return (multiprocessing.cpu_count() * 2) + 1
class StandaloneApplication(BaseApplication):
def __init__(self, app, options=None):
self.options = options or {}
self.application = app
super().__init__()
def load_config(self):
config = {
key: value for key, value in self.options.items()
if key in self.cfg.settings and value is not None
}
for key, value in config.items():
self.cfg.set(key.lower(), value)
def load(self):
return self.application
#api.get("/test")
async def root():
return 'Success'
if __name__ == "__main__":
if os.environ.get('APP_ENV') == "development":
uvicorn.run("api:api", host="0.0.0.0", port=2304, reload=True)
else:
options = {
"bind": "0.0.0.0:2304",
"workers": number_of_workers(),
"accesslog": "-",
"errorlog": "-",
"worker_class": "uvicorn.workers.UvicornWorker",
"timeout": "0"
}
StandaloneApplication(api, options).run()
I had the same problem. After a lot of trial and error, two changes seemed to resolve this for me.
Set uvicorn --timeout-keep-alive to 65. For gunicorn this param is --keep-alive. I'm assuming the Application Load Balancer throws 502 if uvicorn closes the tcp socket before ALB does.
Change the App Runner health check to use HTTP rather than TCP ping to manage container recycling. Currently the AWS UI doesn't allow you to make this change. You will have to do this using aws cli. Use any active URL path for ping check - in your case /test
aws apprunner update-service --service-arn <arn> --health-check-configuration Protocol=HTTP,Path=/test
#2 might just be enough to resolve the issue.
after a working production app was connected to local mongo, we've decided to move to Mongo Atlas.
This caused production errors.
Our stack is docker -> alping 3.6 -> python 2.7.13 -> Flask -> uwsgi (2.0.17) -> nginx running on aws
flask-mongoengine-0.9.3 mongoengine-0.14.3 pymongo-3.5.1
when starting the app in staging/production, The uwsgi is sending No replica set members found yet.
We don't know why.
We've tried different connection settings connect: False which is lazy connecting, meaning not on initializing, but on first query.
It caused the nginx to fail for resource temporarily unavailable error on some of our apps. We had to restart multiple times for the app to finally start serving requests.
I think the issue is with pymongo and the fact that it's not fork-safe
http://api.mongodb.com/python/current/faq.html?highlight=thread#id3
and uwsgi is using forks
I suspect it might be related to the way my app is being initalized.
might by aginst Using PyMongo with Multiprocessing
here is the app init code:
from app import FlaskApp
from flask import current_app
app = None
from flask_mongoengine import MongoEngine
import logging
application, app = init_flask_app(app_instance, module_name='my_module')
def init_flask_app(app_instance, **kwargs):
app_instance.init_instance(environment_config)
application = app_instance.get_instance()
app = application.app
return application, app
# app_instance.py
import FlaskApp
def init_instance(env):
global app
app = FlaskApp(env)
return app
def get_instance():
if globals().get('app') is None:
app = current_app.flask_app_object
else:
app = globals().get('app')
assert app is not None
return app
class FlaskApp(object):
def __init__(self, env):
.....
# Initialize the DB
self.db = Database(self.app)
....
# used in app_instance.py to get the flask app object in case it's None
self.app.flask_app_object = self
def run_server(self):
self.app.run(host=self.app.config['HOST'], port=self.app.config['PORT'], debug=self.app.config['DEBUG'])
class Database(object):
def __init__(self, app):
self.db = MongoEngine(app)
def drop_all(self, database_name):
logging.warn("Dropping database %s" % database_name)
self.db.connection.drop_database(database_name)
if __name__ == '__main__':
application.run_server()
help in debugging this will be appreciated!
I wrote a very simple flask web service that simply returns text hey23 upon being hit and hosted on AWS EC2 t2.micro machine (1GB RAM 1CPU)
To execute this python application I used uwsgi as my app server. Finally I've put my complete setup behind Nginx.
So my stack is Flask+uwsgi+Nginx
Everything is working fine and good. I only have complaint with the execution time. The average latency measured using wrk is ~370ms, which is too much considering the amount of work this service is doing.
Running 30s test # http://XX.XXX.XX.XX/printtest
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 370.98ms 178.90ms 1.96s 91.78%
Req/Sec 93.72 36.72 270.00 69.55%
33124 requests in 30.11s, 5.41MB read
Socket errors: connect 0, read 0, write 0, timeout 15
Non-2xx or 3xx responses: 1173
Requests/sec: 1100.26
Transfer/sec: 184.14KB
hello-test.py
from flask import Flask, jsonify, request
app = Flask(__name__)
#app.route("/print23")
def helloprint():
return "hey23"
if __name__ == "__main__":
app.run(host='0.0.0.0', port=8080, threaded=True)
uwsgi.ini
[uwsgi]
#application's base folder
base = /var/www/demoapp
master = true
#python module to import
app = hello-test
module = %(app)
home = %(base)/venv
pythonpath = %(base)
#socket file's location
socket = /var/www/demoapp/%n.sock
#permissions for the socket file
chmod-socket = 644
#the variable that holds a flask application inside the module imported at line #6
callable = app
disable-logging = True
#location of log files
logto = /var/log/uwsgi/%n.log
max-worker-lifetime = 30
processes = 10
threads = 2
enable-threads = True
cheaper = 2
cheaper-initial = 5
cheaper-step = 1
cheaper-algo = spare
cheaper-overload = 5
Even if i forget about wrk benchmark, even with posting GET requests from my POSTMAN client, I am getting similar latency.
What is wrong here? No matter what, some takeaways
Code cannot be optimised. It just has to return hey23 string.
Nothing can be wrong with Nginx.
I certainly assume ~370ms is not a
good response time for an API doing such a simple task.
Changing the region in which my EC2 machine is hosted may bring some change, but common, this should not be sole reason.
Then what am i missing?
I have a REST API backend with python/flask and want to stream the response in an event stream. Everything is running inside a docker container with nginx/uwsgi (https://hub.docker.com/r/tiangolo/uwsgi-nginx-flask/).
The API works fine until it comes to the event-stream. It seems like something (probably nginx) is buffering the "yields" because nothing is received by any kind of client until the server finished the calculation and everything is sent together.
I tried to adapt the nginx settings (according to the docker image instructions) with an additional config (nginx_streaming.conf) file saying:
server {
location / {
include uwsgi_params;
uwsgi_request_buffering off;
}
}
dockerfile:
FROM tiangolo/uwsgi-nginx-flask:python3.6
COPY ./app /app
COPY ./nginx_streaming.conf /etc/nginx/conf.d/nginx_streaming.conf
But I am not really familiar with nginx settings and sure what I am doing here^^ This at least does not work.. any suggestions?
My server side implementation:
from flask import Flask
from flask import stream_with_context, request, Response
from werkzeug.contrib.cache import SimpleCache
cache = SimpleCache()
app = Flask(__name__)
from multiprocessing import Pool, Process
#app.route("/my-app")
def myFunc():
global cache
arg = request.args.get(<my-arg>)
cachekey = str(arg)
print(cachekey)
result = cache.get(cachekey)
if result is not None:
print('Result from cache')
return result
else:
print('object not in Cache...calculate...')
def calcResult():
yield 'worker thread started\n'
with Pool(processes=cores) as parallel_pool:
[...]
yield 'Somewhere in the processing'
temp_result = doSomethingWith(
savetocache = cache.set(cachekey, temp_result, timeout=60*60*24) #timeout in seconds
yield 'saved to cache with key:' + cachekey +'\n'
print(savetocache, flush=True)
yield temp_result
return Response(calcResult(), content_type="text/event-stream")
if __name__ == "__main__":
# Only for debugging while developing
app.run(host='0.0.0.0', debug=True, port=80)
I ran into the same problem. Try changing
return Response(calcResult(), content_type="text/event-stream")
to
return Response(calcResult(), content_type="text/event-stream", headers={'X-Accel-Buffering': 'no'})
Following the answer from #u-rizwan here, I added this to the /etc/nginx/conf.d/mysite.conf and it resolved the problem:
add_header X-Accel-Buffering no;
I have added it under location /, but it is probably a good idea to put it under the specific location of the event stream (I have a low traffic intranet use case here).
Note: Looks like nginx could be stripping this header by default if it comes from the application: https://serverfault.com/questions/937665/does-nginx-show-x-accel-headers-in-response
I have a question to ask regarding the performance of my flask app when I incorporated uwsgi and nginx.
My app.view file looks like this:
import app.lib.test_case as test_case
from app import app
import time
#app.route('/<int:test_number>')
def test_case_match(test_number):
rubbish = test_case.test(test_number)
return "rubbish World!"
My app.lib.test_case file look like this:
import time
def test_case(test_number):
time.sleep(30)
return None
And my config.ini for my uwsgi looks like this:
[uwsgi]
socket = 127.0.0.1:8080
chdir = /home/ubuntu/test
module = app:app
master = true
processes = 2
daemonize = /tmp/uwsgi_daemonize.log
pidfile = /tmp/process_pid.pid
Now if I run this test case just purely through the flask framework without switching on uwsgi + nginx, using the ab benchmark, I received a response in 31seconds which is expected owning to the sleep function. What I dont get is when I run the app through uwsgi + nginx , the response time I got was 38 seconds, which is an overhead of around 25%. Can anyone enlighten me?
time.sleep() is not time-safe.
From the documentation of time.sleep(secs):
[…] Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.