I use Gunicorn for my python web service with NGINX.
I observe the following problem: even though I have client_max_body_size is big enough in nginx's config, I still get 502 error code when I try to upload big files. I believe that's because uploading is not finished within timeout 90 seconds set when I run my gunicorn workers.
So, my question - is it possible to specify timeout exception for some URLs in Gunicorn command line or config? For example, for file upload URL I want higher timeout and small one for other URLs. If not, what workaround could be implemented?
Thanks in advance!
I'm using mod_wsgi-express to serve a simple Flask app on a CentOS 6 server. The app is currently used pretty infrequently.
I'm having an issue where every few days, the app will crash, either serving up 500 errors or not responding at all. This doesn't seem related to load or use.
Looking through the error log, I'm seeing occasional "[notice] caught SIGTERM, shutting down" messages, after which it appears mod_wsgi-express restarts. I don't know why this is happening.
I'm also getting this message:
[error] (2)No such file or directory: mod_wsgi (pid=24550, process='localhost:8080', application=''): Call to fopen() failed ff
or '/tmp/mod_wsgi-localhost:8080:14699/handler.wsgi'.
I've tried running mod_wsgi-express with both gdb and pdb debuggers enabled and haven't seen any bugs related to the actual execution of the Python script.
I also got a
[info] [client 137.78.237.31] (104)Connection reset by peer: core_output_filter: writing data to the network
which kicked off hundreds of
[notice] child pid 15952 exit signal Segmentation fault (11)
over the course of a couple days.
These messages all seem to happen randomly and independent of each other. The only consistent thing is the SIGTERM signal happening every so often.
I'm confused as to what's going on. The implementation seems pretty simple - it's a simple tool that validates metadata in an uploaded file.
For the 500 errors, use the --server-root option to specify a directory other than in the default of /tmp for its working directory.
The default should only be used for short runs and --server-root used to specify a more permanent home if running it for longer periods.
This is necessary as some operating systems can be set up to remove files from /tmp when they haven't been modified for some period. I have been contemplating adding a warning in the startup output specifically to suggest use of --server-root if the default is being relied upon.
For the segmentation fault issue, please come over the mod_wsgi mailing list to discuss further. StackOverflow is not a discussion forum and that issue will likely need more investigation and discussion to understand what your application is doing.
UPDATE 1
Discussion continuing at:
https://groups.google.com/forum/#!topic/modwsgi/NmyOtp-9Pmg
I have a strange issue, if I restart Apache with my DJANGO Application then static files respond just fine (like the Apache Status Page shows it is handling requests). There are no errors in the log files, and WSGI loads as far as the logfile is concerned.
The problem is that my DJANGO/Python/WSGI Application is not responding to any requests, the web page just times out. I have to restart Apache over and over again, sometimes for up to an hour before the application finally responds. The strange part is that sometimes it will restart immediately. Nothing helps, boot order has no affect, restarting the entire server has not helped. I chased down linked libraries and cleared out the duplicate loading from other requests that I have seen.
A few environment items:
Windows Server Apache 2.2
Python 2.6
Django 1.4.5
MOD_WSGI 3.5 for Windows Apache (http://www.lfd.uci.edu/~gohlke/pythonlibs/#mod_wsgi)
One last detail:
If I block HTTP and HTTPS ports using the firewall and restart Apache then enable the firewall then the service comes right up. So there seams to be some problem with active sessions and Apache/WSGI/Pyhton/Django.
Any ideas on what I can tweak in my WSGI or Apache configuration to get the application to come right up?
Just starting to play around with Flask on a local server and I'm noticing the request/response times are way slower than I feel they should be.
Just a simple server like the following takes close to 5 seconds to respond.
from flask import Flask
app = Flask(__name__)
#app.route("/")
def index():
return "index"
if __name__ == "__main__":
app.run()
Any ideas? Or is this just how the local server is?
Ok I figured it out. It appears to be an issue with Werkzeug and os's that support ipv6.
From the Werkzeug site http://werkzeug.pocoo.org/docs/serving/:
On operating systems that support ipv6 and have it configured such as modern Linux systems, OS X 10.4 or higher as well as Windows Vista some browsers can be painfully slow if accessing your local server. The reason for this is that sometimes “localhost” is configured to be available on both ipv4 and ipv6 socktes and some browsers will try to access ipv6 first and then ivp4.
So the fix is to disable ipv6 from the localhost by commenting out the following line from my hosts file:
::1 localhost
Once I do this the latency problems go away.
I'm really digging Flask and I'm glad that it's not a problem with the framework. I knew it couldn't be.
Add "threaded=True" as an argument to app.run(), as suggested here:
http://arusahni.net/blog/2013/10/flask-multithreading.html
For example: app.run(host="0.0.0.0", port=8080, threaded=True)
The ipv6-disabling solution did not work for me, but this did.
Instead of calling http://localhost:port/endpoint call http://127.0.0.1:port/endpoint.
This removed the initial 500ms delay for me.
The solution from #sajid-siddiqi is technically correct, but keep in mind that the built-in WSGI server in Werkzeug (which is packaged into Flask and what it uses for app.run()) is only single-threaded.
Install a WSGI server to be able to handle multi-threaded behavior. I did a bunch of research on various WSGI server performances. Your needs may vary, but if all you're using is Flask, then I would recommend one of the following webservers.
Update (2020-07-25): It looks like gevent started supporting python3 5 years ago, shortly after I commented that it didn't, so you can use gevent now.
gevent
You can install gevent through pip with the command pip install gevent or pip3 with the command pip3 install gevent. Instructions on how to modify your code accordingly are here: https://flask.palletsprojects.com/en/1.1.x/deploying/wsgi-standalone/#gevent
meinheld
gevent is better, but from all the benchmarks I've looked at that involve real-world testing, meinheld seems to be the most straightforward, simplistic WSGI server. (You could also take a look at uWSGI if you don't mind some more configuration.)
You can also install meinheld through pip3 with the command pip3 install meinheld. From there, look at the sample provided in the meinheld source to integrate Flask: https://github.com/mopemope/meinheld/blob/master/example/flask_sample.py
*NOTE: From my use of PyCharm, the line from meinheld import server highlights as an error, but the server will run, so you can ignore the error.
My problem was solved by "threaded=True", but I want to give some background to distinguish my problem from others for which this may not do it.
My issue only arose when running Flask with python3. Switching to python2, I no longer had this issue.
My problem manifested only when accessing the api with Chrome, at which point, Chrome displayed the expected screen, but everything else hung (curl, ffx, etc) until I either reloaded or closed the Chrome tab, at which point everything else that was waiting around returned a result.
My best guess is that Chrome was trying to keep the session open and Flask was blocking the subsequent requests. As soon as the connection from Chrome was stopped or reset, everything else was processed.
In my case, threading fixed it. Of course, I'm now going through some of the links others have provided to make sure that it's not going to cause any other issues.
threaded=True works for me, but finally I figured out that the issue is due to foxyproxy on firefox. Since when the flask app is running on localhost, slow response happens if
foxyproxy is enabled on firefox
slow response won't happen if
foxyproxy is disabled on firefox
access the website using other browsers
The only solution I found is to disable foxyproxy, tried to add localhost to proxy blacklist and tweak settings but none of them worked.
I used Miheko's response to solve my issue.
::1 localhost was already commented out on my hosts file, and setting Threaded=true didn't work for me. Every REST request was taking 1 second to process instead of being instant.
I'm using python 3.6, and I got flask to be fast and responsive to REST requests by making flask use gevent as its WSGI.
To use gevent, install it with pip install gevent
Afterwards, I used the https://gist.github.com/viksit/b6733fe1afdf5bb84a40#file-async_flask-py-L41 to set flask to use gevent.
Incase the link goes down, here's the important parts of the script:
from flask import Flask, Response
from gevent.pywsgi import WSGIServer
from gevent import monkey
# need to patch sockets to make requests async
# you may also need to call this before importing other packages that setup ssl
monkey.patch_all()
app = Flask(__name__)
# define some REST endpoints...
def main():
# use gevent WSGI server instead of the Flask
# instead of 5000, you can define whatever port you want.
http = WSGIServer(('', 5000), app.wsgi_app)
# Serve your application
http.serve_forever()
if __name__ == '__main__':
main()
I got this error when running on hosts other than localhost as well, so for some, different underlying problems may exhibit the same symptoms.
I switched most of the things I've been using to Tornado, and anecdotally it's helped an amount. I've had a few slow page loads, but things seem generally more responsive. Also, very anecdotal, but I seem to notice that Flask alone will slow down over time, but Flask + Tornado less so. I imagine using Apache and mod_wsgi would make things even better, but Tornado's really simple to set up (see http://flask.pocoo.org/docs/deploying/others/).
(Also, a related question: Flask app occasionally hanging)
I had a different solution here. I just deleted all .pyc from the server's directory and started it again.
By the way, localhost was already commented out in my hosts file (Windows 8).
The server was freezing the whole time and now it works fine again.
I run Python 3.8.10 which works fine, but as soon as I switch to Python 3.10.6 responses will be awful slow.
I could get around this problem with changing the precedence to IPv4 in /etc/gai.conf
I've tried many different ways of writing a simple HTTP server for Python (using IronPython in a hosted environment) and whenever there is an error (e.g. run-time error) the Python environment just hangs, rather than shutting down and reporting the error.
Update:
There is a comment on an answer to another SO question that suggest that calling "self.server.shutdown()" in a request handler also causes a Python web server to hang in Windows.
So possibly run-time exceptions lead to the same problem.
You should probably do something along the lines of:
try:
httpd.serve_forever()
except RuntimeError:
httpd.shutdown()
sys.exit()
btw, this is for python 3 and it assumes that you have the sys module imported.