Uwsgi procces blocks request - python

While testing nginx server with uwsgi and django I am having problem with uwsgi process. I am sending two posts, which are taking a lot of time. Meanwhile server processing I am sending get request from webrowser and I must wait till this two post finished. I am starting uwsgi with this command:
cd /home/pi/cukierek && uwsgi -
-max-requests=5000
--socket /tmp/cukierek.sock
--module config.wsgi
--master-fifo /tmp/cukierek.fifo
--chmod-socket=777 --processes 2
--daemonize /home/pi/cukierek/wsgi.log
--enable-threads
It is possible to get answer from browser while this two post are beeing in progress ? I am using default nginx settings.

You have a uwsgi server configured to spawn 2 processes. Then you run 2 long requests. Those 2 processes are busy with the long requests, so new requests must wait until the long requests finish.
If you want to send new reqeusts to the server while the long requests run, increase the processes to more than 2 (ie --processes 4)

Related

How to fix python uvicorn server returning "426 Upgrade Required"?

I have a python uvicorn app which runs fine locally for my colleagues but not for me. After running python src/main.py, the server connects to database and loads perfectly:
INFO | uvicorn.server:serve:75 - Started server process [49720]
INFO | uvicorn.lifespan.on:startup:47 - Waiting for application startup.
INFO | databases.core:connect:83 - Connected to database postgresql+asyncpg://localhost:5432/faethm_core
INFO | uvicorn.lifespan.on:startup:61 - Application startup complete.
INFO | uvicorn.server:_log_started_message:209 - Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
But the server doesn't take any requests. No matter where I send it from, either curl command, browser request, or an API tool such as Insomnia. I always get the same response 'Upgrade Required'
For example a curl command:
curl --request GET \
--url http://0.0.0.0:8000/health \
--header 'Content-Type: application/json'
would return
Upgrade Required
Things I've tried but failed
Restarting my server and also my computer
Trying to send requests from different browsers and tools
Adding headers to upgrade the protocol to HTTP/2.0. The docs online aren't clear on how to do this
Changing the http connection to https
Does anyone know where this issue is coming from and how to fix it?
A different app was using port 8000 as a daemon running in the background.
Solution
Find any processes that are potentially blocking the port 8000
sudo lsof -i:8080
Find the PID of the processes and kill them
kill $PID
You should now be able to talk to the uvicorn server properly.
Notes
These daemons can get restarted even after you restart your computer, so sometimes they have to be manually killed.
The original status code 426 Upgrade Required didn't help at all to find the source of the problem.
uvicorn server shouldn't have started if the port 8000 was being used by another process or daemon

How to get details about threads running in live gunicorn server

I'm have deployed my Flask in AWS using gunicorn server.
This is my gunicorn configuration in Dockerfile,
CMD gunicorn api:app -w 1 --threads 2 -b 0.0.0.0:8000
It's clear that I'm having one master worker and that worker has 2 threads, the problem I was facing was that server getting stuck sometimes, meaning it was not processing any requests, when I redeployed the app, it started to process the requests once again.
I can increase the number of threads or increase the number of master workers to resolve this issue. But one question I have is how to get information about the threads running in Gunicorn, meaning which thread is processing which request.
Thanks in advance!

Uninterrupted nginx + 2 gunicorns setup

So what's the trick? Nginx is facing the client. Normally the requests are forwarded to gunicorn A at port 80.
You can't run code update in-place, since something might be wrong. So you do a fresh code checkout and launch a separate gunicorn B on some port 5678.
Once you test the new code on a development/testing database, you:
Adjust gunicorn B to point to the database, but do not send any requests.
Stop gunicorn A. Nginx now, ever so briefly, responds with an error.
Set nginx to point to gunicorn B, still at port 5678.
Restart nginx.
Is this about right? Do you just write a script to run the four actions faster and minimize the duration (between steps 2 and 4) the server responds with an error?
Nginx supports configuration reloading. Using this feature, updating your application can work like this:
Start a new instance Gunicorn B.
Adjust the nginx configuration to forward traffic to Gunicorn B.
Reload the nginx configuration with nginx -s reload. After this, Gunicorn B will serve new requests, while Gunicorn A will still finish serving old requests.
Wait for the old nginx worker process to exit (which means all requests initiated before the reload are now done) and then stop Gunicorn A.
Assuming your application works correctly with two concurrent instances, this gives you a zero-downtime update.
The relevant excerpt from the nginx documentation:
Once the master process receives the signal to reload configuration, it checks the syntax validity of the new configuration file and tries to apply the configuration provided in it. If this is a success, the master process starts new worker processes and sends messages to old worker processes, requesting them to shut down. Otherwise, the master process rolls back the changes and continues to work with the old configuration. Old worker processes, receiving a command to shut down, stop accepting new connections and continue to service current requests until all such requests are serviced. After that, the old worker processes exit.

Multithreading Falcon in Python

I'm creating a REST API for an application using Falcon. When launching two or more requests to the API on different endpoints, there's no multi-threaded execution (One request has to be finished to execute the next one)
The problem is coming from a POST endpoint that executes a complex machine learning process (takes dozen of seconds to finish) and the whole API is blocked when the process is being executed, because it waits for the process to be completed to return some results.
I'm using wsgiref simple_server to serve the requests:
if __name__ == '__main__':
httpd = simple_server.make_server('127.0.0.1', 8000, app)
httpd.serve_forever()
Is there any way to make the execution parallel to serve multiple requests in the same time.
Probably the server is not running in multiprocess or multithreaded mode.
But even if it was, it is not a good idea to occupy the web server for long-running tasks. The long running tasks should be run by some other worker processes.
Take a look at Celery
zaher ideally you should use Celery as giorgosp mention but if it is mandatory to return result for API request then you can use Gunicorn
gunicorn --workers 3 -b localhost:8000 main:app --reload
Here, in above code I have mention 3 workers so at a time you can serve/process 3 requests.
Ideally no of workers can be
cpu_count * 2 + 1
You can use any port number you like, but make sure that it is above 1024 and it's not used by any other program.
The main:app option tells Gunicorn to invoke the application object app available in the file main.py.
Gunicorn provides an optional --reload switch that tells Gunicorn to detect any code changes on the fly. This way you can change your code without having to restart Gunicorn.
And if this approach is not suitable for your need than I think you should use Tornado instead of Falcon.
Let me know if any further clarification needed.
This can be easily achieved by coupling Falcon with Gunicorn. With Gunicorn, achieving multi-threading/multi-processing will be relatively easier without needing to implement Celery (Although, nothing is stopping one from implementing it. Celery is awesome!)
gunicorn -b localhost:8000 main:app --threads 3 --workers 3 --reload
The above command will sping up 3 workers with each worker having 3 threads. You as a developer can tweak the number of workers and threads required. I would strongly advise to understand difference between multithreading and multiprocessing before tweaking these settings.

Django Nginx Gunicorn = 504 Timeout

I'm trying to publish a Django application on the production server using Nginx + Gunicorn. When I doing a simple stress test on the server (holding the F5 key for a minute) the server returns a 504 Gateway Time-out error. Why does this happen? This error only appears for the user when doing multiple concurrent requests, or the system will be fully unavailable to everyone?
When you hold down F5:
You've started hundreds of requests.
Those requests have filled your gunicorn request queue.
The request handlers have not been culled as soon as the connection drops.
Your latest requests are stuck in the queue behind all the previous requests.
Nginx times out.
For everyone.
Solutions:
Set up rate-limiting buckets in Nginx, keyed on IP, such that one malicious user can't spam you with requests and DOS your site.
Set up a global rate-limiting bucket in Nginx such that you don't overfill your request queue.
Make Nginx serve a nice "Reddit is under heavy load" style page, so users know that this is a purposeful event
Or:
Replace gunicorn with uwsgi. It's faster, more memory efficient, integrates smoothly with nginx, and most importantly: It will kill the request handler immediately if the connection drops, such that F5 spam can't kill your server.
https://medium.com/#paragsharma.py/504-gateway-timeout-django-gunicorn-nginx-4570deaf0922
504 can be caused by gunicorn timeout you need to start it with --timeout arg like
gunicorn --access-logfile - --workers 3 --timeout 300 --bind unix:/home/ubuntu/myproject/myproject.sock myproject.wsgi:application

Categories

Resources