I have a high-throughput WSGI app that receives many POSTs per second from a remote server and writes documents to a couchdb server. Currently, the writes to couchdb can only happen between request and response. That means
if writes to couchdb are slow, then the client must sit waiting for a response while we write to the database, and
if there are problems writing to the couchdb server, there is no way to wait a few minutes and retry
Are there any existing solutions to queue up the writes to couch in the background (I'm looking at something like celery, for example), or will I need to roll my own solution?
celery could do that, yes, or any other task queue of a similar nature.
(Alternatively you could go one step lower and use a any message queue server, plus your own independent worker process that consumes from the message queue.)
Related
For this question, I'm particularly struggling with how to structure this:
User accesses website
User clicks button
Value x in database increments
My issue is that multiple people could potentially be on the website at the same time and click the button - I want to make sure each user is able to click the button, and update the value and read the incremented value too, but I don't know how to circumvent any synchronisation/concurrency issues.
I'm using flask to run my website backend, and I'm thinking of using MongoDB or Redis to store my single value that needs to be updated.
Please comment if there is any lack of clarity in my question, but this is a problem I've really been struggling with how to solve.
Thanks :)
redis, I think you can use redis hincrby command, or create a distributed lock to make sure there is only one writer at the same time and only the lock holding writer can make the update in your flask framework. Make sure you release the lock after certain period of time or after the writer done using the lock.
mysql, you can start a transaction, and make the update and commit the change to make sure the data is right
To solve this problem I would suggest you follow a micro service architecture.
A service called worker would handle the flask route that's called when the user clicks on the link/button on the website. It would generate a message to be sent to another service called queue manager that maintains a queue of increment/decrement messages from the worker service.
There can be multiple worker service instances running concurrently but the queue manager is a singleton service that takes the messages from each service and adds them to the queue. If the queue manager is busy the worker service will either timeout and retry or return a failure message to the user. If the queue is full a response is sent back to the worker to retry n number of times, and you can count down that n.
A third service called storage manager is run every time the queue is not empty, this service sends the messages to the storage solution (whatever mongo, redis, good ol' sql) and it will ensure the increment/decrement messages are handled in the order they were received in the queue. You could also include a time stamp from the worker service in the message if you wanted to use that to sort the queue.
Generally whatever hosting environment for flask will use gunicorn as the production web server and support multiple concurrent worker instances to handle the http requests, and this would naturally be your worker service.
How you build and coordinate the queue manager and storage manager is down to implementation preference, for instance you could use something like Google Cloud pub/sub system to send messages between different deployed services but that's just off the top of my head. There's a load of different ways to do it, and you're in the best position to decide that.
Without knowing more details about what you're trying to achieve and what's the requirements for concurrent traffic I can't go into greater detail, but that's roughly how I've approached this type of problem in the past. If you need to handle more concurrent users at the website, you can pick a hosting solution with more concurrent workers. If you need the queue to be longer, you can pick a host with more memory, or else write the queue to an intermediate storage. This will slow it down but will make recovering from a crash easier.
You also need to consider handling when messages fail between different services, how to recover from a service crashing or the queue filling up.
EDIT: Been thinking about this over the weekend and a much simpler solution is to just create a new record in a table directly from the flask route that handles user clicks. Then to get your total you just get a count from this table. Your bottlenecks are going to be how many concurrent workers your flask hosting environment supports and how many concurrent connections your storage supports. Both of these can be solved by throwing more resources at them.
I've built a simple application using Django Channels where a Consumer receives data from an external service, the consumer than sends this data to some subscribers on another consumer.
I'm new to websockets and i had a little concern: the consumer is receiving a lot of data, in the order of 100 (or more) JSON records per second. At what point should i be worried about this service crashing or running into performance issues? Is there some sort of limit for what i'm doing?
there is not explicit limit, however it is worth nothing that for each instances (open connection) of the consumer you can only process one WS message at once.
So if you have a single websocket connection and are sending lots and lots of WS messages down that connection if the consumer does work on these (eg write them to the db) the queue of messages might fill up and you will get an error.
their are a few solutions to this,
Open multiple ws connections and share out the load
in your consumer before doing any work that will take time put it onto a work queue and have some background tasks (that you do not await) consume it.
For this second option it is probably a good idea to create this background queue in your on_connect method and handle shutting it down/waiting for it to flush everything in the on disconnect method.
--
if you are expecting a massive amount of data and don't want to fork out for a costly (high memory) VM you might be better of using a server that is no written in python.
My suggestion for a python developer would be https://docs.vapor.codes/4.0/websockets/ this is a Swift server framework, Swift linguistically if very close to TypeAnotated python so is easier than other high performance options to pick up for a python dev.
Using python 3. Lets say I am processing a loop around a large database query. Effectively my loop around the result set cursor can be a billion iterations.
I'd like to give a user an ability to call out to an http server embedded in the same process that would give some statistics on the progress of the query.
So far I have tried this with IOLoop using Tornado http server. The problem is that I have to basically transfer the control back to IOLoop on some number of rows to get the HTTP request to get serviced. That seems wasteful. Transferring that control has a price. Tornado would let me support multiple connections, but I don't actually care for that - one connection is fine.
What I would prefer would be to simply interrupt the loop, service the HTTP request and resume.
I guess this is probably open to too many possiblities...but using tornado I've just started an instance on a thread. Threading issues with python aside, it basically does what i want.
I am working on a django web app that has functions (say for e.g. sync_files()) that take a long time to return. When I use gevent, my app does not block when sync_file() runs and other clients can connect and interact with the webapp just fine.
My goal is to have the webapp responsive to other clients and not block. I do not expect a zillion users to connect to my webapp (perhaps max 20 connections), and I do not want to set this up to become the next twitter. My app is running on a vps, so I need something light weight.
So in my case listed above, is it redundant to use celery when I am using gevent? Is there a specific advantage to using celery? I prefer not to use celery since it is yet another service that will be running on my machine.
edit: found out that celery can run the worker pool on gevent. I think I am a litle more unsure about the relationship between gevent & celery.
In short you do need a celery.
Even if you use gevent and have concurrency, the problem becomes request timeout. Lets say your task takes 10 minutes to run however the typical request timeout is about up to a minute. So what will happen if you trigger the task directly within a view is that the server will start processing it however after a minute a client (browser) will probably disconnect the connection since it will think the server is offline. As a result, your data can become corrupt since you cannot be guaranteed what will happen when connection will close. Celery solves this because it will trigger a background process which will process the task independent of the view. So the user will get the view response right away and at the same time the server will start processing the task. That is a correct pattern to handle any scenarios which require lots of processing.
I believe nginx is event based so with 1 single worker it can take multiple requests, say 100requests/second. These requests will then be pass on to uwsgi to be process and then once it's done it will push the result back to nginx and nginx will push the result to the user that do http request.
Assuming I am only using 1 worker(no thread) for my uwsgi, uwsgi will process this 100 request one by one right? So it will need to do 100 processes to complete the entire requests.
Now what happen if I am planning to use long polling to get a quick update on my front end How does facebook, gmail send the real time notification?
I believe it will force the uwsgi to process a single request(which is the long polling process) and suspend all the other requests, hence causing the entire system to broke down.
Do I have any misconception of how uwsgi work, or is there any other solution to implement long polling?
Thank You
Your analysis is right, long-polling is not well-suited for multiprocesses or multithreads modes (in term of costs). Each process/thread can manage a single request. Lucky enough uWSGI supports dozens of
non-blocking/evented/microthreads-based technologies (like gevent, or lower-levels greenlets), if your app can be adapted to this patterns (and this is not a no-brain task, so do not hope monkey-patching will be enough) you will win.
In addition to this, if you like/tolerate callback-based programming and you do not need uWSGI specific features, i find Tornado a great solution for the problem.