I have never written any code that uses threads.
I have a web application that accepts a POST request, and creates an image based on the data in the body of the request.
Would I want to spin off a thread for the image creation, as to prevent the server from hanging until the image is created? Is this an appropriate use, or merely a solution looking for a problem ?
Please correct any misunderstandings I may have.
Rather than thinking about handling this via threads or even processes, consider using a distributed task manager such as Celery to manage this sort of thing.
Usual approach for handling HTTP requests synchronously is to spawn (or re-use one in the pool) new thread for each request as soon as it comes.
However, python threads are not very good for HTTP, due to GIL and some i/o and other calls blocking whole app, including other threads.
You should look into multiprocessing module for this usage. Spawn some worker processes, and then pass requests to them to process.
Related
We're running a Django project with gunicorn and eventlet.
I'd like to use threading.local to stash some http request data for use later in that thread (with some custom middleware). I'm wondering if this is safe with eventlet.
From the docs
Eventlet is thread-safe and can be used in conjunction with normal Python threads. The way this works is that coroutines are confined to their ‘parent’ Python thread. It’s like each thread contains its own little world of coroutines that can switch between themselves but not between coroutines in other threads.
which sounds like it might be.
But I understand that eventlet, based on reading their docs on 'How the Hubs Work', may suspend a co-routine to process another one. Is it possible, with gunicorn the an http request processing may get suspended and another http request would get picked up and processed by a co-routine in that same initial thread? And if so, does that mean that the threading.local could get shared between two requests?
Can I get away with using threading.local and be certain that each incoming request will get it's own thread.local space?
I also saw this post
the simultaneous connections are handled by green threads. Green threads are not like real threads. In simple terms, green threads are functions (coroutines) that yield whenever the function encounters I/O operation
which makes me think a single "thread" could process multiple requests. And I guess if that is true, then I wonder where exactly is threading.local? at the thread? in a co-routine eventlet (air quotes)thread(air quotes)?
Any pointers would be appreciated here.
Thanks
tl;dr: the answer is yes.
The eventlet coroutines are treated as separate threads so threading.local will work.
A longer discussion is available on the eventlet GitHub issue.
I'm working in a REST service that is basically an wrapper to a library. I'm using flask and gunicorn. Basically each endpoint in the service maps to a different function in the library.
It happens that some of the calls to the library can take a long time to return, and that is making my service run out of workers once the service starts receiving a few requests. Right now I'm using the default gunicorn workers (sync).
I wanted to use gevent workers in order to be able to receive more requests, because not every endpoint takes that long to execute. However the function in the library does not use any of the patchable gevent functions, meaning that it won't cooperatively schedule to another green thread.
I had this idea of using a pool of threads or processes to handle the calls to the library asynchronously, and then each green thread produced by gunicorn would sleep until the process is not finished. Does this idea make sense at all?
Is it possible to use the multiprocessing.Process with gevent? and then have the join method to give up control to another green thread, and only return when the process is finished?
Yes, it makes perfect sense to use (real) threads or processes from within gevent for code that needs to be asynchronous but can't be monkeypatched by gevent.
Of course it can be tricky to get right—first, because you may have monkeypatched threading, and second, because you want your cooperative threads to be able to block on a pool or a pool result without blocking the whole main thread.
But that's exactly what gevent.threadpool is for.
If you would have used concurrent.futures.ThreadPoolExecutor in a non-gevent app, monkeypatch threading and then use gevent.threadpool.ThreadPoolExecutor.
If you would have used multiprocessing.dummy.Pool in a non-gevent app, monkeypatch threading and then use gevent.threadpool.ThreadPool.
Either way, methods like map, submit, apply_async, etc. work pretty much the way you'd expect. The Future and AsyncResult objects play nice with greenlets; you can gevent.wait things, or attach callbacks (which will run as greenlets), etc. Most of the time it just works like magic, and the rest of the time it's not too hard to figure out.
Using processes instead of threads is doable, but not as nice. AFAIK, there's no wrappers for anything as complete as multiprocessing.Process or multiprocessing.Pool, and trying to use the normal multiprocessing just hangs. You can manually fork if you're not on Windows, but that's about all that's built in. If you really need multiprocessing, you may need to do some multi-layered thing, where your greenlets don't talk to a process, but instead talk to a thread that creates a pipe, forks, execs, and then proxies between the gevent world and the child process.
If the calls are taking a long time because they're waiting on I/O from a backend service, or waiting on a subprocess, or doing GIL-releasing numpy work, I wouldn't bother trying to do multiprocessing. But if they're taking a long time because they're burning CPU… well, then you either need to get multiprocessing working, or go lower-level and just spin off a subprocess.Popen([sys.executable, 'workerscript.py']).
I'm quite new to python threading/network programming, but have an assignment involving both of the above.
One of the requirements of the assignment is that for each new request, I spawn a new thread, but I need to both send and receive at the same time to the browser.
I'm currently using the asyncore library in Python to catch each request, but as I said, I need to spawn a thread for each request, and I was wondering if using both the thread and the asynchronous is overkill, or the correct way to do it?
Any advice would be appreciated.
Thanks
EDIT:
I'm writing a Proxy Server, and not sure if my client is persistent. My client is my browser (using firefox for simplicity)
It seems to reconnect for each request. My problem is that if I open a tab with http://www.google.com in it, and http://www.stackoverflow.com in it, I only get one request at a time from each tab, instead of multiple requests from google, and from SO.
I answered a question that sounds amazingly similar to your, where someone had a homework assignment to create a client server setup, with each connection being handled in a new thread: https://stackoverflow.com/a/9522339/496445
The general idea is that you have a main server loop constantly looking for a new connection to come in. When it does, you hand it off to a thread which will then do its own monitoring for new communication.
An extra bit about asyncore vs threading
From the asyncore docs:
There are only two ways to have a program on a single processor do
“more than one thing at a time.” Multi-threaded programming is the
simplest and most popular way to do it, but there is another very
different technique, that lets you have nearly all the advantages of
multi-threading, without actually using multiple threads. It’s really
only practical if your program is largely I/O bound. If your program
is processor bound, then pre-emptive scheduled threads are probably
what you really need. Network servers are rarely processor bound,
however.
As this quote suggests, using asyncore and threading should be for the most part mutually exclusive options. My link above is an example of the threading approach, where the server loop (either in a separate thread or the main one) does a blocking call to accept a new client. And when it gets one, it spawns a thread which will then continue to handle the communication, and the server goes back into a blocking call again.
In the pattern of using asyncore, you would instead use its async loop which will in turn call your own registered callbacks for various activity that occurs. There is no threading here, but rather a polling of all the open file handles for activity. You get the sense of doing things all concurrently, but under the hood it is scheduling everything serially.
To achieve something similar to google app engines 'deferred calls' (i.e., the request is handled, and afterwards the deferred task is handled), i experimented a little and came up with the solution to spawn a thread in which my deferred call is handled.
I am now trying to determine if this is an acceptable way.
Is it possible (according to the WSGI specification) that the process is terminated by the webserver after the actual request is handled, but before all threads run out?
(if there's a better way, that would be also fine)
WSGI does not specify the lifetime of an application process (as WSGI application is a Python callable object). You can run it in a way that is completely independent of the web server, in which case, only you control the lifetime.
There is also nothing in the WSGI that would prohibit you from spawning threads, or processes, or doing whatever the hell you want.
FWIW, also have a read of:
http://code.google.com/p/modwsgi/wiki/RegisteringCleanupCode
The hooking of actions to close() of iterable is the only way within context of the WSGI specification itself for doing deferred work. That isn't in a separate thread though and would occur within the context of the actual request, albeit after the response is supposed to have been flushed back to the client. Thus your deferred action will consume that request thread until the work is complete and so that request thread would not be able to handle other requests until then.
In general, if you do use background threads, there is no guarantee that any hosting mechanism would wait until those background threads complete before shutting process down. In fact, can't even think of any standard deployment mechanism which does wait. There isn't really even a guarantee that atexit handlers will be called on process shutdown, something that the referenced documentation also briefly talks about.
So, I'm writing a python web application using the twisted web2 framework. There's a library that I need to use (SQLAlchemy, to be specific) that doesn't have asynchronous code. Would it be bad to spawn a thread to handle the request, fetch any data from the DB, and then return a response? I'm afraid that if there was a flood of requests, too many threads would be started and the server would be overwhelmed. Is there something built into twisted that prevents this from happening (eg request throttling)?
See the docs, and specifically the thread pool which lets you control how many threads are active at most. Spawning one new thread per request would definitely be an inferior idea!