In the Python documentation it describes how to start and use coroutines.
This section describes how to use a Task.
In the Task section, it states:
Tasks are used to schedule coroutines concurrently
I'm failing to understand, what is happening when I start a coroutines without using Task? Is the code running asynchronously but not concurrently? Does it mean when the code sees an await it goes and does something else?
When I use a Task is it like start two threads and calling join()? I start two or more tasks and wait for the result, correct?
For simple cases, creating Tasks manually is somewhat similar to threads – you can create them, event loop will eventually run them, and you should eventually get result/exception.
But in most cases, your code is built around await coro() – nothing low-level. This means that your code may do some I/O operation inside coro, so process is free to put your implicitly created task into queue, and resume execution later.
Related
I feel like there is a gap in my understanding regarding async/await functionality in python. From my understanding, once is a task is created via asyncio.create_task() it is automatically scheduled, and then a future await call will actually block other code execution until the task has completed. So if you create two tasks, and then await them sequentially, the second task could finish first, but the first task must be completed before code execution can continue. However, code in between the task creation and the await call will obviously proceed immediately (unlike the sync case), which is what I think is the benefit of async/await (please correct me if I am wrong). Are there also other benefits?
Alternatively, one can send off multiple tasks and then use as_completed or gather to handle them as they finish (perhaps out of order).
This flow makes sense in some data aggregating workflow, if you want to send off like 1000 requests simultaneously and want to aggregate or operate on the results sequentially. This all makes sense if you know how many requests you will have before hand, and what exactly the calls will look like, since you are essentially creating async tasks en masse. But what if you want to do async tasks frequently, but not all at once, like sending quotes to a trade matching engine?
I many situations, you likely want to fire a request, continue doing some other work, maybe fire more requests, and then handle the responses upon receiving them (in order of receipt since speed is critical), in a callback mechanism fashion. I don't see much recommendation about add_done_callback which leads me to believe that while I could probably achieve what I am looking for, there must be a gap in my understanding since its not recommended much. What alternatives in the asyncio sphere exist to achieve what I'm talking about? Can an asyncio.Queue be used to achieve this? I'm just confused because nearly every tutorial on the internet involves sending 1000 http requests at once, and handling them using gather or as_completed, but I feel that is such a synthetic and non-real-world workflow.
I am really struggling to understand the interaction between asyncio event loop and multiple workers/threads/processes.
I am using dash: which uses flask internally and gunicorn.
Say I have two functions
def async_download_multiple_files(files):
# This function uses async just so that it can concurrently send
# Multiple requests to different webservers and returns data.
def sync_callback_dash(files):
# This is a sync function that is called from a dash callback to get data
asyncio.run(async_download_multiple_files(files))
As I understand, asyncio.run runs the async function in an event loop but blocks it:
From Python Docs
While a Task is running in the event loop, no other Tasks can run in the same thread.
But what happens when I run a WSGI server like Gunicorn with multiple workers.
Say there are 2 requests coming in simultaneously, presumably there will be multiple calls to sync_callback_dash which will happen in parallel because of multiple Gunicorn workers.
Can both request 1 and request 2 try to execute the asyncio.run in parallel in different threads\processes ? Will one block the other ?
If they can run in parallel, what is the use of having asyncio workers that Gunicorn offers?
I answered this question with the assumption that there is some lack of knowledge on some of the fundamental understandings of threads/processes/async loop. If there was not, forgive me for the amount of detail.
First thing to note is that processes and threads are two separate concepts. This answer might give you some context. To expand:
Processes are run directly by the CPU, and if the CPU has multiple cores, processes can be run in parallel. Inside processes is where threads are run. There is always at least 1 thread per process, but there can be more. If there are more, the process switches between which thread it is executing after every (specific) millisecond (dictated by things out of the scope of this question)- and therefore threads are not run in absolute parallel, but rather constantly switched in and out of the CPU (at least as it pertains to Python, specifically, due to something called the GIL). The async loop runs inside a thread, and switches context relating specifically to I/O-bound instructions (more of this below).
Regarding this question, it's worth noting that Gunicorn workers are processes, and not threads (though you can increase the amount of threads per worker).
The intention of asynchronous code (with the use of async def, await, and asyncio) is to speed-up performance as it specifically relates to I/O bound tasks. Stuff like getting a file from disk, sending/receiving a network request, or anything that requires a physical piece of your computer - whether it is SSD, or the network card - other than the CPU to do some work. It can also be used for large CPU-bound instructions, but this is usually where threads come in. Note that I/O bound instructions are much slower than CPU bound instructions as the electricity inside your computer literally has to travel further distances, as well as perform extra steps in the hardware level (to keep things simple).
These tasks waste the CPU time (or, more specifically, the current process's time) on simply waiting for a reply. Asynchronous code is run with the help of a loop that auto-manages the context switching of I/O bound instructions and normal CPU bound instructions (dependent on the use of await keywords) by leveraging the idea that a function can "yield" control back to the loop, and allow the loop to continue processing other pieces of code while it waits. When async code sends an I/O bound instruction (e.g. grab the latest packet from the network card), instead of sitting still and waiting for a reply it will switch the current process' context to the next task in its list to speed up general execution time (adding that previous I/O bound call to this list to check back in later). There is more to this, but this is the general gist as it relates to your question.
This is what it means when the docs says:
While a Task is running in the event loop, no other Tasks can run in the same thread.
The async loop is not running things in parallel, but rather constantly switching context between different instructions for a more optimized CPU + I/O relationship/execution.
Processes, in the other hand, run in parallel in your CPU assuming you have multiple cores. Gunicorn workers - as mentioned earlier - are processes. When you run multiple async workers with Gunicorn you are effectively running multiple asyncio.loop in multiple (independent, and parallel-running) processes. This should answer your question on:
Can both request 1 and request 2 try to execute the asyncio.run in parallel in different threads\processes ? Will one block the other ?
If there is ever the case that one worker gets stuck on some extremely long I/O bound (or even non-async computation) instruction(s), other workers are there to take care of the next request(s).
With asyncio it is possible to run a separate event loop in each thread. Both will run in parallel (to the extent the Python Interpreter is capable). There are some restrictions. Communication between those loops must use threadsafe methods. Signals and subprocesses can be handled in the main thread only.
Calling asyncio.run in a callback will block until the asyncio part completely finishes. It is not clear from your question if this is what you want.
Alternatively, you could start a long running event loop in one thread and use asyncio.run_coroutine_threadsafe from other threads. Read the docs with an example here.
Disclaimer: this is my first time experimenting with the asyncio module.
I'm using asyncio.wait in the following manner to try to support a timeout feature waiting for all results from a set of async tasks. This is part of a larger library so I'm omitting some irrelevant code.
Note that the library already supports submitting tasks and using timeouts with ThreadPoolExecutors and ProcessPoolExecutors, so I'm not really interested in suggestions to use those instead or questions about why I'm doing this with asyncio. On to the code...
import asyncio
from contextlib import suppress
...
class AsyncIOSubmit(Node):
def get_results(self, futures, timeout=None):
loop = asyncio.get_event_loop()
finished, unfinished = loop.run_until_complete(
asyncio.wait(futures, timeout=timeout)
)
if timeout and unfinished:
# Code options in question would go here...see below.
raise asyncio.TimeoutError
At first I was not worrying about cancelling pending tasks on timeout, but then I got the warning Task was destroyed but it is pending! on program exit or loop.close. After researching a bit I found multiple ways to cancel tasks and wait for them to actually be cancelled:
Option 1:
[task.cancel() for task in unfinished]
for task in unfinished:
with suppress(asyncio.CancelledError):
loop.run_until_complete(task)
Option 2:
[task.cancel() for task in unfinished]
loop.run_until_complete(asyncio.wait(unfinished))
Option 3:
# Not really an option for me, since I'm not in an `async` method
# and don't want to make get_results an async method.
[task.cancel() for task in unfinished]
for task in unfinished:
await task
Option 4:
Some sort of while loop like in this answer. Seems like my other options are better but including for completeness.
Options 1 and 2 both seem to work fine so far. Either option may be "right", but with asyncio evolving over the years the examples and suggestions around the net are either outdated or vary quite a bit. So my questions are...
Question 1
Are there any practical differences between Options 1 and 2? I know run_until_complete will run until the future has completed, so since Option 1 is looping in a specific order I suppose it could behave differently if earlier tasks take longer to actually complete. I tried looking at the asyncio source code to understand if asyncio.wait just effectively does the same thing with its tasks/futures under the hood, but it wasn't obvious.
Question 2
I assume if one of the tasks is in the middle of a long-running blocking operation it may not actually cancel immediately? Perhaps that just depends on if the underlying operation or library being used will raise the CancelledError right away or not? Maybe that should never happen with libraries designed for asyncio?
Since I'm trying to implement a timeout feature here I'm somewhat sensitive to this. If it's possible these things could take a long time to cancel I'd consider calling cancel and not waiting for it to actually happen, or setting a very short timeout to wait for the cancels to finish.
Question 3
Is it possible loop.run_until_complete (or really, the underlying call to async.wait) returns values in unfinished for a reason other than a timeout? If so I'd obviously have to adjust my logic a bit, but from the docs it seems like that is not possible.
Are there any practical differences between Options 1 and 2?
No. Option 2 looks nicer and might be marginally more efficient, but their net effect is the same.
I know run_until_complete will run until the future has completed, so since Option 1 is looping in a specific order I suppose it could behave differently if earlier tasks take longer to actually complete.
It seems that way at first, but it's not actually the case because loop.run_until_complete runs all tasks submitted to the loop, not just the one passed as argument. It merely stops once the provided awaitable completes - that is what "run until complete" refers to. A loop calling run_until_complete over already scheduled tasks is like the following async code:
ts = [asyncio.create_task(asyncio.sleep(i)) for i in range(1, 11)]
# takes 10s, not 55s
for t in ts:
await t
which is in turn semantically equivalent to the following threaded code:
ts = []
for i in range(1, 11):
t = threading.Thread(target=time.sleep, args=(i,))
t.start()
ts.append(t)
# takes 10s, not 55s
for t in ts:
t.join()
In other words, await t and run_until_complete(t) block until t has completed, but allow everything else - such as tasks previously scheduled using asyncio.create_task() to run during that time as well. So the total run time will equal the run time of the longest task, not of their sum. For example, if the first task happens to take a long time, all others will have finished in the meantime, and their awaits won't sleep at all.
All this only applies to awaiting tasks that have been previously scheduled. If you try to apply that to coroutines, it won't work:
# runs for 55s, as expected
for i in range(1, 11):
await asyncio.sleep(i)
# also 55s - we didn't call create_task() so it's equivalent to the above
ts = [asyncio.sleep(i) for i in range(1, 11)]
for t in ts:
await t
# also 55s
for i in range(1, 11):
t = threading.Thread(target=time.sleep, args=(i,))
t.start()
t.join()
This is often a sticking point for asyncio beginners, who write code equivalent to that last asyncio example and expect it to run in parallel.
I tried looking at the asyncio source code to understand if asyncio.wait just effectively does the same thing with its tasks/futures under the hood, but it wasn't obvious.
asyncio.wait is just a convenience API that does two things:
converts the input arguments to something that implements Future. For coroutines that means that it submits them to the event loop, as if with create_task, which allows them to run independently. If you give it tasks to begin with, as you do, this step is skipped.
uses add_done_callback to be notified when the futures are done, at which point it resumes its caller.
So yes, it does the same things, but with a different implementation because it supports many more features.
I assume if one of the tasks is in the middle of a long-running blocking operation it may not actually cancel immediately?
In asyncio there shouldn't be "blocking" operations, only those that suspend, and they should be cancelled immediately. The exception to this is blocking code tacked onto asyncio with run_in_executor, where the underlying operation won't cancel at all, but the asyncio coroutine will immediately get the exception.
Perhaps that just depends on if the underlying operation or library being used will raise the CancelledError right away or not?
The library doesn't raise CancelledError, it receives it at the await point where it happened to suspend before cancellation occurred. For the library the effect of the cancellation is await ... interrupting its wait and immediately raising CancelledError. Unless caught, the exception will propagate through function and await calls all the way to the top-level coroutine, whose raising CancelledError marks the whole task as cancelled. Well-behaved asyncio code will do just that, possibly using finally to release OS-level resources they hold. When CancelledError is caught, the code can choose not to re-raise it, in which case cancellation is effectively ignored.
Is it possible loop.run_until_complete (or really, the underlying call to async.wait) returns values in unfinished for a reason other than a timeout?
If you're using return_when=asyncio.ALL_COMPLETE (the default), that shouldn't be possible. It is quite possible with return_when=FIRST_COMPLETED, then it is obviously possible independently of timeout.
I'm writing a little app in python, consuming some http services, but i really don't understand the difference between using an async function or an Thread for consuming that services.
Anyone can help me to understand?
I have been reading up on the threaded model of programming versus the asynchronous model from this really good article. http://krondo.com/blog/?p=1209
However, the article mentions the following points.
An async program will simply outperform a sync program by switching between tasks whenever there is a I/O.
Threads are managed by the operating system.
I remember reading that threads are managed by the operating system by moving around TCBs between the Ready-Queue and the Waiting-Queue(amongst other queues). In this case, threads don't waste time on waiting either do they?
In light of the above mentioned, what are the advantages of async programs over threaded programs?
In a function there is an entry point and there is an exit point (which is usually a return statement or last statement of function).
Thread: executes all the possible statements from entry point to exit point.
async function :
Functions defined with async def syntax are always coroutine functions
This is from python reference documentation. And coroutines can be entered,exited or resumed from different points anywhere between entry and exit point of the function.
Now, based on your requirement, you can choose which one to use.
I just watch a youtube video where the presenter mentioned that one should design his/her celery to be short. Tasks running several minutes are bad.
Is this correct? What I do see is that I have some long running task, which takes say 10 minutes to finish. When these kind of task is scheduled frequently, the queue is swamped and no other tasks get scheduled. Is this the reason?
If so, what should be used for long running tasks?
Long running tasks aren't great but It's by no means appropriate to say they are bad. The best way to handle long running tasks is to create a queue for just those tasks and have them run on a separate worker then the short tasks.
The problem with long running tasks is that you have to wait for them when you're pushing a new software version on your server. If you don't wait, your task may run possibly incompatible code, especially if you pickled some complex object as a parameter (which is strongly discouraged).
As #user2097159 said its a good practice to keep the long running tasks in a dedicate queue. You should do that by routing using "settings.CELERY_ROUTES" more info here
If you could estimate how long a task can be running, I recommend to use soft_time_limit per task, you will be able to handle it.
There is a gist from a talk I gave here
Augment the basic Task definition to optionally treat the task instantiation as a generator, and check for TERM or soft timeout on every iteration through the generator. Generically inject a "state" dict kwarg into tasks that support it. If it's the first time the task is run, allocate a new one in results cache, otherwise look up the existing one from results cache.
In your task, figure out a good place to yield which results in short execution times. Update the state parameter as necessary.
When control returns to the master task class, check for TERM or soft timeout, and if there is one, save off the state object and respond to the signal.