Develop Reference

Use cases for threading and asyncio in python

I've read quite a few articles on threading and asyncio modules in python and the major difference I can seem to draw (correct me if I'm wrong) is that in,
threading: multiple threads can be used to execute the python program and these threads are juggled by the OS itself. Further only when non blocking I/O is happening on a thread the GIL lock can be released to allow another thread to use it (since GIL makes python interpreter single threaded). This is also more resource intensive than asyncio io, since multiple threads will be utilising multiple resources.
asyncio: one single thread can have multiple tasks/coroutines that multitask cooperatively to achieve concurrency. Here, the issue of GIL doesn't arise since it is on a single thread anyway and whenever one non blocking I/O bound task is happening, python interpreter can be used by another coroutine - and all of this is managed by asyncio's event loop.
Also, one article: http://masnun.rocks/2016/10/06/async-python-the-different-forms-of-concurrency/
says,
if io_bound:
if io_very_slow:
print("Use Asyncio")
else:
print("Use Threads")
else:
print("Multi Processing")
I'd like to understand, just for better clarity, why exactly we can't use asyncio and threading as substitutes for each other, given we have sufficient resources available. Use cases of when to use what would help understand better. Further, since this topic is very new for me, there might be gaps in my understanding, so any kind of resources, explanations and corrections would be really appreciated.

Why is it that only asynchronous functions can yield in asynchronous code?

In the article "I'm not feeling the async pressure" Armin Ronacher makes the following observation:
In threaded code any function can yield. In async code only async functions can. This means for instance that the writer.write method cannot block.
This observation is made with reference to the following code sample:
from asyncio import start_server, run
async def on_client_connected(reader, writer):
while True:
data = await reader.readline()
if not data:
break
writer.write(data)
async def server():
srv = await start_server(on_client_connected, '127.0.0.1', 8888)
async with srv:
await srv.serve_forever()
run(server())
I do not understand this comment. Specifically:
How come synchronous functions cannot yield when inside of asynchronous functions?
What does yield have to do with blocking execution? Why is it that a function that cannot yield, cannot block?

Going line-by-line:
In threaded code any function can yield.
Programs running on a machine are organized in terms of processes. Each process may have one or more threads. Threads, like processes, are scheduled by (and interruptible by) the operating system. The word "yield" in this context means "letting other code run". When work is split between multiple threads, functions "yield" easily: the operating system suspends the code running in one thread, runs some code in a different thread, suspends that, comes back, and works some more on the first thread, and so on. By switching between threads in this way, concurrency is achieved.
In this execution model, whether the code being suspended is synchronous or asynchronous does not matter. The code within the thread is being run line-by-line, so the fundamental assumption of a synchronous function---that no changes occurred in between running one line of code and the next---is not violated.
In async code only async functions can.
"Async code" in this context means a single-threaded application that does the same work as the multi-threaded application, except that it achieves concurrency by using asynchronous functions within a thread, instead of splitting the work between different threads. In this execution model, your interpreter, not the operating system, is responsible for switching between functions as needed to achieve concurrency.
In this execution model, it is unsafe for work to be suspended in the middle of a synchronous function that's located inside of an asynchronous function. Doing so would mean running some other code in the middle of running your synchronous function, breaking the "line-by-line" assumption made by the synchronous function.
As a result, the interpreter will wait only suspend the execution of an asynchronous function in between synchronous sub-functions, never within one. This is what is meant by the statement that synchronous functions in async code cannot yield: once a synchronous function starts running, it must complete.
This means for instance that the writer.write method cannot block.
The writer.write method is synchronous, and hence, when run in an async program, uninterruptible. If this method were to block, it would block not just the asynchronous function it is running inside of, but the entire program. That would be bad. writer.write avoids blocking the program by writing to a write buffer instead and returning immediately.
Strictly speaking, writer.write can block, it's just inadvisable to do so.
If you need to block inside of an async function, the proper way to do so is to await another async function. This is what e.g. await writer.drain() does. This will block asynchronously: while this specific function remains blocked, it will correctly yield to other functions that can run.

“Yield” here refers to cooperative multitasking (albeit within a process rather than among them). In the context of the async/await style of Python programming, asynchronous functions are defined in terms of Python’s pre-existing generator support: if a function blocks (typically for I/O), all its callers that are performing awaits suspend (with an invisible yield/yield from that is indeed of the generator variety). The actual call for any generator is to its next method; that function actually returns.
Every caller, up to some sort of driver that most programmers never write, must participate for this approach to work: any function that did not suspend would suddenly have the responsibility of the driver of deciding what to do next while waiting on the function it called to complete. This “infectious” aspect of asynchronicity has been called a “color”; it can be problematic, as for example when people forget to await a coroutine call that looks correct because it looks like any other call. (The async/await syntax exists to minimize the disruption of the program’s structure from the concurrency by implicitly converting functions into state machines, but this ambiguity remains.) It can also be a good thing: an asynchronous function can be interrupted exactly when it awaits, so it’s straightforward to reason about the consistency of data structures.
A synchronous function therefore cannot yield simply as a matter of definition. The import of the restriction is rather that a function called with a normal (synchronous) call cannot yield: its caller is not prepared to handle such an interaction. (What will happen if it does anyway is of course the same “forgotten await”.) This also affects refactoring: a function cannot be changed to be asynchronous without changing all its clients (and making them asynchronous as well if they are not already). (This is similar to how all I/O works in Haskell, since it affects the type of any function that performs any.)
Note that yield is allowed in its role as a normal generator used with an ordinary for even in an asynchronous function, but that’s just the general fact that the caller must expect the same protocol as the callee: if an enhanced generator (an “old-style” coroutine) is used with for, it just gets None from every (yield), and if an async function is used with for, it produces awaitables that probably break when they are sent None.
The distinction with threading, or with so-called stackful coroutines or fibers, is that no special resumption support is needed from the caller because the actual function call simply doesn’t return until the thread/fiber is resumed. (In the thread case, the kernel also chooses when to resume it.) In that sense, these approaches are easier to use, but with fibers the ability to “sneak” a pause into any function is partially compromised by the need to specify arguments to that function to tell it about the userspace scheduler with which to register itself (unless you’re willing to use global variables for that…). Threads, on the other hand, have even higher overhead than fibers, which matters when great numbers of them are running.

Understanding the difference between Async/Await and Task

In the Python documentation it describes how to start and use coroutines.
This section describes how to use a Task.
In the Task section, it states:
Tasks are used to schedule coroutines concurrently
I'm failing to understand, what is happening when I start a coroutines without using Task? Is the code running asynchronously but not concurrently? Does it mean when the code sees an await it goes and does something else?
When I use a Task is it like start two threads and calling join()? I start two or more tasks and wait for the result, correct?

For simple cases, creating Tasks manually is somewhat similar to threads – you can create them, event loop will eventually run them, and you should eventually get result/exception.
But in most cases, your code is built around await coro() – nothing low-level. This means that your code may do some I/O operation inside coro, so process is free to put your implicitly created task into queue, and resume execution later.

Python: what are the advantages of async over threads? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I've had a hard time trying to understand how and why async functionality works in python and I am still not sure I understand everything correctly (especially the 'why' part). Please correct me if I am wrong.
The purpose of both async methods and threads is to make it possible to process several tasks concurrently.
Threads approach looks simple and intuitive. If python program processes several tasks concurrently we have a thread (may be with sub-threads) for each task, the stack of each thread reflects the current stage of processing of corresponding task. Everything is straightforward, there are easy-to-use mechanisms to start a new thread and wait for results from it.
As I understand the only problem with this approach is that threads are expensive.
Another approach is using async coroutines. I can see several inconveniences with this approach. I'll name couple of them only. We have now two types of methods: usual methods and async methods. 90% of the time the only difference is that you need to remember that this method is async and do not forget to use await keyword when calling this method. And yes, you can't call async method from normal ones. And all this async - await syntactic garbage all around the program is only to indicate that this method is able to yield control to message loop.
Threads approach is free of all these inconveniences. But async - await approach allows to process much more concurrent tasks than threads approach does. How is that possible?
For each concurrent task we still have a call stack, only now it is a coroutine call stack. I am not quite sure, but looks like this is the key difference: usual stacks are operating-system stacks, they are expensive, coroutine stacks are just a python structures, they are much cheaper. Is this my understanding correct?
If this is correct, wouldn't it be better to decouple python threads/call stacks from OS threads/call stacks to make python threads cheaper?
Sorry if this question is stupid. I am sure there are some reasons why async-await approach was selected. Just want to understand these reasons.
Update:
For those who do not think this question is not good and too broad.
Here is an article Unyielding - which starts with explanations why threads are bad and advertises async approach. Main thesis: threads are evil, it's too difficult to reason about a routine that may be executed from arbitrary number of threads concurrently.
Thanks to Nathaniel J. Smith (author of python Trio library) who suggested this link.
By the way, arguments in the article are not convincing for me, but still may be useful.

This article answers your questions.
TL;DR?
Threading in Python is inefficient because of the GIL (Global Interpreter Lock) which means that multiple threads cannot be run in parallel as you would expect on a multi-processor system. Plus you have to rely on the interpreter to switch between threads, this adds to the inefficiency.
asyc/asyncio allows concurrency within a single thread. This gives you, as the developer, much more fine grained control of the task switching and can give much better performance for concurrent I/O bound tasks than Python threading.
The 3rd approach that you don't mention is multiprocessing. This approach uses processes for concurrency and allows programs to make full use of hardware with multiple cores.

Asyncio is a wholly different world, and AFAIK it's the answer of python to node.js which does this things since the start. E.g. this official python doc about asyncio states:
Asynchronous programming is different than classical “sequential” programming
So you'd need to decide if you want to jump into that rabbit hole and learn this terminology. It probably only makes sense if you're faced with either network or disk related heavy tasks. If you are then e.g. this article claims that python 3's asyncio might be faster than node.js and close to the performance of Go.
That said: I've not used asyncio yet, so I cannot really commment on this, but I can comment on a few sentences from your question:
And all this async - await syntactic garbage all around the program is only to indicate that this method is able to yield control to message loop
As far as I can see you have an initial setup of asyncio, but then all the calls have less syntax around it than doing the same things with threads which you need to start() and join() and probably also to check with is_alive(), and to fetch the return value you need to set up a shared object first. So: no, asyncio just looks different but in the end the program will most probably look cleaner than with threads.
As I understand the only problem with this approach is that threads are expensive
Not really. Starting a new thread is very inexpensive and has AFAIK the same cost as starting a "native thread" in C or Java
looks like this is the key difference: usual stacks are operating-system stacks, they are expensive, coroutine stacks are just a python structures, they are much cheaper. Is this my understanding correct?
Not really. Nothing beats creating OS level threads, they are cheap. What asyncio is better at is that you need less thread switches. So if you have many concurrent threads waiting for network or disk then asyncio would probably speed up things.

How is "multiplexed async architecture" asynchronous?

From Python in a Nutshell
In what is sometimes known as a multiplexed async
architecture, your code keeps track of the I/O channels on which
operations may be pending; when you can do no more until one or more
of the pending I/O operations completes, the thread running your
code goes into a blocking wait (this situation is usually referred to
as “your code blocks”), specifically waiting for any completion on
the relevant set of channels. When a completion wakes up the blocking
wait, your code deals with the specifics of that completion (such
“dealing with” may include initiating more I/O operations),
then, usually, goes back to the blocking wait. Python offers
several low-level modules supporting multiplexed async architectures,
but the best one to use is the higher-level selectors module
The meaning of "multiplexed" in "multiplexed async" can't be
explained by its definition from
http://searchnetworking.techtarget.com/definition/multiplexing
Multiplexing (or muxing) is a way of sending multiple signals or streams of information over a communications link at the same time
in the form of a single, complex signal; the receiver recovers the
separate signals, a process called demultiplexing (or demuxing).
What does "multiplexed" mean in "multiplexed async architecture"?
Given that "when you can do no more until one or more of the pending I/O operations completes, the thread running your code goes into a blocking wait", how is "multiplexed async architecture" asynchronous?

The meaning of "multiplexed" in "multiplexed async" can't be explained by its definition
Yes, but it's pretty close. The meaning is:
Multiplexing is a way of performing multiple operations using a single thread at the same time.
What this architecture does is similar to time-division multiplexing.
Given that "when you can do no more until one or more of the pending I/O operations completes, the thread running your code goes into a blocking wait", how is "multiplexed async architecture" asynchronous?
Wikipedia defines "asynchrony" as:
Asynchrony […] refers to the occurrence of […] actions instigated by a program that take place concurrently with program execution, without the program blocking to wait for results.
That is what's happening in this case. That blocking is used as an implementation detail doesn't matter. What matters is that while one operation is waiting, others can continue.