Develop Reference

Python-asyncio - using timers/threading to terminate async

I have an async coroutine that I want to terminate using a timer/thread. The coroutine is based of this example from aiortc.
args = parse_args()
client = Client(connection, media, args.role)
# run event loop
loop = asyncio.get_event_loop()
try:
timer = None
if args.timeout:
print("Timer started")
timer = threading.Timer(args.timeout, loop.run_until_complete, args=(client.close(),))
timer.start()
loop.run_until_complete(client.run())
if timer:
timer.join()
except KeyboardInterrupt:
pass
finally:
# cleanup
loop.run_until_complete(client.close())
This does not work and raises RuntimeError('This event loop is already running')
Why does this raise the error? My guess is that it's because the loop is running on a different thread. However, creating a new loop doesn't work as it gets a future attached to a different loop.
def timer_callback():
new_loop = asyncio.new_event_loop()
new_loop.run_until_complete(client.close())
Following that, how can I use a timer to end the script?

Following that, how can I use a timer to end the script?
You can call asyncio.run_coroutine_threadsafe() to submit a coroutine to an event loop running in another thread:
if args.timeout:
print("Timer started")
timer = threading.Timer(
args.timeout,
asyncio.run_coroutine_threadsafe,
args=(client.close(), loop),
)
timer.start()
Note, however, that since you're working with asyncio, you don't need a dedicated thread for the timer, you could just create a coroutine and tell it to wait before doing something:
if args.timeout:
print("Timer started")
async def close_after_timeout():
await asyncio.sleep(args.timeout)
await client.close()
loop.create_task(close_after_timeout())

This isn't the general solution that I was looking for, I added a timeout variable to the client constructor, and within client.run() I added asyncio.sleep(timeout) which would exit the loop. This is enough for my purposes.

Python asyncio loop already running when using asyncio.run and trying to add tasks

I am very new to asyncio and I find the EchoServer example very confusing. I am trying to achieve a simple situation where a server accepts multiple clients, runs that in a coroutine and handles data, and a UI thread which handles ncurses input. I currently have the following, which, in code, conveys the idea I think. But it does not work.
import asyncio
async def do_ui():
await asyncio.sleep(1)
print('doing')
await do_ui()
async def run_game():
loop = asyncio.get_running_loop()
server = await GameServer.create_server()
loop.create_task(server.serve_forever())
loop.create_task(do_ui())
loop.run_forever()
def run():
asyncio.run(run_game())
The problem starts in GameServer.create_server, where I, for encapsulation reasons, want to delegate creating the server to. However this is an asynchronous action (for some reason) and so has to be awaited. See the server code below:
class GameServer:
#staticmethod
async def create_server():
loop = asyncio.get_running_loop()
return await loop.create_server(
lambda: GameServerProtocol(),
'127.0.0.1', 8888
)
This forces me to make run_game async aswell and await it in the run method, which is my setup.py entrypoint, so I can't do that. Using the asyncio.run method however starts a different event loop and I am not able to access it anymore.
How do I solve this? And to vent my frustration, how is this in any way easier than just using threads?

You cannot use loop.run_forever() whilst the event loop is already running. For example, the following code will not work:
import asyncio
async def main():
loop=asyncio.get_running_loop()
loop.run_forever()
if __name__ == '__main__':
asyncio.run(main())
But this code will work, and has the "run forever" behaviour you appear to be looking for:
import asyncio
async def do_ui():
while True:
await asyncio.sleep(1)
print('doing ui')
async def main():
loop = asyncio.get_running_loop()
loop.create_task(do_ui())
# insert game server code here
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.create_task(main())
loop.run_forever()

Is there a better way to catch `KeyboardInterrupt` than an infinite loop with asyncio?

A program I am developing has a long running process in another thread. I would like to interrupt that thread in the event something goes awry.
Of other SO posts I've seen like this, they use syntax similar to this:
while True:
if condition_here:
break
else:
await asyncio.sleep(1)
which does work in catching KeyboardInterrupts. However, I'm not a big fan of using while loops like this and would like to avoid this if at all possible.
For some example code, here is what I currently have (which does not catch the interrupts until after the thread is done):
import asyncio
import time
from threading import Thread
def some_long_process():
time.sleep(60)
async def main():
thread = Thread(target=some_long_process)
thread.start()
# Doesn't work
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, thread.join)
# await asyncio.wait([loop.run_in_executor(None, thread.join)])
# await asyncio.wait_for(loop.run_in_executor(None, thread.join), None)
# await asyncio.gather(asyncio.to_thread(thread.join))
# Works
# while thread.is_alive():
# await asyncio.sleep(1)
if __name__ == '__main__':
asyncio.run(main())
I'm also open to suggestions to reconsider my entire approach to the way this is designed if this isn't possible. Thanks for your time.

Trouble with Python3 Asyncio creating tasks

Have worked through most examples but am still learning async in Python. I am having trouble why this example of code will not print "i am async".
import asyncio
from threading import Thread
async def cor1():
print("i am async!")
def myasync(loop):
print("Async running")
loop.run_forever()
print("Async ended?")
def main():
this_threads_event_loop = asyncio.get_event_loop()
t_1 = Thread(target=myasync, args=(this_threads_event_loop,));
t_1.start()
print("begining the async loop")
t1 = this_threads_event_loop.create_task(cor1())
print("Finsihed cor1")
main()

Your code attempts to submit tasks to the event loop from a different thread. To do that, you must use run_coroutine_threadsafe:
def main():
loop = asyncio.get_event_loop()
# start the event loop in a separate thread
t_1 = Thread(target=myasync, args=(loop,));
t_1.start()
# submit the coroutine to the event loop running in the
# other thread
f1 = asyncio.run_coroutine_threadsafe(cor1(), loop)
# wait for the coroutine to finish, by asking for its result
f1.result()
print("Finsihed cor1")
Please note that combining asyncio and threads should only be done in special circumstances, such as when introducing asyncio to a legacy application where the new functionality needs to be added gradually. If you are writing new code, you almost certainly want the main to be a coroutine, run from top-level using asyncio.run(main()).
To run a legacy synchronous function from asyncio code, you can always use run_in_executor.

How can I package a coroutine as normal function in event loop?

I am using asyncio for a network framework.
In below code(low_level is our low level function, main block is our program entry, user_func is user-defined function):
import asyncio
loop = asyncio.get_event_loop()
""":type :asyncio.AbstractEventLoop"""
def low_level():
yield from asyncio.sleep(2)
def user_func():
yield from low_level()
if __name__ == '__main__':
co = user_func()
loop.run_until_complete(co)
I want wrap the low_level as normal function rather than coroutine(for compatibility etc.), but low_level is in event loop. How can wrap it as a normal function?

Because low_level is a coroutine, it can only be used by running an asyncio event loop. If you want to be able to call it from synchronous code that isn't running an event loop, you have to provide a wrapper that actually launches an event loop and runs the coroutine until completion:
def sync_low_level():
loop = asyncio.get_event_loop()
loop.run_until_complete(low_level())
If you want to be able to call low_level() from a function that is part of the running event loop, have it block for two seconds, but not have to use yield from, the answer is that you can't. The event loop is single-threaded; whenever execution is inside one of your functions, the event loop is blocked. No other events or callbacks can be processed. The only ways for a function running in the event loop to give control back to the event loop are to 1) return 2) use yield from. The asyncio.sleep call in low_level will never be able to complete unless you do one those two things.
Now, I suppose you could create an entirely new event loop, and use that to run the sleep synchronously from a coroutine running as part of the default event loop:
import asyncio
loop = asyncio.get_event_loop()
#asyncio.coroutine
def low_level(loop=None):
yield from asyncio.sleep(2, loop=loop)
def sync_low_level():
new_loop = asyncio.new_event_loop()
new_loop.run_until_complete(low_level(loop=new_loop))
#asyncio.coroutine
def user_func():
sync_low_level()
if __name__ == "__main__":
loop.run_until_complete(user_func())
But I'm really not sure why you'd want to do that.
If you just want to be able to make low_level act like a method returning a Future, so you can attach callbacks, etc. to it, just wrap it in asyncio.async():
loop = asyncio.get_event_loop()
def sleep_done(fut):
print("Done sleeping")
loop.stop()
#asyncio.coroutine
def low_level(loop=None):
yield from asyncio.sleep(2, loop=loop)
def user_func():
fut = asyncio.async(low_level())
fut.add_done_callback(sleep_done)
if __name__ == "__main__":
loop.call_soon(user_func)
loop.run_forever()
Output:
<2 second delay>
"Done sleeping"
Also, in your example code, you should use the #asyncio.coroutine decorator for both low_level and user_func, as stated in the asyncio docs:
A coroutine is a generator that follows certain conventions. For
documentation purposes, all coroutines should be decorated with
#asyncio.coroutine, but this cannot be strictly enforced.
Edit:
Here's how a user from a synchronous web framework could call into your application without blocking other requests:
#asyncio.coroutine
def low_level(loop=None):
yield from asyncio.sleep(2, loop=loop)
def thr_low_level():
loop = asyncio.new_event_loop()
t = threading.Thread(target=loop.run_until_complete, args(low_level(loop=loop),))
t.start()
t.join()
If a request being handled by Flask calls thr_low_level, it will block until the request is done, but the GIL should be released for all of the asynchronous I/O going on in low_level, allowing other requests to be handled in separate threads.