How do I run asynchronous function in blocking style? I am not allowed to modify signature of my mock function f1() and can't easy switch to async def and so can't use await expression.
async def cc2():
await asyncio.sleep(1)
await asyncio.to_thread(print, "qwerty")
def f1():
t = asyncio.create_task(cc2())
# wait until the task finished before returning
async def main():
f1() # typical call from many places of a program
asyncio.run(main())
I tried asyncio.get_running_loop().run_until_complete(t), but the hack does not work and I get the next error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "<stdin>", line 2, in main
File "<stdin>", line 3, in f1
File "/usr/lib/python3.9/asyncio/base_events.py", line 618, in run_until_complete
self._check_running()
File "/usr/lib/python3.9/asyncio/base_events.py", line 578, in _check_running
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
How do I run asynchronous function in blocking style?
If you are in sync code, you can call it with asyncio.run(async_function()). If you are in async code, you can await async_function() or asyncio.create_task(async_function()), with the latter scheduling it to run in the background. You are not allowed to use asyncio.run() (or run_until_complete, even with a newly created event loop object) inside async code because it blocks and could halt the outer event loop.
But if you need it for testing purposes, you can always do something like:
# XXX dangerous - could block the current event loop
with concurrent.futures.ThreadPoolExecutor() as pool:
pool.submit(asyncio.run, async_function()).result()
Related
I have a web site using Tornado. And I need to call a function (do_work() in the following exmaple) periodically.
import asyncio
import datetime
from typing import Any, Callable, Coroutine
import tornado
import tornado.ioloop
def schedule_next_sync(loop, func, seconds):
time = datetime.datetime.timestamp(
datetime.datetime.now() + datetime.timedelta(seconds=seconds)
)
def wrapper():
loop.run_until_complete(func())
schedule_next_sync(loop, func, seconds)
print("The next run is sheduled for %s", datetime.datetime.fromtimestamp(time))
tornado.ioloop.IOLoop.current().add_timeout(time, wrapper)
async def do_work(x):
"""To be run periodically"""
print(f"...{datetime.datetime.now()}: {x}")
await asyncio.sleep(1)
print('....')
event_loop = asyncio.get_event_loop()
_func = lambda: do_work("test")
schedule_next_sync(event_loop, _func, 3) # every 3 seconds
tornado.ioloop.IOLoop.current().start()
However, the code got the following error?
The next run is sheduled for %s 2022-03-30 01:48:23.141079
ERROR:tornado.application:Exception in callback functools.partial(<function schedule_next_sync.<locals>.wrapper at 0x000001A7D6B99700>)
Traceback (most recent call last):
File "C:\users\xxx\anaconda3\envs\x\lib\site-packages\tornado\ioloop.py", line 741, in _run_callback
ret = callback()
File ".\test.py", line 23, in wrapper
loop.run_until_complete(func())
File "C:\users\xxx\anaconda3\envs\x\lib\asyncio\base_events.py", line 592, in run_until_complete
self._check_running()
File "C:\users\xxx\anaconda3\envs\x\lib\asyncio\base_events.py", line 552, in _check_running
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running
C:\users\xxx\anaconda3\envs\x\lib\site-packages\tornado\ioloop.py:761: RuntimeWarning: coroutine 'do_work' was never awaited
app_log.error("Exception in callback %r", callback, exc_info=True)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
You cannot use loop.run_until_complete() when the event loop is already running.
To schedule the execution of async code from sync code or to run async code concurrently, you need to use asyncio.create_task().
Your example works for me, if I change the wrapper to this:
def wrapper():
asyncio.create_task(func())
schedule_next_sync(loop, func, seconds)
So far I'm trying to load the website and the bot but how can I async await without using an async function in if __name__ == "__main__" so far I got this
from bot.bot import ConchBot
from web.main import flask_thread, run
import time
def run_bot():
bot = ConchBot()
bot.run()
def run_web():
run()
if __name__ == "__main__":
flask_thread(func=run)
print("------")
time.sleep(1)
print("------")
print("Loaded Website")
print("------")
run_bot()
But when I run it and press control + c it raises this error
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\threading.py", line 954, in _bootstrap_inner
self.run()
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\gateway.py", line 150, in run
f = asyncio.run_coroutine_threadsafe(coro, loop=self.ws.loop)
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\asyncio\tasks.py", line 931, in run_coroutine_threadsafe
loop.call_soon_threadsafe(callback)
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 791, in call_soon_threadsafe
self._check_closed()
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 510, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\threading.py:956: RuntimeWarning: coroutine 'DiscordWebSocket.send_heartbeat' was never awaited
self._invoke_excepthook(self)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
And I tried google it and tried using asyncio.run() by making def run_bot(): into async def run_bot(): and adding asyncio.run(run_bot()) it raises another error.
Traceback (most recent call last):
File "c:\Users\pushe\Documents\GitHub\UnsoughtedConchBot\launcher.py", line 20, in <module>
asyncio.run(run_bot())
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete
return future.result()
File "c:\Users\pushe\Documents\GitHub\UnsoughtedConchBot\launcher.py", line 8, in run_bot
bot.run()
File "c:\Users\pushe\Documents\GitHub\UnsoughtedConchBot\bot\bot.py", line 174, in run
super().run(TOKEN, reconnect=True)
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\client.py", line 719, in run
_cleanup_loop(loop)
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\client.py", line 95, in _cleanup_loop
loop.close()
File "C:\Users\pushe\AppData\Local\Programs\Python\Python39\lib\asyncio\proactor_events.py", line 674, in close
raise RuntimeError("Cannot close a running event loop")
RuntimeError: Cannot close a running event loop
If I try to run the bot first. The flask website doesn't load and when I press control+c it loads
How can I fix this?
I'm currently exploring async/await in python.
I'd like to start Pyrogram, then run the cronjob function. I created this code:
import asyncio
from pyrogram import Client, Filters, MessageHandler
import time
app = Client("pihome")
async def cronjob():
print("Cronjob started")
while True:
#print(f"time is {time.strftime('%X')}")
print(int(time.time()))
if int(time.strftime('%S')) == 10:
print("SECONDO 10") # Change this with checks in future
await asyncio.sleep(1)
async def main():
print(f"Starting {app.session_name}...\n")
# Pyrogram
await app.start()
print("Pyrogram started")
# Cronjob
loop = asyncio.new_event_loop()
loop.run_until_complete(loop.create_task(await cronjob()))
print("\nDone")
await Client.idle()
await app.stop()
print(f"\nStopping {app.session_name}...")
if __name__ == "__main__":
asyncio.get_event_loop().run_until_complete(main())
But when i want to stop it with Ctrl+C it gives me this traceback:
^CTraceback (most recent call last):
File "pihome.py", line 39, in <module>
asyncio.get_event_loop().run_until_complete(main())
File "/usr/local/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
self.run_forever()
File "/usr/local/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
self._run_once()
File "/usr/local/lib/python3.8/asyncio/base_events.py", line 1823, in _run_once
event_list = self._selector.select(timeout)
File "/usr/local/lib/python3.8/selectors.py", line 468, in select
fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt
How can I solve it? Also, try/except blocks seem not working
To run the application, it is better to use asyncio.run(main()). In my opinion, it’s more clear.
Replace loop = asyncio.new_event_loop() by loop = asyncio.get_event_loop(). Because using asyncio.new_event_loop() after asyncio.get_event_loop().run_until_complete(main()) you create a second event loop in main thread which is prohibited. Only one event loop per thread is permitted!
Remove await from this line:
loop.run_until_complete(loop.create_task(await cronjob()))
Because create_task need to Coroutine as the first argument but you pass it None
When I run the following python code:
import asyncio
import logging
logging.basicConfig(level=logging.DEBUG)
async def read_future(fut):
print(await fut)
async def write_future(fut):
fut.set_result('My Value')
async def main():
loop = asyncio.get_running_loop()
fut = loop.create_future()
asyncio.gather(read_future(fut), write_future(fut))
asyncio.run(main(), debug=True)
Instead of read_future waiting for the result of fut to be set, the program crashes with the following error:
DEBUG:asyncio:Using selector: KqueueSelector
ERROR:asyncio:_GatheringFuture exception was never retrieved
future: <_GatheringFuture finished exception=CancelledError() created at /Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/tasks.py:642>
source_traceback: Object created at (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/base_events.py", line 566, in run_until_complete
self.run_forever()
File "/Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/base_events.py", line 534, in run_forever
self._run_once()
File "/Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/base_events.py", line 1763, in _run_once
handle._run()
File "/Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "<stdin>", line 4, in main
File "/Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/tasks.py", line 766, in gather
outer = _GatheringFuture(children, loop=loop)
File "/Users/<user>/.pyenv/versions/3.7.4/lib/python3.7/asyncio/tasks.py", line 642, in __init__
super().__init__(loop=loop)
concurrent.futures._base.CancelledError
DEBUG:asyncio:Close <_UnixSelectorEventLoop running=False closed=False debug=True>
What am I doing wrong in this code? I want to be able to await the Future fut and continue after the Future has a value/exception set.
Your problem is that asyncio.gather is itself async (returns an awaitable); by not awaiting it, you never handed control back to the event loop, nor did you store the awaitable, so it was immediately cleaned up, implicitly cancelling it, and by extension, all of the awaitables it controlled.
To fix, just make sure you await the results of the gather:
await asyncio.gather(read_future(fut), write_future(fut))
Try it online!
From https://docs.python.org/3/library/asyncio-future.html#asyncio.Future.result:
result()
Return the result of the Future.
If the Future is done and has a result set by the set_result() method, the result value is returned.
If the Future is done and has an exception set by the set_exception() method, this method raises the exception.
If the Future has been cancelled, this method raises a CancelledError exception.
If the Future’s result isn’t yet available, this method raises a InvalidStateError exception.
(Bold added.)
I'm not sure why the Future is getting cancelled, but that seems to be the cause of the issue.
I'm trying to test some asyncio functionality on Windows. I'm using Python 3.7.
This complains bitterly about some NotImplementedError
import asyncio
import os
import time
import sys
#this works
async def say_after(delay, what):
await asyncio.sleep(delay)
print(what)
#this doesn't
async def test_async(num):
print('Task #{0} start'.format(num))
proc = await asyncio.create_subprocess_shell(
'C:/Python37/python test_repl.py',
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
stdin=asyncio.subprocess.PIPE )
stdout, stderr = await proc.communicate("sleep 10")
cmd = 'python'
print(f'[{cmd!r} exited with {proc.returncode}]')
if stdout:
print(f'[stdout]\n{stdout.decode()}')
if stderr:
print(f'[stderr]\n{stderr.decode()}')
async def test_loop():
task1 = asyncio.create_task(
test_async(1))
task2 = asyncio.create_task(
test_async(2))
print(f"started at {time.strftime('%X')}")
await task1
await task2
print(f"finished at {time.strftime('%X')}")
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
asyncio.run(test_loop())
I've tried several loop types (Protractor and Default). I've tried different functions through the shell and directly invoking the program (_shell and _exec in the subprocess lingo). Nothing seems to work. For a much hype I've read about asyncio, it can't possibly be a Linux only thing. I must be doing something wrong. Can you please point me in the right direction.
The specific error I get is the following:
--------------Async--------------------
started at 22:39:55
Task #1 start
Task #2 start
Traceback (most recent call last):
File "multirun.py", line 45, in <module>
asyncio.run(test_loop())
File "C:\Python37\lib\asyncio\runners.py", line 43, in run
return loop.run_until_complete(main)
File "C:\Python37\lib\asyncio\base_events.py", line 568, in run_until_complete
return future.result()
File "multirun.py", line 38, in test_loop
await task1
File "multirun.py", line 20, in test_async
stdin=asyncio.subprocess.PIPE)
File "C:\Python37\lib\asyncio\subprocess.py", line 202, in create_subprocess_shell
stderr=stderr, **kwds)
File "C:\Python37\lib\asyncio\base_events.py", line 1486, in subprocess_shell
protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs)
File "C:\Python37\lib\asyncio\base_events.py", line 444, in _make_subprocess_transport
raise NotImplementedError
NotImplementedError
Task exception was never retrieved
future: <Task finished coro=<test_async() done, defined at multirun.py:12> exception=NotImplementedError()>
Traceback (most recent call last):
File "multirun.py", line 20, in test_async
stdin=asyncio.subprocess.PIPE)
File "C:\Python37\lib\asyncio\subprocess.py", line 202, in create_subprocess_shell
stderr=stderr, **kwds)
File "C:\Python37\lib\asyncio\base_events.py", line 1486, in subprocess_shell
protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs)
File "C:\Python37\lib\asyncio\base_events.py", line 444, in _make_subprocess_transport
raise NotImplementedError
NotImplementedError
Also, I got the same error when trying to run the notepad.
As of Python 3.8, this issue should no longer exist, as the proactor event loop is now the default event loop on Windows. Original answer follows below.
The problem is that, despite appearances, you're not actually using the ProactorEventLoop. asyncio.run() creates a new event loop based on the current loop creation policy, which you've never changed. Creating a new loop for each run is normally a feature because it guarantees cleanup of the resources associated with the loop - but in this case it's incompatible with the example from the documentation. (Edit: the example has since been removed because ProactorEventLoop was made the default.)
A quick fix is to change asyncio.run(test_loop()) to loop.run_until_complete(test_loop()). A better fix is to set the event loop policy to one that creates the proactor loop.