I am using python 3.8 with this code
async def main():
pass
async def build():
pass
asyncio.create_task(build())
loop = asyncio.get_event_loop()
asyncio.create_task(main())
pending = asyncio.all_tasks()
loop.run_until_complete(asyncio.gather(*pending))
and get the following error
sys:1: RuntimeWarning: coroutine 'build' was never awaited
What am I missing here? shouldn't run until complete wait for all the tasks to finish?
As the loop is not running when creating the task, asyncio is unable to attach tasks to it.
This can be fixed by replacing asyncio.create_task() with loop.create_task().
The full error can be retrieved using asyncio.gather(..., return_exceptions=True) so gather() will raise a RuntimeError: no running event loop.
Related
Is there a way to call an async function from a sync one without waiting for it to complete?
My current tests:
Issue: Waits for test_timer_function to complete
async def test_timer_function():
await asyncio.sleep(10)
return
def main():
print("Starting timer at {}".format(datetime.now()))
asyncio.run(test_timer_function())
print("Ending timer at {}".format(datetime.now()))
Issue: Does not call test_timer_function
async def test_timer_function():
await asyncio.sleep(10)
return
def main():
print("Starting timer at {}".format(datetime.now()))
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
asyncio.ensure_future(test_timer_function())
print("Ending timer at {}".format(datetime.now()))
Any suggestions?
Async functions really do not run in the background: they run always in a single thread.
That means that when there are parallel tasks in async code (normal async code), it is only when you give a chance to the asyncio loop to run that those are executed - this happens when your code uses await, call one of async for, async with or return from a co-routine function that is running as a task.
In non-async code, you have to enter the loop and pass control to it, in order to the async code to run - that is what asyncio.run does - and asyncio.ensure_future does not: this call just registers a task to be executed, whenever the asyncio loop has time for it: but you return from the function without ever passing control to the async loop, so your program just finishes.
One thing that can be done is to establish a secondary thread, where the asyncio code will run: this thread will run its asyncio loop, and you can communicate with tasks in it by using global variables and normal thread data structures like Queues.
The minimal changes for your code are:
import asyncio
import threading
from datetime import datetime
now = datetime.now
async def test_timer_function():
await asyncio.sleep(2)
print(f"ending async task at {now()}")
return
def run_async_loop_in_thread():
asyncio.run(test_timer_function())
def main():
print(f"Starting timer at {now()}")
t = threading.Thread(target=run_async_loop_in_thread)
t.start()
print(f"Ending timer at {now()}")
return t
if __name__ == "__main__":
t = main()
t.join()
print(f"asyncio thread exited normally at {now()}")
(please, when posting Python code, include the import lines and lines to call your functions and make your code actually run: it is not a lot of boiler plate like may be needed in other languages, and turn your snippets in complete, ready to run, examples)
printout when running this snippet at the console:
Starting timer at 2022-10-20 16:47:45.211654
Ending timer at 2022-10-20 16:47:45.212630
ending async task at 2022-10-20 16:47:47.213464
asyncio thread exited normally at 2022-10-20 16:47:47.215417
The answer is simply no. It's not gonna happen in a single thread.
First issue:
In your first issue, main() is a sync function. It stops at the line asyncio.run(test_timer_function()) until the event loop finishes its work.
What is its only task? test_timer_function! This task "does" give the control back to event loop but not to the caller main! So if the event loop had other tasks too, they would cooperate with each other. But within the tasks of the event loop, not between event loop and the caller.
So it will wait 10 seconds. There is no other one here to use this 10 seconds to do its work.
Second issue:
You didn't even run the event loop. Check documentation for ensure_future.
I experimenting with the new asyncio features in Python 3.9, and have the following code:
import asyncio
async def inc(start):
i = start
while True:
print(i)
i += 2
return None
async def main():
x = asyncio.gather(
asyncio.to_thread(inc, 0),
asyncio.to_thread(inc, 1)
)
try:
await asyncio.wait_for(x, timeout=5.0)
except asyncio.TimeoutError:
print("timeout!")
asyncio.run(main())
My expectation is that the program will print numbers starting from 0 and terminate after 5 seconds. However, when I execute the program here, I see no output and the following warning:
/usr/lib/python3.9/asyncio/events.py:80: RuntimeWarning: coroutine 'inc' was never awaited
self._context.run(self._callback, *self._args)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
/usr/lib/python3.9/concurrent/futures/thread.py:85: RuntimeWarning: coroutine 'inc' was never awaited
del work_item
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
The asyncio.to_thread converts a regular function to a coroutine.
All you need is to change the async def inc to def inc. You can do this, because there is no await, it is not a real coroutine.
However, you cannot kill a thread. The timeout will cause to stop waiting, not to stop computing numbers.
UPDATE: In detail, the asyncio.gather awaits the coroutines until the timeout. When cancelled by the timeout, the gather cancels all still running coroutines a and waits for their termination. Cancelling a coroutine means to deliver an exception at the nearest await statement. In this case, there is no await and the coroutines will never receive the cancellation exception. The program hangs at that point.
I'm coding a telegram userbot (with telethon) which sends a message,every 60 seconds, to some chats.
I'm using 2 threads but I get the following errors: "RuntimeWarning: coroutine 'sender' was never awaited" and "no running event loop".
My code:
....
async def sender():
for chat in chats :
try:
if chat.megagroup == True:
await client.send_message(chat, messaggio)
except:
await client.send_message(myID, 'error')
schedule.every(60).seconds.do(asyncio.create_task(sender()))
...
class checker1(Thread):
def run(self):
while True:
schedule.run_pending()
time.sleep(1)
class checker2(Thread):
def run(self):
while True:
client.add_event_handler(handler)
client.run_until_disconnected()
checker2().start()
checker1().start()
I searched for a solution but I didn't find anything...
You should avoid using threads with asyncio unless you know what you're doing. The code can be rewritten using asyncio as follows, since most of the time you don't actually need threads:
import asyncio
async def sender():
for chat in chats :
try:
if chat.megagroup == True:
await client.send_message(chat, messaggio)
except:
await client.send_message(myID, 'error')
async def checker1():
while True:
await sender()
await asyncio.sleep(60) # every 60s
async def main():
await asyncio.create_task(checker1()) # background task
await client.run_until_disconnected()
client.loop.run_until_complete(main())
This code is not perfect (you should properly cancel and wait checker1 at the end of the program), but it should work.
As a side note, you don't need client.run_until_disconnected(). The call simply blocks (runs) until the client is disconnected. If you can keep the program running differently, as long as asyncio runs, the client will work.
Another thing: bare except: are a very bad idea, and will probably cause issues with exception. At least, replace it with except Exception.
There are a few problems with your code. asyncio is complaining about "no running event loop" because your program never starts the event loop anywhere, and tasks can't be scheduled without an event loop running. See Asyncio in corroutine RuntimeError: no running event loop. In order to start the event loop, you can use asyncio.run_until_complete() if you have a main coroutine for your program, or you can use asyncio.get_event_loop().run_forever() to run the event loop forever.
The second problem is the incorrect usage of schedule.every(60).seconds.do(), which is hidden by the first error. schedule expects a function to be passed in, not an awaitable (which is what asyncio.create_task(sender()) returns). This normally would have caused a TypeError, but the create_task() without a running event loop raised an exception first, so this exception was never raised. You'll need to define a function and then pass it to schedule, like this:
def start_sender():
asyncio.create_task(sender())
schedule.every(60).seconds.do(start_sender)
This should work as long as the event loop is started somewhere else in your program.
The problem I think I am having is related to the event loops and calling cancel from a different event loop than it was created in. The code that I think is causing the issue is the synchronous method that is passed to the client library that connects to an external source, when the external source loses connection it calls this method.
The problem then is that the stop and start methods are both async and on their own work fine the start method is created in a task and it waits for the sleep to end then calls stop which cancels the task and seems to work fine, from the sync method I need to create a new event loop to call stop which also works but while you can call cancel the task never seems to close and I get the desired result of reconnect only after the sleep has finished. If I print the self._task variable before and after it would show that it has been cancelled but it clearly keeps running.
<Task pending coro=<AsyncAKSServer.start() running at aks_server.py:88> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7f551e268f18>()]> cb=[<TaskWakeupMethWrapper object at 0x7f551e268e58>()]>
<Task pending coro=<AsyncAKSServer.start() running at aks_server.py:88> wait_for=<Future cancelled> cb=[<TaskWakeupMethWrapper object at 0x7f551e268e58>()]>
Unfortunately I can't give the exact code for IP reasons but below is a sample that shows what I am trying to do, I am unable to update the client library that the sync method is sent to currently as we are in out busy period so changing that to async isn't an option. There is a real chance I have misunderstood the docs and how this is supposed to work any help is appreciated.
import asyncio
class AsyncServer:
def __init__(self):
self._task = None
self._duration = 1
async def run(self):
while True:
self._task = asyncio.create_task(self.start())
try:
await self._task
except asyncio.CancelledError:
print('Start task cancelled')
async def start(self):
await asyncio.sleep(self._duration)
await self.stop()
async def stop(self):
self._taks.cancel()
def sync_request(self):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.run_until_complete(self.stop())
loop.stop()
loop.close()
def main():
server = AsyncServer()
asyncio.run(server.run())
This problem is indeed caused by having to create a new event loop, the fix for this is to store a reference to the event loop when in the start method. You can then use this to call asyncio.run_coroutine_threadsafe(self.stop(), self._loop) which passes the same loop in and makes the cancel work as expected.
I'm having trouble fixing and understanding the problem here. I'm using an example to learn Asyncio but the code I'm using is similar to mine but mine gives an error message saying:
sys:1: RuntimeWarning: coroutine 'run_script' was never awaited
Please any help will be greatly appreciated. Below is my code
async def run_script(script):
print("Run", script)
await asyncio.sleep(1)
os.system("python " + script)
and I'm running it like this
for script in os.listdir():
if script.endswith(".py"):
scripts.append(run_script(script))
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(scripts))
loop.close()
As #dim mentioned what's the typo in your code, you also need to be aware os.system is running in synchronous, which means scripts in your folder will be run in sequence instead of in asynchronous way.
To understand that, add file called hello_world.py:
import time
time.sleep(2)
print('hello world')
if you run you script as follows, it will cost you 2s + 2s = 4s:
loop = asyncio.get_event_loop()
loop.run_until_complete(
asyncio.gather(
*[run_script('hello_world.py') for _ in range(2)]
)
)
So to solve this issue, you can use asyncio.subprocess module:
from asyncio import subprocess
async def run_script(script):
process = await subprocess.create_subprocess_exec('python', script)
try:
out, err = await process.communicate()
except Exception as err:
print(err)
Then it will cost you only 2 sec, because it is running asynchronously.