I get this error:
D:\pythonstuff\demo.py:28: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11.
await asyncio.wait([
Waited 1 second!
Waited 5 second!
Time passed: 0hour:0min:5sec
Process finished with exit code 0
When I run the code:
import asyncio
import time
class class1():
async def function_inside_class(self):
await asyncio.sleep(1)
print("Waited 1 second!")
async def function_inside_class2(self):
await asyncio.sleep(5)
print("Waited 5 second!")
def tic():
global _start_time
_start_time = time.time()
def tac():
t_sec = round(time.time() - _start_time)
(t_min, t_sec) = divmod(t_sec,60)
(t_hour,t_min) = divmod(t_min,60)
print('Time passed: {}hour:{}min:{}sec'.format(t_hour,t_min,t_sec))
object = class1()
async def main():
tic()
await asyncio.wait([
object.function_inside_class(),
object.function_inside_class2()
])
tac()
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
loop.close()
Are there any good alternatives to asyncio.wait? I don't want a warning in console every time I launch my application.
Edit: I don't want to just hide the error, that's bad practice, and I'm looking for other ways to do the same or a similar thing, not another async library to restore the old functionality.
You can just call it this way as it recommends in the docs here
Example from the docs:
async def foo():
return 42
task = asyncio.create_task(foo())
done, pending = await asyncio.wait({task})
So your code would become:
await asyncio.wait([
asyncio.create_task(object.function_inside_class()),
asyncio.create_task(object.function_inside_class2())
])
Related
I was experimenting with some basic constructs in python asyncio. I came across the following scenarios:
Snippet 1
import asyncio
async def A():
await asyncio.sleep(10)
print("A...")
async def B():
await asyncio.sleep(15)
print("B...")
async def main():
t = asyncio.create_task(A())
n = asyncio.create_task(B())
await n
print("CCDD...")
await t
asyncio.run(main())
Snippet 2
import asyncio
async def A():
await asyncio.sleep(10)
print("A...")
async def B():
await asyncio.sleep(15)
print("B...")
async def main():
t = asyncio.create_task(A())
n = asyncio.create_task(B())
await n
await t
print("CCDD...")
asyncio.run(main())
Snippet 3
import asyncio
async def A():
await asyncio.sleep(10)
print("A...")
async def B():
await asyncio.sleep(15)
print("B...")
async def main():
t = asyncio.create_task(A())
n = asyncio.create_task(B())
print("CCDD...")
await n
await t
asyncio.run(main())
I find it difficult to understand how the output produced by the first two snippets above is the same, but the output produced by the last snippet is different than the first two?
Output of snippets 1, 2
A...
B...
CCDD...
Output of snippet 3
CCDD...
A...
B...
It's all a matter of thinking about the sequencing. First off, n/15/B is always the 15-second task and t/10/A is the 10-second one. For all snippets, that means A will be printed before B, as you start them at roughly the same time.
In snippet 1, you start them both then wait for the 15-second task, meaning that both will finish before main prints CCDD (waiting for the 10-second task after that but it's already finished). Hence you see A B CCDD.
In snippet 2, you wait for the both the 15-second and 10-second task to finish before main prints CCDD, resulting in A B CCDD.
In snippet 3, you start both tasks then immediately print CCDD before waiting for them both. This gives you CCDD A B.
I can't figure how to stop loop after one task is finished. In sample when WsServe count to 5 I expect loop to close. But instead stop I got RuntimeError: Cannot close a running event loop
#!/usr/bin/env python
import asyncio
async def rxer():
i=0
while True:
i+=1
print ('Rxer ',i)
await asyncio.sleep(1)
async def WsServe():
for i in range(5):
print ('WsServe',i)
await asyncio.sleep(1)
print ('Finish')
loop.stop()
loop.close()
loop=asyncio.get_event_loop()
loop.create_task(rxer())
loop.run_until_complete(WsServe())
loop.run_forever()
The error comes from calling loop.close() from inside the loop. You don't need to bother with loop.close(), loop.stop() is quite sufficient to stop the loop. loop.close() is only relevant when you want to ensure that all the resources internally acquired by the loop are released. It is not needed when your process is about to exit anyway, and removing the call to loop.close() indeed eliminates the error.
But also, loop.stop() is incompatible with run_until_complete(). It happens to work in this code because the coroutine returns immediately after calling loop.stop(); if you added e.g. an await asyncio.sleep(1) after loop.stop(), you'd again get a (different) RuntimeError.
To avoid such issues, I suggest that you migrate to the newer asyncio.run API and avoid both run_until_complete and stop. Instead, you can just use an event to terminate the main function, and the loop with it:
# rxer() defined as before
async def WsServe(stop_event):
for i in range(5):
print ('WsServe',i)
await asyncio.sleep(1)
print ('Finish')
stop_event.set()
await asyncio.sleep(1)
async def main():
asyncio.get_event_loop().create_task(rxer())
stop_event = asyncio.Event()
asyncio.get_event_loop().create_task(WsServe(stop_event))
await stop_event.wait()
asyncio.run(main())
# python 3.6 and older:
#asyncio.get_event_loop().run_until_complete(main())
Check commented lines of your implementation as below:
import asyncio
async def rxer():
i=0
while True:
i+=1
print ('Rxer ',i)
await asyncio.sleep(1)
async def WsServe():
for i in range(5):
print ('WsServe',i)
await asyncio.sleep(1)
print ('Finish')
#loop.stop()
#loop.close()
loop=asyncio.get_event_loop()
loop.create_task(rxer())
loop.run_until_complete(WsServe())
#loop.run_forever()
And here is the output:
Rxer 1
WsServe 0
Rxer 2
WsServe 1
Rxer 3
WsServe 2
Rxer 4
WsServe 3
Rxer 5
WsServe 4
Rxer 6
Finish
I want speed up some API requests... for that I try to figure out how to do and copy some code which run but when I try my own code its no longer asynchrone. Maybe someone find the fail?
Copy Code (guess from stackoverflow):
#!/usr/bin/env python3
import asyncio
#asyncio.coroutine
def func_normal():
print('A')
yield from asyncio.sleep(5)
print('B')
return 'saad'
#asyncio.coroutine
def func_infinite():
for i in range(10):
print("--%d" % i)
return 'saad2'
loop = asyncio.get_event_loop()
tasks = func_normal(), func_infinite()
a, b = loop.run_until_complete(asyncio.gather(*tasks))
print("func_normal()={a}, func_infinite()={b}".format(**vars()))
loop.close()
My "own" code (I need at the end a list returned and merge the results of all functions):
import asyncio
import time
#asyncio.coroutine
def say_after(start,count,say,yep=True):
retl = []
if yep:
time.sleep(5)
for x in range(start,count):
retl.append(x)
print(say)
return retl
def main():
print(f"started at {time.strftime('%X')}")
loop = asyncio.get_event_loop()
tasks = say_after(10,20,"a"), say_after(20,30,"b",False)
a, b = loop.run_until_complete(asyncio.gather(*tasks))
print("func_normal()={a}, func_infinite()={b}".format(**vars()))
loop.close()
c = a + b
#print(c)
print(f"finished at {time.strftime('%X')}")
main()
Or I m completly wrong and should solve that with multithreading? What would be the best way for API requests that returns a list that I need to merge?
Added comment for each section that needs improvement. Removed some to simply code.
In fact, I didn't find any performance uplift with using range() wrapped in coroutine and using async def, might worth with heavier operations.
import asyncio
import time
# #asyncio.coroutine IS DEPRECATED since python 3.8
#asyncio.coroutine
def say_after(wait=True):
result = []
if wait:
print("I'm sleeping!")
time.sleep(5)
print("'morning!")
# This BLOCKs thread, but release GIL so other thread can run.
# But asyncio runs in ONE thread, so this still harms simultaneity.
# normal for is BLOCKING operation.
for i in range(5):
result.append(i)
print(i, end='')
print()
return result
def main():
start = time.time()
# Loop argument will be DEPRECATED from python 3.10
# Make main() as coroutine, then use asyncio.run(main()).
# It will be in asyncio Event loop, without explicitly passing Loop.
loop = asyncio.get_event_loop()
tasks = say_after(), say_after(False)
# As we will use asyncio.run(main()) from now on, this should be await-ed.
a, b = loop.run_until_complete(asyncio.gather(*tasks))
print(f"Took {time.time() - start:5f}")
loop.close()
main()
Better way:
import asyncio
import time
async def say_after(wait=True):
result = []
if wait:
print("I'm sleeping!")
await asyncio.sleep(2) # 'await' a coroutine version of it instead.
print("'morning!")
# wrap iterator in generator - or coroutine
async def asynchronous_range(end):
for _i in range(end):
yield _i
# use it with async for
async for i in asynchronous_range(5):
result.append(i)
print(i, end='')
print()
return result
async def main():
start = time.time()
tasks = say_after(), say_after(False)
a, b = await asyncio.gather(*tasks)
print(f"Took {time.time() - start:5f}")
asyncio.run(main())
Result
Your code:
DeprecationWarning: "#coroutine" decorator is deprecated since Python 3.8, use "async def" instead
def say_after(wait=True):
I'm sleeping!
'morning!
01234
01234
Took 5.003802
Better async code:
I'm sleeping!
01234
'morning!
01234
Took 2.013863
Note that fixed code now finish it's job while other task is sleeping.
I have several http requests to fire simultaneously. I am trying to use async for to do this.
import asyncio
async def ticker(delay, to):
for i in range(to):
yield i
print(i)
await asyncio.sleep(delay) # instead of aiohttp request
print(i, ' fin')
async def main():
async for x in ticker(1,2):
pass
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
I keep getting subsequent calls with the following:
0
0 fin
1
1 fin
Instead I need the output as shown below:
0
1
0 fin
1 fin
Could you please advise me on how to do this?
The problem is that async for is exactly what you don't need.
async for is designed to iterate while waiting for a task to complete between each iteration; you want to iterate (starting requests) without waiting for the previous task(s) to finish.
You'll want something like
async def do_request():
await asyncio.sleep(1)
async def main():
await asyncio.gather(*[
do_request() for i in range(10)
])
Comment with a follow-up if that doesn't answer your question.
I have 2 functions: The first one, def_a, is an asynchronous function and the second one is def_b which is a regular function and called with the result of def_a as a callback with the add_done_callback function.
My code looks like this:
import asyncio
def def_b(result):
next_number = result.result()
# some work on the next_number
print(next_number + 1)
async def def_a(number):
await some_async_work(number)
return number + 1
loop = asyncio.get_event_loop()
task = asyncio.ensure_future(def_a(1))
task.add_done_callback(def_b)
response = loop.run_until_complete(task)
loop.close()
And it's work perfectly.
The problem began when also the second function, def_b, became asynchronous. Now it looks like this:
async def def_b(result):
next_number = result.result()
# some asynchronous work on the next_number
print(next_number + 1)
But now I can not provide it to the add_done_callback function, because it's not a regular function.
My question is- Is it possible and how can I provide def_b to the add_done_callback function if def_b is asynchronous?
add_done_callback is considered a "low level" interface. When working with coroutines, you can chain them in many ways, for example:
import asyncio
async def my_callback(result):
print("my_callback got:", result)
return "My return value is ignored"
async def coro(number):
await asyncio.sleep(number)
return number + 1
async def add_success_callback(fut, callback):
result = await fut
await callback(result)
return result
loop = asyncio.get_event_loop()
task = asyncio.ensure_future(coro(1))
task = add_success_callback(task, my_callback)
response = loop.run_until_complete(task)
print("response:", response)
loop.close()
Keep in mind add_done_callback will still call the callback if your future raises an exception (but calling result.result() will raise it).
This only works for one future job, if you have multiple async jobs, they will blocks each other, a better way is using asyncio.as_completed() to iterate future list:
import asyncio
async def __after_done_callback(future_result):
# await for something...
pass
async def __future_job(number):
await some_async_work(number)
return number + 1
loop = asyncio.get_event_loop()
tasks = [asyncio.ensure_future(__future_job(x)) for x in range(100)] # create 100 future jobs
for f in asyncio.as_completed(tasks):
result = await f
await __after_done_callback(result)
loop.close()
You can try the aiodag library. It's a very lightweight wrapper around asyncio that abstracts away some of the async plumbing that you usually have to think about. From this example you won't be able to tell that things are running asynchronously since it's just 1 task that depends on another, but it is all running async.
import asyncio
from aiodag import task
#task
async def def_b(result):
# some asynchronous work on the next_number
print(result + 1)
#task
async def def_a(number):
await asyncio.sleep(number)
return number + 1
async def main():
a = def_a(1)
b = def_b(a) # this makes task b depend on task a
return await b
loop = asyncio.get_event_loop()
asyncio.set_event_loop(loop)
response = loop.run_until_complete(main())