I have a blocking, non-async code like this:
def f():
def inner():
while True:
yield read()
return inner()
With this code the caller can choose when to stop the function to generate data. How to change this to async? This solution doesn't work:
async def f():
async def inner():
while True:
yield await coroutine_read()
return inner()
... because yield can't be used in async def functions. If i remove the async from the inner() signature, I can't use await anymore.
Upd:
Starting with Python 3.6 we have asynchronous generators and able to use yield directly inside coroutines.
As noted above, you can't use yield inside async funcs. If you want to create coroutine-generator you have to do it manually, using __aiter__ and __anext__ magic methods:
import asyncio
# `coroutine_read()` generates some data:
i = 0
async def coroutine_read():
global i
i += 1
await asyncio.sleep(i)
return i
# `f()` is asynchronous iterator.
# Since we don't raise `StopAsyncIteration`
# it works "like" `while True`, until we manually break.
class f:
async def __aiter__(self):
return self
async def __anext__(self):
return await coroutine_read()
# Use f() as asynchronous iterator with `async for`:
async def main():
async for i in f():
print(i)
if i >= 3:
break
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
1
2
3
[Finished in 6.2s]
You may also like to see other post, where StopAsyncIteration uses.
Related
I need to return a value in async function.
I tried to use synchronous form of return:
import asyncio
async def main():
for i in range(10):
return i
await asyncio.sleep(1)
print(asyncio.run(main()))
output:
0 [Finished in 204ms]
But it just return value of the first loop, which is not expexted. So changed the code as below:
import asyncio
async def main():
for i in range(10):
yield i
await asyncio.sleep(1)
for _ in main():
print(_)
output:
TypeError: 'async_generator' object is not iterable
by using async generator I am facing with this error. How can I return a value for every loop of async function?
Thanks
You need to use an async for which itself needs to be inside an async function:
async def get_result():
async for i in main():
print(i)
asyncio.run(get_result())
I have the following function to call s(c) every 24 hours.
def schedule_next_sync():
t = datetime.datetime.now()
t = t.replace(hour=0) + datetime.timedelta(hours=24)
def wrapper():
s(c)
schedule_next_sync()
tornado.ioloop.IOLoop.current().add_timeout(datetime.datetime.timestamp(t), wrapper)
However, s() will be changed to an async function.
async def s(c):
How to update schedule_next_sync for async function? Should run s() synchronously? Or change schedule_next_sync() to an async function?
Once s is async, you could use asyncio.sleep() instead of the lower-level add_timeout():
async def schedule_next_sync():
async def call_forever():
while True:
await asyncio.sleep(1)
await s(c)
tornado.ioloop.IOLoop.current().create_task(call_forever())
If you really want to do it with timeouts, something like this should work:
def schedule_next_sync():
t = datetime.datetime.now() + datetime.timedelta(seconds=1)
def wrapper():
loop = asyncio.get_running_loop()
task = loop.create_task(s(c))
task.add_done_callback(lambda _: schedule_next_sync())
loop = tornado.ioloop.IOLoop.current()
loop.add_timeout(datetime.datetime.timestamp(t), wrapper)
I am implementing an asynchronous iterator to be used with async for which should return a new value at a (mostly) regular interval.
We can illustrate such iterator with a simple clock that will increment a counter every ~n seconds:
import asyncio
class Clock(object):
def __init__(self, interval=1):
self.counter = 0
self.interval = interval
self.tick = asyncio.Event()
asyncio.ensure_future(self.tick_tock())
async def tick_tock(self):
while True:
self.tick.clear()
await asyncio.sleep(self.interval)
self.counter = self.__next__()
self.tick.set()
def __next__(self):
self.counter += 1
return self.counter
def __aiter__(self):
return self
async def __anext__(self):
await self.tick.wait()
return self.counter
Is there a better or cleaner approach than using asyncio.Event? More than one coroutine will async for on this iterator.
In my opinion, your approach is fine. Note that since python 3.6, you can also use asynchronous generators:
async def clock(start=0, step=1, interval=1.):
for i in count(start, step):
yield i
await asyncio.sleep(interval)
However, you won't be able to share them between multiple coroutines. You would have to run the clock in a task and make the data available through an asynchronous iteration interface, which is essentially what you did in your code. Here's a possible implementation.
If you're using Python 3.6+ you can use asynchronous generators which are more readable.
async def Clock(interval=1):
counter = 0
while True:
await asyncio.sleep(interval)
counter += 1
yield counter
async def main():
async for i in Clock(1):
print(i)
if i == 4:
break
if __name__ == '__main__':
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(main())
finally:
loop.run_until_complete(loop.shutdown_asyncgens())
loop.close()
I have 2 functions: The first one, def_a, is an asynchronous function and the second one is def_b which is a regular function and called with the result of def_a as a callback with the add_done_callback function.
My code looks like this:
import asyncio
def def_b(result):
next_number = result.result()
# some work on the next_number
print(next_number + 1)
async def def_a(number):
await some_async_work(number)
return number + 1
loop = asyncio.get_event_loop()
task = asyncio.ensure_future(def_a(1))
task.add_done_callback(def_b)
response = loop.run_until_complete(task)
loop.close()
And it's work perfectly.
The problem began when also the second function, def_b, became asynchronous. Now it looks like this:
async def def_b(result):
next_number = result.result()
# some asynchronous work on the next_number
print(next_number + 1)
But now I can not provide it to the add_done_callback function, because it's not a regular function.
My question is- Is it possible and how can I provide def_b to the add_done_callback function if def_b is asynchronous?
add_done_callback is considered a "low level" interface. When working with coroutines, you can chain them in many ways, for example:
import asyncio
async def my_callback(result):
print("my_callback got:", result)
return "My return value is ignored"
async def coro(number):
await asyncio.sleep(number)
return number + 1
async def add_success_callback(fut, callback):
result = await fut
await callback(result)
return result
loop = asyncio.get_event_loop()
task = asyncio.ensure_future(coro(1))
task = add_success_callback(task, my_callback)
response = loop.run_until_complete(task)
print("response:", response)
loop.close()
Keep in mind add_done_callback will still call the callback if your future raises an exception (but calling result.result() will raise it).
This only works for one future job, if you have multiple async jobs, they will blocks each other, a better way is using asyncio.as_completed() to iterate future list:
import asyncio
async def __after_done_callback(future_result):
# await for something...
pass
async def __future_job(number):
await some_async_work(number)
return number + 1
loop = asyncio.get_event_loop()
tasks = [asyncio.ensure_future(__future_job(x)) for x in range(100)] # create 100 future jobs
for f in asyncio.as_completed(tasks):
result = await f
await __after_done_callback(result)
loop.close()
You can try the aiodag library. It's a very lightweight wrapper around asyncio that abstracts away some of the async plumbing that you usually have to think about. From this example you won't be able to tell that things are running asynchronously since it's just 1 task that depends on another, but it is all running async.
import asyncio
from aiodag import task
#task
async def def_b(result):
# some asynchronous work on the next_number
print(result + 1)
#task
async def def_a(number):
await asyncio.sleep(number)
return number + 1
async def main():
a = def_a(1)
b = def_b(a) # this makes task b depend on task a
return await b
loop = asyncio.get_event_loop()
asyncio.set_event_loop(loop)
response = loop.run_until_complete(main())
I want to use generator yield and async functions. I read this topic, and wrote next code:
import asyncio
async def createGenerator():
mylist = range(3)
for i in mylist:
await asyncio.sleep(1)
yield i*i
async def start():
mygenerator = await createGenerator()
for i in mygenerator:
print(i)
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(start())
except KeyboardInterrupt:
loop.stop()
pass
But i got the error:
SyntaxError: 'yield' inside async function
How to use yield generator in async function?
Upd:
Starting with Python 3.6 we have asynchronous generators and able to use yield directly inside coroutines.
import asyncio
async def async_generator():
for i in range(3):
await asyncio.sleep(1)
yield i*i
async def main():
async for i in async_generator():
print(i)
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(main())
finally:
loop.run_until_complete(loop.shutdown_asyncgens()) # see: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.shutdown_asyncgens
loop.close()
Old answer for Python 3.5:
You can't yield inside coroutines. Only way is to implement Asynchronous Iterator manually using __aiter__/__anext__ magic methods. In your case:
import asyncio
class async_generator:
def __init__(self, stop):
self.i = 0
self.stop = stop
async def __aiter__(self):
return self
async def __anext__(self):
i = self.i
self.i += 1
if self.i <= self.stop:
await asyncio.sleep(1)
return i * i
else:
raise StopAsyncIteration
async def main():
async for i in async_generator(3):
print(i)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
0
1
4
Here're two more examples: 1, 2
New Python 3.6 comes with support for asynchronous generators.
PEP 0525
What's new in Python 3.6
PS: On the moment of writing Python 3.6 is still beta. If you are on GNU/Linux or OS X and you cannot wait you can try new Python with pyenv.
This should work with python 3.6 (tested with 3.6.0b1):
import asyncio
async def createGenerator():
mylist = range(3)
for i in mylist:
await asyncio.sleep(1)
yield i*i
async def start():
async for i in createGenerator():
print(i)
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(start())
except KeyboardInterrupt:
loop.stop()
pass