I need to have two function in my Python 3.11 code.
One function must be sync, it retrive some data from a local machine so i need to wait to finish.
Another function must be async, it get the data from the first function and send to the server. Since i don't know how many time can be (5 seconds to 30 seconds) this function must doesn't interrupt the first one
Pratically, the second function start always when the first finish but the first always start and don't care about the second one. This code run H24
My attempt:
import time
import asyncio
async def task1():
print("Recover data... waiting")
time.sleep(3)
print("End data recover")
return "slow"
async def task2(p):
print("I'm so" + p)
time.sleep(10)
print("END--->")
async def main():
while True:
print("create task1 and wait to finish")
x = await task1()
print("create task2 and not wait to finishing")
asyncio.create_task(task2(x))
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
loop.create_task(main())
loop.run_forever()
I dont' need to use asyncio like a requirement, i just want to meet the goal without run out all the memory of the machine. Thanks
basically, "yes". Like in: that is the way you do it. If you need some task to complete before goind on with code in a given place, the thing to do is to wait for that data - if it is in an async func, then you use the await keyword, just as depicted in the code above.
If there are other tasks in parallel to calling main, then it would be nice if the code in task1 would return execution to the asyncio.loop while it waits for its result - That code could run in another thread, with the use of await asyncio.run_in_executor(sync_call, args) or simply await asyncio.sleep(...)`
Related
I have an application which already runs infinitely with asyncio event loop run forever and also I need to run a specific function every 10 seconds.
def do_something():
pass
a = asyncio.get_event_loop()
a.run_forever()
I would like to call the function do_something every 10 seconds. How to achieve this without replacing asynctio event loop with while loop ?
Edited:
I can achieve this with the below code
def do_something():
pass
while True:
time.sleep(10)
do_something()
But I dont want to use while loop to run infinitely in my application instead I would like to go with asyncio run_forever(). So how to call the same function every 10 seconds with asyncio ? is there any scheduler like which will not block my ongoing work ?
asyncio does not ship with a builtin scheduler, but it is easy enough to build your own. Simply combine a while loop with asyncio.sleep to run code every few seconds.
async def every(__seconds: float, func, *args, **kwargs):
while True:
func(*args, **kwargs)
await asyncio.sleep(__seconds)
a = asyncio.get_event_loop()
a.create_task(every(1, print, "Hello World"))
...
a.run_forever()
Note that the design has to be slightly different if func is itself a coroutine or a long-running subroutine. In the former case use await func(...) and in the latter case use asyncio's thread capabilities.
You can achieve it with
async def do_something():
while True:
await asyncio.wait(10)
...rest of code...
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(do_something())
loop.run_forever()
Having read the documents and watched a number of videos, i am testing asyncio as an alternative to threading.
The docs are here:
https://docs.python.org/3/library/asyncio.html
I have constructed the following code with the expectation that it would produce the following.
before the sleep
hello
world
But in fact is produces this (world comes before hello):
before the sleep
world
hello
Here is the code:
import asyncio
import time
def main():
''' main entry point for the program '''
# create the event loop and add to the loop
# or run directly.
asyncio.run(main_async())
return
async def main_async():
''' the main async function '''
await foo()
await bar()
return
async def foo():
print('before the sleep')
await asyncio.sleep(2)
# time.sleep(0)
print('world')
return
async def bar():
print('hello')
await asyncio.sleep(0)
return
if __name__=='__main__':
''' This is executed when run from the command line '''
main()
The main() function calls the async main_async() function which in turn calls both the foo and bar async functions and both of those run the await asyncio.sleep(x) command.
So my question is: why is the hello world comming in the wrong (unexpected) order given that i was expecting world to be printed approximately 2 seconds after hello ?
You awaited foo() immediately, so bar() was never scheduled until foo() had run to completion; the execution of main_async will never do things after an await until the await has completed. If you want to schedule them both and let them interleave, replace:
await foo()
await bar()
with something like:
await asyncio.gather(foo(), bar())
which will convert both awaitables to tasks, scheduling both on the running asyncio event loop, then wait for both tasks to run to completion. With both scheduled at once, when one blocks on an await (and only await-based blocks, because only await yields control back to the event loop), the other will be allowed to run (and control can only return to the other task when the now running task awaits or finishes).
Basically, you have to remember that asyncio is cooperative multitasking. If you're only executing one task, and that task performs an await, there is nothing else to schedule, so nothing else runs until that await completes. If you block by any means other than an await, you still hold the event loop, and nothing else will get a chance to run, even if it's ready to go. So to gain any benefit from asyncio you need to be careful to:
Ensure other tasks are launched in time to occupy the event loop while the original task(s) are blocking on await.
Ensure you only block via await, so you don't monopolize the event loop unnecessarily.
I am pretty new with Async.io and I am using it with Discord.py to create a bot. Once a day, I need to update a spreadsheet, but the problem is that the spreadsheet has gotten a little long so it now triggers the loop's default timeout. Is there anyway to overcome this? I have seen run_until_complete but as you see below there is a await asyncio.sleep(86400) which from my understanding will not work with wait until complete because it will wait for a day? I would also be fine with just changing the timeout for that function and then changing it back after it is complete, but I have not been able to find any resources.
Here is the function that needs to repeat everyday:
async def updateSheet():
while True:
print("Updating Sheet at " + datetime.now().strftime("%H:%M"))
user.updateAllUsers(os.getenv('CID'), os.getenv('CS'), subs) #This is the function that takes too long
print("Done Updating")
await asyncio.sleep(86400)
and here is how I am adding it to the loop (because I am using Discord.py):
#client.event
async def on_ready():
print('We have logged in as {0.user}'.format(client))
client.loop.create_task(updateSheet())
Any and all help will be appreciated since as long as this is down my project loses precious time. :)
If something is blocking, the direct method would be trying to convert it to a task, which might not be possible in your case. So we would have to use something like APS to schedule jobs.
sched = Scheduler()
sched.start()
#sched.cron_schedule(day='mon-fri')
def task():
user.updateAllUsers(os.getenv('CID'), os.getenv('CS'), subs)
Make sure you do this in a separate file, and use async scheduler for tasks.
You can simply measure how much time does the function take to execute and simply subtract it from 86400
import time
async def updateSheet():
while True:
start = time.monotonic()
print("Updating Sheet at " + datetime.now().strftime("%H:%M"))
user.updateAllUsers(os.getenv('CID'), os.getenv('CS'), subs) #This is the function that takes too long
end = time.monotonic()
total = end - start
sleep_time = 86400 - total
await asyncio.sleep(sleep_time)
I really suggest you that you run the blocking functions in a non-blocking way, refer to one of my previous answers for more info, (What does "blocking" mean)
In one of my projects, I need to run three different database updater functions at different intervals.
For instance, function one needs to run every 30 seconds, function two needs to run every 60 seconds and function 3 every 5 minutes (notably due to API call restrictions).
I've been trying to achieve this in python, looking up every possible solution but I cannot seem to find anything that works for my use case. I am rather fresh in python.
Here is (somewhat) what I have, using asyncio.
import asyncio
def updater1(url1, url2, time):
print(f"Doing my thing here every {time} seconds")
def updater2(url1, url2, time):
print(f"Doing my thing here every {time} seconds")
def updater3(url, time):
print(f"Doing my thing here every {time} seconds")
async def func1():
updater1(rankUrl, statsUrl, 30)
await asyncio.sleep(30)
async def func2():
updater2(rankUrl, statsUrl, 60)
await asyncio.sleep(60)
async def func3():
updater3(url, 300)
await asyncio.sleep(300)
# Initiate async loops
while True:
asyncio.run(func1())
asyncio.run(func2())
asyncio.run(func3())
The issue is that these tasks run one after each other, while what I am trying to achieve is that they run independently from each other, with a start time when the script is initiated, and respective to their own individual loop times
Any idea on how this could be done is much appreciated - I am open to new concepts and ideas if you have any for me to explore :)
Don't use asyncio.run() on individual coroutines, as async.run() is itself not asynchronous. The call to asyncio.run() won't return until the funcN() coroutine is done.
Create a single top-level coroutine that then runs others as tasks:
async def main():
task1 = asyncio.create_task(func1())
task2 = asyncio.create_task(func2())
task3 = asyncio.create_task(func3())
await asyncio.wait([task1, task2, task3])
The above kicks off three independent tasks, then waits for all 3 to complete.
I want to execute two tasks forever which are run in parallel by asyncio.gather(). The first task contains some logic, for which I want to wait in the while True: loop. The second task is asyncio.sleep() which is responsible for execution of my tasks in equal timestamps. I want to get the result of the first task in asyncio.gather() just after it is finished.
async def main():
async with aiohttp.ClientSession() as session:
while True:
awaitingTime = calcAwaitingTime()
data, _ = await asyncio.gather(BinanceClient.getLastSample(session), asyncio.sleep(awaitingTime))
if __name__ == '__main__':
loop = asyncio.get_event_loop()
task = loop.create_task(main())
loop.run_forever()
I want to get the result of the BinanceClient.getLastSample() just after it is finished not to wait after asyncio.sleep() also is finished. Maybe do you propose other way of executing these two tasks forever to have better possibility to get the results of them?