My code execution does not reach the print statement: print("I want to display after MyClass has started")
Why is this? I thought the purpose of await asyncio.sleep() is to unblock execution of code so that subsequent lines of code can run. Is that not the case?
import asyncio
class MyClass:
def __init__(self):
self.input = False
asyncio.run(self.start())
print("I want to display after MyClass has started") #This line is never reached.
async def start(self):
while True:
print("Changing state...")
if self.input:
print("I am on.")
break
await asyncio.sleep(1)
m = MyClass()
m.input = True #This line is never reached! Why?
print("I want to display after MyClass is started")
When I execute, it keeps printing "Changing state...". Even when I ctrl+c to quit, the execution continues as shown below. How can I properly terminate the execution? Sorry, I am new to python.
EDIT:
I appreciate the common use of asyncio is for running two or more separate functions asynchronously. However, my class is one which will be responding to changes in its state. For example, I intend to write code in the setters to do stuff when the class objects attributes change -WHILE still having a while True event loop running in the background. Is there not any way to permit this? I have tried running the event loop in it's own thread. However, that thread then dominates and the class objects response times run into several seconds. This may be due to the GIL (Global Interpreter Lock) which we can do nothing about. I have also tried using multiprocessing, but then I lose access to the properties and methods of the object as parallel process run in their own memory spaces.
In the init method of MyClass you invoke asyncio.run() - this method will execute the required routine until that routine terminates. In your case, since the main method includes a while True loop, it will never terminate.
Here is a slight modification of your code that perhaps shows the concurrency effect you're after -
import asyncio
class MyClass:
def __init__(self):
self.input = False
asyncio.run(self.main())
print("I want to display after MyClass has been initialized.") # This line is never reached.
async def main(self):
work1 = self.work1()
work2 = self.work2()
await asyncio.gather(work1, work2)
async def work1(self):
for idx in range(5):
print('doing some work 1...')
await asyncio.sleep(1)
async def work2(self):
for idx in range(5):
print('doing some work 2...')
await asyncio.sleep(1)
m = MyClass()
print("I want to display after MyClass is terminated")
Related
I have a program executed in a subprocess. This program runs forever, reads a line from its stdin, processes it, and outputs a result on stdout. I have encapsulated it as follows:
class BrainProcess:
def __init__(self, filepath):
# starting the program in a subprocess
self._process = asyncio.run(self.create_process(filepath))
# check if the program could not be executed
if self._process.returncode is not None:
raise BrainException(f"Could not start process {filepath}")
#staticmethod
async def create_process(filepath):
process = await sp.create_subprocess_exec(
filepath, stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE)
return process
# destructor function
def __del__(self):
self._process.kill() # kill the program, since it never stops
# waiting for the program to terminate
# self._process.wait() is asynchronous so I use async.run() to execute it
asyncio.run(self._process.wait())
async def _send(self, msg):
b = bytes(msg + '\n', "utf-8")
self._process.stdin.write(b)
await self._process.stdin.drain()
async def _readline(self):
return await self._process.stdout.readline()
def send_start_cmd(self, size):
asyncio.run(self._send(f"START {size}"))
line = asyncio.run(self._readline())
print(line)
return line
From my understanding asyncio.run() is used to run asynchronous code in a synchronous context. That is why I use it at the following lines:
# in __init__
self._process = asyncio.run(self.create_process(filepath))
# in send_start_cmd
asyncio.run(self._send(f"START {size}"))
# ...
line = asyncio.run(self._readline())
# in __del__
asyncio.run(self._process.wait())
The first line seems to work properly (the process is created correctly), but the other throw exceptions that look like got Future <Future pending> attached to a different loop.
Code:
brain = BrainProcess("./test")
res = brain.send_start_cmd(20)
print(res)
So my questions are:
What do these errors mean ?
How do I fix them ?
Did I use asyncio.run() correctly ?
Is there a better way to encapsulate the process to send and retrieve data to/from it without making my whole application use async / await ?
asyncio.run is meant to be used for running a body of async code, and producing a well-defined result. The most typical example is running the whole program:
async def main():
# your application here
if __name__ == '__main__':
asyncio.run(main())
Of couurse, asyncio.run is not limited to that usage, it is perfectly possible to call it multiple times - but it will create a fresh event loop each time. This means you won't be able to share async-specific objects (such as futures or objects that refer to them) between invocations - which is precisely what you tried to do. If you want to completely hide the fact that you're using async, why use asyncio.subprocess in the first place, wouldn't the regular subprocess do just as well?
The simplest fix is to avoid asyncio.run and just stick to the same event loop. For example:
_loop = asyncio.get_event_loop()
class BrainProcess:
def __init__(self, filepath):
# starting the program in a subprocess
self._process = _loop.run_until_complete(self.create_process(filepath))
...
...
Is there a better way to encapsulate the process to send and retrieve data to/from it without making my whole application use async / await ?
The idea is precisely for the whole application to use async/await, otherwise you won't be able to take advantage of asyncio - e.g. you won't be able to parallelize your async code.
using: python 3.8 on windows 10
I am working on a script that runs in a while loop.
At the end of the loop I want it to wait for 5 seconds for the user to give input and then restart, or exit if they do not give input.
I've never actually used it before but I assumed asyncio would be useful because it allows for the definition of awaitables. However bringing things together is proving more difficult than I'd anticipated.
import keyboard, asyncio, time as t
async def get():
keyboard.record('enter', True)
return True
async def countdown(time):
while time > 0:
print(f'This program will exit in {int(time)} seconds. Press enter to start over.', end='\r')
await asyncio.sleep(1)
# t.sleep(1)
time -= 1
print(f'This program will exit in 0 seconds. Press enter to start over.', end='\r')
return False
async def main():
running = True
while running:
# other code
clock = asyncio.create_task(countdown(5))
check = asyncio.create_task(get())
done, pending = await asyncio.wait({clock, check}, return_when=asyncio.FIRST_COMPLETED)
running = next(iter(done)).result()
print(running, end='\r')
print('bye')
asyncio.run(main())
As it stands, the process doesn't end if I wait for five seconds. It also doesn't visibly count down (my best, and most ridiculous, guess is that it may be looping too fast in main? Try holding down the "enter" key - with and without printing running).
Also, when I switch to time.sleep, the display works fine, but it doesn't seem as though the countdown function ever returns.
Previously I'd tried using an input statement instead of keyboard.record; that blocks. I had also tried using asyncio.wait_for; but the timeout never came, though again, the "enter" key was registered (that said, it wouldn't have printed the countdown even if it had worked). I also tried asyncio.as_completed but I was unable to parse anything useful from the iterable. Happy to be wrong there though!
Also, bonus points if you can generalize the countdown to non-integer time spans <3
The wait_for approach:
async def main():
running = True
while running:
try:
running = await asyncio.wait_for(get(), 5)
except asyncio.TimeoutError:
running = False
print(running)
print('bye')
asyncio.run(main())
as_completed
async def main():
running = True
while running:
## other code
clock = asyncio.create_task(countdown(5))
check = asyncio.create_task(get())
for f in asyncio.as_completed({check, clock}):
# print(f.cr_frame)
# print(f.cr_await, f.cr_running, f.cr_origin, f.cr_frame, f.cr_code)
print(f.cr_await, f.cr_running, f.cr_origin)
print('bye')
Also, bonus points if you can generalize the countdown to non-integer time :P
cheers!
If you look in the docs for keyboard.record it says:
Note: this is a blocking function.
This is why your process didn't end after 5 seconds. It was blocking at keyboard.record('enter', True). If you are going to stick with the keyboard module, what you need to do is create a hook on the 'enter' key. I put a quick demo together with your code:
import asyncio
import keyboard
class Program:
def __init__(self):
self.enter_pressed = asyncio.Event()
self._loop = asyncio.get_event_loop()
async def get(self):
await self.enter_pressed.wait()
return True
#staticmethod
async def countdown(time):
while time > 0:
print(f'This program will exit in {int(time)} seconds. Press enter to start over.')
await asyncio.sleep(1)
# t.sleep(1)
time -= 1
print(f'This program will exit in 0 seconds. Press enter to start over.')
return False
def notify_enter(self, *args, **kwargs):
self._loop.call_soon_threadsafe(self.enter_pressed.set)
async def main(self):
running = True
while running:
# other code
keyboard.on_press_key('enter', self.notify_enter)
self.enter_pressed.clear()
clock = asyncio.create_task(self.countdown(5))
check = asyncio.create_task(self.get())
done, pending = await asyncio.wait({clock, check}, return_when=asyncio.FIRST_COMPLETED)
keyboard.unhook('enter')
for task in pending:
task.cancel()
running = next(iter(done)).result()
print(running)
print('bye')
async def main():
program = Program()
await program.main()
asyncio.run(main())
There is callback created notify_enter, that sets an asyncio.Event whenever it's fired. The get() task waits for this event to trigger before it exits. Since I didn't know what your other code is doing, we don't bind a hook to the enter key's key_down event until right before you await both tasks and we unbind it right after one of the tasks completes. I wrapped everything up in a class so the event is accessible in the callback, since there isn't a way to pass parameters in.
I am trying to use asyncio together with threading for a Discord Bot. I've found this script which I changed to my needs:
import time
import threading as th
import asyncio
import discord
class discordvars(object):
client=discord.Client()
TOKEN=('---')
running_discordthread=False
discordloop = asyncio.get_event_loop()
discordloop.create_task(client.start(TOKEN))
discordthread=th.Thread(target=discordloop.run_forever)
def start():
if discordvars.running_discordthread==False:
discordvars.discordthread.start()
print("Discord-Client started...")
discordvars.running_discordthread=True
else:
print("Discord-CLient allready running...")
time.sleep(2)
def stop():
if discordvars.running_discordthread==True:
discordvars.discordloop.call_soon_threadsafe(discordvars.discordloop.stop())
print("Requestet Discord-Client stop!")
discordvars.discordthread.join()
print(discordvars.discordthread.isAlive())
time.sleep(1)
print("Discord-Client stopped...")
discordvars.running_discordthread=False
else:
print("Discord-Client not running...")
time.sleep(2)
#discordvars.client.event
async def on_message(message):
if message.content.startswith('!test'):
embed = discord.Embed(title="test", color=0x0071ce, description="test")
await message.channel.send(embed=embed)
Starting the Script with the start() function works great. Also stopping with the stop() function works somehow. If I call the stop() function it prints: "False" so I am thinking that the thread was stopped. But if I then call the start() function I will get an error:
RuntimeError: threads can only be started once
This script is part of a big project so I am calling the functions from another script. But I think that shouldn't be the problem.
What is the problem? Thanks in advance.
You cannot re-start the existing thread, but you can start a new thread that runs the event loop. You can achieve that by moving the assignment to discordthread to the start function.
And your call to call_soon_threadsafe is wrong. You need to pass discordloop.stop to it, without parentheses. That refers to the actual function without calling it right away, and allows the loop thread to call it, which was intended:
discordloop.call_soon_threadsafe(discordloop.stop)
Finally, your init function is missing a global declaration for the variables you assign that are intended as globals.
I am calling a function from a thread and as soon as the function starts running both the threads stop working until the function has finished executing and after the function execution the program just stops.
class listen(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.playmusicobject = playmusic()
self.objectspeak = speak()
self.apiobject = googleAPI()
def listening(self):
self.r = sr.Recognizer()
threadLock.acquire()
try:
with sr.Microphone() as source:
print("say something")
self.audio = self.r.listen(source)
finally:
threadLock.release()
def checkingaudio(self):
threadLock.acquire()
try:
# a = str(self.r.recognize_google(self.audio))
a = str(self.r.recognize_google(self.audio))
print(a)
if a in greetings:
self.objectspeak.speaking("I am good how are you?")
if a in music:
print("playing music")
self.playmusicobject.play()
if a in stop:
print("stopping")
self.playmusicobject.b()
if a in api:
self.apiobject.distance()
finally:
threadLock.release()
class playmusic:
def play(self):
playsound.playsound("playthisfile")
if __name__ == "__main__":
while 1:
d = listen()
t1 = threading.Thread(target=d.listening)
t1.start()
t2 = threading.Thread(target=d.checkingaudio)
t2.start()
whenever I call the play function in playmusic both the threads supposedly stop and after the play function's execution the program stops. Is there anyway that the function play or for the matter of fact any other function called by the thread running the function CheckingAudio, won't stop the 2 original threads.I tried running the play function as a thread of its own but was unsucessful in doing so, it just gave me an error that said unable to start thread t3, the way was calling the function as a thread was (and this is probably wrong)
if a in music:
print("playing music")
t3 = threading.Thread(target=self.playmusicobject.play)
t3.start()
I tried creating a runplaythread function in playmusic class but still gave me the error cannot start the thread.
Not sure it will fix your problem, but your last thread's target is wrong, you're using self instead of d to call your instance of listen.
This is your code corrected to replace context:
if a in music:
print("playing music")
t3 = threading.Thread(target=d.playmusicobject.play)
t3.start()
NOTE: A comment would probably have been ok, but not enough rep.
EDIT: Reuse of OP's code for context is now explicitly mentioned. (Though that was obvious enough)
EDIT2: Where you are calling your third thread was not clear, but don't seems ok, see comments.
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
self._end = False
self._running = False
def run(self):
self._running = True
while not self._finished:
time.sleep(0.05)
self._end = True
def stop(self):
if not self._running:
return
self._finished = True
while not self._end:
time.sleep(0.05)
I wish to have a thread on which I can call run() and stop(). The stop method should wait for run to complete in an orderly manner. I also want stop to return without any issues if run hasn't even be called. How should I do this?
I create this thread in a setup() method in my test environment and run stop on it in the teardown(). However, in some tests I dont call run().
UPDATE
Here's my second attempt. Is it correct now?
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self._finished = False
def run(self):
while not self._finished:
print("*")
time.sleep(1)
print("Finished Other")
def finish(self):
self._finished = True
self.join()
m = MyThread()
m.start()
print("After")
time.sleep(5)
m.finish()
print("Finished Main")
You do not need to and should not implement this yourself. What you are looking for already exists, at least in large parts. It is, however, not called "stop". The concept you are describing is usually called "join".
Have a look at the documentation for join: https://docs.python.org/3.4/library/threading.html#threading.Thread.join
You write
The stop method should wait for run to complete in an orderly manner.
Join's documentation says: "Wait until the thread terminates." check ✓
You write
I also want stop to return without any issues if run hasn't even be
called
Join's documentation says: "It is also an error to join() a thread before it has been started"
So, the only thing you need to make sure is that you call join() only after you have started the thread via the start() method. That should be easy for you.