I am trying to use asyncio together with threading for a Discord Bot. I've found this script which I changed to my needs:
import time
import threading as th
import asyncio
import discord
class discordvars(object):
client=discord.Client()
TOKEN=('---')
running_discordthread=False
discordloop = asyncio.get_event_loop()
discordloop.create_task(client.start(TOKEN))
discordthread=th.Thread(target=discordloop.run_forever)
def start():
if discordvars.running_discordthread==False:
discordvars.discordthread.start()
print("Discord-Client started...")
discordvars.running_discordthread=True
else:
print("Discord-CLient allready running...")
time.sleep(2)
def stop():
if discordvars.running_discordthread==True:
discordvars.discordloop.call_soon_threadsafe(discordvars.discordloop.stop())
print("Requestet Discord-Client stop!")
discordvars.discordthread.join()
print(discordvars.discordthread.isAlive())
time.sleep(1)
print("Discord-Client stopped...")
discordvars.running_discordthread=False
else:
print("Discord-Client not running...")
time.sleep(2)
#discordvars.client.event
async def on_message(message):
if message.content.startswith('!test'):
embed = discord.Embed(title="test", color=0x0071ce, description="test")
await message.channel.send(embed=embed)
Starting the Script with the start() function works great. Also stopping with the stop() function works somehow. If I call the stop() function it prints: "False" so I am thinking that the thread was stopped. But if I then call the start() function I will get an error:
RuntimeError: threads can only be started once
This script is part of a big project so I am calling the functions from another script. But I think that shouldn't be the problem.
What is the problem? Thanks in advance.
You cannot re-start the existing thread, but you can start a new thread that runs the event loop. You can achieve that by moving the assignment to discordthread to the start function.
And your call to call_soon_threadsafe is wrong. You need to pass discordloop.stop to it, without parentheses. That refers to the actual function without calling it right away, and allows the loop thread to call it, which was intended:
discordloop.call_soon_threadsafe(discordloop.stop)
Finally, your init function is missing a global declaration for the variables you assign that are intended as globals.
Related
I have a waiter() function which I start running. Subsequently, an event such as a click on an ipywidgets.Button should modify the result. My idea is to use event.set() through the Button click, thus I pulled event.set() outside the asyncio context. However,since the waiter() function is blocked, the click-action is also blocked. I think that calling event.set() in some other function should be a common necessity, but I'm using asyncio wrong. What should I be doing instead?
Code modified from:Python Asyncio - Pythonic way of waiting until condition satisfied
import asyncio
from ipywidgets import Button
async def event_done(btn):
event1.set()
print('registred a click')
run_button = Button(description='continue')
run_button.on_click(event_done)
event1 = asyncio.Event()
async def waiter(event1):
# event1 = asyncio.Event()
print('waiting')
# event1.set()
await event1.wait()
print("After waiting")
display(run_button)
asyncio.run(waiter(event1))
Considering the following code:
import asyncio
async def main() -> None:
await asyncio.sleep(2**256)
if __name__ == '__main__':
asyncio.run(main())
What is the most proper way to terminate coroutine main, after it has been called by asyncio.run? When I invoked script and pressed CTRL + C, I saw an ugly traceback.
As I can see from the source code, asyncio.run does a lot of machinery behind the scenes, so I would like to continue using it.
You probably should handle the SIGINT signal.
import asyncio
import functools
import signal
async def main() -> None:
await asyncio.sleep(2**256)
def handler(loop):
...
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.add_signal_handler(signal.SIGINT, functools.partial(handler, loop=loop))
loop.run_until_complete(main())
The question is how the handler should look like? If you want just close program without exception, use sys.exit()
def handler(loop):
sys.exit()
However to close everything gracefully, you need to finish every task and stop the loop. See this topic for more insight.
My code execution does not reach the print statement: print("I want to display after MyClass has started")
Why is this? I thought the purpose of await asyncio.sleep() is to unblock execution of code so that subsequent lines of code can run. Is that not the case?
import asyncio
class MyClass:
def __init__(self):
self.input = False
asyncio.run(self.start())
print("I want to display after MyClass has started") #This line is never reached.
async def start(self):
while True:
print("Changing state...")
if self.input:
print("I am on.")
break
await asyncio.sleep(1)
m = MyClass()
m.input = True #This line is never reached! Why?
print("I want to display after MyClass is started")
When I execute, it keeps printing "Changing state...". Even when I ctrl+c to quit, the execution continues as shown below. How can I properly terminate the execution? Sorry, I am new to python.
EDIT:
I appreciate the common use of asyncio is for running two or more separate functions asynchronously. However, my class is one which will be responding to changes in its state. For example, I intend to write code in the setters to do stuff when the class objects attributes change -WHILE still having a while True event loop running in the background. Is there not any way to permit this? I have tried running the event loop in it's own thread. However, that thread then dominates and the class objects response times run into several seconds. This may be due to the GIL (Global Interpreter Lock) which we can do nothing about. I have also tried using multiprocessing, but then I lose access to the properties and methods of the object as parallel process run in their own memory spaces.
In the init method of MyClass you invoke asyncio.run() - this method will execute the required routine until that routine terminates. In your case, since the main method includes a while True loop, it will never terminate.
Here is a slight modification of your code that perhaps shows the concurrency effect you're after -
import asyncio
class MyClass:
def __init__(self):
self.input = False
asyncio.run(self.main())
print("I want to display after MyClass has been initialized.") # This line is never reached.
async def main(self):
work1 = self.work1()
work2 = self.work2()
await asyncio.gather(work1, work2)
async def work1(self):
for idx in range(5):
print('doing some work 1...')
await asyncio.sleep(1)
async def work2(self):
for idx in range(5):
print('doing some work 2...')
await asyncio.sleep(1)
m = MyClass()
print("I want to display after MyClass is terminated")
I have a program executed in a subprocess. This program runs forever, reads a line from its stdin, processes it, and outputs a result on stdout. I have encapsulated it as follows:
class BrainProcess:
def __init__(self, filepath):
# starting the program in a subprocess
self._process = asyncio.run(self.create_process(filepath))
# check if the program could not be executed
if self._process.returncode is not None:
raise BrainException(f"Could not start process {filepath}")
#staticmethod
async def create_process(filepath):
process = await sp.create_subprocess_exec(
filepath, stdin=sp.PIPE, stdout=sp.PIPE, stderr=sp.PIPE)
return process
# destructor function
def __del__(self):
self._process.kill() # kill the program, since it never stops
# waiting for the program to terminate
# self._process.wait() is asynchronous so I use async.run() to execute it
asyncio.run(self._process.wait())
async def _send(self, msg):
b = bytes(msg + '\n', "utf-8")
self._process.stdin.write(b)
await self._process.stdin.drain()
async def _readline(self):
return await self._process.stdout.readline()
def send_start_cmd(self, size):
asyncio.run(self._send(f"START {size}"))
line = asyncio.run(self._readline())
print(line)
return line
From my understanding asyncio.run() is used to run asynchronous code in a synchronous context. That is why I use it at the following lines:
# in __init__
self._process = asyncio.run(self.create_process(filepath))
# in send_start_cmd
asyncio.run(self._send(f"START {size}"))
# ...
line = asyncio.run(self._readline())
# in __del__
asyncio.run(self._process.wait())
The first line seems to work properly (the process is created correctly), but the other throw exceptions that look like got Future <Future pending> attached to a different loop.
Code:
brain = BrainProcess("./test")
res = brain.send_start_cmd(20)
print(res)
So my questions are:
What do these errors mean ?
How do I fix them ?
Did I use asyncio.run() correctly ?
Is there a better way to encapsulate the process to send and retrieve data to/from it without making my whole application use async / await ?
asyncio.run is meant to be used for running a body of async code, and producing a well-defined result. The most typical example is running the whole program:
async def main():
# your application here
if __name__ == '__main__':
asyncio.run(main())
Of couurse, asyncio.run is not limited to that usage, it is perfectly possible to call it multiple times - but it will create a fresh event loop each time. This means you won't be able to share async-specific objects (such as futures or objects that refer to them) between invocations - which is precisely what you tried to do. If you want to completely hide the fact that you're using async, why use asyncio.subprocess in the first place, wouldn't the regular subprocess do just as well?
The simplest fix is to avoid asyncio.run and just stick to the same event loop. For example:
_loop = asyncio.get_event_loop()
class BrainProcess:
def __init__(self, filepath):
# starting the program in a subprocess
self._process = _loop.run_until_complete(self.create_process(filepath))
...
...
Is there a better way to encapsulate the process to send and retrieve data to/from it without making my whole application use async / await ?
The idea is precisely for the whole application to use async/await, otherwise you won't be able to take advantage of asyncio - e.g. you won't be able to parallelize your async code.
I have got an XMLRPC server and client runs some functions on server and gets returned value. If the function executes quickly then everything is fine but I have got a function that reads from file and returns some value to user. Reading takes about minute(there is some complicated stuff) and when one client runs this function on the server then server is not able to respond for other users until the function is done.
I would like to create new thread that will read this file and return value for user. Is it possible somehow?
Are there any good solutions/patters to do not block server when one client run some long function?
Yes it is possible , this way
#starting the thread
def start_thread(self):
threading.Thread(target=self.new_thread,args=()).start()
# the thread you are running your logic
def new_thread(self, *args):
#call the function you want to retrieve data from
value_returned = partial(self.retrieved_data_func,arg0)
#the function that returns
def retrieved_data_func(self):
arg0=0
return arg0
Yes, using the threading module you can spawn new threads. See the documentation. An example would be this:
import threading
import time
def main():
print("main: 1")
thread = threading.Thread(target=threaded_function)
thread.start()
time.sleep(1)
print("main: 3")
time.sleep(6)
print("main: 5")
def threaded_function():
print("thread: 2")
time.sleep(4)
print("thread: 4")
main()
This code uses time.sleep to simulate that an action takes a certain amount of time. The output should look like this:
main: 1
thread: 2
main: 3
thread: 4
main: 5