I want to create a library of components to work with telegram bot that I could execute in order. Each component would await it's input and allow execution to continue to the next line, like so:
fname = await Question().run('Enter first name')
lname = await Question().run('Enter last name')
While allowing other coroutines to run elsewhere in the code.
I implemented Question like so:
class Question:
def __init__(self):
self.result = None
async def on_message(self, update, context):
self.result = update.message.text
async def run(self, prompt):
# register on_message as handler
# prompt user
while self.result is None:
await asyncio.sleep(1)
# unregister on_message handler
return self.result
This loop would block the on_message functionality and user input is never received
I tried using series of awaited recursive calls instead of the loop. Same result.
I tried using threading and putting the loop into separate Thread. Same result.
I ended up doing this with a chain of callbacks, but this I could have done without any asynchronous stuff, and it is plain ugly.
Please tell me there is a way to achieve what I want.
Related
I'm currently writing a discord bot which needs to be able to run a task that could take anywhere from a few seconds to a minute while still being responsive to other commands. Forgive me if this is a fairly simple question, but I haven't been able to find a solution that worked yet.
Here is an abridged version of the code
class StableCog(commands.Cog, name='Stable Diffusion', description='Create images from natural language.'):
def __init__(self, bot):
self.text2image_model = Text2Image()
self.bot = bot
#commands.slash_command(description='Create an image.')
async def dream(self, -- a ton of arguments -- ):
print(f'Request -- {ctx.author.name}#{ctx.author.discriminator} -- Prompt: {query}')
asyncio.get_event_loop().create_task(src.bot.queue_system.dream_async( -- a ton of arguments -- ))
inside queue_system.py
async def dream_async(-- a ton of arguments --):
await ctx.interaction.response.send_message('Added to queue! You are # in queue')
embed = discord.Embed()
try:
#lots of code, I've removed it since it doesn't have anything to do with the async
await ctx.channel.send(embed=embed, file=discord.File(fp=buffer, filename=f'{seed}.png'))
except Exception as e:
embed = discord.Embed(title='txt2img failed', description=f'{e}\n{traceback.print_exc()}', color=embed_color)
await ctx.channel.send(embed=embed)
However, the discord bot becomes unresponsive until the code in queue_system.py finishes running. Every solution I've tried so far hasn't worked correctly since I'm trying to create a thread to run an asynchronous method. What would be the best way to do so? Ignore the name queue_system.py, it isn't quite a queue system yet, I'm just working out how to run the dream method asynchronously before I work that out.
What is blocking the event loop in the dream_async coroutine ? In that coroutine if you're calling some (non async) functions that have the potential to block the loop, that's an issue, but the real culprit must be in the "lots of code" part :)
A good option would be to use run_in_executor() to run the non async code in a threadpool, and therefore prevent that code from blocking dream_async.
def blocking_stuff(arg):
# this will run in a thread
...
return 'something'
async def dream_async(-- a ton of arguments --):
loop = asyncio.get_event_loop()
await ctx.interaction.response.send_message('Added to queue! You are # in queue')
embed = discord.Embed()
try:
# Run the blocking part in a threadpool
result = await loop.run_in_executor(None, blocking_stuff, 'test')
await ctx.channel.send(embed=embed, file=discord.File(fp=buffer, filename=f'{seed}.png'))
except Exception as e:
embed = discord.Embed(title='txt2img failed', description=f'{e}\n{traceback.print_exc()}', color=embed_color)
await ctx.channel.send(embed=embed)
Hope i didn't misunderstand you.
I am creating a server which needs to make an external request while responding. To handle concurrent requests I'm using Python's asyncio library. I have followed some examples from the standard library. It seems however that some of my tasks are destroyed, printing Task was destroyed but it is pending! to my terminal. After some debugging and research I found a stackoverflow answer which seemed to explain why.
I have created a minimal example demonstrating this effect below. My question is in what way should one counteract this effect? Storing a hard reference to the task, by for example storing asyncio.current_task() in a global variable mitigates the issue. It also seems to work fine if I wrap the future remote_read.read() as await asyncio.wait_for(remote_read.read(), 5). However I do feel like these solutions are ugly.
# run and visit http://localhost:8080/ in your browser
import asyncio
import gc
async def client_connected_cb(reader, writer):
remote_read, remote_write = await asyncio.open_connection("google.com", 443, ssl=True)
await remote_read.read()
async def cleanup():
while True:
gc.collect()
await asyncio.sleep(1)
async def main():
server = await asyncio.start_server(client_connected_cb, "localhost", 8080)
await asyncio.gather(server.serve_forever(), cleanup())
asyncio.run(main())
I am running Python 3.10 on macOS 10.15.7.
It looks that by the time being, the only way is actually keeping
a reference manually.
Maybe a decorator is something more convenient than having
to manually add the code in each async function.
I opted for the class design, so that a class attribute
can hold the hard-references while the tasks run. (A
local variable in the wrapper function would be part
of the task-reference cycle, and the garbage collection
would trigger all the same):
# run and visit http://localhost:8080/ in your browser
import asyncio
import gc
from functools import wraps
import weakref
class Shielded:
registry = set()
def __init__(self, func):
self.func = func
async def __call__(self, *args, **kw):
self.registry.add(task:=asyncio.current_task())
try:
result = await self.func(*args, **kw)
finally:
self.registry.remove(task)
return result
def _Shielded(func):
# Used along with the print sequence to assert the task was actually destroyed without commenting
async def wrapper(*args, **kwargs):
ref = weakref.finalize(asyncio.current_task(), lambda: print("task destroyed"))
return await func(*args, **kwargs)
return wrapper
#Shielded
async def client_connected_cb(reader, writer):
print("at task start")
#registry.append(asyncio.current_task())
# I've connected this to a socket in an interactive session, I'd explictly .close() for debugging:
remote_read, remote_write = await asyncio.open_connection("localhost", 8060, ssl=False)
print("comensing remote read")
await remote_read.read()
print("task complete")
async def cleanup():
while True:
gc.collect()
await asyncio.sleep(1)
async def main():
server = await asyncio.start_server(client_connected_cb, "localhost", 8080)
await asyncio.gather(server.serve_forever(), cleanup())
asyncio.run(main())
Moreover, I wanted to "really see it", so I created a "fake" _Shielded
decorator that would just log something when the underlying task
got deleted: "task complete" is never printed with it, indeed.
Im trying to multithread async functions, one with input, could you please help me?
functions
#client.event
async def on_message(message):
print(f"{message.author.name}: {message.content}")
#client.event
async def on_ready():
while True:
message=input("> ")
run code:
if __name__ == "__main__":
try:
client.run(token, bot=True)
except:
exit(1)
off topic: the default value for bot is true, so you can just do client.run(token)
on topic: async "replaces" threading, https://stackoverflow.com/a/27265877/10192011
also if you want to have a input in discord.py you should look into tasks
Very late to the party but thought I'd share for any future readers.
Sync programming is where python goes line by line and executes every statement in order, while async programming is when you define multiple blocks of code and you set them up to run at a later time when the CPU has spare cycles.
The issue here is that you can run sync functions in an async context, and those lengthy sync tasks will block the thread, stopping also all other async tasks (hence their "blocking tasks" name).
input() is a synchronous blocking task that sits idly waiting for user input until the enter key is pressed, but since it is a blocking function it will stop all other asynchronous code, including discord.py.
What you could do is make use of a separate package, aioconsole, which, among other cool things, includes aioconsole.ainput(). This is entirely similar to input() but works asynchronously:
from aioconsole import ainput
#client.event
async def on_ready():
while True:
message = await ainput("> ")
# do whatever
Do note that the prompt you pass to ainpput() will be printed at the very beginning, if something else (due to this being an asynchronous context) prints while the user hasn't pressed Enter yet, the prompt message will stay where it is (including anything the user has typed so far) and the new printed text added after it, which may result to very confusing interactions...
Using a non async def as the thread target and run the async def from the thread target is one of the solution. Like below.
Example:
async def log_pub(self):
while 1:
sleep(1)
await self.Publish()
#Break the loop based on event
if thread_exit_event.is_set():
break
def handleThread(self):
asyncio.run(self.log_pub())
def startLogging(self):
thread = threading.Thread(target=self.handleThread, args=())
thread.start()
I'm working on a Discord bot with Python and it queues music from YouTube, i'm working on something to autoqueue songs when the player is stopped, while all of the code works perfectly, the only problem is me not being able to check if the player is playing or not every 15 seconds
async def cmd_autoqueue(self,message, player,channel,author, permissions, leftover_args):
print("autoq ran")
if started == True:
if player.is_stopped:
await self.cmd_autoqadd(player, channel, author, permissions,leftover_args,song_url=last_url)
threading.Timer(15.0,await self.cmd_autoqueue(message, player,channel,author, permissions, leftover_args)).start()
i did realise that
threading.Timer(15.0,await self.cmd_autoqueue(message, player,channel,author, permissions, leftover_args)).start()
calls the function, and if i wanted to pass it as something that would be called later i would use lambda: but , async lambda?
also started boolean is managed by other stuff so well its there for the sake of the 'if', here in this question
Solution:
async def cmd_autoqueue(self,message, player,channel,author, permissions, leftover_args):
global started
print("loop")
if started == True:
await asyncio.sleep(15)
if player.is_stopped:
await self.cmd_autoqadd(player, channel, author, permissions,leftover_args,song_url=last_url)
await self.cmd_autoqueue(message, player,channel,author, permissions, leftover_args)
You could use loop.call_later, that returns you an asyncio-compatible handler to cancel the task.
class YourClass:
def init(self):
self._loop = asyncio.get_event_loop()
self._check_handler = None
def schedule_check(self):
self._check_handler = loop.call_later(
15 * 60, # Every 15 minutes
self._loop.create_task, self._periodic_check())
def stop_checking(self):
self._check_handler.cancel()
async def start_player(self):
await do something()
if not check_handler:
schedule_check()
async def stop_player(self):
await do something()
self.stop_checking()
async def _periodic_check(self):
await do_something()
self._schedule_check()
To pass a function (that needs some parameters) as a parameter to another function, that doesn't allow you to pass the parameters, you can bind the parameters using functools.partial.
So i have a basic discord bot which accepts input
import discord
import asyncio
import threading
loop = asyncio.new_event_loop()
bot = discord.Client()
def run_asyncio_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()
Hangman.set_bot(bot)
#bot.event
async def on_message(message):
bot.loop.create_task(Hangman.main(message))
asyncioLoop = threading.Thread(target = run_asyncio_loop, args = (loop,))
asyncioLoop.start()
bot.run(BotConstants.TOKEN)
In this example it calls the hangman game which does not block anything as i have tested this using asyncio.sleep(n) but when i go to do a something in hangman it blocks it.
class Hangman():
async def main(message):
await Hangman.make_guess(message)
async def update_score(message):
sheetLoaded = Spreadsheet.load_ws(...)
userExists = Spreadsheet.user_exists(...)
if (not userExists):
Spreadsheet.add_user(...)
Spreadsheet.add_score(...)
await Hangman.bot.send_message(message.channel, msg)
elif (not sheetLoaded):
await Hangman.bot.send_message(message.channel, msg)
async def make_guess(message):
# perform guess
if (matched):
await Hangman.bot.send_message(message.channel, msg)
Hangman.GAMES.pop(message.server.id)
await Hangman.update_score(message)
When Hangman.update_score() is called it blocks it. so it won't process any commands until the score has been updated which means for about 5 or so seconds (not long but with lots of users spamming it it's an issue) the bot does not accept any other messages
What am i missing to be able to make the process run in the background while still accept new inputs?
Asyncio is still single-threaded. The only way for the event loop to run is for no other coroutine to be actively executing. Using yield from/await suspends the coroutine temporarily, giving the event loop a chance to work. So unless you call another coroutine using yield (from) or await or return, the process is blocked. You can add await asyncio.sleep(0) in between steps of Hangman.update_score to divide the process blocking in multiple parts, but that will only ensure less "hanging" time, not actually speed up your thread.
To make the process actually run in the background, you could try something along the lines of:
from concurrent.futures import ProcessPoolExecutor
executor = ProcessPoolExecutor(2)
asyncio.ensure_future(loop.run_in_executor(executor, Hangman.update_score(message)))