Im trying to multithread async functions, one with input, could you please help me?
functions
#client.event
async def on_message(message):
print(f"{message.author.name}: {message.content}")
#client.event
async def on_ready():
while True:
message=input("> ")
run code:
if __name__ == "__main__":
try:
client.run(token, bot=True)
except:
exit(1)
off topic: the default value for bot is true, so you can just do client.run(token)
on topic: async "replaces" threading, https://stackoverflow.com/a/27265877/10192011
also if you want to have a input in discord.py you should look into tasks
Very late to the party but thought I'd share for any future readers.
Sync programming is where python goes line by line and executes every statement in order, while async programming is when you define multiple blocks of code and you set them up to run at a later time when the CPU has spare cycles.
The issue here is that you can run sync functions in an async context, and those lengthy sync tasks will block the thread, stopping also all other async tasks (hence their "blocking tasks" name).
input() is a synchronous blocking task that sits idly waiting for user input until the enter key is pressed, but since it is a blocking function it will stop all other asynchronous code, including discord.py.
What you could do is make use of a separate package, aioconsole, which, among other cool things, includes aioconsole.ainput(). This is entirely similar to input() but works asynchronously:
from aioconsole import ainput
#client.event
async def on_ready():
while True:
message = await ainput("> ")
# do whatever
Do note that the prompt you pass to ainpput() will be printed at the very beginning, if something else (due to this being an asynchronous context) prints while the user hasn't pressed Enter yet, the prompt message will stay where it is (including anything the user has typed so far) and the new printed text added after it, which may result to very confusing interactions...
Using a non async def as the thread target and run the async def from the thread target is one of the solution. Like below.
Example:
async def log_pub(self):
while 1:
sleep(1)
await self.Publish()
#Break the loop based on event
if thread_exit_event.is_set():
break
def handleThread(self):
asyncio.run(self.log_pub())
def startLogging(self):
thread = threading.Thread(target=self.handleThread, args=())
thread.start()
Related
I'm currently writing a discord bot which needs to be able to run a task that could take anywhere from a few seconds to a minute while still being responsive to other commands. Forgive me if this is a fairly simple question, but I haven't been able to find a solution that worked yet.
Here is an abridged version of the code
class StableCog(commands.Cog, name='Stable Diffusion', description='Create images from natural language.'):
def __init__(self, bot):
self.text2image_model = Text2Image()
self.bot = bot
#commands.slash_command(description='Create an image.')
async def dream(self, -- a ton of arguments -- ):
print(f'Request -- {ctx.author.name}#{ctx.author.discriminator} -- Prompt: {query}')
asyncio.get_event_loop().create_task(src.bot.queue_system.dream_async( -- a ton of arguments -- ))
inside queue_system.py
async def dream_async(-- a ton of arguments --):
await ctx.interaction.response.send_message('Added to queue! You are # in queue')
embed = discord.Embed()
try:
#lots of code, I've removed it since it doesn't have anything to do with the async
await ctx.channel.send(embed=embed, file=discord.File(fp=buffer, filename=f'{seed}.png'))
except Exception as e:
embed = discord.Embed(title='txt2img failed', description=f'{e}\n{traceback.print_exc()}', color=embed_color)
await ctx.channel.send(embed=embed)
However, the discord bot becomes unresponsive until the code in queue_system.py finishes running. Every solution I've tried so far hasn't worked correctly since I'm trying to create a thread to run an asynchronous method. What would be the best way to do so? Ignore the name queue_system.py, it isn't quite a queue system yet, I'm just working out how to run the dream method asynchronously before I work that out.
What is blocking the event loop in the dream_async coroutine ? In that coroutine if you're calling some (non async) functions that have the potential to block the loop, that's an issue, but the real culprit must be in the "lots of code" part :)
A good option would be to use run_in_executor() to run the non async code in a threadpool, and therefore prevent that code from blocking dream_async.
def blocking_stuff(arg):
# this will run in a thread
...
return 'something'
async def dream_async(-- a ton of arguments --):
loop = asyncio.get_event_loop()
await ctx.interaction.response.send_message('Added to queue! You are # in queue')
embed = discord.Embed()
try:
# Run the blocking part in a threadpool
result = await loop.run_in_executor(None, blocking_stuff, 'test')
await ctx.channel.send(embed=embed, file=discord.File(fp=buffer, filename=f'{seed}.png'))
except Exception as e:
embed = discord.Embed(title='txt2img failed', description=f'{e}\n{traceback.print_exc()}', color=embed_color)
await ctx.channel.send(embed=embed)
Hope i didn't misunderstand you.
I am making a bot and want it to ping a user every ten minutes. Here is what I have right now
#client.event
while True:
channel = client.get_channel(717346417592762457)
await message.channel.send(<#!561464966427705374>)
time.sleep(600)
But this results in a lot of errors.
One easy way to do it would be to use discord's tasks extension. You can easily set the amount of time for each loop. The only main drawback is that the loop restarts every time your bot restarts, or rather everytime you run the command, in this example. In the example below, I added the method to start the loop in a command, but you can have it start in an on_ready event or similar.
# New Import
from discord.ext import tasks, commands
# ... Other code ...
#tasks.loop(minutes=10) # #tasks.loop(seconds=600) <-- does the same amount of time
async def ping_loop():
channel = client.get_channel(717346417592762457)
await channel.send("<#!561464966427705374>")
# 1. Don't use message.channel, as you only defined channel and this isn't an event
# 2. As mentioned by Tim Roberts in the comments
# add speech marks to your send() since it's a string
# somewhere else in your code
#client.command()
async def loop(ctx):
try:
ping_loop.start() # start the loop if it isn't currently running
await ctx.send("Started pinging <#!561464966427705374>")
except: # happens if the loop is already running
ping_loop.cancel() # if so, cancel the loop
await ctx.send("Stopped pinging <#!561464966427705374>")
For the specific errors you mentioned:
#client.event
async def somefunction(ctx):
# you need to define a function to call (pass to discord.py)
# also the function has to be async, for the usage of `await`
while True:
channel = client.get_channel(717346417592762457)
await message.channel.send(<#!561464966427705374>)
# not sure if you can mention user like this
await asyncio.sleep(600)
# time.sleep(600) # prefer non-blocking asyncio sleep. make sure `asyncio` is imported
To mention a user, you need to first get user with
user_instance = ctx.guild.get_member(YOUR_USER_discord_id)
Note: You may need to enable the privilege to use the .get_member() function. If you have not enabled it yet, just follow the instructions in the exception.
Then you can sent message with
await ctx.send(f"THE_TEXT_YOU_WANT_TO_SPAM {user_instance.mention}!")
Also, if you are using it in a bot command like me, you may want to add some validation, so that not everyone can cast/invoke the command. I did this with has_role():
#commands.has_role(YOUR_ROLE_ID_THAT_CAN_CAST_THIS_COMMAND)
So in the end, it looks like this:
#bot.command('name': 'spam', 'help':'use at # to mention a user')
#commands.has_role(YOUR_ROLE_ID_THAT_CAN_CAST_THIS_COMMAND)
async def spamMention(ctx, spam_duration_second: int = 3):
user_instance = ctx.guild.get_member(YOUR_USER_discord_id)
for cntr in range(spam_duration_second):
await ctx.send(f"THE_TEXT_YOU_WANT_TO_SPAM {user_instance.mention}!")
await asyncio.sleep(THE_INTERVAL_YOU_WANT_IN_SECOND)
return
It's part of my code so I know it works (my python version: 3.7.9).
I have a Discord bot in Python / Discord.py where people can enter commands, and normally the bot responds very quickly.
However the bot is also gathering/scraping webdata every iteration of the main loop. Normally the scraping is pretty short and sweet so nobody really notices, but from time to time the code is set up to do a more thorough scraping which takes a lot more time. But during these heavy scrapings, the bot is sort of unresponsive to user commands.
#bot.command()
async def sample_command(ctx):
# may actually take a while for this command to respond if we happen to be
# in the middle of a heavier site scrape
await ctx.channel.send("Random message, something indicating bot has responded")
async def main_loop():
sem = asyncio.Semaphore(60)
connector = aiohttp.TCPConnector(limit=60)
async with aiohttp.ClientSession(connector=connector, headers=headers) as session:
while True:
# main guts of loop here ...
scrapers = [scraper_1(session, sem), scraper_2(session, sem), ...]
data = list(chain(*await asyncio.gather(*scrapers))) # this may take a while
# do stuff with data
Is there a way to sort of have it go "Hey, you want to do a heavy scrape, fine go process it elsewhere - meanwhile let's continue with the main loop and I'll hook back up with you later when you're done and we'll process the data then", if that makes sense?
I mainly want to separate this scraping step so it's not holding up the ability for people to actually interact with the rest of the bot.
You can use the discord.py tasks extension docs.
For example:
from discord.ext import tasks
#bot.event()
async def on_ready():
main_loop.start()
#bot.command()
async def sample_command(ctx):
await ctx.channel.send("Random message, something indicating bot has responded")
#tasks.loop(seconds=60)
async def main_loop():
do_something()
Note: It's not recommended to start the tasks in on_ready because the bot will reconnect to discord and the task will start several times, Put it somewhere else or on_ready check if this the first connect.
Another simple tip: you can use await ctx.send() instead of await ctx.channel.send()
You can use asyncio.create_task() to spawn the scraping in the "background":
async def scrape_and_process(...):
scrapers = [scraper_1(session, sem), scraper_2(session, sem), ...]
data = list(chain(*await asyncio.gather(*scrapers))) # this may take a while
# do stuff with data
async def main_loop():
sem = asyncio.Semaphore(60)
connector = aiohttp.TCPConnector(limit=60)
async with aiohttp.ClientSession(connector=connector, headers=headers) as session:
while True:
# main guts of loop here ...
# initiate scraping and processing in the background and
# proceed with the loop
asyncio.create_task(scrape_and_process(...))
You can try to use python threading.
Learn more here
It basically allows you to run it on different threads
example:
import threading
def 1():
print("Helo! This is the first thread")
def 2():
print("Bonjour! This is the second thread")
thread1 = threading.Thread(target=1)
thread2 = Threading.Thread(target=2)
thread1.start()
thread2.start()
How do I remove the async-everywhere insanity in a program like this?
import asyncio
async def async_coro():
await asyncio.sleep(1)
async def sync_func_1():
# This is blocking and synchronous
await async_coro()
async def sync_func_2():
# This is blocking and synchronous
await sync_func_1()
if __name__ == "__main__":
# Async pollution goes all the way to __main__
asyncio.run(sync_func_2())
I need to have 3 async markers and asyncio.run at the top level just to call one async function. I assume I'm doing something wrong - how can I clean up this code to make it use async less?
FWIW, I'm interested mostly because I'm writing an API using asyncio and I don't want my users to have to think too much about whether their functions need to be def or async def depending on whether they're using a async part of the API or not.
After some research, one answer is to manually manage the event loop:
import asyncio
async def async_coro():
await asyncio.sleep(1)
def sync_func_1():
# This is blocking and synchronous
loop = asyncio.get_event_loop()
coro = async_coro()
loop.run_until_complete(coro)
def sync_func_2():
# This is blocking and synchronous
sync_func_1()
if __name__ == "__main__":
# No more async pollution
sync_func_2()
If you must do that, I would recommend an approach like this:
import asyncio, threading
async def async_coro():
await asyncio.sleep(1)
_loop = asyncio.new_event_loop()
threading.Thread(target=_loop.run_forever, daemon=True).start()
def sync_func_1():
# This is blocking and synchronous
return asyncio.run_coroutine_threadsafe(async_coro(), _loop).result()
def sync_func_2():
# This is blocking and synchronous
sync_func_1()
if __name__ == "__main__":
sync_func_2()
The advantage of this approach compared to one where sync functions run the event loop is that it supports nesting of sync functions. It also only runs a single event loop, so that if the underlying library wants to set up e.g. a background task for monitoring or such, it will work continuously rather than being spawned each time anew.
So i have a basic discord bot which accepts input
import discord
import asyncio
import threading
loop = asyncio.new_event_loop()
bot = discord.Client()
def run_asyncio_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()
Hangman.set_bot(bot)
#bot.event
async def on_message(message):
bot.loop.create_task(Hangman.main(message))
asyncioLoop = threading.Thread(target = run_asyncio_loop, args = (loop,))
asyncioLoop.start()
bot.run(BotConstants.TOKEN)
In this example it calls the hangman game which does not block anything as i have tested this using asyncio.sleep(n) but when i go to do a something in hangman it blocks it.
class Hangman():
async def main(message):
await Hangman.make_guess(message)
async def update_score(message):
sheetLoaded = Spreadsheet.load_ws(...)
userExists = Spreadsheet.user_exists(...)
if (not userExists):
Spreadsheet.add_user(...)
Spreadsheet.add_score(...)
await Hangman.bot.send_message(message.channel, msg)
elif (not sheetLoaded):
await Hangman.bot.send_message(message.channel, msg)
async def make_guess(message):
# perform guess
if (matched):
await Hangman.bot.send_message(message.channel, msg)
Hangman.GAMES.pop(message.server.id)
await Hangman.update_score(message)
When Hangman.update_score() is called it blocks it. so it won't process any commands until the score has been updated which means for about 5 or so seconds (not long but with lots of users spamming it it's an issue) the bot does not accept any other messages
What am i missing to be able to make the process run in the background while still accept new inputs?
Asyncio is still single-threaded. The only way for the event loop to run is for no other coroutine to be actively executing. Using yield from/await suspends the coroutine temporarily, giving the event loop a chance to work. So unless you call another coroutine using yield (from) or await or return, the process is blocked. You can add await asyncio.sleep(0) in between steps of Hangman.update_score to divide the process blocking in multiple parts, but that will only ensure less "hanging" time, not actually speed up your thread.
To make the process actually run in the background, you could try something along the lines of:
from concurrent.futures import ProcessPoolExecutor
executor = ProcessPoolExecutor(2)
asyncio.ensure_future(loop.run_in_executor(executor, Hangman.update_score(message)))