So i have a basic discord bot which accepts input
import discord
import asyncio
import threading
loop = asyncio.new_event_loop()
bot = discord.Client()
def run_asyncio_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()
Hangman.set_bot(bot)
#bot.event
async def on_message(message):
bot.loop.create_task(Hangman.main(message))
asyncioLoop = threading.Thread(target = run_asyncio_loop, args = (loop,))
asyncioLoop.start()
bot.run(BotConstants.TOKEN)
In this example it calls the hangman game which does not block anything as i have tested this using asyncio.sleep(n) but when i go to do a something in hangman it blocks it.
class Hangman():
async def main(message):
await Hangman.make_guess(message)
async def update_score(message):
sheetLoaded = Spreadsheet.load_ws(...)
userExists = Spreadsheet.user_exists(...)
if (not userExists):
Spreadsheet.add_user(...)
Spreadsheet.add_score(...)
await Hangman.bot.send_message(message.channel, msg)
elif (not sheetLoaded):
await Hangman.bot.send_message(message.channel, msg)
async def make_guess(message):
# perform guess
if (matched):
await Hangman.bot.send_message(message.channel, msg)
Hangman.GAMES.pop(message.server.id)
await Hangman.update_score(message)
When Hangman.update_score() is called it blocks it. so it won't process any commands until the score has been updated which means for about 5 or so seconds (not long but with lots of users spamming it it's an issue) the bot does not accept any other messages
What am i missing to be able to make the process run in the background while still accept new inputs?
Asyncio is still single-threaded. The only way for the event loop to run is for no other coroutine to be actively executing. Using yield from/await suspends the coroutine temporarily, giving the event loop a chance to work. So unless you call another coroutine using yield (from) or await or return, the process is blocked. You can add await asyncio.sleep(0) in between steps of Hangman.update_score to divide the process blocking in multiple parts, but that will only ensure less "hanging" time, not actually speed up your thread.
To make the process actually run in the background, you could try something along the lines of:
from concurrent.futures import ProcessPoolExecutor
executor = ProcessPoolExecutor(2)
asyncio.ensure_future(loop.run_in_executor(executor, Hangman.update_score(message)))
Related
I am trying to run two different Discord Bots using a single python script using cogs. But when I try to run the 2nd bot it throws an ImportError even-though I didn't use that specific Library. The reaction roles bot works fine without the anti spam bot. Here's my code. FYI I am working inside a Virtual Env.
main.py
if __name__ == "__main__":
try:
reaction_role_bot = commands.Bot(command_prefix=config["reaction_role_bot"]["bot_prefix"], intents=discord.Intents.all())
reaction_slash = SlashCommand(reaction_role_bot, sync_commands=True)
reaction_role_bot.load_extension(f"cogs.{str(os.path.basename('cogs/reaction_roles.py')[:-3])}")
anti_spam_bot = commands.Bot(command_prefix=config["anti_spam_bot"]["bot_prefix"], intents=discord.Intents.default())
spam_slash = SlashCommand(anti_spam_bot, sync_commands=True)
anti_spam_bot.load_extension(f"cogs.{str(os.path.basename('cogs/anti_spam.py')[:-3])}")
event_loop = asyncio.get_event_loop()
event_loop.create_task(reaction_role_bot.run(config["reaction_role_bot"]["token"]))
event_loop.create_task(anti_spam_bot.run(config["anti_spam_bot"]["token"]))
event_loop.run_forever()
except Exception as e:
print(e)
anti_spam.py
import platform
import os
import discord
from discord.ext import commands
from antispam import AntiSpamHandler
from antispam.plugins import AntiSpamTracker, Options
class AntiSpamBot(commands.Cog):
def __init__(self, client):
self.client = client
# Initialize the AntiSpamHandler
self.client.handler = AntiSpamHandler(self.client, options=Options(no_punish=True))
# 3 Being how many 'punishment requests' before is_spamming returns True
self.client.tracker = AntiSpamTracker(self.client.handler, 3)
self.client.handler.register_extension(self.client.tracker)
#commands.Cog.listener()
async def on_ready(self):
print("---------------------------------")
print(f"Logged in as {str(self.client.user)}")
print(f"Discord.py API version: {discord.__version__}")
print(f"Python version: {platform.python_version()}")
print(f"Running on: {platform.system()} {platform.release()} ({os.name})")
await self.client.change_presence(status=discord.Status.idle, activity=discord.Game(name="Head of Security"))
print("---------------------------------\n")
# The code in this event is executed every time a valid commands catches an error
#commands.Cog.listener()
async def on_command_error(context, error):
raise error
#commands.Cog.listener()
async def on_message(self, message):
await self.client.handler.propagate(message)
if self.client.tracker.is_spamming(message):
await message.delete()
await message.channel.send(f"{message.author.mention} has been automatically kicked for spamming.")
await message.author.kick()
await self.client.process_commands(message)
def setup(client):
client.add_cog(AntiSpamBot(client))
Error
Extension 'cogs.anti_spam' raised an error: ImportError: cannot import name 'AsyncMock' from 'unittest.mock' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/mock.py)
I've no experiences using cogs and its bit confusing me. Any kind of help would help me to sort this out! Thanks in advance!
I do not believe this is a cog registration issue. I believe this is an import error from some of the dependencies in your cog file. I googled your error and found something similar, I recommend checking it out here for some more information.
As a blanket statement, I would double check that you have mock installed, and that you're installing it on the version of Python that you think you're installing it on. It can get wonky if you have multiple python versions insealled.
Also, on an unrelated note:: It is best to avoid running multiple bot instances in one python file, but I can help you do it the best way possible.
For starters, you have to realize that Client.run is an abstraction of a couple of more lower level concepts.
There is Client.login which logs in the client and then Client.connect which actually runs the processing. These are coroutines.
asyncio provides the capability of putting things in the event loop for it to work whenever it has time to.
Something like this e.g.
loop = asyncio.get_event_loop()
async def foo():
await asyncio.sleep(10)
loop.close()
loop.create_task(foo())
loop.run_forever()
If we want to wait for something to happen from another coroutine, asyncio provides us with this functionality as well through the means of synchronisation via asyncio.Event. You can consider this as a boolean that you are waiting for:
e = asyncio.Event()
loop = asyncio.get_event_loop()
async def foo():
await e.wait()
print('we are done waiting...')
loop.stop()
async def bar():
await asyncio.sleep(20)
e.set()
loop.create_task(bar())
loop.create_task(foo())
loop.run_forever() # foo will stop this event loop when 'e' is set to true
loop.close()
Using this concept we can apply it to the discord bots themselves.
import asyncio
import discord
from collections import namedtuple
# First, we must attach an event signalling when the bot has been
# closed to the client itself so we know when to fully close the event loop.
Entry = namedtuple('Entry', 'client event')
entries = [
Entry(client=discord.Client(), event=asyncio.Event()),
Entry(client=discord.Client(), event=asyncio.Event())
]
# Then, we should login to all our clients and wrap the connect call
# so it knows when to do the actual full closure
loop = asyncio.get_event_loop()
async def login():
for e in entries:
await e.client.login()
async def wrapped_connect(entry):
try:
await entry.client.connect()
except Exception as e:
await entry.client.close()
print('We got an exception: ', e.__class__.__name__, e)
entry.event.set()
# actually check if we should close the event loop:
async def check_close():
futures = [e.event.wait() for e in entries]
await asyncio.wait(futures)
# here is when we actually login
loop.run_until_complete(login())
# now we connect to every client
for entry in entries:
loop.create_task(wrapped_connect(entry))
# now we're waiting for all the clients to close
loop.run_until_complete(check_close())
# finally, we close the event loop
loop.close()
I have a Discord bot in Python / Discord.py where people can enter commands, and normally the bot responds very quickly.
However the bot is also gathering/scraping webdata every iteration of the main loop. Normally the scraping is pretty short and sweet so nobody really notices, but from time to time the code is set up to do a more thorough scraping which takes a lot more time. But during these heavy scrapings, the bot is sort of unresponsive to user commands.
#bot.command()
async def sample_command(ctx):
# may actually take a while for this command to respond if we happen to be
# in the middle of a heavier site scrape
await ctx.channel.send("Random message, something indicating bot has responded")
async def main_loop():
sem = asyncio.Semaphore(60)
connector = aiohttp.TCPConnector(limit=60)
async with aiohttp.ClientSession(connector=connector, headers=headers) as session:
while True:
# main guts of loop here ...
scrapers = [scraper_1(session, sem), scraper_2(session, sem), ...]
data = list(chain(*await asyncio.gather(*scrapers))) # this may take a while
# do stuff with data
Is there a way to sort of have it go "Hey, you want to do a heavy scrape, fine go process it elsewhere - meanwhile let's continue with the main loop and I'll hook back up with you later when you're done and we'll process the data then", if that makes sense?
I mainly want to separate this scraping step so it's not holding up the ability for people to actually interact with the rest of the bot.
You can use the discord.py tasks extension docs.
For example:
from discord.ext import tasks
#bot.event()
async def on_ready():
main_loop.start()
#bot.command()
async def sample_command(ctx):
await ctx.channel.send("Random message, something indicating bot has responded")
#tasks.loop(seconds=60)
async def main_loop():
do_something()
Note: It's not recommended to start the tasks in on_ready because the bot will reconnect to discord and the task will start several times, Put it somewhere else or on_ready check if this the first connect.
Another simple tip: you can use await ctx.send() instead of await ctx.channel.send()
You can use asyncio.create_task() to spawn the scraping in the "background":
async def scrape_and_process(...):
scrapers = [scraper_1(session, sem), scraper_2(session, sem), ...]
data = list(chain(*await asyncio.gather(*scrapers))) # this may take a while
# do stuff with data
async def main_loop():
sem = asyncio.Semaphore(60)
connector = aiohttp.TCPConnector(limit=60)
async with aiohttp.ClientSession(connector=connector, headers=headers) as session:
while True:
# main guts of loop here ...
# initiate scraping and processing in the background and
# proceed with the loop
asyncio.create_task(scrape_and_process(...))
You can try to use python threading.
Learn more here
It basically allows you to run it on different threads
example:
import threading
def 1():
print("Helo! This is the first thread")
def 2():
print("Bonjour! This is the second thread")
thread1 = threading.Thread(target=1)
thread2 = Threading.Thread(target=2)
thread1.start()
thread2.start()
Im trying to multithread async functions, one with input, could you please help me?
functions
#client.event
async def on_message(message):
print(f"{message.author.name}: {message.content}")
#client.event
async def on_ready():
while True:
message=input("> ")
run code:
if __name__ == "__main__":
try:
client.run(token, bot=True)
except:
exit(1)
off topic: the default value for bot is true, so you can just do client.run(token)
on topic: async "replaces" threading, https://stackoverflow.com/a/27265877/10192011
also if you want to have a input in discord.py you should look into tasks
Very late to the party but thought I'd share for any future readers.
Sync programming is where python goes line by line and executes every statement in order, while async programming is when you define multiple blocks of code and you set them up to run at a later time when the CPU has spare cycles.
The issue here is that you can run sync functions in an async context, and those lengthy sync tasks will block the thread, stopping also all other async tasks (hence their "blocking tasks" name).
input() is a synchronous blocking task that sits idly waiting for user input until the enter key is pressed, but since it is a blocking function it will stop all other asynchronous code, including discord.py.
What you could do is make use of a separate package, aioconsole, which, among other cool things, includes aioconsole.ainput(). This is entirely similar to input() but works asynchronously:
from aioconsole import ainput
#client.event
async def on_ready():
while True:
message = await ainput("> ")
# do whatever
Do note that the prompt you pass to ainpput() will be printed at the very beginning, if something else (due to this being an asynchronous context) prints while the user hasn't pressed Enter yet, the prompt message will stay where it is (including anything the user has typed so far) and the new printed text added after it, which may result to very confusing interactions...
Using a non async def as the thread target and run the async def from the thread target is one of the solution. Like below.
Example:
async def log_pub(self):
while 1:
sleep(1)
await self.Publish()
#Break the loop based on event
if thread_exit_event.is_set():
break
def handleThread(self):
asyncio.run(self.log_pub())
def startLogging(self):
thread = threading.Thread(target=self.handleThread, args=())
thread.start()
How do I remove the async-everywhere insanity in a program like this?
import asyncio
async def async_coro():
await asyncio.sleep(1)
async def sync_func_1():
# This is blocking and synchronous
await async_coro()
async def sync_func_2():
# This is blocking and synchronous
await sync_func_1()
if __name__ == "__main__":
# Async pollution goes all the way to __main__
asyncio.run(sync_func_2())
I need to have 3 async markers and asyncio.run at the top level just to call one async function. I assume I'm doing something wrong - how can I clean up this code to make it use async less?
FWIW, I'm interested mostly because I'm writing an API using asyncio and I don't want my users to have to think too much about whether their functions need to be def or async def depending on whether they're using a async part of the API or not.
After some research, one answer is to manually manage the event loop:
import asyncio
async def async_coro():
await asyncio.sleep(1)
def sync_func_1():
# This is blocking and synchronous
loop = asyncio.get_event_loop()
coro = async_coro()
loop.run_until_complete(coro)
def sync_func_2():
# This is blocking and synchronous
sync_func_1()
if __name__ == "__main__":
# No more async pollution
sync_func_2()
If you must do that, I would recommend an approach like this:
import asyncio, threading
async def async_coro():
await asyncio.sleep(1)
_loop = asyncio.new_event_loop()
threading.Thread(target=_loop.run_forever, daemon=True).start()
def sync_func_1():
# This is blocking and synchronous
return asyncio.run_coroutine_threadsafe(async_coro(), _loop).result()
def sync_func_2():
# This is blocking and synchronous
sync_func_1()
if __name__ == "__main__":
sync_func_2()
The advantage of this approach compared to one where sync functions run the event loop is that it supports nesting of sync functions. It also only runs a single event loop, so that if the underlying library wants to set up e.g. a background task for monitoring or such, it will work continuously rather than being spawned each time anew.
I would like to achieve the following using asyncio:
# Each iteration of this loop MUST last only 1 second
while True:
# Make an async request
sleep(1)
However, the only examples I've seen use some variation of
async def my_func():
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, requests.get, 'http://www.google.com')
loop = asyncio.get_event_loop()
loop.run_until_complete(my_func())
But run_until_complete is blocking! Using run_until_complete in each iteration of my while loop would cause the loop to block.
I've spent the last couple of hours trying to figure out how to correctly run a non-blocking task (defined with async def) without success. I must be missing something obvious, because something as simple as this should surely be simple. How can I achieve what I have described?
run_until_complete runs the main event loop. It's not "blocking" so to speak, it just runs the event loop until the coroutine you passed as a parameter returns. It has to hang because otherwise, the program would either stop or be blocked by the next instructions.
It's pretty hard to tell what you are willing to achieve, but this piece code actually does something:
async def my_func():
loop = asyncio.get_event_loop()
while True:
res = await loop.run_in_executor(None, requests.get, 'http://www.google.com')
print(res)
await asyncio.sleep(1)
loop = asyncio.get_event_loop()
loop.run_until_complete(my_func())
It will perform a GET request on Google homepage every seconds, popping a new thread to perform each request. You can convince yourself that it's actually non-blocking by running multiple requests virtually in parallel:
async def entrypoint():
await asyncio.wait([
get('https://www.google.com'),
get('https://www.stackoverflow.com'),
])
async def get(url):
loop = asyncio.get_event_loop()
while True:
res = await loop.run_in_executor(None, requests.get, url)
print(url, res)
await asyncio.sleep(1)
loop = asyncio.get_event_loop()
loop.run_until_complete(entrypoint())
Another thing to notice is that you're running requests in separate threads each time. It works, but it's sort of a hack. You should rather be using a real asynchronus HTTP client such as aiohttp.
This is Python 3.10
asyncio is single threaded execution, using await to yield the cpu to other function until what is await'ed is done.
import asyncio
async def my_func(t):
print("Start my_func")
await asyncio.sleep(t) # The await yields cpu, while we wait
print("Exit my_func")
async def main():
asyncio.ensure_future(my_func(10)) # Schedules on event loop, we might want to save the returned future to later check for completion.
print("Start main")
await asyncio.sleep(1) # The await yields cpu, giving my_func chance to start.
print("running other stuff")
await asyncio.sleep(15)
print("Exit main")
if __name__ == "__main__":
asyncio.run(main()) # Starts event loop