Develop Reference

Discord cogs doesn't load

I am trying to run two different Discord Bots using a single python script using cogs. But when I try to run the 2nd bot it throws an ImportError even-though I didn't use that specific Library. The reaction roles bot works fine without the anti spam bot. Here's my code. FYI I am working inside a Virtual Env.
main.py
if __name__ == "__main__":
try:
reaction_role_bot = commands.Bot(command_prefix=config["reaction_role_bot"]["bot_prefix"], intents=discord.Intents.all())
reaction_slash = SlashCommand(reaction_role_bot, sync_commands=True)
reaction_role_bot.load_extension(f"cogs.{str(os.path.basename('cogs/reaction_roles.py')[:-3])}")
anti_spam_bot = commands.Bot(command_prefix=config["anti_spam_bot"]["bot_prefix"], intents=discord.Intents.default())
spam_slash = SlashCommand(anti_spam_bot, sync_commands=True)
anti_spam_bot.load_extension(f"cogs.{str(os.path.basename('cogs/anti_spam.py')[:-3])}")
event_loop = asyncio.get_event_loop()
event_loop.create_task(reaction_role_bot.run(config["reaction_role_bot"]["token"]))
event_loop.create_task(anti_spam_bot.run(config["anti_spam_bot"]["token"]))
event_loop.run_forever()
except Exception as e:
print(e)
anti_spam.py
import platform
import os
import discord
from discord.ext import commands
from antispam import AntiSpamHandler
from antispam.plugins import AntiSpamTracker, Options
class AntiSpamBot(commands.Cog):
def __init__(self, client):
self.client = client
# Initialize the AntiSpamHandler
self.client.handler = AntiSpamHandler(self.client, options=Options(no_punish=True))
# 3 Being how many 'punishment requests' before is_spamming returns True
self.client.tracker = AntiSpamTracker(self.client.handler, 3)
self.client.handler.register_extension(self.client.tracker)
#commands.Cog.listener()
async def on_ready(self):
print("---------------------------------")
print(f"Logged in as {str(self.client.user)}")
print(f"Discord.py API version: {discord.__version__}")
print(f"Python version: {platform.python_version()}")
print(f"Running on: {platform.system()} {platform.release()} ({os.name})")
await self.client.change_presence(status=discord.Status.idle, activity=discord.Game(name="Head of Security"))
print("---------------------------------\n")
# The code in this event is executed every time a valid commands catches an error
#commands.Cog.listener()
async def on_command_error(context, error):
raise error
#commands.Cog.listener()
async def on_message(self, message):
await self.client.handler.propagate(message)
if self.client.tracker.is_spamming(message):
await message.delete()
await message.channel.send(f"{message.author.mention} has been automatically kicked for spamming.")
await message.author.kick()
await self.client.process_commands(message)
def setup(client):
client.add_cog(AntiSpamBot(client))
Error
Extension 'cogs.anti_spam' raised an error: ImportError: cannot import name 'AsyncMock' from 'unittest.mock' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/mock.py)
I've no experiences using cogs and its bit confusing me. Any kind of help would help me to sort this out! Thanks in advance!

I do not believe this is a cog registration issue. I believe this is an import error from some of the dependencies in your cog file. I googled your error and found something similar, I recommend checking it out here for some more information.
As a blanket statement, I would double check that you have mock installed, and that you're installing it on the version of Python that you think you're installing it on. It can get wonky if you have multiple python versions insealled.
Also, on an unrelated note:: It is best to avoid running multiple bot instances in one python file, but I can help you do it the best way possible.
For starters, you have to realize that Client.run is an abstraction of a couple of more lower level concepts.
There is Client.login which logs in the client and then Client.connect which actually runs the processing. These are coroutines.
asyncio provides the capability of putting things in the event loop for it to work whenever it has time to.
Something like this e.g.
loop = asyncio.get_event_loop()
async def foo():
await asyncio.sleep(10)
loop.close()
loop.create_task(foo())
loop.run_forever()
If we want to wait for something to happen from another coroutine, asyncio provides us with this functionality as well through the means of synchronisation via asyncio.Event. You can consider this as a boolean that you are waiting for:
e = asyncio.Event()
loop = asyncio.get_event_loop()
async def foo():
await e.wait()
print('we are done waiting...')
loop.stop()
async def bar():
await asyncio.sleep(20)
e.set()
loop.create_task(bar())
loop.create_task(foo())
loop.run_forever() # foo will stop this event loop when 'e' is set to true
loop.close()
Using this concept we can apply it to the discord bots themselves.
import asyncio
import discord
from collections import namedtuple
# First, we must attach an event signalling when the bot has been
# closed to the client itself so we know when to fully close the event loop.
Entry = namedtuple('Entry', 'client event')
entries = [
Entry(client=discord.Client(), event=asyncio.Event()),
Entry(client=discord.Client(), event=asyncio.Event())
]
# Then, we should login to all our clients and wrap the connect call
# so it knows when to do the actual full closure
loop = asyncio.get_event_loop()
async def login():
for e in entries:
await e.client.login()
async def wrapped_connect(entry):
try:
await entry.client.connect()
except Exception as e:
await entry.client.close()
print('We got an exception: ', e.__class__.__name__, e)
entry.event.set()
# actually check if we should close the event loop:
async def check_close():
futures = [e.event.wait() for e in entries]
await asyncio.wait(futures)
# here is when we actually login
loop.run_until_complete(login())
# now we connect to every client
for entry in entries:
loop.create_task(wrapped_connect(entry))
# now we're waiting for all the clients to close
loop.run_until_complete(check_close())
# finally, we close the event loop
loop.close()

How to use asyncio.wait_for to run_until_complete to synchronously call async method in Python

To allow timeouts receiving data via Python websocket, the FAQ: How do I set a timeout on recv()? recommends using asynchronously receive data:
await asyncio.wait_for(websocket.recv(), timeout=10)
Since the function I receive data is not async, I've adapted this answer to run the asyncio loop until data was received or the timeout occurred:
loop.run_until_complete(asyncio.wait_for(ws.recv(), timeout=10))
Unfortunately this statement seems to be not valid, since the following exception occurs:
An asyncio.Future, a coroutine or an awaitable is required
For me, it looks like asyncio.wait_for is no valid parameter for the run_until_complete although the documentation clearly shows an example which awaits for it.
What am I missing here - and what would be the correct way to use asyncio.wait_for in a synchronous method?

You need to feed loop.run_until_complete with a coroutine. To do so, you can wrap your receiving code into an async function:
async def receive_message():
return await asyncio.wait_for(ws.recv(), timeout=10)
loop.run_until_complete(receive_message())
Here's a fully working code:
import asyncio
import websockets
URI = "ws://0.0.0.0:8765"
TIMEOUT = 2
async def create_ws():
return await websockets.connect(URI)
async def receive_message():
ws = await create_ws()
print("Connected")
message = await asyncio.wait_for(ws.recv(), timeout=TIMEOUT)
print(f"Received message in less than {TIMEOUT} seconds: {message}")
if __name__ == "__main__":
asyncio.get_event_loop().run_until_complete(receive_message())

There are two common websocket modules available in Python.
While websockets is imported with import websockets there is also the websocket-client which is imported with import websocket.
Both of the modules offering nearly the same API where ws.recv() allows to receive data.
The fact the modules are so similar to each other may cause confusion which at least lead to my exception.
While websockets are capable of async operations, websocket is not. This means my statement
loop.run_until_complete(asyncio.wait_for(ws.recv(), timeout=10))
will only work for websockets, if used with websocket, the exception
An asyncio.Future, a coroutine or an awaitable is required
will occur.

Asyncio event loop is closed when using asyncio.run()

I'm getting started to AsyncIO and AioHTTP, and i'm writing some basic code to get familiar with the syntax. I tried the following code that should perform 3 requests concurrently:
import time
import logging
import asyncio
import aiohttp
import json
from aiohttp import ClientSession, ClientResponseError
from aiocfscrape import CloudflareScraper
async def nested(url):
async with CloudflareScraper() as session:
async with session.get(url) as resp:
return await resp.text()
async def main():
URL = "https://www.binance.com/api/v3/exchangeInfo"
await asyncio.gather(nested(URL), nested(URL), nested(URL))
asyncio.run(main())
Here is the output:
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
I don't understand why do i get that error, can anyone help me on this?

Update
Originally I was recommending Greg's answer below:
import asyncio
import sys
if sys.platform:
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
Turned out, using WindowsSelectorEventLoop has functionality issues such as:
Can't support more than 512 sockets
Can't use pipe
Can't use subprocesses
due to the fact that Windows uses I/O completion Ports unlike *nix - Therefore SelectorEventLoop is not designed for Windows nor is implemented as full.
If those limitations matters to you - You might be better off using lengthy workaround in this answer.
Check out more about differences at documents.
Or alternatively, consider using Trio over asyncio, which is much more stable and consistent.
import trio
async def task():
await trio.sleep(5)
trio.run(task)
Original post
I've finally figured out how to keep ProactorEventLoop running, preventing unsuccessful IO closure.
Really not sure why windows' Event loop is so faulty, as this also happens for asyncio.open_connection and asyncio.start_server.
To workaround this, you need to run event loop in forever loop and close manually.
Following code will cover both windows and other environments.
import asyncio
from aiocfscrape import CloudflareScraper
async def nested(url):
async with CloudflareScraper() as session:
async with session.get(url) as resp:
return await resp.text()
async def main():
await nested("https://www.binance.com/api/v3/exchangeInfo")
try:
assert isinstance(loop := asyncio.new_event_loop(), asyncio.ProactorEventLoop)
# No ProactorEventLoop is in asyncio on other OS, will raise AttributeError in that case.
except (AssertionError, AttributeError):
asyncio.run(main())
else:
async def proactor_wrap(loop_: asyncio.ProactorEventLoop, fut: asyncio.coroutines):
await fut
loop_.stop()
loop.create_task(proactor_wrap(loop, main()))
loop.run_forever()
This code will check if new EventLoop is ProactorEventLoop.
If so, keep loop forever until proactor_wrap awaits main and schedules loop stop.
Else - possibly all other OS than Windows - doesn't need these additional steps, simply call asyncio.run() instead.
IDE like Pycharm will complain about passing AbstractEventLoop to ProactorEventLoop parameter, safe to ignore.

Whilst this has been answered and accepted. You can fix this issue with one line of code: asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
Event loop is closed is a known issue on Windows (see https://github.com/encode/httpx/issues/914). I suspect this will be fixed in later versions of Python. To get around the error, simply set the event loop policy to WindowsSelectorEventLoopPolicy.
If you plan to run the code on non-windows environment; then you'll want to either add an if statement to prevent error. E.g: if sys.platform == 'win32'. Or add code to set the policies.
Working example:
import asyncio
from aiocfscrape import CloudflareScraper
import sys
async def nested(url):
async with CloudflareScraper() as session:
async with session.get(url) as resp:
print(resp.status)
return await resp.text()
async def main():
URL = "https://www.binance.com/api/v3/exchangeInfo"
await asyncio.gather(nested(URL), nested(URL), nested(URL))
# Only preform check if your code will run on non-windows environments.
if sys.platform == 'win32':
# Set the policy to prevent "Event loop is closed" error on Windows - https://github.com/encode/httpx/issues/914
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())

Asynchronous Python server: fire and forget at startup

I am writing a server that must handle asynchronous tasks. I'd rather stick to asyncio for the asynchronous code, so I chose to use Quart[-OpenAPI] framework with Uvicorn.
Now, I need to run a task (master.resume() in the code below) when the server is starting up without waiting for it to finish, that is, firing and forgetting it.
I'm not sure if it's even possible with asyncio, as I cannot await for this task but if I don't I get a coroutine X was never awaited error. Using loop.run_until_complete() as suggested in this answer would block the server until the task completes.
Here's a skeleton of the code that I have:
import asyncio
from quart_openapi import Pint, Resource
app = Pint(__name__, title="Test")
class Master:
...
async def resume():
await coro()
async def handle_get()
...
#app.route("/test")
class TestResource(Resource):
async def get(self):
print("Received get")
asyncio.create_task(master.handle_get())
return {"message": "Request received"}
master = Master()
# How do I fire & forget master.resume()?
master.resume() # <== This throws "RuntimeWarning: coroutine 'Master.resume' was never awaited"
asyncio.get_event_loop().run_until_complete(master.resume()) # <== This prevents the server from starting
Should this not be achievable with asyncio/Quart, what would be the proper way to do it?

It is see these docs, in summary,
#app.before_serving
async def startup():
asyncio.ensure_future(master.resume())
I'd hold on to the task though, so that you can cancel it at shutdown,
#app.before_serving
async def startup():
app.background_task = asyncio.ensure_future(master.resume())
#app.after_serving
async def shutdown():
app.background_task.cancel() # Or something similar

Python Asyncio run_forever() and Tasks

I adapted this code for using Google Cloud PubSub in Async Python: https://github.com/cloudfind/google-pubsub-asyncio
import asyncio
import datetime
import functools
import os
from google.cloud import pubsub
from google.gax.errors import RetryError
from grpc import StatusCode
async def message_producer():
""" Publish messages which consist of the current datetime """
while True:
await asyncio.sleep(0.1)
async def proc_message(message):
await asyncio.sleep(0.1)
print(message)
message.ack()
def main():
""" Main program """
loop = asyncio.get_event_loop()
topic = "projects/{project_id}/topics/{topic}".format(
project_id=PROJECT, topic=TOPIC)
subscription_name = "projects/{project_id}/subscriptions/{subscription}".format(
project_id=PROJECT, subscription=SUBSCRIPTION)
subscription = make_subscription(
topic, subscription_name)
def create_proc_message_task(message):
""" Callback handler for the subscription; schedule a task on the event loop """
print("Task created!")
task = loop.create_task(proc_message(message))
subscription.open(create_proc_message_task)
# Produce some messages to consume
loop.create_task(message_producer())
print("Subscribed, let's do this!")
loop.run_forever()
def make_subscription(topic, subscription_name):
""" Make a publisher and subscriber client, and create the necessary resources """
subscriber = pubsub.SubscriberClient()
try:
subscriber.create_subscription(subscription_name, topic)
except:
pass
subscription = subscriber.subscribe(subscription_name)
return subscription
if __name__ == "__main__":
main()
I basically removed the publishing code and only use the subscription code.
However, initially I did not include the loop.create_task(message_producer()) line. I figured that tasks were created as they were supposed to however they never actually run themselves. Only if I add said line the code properly executes and all created Tasks run. What causes this behaviour?

PubSub is calling the create_proc_message_task callback from a different thread. Since create_task is not thread-safe, it must only be called from the thread that runs the event loop (typically the main thread). To correct the issue, replace loop.create_task(proc_message(message)) with asyncio.run_coroutine_threadsafe(proc_message(message), loop) and message_producer will no longer be needed.
As for why message_producer appeared to fix the code, consider that run_coroutine_threadsafe does two additional things compared to create_task:
It operates in a thread-safe fashion, so the event loop data structures are not corrupted when this is done concurrently.
It ensures that the event loop wakes up at the soonest possible opportunity, so that it can process the new task.
In your case create_task added the task to the loop's runnable queue (without any locking), but failed to ensure the wakeup, because that is not needed when running in the event loop thread. The message_producer then served to force the loop to wake up in regular intervals, which is when it also checks and executes the runnable tasks.