Im developing a discord covid tracker bot and the information is scraped from a facebook page. I successfully scrape the info and store it to a list, however when I run the bot the bot will work at first but for every 5 minute the bot will be disconnect and not responding because def scrape will be refresh every 5min. So my question is how can I keep the bot to work even the scrape function is looping?
My code:
import discord
import random
import time
import asyncio
from facebook_scraper import get_posts
from discord.ext import commands, tasks
listposts = []
token = 'xxxx'
client = discord.Client()
listposts = []
#tasks.loop(minutes=5)
async def scrape():
wanted = "Pecahan setiap negeri (Kumulatif)" # wanted post
for post in get_posts("myhealthkkm", pages=5):
if post.get("text") is not None and wanted in post.get("text"):
# print("Found", t)
listposts.append(post.get("text"))
else:
pass
# print("Not found")
print(listposts)
#client.event
async def on_message(message):
if message.content.startswith("-malaysiacase"):
await message.channel.send(listposts)
#client.event
async def on_ready():
print("RUN")
scrape.start()
client.run(token)
Result:
After 5 minutes (the scrape function reload), the bot couldn't respond to user request anymore until the scraping process completed
facebook_scraper module is not asynchronous therefore blocking, which will make your bot freeze till it has completed and miss heartbeats which causes disconnect.
Do not use time module inside a discord bot either you have to use asyncio.sleep for the same reason.
Some alternatives you can do are: use BS4 in conjunction with AIOHTTP.
Or look at running your sync function in loop.run_in_executor.
There are some examples here: Python lib beautiful soup using aiohttp
Related
I have a Discord bot in Python / Discord.py where people can enter commands, and normally the bot responds very quickly.
However the bot is also gathering/scraping webdata every iteration of the main loop. Normally the scraping is pretty short and sweet so nobody really notices, but from time to time the code is set up to do a more thorough scraping which takes a lot more time. But during these heavy scrapings, the bot is sort of unresponsive to user commands.
#bot.command()
async def sample_command(ctx):
# may actually take a while for this command to respond if we happen to be
# in the middle of a heavier site scrape
await ctx.channel.send("Random message, something indicating bot has responded")
async def main_loop():
sem = asyncio.Semaphore(60)
connector = aiohttp.TCPConnector(limit=60)
async with aiohttp.ClientSession(connector=connector, headers=headers) as session:
while True:
# main guts of loop here ...
scrapers = [scraper_1(session, sem), scraper_2(session, sem), ...]
data = list(chain(*await asyncio.gather(*scrapers))) # this may take a while
# do stuff with data
Is there a way to sort of have it go "Hey, you want to do a heavy scrape, fine go process it elsewhere - meanwhile let's continue with the main loop and I'll hook back up with you later when you're done and we'll process the data then", if that makes sense?
I mainly want to separate this scraping step so it's not holding up the ability for people to actually interact with the rest of the bot.
You can use the discord.py tasks extension docs.
For example:
from discord.ext import tasks
#bot.event()
async def on_ready():
main_loop.start()
#bot.command()
async def sample_command(ctx):
await ctx.channel.send("Random message, something indicating bot has responded")
#tasks.loop(seconds=60)
async def main_loop():
do_something()
Note: It's not recommended to start the tasks in on_ready because the bot will reconnect to discord and the task will start several times, Put it somewhere else or on_ready check if this the first connect.
Another simple tip: you can use await ctx.send() instead of await ctx.channel.send()
You can use asyncio.create_task() to spawn the scraping in the "background":
async def scrape_and_process(...):
scrapers = [scraper_1(session, sem), scraper_2(session, sem), ...]
data = list(chain(*await asyncio.gather(*scrapers))) # this may take a while
# do stuff with data
async def main_loop():
sem = asyncio.Semaphore(60)
connector = aiohttp.TCPConnector(limit=60)
async with aiohttp.ClientSession(connector=connector, headers=headers) as session:
while True:
# main guts of loop here ...
# initiate scraping and processing in the background and
# proceed with the loop
asyncio.create_task(scrape_and_process(...))
You can try to use python threading.
Learn more here
It basically allows you to run it on different threads
example:
import threading
def 1():
print("Helo! This is the first thread")
def 2():
print("Bonjour! This is the second thread")
thread1 = threading.Thread(target=1)
thread2 = Threading.Thread(target=2)
thread1.start()
thread2.start()
I built a bot that updates the name of the bot to a price in Python. Likewise, the status of the bot updates to a different price as well. The bot works as intended for a bit of time, but then I receive the following message from Discord:
It appears your bot, "MY BOT NAME", has connected to Discord more than 1000 times within a short time period. Since this kind of behavior is usually a result of a bug we have gone ahead and reset your bot's token.
My bot runs every 15 seconds, from a shell script, on a linux server, that also kills the last process which was run before it. I kill the preceding process so they don't eat up my memory and crash my server.
Here is my code:
#!/usr/bin/python
import discord
from discord.ext import commands
import requests
import json
import emoji
import sys
import asyncio
client = commands.Bot(command_prefix = '.')
url = 'URL FOR THE API'
price = requests.get(url)
rapid_gprice = price.json()['data']['price1']
standardp = price.json()['data']['price2']
#client.event
async def on_ready():
guild = client.get_guild(MY GUILD ID)
me = guild.me
await me.edit(nick=standardp)
activity = discord.Game(name=rapid_gprice)
await client.change_presence(status=discord.Status.online, activity=activity)
client.run('MY TOKEN')
I'm fairly certain I need to use some sort of function inside my Python script that loops through the price api and updates the Discord bot accordingly, only having to run the Python script once.
I'm happy to provide any additional information you may need. Thanks in advance.
Running your script every 15 seconds is a really bad idea, there's an build-in discord.py extension called tasks, it lets you run background tasks, hence the name.
from discord.ext import tasks
#tasks.loop(seconds=15)
async def change_nick(guild):
nick = # Get the nick here
await guild.me.edit(nick=nick)
#client.event
async def on_ready():
await client.wait_until_ready()
guild = client.get_guild(MY GUILD ID)
change_nick.start(guild)
Also I see that you're using requests, which is a blocking library, I'd suggest you using aiohttp instead
Take a look at the tasks introduction
This is how I ended up solving my issue. It's probably code overkill but it works. Feel free to help me alter it to make it more efficient:
#!/usr/bin/python
from discord.ext import tasks, commands
import aiohttp
import json
import asyncio
import discord
client = commands.Bot(command_prefix = '.')
url = 'THE API LINK I''M USING'
async def getPrice1():
async with aiohttp.ClientSession() as session:
async with session.get(url) as resp:
response = await resp.json()
price1 = response['data']['priceZYX']
return price1
async def getPrice2():
async with aiohttp.ClientSession() as session:
async with session.get(url) as resp:
response = await resp.json()
price2 = response['data']['priceXYZ']
return price2
#tasks.loop(seconds=7)
async def change_nick(guild):
nick = await getPrice1()
await guild.me.edit(nick=nick)
activity = discord.Game(name=await getPrice2())
await client.change_presence(status=discord.Status.online, activity=activity)
#client.event
async def on_ready():
await client.wait_until_ready()
guild = client.get_guild(MY GUILD ID)
change_nick.start(guild)
I got an error trying to implement the pricedata (from url) to my bot's username.
The error: discord.errors.HTTPException: 400 Bad Request (error code: 50035): Invalid Form Body
In nick: Could not interpret "(the live price)" as string.
If you have the same problem (and using discord rewrite), make sure to label (nick=nick) as a string:
#tasks.loop(seconds=15)
async def change_nick(guild):
nick = await getPrice1()
await guild.me.edit(nick=str(nick))
PS: Im fairly new to coding so please correct me if im wrong, it worked for me. Also thanks for this posts it helped me alot making my bot.
So the goal of this background task is to wait until the bot is ready and then perform the function monitorGame() unfortunately this is not the case when I run the bot.
Code With Problem:
import json
import requests
import discord
from gamestop import monitorGame
from datetime import datetime
from time import sleep
from discord.ext import tasks, commands
from discord.ext.commands import errors
from discord.utils import get
from discord_webhook import DiscordWebhook, DiscordEmbed
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from bs4 import BeautifulSoup
# Prefix
bot = commands.Bot(command_prefix='/')
bot.remove_command("help")
# Bot Event
#bot.event
async def on_ready():
print("[+] Bot Is Alive [+]")
await bot.change_presence(activity=discord.Activity(type=discord.ActivityType.watching, name=f"Game"))
# Start Monitor
#tasks.loop(seconds=1)
async def monitor():
# Wait Until Bot Is Ready
await bot.wait_until_ready()
# Run Function
monitorGame()
# Start Task
monitor.start()
# Run bot
bot.run(token)
Since the function monitorGame() uses selenium, what tells me that the function is running before the bot is ready is that a browser opens before the bot prints Bot Is Alive.
What ends up happening is the function runs before the bot is ready and just screws everything up.
Some side notes:
The function includes selenium and opens a browser.
I am using repl to run this bot.
The function monitorGame() essentially opens a browser and parses the html that's about it.
You could simply check how long it takes to start up the selenium part, then mesure the time and insert a time.sleep('seconds') until it is started up, probably the easiest fix in this case
As mentioned in the title, how do I do it? So far I spent a couple of hours trying to find an answer but since this isn't a popular request there isn't anything about it. Most of what I found is just using youtube-dl for downloading yt audio and playing it.
Edit/Update to use FFmpegPCMAudio to stream.
You'll also need to workout your streaming url for your desired channel. Radio Paradise provides some links and I'm sure there are many others.
You can try the following to get you rolling, of course the member entering the commands needs to be in the voice channel:
import os
from discord import FFmpegPCMAudio
from discord.ext.commands import Bot
from dotenv import load_dotenv
load_dotenv()
TOKEN = os.getenv('DISCORD_TOKEN')
PREFIX = os.getenv('DISCORD_PREFIX')
client = Bot(command_prefix=list(PREFIX))
#client.event
async def on_ready():
print('Music Bot Ready')
#client.command(aliases=['p', 'pla'])
async def play(ctx, url: str = 'http://stream.radioparadise.com/rock-128'):
channel = ctx.message.author.voice.channel
global player
try:
player = await channel.connect()
except:
pass
player.play(FFmpegPCMAudio('http://stream.radioparadise.com/rock-128'))
#client.command(aliases=['s', 'sto'])
async def stop(ctx):
player.stop()
client.run(TOKEN)
Would this be of any use? Potentially use this example as a skeleton for your bot. Not needing to use any YT-DL - proves that you can use local mp3 files.
I have a problem with discord's bot.
I have done a script which calculate the weather every hour and I want to send the result to discord with bot:
import Secret
import discord
result = "temp is ...."
TOKEN = Secret.BOT_TOKEN
client = discord.Client()
client.send(result)
client.run(TOKEN)
I have searched on google but I have not found an answer to send the result automatically.
Can you help me?
If you're using python just put it in a while loop and have it send the message and sleep for an hour.
for sleep you can import time
Something like:
while true:
client.send(result)
sleep(3600)
Important:
Your bot will be 100% inactive while you sleep it, so if you use it for more than just weather this might not be the solution you're looking for.
Using the time module prevents your bot from doing anything else during the time that it is sleeping.
Use tasks. Just put what you want executed in the task() function:
from discord.ext import tasks
#tasks.loop(seconds=60) # repeat once every 60 seconds
async def task():
pass
mytask.start()