Develop Reference

How to make duplicate exclusion for crawled links?

async def test():
old_links = open("old_links.txt", "r", encoding="utf-8")
while True:
base_url = 'https://www.cnbc.com/world/' async with aiohttp.ClientSession() as sess: async with sess.get(base_url, headers={'user-agent': 'Mozilla/5.0'}) as res:
text = await res.text()
soup = bs(text, 'lxml')
title = soup.find("div", attrs={"class":"LatestNews-container"}).a.text.strip()
link = soup.find("div", attrs={"class":"LatestNews-container"}).a["href"]
if link not in old_links:
print(f"title : {title}")
print(f"link : {link}")
f = open("old_links.txt", "w", encoding="utf-8")
f.write(f"{link}\n")
f.close()
else: print("No update")
await asyncio.sleep(3)
The site above is an example. i'm making a discord bot via python, and i'm making a crawling bot for some sites that don't support mailing service. i'm not major learning python. i just succeeded in creating this code through search and simple study, i maded a list with old_links = [] and applied it and used it. but when i reboot the discord bot, the posts that were crawled in the past are sent back to discord as messages. to solve this, i save the link of the post what bot sent the message to as a .txt file and keep it on my computer, and compare it with the link in the text file when the bot runs every certain time. i'm trying to implement sending function. i saving the crawled links in the .txt file was successful, but the function to compare the links stored in the .txt file was not implemented. how should i write the code?

You appear to be opening old_links as a text file without reading its contents. Assuming your existing links are separated by a new line, you can use
old_links = []
with open("old_links.txt", "r", encoding="utf-8") as file:
old_links = file.read().splitlines()
to create a list out of the old links read from the file. The with block makes sure the file is closed properly immediately after execution and limits its scope.
When you want to add a new link, simply append the new link to the array like so:
old_links.append(new_link)
Make sure you write the new lines back to the file after every update so that your changes save correctly:
with open("old_links.txt", "w", encoding="utf-8") as file:
file.write("\n".join(old_links))

Trying to check my .JSON file for a users ID

So my plan is to search the JSON file first and check if the user's ID is in there, if it is it should respond with "You have already registered.", if not then it should continue with the rest of the code.(which is where "data" will be added)
So far I only managed to add the "data" to the JSON file but couldn't figure out how to search for the ID eventually deleting the rest of it to find a solution.
#client.command(aliases = ['Register'])
async def register(ctx):
data = ctx.author.id, currency, Class, Cards
with open('Player_Stats.json', 'w') as f:
json.dump(data, f)
await ctx.send("Account registered!")

You should prefer a database query over a JSON file lookup. If you still want to go with the file you should probably detail the file format. Anyway you will load the file and search for the key or value on the JSON.

Can I find the column index of a csv file in discord.py

I am working on a database project, I can currently find the row in which a specific team is located (discord.py) however I would like to find the index of the row and use that to find the players associated with the team. I am using the print function to test that the bot is recieving the correct info before I put it into a message. Below is the code I have for the command.
Apologies if this is simple, I am new to reading/writing csv files in python.
import os
import csv
from discord.ext import commands
#bot.command()
async def team(ctx, args):
with open('Data.csv', 'r') as data:
csv_file = csv.reader(data)
query = args.upper()
for row in csv_file:
Team = row[0]
if query == Team:
print(args)
else:
await ctx.send("Sorry I cannot find that team")```

Telegram bot - how to return answer with value from parsed csv

I have a certain csv with a couple of fields.
The idea is:
User writes something (text) in the chat.
Bot checks if message (value) user wrote is in the csv.
If there is, bot displays fields from csv for written value.
So far I managed to parse csv file and get a list:
myTable = []
with open("table.csv", 'r', encoding='utf-8-sig') as File:
reader = csv.DictReader(File)
for row in reader:
myTable.append(row)
But I am stuck on handling user's message and returning the answer properly - this way doesn't work:
#bot.message_handler(content_types=['text'])
def send_text(message):
if message.text in myTable:
bot.reply_to(message, "Yes")
else:
bot.reply_to(message, "Nope")
Would appreciate any help.

How would i delete something from a json file

I'm not sure how to delete something from a .json file
I've tryed looking it up and it still nothing :(
#bot.command()
async def afkremoveme(ctx):
#pls help me I'm lost!
no errors

I'm not sure what you want your command to do, but here's an example of how you would implement json into discord.py.
Here, whenever the command is executed, the bot opens a json file, reads the data, and sees if the message author is in the data. If the author is in the data, the key/value pair is deleted, and the data is rewritten into the json file:
import json
#bot.command()
async def afkremoveme(ctx):
f = "yourFile.json"
author = str(ctx.author.id)
with open(f, "r") as read_file:
data = json.load(read_file)
if author in data: # if any key in the dictionary is an integer, it is converted to a string when a json file is written
del data[author]
newData = json.dumps(data, indent=4)
with open(f, "w") as write_file:
write_file.write(newData)
await ctx.send(f"{ctx.author.display_name} is no longer afk...")
This is reading a json file that looks like this (replace 000000 with your id):
{
"000000" : "afk",
"someOtherGuy" : "afk"
}
All of this uses dictionaries and the json module. If you're unfamiliar with either of the concepts, here are a few links to help you out :-)
Python Dictionaries,
Python-Json