Telethon python keeps re-downloading photos - python

I've been scratching my head for the past half hour, but I can't fix this error, let me explain.
Basically what I'm trying to do is to make a simple script that will download both texts and photos from the last 6 days, the script works fine in everything except that when I re-run it, it will download again photos ending up with duplicated images.
This is the code:
from datetime import timedelta
from datetime import datetime
from telethon import TelegramClient
date = datetime.now() + timedelta(-6)
api_id = 123
api_hash = 'XXX'
client = TelegramClient('SESSION', api_id, api_hash)
#Blank files have been already created
async def main():
with open("downloadedids.txt", "r+") as archivio, open("messagesaved.txt", "r+") as scaricare:
async for message in client.iter_messages('test', offset_date=date, reverse=True): ##I want to download only messages 6 days ago to now
if str(message.id) not in str(archivio.read()):
if (message.photo):
await message.download_media("Include")
scaricare.write(str(message.text))
archivio.write(str(message.id) + "\n")
with client:
client.loop.run_until_complete(main())
Maybe the problem is related to async?

It's downloading it again because you are telling it to! Perhaps you want to do a checksum on the existing file and if it matches, don't keep the second one?

Related

telegram bot in python for sending files doesnt work, but I dont know why [duplicate]

Hi i want to send message from bot in specific time (without message from me), for example every Saturday morning at 8:00am.
Here is my code:
import telebot
import config
from datetime import time, date, datetime
bot = telebot.TeleBot(config.bot_token)
chat_id=config.my_id
#bot.message_handler(commands=['start', 'help'])
def print_hi(message):
bot.send_message(message.chat.id, 'Hi!')
#bot.message_handler(func=lambda message: False) #cause there is no message
def saturday_message():
now = datetime.now()
if (now.date().weekday() == 5) and (now.time() == time(8,0)):
bot.send_message(chat_id, 'Wake up!')
bot.polling(none_stop=True)
But ofc that's not working.
Tried with
urlopen("https://api.telegram.org/bot" +bot_id+ "/sendMessage?chat_id=" +chat_id+ "&text="+msg)
but again no result. Have no idea what to do, help please with advice.
I had this same issue and I was able to solve it using the schedule library. I always find examples are the easiest way:
import schedule
import telebot
from threading import Thread
from time import sleep
TOKEN = "Some Token"
bot = telebot.TeleBot(TOKEN)
some_id = 12345 # This is our chat id.
def schedule_checker():
while True:
schedule.run_pending()
sleep(1)
def function_to_run():
return bot.send_message(some_id, "This is a message to send.")
if __name__ == "__main__":
# Create the job in schedule.
schedule.every().saturday.at("07:00").do(function_to_run)
# Spin up a thread to run the schedule check so it doesn't block your bot.
# This will take the function schedule_checker which will check every second
# to see if the scheduled job needs to be ran.
Thread(target=schedule_checker).start()
# And then of course, start your server.
server.run(host="0.0.0.0", port=int(os.environ.get('PORT', 5000)))
I hope you find this useful, solved the problem for me :).
You could manage the task with cron/at or similar.
Make a script, maybe called alarm_telegram.py.
#!/usr/bin/env python
import telebot
import config
bot = telebot.TeleBot(config.bot_token)
chat_id=config.my_id
bot.send_message(chat_id, 'Wake up!')
Then program in cron like this.
00 8 * * 6 /path/to/your/script/alarm_telegram.py
Happy Coding!!!
If you want your bot to both schedule a message and also get commands from typing something inside, you need to put Thread in a specific position (took me a while to understand how I can make both polling and threading to work at the same time).
By the way, I am using another library here, but it would also work nicely with schedule library.
import telebot
from apscheduler.schedulers.blocking import BlockingScheduler
from threading import Thread
def run_scheduled_task():
print("I am running")
bot.send_message(some_id, "This is a message to send.")
scheduler = BlockingScheduler(timezone="Europe/Berlin") # You need to add a timezone, otherwise it will give you a warning
scheduler.add_job(run_scheduled_task, "cron", hour=22) # Runs every day at 22:00
def schedule_checker():
while True:
scheduler.start()
#bot.message_handler(commands=['start', 'help'])
def print_hi(message):
bot.send_message(message.chat.id, 'Hi!')
Thread(target=schedule_checker).start() # Notice that you refer to schedule_checker function which starts the job
bot.polling() # Also notice that you need to include polling to allow your bot to get commands from you. But it should happen AFTER threading!

Discord.py 2.0 on Repl.it

I want to make discord bot that uses buttons I downloaded library and ewerything. It works but because I am using Repl.it i have to use web + uptimerobot so it will never turn off.
Even after that it seems every few hours to quickly turn off and back on, so it will lose all data except databases. I was thinking i can solve this by saving message to database and inside event on_ready delete message and create new one but bot after going off and on cant delete message from varriable.
Code for web:
from flask import Flask
from threading import Thread
app = Flask('')
#app.route('/')
def home():
return "I am working!"
def run():
app.run(host='0.0.0.0',port=8080)
def keep_alive():
t = Thread(target=run)
t.start()
I would like to solve turning off and on problem + it will be helpfull to fix deleting message from database after reseting.
Side question: Do you guys know any easy way to assign instead of reference value to varriable because of circular reference/dependency.
Make a JSON file to save the data and just have it not delete the data on restart. have it autosave when a user uses it.
kinda like this
#client.command()
async def add(ctx, meme_):
def add_memes(meme, file="memes.json"):
with open(file, "r+") as fw:
j = json.load(fw)
j["memes"].append(meme)
with open(file, "w+") as wp:
wp.write(json.dumps(j))
try:
with open("memes.json", "r"):
pass
except:
with open("memes.json", "w+") as wp:
wp.write('{"memes" : []}')
finally:
add_memes(meme_)
await ctx.send("Done!")
but this is a command for saving memes

lyricsgenius lyrics sometimes end with "EmbedShare URLCopyEmbedCopy"

I am making a Discord lyrics bot and to receive the lyrics. I am using genius API (lyricsgenius API wrapper). But when I receive the lyrics, it ends with this:
"away" is the last word in the song but it is accompanied with EmbedShare URLCopyEmbedCopy. Sometimes it is just the plain lyrics without the EmbedShare text.
With the same song:
Is there anyway to prevent that?
Source code for the lyrics command:
#commands.command(help="Gives the lyrics of the song XD! format //lyrics (author) (song name)")
async def lyrics(self, ctx, arg1, arg2):
song = genius.search_song(arg1, arg2)
print(song.lyrics)
name = ("Lyrics for " + arg2.capitalize() + " by " + arg1.capitalize())
gembed = discord.Embed(title=name.capitalize(), description=song.lyrics)
await ctx.send(embed=gembed)
This is a known bug with lyricsgenius and there's an open PR to address this issue: https://github.com/johnwmillr/LyricsGenius/pull/215.
This is because lyricsgenius web scrapes the lyrics from Genius' website, which means if their website updates, lyricsgenius would fail to fetch the lyrics. This library hasn't been updated in 6 months; itself being a web scraping library means that kind of inactivity would render the library severely unstable. Since the library is licensed under MIT, you can fork the library and maintain an up-to-date version for your project/bot. However, it would be much better to use a dedicated API to fetch songs lyrics to guarantee stability.
Also, lyricsgenius uses the synchronous requests library, which means it'll "block" your asynchronous bot while it fetches the lyrics. This is definitely undesirable for a Discord Bot since your bot would be completely unresponsive while it fetches the lyrics. Consider rewriting it using aiohttp or use run_in_executor when calling blocking functions.
Some Random API is something easy to deal with when you are creating a command that will send you the song lyrics.
This is how to do it with some random api,
# these imports are used for this particular lyrics command. the essential import here is aiohttp, which will be used to fetch the lyrics from the API
import textwrap
import urllib
import aiohttp
import datetime
#bot.command(aliases = ['l', 'lyrc', 'lyric']) # adding aliases to the command so they they can be triggered with other names
async def lyrics(ctx, *, search = None):
"""A command to find lyrics easily!"""
if not search: # if user hasnt given an argument, throw a error and come out of the command
embed = discord.Embed(
title = "No search argument!",
description = "You havent entered anything, so i couldnt find lyrics!"
)
return await ctx.reply(embed = embed)
# ctx.reply is available only on discord.py version 1.6.0, if you have a version lower than that use ctx.send
song = urllib.parse.quote(search) # url-encode the song provided so it can be passed on to the API
async with aiohttp.ClientSession() as lyricsSession:
async with lyricsSession.get(f'https://some-random-api.ml/lyrics?title={song}') as jsondata: # define jsondata and fetch from API
if not 300 > jsondata.status >= 200: # if an unexpected HTTP status code is recieved from the website, throw an error and come out of the command
return await ctx.send(f'Recieved poor status code of {jsondata.status}')
lyricsData = await jsondata.json() # load the json data into its json form
error = lyricsData.get('error')
if error: # checking if there is an error recieved by the API, and if there is then throwing an error message and returning out of the command
return await ctx.send(f'Recieved unexpected error: {error}')
songLyrics = lyricsData['lyrics'] # the lyrics
songArtist = lyricsData['author'] # the author's name
songTitle = lyricsData['title'] # the song's title
songThumbnail = lyricsData['thumbnail']['genius'] # the song's picture/thumbnail
# sometimes the song's lyrics can be above 4096 characters, and if it is then we will not be able to send it in one single message on Discord due to the character limit
# this is why we split the song into chunks of 4096 characters and send each part individually
for chunk in textwrap.wrap(songLyrics, 4096, replace_whitespace = False):
embed = discord.Embed(
title = songTitle,
description = chunk,
color = discord.Color.blurple(),
timestamp = datetime.datetime.utcnow()
)
embed.set_thumbnail(url = songThumbnail)
await ctx.send(embed = embed)

how to schedule a telegram bot to send a message?

I am trying to create a Telegram bot that sends a message at a specific time, 5:30pm. However, the ways a was trying are not correct.
I wanted to trigger send_message regarding to the time and without the necessity of the user to send any /command.
import telebot
import datetime
TOKEN = 'MyToken'
bot = telebot.TeleBot(TOKEN)
#bot.message_handler(commands=['start'])
def send_welcome(message):
message_id=message.chat.id
bot.reply_to(message,"Welcome")
bot.polling()
Until now I was trying to add something like that, of course it is not python but kind of pseudocode just to explain:
if currenttime=17:30
send_message(idchat, "mymessage")
Thank you in advance.
If I understand correctly, you need to check your system time before sending a message, you could use the following code [source]:
from datetime import datetime
now = datetime.now()
current_time = now.strftime("%H:%M:%S")
print("Current Time =", current_time)
To send a message you could use the following code [source]:
def test_send_message():
text = 'CI Test Message'
tb = telebot.TeleBot(TOKEN)
ret_msg = tb.send_message(CHAT_ID, text)
assert ret_msg.message_id
To compare the time, you may use:
if current_time=='17:30:00':
test_send_message()
You can use schedule python library.
First of all import the necessary modules
import schedule
import time
import telegram
Then you have to create a function that will repeat in every x time.
Here the bot will send me message in every 5 seconds.
TOKEN='your_token'
bot = telegram.Bot(token=TOKEN)
def job():
bot.sendMessage(chat_id=<chat_id>, text="Hello")
schedule.every(5).seconds.do(job)
while True:
schedule.run_pending()
time.sleep(1)
For more informations about schedule Read the documentation

Python - Poloniex Push API

I am trying to get live data in Python 2.7.13 from Poloniex through the push API.
I read many posts (including How to connect to poloniex.com websocket api using a python library) and I arrived to the following code:
from autobahn.twisted.wamp import ApplicationSession
from autobahn.twisted.wamp import ApplicationRunner
from twisted.internet.defer import inlineCallbacks
import six
class PoloniexComponent(ApplicationSession):
def onConnect(self):
self.join(self.config.realm)
#inlineCallbacks
def onJoin(self, details):
def onTicker(*args):
print("Ticker event received:", args)
try:
yield self.subscribe(onTicker, 'ticker')
except Exception as e:
print("Could not subscribe to topic:", e)
def main():
runner = ApplicationRunner(six.u("wss://api.poloniex.com"), six.u("realm1"))
runner.run(PoloniexComponent)
if __name__ == "__main__":
main()
Now, when I run the code, it looks like it's running successfully, but I don't know where I am getting the data. I have two questions:
I would really appreciate if someone could walk me through the process of subscribing and getting ticker data, that I will elaborate in python, from step 0: I am running the program on Spyder on Windows. Am I supposed to activate somehow Crossbar?
How do I quit the connection? I simply killed the process with Ctrl+c and now when I try to run it agan, I get the error: ReactorNonRestartable.
I ran into a lot of issues using Poloniex with Python2.7 but finally came to a solution that hopefully helps you.
I found that Poloniex has pulled support for the original WAMP socket endpoint so I would probably stray from this method altogether. Maybe this is the entirety of the answer you need but if not here is an alternate way to get ticker information.
The code that ended up working best for me is actually from the post you linked to above but there was some info regarding currency pair ids I found elsewhere.
import websocket
import thread
import time
import json
def on_message(ws, message):
print(message)
def on_error(ws, error):
print(error)
def on_close(ws):
print("### closed ###")
def on_open(ws):
print("ONOPEN")
def run(*args):
# ws.send(json.dumps({'command':'subscribe','channel':1001}))
ws.send(json.dumps({'command':'subscribe','channel':1002}))
# ws.send(json.dumps({'command':'subscribe','channel':1003}))
# ws.send(json.dumps({'command':'subscribe','channel':'BTC_XMR'}))
while True:
time.sleep(1)
ws.close()
print("thread terminating...")
thread.start_new_thread(run, ())
if __name__ == "__main__":
websocket.enableTrace(True)
ws = websocket.WebSocketApp("wss://api2.poloniex.com/",
on_message = on_message,
on_error = on_error,
on_close = on_close)
ws.on_open = on_open
ws.run_forever()
I commented out the lines that pull data you don't seem to want, but for reference here is some more info from that previous post:
1001 = trollbox (you will get nothing but a heartbeat)
1002 = ticker
1003 = base coin 24h volume stats
1010 = heartbeat
'MARKET_PAIR' = market order books
Now you should get some data that looks something like this:
[121,"2759.99999999","2759.99999999","2758.000000‌​00","0.02184376","12‌​268375.01419869","44‌​95.18724321",0,"2767‌​.80020000","2680.100‌​00000"]]
This is also annoying because the "121" at the beginning is the currency pair id, and this is undocumented and also unanswered in the other stack overflow question referred to here.
However, if you visit this url: https://poloniex.com/public?command=returnTicker it seems the id is shown as the first field, so you could create your own mapping of id->currency pair or parse the data by the ids you want from this.
Alternatively, something as simple as:
import urllib
import urllib2
import json
ret = urllib2.urlopen(urllib2.Request('https://poloniex.com/public?command=returnTicker'))
print json.loads(ret.read())
will return to you the data that you want, but you'll have to put it in a loop to get constantly updating information. Not sure of your needs once the data is received so I will leave the rest up to you.
Hope this helps!
I made, with the help of other posts, the following code to get the latest data using Python 3.x. I hope this helps you:
#TO SAVE THE HISTORICAL DATA (X MINUTES/HOURS) OF EVERY CRYPTOCURRENCY PAIR IN POLONIEX:
from poloniex import Poloniex
import pandas as pd
from time import time
import os
api = Poloniex(jsonNums=float)
#Obtains the pairs of cryptocurrencies traded in poloniex
pairs = [pair for pair in api.returnTicker()]
i = 0
while i < len(pairs):
#Available candle periods: 5min(300), 15min(900), 30min(1800), 2hr(7200), 4hr(14400), and 24hr(86400)
raw = api.returnChartData(pairs[i], period=86400, start=time()-api.YEAR*10)
df = pd.DataFrame(raw)
# adjust dates format and set dates as index
df['date'] = pd.to_datetime(df["date"], unit='s')
df.set_index('date', inplace=True)
# Saves the historical data of every pair in a csv file
path=r'C:\x\y\Desktop\z\folder_name'
df.to_csv(os.path.join(path,r'%s.csv' % pairs[i]))
i += 1

Categories

Resources