In this shorted and simplified code I want to wait for all tasks to be completed( queue.join) and then leave.
This code open a network connection and download some data.
Unfortunately I have to push 'ctrl-c' because the 'client.run_until_disconnected()'.
The complete code works but I dont'know how to exit without keyboard interruption.
I have to schedule it with crontab so I can't use 'client.run_until_disconnected()'
EDIT : updated code
#!/usr/bin/env python3.7
import asyncio
import telethon
from telethon import TelegramClient
from telethon import functions, types
from datetime import datetime
api_id = 00000 # your api_id
api_hash = "your api hash"
async def worker(queue):
while True:
queue_book = await queue.get()
book_name = queue_book.file.name
print(book_name)
loop = asyncio.get_event_loop()
await client.download_media(queue_book,book_name)
queue.task_done()
async def main():
#Free ebook medical articles
channel = await client(functions.messages.CheckChatInviteRequest('your channel hash'))
#values message's id depend on the chosen channel
ids = [63529,63528,63527,63526,63525,63524,63523,63522]
queue = asyncio.Queue(1)
workers = [asyncio.create_task(worker(queue)) for _ in range(5)]
for booksId in ids:
async for get_book in client.iter_messages(channel.chat, ids=booksId):
await queue.put(get_book)
await queue.join()
for the_worker in workers:
the_worker.cancel()
async def wait_until(dt):
now = datetime.now()
await asyncio.sleep((dt - now).total_seconds())
async def run_at(dt, coro):
await wait_until(dt)
return await coro
client = TelegramClient("Test", api_id, api_hash)
loop = asyncio.get_event_loop()
client.start()
try:
loop = loop.create_task(run_at(datetime(2021, 2, 19, 11,00),main()))
client.run_until_disconnected()
except KeyboardInterrupt as keyint:
print("KeyboardInterrupt..")
pass
run_until_disconnected() is just a helper function, you don't have an obligation to run it if you want to control when the event loop ends. The minimal change would be to replace client.run_until_disconnected() with loop.run_forever(), and call loop.stop() at the end of main().
A more idiomatic approach would be to leave main() as it is, but instead of starting it with create_task, await it from your top-level coroutine (normally called main by convention, but you already chose that name for the other function). Then you can just call asyncio.run(actual_main()), and the program will automatically exit when main finishes. For example (untested):
# main and other functions as in the question
async def actual_main():
global client
client = TelegramClient("Test", api_id, api_hash)
await client.start()
await run_at(datetime(2021, 2, 19, 11, 00), main())
asyncio.run(actual_main())
Related
Every time a message is sent to the channel the handler function will be called because of the python decorator.
I want to to both recieve messages and print them out. This is how i imagined the program should look like.
But its not working because the event is not passed to the handler function in main.
How can i do that asynchronously, when the #client decorator is controlling the messages and stopping me from using using the handler function inside other functions, and pass on the tokenData
from telethon import TelegramClient, events
client = TelegramClient(username, api_id, api_hash)
client.start()
channel = 'https://t.me/DEXTNewPairsBotBSC'
#client.on(events.NewMessage(chats=channel))
async def handler(event):
# tokenData = event
tokenData = event.message.message
msg.append(tokenData)
await asyncio.sleep(2)
return tokenData
async def dumpIt():
await asyncio.sleep(2)
return msg[0]
async def main():
task1 = asyncio.create_task(handler(event))
task2 = asyncio.create_task(dumpIt())
await asyncio.wait([task1,task2])
client.loop.run_until_complete(main())
Use asyncio.Queue:
import asyncio
from telethon import TelegramClient, events
client = ...
queue = asyncio.Queue()
#client.on(events.NewMessage(chats=channel))
async def handler(event):
queue.put_nowait(event.message.message)
async def dumpIt():
msg = await queue.get()
return msg
async def main():
await dumpIt()
client.loop.run_until_complete(main())
As long as the asyncio event loop is running (which it is while it's "blocked" in queue.get()), Telethon will receive updates.
Binance API & python-binance offers async functionality for non-blocking execution as per discussed in Async basics for Binance.
I am using BinanceSocketManager listening (async non-blocking) to live data via websocket.
In scenarios like network intermittent connection lost, I wish to add an auto-reconnect feature to my project. But I can't seems to find any info with BinanceSocketManager. I was only able to find a guide which uses ThreadedWebsocketManager, but it was not an async implementation.
Does anyone know how to implement a Binance websocket disconnect detection and auto-reconnect mechanism?
Here is some code of what I have so far:
import asyncio
from binance import AsyncClient, BinanceSocketManager
async def main():
client = await AsyncClient.create()
await kline_listener(client)
async def kline_listener(client):
bm = BinanceSocketManager(client)
async with bm.kline_socket(symbol='BTCUSDT') as stream:
while True:
res = await stream.recv()
print(res)
# a way detect websocket error/disconnect, callback 'disconnect_callback'
async def disconnect_callback():
await client.close_connection()
await main() # restart client and kline socket
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
In case someone else is looking at this, for this, you should be looking at the BinanceAPIException. Code could look something like this then:
from binance import AsyncClient, BinanceSocketManager
from binance.exceptions import BinanceAPIException
async def main():
client = await AsyncClient.create()
bm = BinanceSocketManager(client, user_timeout=60)
# start any sockets here, i.e a trade socket
kline_candles = bm.kline_socket('BNBUSDT', interval=client.KLINE_INTERVAL_1MINUTE)
# start receiving messages
try:
status = await client.get_system_status()
print(status['msg'])
async with kline_candles as stream:
for _ in range(5):
res = await stream.recv() # create/await response
await process_message(msg=res, client=client) # process message
except BinanceAPIException as e:
print(e)
await disconnect_callback(client=client)
async def disconnect_callback(client):
await client.close_connection() # close connection
time.sleep(60) # wait a minute before restarting
await main() # restart client and kline socket
async def process_message(msg, client):
if msg['e'] == 'error':
await disconnect_callback(client=client)
print('ERROR OCCURED')
else:
candle = msg['k'] # get only the candle info within the general dict
start_time = datetime.utcfromtimestamp(candle['t']/1000).strftime('%Y-%m-%d %H:%M:%S')
close_time = datetime.utcfromtimestamp(candle['T']/1000).strftime('%Y-%m-%d %H:%M:%S')
print(f'__ start: {start_time}, close: {close_time}')
print(msg)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
The disconnect has not been tested yet, but I assume this will work. If anyone has any additional notes, just let me know.
I have tested above code and it proves rather stable.
Here are some improvements I have made.
I'm not sure what happens if your internet connection is completely gone when this line is executed:
client = await AsyncClient.create()
This could probably be solved like this (I'm open for better ideas):
while True:
try:
client = await AsyncClient.create()
except Exception as error_msg:
print(f"error: {error_msg}")
# should we add a sleep here?
# time.sleep(3)
else:
print("finally got through the loop")
break
Surrounding this with a try/except is a good idea:
bm = BinanceSocketManager(client, user_timeout=60)
The call to stream.recv() should be extended with asyncio.wait_for() to cover the situation, when there is no data coming in for a longer period of time. It usually means there's something wrong.
async with kline_candles as stream:
for _ in range(5):
try:
res = await asyncio.wait_for(stream.recv(), timeout=60) # create/await response
await process_message(msg=res, client=client) # process message
except (asyncio.TimeoutError, websockets.exceptions.ConnectionClosed, asyncio.exceptions.CancelledError, asyncio.exceptions.TimeoutError) as error_msg_1:
print(f"Error! in main loop 1:\n{error_msg_1}")
await disconnect_callback(client=client)
I am writing a bot and I need to implement the following functionality: the bot once every 10 minutes(for example) parse a certain URL and if there were changes from the previous call, writes to the chat.
Since the bot is also engaged in other things, I decided to loop the parsing in the function with sleep at the end. If there are changes, I try to send a message to the chat, but then a problem happens.
Since a successful combination of circumstances does not arise from an event in the chat, I can't pull the "entity" from the "event" for the "send_message" function. therefore, we have to get through the "get_entity" function and links to the chat as a parameter, but for some reason this does not work from another stream. below is a simplified code:
import threading, queue
from time import sleep
import asyncio
from telethon.sync import TelegramClient, events
import config as cfg
bot = TelegramClient('Bot', cfg.api_id, cfg.api_hash)
#bot.on(events.NewMessage(pattern=r'^(?i)(idchat){1}$'))
async def echoidchat(event):
channelaa = await bot.get_entity('https://t.me/elvistest')
await bot.send_message(channelaa, 'ответ')
def parseurls():
for x in range(10):
q.put(x)
pass
async def pre_sendmsg():
while True:
try:
msg = q.get_nowait()
except Exception as e:
await asyncio.sleep(1.0)
else:
await sendmsg(msg)
q.task_done()
async def sendmsg(msg):
channel = await bot.get_entity('https://t.me/elvistest')
await bot.send_message(channel, f'ответ из другого потока {msg}')
if __name__ == '__main__':
q = queue.Queue()
parseurls()
bot.start(bot_token=cfg.bot_token)
threading.Thread(target=asyncio.run, daemon=True, args=(pre_sendmsg(),)).start()
bot.run_until_disconnected()
The thing is that on the line " boot.get_entity" nothing happens. The script execution is lost somewhere and does not go further, that is, the next line with "bot. send_message" is simply not executed. however, "def echoidchat" is working at this time.
Well done!
This is work like I want.
import random
import threading, queue
from time import sleep
import asyncio
from telethon import TelegramClient, events
import config as cfg
bot = TelegramClient('Bot', cfg.api_id, cfg.api_hash)
#bot.on(events.NewMessage(pattern=r'^(?i)(idchat){1}$'))
async def echoidchat(event):
await bot.send_message(event.chat, 'ответ')
async def parseurls():
while True:
ts = abs(int(random.random()*10))
print(f'parseurls({ts})')
await sendmsg(ts)
await asyncio.sleep(ts)
async def sendmsg(msg):
print(f'sendmsg({msg}) - start')
channel = await bot.get_entity('https://t.me/elvistest')
await bot.send_message(channel, f'ответ из другого потока {msg}')
print(f'sendmsg({msg}) - done')
def main():
bot.start(bot_token=cfg.bot_token)
loop = asyncio.get_event_loop()
tasks = [
loop.create_task(parseurls()),
loop.create_task(bot.run_until_disconnected()),
]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
if __name__ == '__main__':
main()
I tried out the example below (from this page):
nc = NATS()
await nc.connect(servers=["nats://demo.nats.io:4222"])
future = asyncio.Future()
async def cb(msg):
nonlocal future
future.set_result(msg)
await nc.subscribe("updates", cb=cb)
await nc.publish("updates", b'All is Well')
await nc.flush()
# Wait for message to come in
msg = await asyncio.wait_for(future, 1)
But this only seems useful for receiving one message. How would I subscribe and keep receiving messages?
I've also seen the package example, but it seems to just play both sides of the conversation, then quit.
You can find a long running service example as well: https://github.com/nats-io/nats.py/blob/master/examples/service.py
import asyncio
from nats.aio.client import Client as NATS
async def run(loop):
nc = NATS()
async def disconnected_cb():
print("Got disconnected...")
async def reconnected_cb():
print("Got reconnected...")
await nc.connect("127.0.0.1",
reconnected_cb=reconnected_cb,
disconnected_cb=disconnected_cb,
max_reconnect_attempts=-1,
loop=loop)
async def help_request(msg):
subject = msg.subject
reply = msg.reply
data = msg.data.decode()
print("Received a message on '{subject} {reply}': {data}".format(
subject=subject, reply=reply, data=data))
await nc.publish(reply, b'I can help')
# Use queue named 'workers' for distributing requests
# among subscribers.
await nc.subscribe("help", "workers", help_request)
print("Listening for requests on 'help' subject...")
for i in range(1, 1000000):
await asyncio.sleep(1)
try:
response = await nc.request("help", b'hi')
print(response)
except Exception as e:
print("Error:", e)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
loop.run_forever()
loop.close()
Don't know much about python, but looks like you are just waiting for one message and program ends. You should look at the subscriber example here. As you can see, there is a loop to wait forever or for the SIGTERM signal.
I am trying to set up a schedule to run a subroutine. I am trying to use the subroutine example to send a message to a discord channel when the schedule is triggered. At first I attempted to try and just send the message but got an error. I then tried looking into how to solve this and have tried different ways using asyncio all of which have not worked.
If anyone is able to give me any pointers on how I could do this then it would be much appreciated.
import discord
import asyncio
import time
import schedule # pip install schedule
client = discord.Client()
#client.event
async def on_ready():
print("Connected!")
async def example(message):
await client.get_channel(CHANNEL ID).send(message)
client.run(SECRET KEY)
def scheduledEvent():
loop = asyncio.get_event_loop()
loop.run_until_complete(example("Test Message"))
loop.close()
schedule.every().minute.do(scheduledEvent)
while True:
schedule.run_pending()
time.sleep(1)
You can't run your blocking schedule code in the same thread as your asynchronous event loop (your current code won't even try to schedule tasks until the bot has already disconnected). Instead, you should use the built in tasks extension which allows you to schedule tasks.
import discord
from discord.ext import tasks, commands
CHANNEL_ID = 1234
TOKEN = 'abc'
client = discord.Client()
#client.event
async def on_ready():
print("Connected!")
#tasks.loop(minutes=1)
async def example():
await client.get_channel(CHANNEL_ID).send("Test Message")
#example.before_loop
async def before_example():
await client.wait_until_ready()
example.start()
clinet.run(TOKEN)