Binance API & python-binance offers async functionality for non-blocking execution as per discussed in Async basics for Binance.
I am using BinanceSocketManager listening (async non-blocking) to live data via websocket.
In scenarios like network intermittent connection lost, I wish to add an auto-reconnect feature to my project. But I can't seems to find any info with BinanceSocketManager. I was only able to find a guide which uses ThreadedWebsocketManager, but it was not an async implementation.
Does anyone know how to implement a Binance websocket disconnect detection and auto-reconnect mechanism?
Here is some code of what I have so far:
import asyncio
from binance import AsyncClient, BinanceSocketManager
async def main():
client = await AsyncClient.create()
await kline_listener(client)
async def kline_listener(client):
bm = BinanceSocketManager(client)
async with bm.kline_socket(symbol='BTCUSDT') as stream:
while True:
res = await stream.recv()
print(res)
# a way detect websocket error/disconnect, callback 'disconnect_callback'
async def disconnect_callback():
await client.close_connection()
await main() # restart client and kline socket
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
In case someone else is looking at this, for this, you should be looking at the BinanceAPIException. Code could look something like this then:
from binance import AsyncClient, BinanceSocketManager
from binance.exceptions import BinanceAPIException
async def main():
client = await AsyncClient.create()
bm = BinanceSocketManager(client, user_timeout=60)
# start any sockets here, i.e a trade socket
kline_candles = bm.kline_socket('BNBUSDT', interval=client.KLINE_INTERVAL_1MINUTE)
# start receiving messages
try:
status = await client.get_system_status()
print(status['msg'])
async with kline_candles as stream:
for _ in range(5):
res = await stream.recv() # create/await response
await process_message(msg=res, client=client) # process message
except BinanceAPIException as e:
print(e)
await disconnect_callback(client=client)
async def disconnect_callback(client):
await client.close_connection() # close connection
time.sleep(60) # wait a minute before restarting
await main() # restart client and kline socket
async def process_message(msg, client):
if msg['e'] == 'error':
await disconnect_callback(client=client)
print('ERROR OCCURED')
else:
candle = msg['k'] # get only the candle info within the general dict
start_time = datetime.utcfromtimestamp(candle['t']/1000).strftime('%Y-%m-%d %H:%M:%S')
close_time = datetime.utcfromtimestamp(candle['T']/1000).strftime('%Y-%m-%d %H:%M:%S')
print(f'__ start: {start_time}, close: {close_time}')
print(msg)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
The disconnect has not been tested yet, but I assume this will work. If anyone has any additional notes, just let me know.
I have tested above code and it proves rather stable.
Here are some improvements I have made.
I'm not sure what happens if your internet connection is completely gone when this line is executed:
client = await AsyncClient.create()
This could probably be solved like this (I'm open for better ideas):
while True:
try:
client = await AsyncClient.create()
except Exception as error_msg:
print(f"error: {error_msg}")
# should we add a sleep here?
# time.sleep(3)
else:
print("finally got through the loop")
break
Surrounding this with a try/except is a good idea:
bm = BinanceSocketManager(client, user_timeout=60)
The call to stream.recv() should be extended with asyncio.wait_for() to cover the situation, when there is no data coming in for a longer period of time. It usually means there's something wrong.
async with kline_candles as stream:
for _ in range(5):
try:
res = await asyncio.wait_for(stream.recv(), timeout=60) # create/await response
await process_message(msg=res, client=client) # process message
except (asyncio.TimeoutError, websockets.exceptions.ConnectionClosed, asyncio.exceptions.CancelledError, asyncio.exceptions.TimeoutError) as error_msg_1:
print(f"Error! in main loop 1:\n{error_msg_1}")
await disconnect_callback(client=client)
Related
I need to listen to User Data Stream, whenever there's an Order Event - order execution, cancelation, and so on - I'd like to be able to listen to those events and create notifications.
So I got my "listenKey" and I'm not sure if it was done the right way but I executed this code and it gave me something like listenKey.
Code to get listenKey:
def get_listen_key_by_REST(binance_api_key):
url = 'https://api.binance.com/api/v1/userDataStream'
response = requests.post(url, headers={'X-MBX-APIKEY': binance_api_key})
json = response.json()
return json['listenKey']
print(get_listen_key_by_REST(API_KEY))
And the code to listen to User Data Stream - which doesn't work, I get no json response.
socket = f"wss://fstream-auth.binance.com/ws/btcusdt#markPrice?listenKey=<listenKeyhere>"
def on_message(ws, message):
json_message = json.loads(message)
print(json_message)
def on_close(ws):
print(f"Connection Closed")
# restart()
def on_error(ws, error):
print(f"Error")
print(error)
ws = websocket.WebSocketApp(socket, on_message=on_message, on_close=on_close, on_error=on_error)
I have read the docs to no avail. I'd appreciate it if someone could point me in the right direction.
You can create a basic async user socket connection from the docs here along with other useful info for the Binance API. Here is a simple example:
import asyncio
from binance import AsyncClient, BinanceSocketManager
async def main():
client = await AsyncClient.create(api_key, api_secret, tld='us')
bm = BinanceSocketManager(client)
# start any sockets here, i.e a trade socket
ts = bm.user_socket()
# then start receiving messages
async with ts as tscm:
while True:
res = await tscm.recv()
print(res)
await client.close_connection()
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
I just figured this out myself and I was able to get mine to work so I'll try my best to guide you. I believe you're just missing this line of code after you create your WebSocket object:
ws.run_forever()
Some other reasons it might not be working; If you want to detect orders on your futures account then you need to use the futures endpoint. I think the one your using is for spot trading (Not sure).
url = 'https://fapi.binance.com'
and just in case it's not clear to you. You must replace:
<listenkeyhere>
in the socket url with your listen key, angle brackets, and all.
I'm trying to run a Telegram client that is capable of sending and receiving messages in parallel, however, I want to use the "event" option, because I think it's more efficient than sending requests for the history of the chat all the time.
The problem is that the "receber" (receiver) function, seem to only star after the "enviar" (sending) function finishs, that is not what I want.
Is there any way to solve this? Anything in the documentation (note: yes, I have checked it, but I couldn't find anything about using events and non-event functions at the same time) will be useful.
from variables import api_id, api_hash
from telethon import TelegramClient, events, utils
import asyncio
import logging
logging.basicConfig(format='[%(levelname) 5s/%(asctime)s] %(name)s: %(message)s',
level=logging.WARNING)
client = TelegramClient('anon', api_id, api_hash);
lock = asyncio.Lock();
async def info_me():
me = await client.get_me();
return me;
async def enviar():
while True:
try:
await lock.acquire();
try:
msg_txt = open("msg.txt", "r");
updates = msg_txt.read();
msg_txt.close();
finally:
lock.release();
print("==================");
print(updates);
print("==================");
msg = input("Enviar: ");
if msg != "//":
msg = msg.split(" | ");
await client.send_message(msg[0], msg[1]);
else:
await asyncio.sleep(0.1);
except(KeyboardInterrupt):
print("Adeus!");
break;
except Exception as e:
print("\n======ERRO======\n");
print(e);
break;
async def receber(event):
try:
sender = await event.get_sender();
name = utils.get_display_name(sender);
message = name + "::::::" + event.text + "\n"; #<--proteger contra input do usuário
await lock.acquire();
try:
file = open('msg.txt', '+a');
file.write(message);
file.close();
finally:
lock.release()
except(KeyboardInterrupt):
print("Adeus!");
except("Cannot send requests while disconnected"):
print("Adeus!");
except Exception as e:
print("\n======ERRO======\n");
print(e);
client.add_event_handler(receber, events.NewMessage)
async def main():
me = await info_me(); #pegando informação sobre a conexão, caso eu precise
enviar_var = asyncio.create_task(enviar());
await enviar_var;
with client:
client.loop.run_until_complete(main());
So, I have found this example on the telethons github. There, you can find this function:
# Create a global variable to hold the loop we will be using
loop = asyncio.get_event_loop()
(...other code...)
async def async_input(prompt):
"""
This is enough to make run in parallel. You can look the actual code to understand better!
NOTE: If you only copy and paste this code, it will run, but there will be a error at the end. Any further comment is welcome!
Python's "input()" is blocking, which means the event loop we set
above can't be running while we're blocking there. This method will
let the loop run while we wait for input.
"""
print(prompt, end='', flush=True)
return (await loop.run_in_executor(None, sys.stdin.readline)).rstrip()
You should look to the documentation to understand better.
Any further comment is welcome!
I am writing a bot and I need to implement the following functionality: the bot once every 10 minutes(for example) parse a certain URL and if there were changes from the previous call, writes to the chat.
Since the bot is also engaged in other things, I decided to loop the parsing in the function with sleep at the end. If there are changes, I try to send a message to the chat, but then a problem happens.
Since a successful combination of circumstances does not arise from an event in the chat, I can't pull the "entity" from the "event" for the "send_message" function. therefore, we have to get through the "get_entity" function and links to the chat as a parameter, but for some reason this does not work from another stream. below is a simplified code:
import threading, queue
from time import sleep
import asyncio
from telethon.sync import TelegramClient, events
import config as cfg
bot = TelegramClient('Bot', cfg.api_id, cfg.api_hash)
#bot.on(events.NewMessage(pattern=r'^(?i)(idchat){1}$'))
async def echoidchat(event):
channelaa = await bot.get_entity('https://t.me/elvistest')
await bot.send_message(channelaa, 'ответ')
def parseurls():
for x in range(10):
q.put(x)
pass
async def pre_sendmsg():
while True:
try:
msg = q.get_nowait()
except Exception as e:
await asyncio.sleep(1.0)
else:
await sendmsg(msg)
q.task_done()
async def sendmsg(msg):
channel = await bot.get_entity('https://t.me/elvistest')
await bot.send_message(channel, f'ответ из другого потока {msg}')
if __name__ == '__main__':
q = queue.Queue()
parseurls()
bot.start(bot_token=cfg.bot_token)
threading.Thread(target=asyncio.run, daemon=True, args=(pre_sendmsg(),)).start()
bot.run_until_disconnected()
The thing is that on the line " boot.get_entity" nothing happens. The script execution is lost somewhere and does not go further, that is, the next line with "bot. send_message" is simply not executed. however, "def echoidchat" is working at this time.
Well done!
This is work like I want.
import random
import threading, queue
from time import sleep
import asyncio
from telethon import TelegramClient, events
import config as cfg
bot = TelegramClient('Bot', cfg.api_id, cfg.api_hash)
#bot.on(events.NewMessage(pattern=r'^(?i)(idchat){1}$'))
async def echoidchat(event):
await bot.send_message(event.chat, 'ответ')
async def parseurls():
while True:
ts = abs(int(random.random()*10))
print(f'parseurls({ts})')
await sendmsg(ts)
await asyncio.sleep(ts)
async def sendmsg(msg):
print(f'sendmsg({msg}) - start')
channel = await bot.get_entity('https://t.me/elvistest')
await bot.send_message(channel, f'ответ из другого потока {msg}')
print(f'sendmsg({msg}) - done')
def main():
bot.start(bot_token=cfg.bot_token)
loop = asyncio.get_event_loop()
tasks = [
loop.create_task(parseurls()),
loop.create_task(bot.run_until_disconnected()),
]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
if __name__ == '__main__':
main()
I am using websocket library in python for one of my projects. It works for me but I am curious to know how it works. I could not find this in the documentation. Specifically, for the example given in the docs,
#!/usr/bin/env python
# WS server example that synchronizes state across clients
import asyncio
import json
import logging
import websockets
logging.basicConfig()
STATE = {"value": 0}
USERS = set()
def state_event():
return json.dumps({"type": "state", **STATE})
def users_event():
return json.dumps({"type": "users", "count": len(USERS)})
async def notify_state():
if USERS: # asyncio.wait doesn't accept an empty list
message = state_event()
await asyncio.wait([user.send(message) for user in USERS])
async def notify_users():
if USERS: # asyncio.wait doesn't accept an empty list
message = users_event()
await asyncio.wait([user.send(message) for user in USERS])
async def register(websocket):
USERS.add(websocket)
await notify_users()
async def unregister(websocket):
USERS.remove(websocket)
await notify_users()
async def counter(websocket, path):
# register(websocket) sends user_event() to websocket
await register(websocket)
try:
await websocket.send(state_event())
async for message in websocket:
data = json.loads(message)
if data["action"] == "minus":
STATE["value"] -= 1
await notify_state()
elif data["action"] == "plus":
STATE["value"] += 1
await notify_state()
else:
logging.error("unsupported event: {}", data)
finally:
await unregister(websocket)
start_server = websockets.serve(counter, "localhost", 6789)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Whenever, I disconnect, the event unregister() gets fired. How does websocket come to know that I have disconnected?
My guess is the line async for message in websocket: has to do something with it but I do not know the details. Any insight on it will be appreciated.
With the websockets module, disconnections are handled as exceptions. For example, if you wanted to handle different kinds of disconnections, you could do something like this:
try:
await websocket.send(state_event())
async for message in websocket:
# Your code...
except websockets.exceptions.ConnectionClosedOK:
print("Client disconnected OK")
except websockets.exceptions.ConnectionClosedError:
print("Client disconnected unexpectedly")
finally:
await unregister(websocket)
More information can be found in the documentation.
I tried out the example below (from this page):
nc = NATS()
await nc.connect(servers=["nats://demo.nats.io:4222"])
future = asyncio.Future()
async def cb(msg):
nonlocal future
future.set_result(msg)
await nc.subscribe("updates", cb=cb)
await nc.publish("updates", b'All is Well')
await nc.flush()
# Wait for message to come in
msg = await asyncio.wait_for(future, 1)
But this only seems useful for receiving one message. How would I subscribe and keep receiving messages?
I've also seen the package example, but it seems to just play both sides of the conversation, then quit.
You can find a long running service example as well: https://github.com/nats-io/nats.py/blob/master/examples/service.py
import asyncio
from nats.aio.client import Client as NATS
async def run(loop):
nc = NATS()
async def disconnected_cb():
print("Got disconnected...")
async def reconnected_cb():
print("Got reconnected...")
await nc.connect("127.0.0.1",
reconnected_cb=reconnected_cb,
disconnected_cb=disconnected_cb,
max_reconnect_attempts=-1,
loop=loop)
async def help_request(msg):
subject = msg.subject
reply = msg.reply
data = msg.data.decode()
print("Received a message on '{subject} {reply}': {data}".format(
subject=subject, reply=reply, data=data))
await nc.publish(reply, b'I can help')
# Use queue named 'workers' for distributing requests
# among subscribers.
await nc.subscribe("help", "workers", help_request)
print("Listening for requests on 'help' subject...")
for i in range(1, 1000000):
await asyncio.sleep(1)
try:
response = await nc.request("help", b'hi')
print(response)
except Exception as e:
print("Error:", e)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
loop.run_forever()
loop.close()
Don't know much about python, but looks like you are just waiting for one message and program ends. You should look at the subscriber example here. As you can see, there is a loop to wait forever or for the SIGTERM signal.