I am trying to connect to multiple channels concurrently and receive messages from a push API through the python websocket library.
Considering the following code below, how would you connect to multiple channels? this code was obtained and slightly modified from here: https://pypi.python.org/pypi/websocket-client
What confuses me is the second last line: ws.on_open = on_open. on_open is defined as a function above and takes 1 argument but no argument is passed when calling the function, I don't recall encountering this before in python code, so I'm unsure what is really going on in this line.
How can I modify this code so that I can pass a variable that contains a string to the function on_open so that I can specify the name of the Chanel that I want to subscribe to? My main goal is to be able to use the multiprocessing library to pass multiple channels to subscribe to concurrently.
Would I accomplish this by creating multiple ws objects or, one ws object and calling on_open multiple times with different channels as arguments?
import websocket
import thread
import time
import json
def on_message(ws, message):
print(message)
def on_error(ws, error):
print(error)
def on_close(ws):
print("### closed ###")
def on_open(ws):
def run(*args):
ws.send(json.dumps({'channel':'channel1'}))
while True:
time.sleep(1)
ws.close()
print("thread terminating...")
thread.start_new_thread(run, ())
if __name__ == "__main__":
websocket.enableTrace(True)
ws = websocket.WebSocketApp("wss://random.example.com",
on_message = on_message,
on_error = on_error,
on_close = on_close)
ws.on_open = on_open
ws.run_forever()
Use partial to pass in additional arguments
from functools import partial
def on_open(ws, channel_name):
"""
Notice the channel_name parameter
"""
# create a new function with the predefined variable
chan1 = partial(on_open, channel_name='channel 1')
# set the new function as the on_open callback
ws1.on_open = chan1
# do the same for the rest
chan2 = partial(on_open, channel_name='channel 2')
ws2.on_open = chan2
As a side note, consider using Tornado or Crossbar.io (aka autobahn). Those are proper asynchronous frameworks and make websocket development simpler as opposed to threads and multiprocessing.
Related
I am using a third party API that provides websocket functionality for continuous stream of messages.
I want to use variables that are defined out of the on_message() function. The structure of the code is as below:
soc = WebSocket()
Live_Data = {}
Prev_Data = {}
def on_message(ws, message):
if <check condition about Prev_Data>:
<Do something on Prev_Data and Live_Data>
def on_open(ws):
print("on open")
def on_error(ws, error):
print(error)
def on_close(ws):
print("Close")
# Assign the callbacks.
soc._on_open = on_open
soc._on_message = on_message
soc._on_error = on_error
soc._on_close = on_close
soc.connect()
But when I run this, it throws error that
Prev_data is referenced before it is assigned
I think this is because the on_message() method is asynchronous and the next on_message() method tries to access the Prev_Data before the first on_message() has finished writing it.
So what is the mechanism to synchronously access the Prev_Data and some other such variables here?
P.S: When I don't use the Prev_Data at all, the code runs fine.
I have a Python asyncio script that needs to run a long running task in a thread. During the operation of the thread, it needs to make network connections to another server. Is there any problem calling network/socket write functions in a thread as opposed to doing it in the main thread?
I know that in the Tiwsted library for example, one must always do network operations in the main thread. Are there any such limitations in asyncio? And if so, how does one get around this problem.
Here's my sample code:
import asyncio
import threading
#
# global servers dict keeps track of connected instances of each protocol
#
servers={}
class SomeOtherServer(asyncio.Protocol):
def __init__(self):
self.transport = None
def connection_made(self,transport):
self.transport=transport
servers["SomeOtherServer"] = self
def connection_lost(self):
self.transport=None
class MyServer(asyncio.Protocol):
def __init__(self):
self.transport = None
def connection_made(self,transport);
self.transport=transport
servers["MyServer"] = self
def connection_lost(self):
self.transport=None
def long_running_task(self,data):
# some long running operations here, then write data to other server
# other_server is also an instance of some sort of asyncio.Protocol
# is it ok to call this like this, even though this method is running in a thread?
other_server = servers["SomeOtherServer"]
other_server.transport.write(data)
def data_received(self,data):
task_thread = threading.Thread(target=self.long_running_task,args=[data])
task_thread.start()
async def main():
global loop
loop = asyncio.get_running_loop()
other_server_obj = await loop.create_server(lambda: SomeOtherServer(),"localhost",9001)
my_server_obj = await loop.create_server(lambda: MyServer(),"localhost",9002)
async with other_server_obj, my_server_obj:
while True:
await asyncio.sleep(3600)
asyncio.run(main())
Note that data_received will set up and call long_running_task in a thread, and long running_task makes a network connection to another server, and does so in the task thread, not the main thread. Is this ok or is there some other way this must be done?
I'm trying to wrap my head around the usage of websocket. Here's my code:
import websocket
def on_message(ws, message):
print(message)
def on_error(ws, error):
print(error)
def on_close(ws):
print('Websocket: closed')
def on_open(ws):
print('Websocket: open')
ws = websocket.WebSocketApp('ws://echo.websocket.org/',
on_message = on_message,
on_error = on_error,
on_close = on_close,
on_open = on_open)
ws.run_forever()
First, why would I put
ws.on_open = on_open
instead of passing it while defining ws? (As I did in the code)
Moreoever, how can I send a message? I could include
ws.send(json.dumps("Hello")) under
def on_open(ws), is there some other way?
Finally, I'm not able to close the connection given that ws.run_forever() never stops, how can I do it? I tried to include ws.close() under def on_open(ws) but then I don't receive any message.
What I'm trying to get is how I can transpose this:
ws = ws.create_connection('ws://echo.websocket.org/')
ws.send(json.dumps("Hello"))
result = json.loads(ws.recv())
print(result)
using WebSocketApp, that is having the messages pushed printed directly, without having to request the result.
I used asyncio for my non-stop server in python and implemented
connection_made , connection_lost , data_received
funtions in my ServerClientProtocol
I used this class first beacause of using multiple times repeatedly sending data to socket class socket
got closed and program exited
and second becuase I thought its async and have parallel answering multiple coming sockets in same time,
but it's not.
how should I use that in one async thread and parallel answering socket?
this is my code:
class ServerClientProtocol(asyncio.Protocol):
def connection_made(self,transport):
self.transport = transport
def connection_lost(self,exc):
pass
def data_received(self, data):
server.server(self,data)
def main(*args):
loop = get_event_loop()
coro = loop.create_server(ServerClientProtocol, '127.0.0.1', 50008)
srv = loop.run_until_complete(coro)
loop.run_forever()
if __name__ == '__main__':
main()
server.server() might be blocking the other connections. If this is a long-running call, try using asyncio.start_server (example here) instead, and call server.server() using await loop.run_in_executor(None, server.server, data)
I am wanting to run a program in Python that sends a message every second via web sockets to a Tornado server. I have been using the example on websocket-client;
This example does not work, because ws.run_forever() will stop the execution of the while loop.
Can somebody give me an example of how to correctly implement this as a threaded class which I can both call the send method of, but also receive messages?
import websocket
import thread
import time
def on_message(ws, message):
print message
def on_error(ws, error):
print error
def on_close(ws):
print "### closed ###"
def on_open(ws):
pass
if __name__ == "__main__":
websocket.enableTrace(True)
ws = websocket.WebSocketApp("ws://echo.websocket.org/", on_message = on_message, on_error = on_error, on_close = on_close)
ws.on_open = on_open
ws.run_forever()
while True:
#do other actions here... collect data etc.
for i in range(100):
time.sleep(1)
ws.send("Hello %d" % i)
time.sleep(1)
There's an example in their github page that does exactly that. It seems like you started out of that example and took the code that sends messages every second out of the on_open and pasted it after the run_forever call, that BTW runs until the socket is disconnected.
Maybe you are having issues with the basic concepts here. There's always going to be a thread dedicated to listening to the socket (in this case the main thread that enters a loop inside the run_forever waiting for messages). If you want to have some other thing going on you'll need another thread.
Below is a different version of their example code, where instead of using the main thread as the "socket listener", another thread is created and the run_forever runs there. I see it as a bit more complicated since you have to write code to assure the socket has connected while you could use the on_open callback, but maybe it will help you understand.
import websocket
import threading
from time import sleep
def on_message(ws, message):
print message
def on_close(ws):
print "### closed ###"
if __name__ == "__main__":
websocket.enableTrace(True)
ws = websocket.WebSocketApp("ws://echo.websocket.org/", on_message = on_message, on_close = on_close)
wst = threading.Thread(target=ws.run_forever)
wst.daemon = True
wst.start()
conn_timeout = 5
while not ws.sock.connected and conn_timeout:
sleep(1)
conn_timeout -= 1
msg_counter = 0
while ws.sock.connected:
ws.send('Hello world %d'%msg_counter)
sleep(1)
msg_counter += 1
In 2023, they have an updated example for dispatching multiple WebSocketApps using an asynchronous dispatcher like rel.
import websocket, rel
addr = "wss://api.gemini.com/v1/marketdata/%s"
for symbol in ["BTCUSD", "ETHUSD", "ETHBTC"]:
ws = websocket.WebSocketApp(addr % (symbol,), on_message=lambda w, m : print(m))
ws.run_forever(dispatcher=rel, reconnect=3)
rel.signal(2, rel.abort) # Keyboard Interrupt
rel.dispatch()
Hope it helps!