I am looking at using websockets in my client (browser) - server (Python) application and I have a hard time to understand how to reuse a websocket channel across threads.
My application will have a few threads, each independently generating some data which I would like to push to the browser. The Getting Started examples make it clear that once a browser connects to the server, a specific function is ran (I copy the example from the linked docs):
import asyncio
import datetime
import random
import websockets
async def time(websocket, path):
while True:
now = datetime.datetime.utcnow().isoformat() + 'Z'
await websocket.send(now)
await asyncio.sleep(random.random() * 3)
start_server = websockets.serve(time, '127.0.0.1', 5678)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Now let's say I have a function (worker)
def thread1(an_existing_webocket):
while True:
an_existing_webocket.send("1")
print("1")
time.sleep(1)
which I would like to start via threading.Thread(target=thread1, args=(an_initialized_websocket,)).start().
How do these two worlds (websockets using coroutines and threading) coexist? And more specifically: what is the websocket I can pass to the workers so that they can use it to send messages upstream?
In other words, how should I create the an_initialized_websocket above?
Related
I am trying to implement WebSocket connection to a server (Python app <=> Django app)
Whole system runs in big Asyncio loop with many tasks. Code snippet is just very small testing part.
I am able to send any data to a server at any moment and many of them will be type request something and wait for response. But I would like to have some "always running" handler for all incoming messages. (When something in Django database will change I want to send changes to python app).
How can Include always running receiver/ or add callback to websocket? I am not able to find any solution for this.
My code snippet:
import asyncio, json, websockets, logging
class UpdateConnection:
async def connect(self,botName):
self.sock = await websockets.connect('ws://localhost:8000/updates/bot/'+botName)
async def send(self,data):
try:
await self.sock.send(json.dumps(data))
except:
logging.info("Websocket connection lost!")
# Find a way how to reconenct... or make socket reconnect automatically
if __name__ == '__main__':
async def DebugLoop(socketCon):
await socketCon.connect("dev")
print("Running..")
while True:
data = {"type": "debug"}
await socketCon.send(data)
await asyncio.sleep(1)
uSocket = UpdateConnection()
loop = asyncio.get_event_loop()
loop.create_task(DebugLoop(uSocket))
loop.run_forever()
My debug server after connection will start sending random messages to the client in random intervals and I would like to somehow handle them in async way.
Thanks for any help :)
You don't have to do it so complicated. First of all I suggest you use the context patterns offered by websockets module.
From the documentation:
connect() can be used as an infinite asynchronous iterator to reconnect automatically on errors:
async for websocket in websockets.connect(...):
try:
...
except websockets.ConnectionClosed:
continue
Additionally, you simply keep the connection alive by awaiting incoming messages:
my_websocket = None
async for websocket in websockets.connect('ws://localhost:8000/updates/bot/' + botName):
try:
my_websocket = websocket
async for message in websocket:
pass # here you could also process incoming messages
except websockets.ConnectionClosed:
my_websocket = None
continue
As you can see we have a nested loop here:
The outer loop constantly reconnects to the server
The inner loop processes one incoming message at a time
If you are connected, and no messages are coming in from the server, this will just sleep.
The other thing that happens here is that my_websocket is set to the active connection, and unset again when the connection is lost.
In other parts of your script you can use my_websocket to send data. Note that you will need to check if it is currently set wherever you use it:
async def send(data):
if my_websocket:
await my_websocket.send(json.dumps(data))
This is just an illustration, you can also keep the websocket object as an object member, or pass it to another component through a setter function, etc.
I'm new to websockets. I've been using the examples on the Getting Started page of the websockets docs, mainly the Synchronization Example.
In this use case, I have a sqlite3 database on localhost. I edit that database from a python GUI program on localhost which just imports the database code layer directly. The client then tells the websocket server to send out some extracted data to all clients.
(Eventually this will be on a LAN, with the server machine running a Flask API.)
This is working, with the code below, but it's not clear if I'm doing it correctly. Basically I want to send websockets messages when certain database activity takes place, and I'm confused about how to do a 'simple' non-async send, when invoked from code, ultimately in response to a GUI interaction, as opposed to doing a send in response to an incoming websocket message. In pseudo-code:
def send(ws,msg):
ws.send(msg)
send(ws,'OK!')
The way I'm accomplishing that is wrapping the async def that does the sending in a non-async 'vanilla' def.
The websocket server code:
# modified from https://websockets.readthedocs.io/en/stable/intro.html#synchronization-example
import asyncio
import websockets
USERS = set()
async def register(websocket):
print("register: "+str(websocket))
USERS.add(websocket)
async def unregister(websocket):
print("unregister: "+str(websocket))
USERS.remove(websocket)
# each new connection calls trackerHandler, resulting in a new USERS entry
async def trackerHandler(websocket, path):
await register(websocket)
try:
async for message in websocket:
await asyncio.wait([user.send(message) for user in USERS])
finally:
await unregister(websocket)
start_server = websockets.serve(trackerHandler, "localhost", 8765)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
in the database interface code (on localhost, this file is just imported directly to the GUI app; but on the LAN server, this is the file specified in the WSGI call in Flask):
import asyncio
import websockets
# uri = "ws://localhost:8765"
# wrap the asynchronous send function inside a synchronous function
def wsSend(uri,msg):
async def send():
async with websockets.connect(uri) as websocket:
# await websocket.send(str.encode(str(msg)))
await websocket.send(json.dumps({"msg":msg}))
# print(f"> {msg}")
# greeting = await websocket.recv()
# print(f"< {greeting}")
asyncio.get_event_loop().run_until_complete(send())
...
...
def tdbPushTables(uri,teamsViewList=None,assignmentsViewList=None,teamsCountText="---",assignmentsCountText="---"):
# uri = "ws://localhost:8765"
if not teamsViewList:
teamsViewList=tdbGetTeamsView()
if not assignmentsViewList:
assignmentsViewList=tdbGetAssignmentsView()
if uri=='pusher':
pusher_client.trigger('my-channel','teamsViewUpdate',teamsViewList)
pusher_client.trigger('my-channel','assignmentsViewUpdate',teamsViewList)
else:
wsSend(uri,json.dumps({
"teamsView":teamsViewList,
"assignmentsView":assignmentsViewList,
"teamsCount":teamsCountText,
"assignmentsCount":assignmentsCountText}))
it's actually the client that initiates the call to tdbPushTables:
def buildLists(self):
self.teamsList=tdbGetTeamsView()
self.assignmentsList=tdbGetAssignmentsView()
self.updateCounts()
tdbPushTables('ws://localhost:8765',self.teamsList,self.assignmentsList,self.teamsCountText,self.assignmentsCountText)
Feels spooky. Is it spooky or is this actually the right way to do it? Should I be using the websockets module for the server, but a different module to do the 'simple'/synchronous sending of the websocket message to the server?
Two known side effects of this solution: 1) it opens and closes the websocket connection on every call - probably not really a problem...?, and 2) it results in non-fatal handled messages like this in the server transcript:
register: <websockets.server.WebSocketServerProtocol object at 0x041C46F0>
Task exception was never retrieved
future: <Task finished coro=<WebSocketCommonProtocol.send() done, defined at C:\Users\caver\AppData\Roaming\Python\Python37\site-packages\websockets\protocol.py:521> exception=ConnectionClosedOK('code = 1000 (OK), no reason')>
Traceback (most recent call last):
File "C:\Users\caver\AppData\Roaming\Python\Python37\site-packages\websockets\protocol.py", line 555, in send
await self.ensure_open()
File "C:\Users\caver\AppData\Roaming\Python\Python37\site-packages\websockets\protocol.py", line 812, in ensure_open
raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedOK: code = 1000 (OK), no reason
unregister: <websockets.server.WebSocketServerProtocol object at 0x041C46F0>
EDIT: looks like the websocket (singular) module has a synchronous interface, and the websockets (plural) docs explain that if you want to go synchronous you should use a different module; so, this works:
(instead of importing asyncio and websockets)
from websocket import create_connection
def wsSend(uri,msg):
ws=create_connection(uri)
ws.send(json.dumps({"msg":msg}))
ws.close()
It does still result in the same handled traceback showing up in the server transcript each time wsSend is called; there's probably a way to silence that traceback output, but, regardless, it still doesn't seem to affect anything.
Your code feels spooky, because you are mixing async code with synchronous code.
Based on personal experience, the code is simpler to follow if you keep most of the code asynchronous.
The structure will become something like:
import asyncio
import websockets
async def main():
# Create websocket connection
async with websockets.connect(uri) as websocket:
await your_function_that_does_some_processing(websocket)
asyncio.get_event_loop().run_until_complete(main())
Have in mind that big sections of blocking code can generate trouble.
I am trying to create a code using rtmidi and websockets, which listenes for midi input and then sends data through websocket to all connected users. The data is beeing sent inside if statement, when some condition is met. In this test, the condition is simply random number comparison. Client can conenct to websocket server, however never receives any dat, even though server confirms that it was sent.
When I added asyncio.sleep(0) just before sending data, everything started working. Why is that? Should I use sleep, or is there a better solution?
I removed everything rtmidi related from example to make it easier, the problem is still the same.
Server code:
import asyncio
import websockets
import random
async def key(websocket, path):
print("port connected ", path)
while True:
rnd=random.randrange(0,300000)
if rnd==0:
#await asyncio.sleep(0) #when not commented everything works
print("sent")
await websocket.send("hello")
start_server = websockets.serve(key, "127.0.0.1", 8765)
print("done")
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Client code:
import asyncio
import websockets
async def gdata():
uri = "ws://127.0.0.1:8765"
async with websockets.connect(uri) as websocket:
while True:
data = await websocket.recv()
print(data)
asyncio.get_event_loop().run_until_complete(gdata())
I have a Python3 program that runs a "while True"-loop until stopped, which occasionally saves data to a MySQL database. I am creating an administrative website, separate from the Python program, where I will be able to observe this data.
I now want to be able to be notified, on the website, when changes have been made to the database. My thought was to set up a websocket connection, so that the Python program can send a message through the socket to all connected clients, i.e. all open browsers, if there has been any changes to the database table.
I have done something similar before, but in that case I had to wait for a websocket connection before the "while True"-loop would start. In the new scenario I want to be able to have multiple website clients at once, and let them connect at any time, as well as disconnect without interrupting the Python programs loop.
This is a simplified version of my previous code, which I now want to update to be able to run both with & without websocket clients.
import asyncio
import websockets
socket_server = websockets.serve(run, "127.0.0.1", 5055)
asyncio.get_event_loop().run_until_complete(socket_server)
console_log("Waiting for socket connection...")
asyncio.get_event_loop().run_forever()
async def run(ws):
while True:
db_has_updated = do_stuff()
if db_has_updated:
await ws.send(data)
I just can't seem to be able to come up with the right search terms to find a solution, so I'm asking here instead.
I figured it out, finally! Here is my solution with a websocket server running in a separate thread from the other logic. I'm probably changing some things to make it neater, but this does everything I need. Feel free to ask any questions.
Be aware that this blocks when messaging all the connected clients. That is the way I needed it to work, but you could always thread/subprocess the logic/data-gen part of the program if you want it to run completely asynchronously.
#!/usr/bin/env python3
import asyncio
import websockets
import threading
import time
import random
def gen_data():
print("Generating data...")
time.sleep(3)
data = random.randint(1, 10)
return data
async def send(client, data):
await client.send(data)
async def handler(client, path):
# Register.
print("Websocket Client Connected.", client)
clients.append(client)
while True:
try:
print("ping", client)
pong_waiter = await client.ping()
await pong_waiter
print("pong", client)
time.sleep(3)
except Exception as e:
clients.remove(client)
print("Websocket Client Disconnected", client)
break
clients = []
start_server = websockets.serve(handler, "localhost", 5555)
asyncio.get_event_loop().run_until_complete(start_server)
threading.Thread(target = asyncio.get_event_loop().run_forever).start()
print("Socket Server Running. Starting main loop.")
while True:
data = str(gen_data())
message_clients = clients.copy()
for client in message_clients:
print("Sending", data, "to", client)
try:
asyncio.run(send(client, data))
except:
# Clients might have disconnected during the messaging process,
# just ignore that, they will have been removed already.
pass
I'm trying to create a unidirectional websocket connection between the client and server (python). The libraries I've currently been prototyping with are websockets and simplesocketserver. I've got a hello world example getting from the server to the client, but I need to be able to send data from the backend to the client unprompted from the client. All of the websockets examples seem to show the server listening to the client and then responding.
So far I've tried:
Using websockets 8.0, sending data from server to client unprompted, this works but only with hard-coded strings, I don't understand how to send real data on demand unprompted from the client
Using simplesocketserver in the same exact manner
Started investigating server sent events - is this more appropriate?
Example from the websockets documentation:
import asyncio
import websockets
async def hello(websocket, path):
name = await websocket.recv()
print(f"< {name}")
greeting = f"Hello {name}!"
await websocket.send(greeting)
print(f"> {greeting}")
start_server = websockets.serve(hello, "localhost", 8765)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Note: the need for unidirectional communication is due to existing architecture.
Any guidance or resources on this would be great, I hope I'm overlooking something easy. Thank you!
I came across a similar issue. After some trial and error I found a solution that works for me, maybe so it does for you. I've tested this on Linux and with Firefox and mobile Safari (parallel).
The InputThread waits for input (words) on the command line and writes them into a list, that is sent to all connected clients each time a new string is appended or a client is connected.
Code snippet
import asyncio
import websockets
import json
from threading import Thread
words = []
clients = []
async def register_client(websocket, path):
# register new client in list and keep connection open
clients.append(websocket)
await send_to_all_clients()
while True:
await asyncio.sleep(10)
async def send_to_all_clients():
global words
for i, ws in list(enumerate(clients))[::-1]:
try:
await ws.send(json.dumps({"words": words}))
except websockets.exceptions.ConnectionClosedError:
# remove if connection closed
del clients[i]
class InputThread(Thread):
def run(self):
global words
async def sending_loop():
while True:
i = input()
if not i.strip():
continue
words.append({"options": i.strip().slit()})
await send_to_all_clients()
asyncio.run(sending_loop())
InputThread().start()
asyncio.get_event_loop().run_until_complete(
websockets.serve(register_client, "localhost", 8765))
asyncio.get_event_loop().run_forever()