http://websockets.readthedocs.io/en/stable/intro.html#consumer contains the following example:
async def consumer_handler(websocket, path):
while True:
message = await websocket.recv()
await consumer(message)
and http://websockets.readthedocs.io/en/stable/intro.html#producer
async def producer_handler(websocket, path):
while True:
message = await producer()
await websocket.send(message)
But there is no example for consumer() and producer() implementation or any explanation. Can somebody provide any simple example for that?
In the first example, consumer_handler listens for the messages from a websocket connection. It then passes the messages to a consumer. In its simplest form, a consumer can look like this:
async def consumer(message):
# do something with the message
In the second example, producer_handler receives a message from a producer and sends it to the websocket connection. A producer can look like this:
async def producer():
message = "Hello, World!"
await asyncio.sleep(5) # sleep for 5 seconds before returning message
return message
Related
I want to send two different messages to a websocket server but with different time intervalls.
For example:
The first message should be send every 2 seconds.
The second message should send every 5 seconds.
async def send_first_message(websocket):
while True:
await websocket.send("FIRST MESSAGE")
response = await websocket.recv()
await asyncio.sleep(2)
async def send_second_message():
while True:
async with websockets.connect(f"ws://{IP}:{PORT}") as websocket:
asyncio.create_task(send_first_message(websocket))
while True:
await websocket.send("SECOND MESSAGE")
response = await websocket.recv()
await asyncio.sleep(5)
asyncio.run(send_second_message())
If I run the code like this I get:
"RuntimeError: cannot call recv while another coroutine is already waiting for the next message"
If I comment out one of the "await websocket.recv()" it works fine for a few seconds and then it throws:
"RuntimeError no close frame received or sent"
There's a bit of a disconnect between what you are trying to do in the tasks (synchronous request-response interaction) and what the protocol and the library expects you to do (asynchronous messages).
When writing asynchronous code, you need to look at what the library/protocol/service expects to be an atomic operation that can happen asynchronously to everything else, and what you want to be a synchronous series of operations. Then you need to find the primitive in the library that will support that. In the case of websockets, the atomic operation is a message being sent in either direction. So you can't expect websockets to synchronize flow over two messages.
Or to put it another way, you are expecting synchronous responses for each send message, but websockets are not designed to handle interleaved synchronous requests. You've sent a message to the websocket server, and you want to get a response to that message. But you've also sent another message on the same websocket and want a response to that too. Your client websocket library can't differentiate between a message intended for the first request and a message intended for the second request (because from the websocket protocol layer, that is a meaningless concept - so the library enforces this by limiting the recv operations on a websocket that can be blocking to one).
So ...
Option 1 - multiple tasks on separate sockets
From the fact the library limits a websocket to one blocking recv, a primitive in the protocol that meets the requirement is the websocket itself. If these are separate requests that you need separate blocking responses to (so only continue in the requesting task once those responses are available) then you could have separate websocket connections and block for the response in each.
client1.py
async def send_first_message():
async with websockets.connect(f"ws://{IP}:{PORT}") as websocket:
while True:
await websocket.send("FIRST MESSAGE")
response = await websocket.recv()
print(response)
await asyncio.sleep(2)
async def send_second_message():
async with websockets.connect(f"ws://{IP}:{PORT}") as websocket:
while True:
await websocket.send("SECOND MESSAGE")
response = await websocket.recv()
print(response)
await asyncio.sleep(5)
async def main():
asyncio.create_task(send_first_message())
asyncio.create_task(send_second_message())
await asyncio.Future()
asyncio.run(main())
Option 1 is however not really the websocket or asynchronous way.
Option 2 - embrace the asynchronous
To do this on a single websocket, you will need to receive the response asynchronous to both sending tasks.
If you don't actually care that the send_* functions get the response, you can do this easily...
client2.py
async def send_first_message(websocket):
while True:
await websocket.send("FIRST MESSAGE")
await asyncio.sleep(2)
async def send_second_message(websocket):
while True:
await websocket.send("SECOND MESSAGE")
await asyncio.sleep(5)
async def receive_message(websocket):
while True:
response = await websocket.recv()
print(response)
async def main():
async with websockets.connect(f"ws://{IP}:{PORT}") as websocket:
asyncio.create_task(send_first_message(websocket))
asyncio.create_task(send_second_message(websocket))
asyncio.create_task(receive_message(websocket))
await asyncio.Future()
asyncio.run(main())
Option 3
But what if you want to line up responses to requests and keep on a single websocket? You need some way of knowing which request any particular response is for. Most web services that need this sort of interaction will have you send an ID in the message to the server, and it will respond once a response is ready using the ID as a reference.
There's also a way of getting your message tasks to block and wait for the response with the right ID by queuing up the responses and checking them periodically.
client3.py
unhandled_responses = {}
async def send_first_message(websocket):
while True:
req_id = random.randint(0,65535)
message = json.dumps({'id': req_id, 'message': 'FIRST MESSAGE'})
await websocket.send(message)
response = await block_for_response(req_id)
print(response)
await asyncio.sleep(2)
async def send_second_message(websocket):
while True:
req_id = random.randint(0,65535)
message = json.dumps({'id': req_id, 'message': 'SECOND MESSAGE'})
await websocket.send(message)
response = await block_for_response(req_id)
print(response)
await asyncio.sleep(5)
async def block_for_response(id):
while True:
response = unhandled_responses.pop(id, None)
if response:
return response
await asyncio.sleep(0.1)
async def receive_message(websocket):
while True:
response = json.loads(await websocket.recv())
unhandled_responses[response['id']] = response
async def main():
async with websockets.connect(f"ws://{IP}:{PORT}") as websocket:
asyncio.create_task(send_first_message(websocket))
asyncio.create_task(send_second_message(websocket))
asyncio.create_task(receive_message(websocket))
await asyncio.Future()
asyncio.run(main())
For completeness, the server code the clients were talking to in my tests.
server.py
import asyncio
import websockets
async def server_endpoint(websocket):
try:
while True:
recv_msg = await websocket.recv()
response = recv_msg
await websocket.send(response)
except Exception as ex:
print(str(ex))
async def main():
async with websockets.serve(server_endpoint, "localhost", 8765):
await asyncio.Future() # run forever
if __name__ == "__main__":
asyncio.run(main())
Quart is a Python web framework which re-implements the Flask API on top of the asyncio coroutine system of Python. In my particular case, I have a Quart websocket endpoint which is supposed to have not just one source of incoming events, but two possible sources of events which are supposed to continue the asynchronous loop.
An example with one event source:
from quart import Quart, websocket
app = Quart(__name__)
#app.websocket("/echo")
def echo():
while True:
incoming_message = await websocket.receive()
await websocket.send(incoming_message)
Taken from https://pgjones.gitlab.io/quart/
This example has one source: the incoming message stream. But what is the correct pattern if I had two possible sources, one being await websocket.receive() and another one being something along the lines of await system.get_next_external_notification() .
If either of them arrives, I'd like to send a websocket message.
I think I'll have to use asyncio.wait(..., return_when=FIRST_COMPLETED), but how do I make sure that I miss no data (i.e. for the race condition that websocket.receive() and system.get_next_external_notification() both finish almost exactly at the same time) ? What's the correct pattern in this case?
An idea you could use is a Queue to join the events together from different sources, then have an async function listening in the background to that queue for requests. Something like this might get you started:
import asyncio
from quart import Quart, websocket
app = Quart(__name__)
#app.before_serving
async def startup():
print(f'starting')
app.q = asyncio.Queue(1)
asyncio.ensure_future(listener(app.q))
async def listener(q):
while True:
returnq, msg = await q.get()
print(msg)
await returnq.put(f'hi: {msg}')
#app.route("/echo/<message>")
async def echo(message):
while True:
returnq = asyncio.Queue(1)
await app.q.put((returnq, message))
response = await returnq.get()
return response
#app.route("/echo2/<message>")
async def echo2(message):
while True:
returnq = asyncio.Queue(1)
await app.q.put((returnq, message))
response = await returnq.get()
return response
I'm coding a small class that should be able to turn lights on when I receive an "ON" message from the mqtt broker. The issue I'm having is that the conn() function connects, and retrieves the message while the on_message() function awaits this. I haven't been able to use the message as I'm awaiting it and I cant find a solution to do this within the await (usually you just do message = conn(), when conn() returns the message). The code below was one of the attempts:
class SpaceCode:
def __init__(self, subscription, broker):
self.subscription = subscription
self.broker = broker
def turn_off(self, message):
pass
async def conn(self):
async with Client(self.broker) as client:
async with client.filtered_messages(self.subscription) as messages:
await client.subscribe(self.subscription)
async for message in messages:
return message
async def main(self):
try:
await asyncio.wait_for(self.movement_detected(), timeout=900)
except asyncio.TimeoutError:
turn_off(topic)
async def movement_detected(self):
await self.on_message()
async def on_message(self):
await self.conn()
message = self.conn()
if "ON" in str(message.payload.decode("utf-8")):
return str(message.payload.decode("utf-8"))
The error it returns then is "AttributeError:'coroutine' object has no attribute payload" which makes sense, I used a coroutine I did not await but even when just doing await message = self.conn() it will result in an error.
Thanks to #NobbyNobbs , the solution was to to await the coroutine and assign it to the variable at the same time. My version:
await self.conn()
message = self.conn()
The correct version:
await message = self.conn()
I tried out the example below (from this page):
nc = NATS()
await nc.connect(servers=["nats://demo.nats.io:4222"])
future = asyncio.Future()
async def cb(msg):
nonlocal future
future.set_result(msg)
await nc.subscribe("updates", cb=cb)
await nc.publish("updates", b'All is Well')
await nc.flush()
# Wait for message to come in
msg = await asyncio.wait_for(future, 1)
But this only seems useful for receiving one message. How would I subscribe and keep receiving messages?
I've also seen the package example, but it seems to just play both sides of the conversation, then quit.
You can find a long running service example as well: https://github.com/nats-io/nats.py/blob/master/examples/service.py
import asyncio
from nats.aio.client import Client as NATS
async def run(loop):
nc = NATS()
async def disconnected_cb():
print("Got disconnected...")
async def reconnected_cb():
print("Got reconnected...")
await nc.connect("127.0.0.1",
reconnected_cb=reconnected_cb,
disconnected_cb=disconnected_cb,
max_reconnect_attempts=-1,
loop=loop)
async def help_request(msg):
subject = msg.subject
reply = msg.reply
data = msg.data.decode()
print("Received a message on '{subject} {reply}': {data}".format(
subject=subject, reply=reply, data=data))
await nc.publish(reply, b'I can help')
# Use queue named 'workers' for distributing requests
# among subscribers.
await nc.subscribe("help", "workers", help_request)
print("Listening for requests on 'help' subject...")
for i in range(1, 1000000):
await asyncio.sleep(1)
try:
response = await nc.request("help", b'hi')
print(response)
except Exception as e:
print("Error:", e)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
loop.run_forever()
loop.close()
Don't know much about python, but looks like you are just waiting for one message and program ends. You should look at the subscriber example here. As you can see, there is a loop to wait forever or for the SIGTERM signal.
The messages are sent in 'waves', i.e. nothing for a couple seconds and then ~5 nearly at the same time. I left out token and channel.
import discord, asyncio
class Bot(discord.Client):
def __init__(self, q, channel):
super().__init__()
self.q = q
self.channel_id = channel
self.bg_task = self.loop.create_task(self.send_messages())
async def on_message(self, message):
if message.author == self.user or message.channel.id != self.channel_id:
return
print(message.content)
async def send_messages(self):
await self.wait_until_ready()
channel = self.get_channel(self.channel_id)
while not self.is_closed():
msg = await self.q.get()
await channel.send(msg)
from threading import Thread
from time import sleep
q = asyncio.Queue()
def f():
while True:
q.put_nowait("hi")
sleep(2)
Thread(target=f).start()
bot = Bot(q, channel)
bot.run(token)
Weirdly, the on_message event seems unaffected, and also, replacing msg = await self.q.get() by
msg = "hi"
await asyncio.sleep(2)
seems to result in the expected behavior.
I'm not sure where things go wrong, so I kept the example more specific to Discord.
Edit
Expanding on the asyncio.sleep behavior, I have replaced the loop in send_messages by
if 0:
msg = await self.q.get()
else:
await asyncio.sleep(0.1)
if self.q.empty():
continue
msg = await self.q.get()
await channel.send(msg)
The if is just to toggle between the original and experiment.
Clearly, one would expect the else part to be at most as fast as the if part, however waiting for the queue to be non-empty seems to completely solve the issue.
I'm starting to think discord and a blocking asyncio Queue interact in unforeseen ways
On the other hand it seems that the channel.send is the blocking line, so maybe it also has something to do with rate.
This is because the rate limit for sending messages through Discord's API is 5 / 5 seconds.
That's why you're seeing 5 messages being sent at once and then a delay as the next messages are rate limited.