I've got a django channels consumer communicating with a client. I've got a view from an external API that wants something from the client. From this view I want then to tell that consumer to ask a request to the client through his socket.
I'm currently exploring django rest framework but I can't find a way for now to directly ask anything to that consumer.
Well I've got an idea but it involves creating another socket and communicate through channels' channel. But I wish I could get rid of this overload.
From your reponse in the comments, it seems you want to send a message to the client through the consumer from your DRF view. You can check out the answer to a similar question.
First, you need to have a method in your consumer that sends a message back to the client:
...
async def send_alert(self, event):
# Send message to WebSocket
await self.send(text_data={
'type': 'alert',
'details': 'An external API api.external.com needs some data from you'
})
...
So now you can send a message to this method. Assuming the client is connected to channel1, you can do this in your view:
from channels.layers import get_channel_layer
from asgiref.sync import async_to_sync
...
channel_layer = get_channel_layer()
async_to_sync(channel_layer.send)("channel1", {
"type": "send.alert"
})
...
async_to_sync usage
Related
I am working with multiple applications that communicate asynchronously using Kafka. These applications are managed by several departments and contract testing is appropriate to ensure that the messages used during communication follow the expected schema and will evolve according to the contract specification.
It sounded like the pact library for python is a good fit because it helps creating contract tests for HTTP and message integrations.
What I wanted to do is to send an HTTP request and to listen from the appropriate and dedicated Kafka topic immediately after. But it seems that the test is forcing me specify an HTTP code even if what I am expecting is a message from a queue without an HTTP status code. Furthermore, it seems that the HTTP request is being sent before the consumer is listening. Here is some sample code.
from pact.consumer import Consumer as p_Consumer
from pact.provider import Provider as p_Provider
from confluent_kafka import Consumer as k_Consumer
pact = p_Consumer('Consumer').has_pact_with(p_Provider('Provider'))
pact.start_service()
atexit.register(pact.stop_service)
config = {'bootstrap.servers':'server', 'group.id':0, 'auto.offset.reset':'latest'}
consumer = k_consumer(config)
consumer.subscribe(['usertopic'])
def user():
while True:
msg = consumer.poll(timeout=1)
if msg is None:
continue
else:
return msg.value().decode()
class ConstractTesting(unittest.TestCase):
expected = {
'username': 'UserA',
'id':123,
'groups':['Editors']
}
pact.given('UserA exists and is not an administrator')
.upon_receiving('a request for UserA')
.with_request(method='GET',path='/user/')
.will_respond_with(200, body=expected)
with pact:
result = user()
self.assertEqual(result,expected)
How would I carry out contract testing in Python using Kafka? It feels like I am going through a lot of hoops to carry out this test.
With Pact message it's a different API you write tests against. You don't use the standard HTTP one, in fact the transport itself is ignored altogether and it's just the payload - the message - we're interested in capturing and verifying. This allows us to test any queue without having to build specific interfaces for each
See this example: https://github.com/pact-foundation/pact-python/blob/02643d4fb89ff7baad63e6436f6a929256c6bf12/examples/message/tests/consumer/test_message_consumer.py#L65
You can read more about message pact testing here: https://docs.pact.io/getting_started/how_pact_works#non-http-testing-message-pact
And finally here are some Kafka examples for other languages that may be helpful: https://docs.pactflow.io/docs/examples/kafka/js/consumer
I have created a socket with Django channels that return the serialized data of Category Object. But in the response, there is no full URL(the IP address is not there). This problem is similar to this question Django serializer Imagefield to get full URL. The difference is that I am calling the Serializer from a Consumer(Django Channels). Whereas in the link, Serializer is called from a View. In a Consumer, there is no request object as mentioned in the solution. The Django Channels says that scope in Consumers is similar to request in Views. So how can I get the full image url in this case?
The Django Channels says that scope in Consumers is similar to request in Views.
Correct; therefore it depends how to setup your events in the AsyncConsumer.
If you could share more about your code or a better explanation with a dummy example.
In general:
Import the serializers in the consumers and then send the same data to the serializers as shown below.
from <app_name>.serializers import <desired_serializer_name>Serializer
from channels.db import database_sync_to_async
#database_sync_to_async
def serializer_checking_saving_data(self, data):
serializer = <desired_serializer_name>Serializer(data=data)
serializer.is_valid(raise_exception=True)
x = serializer.create(serializer.validated_data)#this will create the value in the DB
return <desired_serializer_name>Serializer(x).data
To fetch data from a websocket request:
Setup a receive event (ie channel-layer will receive the data) wherein it would trigger a particular event[for example I will implement to simply display that data]
#write this inside the AsyncWebsocketConsumer
async def receive_json(self, content, **kwargs):
"""[summary]
• All the events received to the server will be evaluated here.
• If websocket has event-type based on these the receive function will execute
the respective function
"""
message_type = content.get('type')
if message_type == 'start.sepsis':
await self.display_the_data(content)
async def display_the_data(self,data)
message = data.get('payload')
print(f"The data sent to the channel/socket is \n {data}")
You can make the websocket request in the following way:-
create a new python file
import json
import websocket
import asyncio
async def making_websocket_request():
ws_pat = websocket.WebSocket()
ws_pat.connect(
'ws://localhost:8000/<ws-router-url>/')
asyncio.sleep(2)#it might take a couple of seconds to connect to the server
ws.send(json.dumps({
'type':'display.the_data'
#the channels will convert "display.the_data" to "display_the_data"
#since "display_the_data" their is an event as defined above it would be called
'payload':{<can-be-any-json-data>}
#this payload will be sent as a parameter when calling the function.
}))
I'm trying to do a get request for a seperate django CRUD API app inside a Django Channels app connect method
So inside the consumers.py I'm doing this
class AssistantConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.commands = requests.get('http://localhost:8000/api/commands')
print(self.commands)
Doing this results in the websocket getting stuck on
WebSocket HANDSHAKING /ws/assistant/user/ [127.0.0.1:64374]
Can anyone tell me why?
The API is working on it's own, I'm posting to it from React, which also connects to the websocket. All that is working - I just need to fetch some data from the database in the consumer.
Couldn't find anything anywhere about this situation.
OK I found a solution - don't know if it's the right one or how stable it is but this package on PyPi does the trick.
https://pypi.org/project/requests-async/
Thanks to Timothee for pointing out that I needed to be doing it async.
This now works.
import requests_async as requests
async def connect(self):
self.commands = await requests.get('http://localhost:8000/api/commands/')
print(self.commands)
await self.accept()
Maybe this will help someone else and if anyone knows of a reason I shouldn't be doing this I'd be interested to know.
async def connect is only called when a client attempts a connection and your routing file sends the incoming connection to your AssistantConsumer. In your case, you are getting stuck on this initial 'handshake'. This means that you are receiving the request to connect from the client but you are not accepting that connection and therefore the WebSocket connection is never opened.
Adding await self.accept() should accept the incoming connection and therefore open the WebSocket.
This is what it would look like:
class AssistantConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.commands = requests.get('http://localhost:8000/api/commands')
print(self.commands)
await self.accept()
I created a simple Django Channels consumer that connects to a Redis channel and receives some data from this channel, i want to send this data to the frontend.
The consumer is able to connect to the consumer and receives the data; the problem is that if i try to load the page when the consumer is running, the page will be stuck on loading. I'm sure this happens because the connection to the Redis channel is a blocking operation, or it could be a problem with threads. I'm new to this concepts, so i decided to make a question for it.
Here is my consumer:
class EchoConsumer(AsyncConsumer):
async def websocket_connect(self, event):
self.send({
'type': 'websocket.accept'
})
self.receive(event)
def receive(self, event):
redis_url = 'redis://localhost:6379/0'
connection = redis.StrictRedis.from_url(redis_url, decode_responses=True)
channel = 'TEST'
params = urllib.parse.parse_qs(self.scope.get('query_string', b'').decode('utf-8'))
pubsub = connection.pubsub(ignore_subscribe_messages=True)
pubsub.subscribe(channel)
for message in pubsub.listen():
# self.send({
# 'type': 'websocket.send',
# 'text': message['data'],
# })
print(message['data'])
async def websocket_disconnect(self, event):
print('DISCONNECTED!')
So what happens is that i can see the data being printed to my console, but if i try to leave that page and reach a different part of my site, the page will get stuck on loading. Can anyone help me fix this?
There are 2 things you could be trying to do here.
Subscribing once and sending this information to all open connections
the issue you are having is pubsub.listen(): will loop forever (never stopping). So your consumer will never continue and be able to process any more messages.
Since (at least in this example) it looks like you are always hitting redis with static values (not dependent on the request from the user) you are better off doing this subscription outside of your consumer. (in a django command https://docs.djangoproject.com/en/3.0/howto/custom-management-commands/)
Then you can have that command then send these messages over a channel layer to your subscribed consumers.
this would make your consumer look like this
class EchoConsumer(AsyncJsonWebsocketConsumer):
async def on_message(self, message):
await self.send_json(message)
then in your management command instread of printing you can send the message using
async_to_sync(channel_layer.group_send)(
"echo_group",
{"type": "on.message", "rate":Rate, "quantity": Quantity, "symbol": Symbol, "order": Order},
)
Subscribing once for each open connection
This you should only do if you expect the subscription to be different for each websocket connection. (eg you are using a value in the url/query/headers or subscribing only when the user sends a ws message to your consumer with a given filter value).
Doing this is a LOT more complex for a few reasons:
Redis will not handle as many open connections to it as you can have websocket connections.
You need to setup a nested async task that can handle the events from redis in such a way that they do not block the rest of the consumer.
If you do still need this functionality i'm happy to update the answer with a solution (but warning it will be long).
I have the following scenario I would like to implement:
User surfs to our website
User enters a bitcoin address.
A websocket is created to the server, passing the address.
The server registers a callback with Blocktrail
When the callback is triggered (a payment was seen by Blocktrail) we send a message back to the browser.
The page the user is browsing is updated to show the message recieved
I'm using webhooks from the Blocktrail API to "listen" to an event, being the reception of coins on an address.
Now, when the event happens, the API does a POST to my URL. This should send a message to the browser that is connected to my server with socket.io (such as 'payment seen on blockchain')
So the question is,
How can I send a message from a route to a socket using flask-socketio
Pseudo code:
#app.route('/callback/<address>')
def callback(id):
socketio.send('payment seen on blockchain')
#socketio.on('address',address)
def socketlisten(address):
registerCallback(address)
I'm going to describe how to solve this using Flask-SocketIO beta version 1.0b1. You can also do this with the 0.6 release, but it is a bit more complicated, the 1.0 release makes addressing individual clients easier.
Each client of a socket connection gets assigned a session id that uniquely identifies it, the so called sid. Within a socket function handler, you can access it as request.sid. Also, upon connection, each client is assigned to a private room, named with the session id.
I assume the metadata that you receive with the callback allows you to identify the user. What you need is to obtain the sid of that user. Once you have it, you can send your alert to the corresponding room.
Example (with some hand-waving regarding how you attach a sid to an address):
#app.route('/callback/<address>')
def callback(address):
sid = get_sid_from_address(address)
socketio.send('payment seen on blockchain', room=sid)
#socketio.on('address')
def socketlisten(address):
associate_address_with_sid(address, request.sid)