I have created a channel that implements some text operations using a shared task which returns the response back to the channel layer.
#consumers.py
import json
import pdb
from asgiref.sync import async_to_sync
from channels.generic.websocket import AsyncWebsocketConsumer
from . import tasks
COMMANDS = {
'help': {
'help': 'Display help message.',
},
'sum': {
'args': 2,
'help': 'Calculate sum of two integer arguments. Example: `sum 12 32`.',
'task': 'add'
},
'status': {
'args': 1,
'help': 'Check website status. Example: `status twitter.com`.',
'task': 'url_status'
},
}
class Consumer(AsyncWebsocketConsumer):
async def receive(self, text_data):
text_data_json = json.loads(text_data)
message = text_data_json['message']
# response_message = 'Please type `help` for the list of the commands.'
message_parts = message.split()
if message_parts:
command = message_parts[0].lower()
if command == 'help':
response_message = 'List of the available commands:\n' + '\n'.join([f'{command} - {params["help"]} ' for command, params in COMMANDS.items()])
elif command in COMMANDS:
if len(message_parts[1:]) != COMMANDS[command]['args']:
response_message = f'Wrong arguments for the command `{command}`.'
else:
getattr(tasks, COMMANDS[command]['task']).delay(self.channel_name, *message_parts[1:])
# response_message = f'Command `{command}` received.'
response_message = message
await self.channel_layer.send(
self.channel_name,
{
'type': 'chat_message',
'message': response_message
}
)
#tasks.py
#shared_task
def add(channel_layer, x, y):
message = '{}+{}={}'.format(x, y, int(x) + int(y))
async_to_sync(channel_layer.send)({"type": "chat.message", "message": message})
I want to share this channel as an api which could be accessed using http request. for which i have written following views.
views.py
#csrf_exempt
#api_view(['POST'])
def api(request):
channel_layer = get_channel_layer()
async_to_sync(channel_layer.send)('test_channel', {'type': 'hello'})
ret = async_to_sync(channel_layer.receive)(channel_name)
return JsonResponse({"msg":ret})
While receiving from the views I get the same message that I have sent. How can I share the channel or handle incoming messages without connecting using WebSockets from the template?
If you just want the POST request to send a message
views.py
#csrf_exempt
#api_view(['POST'])
def api(request):
channel_layer = get_channel_layer()
async_to_sync(channel_layer.send)('test_channel', {'type': 'chat.message'})
return JsonResponse({"msg":"sent"})
You will need to ensure you have subscribed to test_channel in your consumer. And you will need a method on that consumer chat_message.
If you want to wiat for the response in your post
your not going to be able to do this using channel_layer.send since that is async to the extend that you dont have any concept fo response. In addition there might not even be an instance of your consumer running since Channels only creates instances when it has open websocket connections that rout to them.
so I think you can do either:
To create an instance of your consumer and send message to it from synchronise python code is going to be very complex. I suggest you do not do this approach It is complex, dirty and likely to break in all sorts of un-expected
instead I suggest to move the code you want to share between your HTTP view and your websocket view into a single place (not part of the consumer) were they both can call these functions.
Related
I'm attempting to send consumers.py information to display on the client end outside of consumers.py.
I've referenced Send message using Django Channels from outside Consumer class this previous question, but the sub process .group_send or .group_add don't seem to exist, so I feel it's possible I'm missing something very easy.
Consumers.py
from channels.generic.websocket import WebsocketConsumer
from asgiref.sync import async_to_sync
class WSConsumer(WebsocketConsumer):
def connect(self):
async_to_sync(self.channel_layer.group_add)("appDel", self.channel_name)
self.accept()
self.render()
appAlarm.py
def appFunc(csvUpload):
#csvUpload=pd.read_csv(request.FILES['filename'])
csvFile = pd.DataFrame(csvUpload)
colUsernames = csvFile.usernames
print(colUsernames)
channel_layer = get_channel_layer()
for user in colUsernames:
req = r.get('https://reqres.in/api/users/2')
print(req)
t = req.json()
data = t['data']['email']
print(user + " " + data)
message = user + " " + data
async_to_sync(channel_layer.group_send)(
'appDel',
{'type': 'render', 'message': message}
)
It's throwing this error:
async_to_sync(channel_layer.group_send)(
AttributeError: 'NoneType' object has no attribute 'group_send'
and will throw the same error for group_add when stripping it back more to figure out what's going on, but per the documentation HERE I feel like this should be working.
To anyone looking at this in the future, I was not able to use redis or even memurai in Windows OS due to cost. I ended up using server side events (SSE), specifically django-eventstream, and so far it's worked great as I didn't need the client to interact with the server, for a chat application this would not work.
Eventstream creates an endpoint at /events/ the client can connect to and receive a streaming http response.
Sending data from externalFunc.py:
send_event('test', 'message', {'text': 'Hello World'})
Event listener in HTML page:
var es = new ReconnectingEventSource('/events/');
es.addEventListener('message', function (e) {
console.log(e.data);
var source = new EventSource("/events/")
var para = document.createElement("P");
const obj = JSON.parse(event.data)
para.innerText = obj.text;
document.body.appendChild(para)
}, false);
es.addEventListener('stream-reset', function (e) {
}, false);
I am writing these two following function in my code to be able to process incoming messages and respond back to the user on Messenger via a bot:
#app.route('/', methods=['post'])
def webhook():
# endpoint for processing incoming messaging events
data = request.get_json()
print(data) # you may not want to log every incoming message in production, but it's good for testing
if data["object"] == "page":
for entry in data["entry"]:
for messaging_event in entry["messaging"]:
if messaging_event.get("message"): # someone sent us a message
sender_id = messaging_event["sender"]["id"] # the Facebook ID of the person sending you the message
recipient_id = messaging_event["recipient"]["id"] # the recipient's ID, which should be your page's facebook ID
message_text = messaging_event["message"]["text"] # the message's text
responseai = response(message_text, sender_id)
send_message(sender_id, responseai)
if messaging_event.get("delivery"): # delivery confirmation
pass
if messaging_event.get("optin"): # optin confirmation
pass
if messaging_event.get("postback"): # user clicked/tapped "postback" button in earlier message
pass
return "Ok", 200
#app.route('/', methods=['GET'])
def verify():
# when the endpoint is registered as a web hook, it must echo back
# the 'hub.challenge' value it receives in the query arguments
if request.args.get("hub.mode") == "subscribe" and request.args.get("hub.challenge"):
if not request.args.get("hub.verify_token") == os.environs["VERIFY_TOKEN"]:
return "Verification token mismatch", 403
return request.args["hub.challenge"], 200
return "Hello World", 200
When I access my localhost:5000 where my Flask is, only Hello World appears on the browser. How would I know that the function web-hook is working? Should it also display the 'Ok'?
Your browser will not send a POST request, not without some additional HTML and Javascript coding. The simplest way to test if your hook works is by using the curl command-line client.
It can send both GET and POST requests. Testing if your GET handler works correctly:
curl -X GET "localhost:5000/?hub.verify_token=<YOUR_VERIFY_TOKEN>&hub.challenge=CHALLENGE_ACCEPTED&hub.mode=subscribe"
should produce CHALLENGE_ACCEPTED as the output. Then test the POST handler with:
curl -H "Content-Type: application/json" -X POST "localhost:5000/" -d '{"sender":{"id":"<PSID>"}, "recipient":{"id":"<PAGE_ID>"}, "timestamp":1458692752478, "message":{"mid":"mid.1457764197618:41d102a3e1ae206a38", "text":"hello, world!", "quick_reply": {"payload": "<DEVELOPER_DEFINED_PAYLOAD>"}}}'
See the Setting Up Your Webhook section of the Messenger Platform Getting Started documentation, and the message received event for details on what to expect when handling a message event.
Another option is to write Python tests to cover the same:
import os
import pytest
import your_flask_module
#pytest.fixture
def client():
your_flask_module.app.config['TESTING'] = True
yield your_flask_module.app.test_client()
def test_register(client):
args = {
'hub.verify_token': os.environ["VERIFY_TOKEN"],
'hub.challenge': 'CHALLENGE_ACCEPTED',
'hub.mode': 'subscribe',
}
rv = client.get('/', params=args)
assert b'CHALLANGE_ACCEPTED' in rv.data
def test_message_event(client):
event = {
"sender": {"id": "<PSID>"},
"recipient": {"id":"<PAGE_ID>"},
"timestamp": 1458692752478,
"message": {
"mid": "mid.1457764197618:41d102a3e1ae206a38",
"text": "hello, world!",
"quick_reply": {
"payload": "<DEVELOPER_DEFINED_PAYLOAD>"
}
}
}
rv = client.post('/', json=event)
assert rv.status_code == 200
I'm currently trying to create a backend server to communicate with some clients with a websocket. The clients makes some request to the backend and the backend responds directly to the client through a consumer.
In addition, I've got an API that needs to send some requests to the client. It has to go through the opened socket of the consumer. I'm using Django Rest Framework for the API. So I've got 2 apps for now. One for the consumer and one for the API. I want to know if it's the correct way or not.
This is actually the code I'm thinking about right now:
# mybackendapp/consumers.py
class MyConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.client_id = self.scope['url_route']['kwargs']['client_id']
# This line I don't get it very well. It comes from:
# [channels doc: single channels][1]
# I don't know if I should create the Clients myself or if it's
# created automatically
Clients.objects.create(channel_name=self.channel_name,
self.client_id)
self.accept()
async def disconnect(self):
Clients.objects.filter(channel_name=self.channel_name).delete()
async def receive(self, text_data):
self.recv_data = json.loads(text_data)
if self.recv_data[0] == CLIENT_REQUEST:
self.handler = ClientRequestHandler(self.client_id,
self.recv_data)
await self.handler.run()
self.sent_data = self.handler.response
self.send(self.sent_data)
elif self.recv_data[0] == CLIENT_RESPONSE:
self.handler = ClientResponseHandler(self.client_id,
self.recv_data)
channel_layer = get_channel_layer()
# Here I'm not sure but I may have several API requests so
# several row with the same client_id.
# I welcome info on how to deal with that.
api_channel_name = self.another_handler.ext_channel_name
channel_layer.send(api_channel_name, {
"text_data": self.handler.response,
})
async def message_from_api(self, event):
self.api_channel_name = event['channel_name_answer']
# this line is for hiding the fact that I'm only manipulating data
# to send it to a correct format to the socket
self.another_handler = ExternalDataHandler(event['json_data'])
query_to_client = another_handler.get_formatted_query()
self.send(query_to_client)
In receive, this consumer handles differently the messages from the client depending if it's initiated by the client or the rest API. You can see that with CLIENT_REQUEST and CLIENT_RESPONSE constants.
Now from the API view:
# myapi/views.py
from channels.layers import get_channel_layer
def my_api_view(request, client_id):
channel_layer = get_channel_layer()
if request.method == 'POST':
ext_request_data_json = request.data
client_channel_name = Clients.objects.get(
client_id=client_id).channel_name
# I don't know what type is listen_channel_name. I assume it's str
listen_channel_name = async_to_sync(channels_layers.new_channel)()
async_to_sync(channel_layer.send)(
client_channel_name, {
'type': 'message.from.api',
'json_data': ext_request_data_json,
'channel_name_answer': listen_channel_name
}
)
received_msg = channel_layer.receive(listen_channel_name)
I believe that this code should work. I want to know if it's the correct way to do it.
See djangochannelsrestframework as a possible alternative solution.
Django Channels Rest Framework provides a DRF like interface for building channels-v3 websocket consumers.
i am pretty new in django-channels and python, i make a demo using django-channels and django to send data from backend to front activly. the code in consumers.py as below:
websocket_clients = []
class WebsocketConsumer(AsyncWebsocketConsumer):
def __init__(self, *args, **kwargs):
"""
initial a new thread when new websocket instance initialed.
"""
super(WebsocketConsumer, self).__init__(self)
self.index_websocket = GendataThread()
async def connect(self):
self.index_websocket.channels_name = self.channel_name
print('channel_name', self.channel_name)
self.index_websocket.start()
websocket_clients.append(self.channel_name)
await self.accept()
async def disconnect(self, close_code):
"""disconnected a websocket instance, and stop the thread."""
self.index_websocket.stop()
time.sleep(2)
index = websocket_clients.index(self.channel_name)
websocket_clients.pop(index)
async def receive(self, text_data):
"""receive message from connected client, wouldn't work in this project."""
text_data = '{"message": "%s"}' % (text_data)
text_data_json = json.loads(text_data)
async def chat_message(self, event):
"""send message to websocket."""
print('event', event)
await self.send(text_data=json.dumps(
{'message': event['message']}))
def websocket_send_data(data, channel_name):
"""
send data to front in websocket protocol,
:param data: data which will be sent,
:param channel_name: a symbol to mark connected client.
:return:
"""
channel_layer = get_channel_layer()
event = {
"type": "chat.message",
"message": data
}
async_to_sync(channel_layer.send)(channel_name, event)
as you can see, i make a function name: websocket_send_data(data, channel_name),which i can call in GendataThread() as below:
class GendataThread(threading.Thread):
"""new thread
thread for getting data from cluster, and send to front with websocket protocol.
"""
def __init__(self, *args, **kwargs):
super(GendataThread, self).__init__(*args, **kwargs)
self.channels_name = None
def run(self):
while true:
websocker_send_data('1', self.channels_name)
in this case, i can send data from backend to front while django project is running, i connect to the url
ws://127.0.0.1:8000/ws/intime_data
which is set in routing.py as below:
websocket_urlpatterns = [
url(r'^ws/intime_data$', consumers.WebsocketConsumer),
]
i can receive data at front, but as it runs a few minutes, things were going wrong:
aioredis.errors.ReplyError: OOM command not allowed when used memory > 'maxmemory'.
i think is because redis is full by saving data,but i don't want these data,i try to config redis.conf by:
maxmemory-policy allkeys-lru
but still comes error when maxmemory is used. and sometimes will comes such error:
channels.exceptions.Channels Full
i think channels is using redis as a cache, but i don't know why errors come, can anyone helps me? thanks in advance.
you add this code to your settings.py:
CHANNEL_LAYERS = {
"default": {
"BACKEND": "asgi_redis.RedisChannelLayer",
"CONFIG": {
"hosts": [(XXX, XXX)],
},
"ROUTING": "XXXXXchannel_routing"
},
}
I have an isolated python script that simply captures data from Twitter's streaming API and then on the receipt of each message, using redis pubsub it publishes to the channel "tweets". Here is that script:
def main():
username = "username"
password = "password"
track_list = ["apple", "microsoft", "google"]
with tweetstream.FilterStream(username, password, track=track_list) as stream:
for tweet in stream:
text = tweet["text"]
user = tweet["user"]["screen_name"]
message = {"text": text, "user": user}
db.publish("tweets", message)
if __name__ == '__main__':
try:
print "Started..."
main()
except KeyboardInterrupt:
print '\nGoodbye!'
My server side socket.io implementation is done using django-socketio (based off of gevent-socketio) https://github.com/stephenmcd/django-socketio which simply provides a few helper decorators as well as a broadcast_channel method. Because it's done in django, I've simply put this code in views.py simply so that they're imported. My views.py code:
def index(request):
return render_to_response("twitter_app/index.html", {
}, context_instance=RequestContext(request))
def _listen(socket):
db = redis.Redis(host="localhost", port=6379, db=0)
client = db.pubsub()
client.subscribe("tweets")
tweets = client.listen()
while True:
tweet = tweets.next()
tweet_data = ast.literal_eval(tweet["data"])
message = {"text": tweet_data["text"], "user": tweet_data["user"], "type": "tweet"}
socket.broadcast_channel(message)
#on_subscribe(channel="livestream")
def subscribe(request, socket, context, channel):
g = Greenlet.spawn(_listen, socket)
The client side socket.io JavaScript simply connects and subscribes to the channel "livestream" and captures any received messages to that channel:
var socket = new io.Socket();
socket.connect();
socket.on('connect', function() {
socket.subscribe("livestream");
});
socket.on('message', function(data) {
console.log(data);
});
The obvious problem with this code is that each time a new user or browser window is opened to the page, a new _listen method is spawned and the tweets get subscribed to and broadcast for each user resulting in duplicate messages being received on the client. My question is, where would the proper place be to put the _listen method so that it's only created once regardless of the # of clients? Also, keeping in mind that the broadcast_channel method is a method of a socket instance.
The problem was that I was using socket.broadcast_channel when I should have been using socket.send.