I have an isolated python script that simply captures data from Twitter's streaming API and then on the receipt of each message, using redis pubsub it publishes to the channel "tweets". Here is that script:
def main():
username = "username"
password = "password"
track_list = ["apple", "microsoft", "google"]
with tweetstream.FilterStream(username, password, track=track_list) as stream:
for tweet in stream:
text = tweet["text"]
user = tweet["user"]["screen_name"]
message = {"text": text, "user": user}
db.publish("tweets", message)
if __name__ == '__main__':
try:
print "Started..."
main()
except KeyboardInterrupt:
print '\nGoodbye!'
My server side socket.io implementation is done using django-socketio (based off of gevent-socketio) https://github.com/stephenmcd/django-socketio which simply provides a few helper decorators as well as a broadcast_channel method. Because it's done in django, I've simply put this code in views.py simply so that they're imported. My views.py code:
def index(request):
return render_to_response("twitter_app/index.html", {
}, context_instance=RequestContext(request))
def _listen(socket):
db = redis.Redis(host="localhost", port=6379, db=0)
client = db.pubsub()
client.subscribe("tweets")
tweets = client.listen()
while True:
tweet = tweets.next()
tweet_data = ast.literal_eval(tweet["data"])
message = {"text": tweet_data["text"], "user": tweet_data["user"], "type": "tweet"}
socket.broadcast_channel(message)
#on_subscribe(channel="livestream")
def subscribe(request, socket, context, channel):
g = Greenlet.spawn(_listen, socket)
The client side socket.io JavaScript simply connects and subscribes to the channel "livestream" and captures any received messages to that channel:
var socket = new io.Socket();
socket.connect();
socket.on('connect', function() {
socket.subscribe("livestream");
});
socket.on('message', function(data) {
console.log(data);
});
The obvious problem with this code is that each time a new user or browser window is opened to the page, a new _listen method is spawned and the tweets get subscribed to and broadcast for each user resulting in duplicate messages being received on the client. My question is, where would the proper place be to put the _listen method so that it's only created once regardless of the # of clients? Also, keeping in mind that the broadcast_channel method is a method of a socket instance.
The problem was that I was using socket.broadcast_channel when I should have been using socket.send.
Related
class WSTestView(WebsocketConsumer):
def connect(self):
self.accept();
self.send(json.dumps({'status': 'sent'})) # client receives this
def receive(self, text_data=None, bytes_data=None):
notifications = Notification.objects.filter(receiver=text_data) # receives user id
serializer = NotificationSerializer(notifications, many=True).data
self.send(serializer) # client does not receives this
Frontend
// ...
useEffect(() => {
socket.onmessage = (e) => { console.log(e.data) }
}, [])
// ...
I've just started with django-channels and am working on a consumer that sends the user's notifications when it receives the user's id but on the frontend the onmessage event does not receive anything, how can I fix this and is there a better way that I can implement this?
It's probably your front end and not django-channels if you're able to connect to it. The most probable reason is that your onmessage binds after, because of the componenDidMount/useEffect, the server has already sent the message.
Try just as a test, to rule that out. To put the
socket.onmessage = (e) => { console.log(e.data) }
Right after the new WebSocket.... Let me know if that prints something in the console. Another way to test/isolate it is with an online WebSocket tester like this https://www.lddgo.net/en/network/websocket
class WSTestView(WebsocketConsumer):
def connect(self):
self.accept();
self.send(json.dumps({'status': 'sent'}))
def receive(self, text_data=None, bytes_data=None):
notifications = Notification.objects.filter(receiver=text_data) # receives user id
serializer = NotificationSerializer(notifications, many=True).data
self.send(json.dumps(serializer)) # <-------
Had to convert the serialized data into a JSON string.
Frontend
useEffect(() => {
socket.onmessage = (e) => { console.log(e.data) }
}, [socket]) // <-------
Had to put the socket variable into the dependency array
I'm attempting to send consumers.py information to display on the client end outside of consumers.py.
I've referenced Send message using Django Channels from outside Consumer class this previous question, but the sub process .group_send or .group_add don't seem to exist, so I feel it's possible I'm missing something very easy.
Consumers.py
from channels.generic.websocket import WebsocketConsumer
from asgiref.sync import async_to_sync
class WSConsumer(WebsocketConsumer):
def connect(self):
async_to_sync(self.channel_layer.group_add)("appDel", self.channel_name)
self.accept()
self.render()
appAlarm.py
def appFunc(csvUpload):
#csvUpload=pd.read_csv(request.FILES['filename'])
csvFile = pd.DataFrame(csvUpload)
colUsernames = csvFile.usernames
print(colUsernames)
channel_layer = get_channel_layer()
for user in colUsernames:
req = r.get('https://reqres.in/api/users/2')
print(req)
t = req.json()
data = t['data']['email']
print(user + " " + data)
message = user + " " + data
async_to_sync(channel_layer.group_send)(
'appDel',
{'type': 'render', 'message': message}
)
It's throwing this error:
async_to_sync(channel_layer.group_send)(
AttributeError: 'NoneType' object has no attribute 'group_send'
and will throw the same error for group_add when stripping it back more to figure out what's going on, but per the documentation HERE I feel like this should be working.
To anyone looking at this in the future, I was not able to use redis or even memurai in Windows OS due to cost. I ended up using server side events (SSE), specifically django-eventstream, and so far it's worked great as I didn't need the client to interact with the server, for a chat application this would not work.
Eventstream creates an endpoint at /events/ the client can connect to and receive a streaming http response.
Sending data from externalFunc.py:
send_event('test', 'message', {'text': 'Hello World'})
Event listener in HTML page:
var es = new ReconnectingEventSource('/events/');
es.addEventListener('message', function (e) {
console.log(e.data);
var source = new EventSource("/events/")
var para = document.createElement("P");
const obj = JSON.parse(event.data)
para.innerText = obj.text;
document.body.appendChild(para)
}, false);
es.addEventListener('stream-reset', function (e) {
}, false);
I am writing these two following function in my code to be able to process incoming messages and respond back to the user on Messenger via a bot:
#app.route('/', methods=['post'])
def webhook():
# endpoint for processing incoming messaging events
data = request.get_json()
print(data) # you may not want to log every incoming message in production, but it's good for testing
if data["object"] == "page":
for entry in data["entry"]:
for messaging_event in entry["messaging"]:
if messaging_event.get("message"): # someone sent us a message
sender_id = messaging_event["sender"]["id"] # the Facebook ID of the person sending you the message
recipient_id = messaging_event["recipient"]["id"] # the recipient's ID, which should be your page's facebook ID
message_text = messaging_event["message"]["text"] # the message's text
responseai = response(message_text, sender_id)
send_message(sender_id, responseai)
if messaging_event.get("delivery"): # delivery confirmation
pass
if messaging_event.get("optin"): # optin confirmation
pass
if messaging_event.get("postback"): # user clicked/tapped "postback" button in earlier message
pass
return "Ok", 200
#app.route('/', methods=['GET'])
def verify():
# when the endpoint is registered as a web hook, it must echo back
# the 'hub.challenge' value it receives in the query arguments
if request.args.get("hub.mode") == "subscribe" and request.args.get("hub.challenge"):
if not request.args.get("hub.verify_token") == os.environs["VERIFY_TOKEN"]:
return "Verification token mismatch", 403
return request.args["hub.challenge"], 200
return "Hello World", 200
When I access my localhost:5000 where my Flask is, only Hello World appears on the browser. How would I know that the function web-hook is working? Should it also display the 'Ok'?
Your browser will not send a POST request, not without some additional HTML and Javascript coding. The simplest way to test if your hook works is by using the curl command-line client.
It can send both GET and POST requests. Testing if your GET handler works correctly:
curl -X GET "localhost:5000/?hub.verify_token=<YOUR_VERIFY_TOKEN>&hub.challenge=CHALLENGE_ACCEPTED&hub.mode=subscribe"
should produce CHALLENGE_ACCEPTED as the output. Then test the POST handler with:
curl -H "Content-Type: application/json" -X POST "localhost:5000/" -d '{"sender":{"id":"<PSID>"}, "recipient":{"id":"<PAGE_ID>"}, "timestamp":1458692752478, "message":{"mid":"mid.1457764197618:41d102a3e1ae206a38", "text":"hello, world!", "quick_reply": {"payload": "<DEVELOPER_DEFINED_PAYLOAD>"}}}'
See the Setting Up Your Webhook section of the Messenger Platform Getting Started documentation, and the message received event for details on what to expect when handling a message event.
Another option is to write Python tests to cover the same:
import os
import pytest
import your_flask_module
#pytest.fixture
def client():
your_flask_module.app.config['TESTING'] = True
yield your_flask_module.app.test_client()
def test_register(client):
args = {
'hub.verify_token': os.environ["VERIFY_TOKEN"],
'hub.challenge': 'CHALLENGE_ACCEPTED',
'hub.mode': 'subscribe',
}
rv = client.get('/', params=args)
assert b'CHALLANGE_ACCEPTED' in rv.data
def test_message_event(client):
event = {
"sender": {"id": "<PSID>"},
"recipient": {"id":"<PAGE_ID>"},
"timestamp": 1458692752478,
"message": {
"mid": "mid.1457764197618:41d102a3e1ae206a38",
"text": "hello, world!",
"quick_reply": {
"payload": "<DEVELOPER_DEFINED_PAYLOAD>"
}
}
}
rv = client.post('/', json=event)
assert rv.status_code == 200
I'm currently trying to create a backend server to communicate with some clients with a websocket. The clients makes some request to the backend and the backend responds directly to the client through a consumer.
In addition, I've got an API that needs to send some requests to the client. It has to go through the opened socket of the consumer. I'm using Django Rest Framework for the API. So I've got 2 apps for now. One for the consumer and one for the API. I want to know if it's the correct way or not.
This is actually the code I'm thinking about right now:
# mybackendapp/consumers.py
class MyConsumer(AsyncWebsocketConsumer):
async def connect(self):
self.client_id = self.scope['url_route']['kwargs']['client_id']
# This line I don't get it very well. It comes from:
# [channels doc: single channels][1]
# I don't know if I should create the Clients myself or if it's
# created automatically
Clients.objects.create(channel_name=self.channel_name,
self.client_id)
self.accept()
async def disconnect(self):
Clients.objects.filter(channel_name=self.channel_name).delete()
async def receive(self, text_data):
self.recv_data = json.loads(text_data)
if self.recv_data[0] == CLIENT_REQUEST:
self.handler = ClientRequestHandler(self.client_id,
self.recv_data)
await self.handler.run()
self.sent_data = self.handler.response
self.send(self.sent_data)
elif self.recv_data[0] == CLIENT_RESPONSE:
self.handler = ClientResponseHandler(self.client_id,
self.recv_data)
channel_layer = get_channel_layer()
# Here I'm not sure but I may have several API requests so
# several row with the same client_id.
# I welcome info on how to deal with that.
api_channel_name = self.another_handler.ext_channel_name
channel_layer.send(api_channel_name, {
"text_data": self.handler.response,
})
async def message_from_api(self, event):
self.api_channel_name = event['channel_name_answer']
# this line is for hiding the fact that I'm only manipulating data
# to send it to a correct format to the socket
self.another_handler = ExternalDataHandler(event['json_data'])
query_to_client = another_handler.get_formatted_query()
self.send(query_to_client)
In receive, this consumer handles differently the messages from the client depending if it's initiated by the client or the rest API. You can see that with CLIENT_REQUEST and CLIENT_RESPONSE constants.
Now from the API view:
# myapi/views.py
from channels.layers import get_channel_layer
def my_api_view(request, client_id):
channel_layer = get_channel_layer()
if request.method == 'POST':
ext_request_data_json = request.data
client_channel_name = Clients.objects.get(
client_id=client_id).channel_name
# I don't know what type is listen_channel_name. I assume it's str
listen_channel_name = async_to_sync(channels_layers.new_channel)()
async_to_sync(channel_layer.send)(
client_channel_name, {
'type': 'message.from.api',
'json_data': ext_request_data_json,
'channel_name_answer': listen_channel_name
}
)
received_msg = channel_layer.receive(listen_channel_name)
I believe that this code should work. I want to know if it's the correct way to do it.
See djangochannelsrestframework as a possible alternative solution.
Django Channels Rest Framework provides a DRF like interface for building channels-v3 websocket consumers.
I'm very new to django-channels so this is probably a very simple question.
On our website, there is a permanent button "Messages" in the header. I want user to be notified about new message immediately. So I use channels for this purpose. If there is a new message created, I send a number of not readed conversations through channels to client:
class Message(..):
def save(...):
notify_recipient(self)
def notify_recipient(self):
Group('%s' % self.recipient).send({
"text": json.dumps({
"message":{"text":truncatechars(self.text,100)},
"unreaded_conversations":Conversation.objects.get_unreaded_conversations(self.recipient).count(),
}),
})
And in base.html:
const webSocketBridge = new channels.WebSocketBridge();
webSocketBridge.connect('/notifications/');
webSocketBridge.listen(function (action, stream) {
console.log(action, stream);
var conversations_sidebar = $('#id_conversations_sidebar');
var messages_list = $('#messagesList');
if (action.unreaded_conversations) {
$('#id_unreaded_conversations_count').text(action.unreaded_conversations);
}
On the other hand, there is a page /chat/detail/<username>/ where users chat with each other. This chat should be live so I need to recieve messages through channels.
For now, I've added rendered message to the notify_recipient method but the problem is that it has to render the message allways, even when user is not on this /chat/detail/<username>/ url which is not efficient.
Do you know how to recieve rendered messages only when user is in the current chat?
routing.py
#channel_session_user
def message_handler(message):
message.reply_channel.send({"accept": True})
#channel_session_user_from_http
def ws_connect(message,):
Group("%s" % message.user).add(message.reply_channel)
message.reply_channel.send({"accept": True})
channel_routing = [
route("websocket.receive", message_handler), # we register our message handler
route("websocket.connect", ws_connect) # we register our message handler
]