Develop Reference

How to broadcast a single message using Flask and angular

I'm trying to broadcast a message to all the users using flask and angular, I'm trying to achieve this using socketio but since I'm new to socketio, I'm unable to implement it as expected.
Basic idea is to pick a message from db table and broadcast to all users using via API call.
The server should show the new message without the website being refreshed by the user.
Python Code:
import socketio
from aiohttp import web
app = Flask(__name__)
sio = socketio.AsyncServer(async_mode='aiohttp', logger=True, engineio_logger=True)
sio.attach(app)
#sio.on('join_room', namespace='/maintenance')
def handle_message(sid, room):
sio.enter_room(sid, room=room, namespace='/maintenance')
#sio.on('getMessage', namespace='/maintenance')
async def handle_message(sid, maintenance_room):
await sio.emit('getMaintenanceResponse', 'json_object', namespace='/maintenance', room=maintenance_room)
#sio.on('addMessage', namespace='/maintenance')
async def add_message(sid, message_data, maintenance_room):
await sio.emit('getMaintenanceResponse', 'json_object', namespace='/maintenance', room=maintenance_room)
Angular Code:
this.socket.createMaintenanceRoom('maintenance');
this.socket.getMaintenance('maintenance').subscribe(response => {
if (response) {
this.maintenanceNotesData = response;
}
},
err => console.error('Observer got an error: ' + err),
() => console.log('Observer got a complete notification'));

Django channels - unable to subscribe to groups

I'm attempting to send consumers.py information to display on the client end outside of consumers.py.
I've referenced Send message using Django Channels from outside Consumer class this previous question, but the sub process .group_send or .group_add don't seem to exist, so I feel it's possible I'm missing something very easy.
Consumers.py
from channels.generic.websocket import WebsocketConsumer
from asgiref.sync import async_to_sync
class WSConsumer(WebsocketConsumer):
def connect(self):
async_to_sync(self.channel_layer.group_add)("appDel", self.channel_name)
self.accept()
self.render()
appAlarm.py
def appFunc(csvUpload):
#csvUpload=pd.read_csv(request.FILES['filename'])
csvFile = pd.DataFrame(csvUpload)
colUsernames = csvFile.usernames
print(colUsernames)
channel_layer = get_channel_layer()
for user in colUsernames:
req = r.get('https://reqres.in/api/users/2')
print(req)
t = req.json()
data = t['data']['email']
print(user + " " + data)
message = user + " " + data
async_to_sync(channel_layer.group_send)(
'appDel',
{'type': 'render', 'message': message}
)
It's throwing this error:
async_to_sync(channel_layer.group_send)(
AttributeError: 'NoneType' object has no attribute 'group_send'
and will throw the same error for group_add when stripping it back more to figure out what's going on, but per the documentation HERE I feel like this should be working.

To anyone looking at this in the future, I was not able to use redis or even memurai in Windows OS due to cost. I ended up using server side events (SSE), specifically django-eventstream, and so far it's worked great as I didn't need the client to interact with the server, for a chat application this would not work.
Eventstream creates an endpoint at /events/ the client can connect to and receive a streaming http response.
Sending data from externalFunc.py:
send_event('test', 'message', {'text': 'Hello World'})
Event listener in HTML page:
var es = new ReconnectingEventSource('/events/');
es.addEventListener('message', function (e) {
console.log(e.data);
var source = new EventSource("/events/")
var para = document.createElement("P");
const obj = JSON.parse(event.data)
para.innerText = obj.text;
document.body.appendChild(para)
}, false);
es.addEventListener('stream-reset', function (e) {
}, false);

How to limit Autobahn python subscriptions on a per session basis

I am using autobahnpython with twisted (wamp) on server side and autobahnjs in browser. Is there a straight-forward way to allow/restrict subscriptions on a per session basis? For example, a client should not be able to subscribe to topics relavant to other users.
While I am NOT using crossbar.io, I tried using the Python code shown in the 'Example' section at the end of this page http://crossbar.io/docs/Authorization/ where a RPC call is first used to give authorization to a client. Of course, I am using my own authorization logic. Once this authorization is successful, I'd like to give the client privileges to subscribe to topics related only to this client, like 'com.example.user_id'. My issue is that even if auth passes, however, I have not found a way to limit subscription requests in the ApplicationSession class which is where the authorization takes place. How can I prevent a client who authorizes with user_id=user_a from subscribing to 'com.example.user_b'?

You can authorize by creating your own router. To do that, subclass Router() and override (at a minumum) the authorize() method:
def authorize(self, session, uri, action):
return True
This method is pretty simple, if you return a True then the session is authorized to do whatever it is attempting. You could make a rule that all subscriptions must start with 'com.example.USER_ID', so, your python code would split the uri, take the third field, and compare it to the current session id, returning True if they match, false otherwise. This is where things get a little weird though. I have code that does a similar thing, here is my authorize() method:
#inlineCallbacks
def authorize(self, session, uri, action):
authid = session._authid
if authid is None:
authid = 1
log.msg("AuthorizeRouter.authorize: {} {} {} {} {}".format(authid,
session._session_id, uri, IRouter.ACTION_TO_STRING[action], action))
if authid != 1:
rv = yield self.check_permission(authid, uri, IRouter.ACTION_TO_STRING[action])
else:
rv = yield True
log.msg("AuthorizeRouter.authorize: rv is {}".format(rv))
if not uri.startswith(self.svar['topic_base']):
self.sessiondb.activity(session._session_id, uri, IRouter.ACTION_TO_STRING[action], rv)
returnValue(rv)
return
Note that I dive into the session to get the _authid, which is bad karma (I think) because I should not be looking at these private variables. I don't know where else to get it, though.
Also, of note, this goes hand in hand with Authentication. In my implementation, the _authid is the authenticated user id, which is similar to a unix user id (positive unique integer). I am pretty sure this can be anything, like a string, so you should be ok with your 'user_b' as the _auth_id if you wish.
-g

I found a relatively simple solution using a Node guest. Here's the code:
// crossbar setup
var autobahn = require('autobahn');
var connection = new autobahn.Connection({
url: 'ws://127.0.0.1:8080/ws',
realm: 'realm1'
}
);
// Websocket to Scratch setup
// pull in the required node packages and assign variables for the entities
var WebSocketServer = require('websocket').server;
var http = require('http');
var ipPort = 1234; // ip port number for Scratch to use
// this connection is a crossbar connection
connection.onopen = function (session) {
// create an http server that will be used to contain a WebSocket server
var server = http.createServer(function (request, response) {
// We are not processing any HTTP, so this is an empty function. 'server' is a wrapper for the
// WebSocketServer we are going to create below.
});
// Create an IP listener using the http server
server.listen(ipPort, function () {
console.log('Webserver created and listening on port ' + ipPort);
});
// create the WebSocket Server and associate it with the httpServer
var wsServer = new WebSocketServer({
httpServer: server
});
// WebSocket server has been activated and a 'request' message has been received from client websocket
wsServer.on('request', function (request) {
// accept a connection request from Xi4S
//myconnection is the WS connection to Scratch
myconnection = request.accept(null, request.origin); // The server is now 'online'
// Process Xi4S messages
myconnection.on('message', function (message) {
console.log('message received: ' + message.utf8Data);
session.publish('com.serial.data', [message.utf8Data]);
// Process each message type received
myconnection.on('close', function (myconnection) {
console.log('Client closed connection');
boardReset();
});
});
});
};
connection.open();

How to deliver a response to a client after a Node.js server gets a message from a AMQP queue?

When a client makes a get request to '/test' a simple string is exchanged between node.js and python via AMQP, but I don't know how to transmit the response back to the client (since the process is async).
test.py
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='task_queue', durable=True)
print ' [*] Waiting for messages. To exit press CTRL+C'
def callback(ch, method, props, body):
print " [x] Received %r" % (body,)
response = body + " MODIFIED"
#response = get_a_concept()
print " [x] Done"
ch.basic_publish(exchange='',
routing_key=props.reply_to,
properties=pika.BasicProperties(correlation_id = \
props.correlation_id),
body=str(response))
ch.basic_ack(delivery_tag = method.delivery_tag)
channel.basic_qos(prefetch_count=1)
channel.basic_consume(callback,
queue='task_queue')
channel.start_consuming()
app.js
var connection = amqp.createConnection({ host: 'localhost' });
connection.addListener('ready', function() {
var exchange = connection.exchange('', {
'type' : 'direct',
durable : false
}, function() {
var queue = connection.queue('incoming', {
durable : false,
exclusive : true }, function() {
queue.subscribe(function(msg) {
// got response here, how to transmit it to the node that made the exchange?
console.log("received message: ");
console.log(msg.data.toString());
});
});
});
});
User request makes a publish to python, but how to reply it back to the user once it's finished?
app.get('/test', loadUser, function(req, res) {
console.log("sent");
exchange.publish('task_queue', "funciona!", {
'replyTo' : 'incoming'
});
res.redirect('/home');
});
(Note: I'm not sure if this is the best implementation. Hints, suggestions welcomed!)

I solved it as follows:
The requesting side sets the reply-to and correlation-id headers when sending a message and stores the information needed to process the reply (in my case, NodeJS, a callback) in a list with the correlation-id as index.
The responding side publishes to a direct exchange with reply-to as routing key and sets the correlation-id on the message.
Now when the message arrives back at the requester, it simply gets (and removes) the needed info from the list and handles the response.
Edit: Ofcourse there's some more work to be done if you want to handle timeouts, etc, but that all depends on your usecase.

Django/gevent socket.IO with redis pubsub. Where do I put things?

I have an isolated python script that simply captures data from Twitter's streaming API and then on the receipt of each message, using redis pubsub it publishes to the channel "tweets". Here is that script:
def main():
username = "username"
password = "password"
track_list = ["apple", "microsoft", "google"]
with tweetstream.FilterStream(username, password, track=track_list) as stream:
for tweet in stream:
text = tweet["text"]
user = tweet["user"]["screen_name"]
message = {"text": text, "user": user}
db.publish("tweets", message)
if __name__ == '__main__':
try:
print "Started..."
main()
except KeyboardInterrupt:
print '\nGoodbye!'
My server side socket.io implementation is done using django-socketio (based off of gevent-socketio) https://github.com/stephenmcd/django-socketio which simply provides a few helper decorators as well as a broadcast_channel method. Because it's done in django, I've simply put this code in views.py simply so that they're imported. My views.py code:
def index(request):
return render_to_response("twitter_app/index.html", {
}, context_instance=RequestContext(request))
def _listen(socket):
db = redis.Redis(host="localhost", port=6379, db=0)
client = db.pubsub()
client.subscribe("tweets")
tweets = client.listen()
while True:
tweet = tweets.next()
tweet_data = ast.literal_eval(tweet["data"])
message = {"text": tweet_data["text"], "user": tweet_data["user"], "type": "tweet"}
socket.broadcast_channel(message)
#on_subscribe(channel="livestream")
def subscribe(request, socket, context, channel):
g = Greenlet.spawn(_listen, socket)
The client side socket.io JavaScript simply connects and subscribes to the channel "livestream" and captures any received messages to that channel:
var socket = new io.Socket();
socket.connect();
socket.on('connect', function() {
socket.subscribe("livestream");
});
socket.on('message', function(data) {
console.log(data);
});
The obvious problem with this code is that each time a new user or browser window is opened to the page, a new _listen method is spawned and the tweets get subscribed to and broadcast for each user resulting in duplicate messages being received on the client. My question is, where would the proper place be to put the _listen method so that it's only created once regardless of the # of clients? Also, keeping in mind that the broadcast_channel method is a method of a socket instance.

The problem was that I was using socket.broadcast_channel when I should have been using socket.send.