Develop Reference

Subscribe to Google PubSub from Azure Function App

I'm trying to figure out the best way to subscribe to a Google PubSub subscription from Azure Function App. My solution below "works" -- meaning I can start it up locally and it will pull messages that are published to the subscribed-to topic. But this can't be the right way of doing this, so I'm looking for better ideas.
I'm kicking off a timer and then listening for messages. But would a Durable app be better? The described patterns in the documentation don't fit this.
Another approach might be using the timer, but then pulling instead of listening asynchronously?
I want to log these messages and publish them to Azure ESB.
Any advice or examples would be appreciated. I'm happy to use C# or Python -- or any language for that matter.
using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
using Google.Cloud.PubSub.V1;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace NetEaiDemo
{
public class Function1
{
[FunctionName("Function1")]
public async Task Run([TimerTrigger("*/30 * * * * *")]TimerInfo myTimer, ILogger log)
{
log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
string projectId = "project1";
string subscriptionId = "subscription1";
SubscriptionName subscriptionName = new SubscriptionName(projectId, subscriptionId);
SubscriberClient subscriber = await SubscriberClient.CreateAsync(subscriptionName);
List<PubsubMessage> receivedMessages = new List<PubsubMessage>();
// Start the subscriber listening for messages.
await subscriber.StartAsync((msg, cancellationToken) =>
{
receivedMessages.Add(msg);
Console.WriteLine($"Received message {msg.MessageId} published at {msg.PublishTime.ToDateTime()}");
Console.WriteLine($"Text: '{msg.Data.ToStringUtf8()}'");
// Stop this subscriber after one message is received.
// This is non-blocking, and the returned Task may be awaited.
subscriber.StopAsync(TimeSpan.FromSeconds(15));
// Return Reply.Ack to indicate this message has been handled.
return Task.FromResult(SubscriberClient.Reply.Ack);
});
}
}
}

Flask SocketIO duplicating saved messages

I am building a chat app based on Flask SocketIO only, no database. One of the requirements is when a new user connects, the app should show previous messages. I save every message into a array on each send event. Okay, so now the problem is, when User A connects and creates some messages, and then User B connects, all the previous messages from User B is shown, but then User A also gets that messages, so User A ends up duplicated messages.
Client Side JS
function myFunction() {
document.getElementById('demo').style['text-decoration']='underline';
}
const socket = io.connect("http://127.0.0.1:5000");
socket.on('connect', function() {
socket.emit('sync_messages');
socket.emit('sync_channels');
});
Flask App Code
#socketio.on('sync_messages')
def handle_sync():
socketio.emit('show_all_messages', messages)
#socketio.on('sync_channels')
def handle_sync_channels():
socketio.emit('show_all_channels', channels)
Visual representation of what is happening
what the actual bug is

The socketio.emit() function is not context aware, it defaults to broadcasting to all connected users. Try using emit() which is the Flask-friendly wrapper:
#socketio.on('sync_messages')
def handle_sync():
emit('show_all_messages', messages)
#socketio.on('sync_channels')
def handle_sync_channels():
emit('show_all_channels', channels)

Python Proton sending binary data to Active MQ

I am trying to write a simple string message to an ActiveMQ queue:
def write_to_amq(message, host_name, port, queue):
conn = BlockingConnection(f'{host_name}:{port}')
sender = conn.create_sender(queue)
sender.send(Message(body='message'))
conn.close()
The message gets to the queue just fine, but it appears to have some binary data in it when I view it on the ActiveMQ web UI. It reports the contents as SpESsESw�message. I was expecting the contents to just be message
[Additional data point]
I am also seeing this in a separate Go program I have written using the pack.ag/amqp package.
func (s *amqpSender) SendResult(data string) error {
session, err := s.client.NewSession()
if err != nil {
return fmt.Errorf("failure creating AMQP session: %s", err)
}
ctx := context.Background()
sender, err := session.NewSender(
amqp.LinkTargetAddress(s.workQueueName),
)
if err != nil {
return fmt.Errorf("failure creating sender link: %s", err)
}
ctx, cancel := context.WithTimeout(ctx, s.timeout)
defer func() {
cancel()
sender.Close(ctx)
}()
err = sender.Send(ctx, amqp.NewMessage([]byte(data)))
if err != nil {
return fmt.Errorf("failure sending message: %s", err)
}
return nil
}
When I send a different message to ActiveMQ, I get similar behavior, seeing Su�vMy message in the ActiveMQ Message Details. Could this just be a web UI anomaly?

It is not a web anomaly, if you receive the message under the openwire protocol, you'll see the same thing as you see on the webpage. It seems that ActiveMQ encodes the properties inside the message payload so there are those weird characters at the start. My hypothesis is that ActiveMQ encodes these properties you see at the top right of the webpage at the start of the body:
Furthermore, if you were to send a message with text-based protocols like OpenWire or STOMP, you won't see any properties nor weird bytes at the start of the body.
There are 3 potential solutions to this issue:
If you want to keep using ActiveMQ
As said here, you can add
<transportConnectors>
<transportConnector name="amqp" uri="amqp://localhost:5672?transport.transformer=jms"/>
</transportConnectors>
inside the ActiveMQ configuration so AMQP messages are transformed into JMS TextMessages. However, if you're using the managed ActiveMQ service of AWS, Amazon MQ, this configuration is currently not available. So please open a ticket so they prioritize this.
Use a text-based protocol instead of AMQP such as OpenWire or STOMP
If you don't mind using another broker then ActiveMQ
Consider switching to RabbitMQ where AMQP is a first-class citizen.
I know the question is over a year old, but I hope this will still be helpful!

You need to tell the python binding that you want to encode the body as a String value by adding the unicode encoding prefix to you string so that the python binding knows what to do with the data you are encoding. The way you are currently handing off the body is resulting in a binary encoding instead and so the broker will show you the garbage data on the console as it views this as a BytesMessage instead of a TextMessage
It should work if you did something like the following:
sender.send(Message(body=u"message"))
More on python string encoding and decoding here.

Is there a way to broadcast a message to all (or filtered) WebSocket clients connected to a WebSocket server? [duplicate]

I'm assuming this isn't possible, but wanted to ask in case it is. If I want to provide a status information web page, I want to use WebSockets to push the data from the server to the browser. But my concerns are the effect a large number of browsers will have on the server. Can I broadcast to all clients rather than send discrete messages to each client?

WebSockets uses TCP, which is point to point, and provides no broadcast support.

Not sure how is your client/server setup, but you can always just keep in the server a collection of all connected clients - and then iterate over each one and send the message.
A simple example using Node's Websocket library:
Server code
var WebSocketServer = require('websocket').server;
var clients = [];
var socket = new WebSocketServer({
httpServer: server,
autoAcceptConnections: false
});
socket.on('request', function(request) {
var connection = request.accept('any-protocol', request.origin);
clients.push(connection);
connection.on('message', function(message) {
//broadcast the message to all the clients
clients.forEach(function(client) {
client.send(message.utf8Data);
});
});
});

As noted in other answers, WebSockets don't support multicast, but it looks like the 'ws' module maintains a list of connected clients for you, so it's pretty easy to iterate through them. From the docs:
const WebSocketServer = require('ws').Server;
const wss = new WebSocketServer({ port: 8080 });
wss.broadcast = function(data) {
wss.clients.forEach(client => client.send(data));
};

Yes, it is possible to broadcast messages to multiple clients.
In Java,
#OnMessage
public void onMessage(String m, Session s) throws IOException {
for (Session session : s.getOpenSessions()) {
session.getBasicRemote().sendText(m);
}
}
and here it is explained.
https://blogs.oracle.com/PavelBucek/entry/optimized_websocket_broadcast.

It depends on the server-side really. Here's an example of how it's done using Tomcat7:
Tomcat 7 Chat Websockets Servlet Example
and an explanation of the how it's constructed here.

Yes you can and there are many socket servers out there written in various scripting languages that are doing it.

The Microsoft.Web.WebSockets namespace has a WebSocketCollection with Broadcast capability. Look for the assembly in Nuget. The name is Microsoft.WebSockets.

Channel API python can't understand one part

I am trying to build a chat application using channel API in Google App Engine.
When going through http://developers.google.com/appengine/docs/python/channel/overview
I could not understand the opening the socket part using the XMLHttpRequest();
Any help will be appreciated!!
Thanks

Basically, when the socket is opened, the code below updates the user interface of an example Tic Tac Toe game, and sends a POST message to the server asking for the latest game state.
The code is not Python, it's client-side Javascript. I have commented it below:
sendMessage = function(path, opt_param) {
/* path variable is part of a URL being maintained */
path += '?g=' + state.game_key; /* Saving game key in URL */
if (opt_param) {
path += '&' + opt_param; /* Adding optional parameters to the path */
}
var xhr = new XMLHttpRequest(); /* Used for Ajax in Javascript */
xhr.open('POST', path, true); /* Asynchronously POST, via HTTP, the path */
xhr.send(); /* Start the POST above */
};
onOpened = function() {
connected = true; /* Set boolean value, which lets us know we're connected */
sendMessage('opened'); /* We can now send messages to the server */
updateBoard(); /* Update user interface to reflect that socket is open */
};
Note that the application defines sendMessage() as a wrapper around XmlHttpRequest, which the client uses to send messages to the server.

The socket is "opened" when the channel between client and server is established. At that point the OnOpened callback is called. The callback does a POST request back to the server to get the current state of the game. So the XMLHttpRequest is not related to the opening of the socket but rather is just a common coding pattern to have a two way communication since the channel is just one way (server to client). The other route (client to server) is done through these HTTP requests. Almost every time you get a message in the channel from the server you will want to send something back to the server (a response, an update, etc.).
Hope this helps.