I am trying to create and node.js api to which users can subscribe to get event notifications?
I created the below API and was able to call the API using python ,however its not clear to me how can folks subscribe to it?
How can folks subscribe to this API to get notification of New root build released?what do I need to change?
node.js API
app.get("/api/root_event_notification", (req, res, next) => {
console.log(req.query.params)
var events = require('events');
var eventEmitter = new events.EventEmitter();
//Create an event handler:
var myEventHandler = function () {
console.log('new_root_announced!');
res.status(200).json({
message: "New root build released!",
posts: req.query.params
});
}
import requests
python call
input_json = {'BATS':'678910','root_version':'12A12'}
url = 'http://localhost:3000/api/root_event_notification?params=%s'%input_json
response = requests.get(url)
print response.text
OUTPUT:-
{"message":"New root build released!","posts":"{'root_version': '12A12', 'BATS': '678910'}"}
You can't just postpone sending an http response for an arbitrary amount of time. Both client and server (and sometimes the hosting provider's infrastructure) will timeout the http request after some number of minutes. There are various tricks to try to keep the http connection alive, but all have limitations.
Using web technologies, the usual options for get clients getting updated server data:
http polling (client regularly polls the server). There's also a long polling adaptation version of this that attempts to improve efficiency a bit.
Websocket. Clients makes a websocket connection to the server which is a lasting, persistent connection. Then either client or server can send data/events of this connection at any time, allowing the server to efficiently send notifications to the client at any time.
Server Sent Events (SSE). This is a newer http technology that allows one-way notification from server to client using some modified http technology.
Since a server cannot typically connect directly to a client due to firewall and public IP address issues, the usual mechanism for a server to notify a client is to use either a persistent webSocket connection from client to server over which either side can then send webSocket packets or use the newer SSE (server sent events) which allows some server events to be sent to a client over a long lasting connection.
The client can also "poll" the server repeatedly, but this is not really an event notification system (and not particularly efficient or timely) as much as it is some state that the client can check.
Related
I am quite new to MQTT and brokers, but I am having an issue with VerneMQ not sending offline messages to clients. Here is my setup. I have a backend written in Python which is using the Paho Eclipse MQTT library's single() method to send messages to a connected client. The client, a virtual machine on my development station, has a client which is written in go-lang, using paho.mqtt.golang to connect to the broker and subscribe.
The call to single() on the backend looks like this:
def send_message(device_id, payload):
token = get_jwt('my_token').decode()
mqtt.single(
f'commands/{device_id}',
payload=payload,
qos=2,
hostname=MESSAGING_HOST,
port=8080,
client_id='client_id',
auth={'username': 'username', 'password': f'Bearer {token}'},
transport='websockets'
)
On the client, the session is established with the following options:
func startListenerRun(cmd *cobra.Command, args []string) {
//mqtt.DEBUG = log.New(os.Stdout, "", 0)
mqtt.ERROR = log.New(os.Stdout, "", 0)
opts := mqtt.NewClientOptions().AddBroker(utils.GetMessagingHost()).SetClientID(utils.GetClientId())
opts.SetKeepAlive(20 * time.Second)
opts.SetDefaultPublishHandler(f)
opts.SetPingTimeout(5 * time.Second)
opts.SetCredentialsProvider(credentialsProvider)
opts.SetConnectRetry(false)
opts.SetAutoReconnect(true)
opts.willQos=2
opts.SetCleanSession(false)
I am not showing all the code, but hopefully enough to illustrate how the session is being set up.
I am running VerneMQ as a docker container. We are using the following environment variables to change configuration defaults in the Dockerfile:
ENV DOCKER_VERNEMQ_PLUGINS.vmq_diversity on
ENV DOCKER_VERNEMQ_VMQ_DIVERSITY.myscript1.file /etc/vernemq/authentication.lua
ENV DOCKER_VERNEMQ_VMQ_ACL.acl_file /etc/vernemq/vmq.acl
ENV DOCKER_VERNEMQ_PLUGINS.vmq_acl on
ENV DOCKER_VERNEMQ_RETRY_INTERVAL=3000
As long as the client has an active connection to the broker, the server's published messages arrive seamlessly. However, if I manually close the client's connection to the broker, and then publish a message on the backend to that client, when the client's connection reopens, the message is not resent by the broker. As I said, I am new to MQTT, so I may need to configure additional options, but so far I've yet to determine which. Can anyone shed any light on what might be happening on my setup that would cause offline messages to not be sent? Thanks for any information.
As thrashed out in the comments
Messages will only be queued for an offline client that has subscribed at greater than QOS 0
More details can be found here
You need to make QOS to 1 or 2 depending on your requirement and also you can use --retain flag which is quite useful. retain flag will make sure that last message will be delivered irrespective of any failure. You can know the last status of device. Check this http://www.steves-internet-guide.com/mqtt-retained-messages-example/
I'm trying to make JavaScript client to a Python websocket server through an Apache2 proxy.
The client is dead simple:
const socket = io({
transports: ['websocket']
});
I have a NodeJS websocket server and a working Apache2 reverse proxy setup.
Now I want to replace the NodeJS server with a Python server - but none of the example implementations from socket.io works. With each of the my client reports an "error 400" when setting up the websocket connection.
The Python server examples come from here:
https://github.com/miguelgrinberg/python-socketio/tree/master/examples/server
Error 400 stands for "Bad Request" - but I know that my requests are fine because my NodeJS server understands them.
When not running behind a proxy then all Python examples work fine.
What could be the problem?
I found the solution - all the Python socket.io server examples that I refered to are not configured to run behind a reverse proxy. The reason is, that the socket.io server is managing a list of allowed request origins and the automatic list creation is failing in the reverse proxy situation.
This function creates the automatic list of allowed origins (engineio/asyncio_server.py):
def _cors_allowed_origins(self, environ):
default_origins = []
if 'wsgi.url_scheme' in environ and 'HTTP_HOST' in environ:
default_origins.append('{scheme}://{host}'.format(
scheme=environ['wsgi.url_scheme'], host=environ['HTTP_HOST']))
if 'HTTP_X_FORWARDED_HOST' in environ:
scheme = environ.get(
'HTTP_X_FORWARDED_PROTO',
environ['wsgi.url_scheme']).split(',')[0].strip()
default_origins.append('{scheme}://{host}'.format(
scheme=scheme, host=environ['HTTP_X_FORWARDED_HOST'].split(
',')[0].strip()))
As you can see, it only adds URLs with {scheme} as a protocol. When behind a reverse proxy, {scheme} will always be "http". So if the initial request was HTTPS based, it will not be in the list of allowed origins.
The solution to this problem is very simple: when creating the socket.io server, you have to either tell him to allow all origins or specify your origin:
import socketio
sio = socketio.AsyncServer(cors_allowed_origins="*") # allow all
# or
sio = socketio.AsyncServer(cors_allowed_origins="https://example.com") # allow specific
I have a vb.net application that opens a socket and listens on it.
I need to communicate via this socket to that application using a javascript running on a browser. That is i need to send some data on this socket so that the app which is listening on this socket can take that data, do some stuff using some remote calls and get some more data and put it back on the socket that my javascript needs to read and print it in the browser.
Ive tried, socket.io, websockify but none have proved to be useful.
Hence the question, is what i am trying even possible? Is there a way that a javascript running in a browser can connect to a tcp socket and send some data and listen on it for some more data response on the socket and print it to the browser.
If this is possible can some one point me in the right direction as to which would help me establish the goal.
As for your problem, currently you will have to depend on XHR or websockets for this.
Currently no popular browser has implemented any such raw sockets api for javascript that lets you create and access raw sockets, but a draft for the implementation of raw sockets api in JavaScript is under-way. Have a look at these links:
http://www.w3.org/TR/raw-sockets/
https://developer.mozilla.org/en-US/docs/Web/API/TCPSocket
Chrome now has support for raw TCP and UDP sockets in its ‘experimental’ APIs. These features are only available for chrome apps and, although documented, are hidden for the moment. Having said that, some developers are already creating interesting projects using it, such as this IRC client.
To access this API, you’ll need to enable the experimental flag in your extension’s manifest. Using sockets is pretty straightforward, for example:
chrome.experimental.socket.create('tcp', '127.0.0.1', 8080, function(socketInfo) {
chrome.experimental.socket.connect(socketInfo.socketId, function (result) {
chrome.experimental.socket.write(socketInfo.socketId, "Hello, world!");
});
});
This will be possible via the navigator interface as shown below:
navigator.tcpPermission.requestPermission({remoteAddress:"127.0.0.1", remotePort:6789}).then(
() => {
// Permission was granted
// Create a new TCP client socket and connect to remote host
var mySocket = new TCPSocket("127.0.0.1", 6789);
// Send data to server
mySocket.writeable.write("Hello World").then(
() => {
// Data sent sucessfully, wait for response
console.log("Data has been sent to server");
mySocket.readable.getReader().read().then(
({ value, done }) => {
if (!done) {
// Response received, log it:
console.log("Data received from server:" + value);
}
// Close the TCP connection
mySocket.close();
}
);
},
e => console.error("Sending error: ", e)
);
}
);
More details are outlined in the w3.org tcp-udp-sockets documentation.
http://raw-sockets.sysapps.org/#interface-tcpsocket
https://www.w3.org/TR/tcp-udp-sockets/
Another alternative is to use Chrome Sockets
Creating connections
chrome.sockets.tcp.create({}, function(createInfo) {
chrome.sockets.tcp.connect(createInfo.socketId,
IP, PORT, onConnectedCallback);
});
Sending data
chrome.sockets.tcp.send(socketId, arrayBuffer, onSentCallback);
Receiving data
chrome.sockets.tcp.onReceive.addListener(function(info) {
if (info.socketId != socketId)
return;
// info.data is an arrayBuffer.
});
You can use also attempt to use HTML5 Web Sockets (Although this is not direct TCP communication):
var connection = new WebSocket('ws://IPAddress:Port');
connection.onopen = function () {
connection.send('Ping'); // Send the message 'Ping' to the server
};
http://www.html5rocks.com/en/tutorials/websockets/basics/
Your server must also be listening with a WebSocket server such as pywebsocket, alternatively you can write your own as outlined at Mozilla
ws2s project is aimed at bring socket to browser-side js. It is a websocket server which transform websocket to socket.
ws2s schematic diagram
code sample:
var socket = new WS2S("wss://ws2s.feling.io/").newSocket()
socket.onReady = () => {
socket.connect("feling.io", 80)
socket.send("GET / HTTP/1.1\r\nHost: feling.io\r\nConnection: close\r\n\r\n")
}
socket.onRecv = (data) => {
console.log('onRecv', data)
}
See jsocket. Haven't used it myself. Been more than 3 years since last update (as of 26/6/2014).
* Uses flash :(
From the documentation:
<script type='text/javascript'>
// Host we are connecting to
var host = 'localhost';
// Port we are connecting on
var port = 3000;
var socket = new jSocket();
// When the socket is added the to document
socket.onReady = function(){
socket.connect(host, port);
}
// Connection attempt finished
socket.onConnect = function(success, msg){
if(success){
// Send something to the socket
socket.write('Hello world');
}else{
alert('Connection to the server could not be estabilished: ' + msg);
}
}
socket.onData = function(data){
alert('Received from socket: '+data);
}
// Setup our socket in the div with the id="socket"
socket.setup('mySocket');
</script>
In order to achieve what you want, you would have to write two applications (in either Java or Python, for example):
Bridge app that sits on the client's machine and can deal with both TCP/IP sockets and WebSockets. It will interact with the TCP/IP socket in question.
Server-side app (such as a JSP/Servlet WAR) that can talk WebSockets. It includes at least one HTML page (including server-side processing code if need be) to be accessed by a browser.
It should work like this
The Bridge will open a WS connection to the web app (because a server can't connect to a client).
The Web app will ask the client to identify itself
The bridge client sends some ID information to the server, which stores it in order to identify the bridge.
The browser-viewable page connects to the WS server using JS.
Repeat step 3, but for the JS-based page
The JS-based page sends a command to the server, including to which bridge it must go.
The server forwards the command to the bridge.
The bridge opens a TCP/IP socket and interacts with it (sends a message, gets a response).
The Bridge sends a response to the server through the WS
The WS forwards the response to the browser-viewable page
The JS processes the response and reacts accordingly
Repeat until either client disconnects/unloads
Note 1: The above steps are a vast simplification and do not include information about error handling and keepAlive requests, in the event that either client disconnects prematurely or the server needs to inform clients that it is shutting down/restarting.
Note 2: Depending on your needs, it might be possible to merge these components into one if the TCP/IP socket server in question (to which the bridge talks) is on the same machine as the server app.
The solution you are really looking for is web sockets. However, the chromium project has developed some new technologies that are direct TCP connections TCP chromium
I building a Socket.io App.
This is the Node.Js Server Code, running in AWS instance:
var server = require('http').createServer();
var io = require('socket.io')(server);
io.on('connection', function(client){
io.sockets.emit("welcome"); //This is received by everyone
client.on('message', function(msg){
console.log("message arrived"); //This is executed
io.sockets.emit("welcome"); //This is not received by Python Client
});
});
server.listen(8090);
I have different clients, running on web page with Javascript and one Python client running in my local computer.
This is the Python client:
from socketIO_client import SocketIO
socket_url = "http://xxxxxxxxxxxxxx.eu-central- 1.compute.amazonaws.com"
socketIO = SocketIO(socket_url, 8090, verify=False)
def welcome():
print('welcome received')
socketIO.on('welcome', welcome)
socketIO.wait(seconds=1)
while True:
pass
The problem:
Python client receives the "welcome" only when the socket start, but when other clients send the "message" to the server, and it re-transmit the "welcome" to all clients, the Python client does not receive it. The others clients receive this specific "welcome", so the problem is the Python client.
I am using https://pypi.python.org/pypi/socketIO-client for the Python client. And Socket.io npm version 1.7.2 due to this issue
https://github.com/invisibleroads/socketIO-client/issues/159
I found the problem.
It was the line:
socketIO.wait(seconds=1)
SocketIO was responding only for 1 second, so everything that arrive after 1 second is ignored.
socketIO.wait()
Solve the problem
i have a simple twisted application, with the websockets module. the application is serving a lineReceiver protocol, and i'd like to inspect the contents of incoming connections, including seeing which cookies are set on them, something like this:
class Echo(basic.lineReceiver):
def connectionMade(self):
print "Got new client!"
#print request.cookies
is it possible for me to access request (client-side) cookies from a protocol that's served over websockets? if so, how do I do it?
Here is a complete example that includes a WebSocket client and a server that set/get custom HTTP headers. Adapting this you can access cookies sent by e.g. a browser during the initial WebSocket opening handshake on server-side.
Disclosure: I am original author of Autobahn and work for Tavendo.