Develop Reference

Mosquitto - unable to connect from remote computer

I am trying to test a Mosquitto MQTT server. My plan is to hopefully get it to ingest some notifications from our monitoring system in my work IT dept. Seems perfect for the task. I was able to install Mosquitto on a Ubuntu 20.04 server, and testing with the mosquitto_sub and mosquitto_pub clients worked right away (see attached screenshot). I also was able to Google around and set up some Python script using paho mqtt module for subscribe and publish. I set up the subscribe script as a service and it runs fine. I can receive published messages from both mosquitto_pub and the "publish" Python script...
Now where the issues come up is when I try publishing to the Mosquitto server from a remote computer within my office. For the life of me I cannot determine what the issue is. :(
I am able to telnet to the Mosquitto server on port 1883 so it does not seem to be a firewall issue on the Mosquitto server... Whenever I try to use the publish Python script from the remote computer, it executes but it does not work -- no messages are received at the subscriber running on the server. One odd thing is that the "on_connect" portion on the script does not work when the script is run on the remote computer; no "Connected to MQTT Broker!" message is printed when the script is run remotely. (it does work when it's run locally on the Mosquitto server).
I am attaching a screenshot of my /etc/mosquitto/conf.d/default.conf file. I have tried all sorts of combinations on the config and each time things work when run on the server itself but not remotely. I tried putting the IP address for the server on the config as "listener 1883 10.x.x.x". (Note for this post using "10.x.x.x" instead of my real IP address.) The last thing I tried was putting the adapter name, and still does not work. As you can see I have also defined authentication (usernames, passwords) for a pubuser and subuser. The account info does work with the mosquitto_sub and mosquitto_pub clients, and when used in Python scripts (when script are run on Mosquitto server) so I don't think it's an account info issue. All signs are pointing to either some misconfiguration I have in Mosquitto, or some firewall policy in my work location. Please help!
Please share any tips or possible fixes for me. I might be doing something wrong in my config; please do let me know if you see something amiss. I am a newbie with Mosquitto and MQTT but would love for it to work, kind of ruins the point if I cannot publish to the server form a remote computer though :)..
Below is example Python code I am using on the remote computer to publish (works run locally on the Mosquitto server). I redacted the IP address of the server with 10.x.x.x).
When I run the script from the remote computer I just get "Failed to send message to topic python/mqtt".
:
import random
import time
from paho.mqtt import client as mqtt_client
broker = '10.x.x.x'
port = 1883
topic = "python/mqtt"
# generate client ID with pub prefix randomly
#client_id = f'python-mqtt-{random.randint(0, 1000)}'
#Set the static client id
client_id = 'python-mqtt01'
username = 'pubuser'
password = 'something'
def connect_mqtt():
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected to MQTT Broker!")
else:
print("Failed to connect, return code %d\n", rc)
client = mqtt_client.Client(client_id)
client.username_pw_set(username, password)
client.on_connect = on_connect
client.connect(broker, port)
return client
def publish(client):
msg_count = 0
while True:
time.sleep(1)
msg = f"messages: {msg_count}"
result = client.publish(topic, msg)
# result: [0, 1]
status = result[0]
if status == 0:
print(f"Send `{msg}` to topic `{topic}`")
else:
print(f"Failed to send message to topic {topic}")
msg_count += 1
def run():
client = connect_mqtt()
client.loop_start()
publish(client)
if __name__ == '__main__':
run()
Thanks for reading! Hopefully someone out there has encountered the same issue and fixed it!
P.S. - forgot to mention I also disabled the firewall on the Ubuntu server until I get things working (ufw disable).
Example of my default.cfg (before) removing adapter and adding log_type all:
listener 1883
allow_anonymous false
bind_interface ens33

VerneMQ single publish messages lost when client is offline

I am quite new to MQTT and brokers, but I am having an issue with VerneMQ not sending offline messages to clients. Here is my setup. I have a backend written in Python which is using the Paho Eclipse MQTT library's single() method to send messages to a connected client. The client, a virtual machine on my development station, has a client which is written in go-lang, using paho.mqtt.golang to connect to the broker and subscribe.
The call to single() on the backend looks like this:
def send_message(device_id, payload):
token = get_jwt('my_token').decode()
mqtt.single(
f'commands/{device_id}',
payload=payload,
qos=2,
hostname=MESSAGING_HOST,
port=8080,
client_id='client_id',
auth={'username': 'username', 'password': f'Bearer {token}'},
transport='websockets'
)
On the client, the session is established with the following options:
func startListenerRun(cmd *cobra.Command, args []string) {
//mqtt.DEBUG = log.New(os.Stdout, "", 0)
mqtt.ERROR = log.New(os.Stdout, "", 0)
opts := mqtt.NewClientOptions().AddBroker(utils.GetMessagingHost()).SetClientID(utils.GetClientId())
opts.SetKeepAlive(20 * time.Second)
opts.SetDefaultPublishHandler(f)
opts.SetPingTimeout(5 * time.Second)
opts.SetCredentialsProvider(credentialsProvider)
opts.SetConnectRetry(false)
opts.SetAutoReconnect(true)
opts.willQos=2
opts.SetCleanSession(false)
I am not showing all the code, but hopefully enough to illustrate how the session is being set up.
I am running VerneMQ as a docker container. We are using the following environment variables to change configuration defaults in the Dockerfile:
ENV DOCKER_VERNEMQ_PLUGINS.vmq_diversity on
ENV DOCKER_VERNEMQ_VMQ_DIVERSITY.myscript1.file /etc/vernemq/authentication.lua
ENV DOCKER_VERNEMQ_VMQ_ACL.acl_file /etc/vernemq/vmq.acl
ENV DOCKER_VERNEMQ_PLUGINS.vmq_acl on
ENV DOCKER_VERNEMQ_RETRY_INTERVAL=3000
As long as the client has an active connection to the broker, the server's published messages arrive seamlessly. However, if I manually close the client's connection to the broker, and then publish a message on the backend to that client, when the client's connection reopens, the message is not resent by the broker. As I said, I am new to MQTT, so I may need to configure additional options, but so far I've yet to determine which. Can anyone shed any light on what might be happening on my setup that would cause offline messages to not be sent? Thanks for any information.

As thrashed out in the comments
Messages will only be queued for an offline client that has subscribed at greater than QOS 0
More details can be found here

You need to make QOS to 1 or 2 depending on your requirement and also you can use --retain flag which is quite useful. retain flag will make sure that last message will be delivered irrespective of any failure. You can know the last status of device. Check this http://www.steves-internet-guide.com/mqtt-retained-messages-example/

python socket.io client can't receive broadcasting messages

I building a Socket.io App.
This is the Node.Js Server Code, running in AWS instance:
var server = require('http').createServer();
var io = require('socket.io')(server);
io.on('connection', function(client){
io.sockets.emit("welcome"); //This is received by everyone
client.on('message', function(msg){
console.log("message arrived"); //This is executed
io.sockets.emit("welcome"); //This is not received by Python Client
});
});
server.listen(8090);
I have different clients, running on web page with Javascript and one Python client running in my local computer.
This is the Python client:
from socketIO_client import SocketIO
socket_url = "http://xxxxxxxxxxxxxx.eu-central- 1.compute.amazonaws.com"
socketIO = SocketIO(socket_url, 8090, verify=False)
def welcome():
print('welcome received')
socketIO.on('welcome', welcome)
socketIO.wait(seconds=1)
while True:
pass
The problem:
Python client receives the "welcome" only when the socket start, but when other clients send the "message" to the server, and it re-transmit the "welcome" to all clients, the Python client does not receive it. The others clients receive this specific "welcome", so the problem is the Python client.
I am using https://pypi.python.org/pypi/socketIO-client for the Python client. And Socket.io npm version 1.7.2 due to this issue
https://github.com/invisibleroads/socketIO-client/issues/159

I found the problem.
It was the line:
socketIO.wait(seconds=1)
SocketIO was responding only for 1 second, so everything that arrive after 1 second is ignored.
socketIO.wait()
Solve the problem

Tornado secure websocket timeout

My server got 2 Ip's (ip1 & ip2) i recently added ip2. When i try to open my tornado websocketserver on ip1 (where apache2 is running) everything is fine, i specify a port e.g. 22000 and can connect to my socket via wss://domain.tld:22000/sub
However as soon as i configured tornado to listen on ip2 (where apache is not running), because i have to use the port 443, which is blocked by apache on ip1, I can't connect to it via wss://sockets.domain.tld:443/sub. The DNS A record points to ip2.
The connection times out. No matter which port or protocol (wss / ws) i use.
My python code:
from tornado import web
from tornado import ioloop
from tornado import websocket
from tornado import httpserver
import ssl
import json
import random
import re
import os
application = web.Application([(r"/sub", Client)])
http_server = httpserver.HTTPServer(application, ssl_options = {
"certfile": os.path.join(LIB_DIR, "certificate.crt"),
"keyfile": os.path.join(LIB_DIR, "certificate.key"),
})
http_server.bind(443, address = "ip2")
print("Listening to ip2:443")
ioloop.IOLoop.current().start()
My Server is running on Ubuntu 12.2, I opened the ports and checked with an external tool if they are open.
How can i fix this? Has it something to do with my server?
UPDATE
I'm quite sure it has to do with http_server.bind(...), the code does work with .listen(port), but ip1 and bind does also not work.

According to the documentation, after the call to bind, you should call start on the server. So
http_server.bind(443, address = "ip2")
print("Listening to ip2:443")
http_server.start()
ioloop.IOLoop.current().start()
should work.

Python TLS1.2 socket server

I have an assignment to create a secure socket server using TLS version 1.1 or 1.2. I'm using python 3.4 (as that's the only version with native TLS 1.1/1.2 support). I've made a self-signed CA and signed both the client and the server. A snippet of the code is as follows:
In my server:
tls_server = ssl.wrap_socket(server, ssl_version=ssl.PROTOCOL_TLSv1_2,
cert_reqs=ssl.CERT_NONE, server_side=True,
keyfile='./server.key', certfile='./server.crt',
ca_certs='./SigningCA/signing-ca.crt')
and in the client:
tls_client = ssl.wrap_socket(client, keyfile="./client.key",
certfile="./client.crt", ssl_version=ssl.PROTOCOL_TLSv1_2,
cert_reqs=ssl.CERT_REQUIRED, ca_certs='./SigningCA/signing-ca.crt')
The connection works fine, I get a request and response. But when I print out the results of the client or server cipher() method, I get the following:
('AES256-SHA', 'TLSv1/SSLv3', 256)
which seems to indicate I'm still running TLSv1/SSLv3. Does anyone have some insight into this? Any help would be appreciated.