I am playing with a MQTT docker-compose env and trying to use a username and password in the subscriber. Without the password both publish and subscribe work correctly.
My docker-compose.yml looks like this:
version: "3"
services:
mqtt:
image: eclipse-mosquitto:latest
container_name: mqtt
network_mode: bridge
ports:
- 1883:1883
volumes:
- ./conf:/mosquitto/config
- ./log:/mosquitto/log
My mosquitto.conf in /mosquitto/config/ on the container looks like this:
persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log
listener 1883
allow_anonymous false
password_file /mosquitto/config/passwd.txt
My password file is in /mosquitto/config/passwd.txt and has a line with username and a clear password. I shelled into the container to the location of this file and did this
mosquitto_passwd -U /mosquitto/config/passwd.txt
which encrypts the file in place. When I look at the passwd.txt it has an entry with encrypted text:
user:$7$101$vOlxOuwvzHmJiYWC$N8MKHb4fczMZNPzCBfXK4k7mUbOp+PzwT2Yb4IeU1KMKABP8hOsvDjqe+DcK7q6ksVuGmdODfWjrjQNAfJMjZw==
When I run the publish it works fine. Along these lines:
import paho.mqtt.client as mqtt
import time
def on_publish(client,userdata,result):
print("[PUB] Data published = %s" % result)
mqtt_broker ="localhost"
client = mqtt.Client("test")
client.on_publish = on_publish
client.connect(mqtt_broker, 1883)
for i in range(10):
temp = i
client.publish("Temp", temp)
print("[PUB] Just published %.4f to topic Temp" % temp)
time.sleep(1)
The subscribe however never gets it:
import paho.mqtt.client as mqtt
username='user'
password='passwd'
def on_message(client, userdata, message):
print("[SUB] received message: %s" % message.payload.decode("utf-8"))
def on_connect(client, userdata, flags, rc):
print("[SUB] Connected with result code "+str(rc))
client.subscribe("/Temp")
mqtt_broker = "localhost"
client = mqtt.Client("sub")
ttl = 120
client.username_pw_set(username, password)
client.connect(mqtt_broker, 1883, ttl)
client.loop_start()
client.subscribe("Temp")
print("[SUB] Awaiting notifications")
client.loop_forever()
In the logs I see not authorised errors:
1670429277: New connection from 172.17.0.1:59538 on port 1883.
1670429277: Client mac disconnected, not authorised.
What could be the issue?
As per the comments the issue in this case was that your "publish" did not set the username and password (but you do in your subscribe code). It's worth using a known good program (e.g. mosquitto_sub/mosquitto_pub) when testing (otherwise you might have bugs in both your subscribe and publish code which makes debugging confusing!). I'll leave the rest of my answer in case it helps anyone else.
I have been able to run your code successfully after making a few tweaks. This may not be a full answer because I have not been able to replicate your issue, but can point you towards code that works.
The main issue I can see in your code is that you are not configuring client to use your callbacks. e.g.:
client.on_message = on_message
client.on_connect = on_connect
This means that your program will not subscribe (and would do nothing with any messages received).
In addition to the above you are using both the threaded and blocking loop functions. This will cause issues because you end up running multiple instances of the network loop. I fixed these issues and made a few other small changes:
import paho.mqtt.client as mqtt
import time
username='user'
password='passwd'
def on_message(client, userdata, message):
print("[SUB] received message: %s" % message.payload.decode("utf-8"))
def on_connect(client, userdata, flags, rc):
print("[SUB] Connected with result code "+str(rc))
client.subscribe("/Temp")
time.sleep(5)# Allow time for mosquitto to startup
mqtt_broker = "mqtt"
client = mqtt.Client("sub")
client.on_message = on_message
client.on_connect = on_connect
ttl = 120
client.username_pw_set(username, password)
client.connect(mqtt_broker, 1883, ttl)
#client.loop_start() # Pick either threaded or non-threaded, not both!
client.subscribe("Temp")
print("[SUB] Awaiting notifications")
client.loop_forever()
I ran your code and mosquitto using docker - config follows (happy to add Dockerfile etc if needed):
version: '3.4'
services:
mqtt:
image: eclipse-mosquitto:latest
ports:
- "1883:1883"
volumes:
- ./conf:/mosquitto/config
- ./log:/mosquitto/log
pythondocker:
image: pythondocker
build:
context: .
dockerfile: ./Dockerfile
With this running I used mosquitto_pub.exe -h 127.0.0.1 -u user -P passwd -t Temp -m "foo" on the host to send a message which was successfully received.
Attaching to pythondocker-mqtt-1, pythondocker-pythondocker-1
pythondocker-pythondocker-1 | [SUB] Awaiting notifications
pythondocker-pythondocker-1 | [SUB] Connected with result code 0
pythondocker-pythondocker-1 | [SUB] received message: foo
If I modify the python, changing the pasword to passwd2 then I do get the error you were seeing (not sure why you are getting this - I am using the passwd.txt you provided):
1670453773: New connection from 172.26.0.2:40037 on port 1883.
1670453773: Client sub disconnected, not authorised.
Related
i've been trying to make this example running for many hours. I was building an example so my friend can learn some python but i've end up frustrated on my own.
My python knowledge is quite limited. Something is causing the program thread to finish no matter how much I try delaying the execution with time.sleep (i've removed that part of the code).
Expect result: sender container should be started after the receiver one. So the receiver is subscribed to the broker and waiting for messages.
Given result: receiver container starts and then dies.
Thanks in advance.
I have a docker compose as follows:
services:
mqtt_broker:
image: eclipse-mosquitto
volumes:
- "./mosquitto.conf:/mosquitto/config/mosquitto.conf"
client_send:
build:
context: ./client_send/
environment:
BROKER_HOST: mqtt_broker
depends_on:
- client_receive
client_receive:
build:
context: ./client_receive/
environment:
BROKER_HOST: mqtt_broker
depends_on:
- mqtt_broker
Then I have client code for each of these clients:
Receiver:
import os
import paho.mqtt.client as mqtt
def on_connect(client, userdata, flags, rc):
print("[receiver] Connected with result code " + str(rc))
client.subscribe("sample_topic")
def on_message(client, userdata, msg):
print("[receiver] got a message: " + str(msg.payload.decode()))
client.loop_stop()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(os.environ["BROKER_HOST"], 1883, 60)
client.loop_start()
Sender:
import os
import paho.mqtt.client as mqtt
def run():
print("[sender] will send a message")
client.publish("sample_topic", "message from sender")
client.loop_stop()
def on_connect(client, userdata, flags, rc):
print("[sender] Connected with result code " + str(rc))
run()
client = mqtt.Client()
client.on_connect = on_connect
client.connect(os.environ["BROKER_HOST"], 1883, 60)
client.loop_start()
I was using loop_forever without too much success bcoz print() calls were not logging anything since the main thread was blocked so I couldn't see if my code was working.
EDIT: previous paragraph is just not correct. loop_forever will work taking this into account: Python app does not print anything when running detached in docker
Finally got it working as suggested by #Brits (see comments) just by running exit or disconnecting the client (works using exit too)
I also keep the depends_on so the docker-compose.yml was not changed
This is the receiver:
import os
import paho.mqtt.client as mqtt
def on_connect(client, userdata, flags, rc):
print("[receiver] Connected with result code " + str(rc))
client.subscribe("sample_topic")
def on_message(client, userdata, msg):
print("[receiver] got a message: " + str(msg.payload.decode()))
client.disconnect()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(os.environ["BROKER_HOST"], 1883, 60)
client.loop_forever()
As a side note, if the main thread is blocked you will never be able to see the output of the program even if you got the message back from the sender. If you don't disconnect the client it might actually work if your application does not rely on console output. Soo freeing the main thread allows for the system to release the output logs to docker.
My client is unable to receive message payload from the MQTT server. I had set mosquitto on my PC to act as both the server and publisher and execute the following command:
mosquitto_pub -h 192.168.1.2 -t topic/test -m "testing" -i simulator -d
Subscribing myself works and payload is received.
However, my client which is using paho mqtt only received successful connection without the payload message. So i have the following code here:
def on_connect(client, userdata, flags, rc):
print("connection ok")
client.subscribe("topic/test")
def on_message(client, userdata, msg):
print("Message received! " + str(msg.payload))
if msg.payload == "testing":
print("Message received")
# Do something
...
def main():
#MQTT
broker = "192.168.1.2" #broker ip
client = mqtt.Client("simulator")
client.on_connect = on_connect
client.on_message = on_message
client.connect(broker, 1883, 60)
client.loop_start()
while(1):
...
So it did managed to print "connection ok", but not "Messaged received! ..."
You are using the same client-id (simulator) with mosquitto_pub and in your python. As per the spec:
If the ClientId represents a Client already connected to the Server then the Server MUST disconnect the existing Client
So what is happening is:
Python client connects and subscribes.
mosquitto_pub connects, broker disconnects python client.
mosquitto_pub publishes the message (but as nothing is subscribed the QOS 0 message will be dropped by the broker).
Python client reconnects and subscribes again.
Try using a different client-id with mosquitto_pub (your code works fine when I test it in that way - added a time.sleep(1) in the while(1)).
I am writing a django application which should act as MQTT publisher and as a subscriber.
Where should I start the paho client and run loop_forever() function.
Should it be in wsgi.py ?
Update:
If you need Django running in multiple threads then to publish messages from your Django app you can use helper functions from Publish module of Paho - https://eclipse.org/paho/clients/python/docs/#id17
You don't need to create an instance of mqtt client and start a loop in this case. And to subscribe to some topic consider running mqtt client as a standalone script and import there needed modules of your Django app (and don't forget to setup the Django environment in the script).
The answer below is good only if you run Django in a single thread, which is not usual in production.
Create mqtt.py in your application folder and put all related code there. For example:
import paho.mqtt.client as mqtt
def on_connect(client, userdata, rc):
client.subscribe("$SYS/#")
def on_message(client, userdata, msg):
# Do something
pass
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("iot.eclipse.org", 1883, 60)
Don't call loop_forever() here!
Then in your application __init__.py call loop_start():
from . import mqtt
mqtt.client.loop_start()
Using loop_start() instead of loop_forever() will give you not blocking background thread.
If you are using ASGI in your Django application you can use MQTTAsgi.
It's a complete protocol server for Django and MQTT.
Edit: Full disclosure I'm the author of MQTTAsgi.
Edit 2: To utilize the mqtt protocol server you can run your application, first you need to create a MQTT consumer:
from mqttasgi.consumers import MqttConsumer
class MyMqttConsumer(MqttConsumer):
async def connect(self):
await self.subscribe('my/testing/topic', 2)
async def receive(self, mqtt_message):
print('Received a message at topic:', mqtt_mesage['topic'])
print('With payload', mqtt_message['payload'])
print('And QOS:', mqtt_message['qos'])
pass
async def disconnect(self):
await self.unsubscribe('my/testing/topic')
Then you should add this protocol to the protocol router:
application = ProtocolTypeRouter({
'websocket': AllowedHostsOriginValidator(URLRouter([
url('.*', WebsocketConsumer)
])),
'mqtt': MyMqttConsumer,
....
})
Then you can run the mqtt protocol server with*:
mqttasgi -H localhost -p 1883 my_application.asgi:application
*Assuming the broker is in localhost and port 1883.
Thanks for the comments!
I'm using the following code from MQTT Paho project to subscribe to messages from my mqtt broker. I tested the connection using mosquitto_sub and I receive the messages there. However, when I run the following code it doesn't receive any messages and no output is printed. I checked the topic and host.
import paho.mqtt.client as mqtt
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("test")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 1883, 60)
client.loop_forever()
The following error is logged by the broker:
Invalid protocol "MQTT" in CONNECT from ::1.
Socket read error on client (null), disconnecting.
EDIT Thanks to #hardillb for pointing out the outdated MQTT version.
Everything worked after I did the following:
sudo apt-get purge mosquitto
sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa
sudo apt-get update
sudo apt-get install mosquitto
This is most likely because you are using an old version of mosquitto and the python is expecting a newer build that supports MQTT 3.1.1
Try changing the code as shown:
import paho.mqtt.client as mqtt
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("test")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
# Change made HERE
client = mqtt.Client(protocol=MQTTv31)
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 1883, 60)
client.loop_forever()
You should also upgrade your broker as soon as possible, that version is incredibly out of date and has a number of known issues, the current version is 1.4.10
I have a remote server running mosquitto. I can connect to this server and exchange messages using mosquitto_pub and mosquitto_sub. If i try the same using some python with paho.mqtt.client I get no connection. My script keeps running but the on_connection hook is never called.. The same scripts however work flawlessly with my local mosquitto server.
What could possibly be the reason for the connection problems? How can I get some more feedback on what's happening? Any suggestions?
EDIT: I added a minimal code example
import paho.mqtt.client as mqtt
def on_connect(client, userdata, flags, rc):
print("Yeeha")
client.subscribe("botgrid/init", qos=2)
def on_message(client, userdata, msg):
print(msg.payload)
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost")
print("Waiting for connection...")
client.loop_forever()
EDIT 2: While playing around, I noted that replacing "localhost" with "test.mosquitto.org" resulted in OSError: [Errno 101] Network is unreachable although I have no problems connecting to it via mosquitto_sub
Does this code produce the same problem? This is probably the equivalent to the code at the point that it is failing.
import socket
sock = socket.create_connection(("test.mosquitto.org", 1883))