i've been trying to make this example running for many hours. I was building an example so my friend can learn some python but i've end up frustrated on my own.
My python knowledge is quite limited. Something is causing the program thread to finish no matter how much I try delaying the execution with time.sleep (i've removed that part of the code).
Expect result: sender container should be started after the receiver one. So the receiver is subscribed to the broker and waiting for messages.
Given result: receiver container starts and then dies.
Thanks in advance.
I have a docker compose as follows:
services:
mqtt_broker:
image: eclipse-mosquitto
volumes:
- "./mosquitto.conf:/mosquitto/config/mosquitto.conf"
client_send:
build:
context: ./client_send/
environment:
BROKER_HOST: mqtt_broker
depends_on:
- client_receive
client_receive:
build:
context: ./client_receive/
environment:
BROKER_HOST: mqtt_broker
depends_on:
- mqtt_broker
Then I have client code for each of these clients:
Receiver:
import os
import paho.mqtt.client as mqtt
def on_connect(client, userdata, flags, rc):
print("[receiver] Connected with result code " + str(rc))
client.subscribe("sample_topic")
def on_message(client, userdata, msg):
print("[receiver] got a message: " + str(msg.payload.decode()))
client.loop_stop()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(os.environ["BROKER_HOST"], 1883, 60)
client.loop_start()
Sender:
import os
import paho.mqtt.client as mqtt
def run():
print("[sender] will send a message")
client.publish("sample_topic", "message from sender")
client.loop_stop()
def on_connect(client, userdata, flags, rc):
print("[sender] Connected with result code " + str(rc))
run()
client = mqtt.Client()
client.on_connect = on_connect
client.connect(os.environ["BROKER_HOST"], 1883, 60)
client.loop_start()
I was using loop_forever without too much success bcoz print() calls were not logging anything since the main thread was blocked so I couldn't see if my code was working.
EDIT: previous paragraph is just not correct. loop_forever will work taking this into account: Python app does not print anything when running detached in docker
Finally got it working as suggested by #Brits (see comments) just by running exit or disconnecting the client (works using exit too)
I also keep the depends_on so the docker-compose.yml was not changed
This is the receiver:
import os
import paho.mqtt.client as mqtt
def on_connect(client, userdata, flags, rc):
print("[receiver] Connected with result code " + str(rc))
client.subscribe("sample_topic")
def on_message(client, userdata, msg):
print("[receiver] got a message: " + str(msg.payload.decode()))
client.disconnect()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(os.environ["BROKER_HOST"], 1883, 60)
client.loop_forever()
As a side note, if the main thread is blocked you will never be able to see the output of the program even if you got the message back from the sender. If you don't disconnect the client it might actually work if your application does not rely on console output. Soo freeing the main thread allows for the system to release the output logs to docker.
Related
I am playing with a MQTT docker-compose env and trying to use a username and password in the subscriber. Without the password both publish and subscribe work correctly.
My docker-compose.yml looks like this:
version: "3"
services:
mqtt:
image: eclipse-mosquitto:latest
container_name: mqtt
network_mode: bridge
ports:
- 1883:1883
volumes:
- ./conf:/mosquitto/config
- ./log:/mosquitto/log
My mosquitto.conf in /mosquitto/config/ on the container looks like this:
persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log
listener 1883
allow_anonymous false
password_file /mosquitto/config/passwd.txt
My password file is in /mosquitto/config/passwd.txt and has a line with username and a clear password. I shelled into the container to the location of this file and did this
mosquitto_passwd -U /mosquitto/config/passwd.txt
which encrypts the file in place. When I look at the passwd.txt it has an entry with encrypted text:
user:$7$101$vOlxOuwvzHmJiYWC$N8MKHb4fczMZNPzCBfXK4k7mUbOp+PzwT2Yb4IeU1KMKABP8hOsvDjqe+DcK7q6ksVuGmdODfWjrjQNAfJMjZw==
When I run the publish it works fine. Along these lines:
import paho.mqtt.client as mqtt
import time
def on_publish(client,userdata,result):
print("[PUB] Data published = %s" % result)
mqtt_broker ="localhost"
client = mqtt.Client("test")
client.on_publish = on_publish
client.connect(mqtt_broker, 1883)
for i in range(10):
temp = i
client.publish("Temp", temp)
print("[PUB] Just published %.4f to topic Temp" % temp)
time.sleep(1)
The subscribe however never gets it:
import paho.mqtt.client as mqtt
username='user'
password='passwd'
def on_message(client, userdata, message):
print("[SUB] received message: %s" % message.payload.decode("utf-8"))
def on_connect(client, userdata, flags, rc):
print("[SUB] Connected with result code "+str(rc))
client.subscribe("/Temp")
mqtt_broker = "localhost"
client = mqtt.Client("sub")
ttl = 120
client.username_pw_set(username, password)
client.connect(mqtt_broker, 1883, ttl)
client.loop_start()
client.subscribe("Temp")
print("[SUB] Awaiting notifications")
client.loop_forever()
In the logs I see not authorised errors:
1670429277: New connection from 172.17.0.1:59538 on port 1883.
1670429277: Client mac disconnected, not authorised.
What could be the issue?
As per the comments the issue in this case was that your "publish" did not set the username and password (but you do in your subscribe code). It's worth using a known good program (e.g. mosquitto_sub/mosquitto_pub) when testing (otherwise you might have bugs in both your subscribe and publish code which makes debugging confusing!). I'll leave the rest of my answer in case it helps anyone else.
I have been able to run your code successfully after making a few tweaks. This may not be a full answer because I have not been able to replicate your issue, but can point you towards code that works.
The main issue I can see in your code is that you are not configuring client to use your callbacks. e.g.:
client.on_message = on_message
client.on_connect = on_connect
This means that your program will not subscribe (and would do nothing with any messages received).
In addition to the above you are using both the threaded and blocking loop functions. This will cause issues because you end up running multiple instances of the network loop. I fixed these issues and made a few other small changes:
import paho.mqtt.client as mqtt
import time
username='user'
password='passwd'
def on_message(client, userdata, message):
print("[SUB] received message: %s" % message.payload.decode("utf-8"))
def on_connect(client, userdata, flags, rc):
print("[SUB] Connected with result code "+str(rc))
client.subscribe("/Temp")
time.sleep(5)# Allow time for mosquitto to startup
mqtt_broker = "mqtt"
client = mqtt.Client("sub")
client.on_message = on_message
client.on_connect = on_connect
ttl = 120
client.username_pw_set(username, password)
client.connect(mqtt_broker, 1883, ttl)
#client.loop_start() # Pick either threaded or non-threaded, not both!
client.subscribe("Temp")
print("[SUB] Awaiting notifications")
client.loop_forever()
I ran your code and mosquitto using docker - config follows (happy to add Dockerfile etc if needed):
version: '3.4'
services:
mqtt:
image: eclipse-mosquitto:latest
ports:
- "1883:1883"
volumes:
- ./conf:/mosquitto/config
- ./log:/mosquitto/log
pythondocker:
image: pythondocker
build:
context: .
dockerfile: ./Dockerfile
With this running I used mosquitto_pub.exe -h 127.0.0.1 -u user -P passwd -t Temp -m "foo" on the host to send a message which was successfully received.
Attaching to pythondocker-mqtt-1, pythondocker-pythondocker-1
pythondocker-pythondocker-1 | [SUB] Awaiting notifications
pythondocker-pythondocker-1 | [SUB] Connected with result code 0
pythondocker-pythondocker-1 | [SUB] received message: foo
If I modify the python, changing the pasword to passwd2 then I do get the error you were seeing (not sure why you are getting this - I am using the passwd.txt you provided):
1670453773: New connection from 172.26.0.2:40037 on port 1883.
1670453773: Client sub disconnected, not authorised.
I am new to using MQTT and I would like to run two Python programs simultaneously: one to publish messages to a topic and the other to print out the messages as they are published to the topic. However, the callback function to print out the message isn't working for me.
These are the two programs I have:
Publish messages:
import paho.mqtt.client as mqtt
def on_connect(client, userdata, flags, rc):
print("Connected! Result code: " + str(rc))
client = mqtt.Client(client_id="clientID")
client.on_connect = on_connect
client.username_pw_set(username="username", password="password")
client.loop_start()
client.connect("node02.myqtthub.com", 1883)
while True:
client.publish("my/topic", "test")
Subscribe to and print messages:
import paho.mqtt.client as mqtt
def on_message(client, userdata, msg):
print(msg.payload.decode())
def on_connect(client, userdata, flags, rc):
print("Connected! Result code: " + str(rc))
client.subscribe("my/topic")
client = mqtt.Client(client_id="clientID")
client.on_connect = on_connect
client.on_message = on_message
client.username_pw_set(username="username", password="password")
client.connect("node02.myqtthub.com", 1883)
client.loop_forever()
Could someone help me fix this issue?
Both programs are using client_id="clientID"; the mqtt spec states
If the ClientId represents a Client already connected to the Server then the Server MUST disconnect the existing Client [MQTT-3.1.4-2].
So whichever app connects second will cause the first to be disconnected. To fix this change the client id in one of the apps.
Note: This is a comment on something obviously wrong. When asking a question please state what you see happening (any output, errors, what you have tried etc) because there may be another problem.
I have written a code that can do both publishing and subscribing in the same python file. My aim is to be able to send endlessly color temperature to a lamp and receive back its state (ON/OFF or online/offline).
However, when I run the code, I can send messages to the lamp but I don't receive anything (note that publisher and subscriber worked well separately)
import paho.mqtt.client as mqtt
import sys
import time
topic = "testtopic/3"
topic_casa = "testtopic/1"
def on_connect(client, userdata, flags, rc):
print("Connection to the broker. Result : "+str(rc))
client.subscribe(topic_casa, qos=1)
client.publish(topic, light.circadianLight())
def on_message(client, userdata, msg):
Message = str(msg.payload)
print("[MSG RECEBIDA] Topico: "+msg.topic+" / Mensagem: "+Message)
def on_publish(client,userdata,result):
client.publish(topic, light.circadianLight())
time.sleep(10)
def on_disconnect(client, userdata, rc):
if rc != 0:
print("Unexpected disconnection.")
try:
print("[STATUS] Inicializando MQTT...")
#inicializa MQTT:
client = mqtt.Client()
client.username_pw_set(username, password)
client.on_connect = on_connect
client.on_message = on_message
client.on_disconnect = on_disconnect
client.on_publish = on_publish #with this callback we won't be able to
#receive messages
client.connect(Broker, PortaBroker, KeepAliveBroker)
client.loop_forever()
except KeyboardInterrupt:
print ("\nCtrl+C pressionado, encerrando aplicacao e saindo...")
sys.exit(0)
The problem is two fold, first that you are calling time.sleep(10) in the on_publish() callback.
This is blocking the MQTT client network loop so it will not send/receive any messages for 10 seconds every time you publish a message.
You should not be doing any blocking or long running tasks in any of the MQTT client callbacks.
The other problem is that without the sleep the client will spend all it's time publishing messages (because everytime it finishes publishing a message it trigger the on_publish() callback which kicks off another publish. It will never get round to handing the incoming messages.
If you want to publish messages at a regular interval then you need to start a separate thread to do it, not try to do it in the callbacks.
I'm using the following code from MQTT Paho project to subscribe to messages from my mqtt broker. I tested the connection using mosquitto_sub and I receive the messages there. However, when I run the following code it doesn't receive any messages and no output is printed. I checked the topic and host.
import paho.mqtt.client as mqtt
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("test")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 1883, 60)
client.loop_forever()
The following error is logged by the broker:
Invalid protocol "MQTT" in CONNECT from ::1.
Socket read error on client (null), disconnecting.
EDIT Thanks to #hardillb for pointing out the outdated MQTT version.
Everything worked after I did the following:
sudo apt-get purge mosquitto
sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa
sudo apt-get update
sudo apt-get install mosquitto
This is most likely because you are using an old version of mosquitto and the python is expecting a newer build that supports MQTT 3.1.1
Try changing the code as shown:
import paho.mqtt.client as mqtt
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe("test")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print(msg.topic+" "+str(msg.payload))
# Change made HERE
client = mqtt.Client(protocol=MQTTv31)
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 1883, 60)
client.loop_forever()
You should also upgrade your broker as soon as possible, that version is incredibly out of date and has a number of known issues, the current version is 1.4.10
I'm trying to send a jpg image using MQTT mosquitto broker(pub and sub) in Raspberry Pi2.
This is my python code pub.py(modified)
import paho.mqtt.client as mqtt
def on_publish(mosq, userdata, mid):
mosq.disconnect()
client = mqtt.Client()
client.connect("test.mosquitto.org", 1883, 60)
client.on_publish = on_publish
f=open("b.jpg", "rb") #3.7kiB in same folder
fileContent = f.read()
byteArr = bytearray(fileContent)
client.publish("image",byteArr,0)
client.loop_forever()
and it's sub.py(modified)
import paho.mqtt.client as mqtt
def on_connect(client, userdata, rc):
print("Connect" + str(rc))
client.subscribe("image")
def on_message(client, userdata, msg):
print "Topic : ", msg.topic
f = open("/tmp/output.jpg", "w") #there is a output.jpg which is different
f.write(msg.payload)
f.close()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("test.mosquitto.org", 1883, 60)
client.loop_forever()
My python verson is 2.7.9.
After I solved some error, It seems to work but It doesn't.
When I implement the sub.py, It connects successfully so I implement the pub.py in other terminal.
However, There isn't any reaction without connect message which is "Connect with result code 0"
There is no error message so I don't know what my mistake is.
Tested Code:
Requirements:
Install Mosquitto Broker
Install paho.mqtt package.
pub.py
import paho.mqtt.publish as publish
MQTT_SERVER = "xxx.xxx.xxx.xxx" #Write Server IP Address
MQTT_PATH = "Image"
f=open("image_test.jpg", "rb") #3.7kiB in same folder
fileContent = f.read()
byteArr = bytearray(fileContent)
publish.single(MQTT_PATH, byteArr, hostname=MQTT_SERVER)
One small modification, file permission is write byte instead of write mode.
sub.py
import paho.mqtt.client as mqtt
MQTT_SERVER = "localhost"
MQTT_PATH = "Image"
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe(MQTT_PATH)
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
# more callbacks, etc
# Create a file with write byte permission
f = open('output.jpg', "wb")
f.write(msg.payload)
print("Image Received")
f.close()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(MQTT_SERVER, 1883, 60)
# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a
# manual interface.
client.loop_forever()
Expected Output:
Able to transfer the Image from Raspberry Pi to server machine.
In sub.py you have 2 on_public functions which should be renamed on_connect and on_publish respectively.
In pub.py you need to actually set the on_publish method on the client so it will be called once the publish has been completed.
...
client.connect("test.mosquitto.org", 1883, 60)
client.on_publish = on_public
...
Also as #ralight pointed out in his answer to your previous question, you should change client.loop(5) to client.loop_forever() it will still exit once the message has been sent because of the mosq.disconnect()
In your sub.py you need a callback which handles incoming messages for your subscription. The standard callback for that is on_message.
Just rename in your sub.y on_publish(client, userdata, msg) to on_message(client, userdata, msg)and assign client.on_message = on_message.