I'm writing a python project which need to send message by mqtt. I find a problem that when I send a order which need subscriber to download a big file which need to spend few minutes, whereagfter subscriber called on_connect func again, at the moment the subscriber cannot receive any message it subscribed. this bug happens occasionally.
After many test, I find that ss long as the on_connect function is called after downloading a large file, the subscriber cannot receive other messages.
And subscriber can send a message stating that mqtt has no problem, can also receive after opening a terminal subscription message.
So, I guess the subscription was disconnected after downloading the large file. I need to check the topic of the broker's internal connection subscription to verify my guess.
But I don't konw how to check it. Please tell me the method to inspect broker and how to fix this question if the guess is proved correct
Because there's too much code, I'm going to outline it
cloud send a series of order including download file, modified and the likes by mqtt
devices receive message by mqtt, then operate order and feedback
After devices download big file, there is a probability that other MQTT messages cannot be received when terminal print the result that wait handle : Connected with result code 0 of on_connect func which is type
def on_connect(client, userdata, flags, rc):
print("wait handle : Connected with result code " + str(rc))
The problem is most likely that you are doing long running tasks in either the on_connect or on_message callbacks.
These callbacks run on the MQTT client's network thread, this thread is used to handle the sending and receiving of network packets. If it blocks for too long then the keep alive (time between MQTT packets) will expire and the broker will disconnect the client.
If you have long running tasks they need to be run on a separate thread.
Using subprocess and waiting for it to finish so you can get the output then you are blocking for the length of time the process takes to run, so you might as well be running it on the same thread.
Related
I am new to using MQTT. I have set up an Arduino publishing MQTT messages to ActiveMQ. I also have a Python script subscribed to the same topic (using paho) which gets the data from the broker and inserts it into a database.
The problem is if this Python script is down or loses connection for any reason I lose all messages being published while its down. How can I ensure all the data is inserted into the database? I see ActiveMQ has some sort of storage is it possible to retrieve historic data from it?
If you want to ensure that your subscription persists and receives messages even if the subscriber gets disconnected then you need to set CleanSession=false on your MQTT client (assuming you're using MQTT 3.x) and use the same client ID when you reconnect.
Also, if you want messages to survive a broker restart or crash you need to send them with QoS 1.
I think the PubSubClient Arduino library only publishes with fire and forget (QoS 0). I've instead used the stomp.py library to create a durable consumer subscribed to the same topic. If there's no connection on Arduino It's saved to the SD card and if there is connection but no client at the time the messages are stored by ActiveMQ until the client is active.
This solved my problem but I'm still testing the durability
Do python-socketio or underlying python-engineio have any kind of confirmation that specific message was completely delivered to other side, similar to what TCP does to ensure all data was successfully transferred to other side?
I have kind of pubsub service built on python-socketio server, which sends back ok/error status when request has been processed. But in my python-socketio client sometimes I just need fire and forget some message to pubsub but I have to wait it was completely delivired before I terminate my application.
So, my naive code:
await sio.emit("publish", {my message})
it seems the await above is just scheduling send over wire to asyncio, but does not wait for send to complete. I suppose it's by design. Just need to know is it possible to know when send is complete or not.
Socket.IO has ACK packets that can be used for the receiving side to acknowledge receipt of an event.
When using the Python client and server, you can replace the emit() with call() to wait for the ack to be received. The return value of call() is whatever data the other side returned in the acknowledgement.
But not that for this to work the other side also needs to be expanded to send this ACK packets. If your other side is also Python, an event handler can issue an ACK simply by returning something from the handler function. The data that you return is included in the ACK packet. If the other side is JavaScript, you get a callback function passed as a last argument into your handler. The handler needs to call this function passing any data that it wants to send to the other side as response.
I am a writing a wrapper for a C-API with Cython. The goal is to build a client which connects to a mediator server using the API. The client process a audio file, splits it into packages and sends them to the mediator. Short before sending the audio packets he starts an additional thread for receiving incoming packets from the mediator. The API method for receiving packets from the mediator is called with released gil no gil. Nevertheless, after finishing to send all the outgoing packets the client has to wait. He is still receiving packets from the mediator and has to wait for a done message on mediators part. Only then he is allowed to disconnect from the mediator.
DO STUFF
...
t = threading.Thread(target=_recv_thread, args=(user_data,))
t.start()
...
send_audio_packets()
while True:
if user_data.proceed is 1:
print("Still waiting")
time.sleep(2)
else:
disconnect()
To disconnect the user_data.proceed value has to be set to 0 after receiving the last packet.
The problem I am now encountering is that the time.sleep() causes a double free or corruption (fasttop) Aborted. Is it possible that this error has to do something with the other "thread"?
A message queue stores message until they are consumed When you use a message queue, each incoming message is stored in the queue until it is picked up by a client (often called a consumer). If no client picks up the message, the message remains stuck in the queue and waits to be consumed. In a message queue, it is not possible for a message not to be processed by any client, as it is in MQTT if nobody subscribes to a topic.
What will happen if the subscriber receives the message but is unable to download it due to lack of connection? What measures can be taken to make sure the client gets the message even after this?
I'm developing an application with Paho MQTT and QOS = 1. I know for sure that if the network is not available for a certain period of time the client puts every message in a queue and sends them when the connection is established again.
What I want to understand it's if the queue is saved even when the application itself gets restarted (for instance a machine reboot). If not, how do you suggest to implement it?
Thanks for your help.