Re: [mosquitto-dev] Retained message $SYS/broker/messages/stored not alw

["c@xxxxxxxx" <c@xxxxxxxx>]

Hi Wim,

I can confirm your observation with mosquitto 2.0.14.

Most of the $SYS messages do expire after 60 seconds but are republished only if the values changed since the last publish after `sys_interval`.

This means if the number of stored messages does not change within 60 seconds , the topic `$SYS/broker/messages/stored`Is deleted.

I am not sure if this is the intended behaviour introduced with MQTT v5 in 2019 with commit ca3782b.

With regards to your tool, you cannot rely on those $SYS topics to be present.

Regards,

   Christoph

Snippet from sys-tree.c:

if(db.msg_store_count != msg_store_count){ msg_store_count = db.msg_store_count; len = (uint32_t)snprintf(buf, BUFLEN, "%d", msg_store_count); db__messages_easy_queue(NULL, "$SYS/broker/messages/stored", SYS_TREE_QOS, len, buf, 1, 60, NULL); db__messages_easy_queue(NULL, "$SYS/broker/store/messages/count", SYS_TREE_QOS, len, buf, 1, 60, NULL); }

On 21. Aug 2022, at 22:04, Wim Nelis via mosquitto-dev <mosquitto-dev@xxxxxxxxxxx> wrote:

On a Raspberry Pi running Raspberry OS 11 (bullseye), a monitoring tool regularly retrieves the wildcard topic '$SYS/broker/messages/#' from the local mosquitto service. To retrieve these statistics, the monitoring program goes through the following actions at each poll:

• it builds a session to the broker (on localhost),
  
• it subscribes to the aforementioned topic,
• it starts a watchdog timer of 0.5 seconds,
• it stops if either 3 messages are received or the watchdog timer is expired.

This should deliver all retained messages with a matching topic. However, regularly no message with topic '$SYS/broker/messages/stored' is received. Increasing the wait period in the watchdog timer does not solve this issue.

Mosquitto is installed using apt. It's version is 2.0.11.

The problem can be shown using utility mosquitto_sub. The output of three invocations is shown below.

pi@rpi30:~/tmp $ date ; mosquitto_sub -h 127.0.0.1 -d -t '$SYS/broker/messages/#' ; date
2022-08-21 21:03:00+02:00
Client (null) sending CONNECT
Client (null) received CONNACK (0)
Client (null) sending SUBSCRIBE (Mid: 1, Topic: $SYS/broker/messages/#, QoS: 0, Options: 0x00)
Client (null) received SUBACK
Subscribed (mid: 1): 0
Client (null) received PUBLISH (d0, q0, r1, m0, '$SYS/broker/messages/received', ... (4 bytes))
7403
Client (null) received PUBLISH (d0, q0, r1, m0, '$SYS/broker/messages/sent', ... (4 bytes))
5179
^CClient (null) sending DISCONNECT
2022-08-21 21:03:01+02:00
pi@rpi30:~/tmp $ date ; mosquitto_sub -h 127.0.0.1 -d -t '$SYS/broker/messages/#' ; date
2022-08-21 21:03:09+02:00
Client (null) sending CONNECT
Client (null) received CONNACK (0)
Client (null) sending SUBSCRIBE (Mid: 1, Topic: $SYS/broker/messages/#, QoS: 0, Options: 0x00)
Client (null) received SUBACK
Subscribed (mid: 1): 0
Client (null) received PUBLISH (d0, q0, r1, m0, '$SYS/broker/messages/stored', ... (2 bytes))
53
Client (null) received PUBLISH (d0, q0, r1, m0, '$SYS/broker/messages/received', ... (4 bytes))
7407
Client (null) received PUBLISH (d0, q0, r1, m0, '$SYS/broker/messages/sent', ... (4 bytes))
5184
^CClient (null) sending DISCONNECT
2022-08-21 21:03:11+02:00
pi@rpi30:~/tmp $ date ; mosquitto_sub -h 127.0.0.1 -d -t '$SYS/broker/messages/#' ; date
2022-08-21 21:38:24+02:00
Client (null) sending CONNECT
Client (null) received CONNACK (0)
Client (null) sending SUBSCRIBE (Mid: 1, Topic: $SYS/broker/messages/#, QoS: 0, Options: 0x00)
Client (null) received SUBACK
Subscribed (mid: 1): 0
Client (null) received PUBLISH (d0, q0, r1, m0, '$SYS/broker/messages/received', ... (4 bytes))
7944
Client (null) received PUBLISH (d0, q0, r1, m0, '$SYS/broker/messages/sent', ... (4 bytes))
5555
^CClient (null) sending DISCONNECT
2022-08-21 21:38:29+02:00
pi@rpi30:~/tmp $

Up until a few days ago, the Raspberry Pi was running Raspberry OS 10 (buster). The same monitoring script was in use, and in the two years it has been in use, this problem has never occurred.

I've tried to use '$SYS/broker/store/messages/count' instead, but that did not solve the issue.

Changing the script to subscribe to the individual topics, including '$SYS/broker/messages/stored', did not solve the issue either. Most of the times the script would run into a time-out.

(There is a pattern. The script is invoked every 5 minutes. The script will run into a time-out 10 times, followed by one run when it completes without a time-out. Then it repeats.)

How to solve this issue? For the time being, I circumvent the issue by not requesting '$SYS/broker/messages/stored'.

Regards,

  Wim Nelis.

_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/mosquitto-dev