Re: [paho-dev] controing generated persistent messages

[Peter Silva <peter@xxxxxxxxxxxxxxx>]

Yeah, I don't want to suffer the excess overhead of QoS==2, so I use QoS==1... the continuous retransmission thing I had heard that MQTT does... The thing is, for the protocol to work, it has to leave some time for the acks to happen at all, so it cannot be too frequent.  This guy did the homework:

http://www.steves-internet-guide.com/understanding-mqtt-qos-levels-part-1/

It shows retransmissions happening 9 seconds, 7 seconds, then 20 seconds later... 

In the best case, there is no queueing on the client, and processing on the far end fits in such an interval, so it doesn't make any difference.

IF the library is as dumb as described, where it start resending rapidly and unconditionally, then

If you have large queueing on the client... like 5 minutes... well it also doesn't matter whether you are acknowledging in the library or not... so... it is going to behave very badly no matter what level the ack is done at.

It is only in the middle case, where you have either: a little queueing, large propagation delay, or a potentially large processing time, where it will cause extra message sends by the broker.

Does anyone know of a reference that describes how brokers are supposed to decide when to resend with QoS==1 ?  I looked through the OASIS standard, and did not see anything,

Ideally, the broker would take hints from the rate at which acknowledgements from the consumer arrives, and only resend when it has good reason to believe they are "late".

In such as case, the broker will see the receiver acknowledging messages gradually later and later, and it should accumulate, say a mean time to acknowledgement receipt, and only resend after... oh... twice that time has gone by...

That would be robust... anybody know if any brokers actually do this sort of thing?

I'll take a look at mosquitto source... but someone who knows might be easier.

On Fri, Mar 26, 2021 at 6:35 PM Frank Pagliughi <fpagliughi@xxxxxxxxxxxxxx> wrote:

It's an interesting idea about user-controlled ACKs. I always go with AMQP over MQTT if I have reliable connections and powerful enough machines, partly due to this and the various other use cases AMQP brings.

I wonder if the thinking is that AMQP is more of a pull model while MQTT is more push. If you don't ACK a QoS=1 message, the broker will keep throwing the message at you at some undefined rate. Is that what you want?  And then how do you handle the full handshake of QoS=2?

Frank

On 3/26/21 1:00 PM, Peter Silva wrote:

(repost because the subject was a mistaken post of message content... sorry about that.)

An easier answer to this question is to allow application control over acknowledgements.  I come from AMQP... and use explicit acks over there.
You only acknowledge messages when you know processing is done...
I have a pull request to add explicit ack mode to the python-paho library here:

https://github.com/eclipse/paho.mqtt.python/pull/554

23 days so far with no response...  feedback welcome.

fwiw, the lack of this kind of user control of acks is really crippling for a lot of use cases.


On 3/12/21 12:01 PM, Frank Pagliughi wrote:
Persist your application queue?

Create a file when the connection is lost, then append messages to the end of the file. Be sure to flush each time you add data. That should be easy enough. Depending on your system you could also use a little cache or DB, like Redis (with persistence enabled) or SQLite.

Frank

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/paho-dev