Re: [paho-dev] MqttAsyncClient - inflight message limit

[Davy De Waele <ddewaele@xxxxxxxxx>]

The buffer capability would be a very welcome feature ... Currently one has to implement his own persistence mechanism in order to get true QoS 2 so looking forward to that one.

While we're on the topic of in-flight messages, during testing I was in a situation where my broker was "throttling" (read: rejecting)" all messages to a particular topic :

/var/log/activemq/activemq.log.3:2015-01-26 12:14:40,202 [Q NIO Worker 80] INFO  Topic                          - topic://testTopic, Usage Manager memory limit reached 1048576. Producers will be throttled to the rate at which messages are removed from this destination to prevent flooding it. See http://activemq.apache.org/producer-flow-control.html for more info.

This resulted in Paho being unable to clear the current 10 inflight-messages for that topic. As a result Paho couldn't publish anything anymore. (not even to different topics). 

Perhaps it would be interesting to maintain a max number of in-flight messages on a per topic basis.  That way one topic that is "mis-behaving" would not bring down the whole system.  But perhaps the buffer will solve that problem.

On Mon, Jan 26, 2015 at 2:16 PM, Dave Locke <locke@xxxxxxxxxx> wrote:

and one other enhancement that is on the wish list is to add a buffer capability that can accept and store messages when connectivity is not available but would also handle the case where the "in-flight" window is maxed out,.


All the best
Dave

 



From:        Ian Craggs <icraggs@xxxxxxxxxxxxxxxxxxxxxxx>
To:        paho-dev@xxxxxxxxxxx
Date:        26/01/2015 12:58
Subject:        Re: [paho-dev] MqttAsyncClient - inflight message limit
Sent by:        paho-dev-bounces@xxxxxxxxxxx

---

Hi Davy,

1) You can set a callback on the publish method, which will be called when each publish has completed.  This way you can count how many messages are in flight and delay the call to publish.

2) We also have an enhancement currently under review which allows the application to set the maximum number of inflight messages allowed.  I hadn't got to reviewing it yet -- I hope to do so this week or next.

Ian


On 01/26/2015 11:07 AM, Davy De Waele wrote:
Hi,

I started implementing a solution based on the java MqttAsyncClient since I needed to obtain the IMqttToken for message bookkeeping due to some limitiations with the syncClient.

However I noticed the following :

When sending a substantial number of msgs in a short timestamp, the risk of running into the max inflight messages is very high (writing a loop that sends 20 msgs to a topic results in "Too many publishes in progress (32202)" about half of the time. 

This is not the case with the syncClient because you allow for some time for the message to get sent before moving on to the next one.

How should we handle this ? Our broker is perfectly capable of handling this type of load, but here it seems that the client is the bottleneck.

Should the asyncClient always be used by specifying a waitForCompletion on the mqttDeliveryToken to avoid having too much inflight messages in the client ?

mqttDeliveryToken.waitForCompletion(200);

Regards,
Davy



_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/paho-dev

--
Ian Craggs                          
icraggs@xxxxxxxxxx                 IBM United Kingdom
Paho Project Lead; Committer on Mosquitto

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/paho-dev

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/paho-dev