Re: [paho-dev] Throttling on massive message publish in Java client

[Lars Schuetze <dev@xxxxxxxxxxxxxxx>]

Hi Ian,

thank you very much for your response.

So, for 1)
I did tests with mosquitto, as well as with Mosca, but configured it
to use mosquitto as backend. There is a broker effect and pure
mosquitto is way faster.
[1] shows a graph where the y-axis is the latency (where start = 0) so
it should be a simple line. But at the end it starts going horrible
slow and the latency is increasing rapidly. But that will be a problem
of the TCP packets flying around. This was tested with AsyncClient and
QoS0.

Since I tested it on the same machine the many connections (client <->
broker (TCP), broker <-> webclient (websocket, so TCP)) seems to be
too much.
Having the full pipeline and make use of mosquitto as a parent broker
for Mosca I end up with having packages sent from the beginning (ca.
2600mps arriving), but after stopping it still sends packages for some
time. So this is a Mosca problem and I will not go deeper into this.

For 2)
If I hit this exception I make the client to wait 500ms and then
continue to send.
However, I get a second exception, too: There are no more message IDs
available (which is set to max 65534). But I read up the standard and
there is no need for package identifiers[3] when using QoS0 but
MqttWireMessage[2] always returns true in this regard (and thus is
causing exceptions when hitting [4]). I am not so much familar with
your code base, so it could be I am mixing the standards package
identifier with your message identifier.

I dont get the exception with the sync client, but it sends barely
1600 mps (but there is almost no latency).

Best Regards,
    Lars


[1] https://drive.google.com/file/d/0B2mHWmdGeL4pZHJXOWM2V18xMGM/view?usp=sharing
[2] http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.java.git/tree/org.eclipse.paho.client.mqttv3/src/main/java/org/eclipse/paho/client/mqttv3/internal/wire/MqttWireMessage.java#n143
[3] http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.pdf , line
(numbers on the left) 1231 (or page 52)
[4] http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.java.git/tree/org.eclipse.paho.client.mqttv3/src/main/java/org/eclipse/paho/client/mqttv3/internal/ClientState.java#n416

2015-01-27 22:43 GMT+01:00 Ian Craggs <icraggs@xxxxxxxxxxxxxxxxxxxxxxx>:
> Hi Lars,
>
> two things to investigate first come to my mind.
>
> 1) Try similar tests with Mosquitto as well, to see if we can see any broker
> effect.  This can happen especially with QoS 0 as there is no throttling
> available in MQTT for QoS 0 (no acks).
>
> 2) In the async client, what action are you taking when you get the "max
> messages inflight" condition?  Do you see that condition with the sync
> client as well?
>
> Ian
>
>
>
> On 01/27/2015 12:55 PM, Lars Schuetze wrote:
>>
>> Hello,
>>
>> I am using Ecipse Paho Java (1.0.1) implementation in a height
>> throughput environment.
>> I did some experiments and am experiencing some weird things.
>>
>> The scenario is a client using Paho, and a broker using Mosca (NodeJS
>> implementation, said to handle 20k msg/s). We have a websocket browser
>> application for visualization.
>> I have tried the synchronous and asynchronous client as well as
>> different QoS parameters.
>>
>> First of all, the async client is much much slower when using QoS 1
>> then using the synchronous client directly, which should be
>> equivalent. But I am achieving the below listed values:
>>
>> MqttClient (QoS1): 1500 - 1700 mps
>> MqttAsyncClient (QoS1): 20 - 100 mps
>> MqttAsyncClient (QoS0):
>>      - first second 1-2mps then 0 mps
>>      - some seconds pause
>>      - then 4000 - 8000 mps
>>
>> The pictures [1,2] are in the done with the MqttAsyncClient and QoS 1.
>> Where [1] shows the counted packages per second arriving at the
>> broker. As you can see there are some packages sent (99, 1131) until
>> the throughout drops dramatically. After pausing the client the
>> throughput stays some seconds low until it explodes. In [2] you see
>> the graph in which the x-axis is time and y-axis is millisecond
>> between send <-> receive (broker) <-> receive (webclient). The huge
>> increase in time is the point in time where I made the client stop
>> sending. This is where in [1] those (1675, 2040, 1577) packages begin
>> to fly in.
>>
>> The scenario is the very same when using the async client with QoS 0.
>>
>> I've been using Fedora 21, Ubuntu 14.10 for testing this. Those
>> pictures where taken in a local environment. Using Wireshark I
>> calculated that by requesting the client to send 10 bytes messages the
>> TCP package arrived will be 113 bytes. But doing the math I cannot see
>> that my 100/10 Ethernet will be the limiting factor.
>>
>> Maybe you have some ideas on how to investigate further.
>>
>> Regards,
>>      Lars
>>
>> [1]
>> https://drive.google.com/file/d/0B2mHWmdGeL4pM0J4VHBlTlBUd0E/view?usp=sharing
>> [2]
>> https://drive.google.com/file/d/0B2mHWmdGeL4peHFsdEFPbHdCNE0/view?usp=sharing
>> _______________________________________________
>> paho-dev mailing list
>> paho-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/paho-dev
>
>
> --
> Ian Craggs
> icraggs@xxxxxxxxxx                 IBM United Kingdom
> Paho Project Lead; Committer on Mosquitto
>
> _______________________________________________
> paho-dev mailing list
> paho-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/paho-dev