Hi Jianhui
@Oegma2, I have tested your scenario in 3 brokers mosquitto cluster by Tsung, include 100 clients connected with will msg, and another client subscribed with topic filter "#", and found that only 25 will msgs received by the subscriber, so I have checked the mosquitto.log and found that the subscription was happend AFTER 75 clients' disconnection, so it only receive 25 will msgs for remained disconnection. Could you check with this point in your env.?
I've setup two clustered servers yesterday using t2.micro EC2 instances on AWS using the new LB (TCP edition) and got 2000 clients connected without any issues - got my LWT message on the subscriber side using mqttspy and sending a ping message to all 2000, i got pong 2000 message back with my details. I did remove a lot of the config's "YES" parts..like not doing any persistence or TLS or any special options
In short, using the clustered mosquitto - and correct sub client tool "mqttspy" - things looking much better in terms of testing setup
Setup that I've tested yesterday (AWS Region eu-west-1):
2x EC2 t2.micro instances running mosquitto on Ubuntu 16.04
1x ELB (new edition TCP) forwarding traffic to both EC2 servers
Route53 DNS mapping to the ELB
Testing setup on my end - running in South Africa "very long distance from eu-west-1 :)"
Client code on my side update to point to new DNS Route53
Spawned 2000 of those clients each doing the following steps
1. Connection contains "LWT part when client goes offline"
2. OnConnect -> Pub message to topic "STATUS" as online
3. Sub -> Test/PingMe
Used mqtt-spy-1.0.0.jar as "worker" or subscriber to topic "#"
Using that setup
1. Got 2000 message on topic "STATUS"
2. When I close my loadtest, got 2000 LWT responses
3. Restarted clients and publish message to Test/PingMe ...got 2000 message back ;)
"I have' started to map metric in terms of msg/sec yet} - first trying to establish a working baseline"
Next test will be to setup 3 clustered mosquitto and try 3000 clients with the same scenario
Summary:
* Working 2xmosquitto cluster with 2000 clients - everything working, including LWT
* Suggest using mqtt-spy-1.0.0 as subscriber on topic "#"