Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[mosquitto-dev] ~3 orders of magnitude lower performance with websockets than raw mqtt 
Hi,

While evaluation performance of mosquitto compared to other brokers I've
noticed a very significant difference between raw MQTT and MQTT over
websockets.

E.G. running https://github.com/krylovsk/mqtt-benchmark against
mosquitto on localhost I see:

mqtt-benchmark --broker tcp://localhost:1883 --clients 1 --count 1000 --size 1024 --qos 1
2017/04/21 23:59:33 Starting client  0
2017/04/21 23:59:33 CLIENT 0 is connected to the broker tcp://localhost:1883
2017/04/21 23:59:33 CLIENT 0 published 100 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 200 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 300 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 400 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 500 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 600 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 700 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 800 messages and keeps publishing...
2017/04/21 23:59:33 CLIENT 0 published 900 messages and keeps publishing...
======= CLIENT 0 =======
2017/04/21 23:59:33 CLIENT 0 is done publishing
Ratio:               1.000 (1000/1000)
Runtime (s):         0.072
Msg time min (ms):   0.035
Msg time max (ms):   1.556
Msg time mean (ms):  0.069
Msg time std (ms):   0.078
Bandwidth (msg/sec): 13816.009

========= TOTAL (1) =========
Total Ratio:                 1.000 (1000/1000)
Total Runtime (sec):         0.072
Average Runtime (sec):       0.072
Msg time min (ms):           0.035
Msg time max (ms):           1.556
Msg time mean mean (ms):     0.069
Msg time mean std (ms):      NaN
Average Bandwidth (msg/sec): 13816.009
Total Bandwidth (msg/sec):   13816.009

vs:
mqtt-benchmark --broker ws://localhost:1884 --clients 1 --count 1000 --size 1024 --qos 1
2017/04/21 23:59:43 Starting client  0
2017/04/21 23:59:43 CLIENT 0 is connected to the broker ws://localhost:1884
2017/04/22 00:00:04 CLIENT 0 published 100 messages and keeps publishing...
2017/04/22 00:00:24 CLIENT 0 published 200 messages and keeps publishing...
2017/04/22 00:00:44 CLIENT 0 published 300 messages and keeps publishing...
2017/04/22 00:01:04 CLIENT 0 published 400 messages and keeps publishing...
2017/04/22 00:01:24 CLIENT 0 published 500 messages and keeps publishing...
2017/04/22 00:01:44 CLIENT 0 published 600 messages and keeps publishing...
2017/04/22 00:02:04 CLIENT 0 published 700 messages and keeps publishing...
2017/04/22 00:02:24 CLIENT 0 published 800 messages and keeps publishing...
2017/04/22 00:02:44 CLIENT 0 published 900 messages and keeps publishing...
======= CLIENT 0 =======
2017/04/22 00:03:04 CLIENT 0 is done publishing
Ratio:               1.000 (1000/1000)
Runtime (s):         200.820
Msg time min (ms):   199.551
Msg time max (ms):   203.438
Msg time mean (ms):  200.468
Msg time std (ms):   0.171
Bandwidth (msg/sec): 4.980

========= TOTAL (1) =========
Total Ratio:                 1.000 (1000/1000)
Total Runtime (sec):         200.820
Average Runtime (sec):       200.820
Msg time min (ms):           199.551
Msg time max (ms):           203.438
Msg time mean mean (ms):     200.468
Msg time mean std (ms):      NaN
Average Bandwidth (msg/sec): 4.980
Total Bandwidth (msg/sec):   4.980

E.G. ~3000x more msg/sec with raw MQTT vs websockets.

This is with mosquitto 1.4.11 and libwebsockets 2.1.1. The configuration
is simply:

# cat /etc/mosquitto/mosquitto.conf
listener 1883
listener 1884
protocol websockets

mosquitto -c /etc/mosquitto/mosquitto.conf

strace'ing mosquitto seems to show that it hangs in a poll() for 100ms
before timing out and replying to the client:

strace -e recvfrom,sendto,poll -r  mosquitto -c /etc/mosquitto/mosquitto.conf
..
     0.000343 recvfrom(9, "\202\376\4\f[d\275_i\355\265_^K\311:(\20\275^[d\275_[d\275_[d\275_"..., 4096, 0, NULL, NULL) = 1044
     0.000549 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}], 2, 100) = 0 (Timeout)
     0.100796 poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=9, events=POLLIN|POLLOUT}], 3, 0) = 1 ([{fd=9, revents=POLLOUT}])
     0.000582 poll([{fd=9, events=POLLOUT}], 1, 0) = 1 ([{fd=9, revents=POLLOUT}])
     0.000397 sendto(9, "\202\4@\2\0\1", 6, MSG_NOSIGNAL, NULL, 0) = 6
     0.000828 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}], 2, 100) = 0 (Timeout)
     0.100635 poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}, {fd=9, events=POLLIN}], 3, 0) = 1 ([{fd=9, revents=POLLIN}])
     0.000372 recvfrom(9, "\202\376\4\f\200\205\307\230\262\f\317\230\205\252\263\375\363\361\307\231\200\205\307\230\200\205\307\230\200\205\307\230"..., 4096, 0, NULL, NULL) = 1044

Which seems to come from the poll() in loop.c:

#ifndef WIN32
		sigprocmask(SIG_SETMASK, &sigblock, &origsig);
		fdcount = poll(pollfds, pollfd_index, 100);
		sigprocmask(SIG_SETMASK, &origsig, NULL);
#else
		fdcount = WSAPoll(pollfds, pollfd_index, 100);
#endif
		if(fdcount == -1){
			_mosquitto_log_printf(NULL, MOSQ_LOG_ERR, "Error in poll: %s.", strerror(errno));
		}else{
			loop_handle_reads_writes(db, pollfds);


Whereas with raw MQTT it replies right away:

     0.000061 read(9, "2", 1)           = 1
     0.000029 read(9, "\211", 1)        = 1
     0.000028 read(9, "\10", 1)         = 1
     0.000029 read(9, "\0\5/test\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1033) = 1033
     0.000049 write(9, "@\2\0\1", 4)    = 4
     0.000134 poll([{fd=6, events=POLLIN}, {fd=8, events=POLLIN}], 2, 0) = 0 (Timeout)
     0.000107 poll([{fd=3, events=POLLIN}, {fd=4, events=POLLIN}, {fd=9, events=POLLIN}], 3, 100) = 1 ([{fd=9, revents=POLLIN}])

Is this a known issue? Anything I'm doing wrong?

-- 
Bye, Peter Korsgaard

Back to the top