[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [mosquitto-dev] Moving away from libwebsockets
|
Forgot to include the rate at which I'm publishing. There are 450 publishers pushing 16kb messages at a rate of 2x a second. In addition, the 3000 consumers publish a small msg around 64bytes every 10 seconds.
V2 isn't as performant compared to V1 when sending large messages (16kb) over web sockets. I'm bringing this up because I saw the thread about moving away from libwebockets from a few years ago. Trying to get some traction and direction. I'll likely check out libslay and report back.
The python paho mqtt library msg queue backs up and exhausts memory under high load. More specifically 450 publishers of 16kb messages consumed by 3000 consumers will cause a back up on the python process and exhaust memory rather quickly. Doesn't seem to matter how I divy up the clients/threads to communicate with mosquitto. Once mosquitto gets above 2800 clients, it doesn't keep up with the publishing load.
I've built mosquitto against many versions and configured options of lws and the relevant variable seems to be external poll support. External poll isn't required for mosquitto <2. Without external poll on v2, large messages aren't reliable. With external poll enabled, mosquitto can experience problems keeping up with publishing when a high volume of web socket clients are present. There's a related thread about this.
https://github.com/eclipse/mosquitto/issues/2060. The difference here is I've built with external poll enabled, and have encountered load which causes the feeding service to exhaust memory because mosquitto isn't keep up with the publishing load.
I'm not against leaving libwebsockets as it's been a pain in the rear on multiple projects. Just wanted to point this out as the shift seems to have paused. Thanks for all your hard work.
