[mosquitto-dev] Multithreading

[Wiebe Cazemier <wiebe@xxxxxxxxxxxx>]

Hi, 

After bench-marking some MQTT servers, it was clear that Mosquitto is one of the fastest, it's just single threaded. I decided to have a look at the code, and it seems to me multithreading could be implemented. With edge-triggered epoll, you can call epoll_wait() on several threads. One could do pthread_create() for each CPU core at [1] where it now calls 'while(run)' in mosquitto_main_loop(), to have a loop per CPU. This will process connections rapidly, without context switching. What manner of mutexing on the subscription hierarchy, message store, log file and such are necessary, or whether it should be converted to a producer/consumer model, I'm still looking into. And I'm just learning the intricacies of the source code first, of course.

I'm working towards a first (sloppy) proof of concept. I'm happy to continue my efforts, but like to just mention it, as to not reinvent the wheel or go down paths that have been tried. I know there are some obstructions, like the Windows build without epoll, and auth plugins would need to be thread-safe, but one thing at a time.

I do have one question already. There are some (dummy) mutexes present in 'struct mosquitto' already, used in packet_mosq.c, 'int packet__write(struct mosquitto *mosq)' for instance. I read on an old wiki [2] that even though not threaded, code was written with threading in mind, but I'm not following the logic. Mutexes here suggest multiple threads are working on the same connection?

Regards,

Wiebe


[1] https://github.com/eclipse/mosquitto/blob/a9d1a9f11524d117804635f3e44869f69f294968/src/loop.c#L211
[2] https://wiki.eclipse.org/Mosquitto/Multi-Thread