Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[mosquitto-dev] Question about storing messages in the heap of broker

Hello, everyone!

I am investigating a possible memory leak issue in mosquitto and would like to ask for some help: I need to understand how it manages the published messages.

The use case I am testing is relatively simple: mosquitto broker without bridge, one client (both publisher and subscriber). Client subscribes for the topic, sends in it 100 identical messages with QOS = 2 and falls into busyloop (with responding to MQTT-pings).

In order to track memory allocations I had to modify the code in lib/memory_mosq.c. Particularly, I have added a list where I add/remove allocations. Based on this list I am calculating the total used memory. The items in this list also store the number of the line where mosquitto__malloc/calloc/realloc/strdup was invocated and a kind of a lifetime counter which provides understanding of how long this chunk is in the heap. (And also I found that some allocations are created using malloc/calloc/strdup bypassing mosquitto__*alloc functions. So, I also changed them.)

Using this approach I have found following strange behavior: after client finishes its publishing, broker still keeps some allocated objects in heap. Particularly:

  • allocations from sub__add_hier_entry():
    • mosquitto__calloc(1, sizeof(struct mosquitto__subhier))
    • mosquitto__malloc(len+1)
    • mosquitto__malloc(sizeof(UT_hash_table)) hidden in macro HASH_ADD_KEYPTR
  • allocations from db__messages_easy_queue()/db__message_store():
    • mosquitto__strdup(topic)
    • mosquitto__calloc(1, sizeof(struct mosquitto_msg_store))

The tracking list shows that each of these non-freed allocations appears at least 50 times which is quite a lot for 100 publications. I also tried to change QOS and found that total memory is decreasing when one changes QOS from 2 to 1 or 0. The decrease is about 1K per each QOS downgrade. This looks like a memory leak.

At the same time, I believe that mosquitto is a project developed by professionals which knows how to manage memory. So, probably, I am missing something - maybe there is a memory reclaiming/re-using procedure which frees/re-uses these resources after some time?

Thank you in advance for help!

Kind regards,

Sergey.

Back to the top