[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [mosquitto-dev] Question about storing messages in the heap of broker
|
Hello, Roger!
Yes, the increase of the memory is very
similar to described behavior with retained bit set. Are all
messages kept in the memory if retained bit set?
By the way, is there a kind of a high
level design document for mosquitto?
I have made a long test and it seems
that I am mistaken about memleak: most probably I am missing some
free procedure and miscalculating total used memory.
Kind regards,
Sergey.
Hi Sergey,
I've just tried to reproduce this using
Mosquitto 1.6.9 running under massif, and a simple client that
sends qos 2 messages to topic1, topic2, topic3, ...
In the first case, the heap usage increases to a
low level just after startup, then remains essentially
constant.
In the second case, I changed the client
behaviour to publish with the retained bit set. The behaviour
there is increasing heap usage.
A retained message requires a number of things
to be stored: the topic on which it was published, the tree
items, the message itself, and a few other bits. This is
entirely consistent with what you are seeing, and may be the
cause. There are other situations where the same outcome
occurs - like if a client makes a subscription, then the topic
tree items will remain in memory. Or if a persistent client is
disconnected, then queued messages for it will also exhibit
this behaviour, even if the messages aren't retained.
If your situation is different, please let us
know where it differs and we can see what is happening.
Regards,
Roger
Hello, everyone!
I am investigating a possible memory leak issue in
mosquitto and would like to ask for some help: I need to
understand how it manages the published messages.
The use case I am testing is relatively simple: mosquitto
broker without bridge, one client (both publisher and
subscriber). Client subscribes for the topic, sends in it
100 identical messages with QOS = 2 and falls into
busyloop (with responding to MQTT-pings).
In order to track memory allocations I had to modify the
code in lib/memory_mosq.c. Particularly, I have added a
list where I add/remove allocations. Based on this list I
am calculating the total used memory. The items in this
list also store the number of the line where
mosquitto__malloc/calloc/realloc/strdup was invocated and
a kind of a lifetime counter which provides understanding
of how long this chunk is in the heap. (And also I found
that some allocations are created using
malloc/calloc/strdup bypassing mosquitto__*alloc
functions. So, I also changed them.)
Using this approach I have found following strange
behavior: after client finishes its publishing, broker
still keeps some allocated objects in heap. Particularly:
- allocations from sub__add_hier_entry():
- mosquitto__calloc(1, sizeof(struct
mosquitto__subhier))
- mosquitto__malloc(len+1)
- mosquitto__malloc(sizeof(UT_hash_table)) hidden in
macro HASH_ADD_KEYPTR
- allocations from
db__messages_easy_queue()/db__message_store():
- mosquitto__strdup(topic)
- mosquitto__calloc(1, sizeof(struct
mosquitto_msg_store))
The tracking list shows that each of these non-freed
allocations appears at least 50 times which is quite a lot
for 100 publications. I also tried to change QOS and found
that total memory is decreasing when one changes QOS from
2 to 1 or 0. The decrease is about 1K per each QOS
downgrade. This looks like a memory leak.
At the same time, I believe that mosquitto is a project
developed by professionals which knows how to manage
memory. So, probably, I am missing something - maybe there
is a memory reclaiming/re-using procedure which
frees/re-uses these resources after some time?
Thank you in advance for help!
Kind regards,
Sergey.
_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/mosquitto-dev
_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/mosquitto-dev