Re: [mosquitto-dev] max system memory and mosquitto persistence file siz

[Kiran Pradeep <kiran.happy@xxxxxxxxx>]

Hi Karl,

Thanks for the max_queued_bytes feature addition and your reply. I
checked out develop branch and tested `max_queued_bytes`.

The docs mentions queueing is per client. I noticed by per client, it
is meant as per subscriber client id and not per publisher client id.
And for a bridge, the limit is per bridge ( again the remote
subscriber ). It would be helpful to future readers if that is made
unambigous.

Of lesser importance: The first message per subscriber client id is
not dropped by broker even if it is above byte limits. This is not an
issue and ignore if already known.

Thanks Again,
Kiran.

On 12/22/16, Karl Palsson <karlp@xxxxxxxxxxxx> wrote:
>
>
> You're (unsurprisingly) not the first person with such desires :)
>
> You may find byte queue limits to be at least a partial fix,
> which is now implemented in the development branch:
> https://github.com/eclipse/mosquitto/commit/c6aac741c2c294b5e2b469ee8c05c1aad5c80c59
> (and a few before that)
>
> There's still no overall knobs on memory, but having the per
> client queues tracked in byte levels should certainly make it
> easier for such a knob to be added.
>
> Sincerely,
> Karl Palsson
>
> Kiran Pradeep <kiran.happy@xxxxxxxxx> wrote:
>> I am trying to limit memory consumed by mosquitto( in memory
>> queue ). The intention here is that, when running on low RAM
>> devices ( say 512MB ), I would like mosuqitto to consume not
>> more than half of system memory( so as not to impact the
>> system). Currently there is no cap on the memory consumed by
>> mosquitto broker.
>>
>> There is a `max_queued_messages` configuration paramter. But
>> that limits by number of queued messages and not by memory
>> consumption.
>>
>> I assume using linux cgroups, will result in OOM killer
>> terminating mosquitto broker.
>>
>> Is there any modification, I could do to mosquitto broker, so
>> that I could limit clients (publishing further messaging) by
>> memory consumed? Kindly suggest.
>>
>> Thanks,
>> Kiran.
>>
>> On 12/12/16, Roger Light <roger@xxxxxxxxxx> wrote:
>> > Hi Kiran,
>> >
>> >> 1. Mosquitto *always* maintains *all messages* in memory db.
>> >
>> > This is correct for current versions of mosquitto. It is unlikely that
>> > this will remain true for future versions.
>> >
>> >> 2. I would like to limit max memory used by mosquitto. I am planning
>> >> to do this enabling REAL_WITH_MEMORY_TRACKING in memory_mosq.c, and
>> >> returning NULL( ignoring errno ) from _mosquitto_*alloc, functions.
>> >> Any further clients requests to connect / publish will fail. Mosquitto
>> >> will gracefully continue to run mainloop. But client interactions will
>> >> be possible only after mosquitto frees some allocated memory with
>> >> _mosquitto_free ( possibly after a network connectivity restore ).
>> >
>> > That's an interesting idea. In principle I can see what you're trying
>> > to do, in practice I wouldn't want to guarantee that you'll get useful
>> > behaviour. For example, if the call in mosquitto_main_loop()
>> > allocating memory for the pollfds variable fails, the broker will
>> > exit.
>> >
>> > Regards,
>> >
>> > Roger
>> > _______________________________________________
>> > mosquitto-dev mailing list
>> > mosquitto-dev@xxxxxxxxxxx
>> > To change your delivery options, retrieve your password, or unsubscribe
>> > from
>> > this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/mosquitto-dev
>> >
>> _______________________________________________
>> mosquitto-dev mailing list
>> mosquitto-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/mosquitto-dev