Re: [mosquitto-dev] v5 API message modification

[Abilio Marques <abiliojr@xxxxxxxxx>]

Hi Roger,

Thanks for the reply :) . Of course performance can always be an issue. But as long as it does not impact if unused, I guess is a final user tradeoff, depending on the application.

Other examples I can think of where modifying the output messages can be useful for some people:

• Mounting/namespacing/jailing some users to a specific level, e.g., an application where 'sensors/temp/{client_id}' is the root for all the temp sensors in the system, while the sensors think they are publishing and subscribing to "/". I imagine this being really useful to build a secure system.
  
• MQTT 5 to 3.1.1 translation, where a plugin can be made to append the properties to the payload for 3.1.1 clients, while keeping the rest of the clients using MQTT 5.

Any suggestions on how to do it? Should I expand the existing message event struct with an argument that indicates "read/write" or should I create a new callback altogether?

Kind Regards,

Abilio

On Sat, Mar 5, 2022 at 11:17 AM Roger Light <roger@xxxxxxxxxx> wrote:

Hi Abilio,

Thank you, I'm pretty happy with how the plugin interface is going.
The examples directory at
https://github.com/eclipse/mosquitto/tree/develop/plugins/examples is
filling up nicely.

I'm keen to see the interface grow and become more useful, in
particular I want to widen the broker function API so plugins can have
greater capability to interrogate and control - like finding client
and topic information for example. Likewise more events as required.

I suppose the main reason against an "out" message event would be that
it happens per client that will receive the message, so there is
potentially a very big fanout and duplication of effort - if you're
talking about doing per message decompression then that could be a
factor. Perhaps if we ensure that plugins know the store_id of the
base message then they can use that to determine when there is a new
message.

Other than that I can't think of a reason against it.

Cheers,

Roger



On Sat, 5 Mar 2022 at 09:25, Abilio Marques <abiliojr@xxxxxxxxx> wrote:
>
> With excitement I've been watching the newly improved vr5 api for Mosquitto. Kudos to Roger for such a great work.
>
> There is one feature I loved: the ability to build plugins that can modify messages, altering payloads and topics. I've been doing many experiments with it, and I see a lot of potential. But I couldn't help noticing that message modification happens only when messages come in (and not when they go out).
>
> Potential uses of modifying outgoing messages:
>
> Selective modification of properties based on destination.
> Topic remapping.
> "Redacting" portions of the payload.
> Decompression of retained messages (compressed when they come in) to reduce memory footprint.
>
> I found a way to circumvent the lack of modification on outgoing messages. I intercept them on the read ACL callback and re-publish them to the specific client ID. This works fine as long as the message is not stale (retained). In this case the retain bit is always forced to 0, and this breaks the rule that "stale" messages must be marked with a 1.
>
> I can imagine at least 3 reasons why modification on output is not implemented:
>
> Is not implemented "yet"
> There are reasons to consider is a bad idea
> There are ways to achieve the same without it
>
> If anyone knows of a way to achieve this without further modifications to mosquitto, in a way that is compatible with retained/stale, could you please share your ideas.
>
> If the whole concept is a bad idea, could someone mention why?
>
> And if it's just a missing feature, I'm thinking of prototyping an implementation, but looking for good ideas first.
>
> Regards,
> Abilio
> _______________________________________________
> mosquitto-dev mailing list
> mosquitto-dev@xxxxxxxxxxx
> To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/mosquitto-dev
_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/mosquitto-dev