Re: [paho-dev] sharing subscriptions among connected group members

[Peter Silva <peter@xxxxxxxxxxxxxxx>]

If you're telling me that is, according to the standard, how it is supposed to work... that is good and helpful.

yes. I can do interop tests... I figure mosquitto is the lowest bar, in terms of being the lightest one likely the most

conservative in implementation.  If different brokers do different things... That is sub-optimal, as I want it to work

with at least a good subset of them.  Is there a list somewhere that compares brokers for these sorts of things? I have experimented with: emqx, verneMQ, and mosquitto, so far.  I can add Hive, I guess. In earlier work they seemed to all work fine, but I wasn't getting as detailed as now. The wikimedia MQTT implementation page is long.  MQTT's origins are in SCADA, and with mqtt v5 it has much better support for message queueing apps that are a little different... hard to know which brokers to try for an app from a message queueing background.  We want reliable messaging (no loss, yes queueing) good load balancing (dynamic shared subscriptions?) and error recovery (which seems to mean different things to different people.) It's very close to amqp stuff... just can't quite tell if it is really there yet.

more concretely:  It occurred to me that brokers might want to load balancing based on ReceiveMaximum... which mosquitto appears to do... in that it doesn't queue data for any client that has reached it's receive Maximum... but that seems to mean that if there are no clients connected who have < receiveMaximum messages pending... it looks like it drops the message. Sigh... that's not what I'd prefer either... It should queue as long as it sees the client is connected, and receiveMaximum should control the number of messages in flight.

I'm searching for a way to provide backpressure... My only guess for now is to disconnect temporarily...

 

On Thu, Apr 8, 2021 at 4:35 AM Dominik Obermaier <dominik.obermaier@xxxxxxxxx> wrote:

Hi Peter,

the behavior you described that you want only online clients to receive messages is the behavior that should be expected with Shared Subscriptions. Also that unacked messages are redistributed among subscribers that are online is something that some brokers implement. The MQTT 5 spec has some room for implementation-specific behavior.

If you want to verify that what you are seeing is not an application bug but the behavior you see is a mosquitto implementation detail, you could also test against other (open source) MQTT brokers like HiveMQ Community Edition or you could quickly test against a free online MQTT broker like HiveMQ Cloud.

Best,
Dominik

On 8 Apr 2021, at 2:52, Peter Silva wrote:

Hi there,

Example: client A, B, and C establish sessions subscribed to:

$share/G/test/#

All three clients are part of group G and get 1/3 messages for topic:
test/#
So far, so good.. but let's say B dies... ... so sad... Once we have
determined that B is dead, I would want the broker to send half the
messages to A and the other half to C.

My tests are really short (a few minutes) so perhaps I didn't give it
enough of a chance, but as far as I can tell, it continues to split 1/3,
1/3, 1/3 forever queueing up for the B that is gone. If D comes in... then
it gets 1/4, as do A and C, and 1/4 piles up with B. I imagine I could
connect with B again, and do an unsubscribe, to tell the broker that B has
left the group... but that relies on b being healthy enough to do so.

It would be nice if there were a way to tell the broker to round-robin
among the clients that are currently connected? so the collective work
continues at 1/3 slower aggregate rate, but without queueing 1/3 of the
work forlornly for B to return. When you decide B is dead, it should also
re-distribute the queue of unacknowledged messages in it's queue to the
remaining group members.

Leaving a queue for B will mean the load will be asymmetrical for some time
after it returns, as it will be processing its backlog.

Does this seem reasonable, and is there a way to convince a broker to
behave this way (e.g. mosquitto?)
_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/paho-dev

_______________________________________________
paho-dev mailing list
paho-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/paho-dev