Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [mosquitto-dev] Cassandra as a retained message store
Yes - preliminary discussions have started on adding retained storage and other capabilities to RabbitMQ MQTT.

We have a "rabbits everywhere" architecture.

So we will pursue several options simultaneously, most likely.

By 'we' I mean me for now, but my team is interested too - also She Who Must Be Obeyed.

Unfortunately, we have production dev/ops to support as well, and a feature pipeline, so its a delicate dance :)

On Wed, Dec 10, 2014 at 9:43 AM, Oegma2 <oegma2@xxxxxxxxx> wrote:
Hi Michael

Have you guys looked at RabbitMQ's MQTT plugin as a gateway/broker?


On Thu, Dec 4, 2014 at 2:53 PM, Laing, Michael <michael.laing@xxxxxxxxxxx> wrote:
Briefly, nytfabrik is a global message bus, meshed across multiple AWS regions/zones using RabbitMQ, with a single, pervasive, eventually consistent memory provided by Cassandra.

The client-facing node groups autoscale, each node (gateway) handling 10-100K subscribers.

We are exploring implementing the gateways as MQTT brokers to augment or replace our custom code, expand the reach of nytfabrik to more devices, and standardize the interaction with them.

The mosquitto code base is one option. We are also in discussion with the RabbitMQ folks. We will pursue several options.

An efficient, scalable Cassandra implementation for retained messages was an identifiable hurdle, now solved for our use case IMHO.

Others may be happy with SQL- or other NOSQL-based solutions, hence a flexible approach to a retained message store may be useful.

Cheers,

ml

On Thu, Dec 4, 2014 at 4:24 AM, Oegma2 <oegma2@xxxxxxxxx> wrote:
Hi Michael

This is really amazing "options" you have and I am also trying different angles with MQTT 

Have you looked at using Erlang  with streams, queries, response....
Wonder if Mosquitto can be mapped to Erlang - meaning, Devices-->Mosquitto-->Erlang (streams/workers)..... uhm

Sorry for asking, but you have RabbitMQ ...where do Cassandra fit into all this? Is the RabbitMQ not keeping the message at this stage for you?

I myself is busying looking a pattern CQRS and Event-sourcing , with MQTT...and funny enough - looking at RabbitMQ, Erlang but still pondering on the event-sourcing store :) keeping scaling in mind and security - also looking at Kafka and couple other options for scaling MQTT - even geolocation dns loadbalancing :)

Any case, this is really exiting stuff Michael and i can definitely see a overlapping of domain problems - my biggest issue at this stage is to scale the "broker" to handle the incoming MQTT messages....





On Thu, Dec 4, 2014 at 12:33 AM, Laing, Michael <michael.laing@xxxxxxxxxxx> wrote:
Just an update.

I have increased the wildcard level span to 9 by combining inversion and indexing strategies. It is still very fast and retrieves all results in a single query.

To scale, it requires one or 2 more strategies to increase the cardinality of the partition keys - the 'row' in the big table model. In the current nytfabrik, we use the 0 level as an index into metadata that lets us do this, in effect moving some levels into the partition key and, sometimes, a rarely wildcarded level as a hash into a partition key shard.

For contrast, I have done the same thing using Postgres - much easier, of course, as the db does the query planning. Also fast.

But now I am thinking about how to implement these in mosquitto. They could be embedded, but maybe it would be more flexible to use something like zmq to asynchronously stream queries and responses. Then I could do my Cassandra and Postgres backends in python - others may want to do mongo or redis, etc., using their language of choice.

Cheers,

Michael

On Mon, Dec 1, 2014 at 2:07 AM, Oegma2 <oegma2@xxxxxxxxx> wrote:
Hi Michael

I will definitely be interested to hear more ^_^

On Thu, Nov 27, 2014 at 10:43 AM, Roger Light <roger@xxxxxxxxxx> wrote:
Hi Michael,

I'd definitely be interested to hear more.

Regards,

Roger


On Wed, Nov 26, 2014 at 3:23 PM, Laing, Michael
<michael.laing@xxxxxxxxxxx> wrote:
> I am experimenting with MQTT as a way to extend the nytfabrik at the edge,
> complementing and perhaps replacing our custom WebSocket/SockJS protocol.
>
> We use RabbitMQ with topic routing on the nytfabrik global backbone mesh, so
> we are familiar with wildcarding, subscription, etc.
>
> And we have lots of 'retained' messages - 10's of millions growing to 100's
> of millions.
>
> This is no problem for Cassandra but, as we look to support MQTT, we need a
> technical solution for query with wildcards on retained messages.
>
> Cassandra has a limited query language but can be very fast. I was looking
> for a way to run 'close to the metal' (no additional layers) and now have a
> working prototype architecture and algorithms for store and query that are
> simple, fast and which scale. So I think that technical blocker is overcome
> for our usage cases.
>
> Anyway, if this is of interest, I'll keep this list informed, and maybe
> solicit some advice.
>
> We're not sure yet how we'll add MQTT to the nytfabrik, but most likely we
> would remix mosquitto with our own code, package it up, and run it under
> python on our autoscaling edge instances.
>
> Cheers,
>
> Michael
>
> _______________________________________________
> mosquitto-dev mailing list
> mosquitto-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/mosquitto-dev
_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev


_______________________________________________
mosquitto-dev mailing list
mosquitto-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/mosquitto-dev

Back to the top