|Re: [m2m-iwg] General MQTT Protocol Question|
On 5/29/12 8:51 AM, Rick Bullotta wrote:
Hello Rick. We're all glad you're exploring MQTT and hopefully adopt it. I'll agree with Andy in suggesting the MQTT Google Groups as another resource on the MQTT protocol. I have a welcome posting there that explains some of the background requirements that we considered when MQTT was developed. The short list here is:
1 - Simple to implement.
2 - Provide a Quality of Service Data Delivery
3 - Lightweight and Bandwidth Efficient
4 - Data Agnostic
5 - Continuous Session Awareness
I trust we can answer your questions here and please post any more as they come up.
As noted, if a message is published with the "retain" flag set to true, the "Last Known Value" will be published as soon as you subscribe again. This was basically to retain state in a real-time use case knowing that new values would be arriving. MQTT is really all about the efficient delivery of data with known state and QoS. Historical views of the data beyond the last publish are probably better left to either the business logic running on the MQTT client or backend database architecture.
No, not from the "MQTT protocol" point of view. Business logic could be implemented to trigger a "re-publish" of some data from end devices, or the server infrastructure could be placing the historical payloads into a longer term persistent data store. (See my comments below on using Publish messages as information request triggers).
MQTT as a transport protocol was invented for its bandwidth efficiency. You can indeed place any data representation you want to into the payload. We've used XML, JSON, OBIX, and HL7 just to name a few. But all of these tend to be too verbose to stand the associated costs of remote/M2M communications systems. MQTT was first invented to run over very expensive VSAT systems where every byte cost. Even today, customers are trying to cram as much as possible into a 100K/mo cellular data plan. So everyone starts with a human readable payload, sees the communications bill over the next month, then drops back to packed binary :-) . Once on the server side where bandwidth is not a concern, the MQTT payloads can be transformed and re-published into much more verbose representations. I can say that I believe that the very fact that the MQTT payloads are data agnostic has been a huge advantage.
I know customers that have been running MQTT based SCADA systems for over 10 years with %99.999 availability. This usually involves 2-4 MQTT brokers in a cluster configuration. The typical number of data points (individual sensor readings) tend to range over the 25K to 50K per broker. In these use cases there were typically 400 to 600 connected MQTT clients. I know there are a lot of additional use cases out there.
The MQTT protocol itself does not dictate any particular implementation. But the nice thing about MQTT is that it was designed to take into account that VSAT/Cellular/PSTN Modems/Frame Relay are all unreliable transports. For reliable systems we usually have multiple communications paths and multiple brokers. If a MQTT client fails off of one broker it can just pick another comms path, re-connect, and pick up where it left off.
I've seen a LOT of different implementations of "Topic Dictionaries" that have been developed. In most cases, the namespace is built up within the infrastructure where consumer applications know what they are looking to subscribe to. As Andy has stated, this is probably outside the scope of the MQTT protocol and much more in the realm of an interesting backend service play. Remember that MQTT is a bi-directional creature. Clients can "Publish" messages which are really just triggers to cause another app in the infrastructure to "do something". In this case a client could "publish" a topic request message which is then consumed by another application (in this case a simple MQTT Topic Discovery service) that could publish the results back. This and many, many additional services could be envisaged and discussed within this forum.