Re: [paho-dev] MQTT <-> REST+websocket mapping

["Nicholas O'Leary" <nick.oleary@xxxxxxxxx>]

Clearly someone was busy at eclipse, and they got the wiki working in
the time it took me to write my reply.

Here's the link for the MQTT/Websockets stuff -
http://wiki.eclipse.org/Paho/Paho_Websockets

On 15 March 2013 15:22, Nicholas O'Leary <nick.oleary@xxxxxxxxx> wrote:
> Hi Matteo,
>
> A couple thoughts and links for your first two questions:
>
>> 1) The websocket tunneling issue is longstanding in this community. Some
>> previous attempt has been done, and there was even a proposal for routing
>> binary MQTT inside a websocket tunnel.
>> I believe that the best approach to make this work is to map MQTT to JSON in
>> a standard way, and then route it inside websocket & polyfills (es.
>> socket.io or sock.js and the like)
>> This will simplify implementations and diffusion.
>
> There is already a proposal for using MQTT with Websockets on the Paho
> wiki (which I would link to if the entire eclipse wiki wasn't down
> right now). It takes the approach of tunnelling the MQTT binary
> packets through the websocket.
> This is the approach taken by the Mosquitto websocket implementation afaik.
>
> In my opinion, mapping it to a JSON payload across the websocket
> (which is what we did with earlier versions when browser support for
> ArrayBuffer etc was not as wide spread as it is today) means you lose
> the benefit of the light-weight packet size over the network - one of
> the core principles of the protocol.
>
>> 2) While creating an interface for publishing messages is easy, the core
>> issue is on reading them.
>> For example, ActiveMQ uses a DELETE to read a message, which can timeout if
>> no message is published in a given period of time. This approach is not
>> really web-compatible,
>
> can you explain what you mean by "not really web-compatible". DELETE
> is absolutely the right verb to use if you request a resource that as
> a side-effect removes it from the server. From javascript you can
> issue a DELETE XHR request just as easily as any of the other verbs.
>
>> so QEST adopt a standard GET approach, exposing to
>> the web every published message (here you can find is my home temperature:
>> http://qest.me/topics/temp). Every message is stored on a Redis database.
>> However, exposing the last message on the web is extremely similar to the
>> retained message behavior. In QEST, I did assimilate the two concepts
>> stating that every message is retained, even on MQTT.
>> At the moment, I am revising this decision to be MQTT-compliant. Still, the
>> web behavior needs to be clear. One possible solution is to expose on the
>> web only retained messages, or ignore the 'retained' flag at the web level
>> and expose everything.
>
> You cannot only expose retained messages and call that MQTT-compliant.
>
> A possible suitable approach would be: When an http-client issues its
> first 'subscribe' to a topic, return any retained message on that
> topic. Then, as you would with an MQTT non-clean session client,
> subscribe to the topic in the MQTT-sense for them and start queuing up
> messages for that client. Subsequent requests from that client will
> then consume messages off that client-specific delivery queue. A set
> of custom HTTP headers could be defined to say how long that
> subscription should be maintained for before discarding the queued up
> messages.
>
> It is worth noting this is well trodden ground. A few years ago, we
> released a support pack for WebSphere MQ that provided an HTTP
> interface. This isn't explicitly MQTT as it covers queues as well, but
> it did examine a lot of these issues. This has since been rolled into
> the base product, but you can see the original support pack
> documentation here to see how they defined the interface
> http://www-01.ibm.com/support/docview.wss?rs=171&uid=swg24016142.
> Again - this was for a more complex messaging system than MQTT in
> terms of options and properties, but the basic concepts are there. I
> should add, I don't like how they chose to do subscriptions in the
> support pack - they went for the 'do a request with a timeout and only
> deliver a message that happens to arrive in that period' approach. But
> interesting background material.
>
>
> Regards,
> Nick
>
>
>
> On 15 March 2013 14:57, Matteo Collina <hello@xxxxxxxxxxxxxxxxx> wrote:
>>
>> Hi Everyone,
>>
>> I am working on a web adapter for MQTT, and in general to the whole Internet
>> of Things and M2M worlds. My preliminary work is available at:
>> http://qest.me/.
>> It seems everyone is working on this problem, however there is not a shared
>> effort on it, and it is time to reignite the discussion.
>>
>> I am identifying the requirements for a general solution to this problem:
>> 1) Web-tunnelling of MQTT to achieve compatibility with web apps..
>> 2) REST interface that must enable to read and write the last message from a
>> topic.
>> 3) REST interface to read the history of the messages on a topic/query.
>> 4) No modification to MQTT and pluggable to any MQTT server.
>> 5) Security & Access Control Systems.
>> 6) Support for webhooks, to ease the integration in other server-side
>> systems.
>>
>> In order to clarify more, I will expand some points.
>>
>> 1) The websocket tunneling issue is longstanding in this community. Some
>> previous attempt has been done, and there was even a proposal for routing
>> binary MQTT inside a websocket tunnel.
>> I believe that the best approach to make this work is to map MQTT to JSON in
>> a standard way, and then route it inside websocket & polyfills (es.
>> socket.io or sock.js and the like)
>> This will simplify implementations and diffusion.
>>
>> 2) While creating an interface for publishing messages is easy, the core
>> issue is on reading them.
>> For example, ActiveMQ uses a DELETE to read a message, which can timeout if
>> no message is published in a given period of time. This approach is not
>> really web-compatible, so QEST adopt a standard GET approach, exposing to
>> the web every published message (here you can find is my home temperature:
>> http://qest.me/topics/temp). Every message is stored on a Redis database.
>> However, exposing the last message on the web is extremely similar to the
>> retained message behavior. In QEST, I did assimilate the two concepts
>> stating that every message is retained, even on MQTT.
>> At the moment, I am revising this decision to be MQTT-compliant. Still, the
>> web behavior needs to be clear. One possible solution is to expose on the
>> web only retained messages, or ignore the 'retained' flag at the web level
>> and expose everything.
>>
>> 3) An History interface has to be provided. The best work done in that area
>> is Cosm.
>> However, a standard representation of message meta-data has to be defined.
>>
>> 5) There is much discussion on how to expose M2M and IoT to end users.
>> Exposing it to the web is a key point, and doing it a safe way for both
>> users and devices is extremely important.
>> In particular, both the provisioning of devices and the pairing between
>> users and devices must be addressed.
>> This will necessarily lead to a paring protocol on top of both MQTT and
>> HTTP, and might include an OAuth step.
>>
>> Have you got some suggestions on these topics?
>>
>> Cheers,
>>
>> Matteo
>>
>>
>> _______________________________________________
>> paho-dev mailing list
>> paho-dev@xxxxxxxxxxx
>> http://dev.eclipse.org/mailman/listinfo/paho-dev
>>