[m2m-iwg] Xively - was Re: MQTT <-> REST+websocket mapping

[UOMo <uomo@xxxxxxxxxxx>]

Rick/all,

Following our discussion about Pachube/Cosm a few months ago, some of you may have heard, it's been renamed once again into "Xively" (not sure, why they chose a name even harder to pronounce than "Pachube"?) Having learned from problems like the one you mentioned or missing unit cases I outlined, Xively not only supports a variety of languages:

https://xively.com/dev/libraries/ (no Lua right now, sorry, at least they don't seem to consider it as important as Java, Android, Python or Ruby)

Among those at least the Java, Python and Ruby API include Unit System awareness based on the old Pachube-driven EEML http://www.eeml.org/#specification It's not as strict as SensorML, UnitsML or UCUM, but at least knows the unit and type of unit, so it'll distinguish between 

Unit celsius= new Unit();

celsius.setSymbol("C");

celsius.setLabel("Celsius");

celsius.setUnitType(DERIVED_SI);

and

Unit fahrenheit= new Unit();

fahrenheit.setSymbol("F");

fahrenheit.setLabel("Fahrenheit");

fahrenheit.setUnitType(CONVERSION_BASED_UNITS);

I find the comment to deprecate the UnitType a little disturbing, though, if at least Celsius and Fahrenheit can be told apart that's already a giant leap compared to so many other M2M and IoT solutions not even bothering to distinguish, or just calling it "Degree"

Especially since full scale ICU4J could be quite heavy on small devices, I'll explore a fork of the Java client to see, how it may benefit from implementing Unit-API. 

Have fun in France those who are there,

-- 

Werner Keil | JCP Executive Committee Member | Eclipse UOMo Lead, Babel Language Champion | Java Godfather

Twitter @wernerkeil | @JSR354 | #EclipseUOMo | #Java_Social | #OpenDDR

Skype werner.keil | Google+ gplus.to/wernerkeil

* Eclipse DemoCamps Kepler 2013: June 19-28 2013, Germany, Denmark, Austria. Werner Keil, UOMo Lead, Mærsk DevOps Build Manager will present "Triple-E’class DevOps with Hudson, Maven, Kokki, Multiconf & PyDev", "M4M 2 the Rescue of M2M"

On Mon, Mar 18, 2013 at 5:54 PM, UOMo <uomo@xxxxxxxxxxx> wrote:

Rick,

Thanks for the quick reply. I can very well imagine, and I'm sure, some people were probably confronted with Radiation Levels based on a "Japanese Imperial System"?

The way this mailing list processed the thread and special characters like "Degree" sign shows quite drastically, that a practice by a few Cosm/Pachube feeds I once saw to exchange such special characters or take them directly as URL arguments can also be tricky.

While the Slovenian example I gave still has the best Human readable rendering IMHO, the one from Portland saying "Deg C" feels a bit like a UCUM-inspired representation of Celsius, except the unique UCUM code for Degrees Celsius is "Cel", that for Degrees Fahrenheit "degF". Don't ask me why, but they are all unique, that's the most important thing about UCUM. Each code maps to a "print" representation, the nice one with graphical Unicode special characters, but only the use of either case-sensitive or ALL-UPPERCASE UCUM code is encouraged for data exchange of any kind and format.

One of the reasons why I suggested the Lua side of this IWG might consider looking at a UCUM parser/processor, similar to that, UOMo UCUM alraedy has for Java.

Werner 

Message: 2
Date: Mon, 18 Mar 2013 16:41:14 +0000
From: Rick Bullotta <rick.bullotta@xxxxxxxxxxxxx>

To: m2m Industry Working Group <m2m-iwg@xxxxxxxxxxx>

Subject: Re: [m2m-iwg] MQTT <-> REST+websocket mapping
Message-ID:

        <C7D31C647F791B439B6E65E10053D5370D99204E@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>

Content-Type: text/plain; charset="iso-8859-1"

Your point is well taken.

This was actually a problem during the Fukushima disaster.  It was awesome how the community rallied and leveraged Pachube to provide visibility to radiation data, but the inconsistencies in measurement units were a challenge.  It made it difficult to reliably interpret and act upon the data.


From: m2m-iwg-bounces@xxxxxxxxxxx [mailto:m2m-iwg-bounces@xxxxxxxxxxx] On Behalf Of UOMo
Sent: Monday, March 18, 2013 12:36 PM
To: m2m-iwg@xxxxxxxxxxx

Subject: Re: [m2m-iwg] MQTT <-> REST+websocket mapping

Matteo/all,

Thanks for the updates. Probably have to see what's discussed on the Paho mailing list, too.

2) I tried that URL and at first it it empty. When you click "Edit" and enter any sort of value, like 56 or 32, it keeps coming back with "18" as a result. Given the different input, I assume it isn't temperature conversion, but I would have hoped to see that "18" in the first place. And while one may guess that your home in Italy is measuring temperature in Degree Celsius, in the World Wide Internet of Things this isn't obvious at all.
A classical example of Units that must be taken into consideration.

3)
See above, I had a few discussions with the makers of Cosm when it was still called Pachube, as their datafeeds vary a lot. Some have decent handling of measurement values and units, e.g.

https://cosm.com/feeds/40318
Energy, wind speed and temperature

none of them with a proper unit[cid:image001.gif@01CE23D5.E0649F20]

A feed from Belgium: https://cosm.com/feeds/66151

says "?" but is it Celsius, Fahrenheit or Rankine?[cid:image002.gif@01CE23D5.E0649F20]

This one has no data feed, but the temperature unit is said to be "c":
https://cosm.com/feeds/63802

Not standard compliant, but at least gives a vague idea it seems Celsius[cid:image003.gif@01CE23D5.E0649F20]

This air quality Egg from Portland, OR
https://cosm.com/feeds/81913
shows "Deg C" among other units, even a bit better.
And this one from Slowenia
https://cosm.com/feeds/96652

wins the "World Cup", not just of Alpine Skiing, by properly declaring "?C"[cid:image004.gif@01CE23D5.E0649F20]


If any of these values shall make sense in a Global M2M environment, metadata must contain units of measurement, and a unit or code system. That could be ISO, USCustomary, Imperial, UCUM or whatever, as long as both sides can adjust to the right system to understand each other[cid:image005.gif@01CE23D5.E0649F20]

5) Cosm uses OAuth 2.0, at least there is a Beta trial, so is for MQTT btw, although it says, it may go away with no further warning[cid:image006.gif@01CE23D5.E0649F20] https://cosm.com/docs/beta/mqtt/

Cheers,
Werner

On Mon, Mar 18, 2013 at 5:00 PM, <m2m-iwg-request@xxxxxxxxxxx<mailto:m2m-iwg-request@xxxxxxxxxxx>> wrote:
Send m2m-iwg mailing list submissions to

        m2m-iwg@xxxxxxxxxxx<mailto:m2m-iwg@xxxxxxxxxxx>

To subscribe or unsubscribe via the World Wide Web, visit
        http://dev.eclipse.org/mailman/listinfo/m2m-iwg
or, via email, send a message with subject or body 'help' to

        m2m-iwg-request@xxxxxxxxxxx<mailto:m2m-iwg-request@xxxxxxxxxxx>

You can reach the person managing the list at

        m2m-iwg-owner@xxxxxxxxxxx<mailto:m2m-iwg-owner@xxxxxxxxxxx>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of m2m-iwg digest..."


Today's Topics:

   1. Re: MQTT <-> REST+websocket mapping (Peter Niblett)


----------------------------------------------------------------------

Message: 1
Date: Mon, 18 Mar 2013 14:22:48 +0000

From: Peter Niblett <peter_niblett@xxxxxxxxxx<mailto:peter_niblett@xxxxxxxxxx>>
To: m2m Industry Working Group <m2m-iwg@xxxxxxxxxxx<mailto:m2m-iwg@xxxxxxxxxxx>>
Cc: m2m-iwg-bounces@xxxxxxxxxxx<mailto:m2m-iwg-bounces@xxxxxxxxxxx>,        m2m Industry Working Group
        <m2m-iwg@xxxxxxxxxxx<mailto:m2m-iwg@xxxxxxxxxxx>>,  Dave Locke <locke@xxxxxxxxxx<mailto:locke@xxxxxxxxxx>>

Subject: Re: [m2m-iwg] MQTT <-> REST+websocket mapping
Message-ID:

        <OFCE2747EE.F0DFBAD0-ON80257B32.004E10EB-80257B32.004F0233@xxxxxxxxxx<mailto:OFCE2747EE.F0DFBAD0-ON80257B32.004E10EB-80257B32.004F0233@xxxxxxxxxx>>

Content-Type: text/plain; charset="us-ascii"

Hi Matteo

I think there's been some parallel discussion of your first point on the
Paho mailing list. I am curious about your remarks about tunnelling MQTT
over websockets.  Since RFC6455 is a binary protocol, it seems a more
natural fit to run binary MQTT over it. That way
a) We avoid having to specify and standardise a new version of MQTT
b) Applications that are operating under tight bandwidth constraints can
still do so, without suffering the increase in message size that JSON
would imply

I'm only talking about the headers here. If you want to use JSON to encode
the application data (i.e. send JSON in an MQTT message) that's fine, and
of course you might well want to do that even when running MQTT directly
over TCP/IP.

Regards


Peter Niblett
IBM Senior Technical Staff Member
Member of the IBM Academy of Technology

+44 1962 815055<tel:%2B44%201962%20815055>
+44 7825 657662<tel:%2B44%207825%20657662> (mobile)




From:   Matteo Collina <matteo.collina@xxxxxxxxx<mailto:matteo.collina@xxxxxxxxx>>
To:     m2m Industry Working Group <m2m-iwg@xxxxxxxxxxx<mailto:m2m-iwg@xxxxxxxxxxx>>,

Date:   03/15/2013 02:53 PM
Subject:        [m2m-iwg] MQTT <-> REST+websocket mapping

Sent by:        m2m-iwg-bounces@xxxxxxxxxxx<mailto:m2m-iwg-bounces@xxxxxxxxxxx>

Hi Everyone,

I am working on a web adapter for MQTT, and in general to the whole
Internet of Things and M2M worlds. My preliminary work is available at:
http://qest.me/.
It seems everyone is working on this problem, however there is not a
shared effort on it, and it is time to reignite the discussion.

I am identifying the requirements for a general solution to this problem:
1) Web-tunnelling of MQTT to achieve compatibility with web apps..
2) REST interface that must enable to read and write the last message from
a topic.
3) REST interface to read the history of the messages on a topic/query.
4) No modification to MQTT and pluggable to any MQTT server.
5) Security & Access Control Systems.
6) Support for webhooks, to ease the integration in other server-side
systems.

In order to clarify more, I will expand some points.

1) The websocket tunneling issue is longstanding in this community. Some
previous attempt has been done, and there was even a proposal for routing
binary MQTT inside a websocket tunnel.
I believe that the best approach to make this work is to map MQTT to JSON
in a standard way, and then route it inside websocket & polyfills (es.

socket.io<http://socket.io> or sock.js and the like)

This will simplify implementations and diffusion.

2) While creating an interface for publishing messages is easy, the core
issue is on reading them.
For example, ActiveMQ uses a DELETE to read a message, which can timeout
if no message is published in a given period of time. This approach is not
really web-compatible, so QEST adopt a standard GET approach, exposing to
the web every published message (here you can find is my home temperature:
http://qest.me/topics/temp). Every message is stored on a Redis database.
However, exposing the last message on the web is extremely similar to the
retained message behavior. In QEST, I did assimilate the two concepts
stating that every message is retained, even on MQTT.
At the moment, I am revising this decision to be MQTT-compliant. Still,
the web behavior needs to be clear. One possible solution is to expose on
the web only retained messages, or ignore the 'retained' flag at the web
level and expose everything.

3) An History interface has to be provided. The best work done in that
area is Cosm.
However, a standard representation of message meta-data has to be defined.

5) There is much discussion on how to expose M2M and IoT to end users.
Exposing it to the web is a key point, and doing it a safe way for both
users and devices is extremely important.
In particular, both the provisioning of devices and the pairing between
users and devices must be addressed.
This will necessarily lead to a paring protocol on top of both MQTT and
HTTP, and might include an OAuth step.

Have you got some suggestions on these topics?

Cheers,

Matteo_______________________________________________
m2m-iwg mailing list

m2m-iwg@xxxxxxxxxxx<mailto:m2m-iwg@xxxxxxxxxxx>

http://dev.eclipse.org/mailman/listinfo/m2m-iwg


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/07327e5b/attachment.html>

------------------------------

_______________________________________________
m2m-iwg mailing list

m2m-iwg@xxxxxxxxxxx<mailto:m2m-iwg@xxxxxxxxxxx>

http://dev.eclipse.org/mailman/listinfo/m2m-iwg


End of m2m-iwg Digest, Vol 17, Issue 24
***************************************

-------------- next part --------------
An HTML attachment was scrubbed...

URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/33526d23/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 104 bytes
Desc: image001.gif
URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/33526d23/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 540 bytes
Desc: image002.gif
URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/33526d23/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.gif
Type: image/gif
Size: 186 bytes
Desc: image003.gif
URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/33526d23/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.gif
Type: image/gif
Size: 96 bytes
Desc: image004.gif
URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/33526d23/attachment-0003.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.gif
Type: image/gif
Size: 257 bytes
Desc: image005.gif
URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/33526d23/attachment-0004.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.gif
Type: image/gif
Size: 100 bytes
Desc: image006.gif
URL: <http://dev.eclipse.org/mailman/private/m2m-iwg/attachments/20130318/33526d23/attachment-0005.gif>

------------------------------

_______________________________________________
m2m-iwg mailing list
m2m-iwg@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/m2m-iwg

End of m2m-iwg Digest, Vol 17, Issue 26
***************************************

--

Werner Keil | Eclipse UOMo Lead, Babel Language Champion | Java Godfather

Twitter @wernerkeil | @JSR354 |#EclipseUOMo

Skype werner.keil | Google+ gplus.to/wernerkeil

* Eclipse Stammtisch: May 15 2013, Zurich, Switzerland. Werner Keil, UOMo Lead, JSR 360 EG Member will present "M4M 2 the Rescue of M2M"

* GeeCON: May 16-17 2013, Krakow, Poland. Werner Keil, JCP Executive Committee Member, JSR 360 EG Member will present "Standards for the Future of Java Embedded"

* GR8Conf Hackergarten: May 22 2013, Copenhagen, Denmark. Werner Keil, JCP EC Member, JSR 354 EG Member will present "Groovy/Grails - Best way for Money using JSR 354"