[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
RE: [hyades-dev] XML-based command data
|
I would vote yes to having XML and the
prime vehicle and leaving the binary door open but not used by any Hyades
tools.
Thanks for your time.
--------------------------------------------------------------------------
Harm Sluiman, STSM,
phone:905-413-4032 fax: 4920
cell: 416-432-9754
mailto:sluiman@xxxxxxxxxx
Admin : Gabrielle Chapman chapmaga@xxxxxxxxxx Tie: 969-2323
Victor Havin <vhavin@xxxxxxxxxx>
Sent by: hyades-dev-admin@xxxxxxxxxxx
09/21/2004 04:42 PM
Please respond to
hyades-dev |
|
To
| hyades-dev@xxxxxxxxxxx
|
cc
|
|
Subject
| RE: [hyades-dev] XML-based
command data |
|
Using XML as the primary format assumes there is an alternative format.
Do you want to reserve a possibility to send unformatted binary stream
through the command pipe?
With that in mind I vote 'yes' on XML.
--Victor
"Kaylor, Andrew"
<andrew.kaylor@xxxxxxxxx>
Sent by: hyades-dev-admin@xxxxxxxxxxx
09/21/2004 09:14 AM
Please respond to
hyades-dev |
|
To
| <hyades-dev@xxxxxxxxxxx>
|
cc
|
|
Subject
| RE: [hyades-dev] XML-based
command data |
|
We definitely need to account for large volumes of data, but my question
is whether we can use only the data channel for these large blocks and
assume that the control channel will only be used for small blocks. I
thought I understood Harm to be saying that there was some need for large
amounts of data on the control channel, but maybe I misunderstood.
I believe the issue with encoding the data into a standard XML stream is
the time it would take to perform the decoding (which is probably a serious
issue for the control channel). On the other hand, the problem with
not encoding the data is that non-encoded streams would be rejected by
IT filters in the firewall scenario.
Anyway, I feel like this is heading down a rabbit trail.
The real question I want to get answered is, should we pursue XML as our
primary format for the control channel?
I’d appreciate if everyone with an interest in this could drop a quick
‘yes’ or ‘no’ (just so I’m clear on where we are) and also voice their
particular concerns if applicable. We can address the particular
concerns as necessary on this list.
-Andy
-----Original Message-----
From: hyades-dev-admin@xxxxxxxxxxx [mailto:hyades-dev-admin@xxxxxxxxxxx]
On Behalf Of Victor Havin
Sent: Monday, September 20, 2004 6:30 PM
To: hyades-dev@xxxxxxxxxxx
Cc: hyades-dev@xxxxxxxxxxx; hyades-dev-admin@xxxxxxxxxxx
Subject: Re: [hyades-dev] XML-based command data
I think we all agree that scenarios involving large volumes of data should
be properly addressed.
I am not in a position to make recommendations as I have only limited experience
with XML and communication protocols in general. I just want to point out
that XML is not absolutely foreign to binary data transfer. For example,
SMIL standard that was introduced some time ago (appendix 1) deals explicitly
with transmission of large volumes of binary data over the Internet. Another
example is MIME multipart stream that can carry binary data along with
XML parts referencing it (appendix 2). XML itself is agile enough to carry
any kind of context, including unparsed data (appendix 3). If the concern
here is data inflation caused by binary stream encoding (eg. base64) then
we should address it explicitly. Given we are talking about protocol-independent
data streams we have to deal with encoding anyway. We can probably shift
this responsibility to payload normalizers, envelops and other internal
facilities. If we are ready to invest heavily into the internal infrastructure
because of the scalability requirements, then we can take this approach.
In general I see at least three possible outcomes here:
1. We follow the XML trail and hope it takes care of the most data marshalling
issues.
2. We use XML as the main command delivery standard, but leave the door
open for occasionally sending a raw binary blob. We do realize that this
blob requires special parsing on every potential target platform. This
blob should be properly marked and carry an envelope with detailed description
of data origin and format.
3. We always use proprietary binary format and just deal with it
on all target platforms.
The first approach promises maximum flexibility and probably the heaviest
overhead in simple 'peer-to-peer over socket' cases. The last one can be
very optimized at the expense of our long work hours. The second is in
between and I would personally vote for it.
--Victor
Appendix 1.
SMIL URL:
http://www.w3.org/TR/REC-smil/)
Appendix 2.
Multipart MIME document.
RFC2387:
http://www.faqs.org/rfcs/rfc2387.html
Example:
Content-Type:
multipart/related; boundary=--xxxxxxxxxx;
--xxxxxxxxxx
Content-Type: text/xml
Content-ID: Contents
<?xml version="1.0" ?>
<objectDef uid="?">
<property><name>Width</name>
<value><i4>1024</i4></value>
</property>
<property><name>Height</name>
<value><i4>1024</i4></value>
</property>
<property><name>BitCount</name>
<value><i4>16</i4></value>
</property>
<property><name>BitCount</name>
<value><i4>16</i4></value>
</property>
<property><name>Pixels</name>
<value><stream href="" /></value>
</property>
--xxxxxxxxxx
Content-Type: application/binary
Content-Transfer-Encoding: Little-Endian
Content-ID: Pixels
Content-Length: 524288
....binary data here...
--xxxxxxxxxx |
Appendix 3.
Example of binary substream in HCE message
<hce:message>
...
<hce:data hce:encoding="binary.base64">Z53815Zb82b</hce:data>
... |
</hce:message>
----------------------------------------------------------------------------------------------------------------------------
"Kaylor, Andrew"
<andrew.kaylor@xxxxxxxxx>
Sent by: hyades-dev-admin@xxxxxxxxxxx
09/20/2004 11:04 AM
Please respond to
hyades-dev |
|
To
| <hyades-dev@xxxxxxxxxxx>
|
cc
|
|
Subject
| [hyades-dev] XML-based command
data |
|
At last week's face-to-face we discussed the possibility of switching the
command data format for proposed HCE protocol from a binary, byte-stream-based
format to an XML-based format. Rather than make any hasty changes
to the current proposal, I wanted to air this explicitly on this mailing
list and give everyone (particularly those who weren’t present last week)
a chance to voice whatever concerns they might have.
I believe there was some consensus that this was a good idea so long as
we kept it to a simple description of the data (perhaps leveraging some
existing standard).
However, at some point on Thursday Harm raised the possibility that there
are scenarios in which it is necessary to send large amounts of binary
data on the command channel. I would have serious reservations about
using XML some of the time but occasionally switching to binary, but if
someone familiar with this case can propose something reasonable, I'm willing
to listen.
We agreed, I think, on Friday that this would apply only to the command
header and command data, not to the message envelope (which is specific
to the transport layer being used).
One concern I have is that if we use XML to describe command data, I want
to be sure we have very clearly defined rules for how and when additional
fields can be added, including the expected behavior if a command with
the new data is sent to an object that isn't expecting it (i.e. what are
the implications of ignoring the data) and the expected behavior if a command
without the data is sent to an object that is looking for it (i.e. what
is the default).
I don't want the interfaces to become amorphous.
Other comments?
Should we switch to XML-based command data?