RE: [hyades-dev] XML-based command data, binary data, pipelines

["Kaylor, Andrew" <andrew.kaylor@xxxxxxxxx>]

I thought there might have been two different "binary door" ideas
floating around.  If we take the base-protocol to XML, I doubt it will
ever go back to binary.

Is there a reason that we can't send the binary data on the data channel
and keep everything XML on the control channel?  I know that just moves
the problem (and we haven't talked about what's going on with the data
channel yet), but at least it gives us one clean channel.  

Otherwise, I suppose I can live with an optional binary blob tacked onto
our XML block.

I don't understand why you think the HCE would ever need to know the
internal parameters of the commands.  Whether we're using binary or XML,
we can have a part of it that the HCE just doesn't crack open, right?

I don't think the format of the data going into the HCE is something
that needs to be replaceable.  The job of the pipeline up to that point
is to get it into the expected format.  The end-point, though, needs to
be fixed.

If the HCE doesn't get the command in XML, then we'll end up with a
scenario where a client-to-agent message will look like this: (1) The
client converts its internal representation of the command (probably
binary) to XML, (2) The incoming pipeline converts XML into binary, (3)
the outgoing pipeline converts the binary back to XML (to send it to the
agent), (4) the agent converts the command to its internal
representation (probably binary).  There's a lot of unnecessary
conversion going on there.  But if the HCE takes the command as XML, we
can eliminate (2) and (3).

-Andy


-----Original Message-----
From: hyades-dev-admin@xxxxxxxxxxx [mailto:hyades-dev-admin@xxxxxxxxxxx]
On Behalf Of Allan K Pratt
Sent: Wednesday, September 22, 2004 12:33 PM
To: hyades-dev@xxxxxxxxxxx
Subject: RE: [hyades-dev] XML-based command data, binary data, pipelines

Harm wrote: "I would vote yes to having XML and the prime vehicle and 
leaving the binary door open but not used by any Hyades tools."

Let's talk about the "binary door" for a second.

I think the "binary door" that Andy sees is that the whole
communications 
pipeline can be replaced. Victor and I have talked this over, and that's

not what Victor means by it. Maybe not Harm either.

First, why talk about a binary door at all? Answer: to satisfy the 
requirement (which I now see isn't captured) that a low-overhead path 
exist from client to agent for bulk binary data. This data is expected
to 
include Java classes, test drivers, data pools, and even whole 
applications.

This data might never reside in a file, so the file transfer requirement

doesn't already satisfy this. We can back our way in by telling users, 
"Put it in a file and use file transfer," but this could be
inconvenient. 
It's all about tradeoffs.

One way to get binary data through the XML command path is to use base64

encoding or similar. This has space (1.4x) and encode/decode memory and 
time costs.

Another way is to split the payload into two parts: an XML part and a 
(possibly empty) binary part. Most commands wouldn't use the binary
part, 
but it would be there for those that wanted it. We still get the
benefits 
of the XML part, but we also get the performance (in time, space, and 
bandwidth) of un-encoded binary data for those situations where it's 
desired. (Note that an HTTP transport system will have to base64 encode 
the binary part after all, but at least the TCP/IP transport won't have 
to.)

As for "replacing the whole communications pipeline," I'm not sure quite

what that means any more. We drew boxes with a "command extractor" on
the 
right and an arrow down from there into the HCE. What exactly is that 
interface? At one time we said "it's the binary data represented as 
structs" but I don't know if that's true any more. Does the HCE have a
big 
"union" somewhere that knows the internal parameter data for every 
command? I think not. Does the command extractor use DOM parsing to
create 
a structured representation of the XML? I didn't think we wanted that. 
Does it pass the raw XML without processing it? Maybe. Now I don't know 
what that looks like any more. And remember, that's one end of the 
pipeline we're supposed to be able to replace.

So: if the mechanism for binary data is to use a whole different
pipeline, 
what does the end of that pipeline look like? And if not, what about an 
optional binary blob following the XML?

-- Allan Pratt, apratt@xxxxxxxxxx
Rational software division of IBM

_______________________________________________
hyades-dev mailing list
hyades-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/hyades-dev