| [hyades-dev] Datacollection protocols and deadlock |
|
Mike and I are worried about the new
protocol specifications for the Hyades datacollection transport specification
with respect to distributed deadlock.
From what Andy said, agents will be sending around
asynchronous messages and eager readers on the agents' incoming streams are
expected to prevent deadlock. A system like this can still
deadlock.
Mike wants the Choreography Engine's
transport layer and the Datacollection Engine's transport
layer to converge; partly so that the Datacollection Engine can benefit
from the Choreography Engine's deadlock free transport layer and partly so that
there is the potential for both engines to use the same transport
layer specification.
I don't entirely understand what the flow of
communication will be in the Datacollection Engine and it may be that the message flows for the datacollection engine
mean the datacollection engine is not able to deadlock at points where the
choreography engine is. However, on the face of it it does sound like
there will be problems and the transport layer for the Choreography Engine is
deadlock free so picking it up guarantees we won't see these problems at
any point.
We need to discuss this and come to some agreement
on it before we start specifying layers too high up. I think that layer 0
will probably be unaffected as its only handshaking and simple point to point
messaging but we need to discuss this before we go too far with the layers
above.
There doesn't seem to be a lot of common knowledge
about these layers so I think probably the fastest way to progress this stuff is
for me to produce a document that explains the deadlock protection layers in
detail and then when we've discussed that we can start to come up with some
protocols for them.
For the moment though, here is a quick description
of the Choreography Engine's bottom layers:
-----------------------------------
point to point transaction layer (including buffer planes - Merlin and
Schweitzer's algorithm) -----------------------------------
finite-buffered (through remote acks) unidirectional
streams -----------------------------------
multiplexing (needed for acks above to not
interfere with normal messages)
e.g. data is sent on up + down
stream 1, acks are sent on up + down stream 2
-----------------------------------
point to point (e.g. socket
connection)
-----------------------------------
This description probably isn't very enlightening to most but I'll email
out the detailed document as soon as I can.
I'd appreciate any agreement / disagreement / feedback from anyone
participating in the data collection protocol discussion.
cheers
Antony
|