RE: [dsdp-tcf-dev] Applicability of TCF agent for remote tracing

["Burton, Felix" <Felix.Burton@xxxxxxxxxxxxx>]

Hi Michel,

This is an important use case that has been considered.  We have
identified several cases where there is a need for a transparent high
performance stream to transfer of data to and from a target while using
TCF for the control aspect of the data.  

The easiest and fastest solution would probably be to open a new TCP/IP
channel for the data part, but this has several drawbacks:

- It only works only if the connection between the host and the target
supports TCP/IP 
- It opens additional routing, firewall and security issues, i.e. some
customers want to protect the production network from the board lab by
using a firewall and in such cases only the host could initiate the
connection.

Because of these drawbacks we implemented a general service to
efficiently stream binary data over the TCF channel, see
streamservice.[ch] in the TCF prototype (unfortunately we don't have any
documentation for this service this yet).  Basically what this service
does is to creation of arbitrary number of byte streams over a TCF
channel.  The commands a designed to minimize round trips and allow
multiple concurrent read and write commands can be issued asynchronously
so there are no delays between commands.

The stream service is using standard encoding format, so there is some
overhead as compared to a pure TCP/IP stream, however, there are plans
to add support for raw and compressed formats for transferring binary
data to TCF by extend the JSON format when both peers supports it.  The
message header overhead can also be minimized by reading/writing larger
chunks of data.  

Overall, I believe this will performance very well.  Of course it can
never reach as good performance as the underlying transport, so when the
data rate is really high some alternative approach which might be
supported.

Thoughts?

Best Regards,
Felix

> -----Original Message-----
> From: dsdp-tcf-dev-bounces@xxxxxxxxxxx [mailto:dsdp-tcf-dev-
> bounces@xxxxxxxxxxx] On Behalf Of Michel Dagenais
> Sent: Friday, May 08, 2009 2:07 PM
> To: dsdp-tcf-dev@xxxxxxxxxxx
> Subject: [dsdp-tcf-dev] Applicability of TCF agent for remote tracing
> 
> Hi, we are in the process of revamping our tracing daemon (lttd in
> LTTng
> project at lttng.org). The new daemon should map tracing buffers from
> the Linux kernel and from several traced user space applications and
> receive notification when a buffer is ready (buffer full or maximum
> time
> elapsed between flushing of data). It should then either write the
> tracing data to files or send the binary tracing data efficiently over
> the network to a tracing client. The tracing client should also be
able
> to list, enable and disable the available tracepoints in the kernel
and
> in the applications.
> 
> Enabling and disabling tracepoints is not high volume and thus
> performance sensitive. The JSON ascii format is fine. The binary
> tracing
> data must not be encoded in any way, as this would be an unacceptable
> overhead for the traced system (using base 64 encoding, copying to the
> stream buffer...).
> 
> What do you suggest to send the content of tracing buffers "zero copy"
> to the remote client? There could be a command to open a new TCP
> connection and use that connection. Is there a more integrated way,
for
> instance getting some raw TCP channel for the tracing data?
> 
> _______________________________________________
> dsdp-tcf-dev mailing list
> dsdp-tcf-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/dsdp-tcf-dev