[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [dsdp-tcf-dev] Applicability of TCF agent for remote tracing
|
Hi,
What we are using right now to transfer the tracing data for the kernel
memory to a file in our local deamon is the splice system call from
linux. This way, we can ask the kernel to do the transfer for us so we
don't get any useless copy. It's what we wanted to use for the network
transfer too to get the smallest possible overhead. I think this won't
be easily feasible in tcf because we can't have access directly to the
file descriptor or the socket, am I wrong? Maybe we could think of
another way of doing it with mmap and bloc write to get something
similar, but getting the file descriptor would probably be the most
efficient way of doing it.
For the sending side of the agent, were you thinking of flushing the
write buffer stream (writing it to the socket) before every blocwrite of
data and then sending the bloc directly?
I also have the concern of how we are going to tell that a bloc of
uncompressed data is in the message. I thought of inserting another
escape sequence to let the reader know about the data (ESCcode size
[data]). Because right now, the inputbuf is scanning the data for the
end of message sequence (ibuf_read_done and ibuf_get_more). It would
also have to check for the "bloc escape sequence" and skip the binary
data so there are some other changes that would need to be done there?
Thank you
Michael
Burton, Felix wrote:
Hi Michael,
-----Original Message-----
From: dsdp-tcf-dev-bounces@xxxxxxxxxxx [mailto:dsdp-tcf-dev-
bounces@xxxxxxxxxxx] On Behalf Of Michael Sills-Lavoie
Sent: Monday, May 25, 2009 6:33 AM
...
Correct, it is separate from zero copy. It provides a general
purpose
way of streaming data over TCF. As Eugene pointed out it would be
fairly easy to do a general enhancement to the TCF channel stream to
allow zero copy of the data.
What would you suggest in order to implement zero copy transfer? How
would you do it?
The way Eugene and I discussed how to implement zero copy data transfer
is to:
1) Enhance OutputStream interface in to support an additional function
to write a block of data to the channel.
2) Enhance channel_tcp.c to implement the block write function such that
it adds it prefixes the block with a marker specifying the size of the
block so no escaping is necessary on the block. This way the whole
block can be given to the OS in a single operation. In the case of
Linux, the write is going to copy the data from user space to system
space, but for other OSes a true zero copy might be used.
3) Extend JSON to add ways to encode binary data instead of using a
base64 encoded strings. For zero copy we need to know how many bytes
will follow, but I would also add the capability to encode compressed
binary data of different formats.
I believe this should cover it for the sending side which I believe it
the primary concern, right?
Of course these are non-backwards compatible changes, so we need to have
a way for peers to negotiate support for this. The solution we have
discussed for this is to add a service name which indicates the TCF
protocol version, e.g. "tcf-2". If both peers announce this service,
then the new TCP channel marker and JSON extensions can be used,
otherwise the implementation would fall back to non-zero copy mode.
Thoughts?
Felix
_______________________________________________
dsdp-tcf-dev mailing list
dsdp-tcf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/dsdp-tcf-dev
