RE: [dsdp-tcf-dev] Applicability of TCF agent for remote tracing

["Burton, Felix" <Felix.Burton@xxxxxxxxxxxxx>]

Hi Michael,

The only way we can accept contributions are through a patch attached to
an Eclipse bugzilla entry.

You will also need to write a legal message about license of the patch
following the template 

Legal Message: I, {name}, declare that I developed attached code from
scratch, without referencing any 3rd party materials except material
licensed under the EPL and EDL. {I am authorized by my employer to make
this contribution under the EPL and EDL.}

See
http://www.eclipse.org/dsdp/tm/development/committer_howto.php#external_
contrib

Hope that helps!

Thanks,
Felix

> -----Original Message-----
> From: dsdp-tcf-dev-bounces@xxxxxxxxxxx [mailto:dsdp-tcf-dev-
> bounces@xxxxxxxxxxx] On Behalf Of Michael Sills-Lavoie
> Sent: Thursday, May 28, 2009 7:29 AM
> To: DSDP TCF dev list
> Subject: Re: [dsdp-tcf-dev] Applicability of TCF agent for remote
> tracing
> 
> Burton, Felix wrote:
> > Hi Michael,
> >
> > See comments inline.
> >
> >
> >> -----Original Message-----
> >> From: dsdp-tcf-dev-bounces@xxxxxxxxxxx [mailto:dsdp-tcf-dev-
> >> bounces@xxxxxxxxxxx] On Behalf Of Michael Sills-Lavoie
> >> Sent: Wednesday, May 27, 2009 12:48 PM
> >>
> > ...
> >
> >> What we are using right now to transfer the tracing data for the
> >>
> > kernel
> >
> >> memory to a file in our local deamon is the splice system call from
> >> linux. This way, we can ask the kernel to do the transfer for us so
> we
> >> don't get any useless copy. It's what we wanted to use for the
> network
> >> transfer too to get the smallest possible overhead. I think this
> won't
> >> be easily feasible in tcf because we can't have access directly to
> the
> >> file descriptor or the socket, am I wrong? Maybe we could think of
> >> another way of doing it with mmap and bloc write to get something
> >> similar, but getting the file descriptor would probably be the most
> >> efficient way of doing it.
> >>
> >
> > In addition to a block write function we could add a
> > copy-block-from-file function which under the hood is using splice.
> I
> > have not used splice before, but the documentation states that one
> side
> > must be a pipe.  Do you know if a socket considered to be a pipe or
a
> > file from the splice system call point of view?
> >
> >
> 
>  From what I saw, the socket is considered to be a file. In order to
> use
> splice, you need to create a temporary pipe. For example, when you
want
> to transfer data from a file to another file (or socket), you create a
> pipe, you splice some data into it from the first file, then you
splice
> the data from the pipe into the other file. As you can see in the man
> pages, there is no actual copy of the data into the pipe, it's only
> used
> to tell the kernel what to do :
> 
> "Though we talk of copying, actual copies are generally avoided.  The
> kernel does this by implementing a pipe buffer as a set  of
> reference-counted  pointers  to pages  of  kernel  memory.   The
kernel
> creates "copies" of pages in a buffer by creating new pointers (for
the
> output  buffer)  referring  to  the  pages,  and increasing the
> reference counts for the pages: only pointers are copied, not the
pages
> of the buffer."
> 
> >> For the sending side of the agent, were you thinking of flushing
the
> >> write buffer stream (writing it to the socket) before every
> blocwrite
> >> of data and then sending the bloc directly?
> >>
> >
> > Yes, the block write function would write any pending data in the
> fifo
> > to the socket, preferable in such a way that it does not cause a
> package
> > to be sent to over the network.
> >
> >
> >> I also have the concern of how we are going to tell that a bloc of
> >> uncompressed data is in the message. I thought of inserting another
> >> escape sequence to let the reader know about the data (ESCcode size
> >> [data]). Because right now, the inputbuf is scanning the data for
> the
> >> end of message sequence (ibuf_read_done and ibuf_get_more). It
would
> >> also have to check for the "bloc escape sequence" and skip the
> binary
> >> data so there are some other changes that would need to be done
> there?
> >>
> >
> > Correct, the receiving side would have to check the a block marker
> and
> > skip any marker checks in that range.
> >
> 
> I'll work on the implementation of this in the next days and I'll
> submit
> my work to you when it's finished. Do you have any particular format
> that you want me to use for the changes?
> 
> Thank you,
> 
> Michael
> 
> > Thanks,
> > Felix
> >
> > _______________________________________________
> > dsdp-tcf-dev mailing list
> > dsdp-tcf-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/dsdp-tcf-dev
> >
> 
> _______________________________________________
> dsdp-tcf-dev mailing list
> dsdp-tcf-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/dsdp-tcf-dev