Re: [linuxtools-dev] LTTng/TMF live trace streaming support plan?

[Aaron Spear <aspear@xxxxxxxxxx>]

Hi Alexandre,

Thanks for the helpful info, some more dialogue below:

> Since LTTng is our reference implementation, our plan was to wait until
> live-reading was supported in LTTng itself (at least in beta form)
> before starting to work on supporting it in TMF. From what I hear,
> live-reading will be part of LTTng 2.3 or 2.4 (2.2 is the version
> currently in development).
> 
> Right now we are working on connecting to a locally-running relay
> daemon, so that one can start a trace on a remote system and bring it to
> their machine. But only post-mortem reading is possible, for now. That's
> probably all that will be in the Kepler release, next June.
> 
> We did talk about eventually re-implementing that relayd protocol in the
> Eclipse plugin itself. This would allow someone to start a trace on a
> remote system, and look at it live in TMF (and *optionally* save it to
> disk), without needing a local, arch-specific relayd running. Which is
> pretty cool! But that's still a long way off...

Yes, very cool! 
 
> If you would like to contribute to making it happen faster, the LTTng
> guys could probably use a hand in defining/testing the live-reading
> spec. Or we, on the Eclipse side, could always use help in implementing
> the relayd-in-Eclipse. Pick your poison ;)

I will likely do both.  It seems to me that the ideal solution would be able to support connecting to a LTTng relayd (and speak that protocol), but it also seems to me that ideally this layer that does the communication would be pluggable so that I could accomplish my use case as well, which is speaking some different protocol for events, but translating those to CTF events on the fly.

I tend to think that this pattern could be generally useful for a number of things.  Consider a use case where you wanted to follow a *remote* log file extracting events from it, i.e. "ssh root@xxxxxxxxx tail -f /path/to/file.log"  will read all lines in the remote log file, and then wait for any new lines as they are appended to the file.  Instead of an implementation like the current  org.eclipse.linuxtools.internal.tmf.ui.parsers.custom.CustomTxtTrace, which does nice parsing of static log files in the local file system, why not have a possible implementation which has a provider interface for the events, and then it persists those events optionally as CTF (the underlying "trace" would be CTF, and thus "rank" and ITmfLocation would be CTF file values).  So then there can be an interface used to provide the parser with a block of lines, which it parses to events and then it passes those events to a model implementation which persists to CTF and then also notifies the GUI that new events are available. (Maybe this is just a decorator on CtfTmfTrace?)

I will be doing some experimenting with this, so I will keep my eyes open for where there is a nice opportunity for reuse here, and am very interested in the thoughts of anyone else as well (especially if they are more brilliant (meaning simpler!) than those above :-)

Aaron Spear - VMware