Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [linuxtools-dev] FW: Java code for CTF trace writing?

Some notes on the trace synchronization in TMF

1) What I have implemented in patch https://git.eclipse.org/r/#/c/10869/ is a synchronization framework, with a first use case using Tcp packets matching in a lttng kernel trace where the tracer use a extra addon kernel module developed by Francis Giraldeau at Polytechnique (https://github.com/giraldeau/lttng-modules/tree/addons)

2) Once this feature is accepted in TMF, it would be easy to add event matching for different kinds of traces and synchronize them together. You would just need to put the traces in a experiment and synchronize the experiment

3) As long as you have events from a trace that can be matched to events from another trace, there is no need to synchronize clocks on both computers. As Michel said, the algorithm is very precise.

4) Synchronizing 2 machines with network packets will eventually be supported in the mainline lttng kernel tracer. Patch have been submitted on lttng-dev mailing list and are waiting for review. There will be no need for Francis's addon module.

5) The synchronization algorithm was designed to support live tracing, so when live tracing is supported in TMF, synchronization could be done on the fly.

Hope this helps,
Cheers,
Geneviève


On 05/07/2013 06:22 PM, Alexandre Montplaisir wrote:
CC'ing Geneviève, who has worked on the trace synchronization framework.
She can correct me if I'm completely wrong ;)


On 13-05-07 03:56 PM, Ostermueller, Erik wrote:
So that synchronization sounds like a manual process, right?
Some work has to be done first in the trace type definition. But once
you have the "recipe" for how to synchronize traces of type "JVM" with
traces of type "LINUX", if you open both in one experiment and click the
eventual Synchronize button, all the work should be done automatically.

Say that two JVM's take part in a single trace-able event, like a JSP request  on machine A to a SOAP server on machine B.
This is a garden variety scenario for us.  To include linux events, we'd than have four trace files:

JVM-A
LINUX-A
JVM-B
LINUX-B

So for sure, we'd want the manual synchronization process to handle more than just 2 files at one time.
But manually collecting four trace files, and potentially more, will definitely discourage use of the tracing tool.
Depending on the tracer used, all the traces could be put in the same
directory. Then we can use something like the new Batch Import to
automatically import everything under a given directory (automatic trace
type detection NYI).

We'll also need to keep the clocks of machine A & B in sync, right?
Here another type of synchronization could be used, for example there's
a reference implementation that uses TCP packets.
So first you can synchronize the -A traces together, and the -B traces
together. And then synchronize both experiments together to "fix" the
network delays.

Sooooo, it would be nice to have an interface layer that would allow:
1) First iteration, Java-CTF-trace-to-file-on-disk writer that requires manual synchronization merges, like (I think) you've detailed.
2) Subsequent Iteration: super-duper implementation that synchronized trace events on the fly from multiple trace sources, like CompuWare DynaTrace and CA Introscope.
3) Even Later implementation provides cluster-able event collection to deal with heavy load, like CompuWare DynaTrace and CA Introscope.

So my suggestion is to spend some time to make the interface good, and the implementation pluggable.
Sure. We haven't started integrating the synchronization stuff yet (this
is planned for after the Kepler release in June), so there's definitely
a lot of work to be done to make it useable!


Cheers,
Alexandre

Thanks,
--Erik

-----Original Message-----
From: Alexandre Montplaisir [mailto:alexmonthy@xxxxxxxxxxxx]
Sent: Monday, May 06, 2013 4:01 PM
To: Ostermueller, Erik
Cc: Linux Tools developer discussions
Subject: Re: [linuxtools-dev] FW: Java code for CTF trace writing?

Hi Erik,

Typically, you would have one trace for your kernel events, and a separate one from the Java parser.

In TMF, you can open multiple traces at the same time in what we call experiments. This allows you to see events coming from different traces in the same list.

With regards to trace synchronization, an initial synchronization infrastructure was submitted at https://urldefense.proofpoint.com/v1/url?u=https://git.eclipse.org/r/%23/c/10869/&k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0A&r=k0mtagpeWG6phJwAIzKq%2B4lTFA1JCM2OiWhVYmwwWeM%3D%0A&m=7wZCs%2Fzc0UXoImIqO%2F7dsVBRRAbFPNp%2FkmKCA8GhyLc%3D%0A&s=1b3c6073cef1a6a95ea1118e922a1371ec16ae3026c1c4b571168eeb47b7d5c1 .
If you have events in both your kernel and your Java trace that can be used to synchronize them together, it could be hooked into this infrastructure. We are hoping to merge it shortly after the Kepler release.

Cheers,
Alexandre


On 13-05-06 10:37 AM, Ostermueller, Erik wrote:
-----Original Message-----
From: Ostermueller, Erik [mailto:Erik.Ostermueller@xxxxxxxxxxxxx]
Sent: Monday, May 06, 2013 9:34 AM
To: Aaron Spear; Matthew Khouzam
Cc: Philippe Proulx; Dominique Toupin; lttng-dev@xxxxxxxxxxxxxxx;
Linux Tools developer discussions
Subject: Re: [lttng-dev] Java code for CTF trace writing?

Hello all,

I've got a question about the Java code that writes CTF traces.
Would this design provide a single trace with Java events and OS events in the same trace?
Here is an example:

Consider an all-Java program that sends an XML requests request over HTTP.
Would like the trace detail to show Java method activity as well as OS activity that opens/closes the socket.

This could be very helpful for unveiling the details of how Java code consumes hardware resources.

Thanks,
--Erik

-----Original Message-----
From: Aaron Spear [mailto:aspear@xxxxxxxxxx]
Sent: Tuesday, April 23, 2013 3:51 PM
To: Matthew Khouzam
Cc: Philippe Proulx; Dominique Toupin; lttng-dev@xxxxxxxxxxxxxxx;
Linux Tools developer discussions
Subject: Re: [lttng-dev] Java code for CTF trace writing?

When would you be available to work on this feature, I would love to
sync up our efforts for this.
The way things look right now, I will have bandwidth for it in another month, so middle to end of May. Perhaps sooner if all the stars align.

Aaron
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_______________________________________________
linuxtools-dev mailing list
linuxtools-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/linuxtools-dev



Back to the top