[linuxtools-dev] TMF: pluggable state provider contribution(s)

[Aaron Spear <aspear@xxxxxxxxxx>]

Hi Alexandre (and others), 

I am picking up a discussion we were having a month ago or so about contributing the data driven state provider stuff that I have been working on. Even based on changes that I just saw in master (the "partial history" support), it seems clear that there is some synchronization in plans and vision that needs to happen, so I am hoping that you could kindly review what I have done and offer any comments on architecture changes you would like to see, or things you don't like or whatever, so that I can get the ball rolling. 

GETTING STARTED: 
I spent some time cleaning up the code a bit and then created a fork of linuxtools on github, and then merged my changes in to a "state-system-extensions" branch there. I am quite sure that I am going to have to retool the branch to split changes into chunks that can be contributed incrementally. https://github.com/aspear/linuxtools

There are full instructions on how to get started and reproduce the views that I previously sent screenshots of here: 
https://github.com/aspear/linuxtools/blob/state-system-extensions/lttng/org.eclipse.linuxtools.tmf.stateflow.ui.tests/README.txt 

(Note that this "test" plugin really doesn't have any tests in it yet, I just wanted to have a container to dump in my example traces and screenshots and such.)

HIGH LEVEL OVERVIEW OF WHAT I DID: 

STATE SYSTEM "PRESENTATION INFO":
1) I added a new method to ITmfStateSystem, IStateSystemPresentationInfo  ITmfStateSystem.getStatePresentationInfo();  The idea here is that optionally a state system can provide some metadata that can help a trace agnostic viewer to parse the data.  ITmfStateSystemBuilder adds a setter for this IStateSystemPresentationInfo info.  My thought was that the derived IStateChangeInput, which knows about the state system and the state schema, could call the setter to set the presentation info in the state system.

2) I added an entire GUI view that I cloned from ControlFlowView that knows how to use this presentation info.  More on that later.

STATE SYSTEM FACTORY: 
3) I added a new "org.eclipse.linuxtools.tmf.ui.statesystemfactory" extension point to the org.eclipse.linuxtools.tmf.ui plugin. In reality extensions that implement this are headless and simply providing the implementation of a new IStateSystemFactory interface. I added it to this plugin because it seemed to be the only one that was providing extension points right now?  It belongs in org.eclipse.linuxtools.tmf.core, should I add it to that? (seems like the trace extension belongs there too right?) 

4) In the org.eclipse.linuxtools.tmf.core plugin I added org.eclipse.linuxtools.tmf.core.statesystem.StateSystemFactoryManager, which catalogs and caches the implementations of this factory.  I would think this class could be merged into StateSystemManager if you find the approach OK.   StateSystemFactoryManager is a singleton that can be used to get the factory and then create a ITmfStateSystem instance for a given trace.  Note that I did this, as opposed to creating a IStateChangeInput from the factory, because there were flows through the current code, in the case that the state database file already exists where there is no IStateChangeInput created at all, and that screwed me over since I then had no way to get the presentation.  I am sure there is a more simple and graceful way to do this initialization... 

5) TmfTrace.buildStateSystem is extended so that it attempts to use the factory to create a state system for an arbitrary trace.  There are ways to say which factory you want it to use (this info is cached) but if this doesn't exist, then all factories are iterated and the first one that works is used.  Of course a derived trace can still do whatever it wants, so no existing functionality is changed.

XML DATA DRIVEN STATE:
6) My stateflow.ui plugin provides the implementation of the state factory extension point so that it looks for an XML file with the same name as the trace but with ".state-schema.xml" appended to the end of it.  This is, of course, a pretty sucky work flow because it means you have to copy these XML files all over the place.  I would note that this is only one way to do it.  The factories can be much smarter if they wish. In fact in the case of my Java function call tracer, the derived trace actually reads the XML from a file that lives in the plugin, which is seamless.  There can be other ways to specify mappings between file patterns and schema files to use and such as well.

STATE FLOW VIEW:
7) I cloned ControlFlowView and rewrote it a bit to understand this schema.  I am still not entirely pleased with the way that it is setup today.  I will likely remove all of the columns on the table except for the name, and then put the trace in as the top level context node in the tree.   From there, the nodes in the context are specific to the trace.  Note that I am also going to extend this view so that it supports multiple traces at the same time (very important) so that you can correlate state changes from multiple contexts.  Of course there is a lot to think about here.  Really all of the these viewers should be refactored so that any trace can provide a state flow that can then be correlated (e.g. UST + LTTng kernel + my state flow).

-------------------  
There is a lot to say and a lot that I have planned to build, but this is a good start, so I will eagerly await any feedback.  

Note that I am especially pleased that Florian Wininger, a grad student studying under Michel Dagenais, is willing to jump into this same data driven trace state work as well.  He already has my prototype working and was going to contrast what I have done with his own requirements, so I think there will be good things coming.

BTW, I looked through the content at http://wiki.eclipse.org/Gerrit, and it is not entirely clear to me how exactly I can contribute my code for review with Gerrit.  Must any changes contributed there be to the master branch?  Or can I push another branch for review?  Any tips to docs or examples gratefully accepted.

best regards,
Aaron Spear