Re: [linuxtools-dev] TMF: pluggable state provider contribution(s)

[Aaron Spear <aspear@xxxxxxxxxx>]

Hi Alexandre,

replies below:

----- Original Message -----
> - I think what you call "context" in the stateflow plugin is pretty much
> the same thing we call "attribute" in the TMF state system. You've
> probably seen o.e.l.internal.tmf.core.statesystem.Attribute. We already
> use the notion of "context" to represent a specific location in a trace,
> so it could lead to confusion. Perhaps we could tweak Attribute a bit
> and make it public, or maybe have them share a common interface.

I am trying to use context in the same sense that we intended it while
developing the CTF spec, that is that it is a value that indicates 
information about the environment of a given event.  So that typically means
board/core/process/thread or the like.  Of course this information *can* and
in this case is an attribute of an event, but it is not necessarily an 
attribute as it could be implied for a given trace too.  I think that the 
important point is that for a given trace with this "state flow" view, the 
context value is something that is completely trace specific and is a 
hierarchical Y axis value

If I understand you correctly you are saying that you use "context" to 
refer to an events position in time?  How do you differentiate between an 
attribute of an event that is say the count of a semaphore vs. an attribute
that is the thread that it occurred in?  They are both "attributes" no?

> - In the XML files to define the state changes, you separate the actions
> of "switching context" and "applying state change to the current
> context". Is there any reason for this? This can make sense / be easier
> when dealing with single-threaded applications, but in multi-threaded
> applications or systems, a series of events in the trace could relate to
> completely different attributes.

I realize that is a little odd to get your brain wrapped around it.  It is like
this because it was the most simple way that I could, in the same data driven 
implementation, be able to support the notion of a call stack on a given thread.
The events that come across for the function tracer that I did basically have a 
thread id in every event, and then a function/entry or exit as the event type.  
Because it is multi-threaded you have no idea what order any events will come in, 
and this means that you must, in the viewer, track the call stack for each thread
individually. So I FIRST check to see if we should "switch context" for the given
event, and then switch to a data structure that tracks the call stack for that 
context/thread.  Then if that same event is setup for any "push state change" or
"pop state change" I apply those.  I did want to keep this information (of the 
call stack for the thread) in the state system, but could not really figure out
an efficient way to do this.  I would certainly be open to better ideas here.

> Would it be more generic to have only one entry per state change, which
> would define both the attribute/context to modify and the value that is
> being given?

Well, it is trickier than that though.  Consider the use case of this function
tracer.  In that case, you have only have the following two different events 
-thread id
-event type: function entry
-new function name

-thread id
-event type: function exit
-name of function you are EXITING

There are two reasons then why you must have the "push", "pop" in addition to a
switch:

-You want to be able to indicate functions for a given thread mutually 
exclusively so that it is clear what functions are running and what functions
are up stack at any point. 
-If you didn't care about showing what was up stack in a different color you
could do a switch for the function entry, since you have all the information
you need in the event.  However, for function exit, you only have that you are
exiting a function, and thus you do not know what you need to switch to.  You
must track this outside.

Yes, you could have a single entry that can do all of this, but it ends up being
"6 of one, half dozen of the other".  If you do this, then to support my 
"call stack" use case you must indicate that this particular event is either a 
"switch only" in which case the new value in the event is the complete value, 
or a "push" (in which case the new value is pushed on to a stack for the given
context) or a "pop".

Note that doing this does open up some interesting visualization possibilities
for state machines in addition to function calls.

> Very crude and unoriginal example, let's suppose I have an event
> "sched_switch", and I want to update the value of the attribute
> "Thread/[thread tid]/Status" to the value "running":
> 
> <eventHandler eventName="sched_switch">
>   <stateChange>
>     # Define the path to the attribute to be affected
>     <attribute name="Threads" /> # hard-coded value
>     <attribute fieldvalue="tid" /> # a field in sched_switch
>     <attribute name="Status" />
>     # Apply the state change
>     <modify valueType="string" value="Running" />
>   </stateChange>
> </eventHandler>
> 
> I put a String value, but of course we'd probably want an integer value
> here instead. I also put the <stateChange> under the event name itself,
> since sometimes one event can cause more than one state change. Any
> thoughts?

One thing that does occur to me is that there is probably some value in
decoupling the logic used to populate the state system with what you want
to display.  What you have above is good in that it outlines clearly what
event information is going into the state system, but of course you don't
know from this alone what is supposed to be displayed on the Y axis at all.
You would need something else to tell the view what the hierarchy is, how
to get the values from the state system, and the display information for those
states (and of course my xml file does all of this in one file at the moment)
 
> - As you mentionned, your StateSystemFactory and the existing
> StateSystemManager could eventually be merged together. If I understand
> correctly, with the factory you can only build one state system per
> trace. In TMF we are now often building multiple state systems per
> trace. Some of them can be defined in base classes. For example we're
> looking at having:
> 
> TmfTrace -> generic statistics (event totals, and count per event types)
> LttngTrace -> LTTng-specific statistics (CPU time per process, etc.)
> LttngKernelTrace -> Kernel-specific state system, used to populate the
> Control Flow and Resource views.
> 
> This way, a "LttngKernelTrace" will own 3 state system types, and the
> relevant views can pick the right one with its ID. It's even possible
> for views to be completely oblivious to the trace type itself: they can
> simply look if their expected state system is present, and if so they
> use it to populate themselves. (This only applies to views using a state
> system directly, obviously. Some views could rely on straight trace events.)

Yeah, I noticed this when I re-based before putting this code up.  I started 
with the code where there was a single state system...  I will refactor this 
so that it can be one of many that is added.

> Also, you might have noticed, but in master we recently merged the State
> System Explorer view, which simply displays the contents of the current
> trace's state system(s) at the currently-selected timestamp. This is
> very useful for debugging states, and simply making sure your provider
> actually builds what you are looking for. It's available under the
> Tracing category.

Hadn't seen this!  will take a look as that sounds very useful.

regards,
Aaron