A follow-up to the discussion on state started earlier. Statistics extracted from tracing data are similar and closely related to state. In an extensible system, when a developer defines new tracepoints for his subsystem, he may also want to define associated modeled state to update, and statistics to gather.
In LTTng, the statistics are computed as follows. The number of events of each type and the time spent by each process in each state (running, runnable, waiting) are summed by CPU, mode (user, syscall, trap, irq), submode (specific system call) and possibly function. These are then aggregated in different ways (by process, by CPU...). In some cases, some finer grained value is summed, like the number of bytes read or written in read or write events.
The statistics can then be reported as total counts and histograms (count for an interval) for events, or as durations (total CPU time for a process or proportion of CPU usage) for state. Currently, only sums for the whole trace are reported, it will be simple to provide the same information for time intervals. The challenge is to be able to pre-compute some values in order to quickly provide statistics for such intervals.
It should be relatively easy to identify event types or fields within event types to "count" and display as statistics. There could be as well interesting state values to display which are numeric and vary in time, for instance the number of requests in a queue. Similarly, certain states (their duration) could be identified as interesting to display (e.g. CPU time used by each process in a time period, proportion of time where a disk or CPU was busy...).
Do you have links to systems which have addressed these issues and have a flexible and extensible framework to compute and display statistics for tracing or similar tasks?
