| [linuxtools-dev] TMF indexing performance |
|
How is it planned to efficiently creating the index
of trace files that are holding multiple millions of events? It seems to be quite slow to parse every single event
in a trace log file just for creating the index. I see that synchronization of multiple
traces might be a problem but it can handled more efficient when we have a
getNextEvent function that is able to skip a given amount of events. It might work this way: The framework calls getNextEvent( traces, contexts,
numberOfEventsToSkip ); The getNextEvent implementation calls the
getNextEvent of the trace with the smallest timestamp and gives the number of
events to skip and the amount of time (where the next trace in time is currently
positioned). Given on that parameters the trace will either skip the number of
given events or stop at the given timestamp so that the framework can switch
over to the next trace and continue with the remaining amount of events to
skip. The parseEvent function should have the same
parameters (numbers of events to skip and timestamp where to stop skipping) so that
this functionality can efficiently handled in JNI. This would help the indexing performance a lot
because the underlying JNI implementation may know how to efficiently skip
events in the stream. What do you think? br, Franz |