Re: [tracecompass-dev] Performance of CPU usage analyses with many threa

[rreinsberger@xxxxxxxxx]

Hello Geneviève,

In the patch review thread there was an example with 4 levels of 
checkboxes discussed - mentioned by Bernd.
I wonder if it is a specified use case to have several levels of 
sub-threads in the CPU usage view, respectively if it is, can this 
example be made available please (e.g. as a test trace in the 
repository)?

Regarding the drawing performance of the usage graphs in the SWTChart 
component: As experiments showed drawing poly lines instead of drawing 
lines (i.e. drawPolyline() instead of drawLine()) significantly improves 
rendering performance. Are you aware of any caveats attached to the 
usage of the poly line drawing routine?

Best Regards,
Rudolf


Am 2019-03-07 22:56, schrieb Genevieve Bastien:
> Hi Rudolf,
>
> Thanks for looking into this.
>
> Some of us started to look at this, at patch
> https://git.eclipse.org/r/#/c/137404/  But some things were not working
> in our cases.
>
> You're welcome to contribute your solution! We'll gladly review it.
> Here's the contributor guidelines
> https://wiki.eclipse.org/Trace_Compass/Contributor_Guidelines
>
> Regards,
>
> Geneviève
>
>
> On 2019-03-07 8:38 a.m., rreinsberger@xxxxxxxxx wrote:
> > Hello Genevieve,
> >
> > We now did some debugging ourselves regarding the CPU usage check box
> > tree and the rendering performance.
> >
> > As it seems the pure graph rendering for the >1500 threads is done in
> > about 20s (without any check box handling) - which is acceptable.
> >
> > With our improvements on the check box handling (in
> > TriStateFilteredCheckboxTree.java) we are down to 40s from selecting
> > all 1500 threads (i.e. check boxes) to finishing the graph rendering.
> > We would be ready to submit these improvements but as we are no TC
> > committers (yet) how can we best do that?
> >
> > However, although this is a huge improvement from the application
> > previously becoming unresponsive after 30 minutes, even the remaining
> > check box handling overhead is somewhat unsatisfyingly high and we
> > should consider further optimizations.
> >
> > Best Regards,
> > Rudolf
> >
> >
> > Am 2019-02-21 18:43, schrieb Genevieve Bastien:
> > > Hi Rudolf,
> > >
> > >
> > > There was no benchmark for the CPU Usage analysis, so I made one that
> > > queries the backend. For manyThreads, it takes 2.15 minutes to query
> > > 10 times the tree on the left and the CPU usage for all threads. 
> > > There
> > > was an easy performance fix, got that down to 1.5 minutes for 10x the
> > > full queries (so ~9s for one query of both tree and graph data). So
> > > the problem is definitely not the backend (the algorithm is not
> > > linear).
> > >
> > > After doing some more profiling according to your comments, one would
> > > think that if the checkboxes are not visible, they would not be
> > > rendered, but that is apparently not true, it is still checking the
> > > checked state of each box, no matter if it's visible or not. And that
> > > still takes forever. Somebody in the team is looking into this.
> > > Hopefully we'll "fix" that pretty soon.
> > >
> > >
> > > Thanks and don't hesitate if you notice any other such performance 
> > > hit!
> > >
> > > Regards,
> > >
> > > Geneviève
> > >
> > >
> > > On 2019-02-21 4:36 a.m., rreinsberger@xxxxxxxxx wrote:
> > > > Hello Geneviève,
> > > >
> > > > I tried the same with a collapsed tree so that only the topmost
> > > > check box remains. So, selecting this one implicitly selects all
> > > > 1535 collapsed threads. Although there is nothing apparently to
> > > > render in this case (in terms of individual thread checkboxes) the
> > > > result is the same as with the expanded tree view.
> > > >
> > > > However, trying to render just individual threads from this example
> > > > takes between 5s to 7s on my machine (which is well aged but with an
> > > > i7 core after all). That would roughly mean about 2 hours to render
> > > > all 1535 threads. So, I am not that sure that it's definitely the
> > > > GUI check boxes causing this run time?
> > > >
> > > > As the many-threads example I use is from the test repository do you
> > > > know if there is any test protocol for this test case telling about
> > > > the expected results and run time?
> > > >
> > > > Regards,
> > > > Rudolf
> > > >
> > > >
> > > >
> > > > Am 2019-02-18 16:44, schrieb Genevieve Bastien:
> > > > > Hi Rudolf,
> > > > >
> > > > > Wow, reading this bug report brings back memories... Claudette
> > > > > nodes...
> > > > >
> > > > > Anyway, indeed, on the backend side, there should be no performance
> > > > > problems anymore, and from doing a bit of sampling after reading 
> > > > > your
> > > > > mail, the performance problem you're experience comes from the
> > > > > frontend: The UI spends most of its time rendering checkboxes of 
> > > > > the
> > > > > tree items... Because there's a lot of items in this tree, yet only 
> > > > > a
> > > > > few of them are visible. I think the 'tree' widget of those views
> > > > > should be more lazy and display only the visible items instead of 
> > > > > all
> > > > > of them. There should be a bug for that. Especially that we have 
> > > > > more
> > > > > and more of that type of views with a lot of items...
> > > > >
> > > > > Thanks for bringing this up,
> > > > >
> > > > > Regards,
> > > > >
> > > > > Geneviève
> > > > >
> > > > >
> > > > > On 2019-02-18 8:05 a.m., rreinsberger@xxxxxxxxx wrote:
> > > > > > Hello,
> > > > > >
> > > > > > Evaluating the CPU usage analyses and view I used sample/test data
> > > > > > from here:
> > > > > >
> > > > > > https://github.com/tracecompass/tracecompass-test-traces/tree/master/ctf/src/main/resources/many-threads
> > > > > >
> > > > > >
> > > > > > … and from here (see file attachment in the report):
> > > > > >
> > > > > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=460261
> > > > > >
> > > > > > Trying to update the CPU usage graph with all threads selected the
> > > > > > analyses took a very long time (>30 min) until Trace Compass
> > > > > > (v4.2) was eventually rendered unresponsive.
> > > > > >
> > > > > > It seems that in the bug report referred to above this problem (or
> > > > > > at least parts of it) were addressed. However, I am not sure what
> > > > > > performance and results are to be expected for this test traces.
> > > > > > Is the CPU usage supposed to work and produce results with this
> > > > > > trace data in TC version 4.2?
> > > > > >
> > > > > > Regards,
> > > > > > Rudolf
> > > > > > _______________________________________________
> > > > > > tracecompass-dev mailing list
> > > > > > tracecompass-dev@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe from this list, visit
> > > > > > https://www.eclipse.org/mailman/listinfo/tracecompass-dev
> > > > > _______________________________________________
> > > > > tracecompass-dev mailing list
> > > > > tracecompass-dev@xxxxxxxxxxx
> > > > > To change your delivery options, retrieve your password, or
> > > > > unsubscribe from this list, visit
> > > > > https://www.eclipse.org/mailman/listinfo/tracecompass-dev
> > _______________________________________________
> > tracecompass-dev mailing list
> > tracecompass-dev@xxxxxxxxxxx
> > To change your delivery options, retrieve your password, or
> > unsubscribe from this list, visit
> > https://www.eclipse.org/mailman/listinfo/tracecompass-dev
>
> _______________________________________________
> tracecompass-dev mailing list
> tracecompass-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/tracecompass-dev