[platform-core-dev] core.resources Performance Test question

["Oberhuber, Martin" <Martin.Oberhuber@xxxxxxxxxxxxx>]

Hi all,

 

as I have mentioned in the Arch call yesterday, I have implemented a change to core.resources that should greatly reduce memory consumption during "Refresh Project", and now I'd like to verify that it doesn't take longer to run and in fact does reduce memory consumption.

 

In anticipation of tonight's 3.6 run with performance tests, I investigated the previous performance run on I20091013-1302:

http://download.eclipse.org/eclipse/downloads/drops/I20091013-1302/performance/org.eclipse.core.php?fp_type=0

 

My questions:

• For the "Refresh Project" scenario that I'm interested in, the test shows 68% improvement (compared to 3.5) on SLED10, but 53% degradation on Windows (on no change at 2.3% on RHEL5). These numbers do not seem reasonable at all. I'm assuming that this is due to the outlier in elapsed process time (see below) -- right? Or how else to interpret this?
• Digging deeper: http://download.eclipse.org/eclipse/downloads/drops/I20091013-1302/performance/eplnx1/Scenario166.html the baseline shows a complete outlier in terms of elapsed process time. How to interpret that? Shouldn't the baseline always remain the same? And all percentage comparison be against a mean value of baseline, rather than an outlier as done here?
• Why do the tests (when looking on the highest level) look at Elapsed Process time rather than CPU time, which seems stable here?
• I'm interested in memory consumption, but the test data does not even seem to record that... or where can I find it?

Thanks,

Martin

--

Martin Oberhuber, Senior Member of Technical Staff, Wind River

Target Management Project Lead, DSDP PMC Member

http://www.eclipse.org/dsdp/tm