Bug 207215

Summary: Regression since 3.4M1 in performance test StatePerformanceTest#testResolution1000()
Product: [Eclipse Project] Equinox Reporter: Frederic Fusier <frederic_fusier>
Component: FrameworkAssignee: Thomas Watson <tjwatson>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 Keywords: performance, test
Version: 3.4   
Target Milestone: 3.4 M3   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
patch none

Description Frederic Fusier CLA 2007-10-23 15:11:03 EDT 
org.eclipse.osgi.tests.perf.StatePerformanceTest#testResolution1000()
 was between +18.7% and +22% in 3.4M1 performance test results. But since build N20070906-0010 it came down bewteen -16.9% and +0.4%. Last results on N20071021-0010 were bewteen -8.5% and +1.1%.

The standard error of this test is OK (always below the 3% threshold).

Note that test duration is between 1.5s and 2.9s approximately, so this
fall may be noticeable by users => open with normal severity
Comment 1 Thomas Watson CLA 2007-10-23 15:55:07 EDT 
Looks like the fixes for bug 165964 or bug 201489 may have caused some slowdown.  Investigate for M3.
Comment 2 Thomas Watson CLA 2007-10-23 16:41:12 EDT 
I confirmed the fix in bug 201489 caused the slowdown.  The fix is using an ArrayList for a collection when it should be using a Set to keep track of processed fragment BSNs.
Comment 3 Thomas Watson CLA 2007-10-23 16:45:53 EDT 
Created attachment 80994 [details]
patch
Comment 4 Thomas Watson CLA 2007-10-23 17:07:20 EDT 
Patch released to HEAD.  This gets the numbers close to I20070904-0800 results on my machine but it is still a little slower than before the fix to bug 201489 (~20ms) but this is probably expected because we still do some extra work to fix the original bug.  Fortunately other performance improvements outweigh the tiny slowdown and we should see an overall improvement in the resolver performance results against 3.3.