Bug 207215 - Regression since 3.4M1 in performance test StatePerformanceTest#testResolution1000()
Summary: Regression since 3.4M1 in performance test StatePerformanceTest#testResolutio...
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: Framework (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.4 M3   Edit
Assignee: Thomas Watson CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance, test
Depends on:
Blocks:
 
Reported: 2007-10-23 15:11 EDT by Frederic Fusier CLA
Modified: 2007-11-08 03:51 EST (History)
0 users

See Also:

Attachments
patch (1.39 KB, patch)
2007-10-23 16:45 EDT, Thomas Watson CLA 		no flags Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Frederic Fusier CLA 2007-10-23 15:11:03 EDT 
org.eclipse.osgi.tests.perf.StatePerformanceTest#testResolution1000()
 was between +18.7% and +22% in 3.4M1 performance test results. But since build N20070906-0010 it came down bewteen -16.9% and +0.4%. Last results on N20071021-0010 were bewteen -8.5% and +1.1%.

The standard error of this test is OK (always below the 3% threshold).

Note that test duration is between 1.5s and 2.9s approximately, so this
fall may be noticeable by users => open with normal severity
Comment 1 Thomas Watson CLA 2007-10-23 15:55:07 EDT 
Looks like the fixes for bug 165964 or bug 201489 may have caused some slowdown.  Investigate for M3.
Comment 2 Thomas Watson CLA 2007-10-23 16:41:12 EDT 
I confirmed the fix in bug 201489 caused the slowdown.  The fix is using an ArrayList for a collection when it should be using a Set to keep track of processed fragment BSNs.
Comment 3 Thomas Watson CLA 2007-10-23 16:45:53 EDT 
Created attachment 80994 [details]
patch
Comment 4 Thomas Watson CLA 2007-10-23 17:07:20 EDT 
Patch released to HEAD.  This gets the numbers close to I20070904-0800 results on my machine but it is still a little slower than before the fix to bug 201489 (~20ms) but this is probably expected because we still do some extra work to fix the original bug.  Fortunately other performance improvements outweigh the tiny slowdown and we should see an overall improvement in the resolver performance results against 3.3.