Bug 209807 - Regression in performance test PerfVariableDetailTests#testToStringDetails()
Summary: Regression in performance test PerfVariableDetailTests#testToStringDetails()
Status: VERIFIED WORKSFORME
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Debug (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 minor (vote)
Target Milestone: 3.4 M4   Edit
Assignee: JDT-Debug-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance, test
Depends on:
Blocks:
 
Reported: 2007-11-14 09:50 EST by Frederic Fusier CLA
Modified: 2008-03-07 05:30 EST (History)
1 user (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Frederic Fusier CLA 2007-11-14 09:50:43 EST 
org.eclipse.jdt.debug.tests.performance.PerfVariableDetailTests#testToStringDetails()
 results are over 10% regression since N20071108-0010:

RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB):
I20071107-1300	0.0%
N20071108-0010	-15.2%
N20071111-0010	-15.8%
I20071113-0800	-13.9%

Win XP Sun 1.4.2_10 (3 GHz 2 GB)
I20071107-1300	-1.7%
N20071108-0010	-16.8%
N20071111-0010	-17.2%
I20071113-0800	-49.6% <-- Seems to have also other troubles on the box

RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)
I20071107-1300	-2.3%
N20071108-0010	-20.2%
N20071111-0010	-21.6%
I20071113-0800	-21.2%

Win XP Sun 1.4.2_10 (2 GHz 512 MB)
I20071107-1300	0.5%
N20071108-0010	-17.1%
N20071111-0010	-18.1%
I20071113-0800	-17.6%

RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)
I20071107-1300	-1.3%
N20071108-0010	-24.2%
N20071111-0010	-25.2%
I20071113-0800	-25.6%

The standard error of this test is really good (between 0.4% and 1.1%), so it seems that this regression needs to be investigated.

Note that this test lasts between 200ms and 400ms and so make this regression not really noticeable by users => open as minor
Comment 1 Darin Wright CLA 2007-11-21 09:53:55 EST 
In I20071120-0800 this test is now back to normal. We have made no code changes in this area - I suspect something else caused the test to worsen and then return to normal.

Marking as "works for me".
Comment 2 Frederic Fusier CLA 2008-03-07 05:30:50 EST 
Verified that this test is constantly OK since 3.4M4...