Community
Participate
Working Groups
Looking at M20080820-0800 performance results, it looks like the scenario: 'org.eclipse.swt.test.Draw on a window (Win XP Sun 1.4.2_10 (2 GHz 512 MB))' shows a severe regression. Here are the numbers got from the database: M1 I20070809-1105 21.4% M3 I20071101-2000 -3.1% M4 I20071213-1700 3.0% M5 I20080207-1530 -7.2% M6a I20080409-1425 0.1% M7 I20080502-0100 19.4% RC1 I20080516-1333 0.9% RC2 I20080527-2000 10.8% RC3 I20080530-1730 -6.9% RC4 I20080609-1311 -0.3% R3_4 I20080617-2000 -6.6% M20080703-0800 0.1% M20080723-0800 -22.4% M20080730-0800 -22.4% M20080731-1200 -22.6% M20080808-0800 -35.9% M20080813-0800 -22.8% M20080820-0800 -35.5% This test was a little bit unstable until 3.4.0, but it seems that something happened between M20080703-0800 and M20080723-0800 as since then, it shows a performance regression over 20%... Note that other test boxes (even the other WinXP one) do not show similar regression. So, not sure whether a code change can explain this regression or a problem on the test box itself?
This is the test that I was talking about that was failing. We believe the test is unstable but we need to get to the bottom of it ... after 3.4.1 testing.
Something wrong with the machine. Here some SWT version I got from the webpage: 3.4 org.eclipse.swt=v3448f -6.6% M20080808-0800 org.eclipse.swt=v3448l -35.9% M20080813-0800 org.eclipse.swt=v3448l -22.8% M20080820-0800 org.eclipse.swt=v3448m -35.5% I compared v3448f and v3448m. No differences at all the code that is being exercised by the testcase. Notice that M20080808-0800 and M20080813-0800 are running the same version of SWT and the performance is different. I'm moving the problem to rel-eng. Not sure what is best: to remove the testcase or to discommission the machine.
This machine will be decomissioned in 3.5, we will be getting new performance for this stream of builds.
Kim, I think this machine should be decommission asap. We also see bad results for other components on this machine and this may introduce doubt in users mind about Eclipse performance for this kind of box... I also think we should re-generate the 3.4.1 results without the results of this box. Philippe, what's your opinion?
Can this bug be closed? We'll be using new hardware in 3.5 that will exclude this machine. Not sure if we will be changing the 3.4.1 performance infrastructure. I doubt it.
Closing.
Created attachment 121670 [details] patch to disable perf tests on eclipseperfwin1 released for next build - M20090107-0800.
(In reply to comment #7) > Created an attachment (id=121670) [details] > patch to disable perf tests on eclipseperfwin1 > > released for next build - M20090107-0800. > I took me a long time to realize that with the change done in helper.xml in this patch makes the perf generation facing bug 248252... That explain why no results was generated on maintenance stream since several weeks! I'll try to fix bug 248252 before next maintenance build
I've reverted the change done in helper.xml to workaround bug 248252 in the maintenance stream.
