[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [platform-releng-dev] Questions/Remarks about 3.1 M6 JDT/Core performances results
|
As suggested previously by others, the reference/baseline 3.0 performance
tests should be run on a regular basis and their graphs should show a flat
line with a slope of 0.
This flat line would prove that all of the test conditions remain the same
from week to week, and that the performance hits/improvements in the 3.1
stream are real.
Wassim.
Sonia
Dimitrov/Ottawa/I
BM@IBMCA To
Sent by: "Eclipse platform release
platform-releng-d engineering list."
ev-bounces@eclips <platform-releng-dev@xxxxxxxxxxx>
e.org cc
Subject
04/04/2005 02:54 Re: [platform-releng-dev]
PM Questions/Remarks about 3.1 M6
JDT/Core performances results
Please respond to
"Eclipse platform
release
engineering
list."
See my comments below denoted by sd>
Sonia
Frederic Fusier
<frederic_fusier@xxxxxx.c
om> To
Sent by: platform-releng-dev@xxxxxxxxxxx
platform-releng-dev-bounc cc
es@xxxxxxxxxxx
Subject
[platform-releng-dev]
04/04/2005 06:24 AM Questions/Remarks about 3.1 M6
JDT/Core performances
results
Please respond to
"Eclipse platform
release engineering
list."
I was just looking at 3.1 M6 performance results
(http://fullmoon.torolab.ibm.com/downloads/drops/S-3.1
M6-200504011645/performance/org.eclipse.jdt.core.php?)
and have several remarks/questions about them.
1) New tests on Scanner and Parser
We added these tests last week. We expect them to reflect a
performance increase around 30% regarding to 3.0 version.
Unfortunately, it seems that last Friday, 3.0 reference was not
rebuilt and results are negative :-(.
Would it be possible to make this reference as soon as possible
and update JDT/Core performance results page to be able to see
the *real* deltas for these tests?
sd> we will rerun the 3.1M6 performance tests on a fresh baseline on
Wednesday. We will rebuild and rerun the baseline tests following
tomorrow's integration build. Due to all the build activity surrounding
3.1M6 (not to mention the team being at 50%), there was no time to run
baseline tests last Friday.
Waiting for this 3.0 rebuild, I wonder why same tests with same code
(no change has been done for Scanner and Parser since tests creation)
produce deltas going from -2.6% to +7.2% (ie. dispersion near 10%).
Does it means that precision for performance tests is not better
than 10%? Or is there any issue with test box during last week?
sd> good question, we need time to investigate further to answer with some
precision.
2) Full source workspace build without comment
I do not understand why there's no 3.0 reference on Win/XP test box
although there's one on RHEL one?
What does 3.0_200406250800_200412241024 mean?
sd> this is the "substitute" build id (this one happens to be a baseline
(3.0) we ran on Dec. 24 in 2004, but could be an I or N build) used by the
performance framework to assert against when the specified reference does
not exist. Andre could comment best on how the substitute is chosen.
sd> as for the missing reference on Windows, this information is in the
consolelog. I can investigate this further with a local member of the JDT
Core team.
[java] !ENTRY org.eclipse.test.performance 2 0 2005-04-01 23:24:50.484
[java] !MESSAGE no reference data named '{config=relengbuildwin2,
build=3.0_200406251208_200503271002, jvm=sun1.4.2_06}' found
[java] !ENTRY org.eclipse.test.performance 2 0 2005-04-01 23:30:14.937
[java] !MESSAGE no reference data named '{config=relengbuildwin2,
build=3.0_200406251208_200503271002, jvm=sun1.4.2_06}' found
[java] !ENTRY org.eclipse.test.performance 2 0 2005-04-01 23:31:51.265
[java] !MESSAGE no reference data named '{config=relengbuildwin2,
build=3.0_200406251208_200503271002, jvm=sun1.4.2_06}' found
[java] !ENTRY org.eclipse.test.performance 2 0 2005-04-01 23:32:28.359
[java] !MESSAGE no reference data named '{config=relengbuildwin2,
build=3.0_200406251208_200503271002, jvm=sun1.4.2_06}' found
[java] !ENTRY org.eclipse.test.performance 2 0 2005-04-01 23:33:57.546
[java] !MESSAGE no reference data named '{config=relengbuildwin2,
build=3.0_200406251208_200503271002, jvm=sun1.4.2_06}' found
[java] -------------------------------------
[java] DOM/AST creation performance test statistics:
[java] - 6,156,412 nodes have been parsed.
[java] - 49,713 nodes have extended start position.
3) Difference between 2 test boxes
It seems that Win/XP results have changed a lot since last week and
do not really match reality. I wonder what happens on this machine?
Perhaps is there any change on it or something wrong happens during
reference 3.0 build which makes these results inaccurate?
Do you have any clue on this?
Cordialement/Regards,
Frédéric
_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-releng-dev
_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-releng-dev
