Bug 267789

Description Graeme Vetterlein 2009-03-10 05:07:15 EDT

Build ID: Version: 3.4.1 Build id: M20080911-1700

Steps To Reproduce:
1.download eclipse-Automated-Tests-3.4.zip  (found here http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/)
2.Install IBM JDK
3.Use runtests to run the test jdtcoreperf


More information:
Also found here: http://www.eclipse.org/newsportal/article.php?id=80349&group=eclipse.platform#80349

In essence this runs with the SUN JDK but not with the IBM JDK. Looks like it does not have a buildpath to find the basic java classes

I get build errors using IBM JVM but not with SUN JVM. There
is a defect from 2002 which seems similar.



eclipse-Automated-Tests-3.4.zip  (found here http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/)

I have this running but I've been investigating the variation in results. On the website they show some results like:

http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/performance/html/org.eclipse.jdt.core.tests.performance_linux.gtk.perf.html

Now I can get a similar html table (see attached) ... all looking fine . However if I then look at ....eclipse-testing/results/linux.gtk.x86/org.eclipse.jdt.core.tests.performance.AllPerformanceTests.txt

================================================================================
Running FullSourceWorkspaceBuildTests#testParser...
Scenario 'org.eclipse.jdt.core.tests.performance.FullSourceWorkspaceBuildTests#testParser()' (average over 10 samples):
   System Time:            2.86s         (95% in [2.38s, 3.35s]) Measurable effect: 862ms (1.3 SDs) (required sample size for an effect of 5% of mean: 363)
   Used Java Heap:        77.75M         (95% in [77.75M, 77.75M])
   Working Set:             819          (95% in [-416, 2.01K]) Measurable effect: 2.13K (1.3 SDs) (required sample size for an effect of 5% of stdev: 6400)
   Elapsed Process:        2.86s         (95% in [2.38s, 3.35s]) Measurable effect: 862ms (1.3 SDs) (required sample size for an effect of 5% of mean: 363)
   Kernel time:              13ms        (95% in [-5ms, 31ms]) Measurable effect: 32ms (1.3 SDs) (required sample size for an effect of 5% of stdev: 6400)
   CPU Time:               2.89s         (95% in [2.41s, 3.36s]) Measurable effect: 843ms (1.3 SDs) (required sample size for an effect of 5% of mean: 342)
   Hard Page Faults:          0          (95% in [0, 0])
   Soft Page Faults:       2.03K         (95% in [-2.46K, 6.52K]) Measurable effect: 7.94K (1.3 SDs) (required sample size for an effect of 5% of stdev: 6400)
   Text Size:                 0          (95% in [0, 0])
   Data Size:                 0          (95% in [0, 0])
   Library Size:              0          (95% in [0, 0])

================================================================================
Running FullSourceWorkspaceBuildTests#testFullBuildDefault...
FullBuildDefault: Unexpected ERROR marker(s):
org.eclipse.update.ui:
The project cannot be built until its prerequisite org.eclipse.ui is built. Cleaning and building all projects is recommended
org.eclipse.update.core:
The project was not built since its build path is incomplete. Cannot find the class file for java.io.Serializable. Fix the build path then try building this project
UpdateCommand.java:

So the .html table is correctly reporting the eclipse results ... i.e. eclipse running OK and producing these timings ... many test work fine but the 'compile' type
tests seem to have a lot of compilation failures . These in turn seem to relate to bad build paths. I found this historic bug:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=22390


Which seems like the same idea ... so my current guess is that he buildpath is being incorrectly generated.  I've gone back and recreated the install etc on two different machines
but this always seems to happen. 

I am running these tests with the IBM Java6 JVM . I repeated these
test with Sun Java6 JVM and it worked fine. This feels like a resurrection of an old defect.