Community
Participate
Working Groups
Build ID: Version: 3.4.1 Build id: M20080911-1700 Steps To Reproduce: 1.download eclipse-Automated-Tests-3.4.zip (found here http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/) 2.Install IBM JDK 3.Use runtests to run the test jdtcoreperf More information: Also found here: http://www.eclipse.org/newsportal/article.php?id=80349&group=eclipse.platform#80349 In essence this runs with the SUN JDK but not with the IBM JDK. Looks like it does not have a buildpath to find the basic java classes I get build errors using IBM JVM but not with SUN JVM. There is a defect from 2002 which seems similar. eclipse-Automated-Tests-3.4.zip (found here http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/) I have this running but I've been investigating the variation in results. On the website they show some results like: http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/performance/html/org.eclipse.jdt.core.tests.performance_linux.gtk.perf.html Now I can get a similar html table (see attached) ... all looking fine . However if I then look at ....eclipse-testing/results/linux.gtk.x86/org.eclipse.jdt.core.tests.performance.AllPerformanceTests.txt ================================================================================ Running FullSourceWorkspaceBuildTests#testParser... Scenario 'org.eclipse.jdt.core.tests.performance.FullSourceWorkspaceBuildTests#testParser()' (average over 10 samples): System Time: 2.86s (95% in [2.38s, 3.35s]) Measurable effect: 862ms (1.3 SDs) (required sample size for an effect of 5% of mean: 363) Used Java Heap: 77.75M (95% in [77.75M, 77.75M]) Working Set: 819 (95% in [-416, 2.01K]) Measurable effect: 2.13K (1.3 SDs) (required sample size for an effect of 5% of stdev: 6400) Elapsed Process: 2.86s (95% in [2.38s, 3.35s]) Measurable effect: 862ms (1.3 SDs) (required sample size for an effect of 5% of mean: 363) Kernel time: 13ms (95% in [-5ms, 31ms]) Measurable effect: 32ms (1.3 SDs) (required sample size for an effect of 5% of stdev: 6400) CPU Time: 2.89s (95% in [2.41s, 3.36s]) Measurable effect: 843ms (1.3 SDs) (required sample size for an effect of 5% of mean: 342) Hard Page Faults: 0 (95% in [0, 0]) Soft Page Faults: 2.03K (95% in [-2.46K, 6.52K]) Measurable effect: 7.94K (1.3 SDs) (required sample size for an effect of 5% of stdev: 6400) Text Size: 0 (95% in [0, 0]) Data Size: 0 (95% in [0, 0]) Library Size: 0 (95% in [0, 0]) ================================================================================ Running FullSourceWorkspaceBuildTests#testFullBuildDefault... FullBuildDefault: Unexpected ERROR marker(s): org.eclipse.update.ui: The project cannot be built until its prerequisite org.eclipse.ui is built. Cleaning and building all projects is recommended org.eclipse.update.core: The project was not built since its build path is incomplete. Cannot find the class file for java.io.Serializable. Fix the build path then try building this project UpdateCommand.java: So the .html table is correctly reporting the eclipse results ... i.e. eclipse running OK and producing these timings ... many test work fine but the 'compile' type tests seem to have a lot of compilation failures . These in turn seem to relate to bad build paths. I found this historic bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=22390 Which seems like the same idea ... so my current guess is that he buildpath is being incorrectly generated. I've gone back and recreated the install etc on two different machines but this always seems to happen. I am running these tests with the IBM Java6 JVM . I repeated these test with Sun Java6 JVM and it worked fine. This feels like a resurrection of an old defect.
There's a similar issue with Sun JDK 6.0 which has been fixed in HEAD and perf_34x streams. Unfortunately the 3.4.x stream is closed and there's no way to have a new build on it.
Created attachment 128188 [details] Proposed jar file including patch fixing this issue Could you make a try with this jar file and let me know if it fixes your issue?
I also have a question, why do you consider this problem as a blocker? It's about performance test... and it does not prevent to use 3.4.2. So, I would have set it to major but not more unless there's some hidden important reason?
I understand this is blocker for you, but it is not for Eclipse, hence reducing the severity to critical. In parallel I increment the priority to P2 as I'll try to rapidly figure out what kind of solution might be applied there to fix this peculiar issue...
Created attachment 129068 [details] Proposed patch This patch does not assume anything for the JVM jars and put all of them onto the classpath of each project, hence no compiler error should ever occur while running the tests of FullSourceWorkspaceBuidTests suite...
Fixed for 3.5M7 in HEAD stream.
Verified for 3.5M7
