Community
Participate
Working Groups
3.7 but also on the baseline. CodeCompletionPerformanceTest#testApplicationWithParamterGuesses2() is unreliable. The test jumps up and down. In 3.6 it looked good. We need to investigate why this now happens in 3.6 baseline builds and 3.7.
This test is unreliable only on RHEL machine. Plus CodeCompletionPerformanceTest#testApplicationWithParamterNames() is also unreliable on the same machine. Both the tests work reliably on other 3 machines.
(In reply to comment #0) > In 3.6 it looked good. > > We need to investigate why this now happens in 3.6 baseline builds and 3.7. Nope, the problem started sometime during 3.6 and that too only on RHEL. http://archive.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/performance/eplnx2/Scenario32.html http://archive.eclipse.org/eclipse/downloads/drops/R-3.6-201006080911/performance/eplnx2/Scenario28.html
For one thing CodeCompletionPerformanceTest.testApplicationWithParamterGuesses() is wrong. In the third line the method invocation should be measureApplicationWithParamterGuesses instead of measureCompletionWithParamterNames :) Once this method is corrected, the problem should surface in this method as well.
Created attachment 194802 [details] change mentioned in comment 3 Committed the patch to HEAD.
(In reply to comment #4) > Created attachment 194802 [details] [diff] > change mentioned in comment 3 Committed the patch to perf_36x as well.
testApplicationNoParamters works fine. The three problematic methods are - testApplicationWithParamterNames testApplicationWithParamterGuesses testApplicationWithParamterGuesses2 So the problem should be either - in the computation of parameter suggestions or - in showing the popup with the parameter suggestions. Since the problem happens only on RHEL, I think it is the second. On my RHEL machine the CPU and Elapsed process times for the three tests in question are about 9-10 seconds. On the build machine these are 12-13 seconds. However I have not been able to reproduce the variable nature of the test. On my WinXP machine the numbers for the same three tests are 1.x seconds, i.e. similar to the numbers on the build machine.
(In reply to comment #6) > testApplicationNoParamters works fine. > > The three problematic methods are - > testApplicationWithParamterNames > testApplicationWithParamterGuesses > testApplicationWithParamterGuesses2 > > So the problem should be either > - in the computation of parameter suggestions or > - in showing the popup with the parameter suggestions. > Since the problem happens only on RHEL, I think it is the second. > > On my RHEL machine the CPU and Elapsed process times for the three tests in > question are about 9-10 seconds. On the build machine these are 12-13 seconds. > However I have not been able to reproduce the variable nature of the test. > > On my WinXP machine the numbers for the same three tests are 1.x seconds, i.e. > similar to the numbers on the build machine. Does the computation use lots of memory so that sometimes a GC happens?
(In reply to comment #7) > Does the computation use lots of memory so that sometimes a GC happens? Not particularly. I was profiling using yourkit today, there isn't much difference on Windows and Linux as far as GC is concerned. However on Linux Shell.setVisible(boolean) takes a bit of time (this in turn calls OS._g_main_context_iteration(int,boolean) which takes most of the time in the setVisible call). On Windows, Shell.setVisible(boolean) is extremely fast. The call to Shell.setVisible(boolean) comes from CompletionProposalPopup2.displayProposals() line 683. This method includes this comment - "// see bug 47511: setVisible may run the event loop on GTK and trigger a rentrant call - have to check whether we are still visible".
(In reply to comment #8) > However on Linux Shell.setVisible(boolean) takes a bit of time (this in turn > calls OS._g_main_context_iteration(int,boolean) which takes most of the time in > the setVisible call). On Windows, Shell.setVisible(boolean) is extremely fast. Filed bug 345093 for this. This explains why the test takes more time on Linux, but I still do not know the reason for the unreliable nature of the test.
With I20110504-0800 the test runs fast(2.24 seconds) on the build machine but on my RHEL machine it runs quite slow (11 seconds).
We need to add tracing information so that we can see what's going on. This will need more time and we don't want to play around with the tests at this point. ==> 3.8.
Things look better here after bug 345093 got fixed. I verified on my linux machine with I20111005-0925 that the CPU and Elapsed process times for the three tests in question are about 1.x second. Keeping this bug open till this is verified on the build machine, which may take a while as the performance tests for 3.8/4.2 are not yet running there.
Things look better in 3.8 http://download.eclipse.org/eclipse/downloads/drops/S-3.8M4-201112091447/performance/eplnx2/Scenario39.html ... as compared to 3.7 http://download.eclipse.org/eclipse/downloads/drops/R-3.7-201106131736/performance/eplnx2/Scenario32.html
