Bugzilla – Full Text Bug Listing
| Summary: | FullSourceWorkspaceCompletionTests doesn't run correctly | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] JDT | Reporter: | David Audel <david_audel> | ||||
| Component: | Core | Assignee: | Frederic Fusier <frederic_fusier> | ||||
| Status: | VERIFIED FIXED | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | P3 | CC: | philippe_mulet | ||||
| Version: | 3.3 | Keywords: | performance | ||||
| Target Milestone: | 3.3 M6 | ||||||
| Hardware: | PC | ||||||
| OS: | Windows XP | ||||||
| Whiteboard: | |||||||
| Attachments: |
|
||||||
Created attachment 60916 [details] Proposed patch Here are the results now correctly printed with this patch: ------------------------------------- Completion performance test statistics: - 400 completions have been performed - following proposals have been done: + test 0: 29,110 proposals + test 1: 67,240 proposals + test 2: 304,630 proposals + test 3: 72,980 proposals + test 4: 333,330 proposals + test 5: 72,570 proposals + test 6: 2,460 proposals + test 7: 19,680 proposals + test 8: 333,330 proposals ------------------------------------- Note that all tests have been renamed as time is completely different without this fix. Here are numbers got while running these tests with the fix on our local performance Linux test box: test0 test1 test2 test3 test4 test5 test6 test7 test8 ----------------------------------------------------------------------------- R3_2 13741 9798 23686 9685 24465 9645 9006 6999 21828 M5 13489 9446 23215 9346 23599 9264 22353 6705 21187 HEAD 13447 9437 21416 9935 22463 9805 9139 7287 20005 ============================================================================= Difference using R3_2 as reference: test0 test1 test2 test3 test4 test5 test6 test7 test8 ----------------------------------------------------------------------------- M5 1,83% 3,59% 1,99% 3,50% 3,54% 3,95% -148,2% 4,20% 2,94% HEAD 2,13% 3,68% 9,58% -2,59% 8,19% -1,66% -1,48% -4,12% 8,35% ============================================================================= Problem on test5 (M5) is due to bug 176971 which is fixed in HEAD stream. Perfs regression on test3, 5, 6 & 7 is due functionality addition while fixing bug 162865... You should read "Problem on test6 (M5)" in previous comment Released for 3.3 M6 in HEAD stream. Released in perf_32x stream to synchronize changes in reference. Note that for 3.2, statistics are a little bit different: ------------------------------------- Completion performance test statistics: - 400 completions have been performed - following proposals have been done: + test 0: 28,700 proposals + test 1: 67,240 proposals + test 2: 304,630 proposals + test 3: 71,750 proposals + test 4: 333,330 proposals + test 5: 71,750 proposals + test 6: 2,460 proposals + test 7: 18,450 proposals + test 8: 333,330 proposals ------------------------------------- Verified for 3.3 M6 using build I20070320-0010 |
An IndexOutOfBoundsException is thrown at the first accepted completion proposal and is caught inside CompletionEngine#complete(). The exception come from: public class FullSourceWorkspaceCompletionTests { ... class TestCompletionRequestor extends CompletionRequestor { public void accept(CompletionProposal proposal) { PROPOSAL_COUNTS[TESTS_LENGTH-TESTS_COUNT]++; } } ... } The problem is that TESTS_LENGTH is never initialized. This problem was introduced in revision 1.4 o FullSourceWorkspaceCompletionTests (11/22/05). So these performance tests aren't accurate since this date.