Bug 306172

Summary: [perfs] Invalid test duration for FullSourceWorkspaceTypeHierarchyTests#testPerSuperTypes()
Product: [Eclipse Project] JDT Reporter: Frederic Fusier <frederic_fusier>
Component: CoreAssignee: Satyam Kandula <satyam.kandula>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: Olivier_Thomann
Version: 3.6   
Target Milestone: 3.6 M7   
Hardware: All   
OS: All   
Whiteboard:
Bug Depends on:    
Bug Blocks: 286955    
Attachments:
Description Flags
Proposed patch
frederic_fusier: review-
Proposed patch
frederic_fusier: iplog+, frederic_fusier: review+
Proposed patch for perf_35x frederic_fusier: iplog+, frederic_fusier: review+

Description Frederic Fusier CLA 2010-03-17 07:37:27 EDT 
Verifying 3.6M6 and I201016-0859 perf results, I noticed an invalid duration for FullSourceWorkspaceTypeHierarchyTests#testPerSuperTypes() perf test.

Note also that the test summary is the same than FullSourceWorkspaceTypeHierarchyTests#testPerfAllTypes() and should be fixed as well...
Comment 1 Satyam Kandula CLA 2010-04-21 02:01:08 EDT 
Created attachment 165527 [details]
Proposed patch

Put in a better tag, increased the WARMUP_COUNT and put it in a loop inside the measuring loop. Now, this test takes a decent time of 160ms on my m/c.
Comment 2 Frederic Fusier CLA 2010-04-21 04:39:19 EDT 
Comment on attachment 165527 [details]
Proposed patch

Do not put the test in the fingerprint until we get enough history to see whether it's stable enough.

Also, is there a specific reason to have increased the number of loops for the warmup? The time does not matter for those loops except that it will make the global time to run the test suite longer...

Do you have run this test on top of perf_35x? I would like to have a patch for this to be ready before the Friday's baseline run and also have an idea of its duration to figure out whether we should expect an improvement or a regression...

TIA
Comment 3 Satyam Kandula CLA 2010-04-22 01:28:58 EDT 
Created attachment 165689 [details]
Proposed patch

Removed the finger print and also changed the inner loop to 20.
Comment 4 Satyam Kandula CLA 2010-04-22 01:31:22 EDT 
Created attachment 165690 [details]
Proposed patch for perf_35x 

Similar as the eariler patch for 3.5
Comment 5 Satyam Kandula CLA 2010-04-22 01:42:09 EDT 
Frederic,
The finger print is removed as for your comment. The warmup count is increased, because, otherwise I was seeing lot of variance in the test results. 
Had to increase the inner loop to 20, as I see that 10 was taking lesser time -- probably my laptop was running on the battery then. 
Now, the test takes 140ms on both Head and 3.5 maintenance branch. The fix that I did for which I had included this test was delivered on 3.5.2 also. Hence, the time should be similar and they are as expected :).
Comment 6 Frederic Fusier CLA 2010-04-22 13:06:24 EDT 
(In reply to comment #4)
> Created an attachment (id=165690) [details]
> Proposed patch for perf_35x 
> 
> Similar as the eariler patch for 3.5

Released in perf_35x branch. I've versionned the project as v_963_perf35xc and updated the map file for today's 6pm baseline run...
Comment 7 Frederic Fusier CLA 2010-04-22 13:09:49 EDT 
(In reply to comment #3)
> Created an attachment (id=165689) [details]
> Proposed patch
> 
> Removed the finger print and also changed the inner loop to 20.

Released for 3.6M7 in HEAD stream.
Comment 8 Frederic Fusier CLA 2010-04-28 09:35:32 EDT 
Verified with performance results of I20100425-2000 build.