Bug 270750 - [perfs] Some 'Delete *' tests do not last long enough
Summary: [perfs] Some 'Delete *' tests do not last long enough
Status: VERIFIED WORKSFORME
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Text (show other bugs)
Version: 3.5   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.5 M7   Edit
Assignee: JDT-Text-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance, test
Depends on:
Blocks: 270824
  Show dependency tree
 
Reported: 2009-04-01 07:51 EDT by Frederic Fusier CLA
Modified: 2009-04-15 10:00 EDT (History)
2 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Frederic Fusier CLA 2009-04-01 07:51:34 EDT 
While verifying perf results for I20090331-0901 build, I saw that some JDT/Text 'Delete *' tests do not last enough (<100ms).

Here are the numbers on RHEL 5.0 machine:
 - DocumentPerformanceTest#measureDeleteInsert(): 18ms
 - DocumentPerformanceTest#measureInsertAtEnd():  33ms
 - DocumentPerformanceTest#measureInsertAtStart():76ms
Comment 1 Dani Megert CLA 2009-04-01 10:35:15 EDT 
We have them so that we know when they go through the roof.
Comment 2 Frederic Fusier CLA 2009-04-01 10:39:08 EDT 
(In reply to comment #1)
> We have them so that we know when they go through the roof.
> 
Ok, so it could be interesting to think about a mechanism in the framework to identify that such test duration is expected... And also report a warning otherwise...
Comment 3 Dani Megert CLA 2009-04-01 10:47:58 EDT 
This is already possible: we could specify an absolute band but so far I didn't see the real benefit given bug 89804 is marked as WONTFIX.
Comment 4 Frederic Fusier CLA 2009-04-08 09:29:08 EDT 
(In reply to comment #1)
> We have them so that we know when they go through the roof.
> 
Reopen as I think then this test should not be put in the fingerprints. One of the goal of fingerprint is to show tests which time is meaningful for users.

Other tests should be only visible in the status table.
Comment 5 Dani Megert CLA 2009-04-08 10:34:28 EDT 
Sorry, but even if the number is below 100 it can interest the user whether it became (much) slower or faster. And document manipulation is central to Text.

Also, I agree the fingerprints are targeted to the user but let's be serious: the main consumer/user are we.
Comment 6 Frederic Fusier CLA 2009-04-15 10:00:33 EDT 
Verified for 3.5M7