Bug 567986 - [tests] Performance Regression
Summary: [tests] Performance Regression
Status: NEW
Alias: None
Product: QVTo
Classification: Modeling
Component: Engine (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 10
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-10-19 04:07 EDT by Ed Willink CLA
Modified: 2020-10-19 04:09 EDT (History)
0 users

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Willink CLA 2020-10-19 04:07:31 EDT 
Bug 486487 was REOPENED since the fix caused a performance degradation. Such degradations are inevitable especially now that there is very little review input from any of the original development team.

At least some of the tests should run standalone with unduly poor performance diagnosed.

---

How do we accommodate perhaps 5-fold uncertainty in respect of the current CPU speed?

How do we accommodate perhaps 10-fold uncertainty in respect of concurrent activities, particularly on a Jenkins server?

How do we isolate JVM ccentricities?

---

? Ensure that all I/O is performed before/after the instrumented execution time.

? Ensure that the instrumented execution is run at leasdt ten times before use to warm up the JVM.

? Run a CPU benchmark to normalize wrt the current CPU speed

? Maintain a test+QVTo-version+Eclipse-version+Java-version+platform+time+normalized-time database so that trends can be observed.