Bug 572413 - Performance tests for bug 543480 fluctuate in CI - especially with higher load
Summary: Performance tests for bug 543480 fluctuate in CI - especially with higher load
Status: CLOSED DUPLICATE of bug 572393
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.20   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 4.21   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: helpwanted
Depends on:
Blocks:
 
Reported: 2021-03-29 10:00 EDT by Manoj N Palat CLA
Modified: 2021-05-04 16:29 EDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Manoj N Palat CLA 2021-03-29 10:00:13 EDT
Related to bug 551899

For the record, from https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit/4337/testReport/junit/org.eclipse.jdt.core.tests.compiler.regression/GenericTypeTest/testBug543480BasedOnTest2FromComment4ToSameSameOptimization___1_8/

----------
Potential fluctuation of a performance test: average long compile time 1072.1666666666667ms should be less than 10.0x the average short compile time 6.166666666666667ms
long compile times: [#9 349ms (excluded), #10 366ms (excluded), #8 575ms, #5 1008ms, #4 1103ms, #7 1177ms, #3 1180ms, #2 1390ms, #1 1724ms (excluded), #6 9817ms (excluded)]
short compile times: [#3 5ms (excluded), #2 6ms (excluded), #4 6ms, #7 6ms, #8 6ms, #9 6ms, #10 6ms, #1 7ms, #6 7ms (excluded), #5 15ms (excluded)]
----------

Failure seems to happen more frequently (no data to back this claim though) when the load is high.

Would it mean that we need to fall back on Stephan Herrmann's suggestion of Plan B in bug 543480 comment 62 - quote : "As a plan B I would suggest to add test-only API to indeed measure the number of bounds created, even during a full compilation."
Comment 1 Stephan Herrmann CLA 2021-05-04 16:29:13 EDT

*** This bug has been marked as a duplicate of bug 572393 ***