Community
Participate
Working Groups
for some tests an absolute threshold should be used, i.e., switching a perspective should not be slower than 100ms. This is supported in the tests infrastructure (assertPerformanceInAbsoluteBand()), but there is currently no way to render these tests in a summary.
What do you think about using a line graph represent this? We could draw a horizontal line across the graph to represent the threshold value then plot points in green or red depending on where it falls in relation to the threshold.
from my side nothing is planned for 3.2
Closing.
I'll try to figure out what can be done for this...
> This is supported in the tests infrastructure > (assertPerformanceInAbsoluteBand()), but > there is currently no way to render these tests in a summary. It's not only about the summary/fingerprint. Performance#assertPerformanceInAbsoluteBand(..) does not seem to be used in the test results table as well. E.g. on http://download.eclipse.org/eclipse/downloads/drops/I20090414-0800/performance/org.eclipse.jdt.ui.php?fp_type=0 , the test PackageExplorerWarmPerfTest#testOpen() ends with this: Performance.getDefault().assertPerformanceInAbsoluteBand(fPerformanceMeter, Dimension.ELAPSED_PROCESS, 0, 500); But the test still gets an X in the table and the hover says: Performance criteria not met when compared to 'R-3.4-200806172000_200904101800': Elapsed Process= 80ms is not within [0%, 110'%] of 70ms".
I also think the implementation of the lower band in AbsoluteBandChecker#test(..) is wrong: if (actual > fUpperBand + test || actual < test - fLowerBand) { message.append( ... + " is not within [-" + dimension.getDisplayValue(new Scalar(null, fLowerBand)) + ... ); return false; } The Javadoc of Performance#assertPerformanceInAbsoluteBand(..) says: * @param lowerBand a negative number indicating the absolute amount the measured value is allowed to be smaller than some reference value * @param upperBand a positive number indicating the absolute amount the measured value is allowed to be greater than some reference value => Javadoc says lowerBand should be negative, but the implementation assumes fLowerBand is positive.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.
Perf tests need to publish results first in order to see current state and improve after that. 12 years old bugs are not helpful
