Bug 560837 - org.eclipse.ui.editors.tests LargeFileTest.openLargeFileWithAnnotation test fails intermittently
Summary: org.eclipse.ui.editors.tests LargeFileTest.openLargeFileWithAnnotation test f...
Status: NEW
Alias: None
Product: Platform
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.15   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Platform-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-03-05 23:41 EST by Kalyan Prasad Tatavarthi CLA
Modified: 2021-02-19 00:17 EST (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kalyan Prasad Tatavarthi CLA 2020-03-05 23:41:32 EST
The test org.eclipse.ui.editors.tests.LargeFileTest.openLargeFileWithAnnotation failed in the build I20200305-0155

https://download.eclipse.org/eclipse/downloads/drops4/I20200305-0155/testresults/html/org.eclipse.ui.editors.tests_ep415I-unit-cen64-gtk3-java13_linux.gtk.x86_64_13.html

The error reported was

Opening large file took too long: 750.82983ms with marker vs. 291.90768ms without

java.lang.AssertionError: Opening large file took too long: 750.82983ms with marker vs. 291.90768ms without
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.eclipse.ui.editors.tests.LargeFileTest.openLargeFileWithAnnotation(LargeFileTest.java:132)
Comment 2 Thomas Wolf CLA 2020-05-29 03:27:14 EDT
This test was done for bug 550396, which was a 15 second UI freeze when opening a large file with a marker. The test has a comment at the assert:

  // Be generous here; all this timing is approximate. Fail if the attempt
  // with the marker takes more than twice as long. If the OverviewRuler
  // is badly implemented, opening with the marker will take much longer.

The assert fails if opening the file with a marker takes more than twice as long as opening it without marker.

I don't see any changes in OverviewRuler that would have made this file opening slower again. So it's either just normal performance variation between CI job runs, or something changed somewhere else (StyledText? -- The underlying cause of bug 550396 was line height computation) that really made opening the file with marker much slower again.

If it's only CI job performance variation, feel free to be even more generous in the assert and fail only if it takes 3 or 4 times as long.

Probably the test should also have a second assert on the absolute time. As a user, 1 second for opening a file of 40000 lines might be barely accesptable. 10 seconds definitely isn't, so failing if opening takes more than 10 seconds is an additional option.
Comment 4 Kalyan Prasad Tatavarthi CLA 2021-02-19 00:04:38 EST
(In reply to Kalyan Prasad Tatavarthi from comment #3)
> This test Failed on Mac for build I20210217-1800
> 
> https://download.eclipse.org/eclipse/downloads/drops4/I20210217-1800/
> testresults/html/org.eclipse.ui.editors.tests_ep419I-unit-mac64-
> java11_macosx.cocoa.x86_64_11.html

Error reported as below Opening large file took too long: 1090.01ms with marker vs. 456.69934ms without

java.lang.AssertionError: Opening large file took too long: 1090.01ms with marker vs. 456.69934ms without
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.eclipse.ui.editors.tests.LargeFileTest.openLargeFileWithAnnotation(LargeFileTest.java:135) .

Could be intermittent as the mac machine is very slow.