Bug 559476 - [Tests] Some UI tests leak modal shells and tests should fail for this
Summary: [Tests] Some UI tests leak modal shells and tests should fail for this
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.14   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 4.15 M3   Edit
Assignee: Paul Pazderski CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 544675 552614
  Show dependency tree
 
Reported: 2020-01-23 18:22 EST by Paul Pazderski CLA
Modified: 2020-01-26 14:03 EST (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Pazderski CLA 2020-01-23 18:22:08 EST
Well 'some' is a little exaggerated. I found two tests opening but not closing a modal/modeless shell/dialog.

Because this text might get a bit long here is a summary:
 - fix tests leaking modal shells
 - as consequence TextHandlerTests should not fail anymore or fail for another reason
 - add protection to prevent further modal shell leaks.
 - fix testPreviousChoicesAvailable test


But the consequences of those leaks are awful. It is responsible for the test failure of TextHandlerTest testEditableText (Bug 544675) and testNonEditableText (Bug 552614). One reason I did not directly attached the patch there is that I'm very sure for testEditableText and quite sure for testNonEditableText that they will continue failing on Windows for another reason after this is fixed. (failing on another assert so we can see if my assumption is correct)

So why does a leaked modal shell fail this test and why does it not fail on Linux? The tests fail on the first enablement of the copy handler. The text widget is empty therefor the handler should be disabled. But it is not on Mac and Windows. The reason lies in the enablement update code (I do not plan to change) if there is an active text widget the selection determined the enablement (select -> enabled). If no text widget is active the actions enablement state is used which is always true for this test.

The 'active text widget' is set from focus events. And here is the failure. Due to the open modal shell the text widget cannot given focus. It does not work on Windows and I assume it does not work on Mac as well.
On Linux however the set focus works regardless of the open modal dialog.


One test missing cleanup can fail an entirely different test without clear connection and maybe hard to debug defects in the actual correct test.
To prevent such problems I'd like to add a check in UITestCase.tearDown to close any leaked modal shell and fail the offending test.
This is not perfect because not any test is using UITestCase and if such a test does not close a modal dialog the next UITestCase would report the failure. However any test failure on a gerrit build should be investigated and even if this is not always the case due to unstable tests a leaked modal shell should always considered a sever failure.


Something that might look unrelated but is probably not is QuickAccessDialogTest.testPreviousChoicesAvailable this test failed after I correctly closed the dialogs in the two bad tests. I cannot fully explain the connection. It seem that Mickael already had some trouble [1] with this test which might be related to those shell leaks.
Comment 1 Eclipse Genie CLA 2020-01-23 18:24:27 EST
New Gerrit change created: https://git.eclipse.org/r/156479
Comment 2 Eclipse Genie CLA 2020-01-23 18:24:47 EST
New Gerrit change created: https://git.eclipse.org/r/156480
Comment 3 Eclipse Genie CLA 2020-01-23 18:25:32 EST
New Gerrit change created: https://git.eclipse.org/r/156481
Comment 6 Eclipse Genie CLA 2020-01-25 06:39:19 EST
New Gerrit change created: https://git.eclipse.org/r/156559
Comment 7 Paul Pazderski CLA 2020-01-25 06:44:32 EST
(In reply to Paul Pazderski from comment #0)
> One reason I did not directly attached the
> patch there is that I'm very sure for testEditableText and quite sure for
> testNonEditableText that they will continue failing on Windows for another
> reason after this is fixed. (failing on another assert so we can see if my
> assumption is correct)

Well, was not so lucky with my prediction. The good news is that closing the modal shells helped the failing TextHandlerTests. Both succeeded on Windows and one failed on Mac on another assert while the other passed.
For testEditableText my prediction was wrong because testNonEditableText was run before. And for testNonEditableText I had seen locally a failure which was a race condition and obviously did not happened on the test machine.