Bug 577185

Summary:	RuntimeProcessTests.testOutputAfterDestroy fails
Product:	[Eclipse Project] Platform	Reporter:	Andrey Loskutov <loskutov>
Component:	Debug	Assignee:	Jonah Graham <jonah>
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	P3	CC:	jonah, sarika.sinha, torbjorn.svensson
Version:	4.22	Keywords:	regression
Target Milestone:	4.22 M3
Hardware:	PC
OS:	Linux
See Also:	https://bugs.eclipse.org/bugs/show_bug.cgi?id=577189 https://git.eclipse.org/r/c/platform/eclipse.platform.debug/+/187593 https://git.eclipse.org/c/platform/eclipse.platform.debug.git/commit/?id=b703e4db3c9ed4e049a713c182b2d9417eb58e6e
Whiteboard:
Bug Depends on:
Bug Blocks:	576945

Description Andrey Loskutov

2021-11-10 08:51:46 EST

We see unstable test results and failures on Windows/Linux, introduced via bug 576945.

https://download.eclipse.org/eclipse/downloads/drops4/I20211109-1800/testresults/html/org.eclipse.debug.tests_ep422I-unit-cen64-gtk3-java11_linux.gtk.x86_64_11.html

N/A

java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at org.eclipse.debug.tests.console.RuntimeProcessTests.testOutputAfterDestroy(RuntimeProcessTests.java:235)


https://download.eclipse.org/eclipse/downloads/drops4/I20211109-1800/testresults/html/org.eclipse.debug.tests_ep422I-unit-win32-java11_win32.win32.x86_64_11.html

expected:<-1> but was:<76>

java.lang.AssertionError: expected:<-1> but was:<76>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:633)
at org.eclipse.debug.tests.console.RuntimeProcessTests.testOutputAfterDestroy(RuntimeProcessTests.java:232)

Torbjörn: could you please check?

Comment 1 Sarika Sinha

2021-11-10 09:00:37 EST

@Torbjörn,
Can you look at this?
Else we will have to disable this test.

Comment 2 Andrey Loskutov

2021-11-10 09:08:09 EST

Also fails on Mac, see https://download.eclipse.org/eclipse/downloads/drops4/I20211109-1800/testresults/html/org.eclipse.debug.tests_ep422I-unit-mac64-java17_macosx.cocoa.x86_64_17.html

N/A

java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at org.eclipse.debug.tests.console.RuntimeProcessTests.testOutputAfterDestroy(RuntimeProcessTests.java:235)

Comment 3 Jonah Graham

2021-11-10 09:11:05 EST

This test is failing because (as identified when fix went in for Bug 576945[1]) the problem is much improved, but not 100% fixed.

MockProcess has a comment on line 182 about adding a sleep - with that sleep the test fails reliably.

I suspect that the test should be disabled for now until other issues can be resolved.

@Torbjörn - do you agree with this conclusion?

Also, we should have a bug about the remaining issue identified (that it is still possible to miss some output)

[1] https://git.eclipse.org/r/c/platform/eclipse.platform.debug/+/187134/6#message-442a1a8f86505087075b577e92aff70e4d241f21

[2] https://git.eclipse.org/r/c/platform/eclipse.platform.debug/+/187134/6/org.eclipse.debug.tests/src/org/eclipse/debug/tests/console/MockProcess.java

Comment 4 Torbjörn Svensson

2021-11-10 09:19:09 EST

If the test case is that unreliable, I suppose we should disable it for now.
I've executed it several times on at least 3 different systems (2 Linux and 1 Windows 10) and so far, I've not got any failure as long as the commented sleep was not used.

Regarding the remaining error; @Jonah, can you create that bug as I think you had an idea on how to solve it.

Comment 5 Jonah Graham

2021-11-10 09:31:30 EST

OK - I will create a bug and disable the test.

Comment 6 Jonah Graham

2021-11-10 09:54:30 EST

Because I mentioned two bugs in commit comment the gerrit was attributed to the other bug. Manually entering the gerrit to disable test here.

@Sarika - I assume it is ok to submit disabling this test for M3? I have provisionally changed the Target Milestone.

Comment 7 Jonah Graham

2021-11-10 10:03:40 EST

Thanks Andrey for raising this, and for reviewing/merging the gerrit.

Comment 8 Sarika Sinha

2021-11-10 12:53:59 EST

(In reply to Jonah Graham from comment #6)
> Because I mentioned two bugs in commit comment the gerrit was attributed to
> the other bug. Manually entering the gerrit to disable test here.
> 
> @Sarika - I assume it is ok to submit disabling this test for M3? I have
> provisionally changed the Target Milestone.

Thanks!