Community
Participate
Working Groups
We see ~20 test failures on Windows related to bug 571055 tests. Example stack: junit.framework.ComparisonFailure: Unexpected error output for invocation with arguments [-annotationpath "C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\annots" -1.8 -proc:none -err:+nullAnnot -warn:+null "C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression"]. ----------- Expected ------------ ----------\n 1. ERROR in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 7)\n String result = api.m(null);\n ^^^^\n Null type mismatch: required '@NonNull String' but the provided value is null\n ----------\n 2. WARNING in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 8)\n System.out.println(result.toUpperCase());\n ^^^^^^\n Potential null pointer access: The variable result may be null at this location\n ----------\n 2 problems (1 error, 1 warning)\n ------------ but was ------------ ----------\r\n 1. ERROR in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 7)\r\n String result = api.m(null);\r\n ^^^^\r\n Null type mismatch: required '@NonNull String' but the provided value is null\r\n ----------\r\n 2. WARNING in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 8)\r\n System.out.println(result.toUpperCase());\r\n ^^^^^^\r\n Potential null pointer access: The variable result may be null at this location\r\n ----------\r\n 2 problems (1 error, 1 warning)\r\n --------- Difference is ---------- expected:<----------\[n 1. ERROR in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 7)\n String result = api.m(null);\n ^^^^\n Null type mismatch: required '@NonNull String' but the provided value is null\n ----------\n 2. WARNING in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 8)\n System.out.println(result.toUpperCase());\n ^^^^^^\n Potential null pointer access: The variable result may be null at this location\n ----------\n 2 problems (1 error, 1 warning)]\n > but was:<----------\[r\n 1. ERROR in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 7)\r\n String result = api.m(null);\r\n ^^^^\r\n Null type mismatch: required '@NonNull String' but the provided value is null\r\n ----------\r\n 2. WARNING in C:\Users\jenkins_vnc\AppData\Local\Temp\comptest\run.1613330339929\regression\test1\Test1.java (at line 8)\r\n System.out.println(result.toUpperCase());\r\n ^^^^^^\r\n Potential null pointer access: The variable result may be null at this location\r\n ----------\r\n 2 problems (1 error, 1 warning)\r]\n > at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertStringEquals(TestCase.java:266) at org.eclipse.jdt.core.tests.junit.extension.TestCase.assertEquals(TestCase.java:242) at org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest.runTest(AbstractBatchCompilerTest.java:655) at org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest.runNegativeTest(AbstractBatchCompilerTest.java:501) at org.eclipse.jdt.core.tests.compiler.regression.NullAnnotationBatchCompilerTest.runTestBug571055(NullAnnotationBatchCompilerTest.java:1117) at org.eclipse.jdt.core.tests.compiler.regression.NullAnnotationBatchCompilerTest.testBug571055_dedicatedAnnotationPath(NullAnnotationBatchCompilerTest.java:1032) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at junit.framework.TestCase.runTest(TestCase.java:177) at org.eclipse.jdt.core.tests.junit.extension.TestCase.runTest(TestCase.java:957) at junit.framework.TestCase.runBare(TestCase.java:142) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:130) at junit.framework.TestSuite.runTest(TestSuite.java:241) at junit.framework.TestSuite.run(TestSuite.java:236) at junit.framework.TestSuite.runTest(TestSuite.java:241) at junit.framework.TestSuite.run(TestSuite.java:236) at org.eclipse.jdt.core.tests.util.CompilerTestSetup.run(CompilerTestSetup.java:59) at junit.framework.TestSuite.runTest(TestSuite.java:241) at junit.framework.TestSuite.run(TestSuite.java:236) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/176350
This was a freaky failure. We get totally entangled in three different possible reasons why strings might differ: (1) line ends should always be normalized (2) some magic is needed for matching the file path in the error message (3) error messages could substantially differ According to (1) line ends are not the problem. According to the message of the failed assert (2) didn't look like the problem, nor is (3) to blame. It turns out the left hand of the test infrastructure doesn't know what the right hand does: We did detect a difference in paths, but this difference doesn't show in the failure. What's more: the detected difference is only *created* by the test infrastructure, by normalizing the path in the actual stderr text by replacing the real location with the string "---OUTPUT_DIR_PLACEHOLDER---". (In reply to Eclipse Genie from comment #1) > New Gerrit change created: > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/176350 Fixes two issues: * fix the actual test by using "---OUTPUT_DIR_PLACEHOLDER---" instead of the real path OUTPUT_DIR. * add assertions that detect bogus tests: output dir normalization should never have an effect on the *expected* out / err string. Since I don't have a Windows machine, the actual success can only be determined by the next integration build ...
@Andrey, I'm a bit out of sync: on the one hand gerrit tells me, I should not submit this right now, but there used to be an exception for test-only changes anyway. Is this exception still in place? If not, does the "regression"-exception apply to this? Or should I really wait till after M3?
(In reply to Stephan Herrmann from comment #3) > @Andrey, I'm a bit out of sync: on the one hand gerrit tells me, I should > not submit this right now, but there used to be an exception for test-only > changes anyway. Is this exception still in place? Sure, so once the gerrit will be green in actual JDT tests, we can merge. The gerrit warning is for committers that don't read/don't understand releng mails about freeze periods.
Gerrit change https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/176350 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=4ffde08df16264cfbff159024893e456a508cde8
(In reply to Eclipse Genie from comment #5) > Gerrit change https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/176350 was > merged to [master]. > Commit: > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/ > ?id=4ffde08df16264cfbff159024893e456a508cde8 Released for 4.19 M3
Verified for Eclipse 4.19 M3 with test results at https://download.eclipse.org/eclipse/downloads/drops4/I20210217-1800/testResults.php
