Bug 486013 - [1.9][tests] Test case issues
Summary: [1.9][tests] Test case issues
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.6   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Jay Arthanareeswaran CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
: 529981 530212 (view as bug list)
Depends on: 477387
Blocks: 457413
  Show dependency tree
 
Reported: 2016-01-18 02:25 EST by Manoj N Palat CLA
Modified: 2022-05-26 20:18 EDT (History)
7 users (show)

See Also:


Attachments
tracing output (71.79 KB, text/plain)
2017-11-14 07:08 EST, Stephan Herrmann CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Manoj N Palat CLA 2016-01-18 02:25:22 EST
Umbrella bug for addressing all 1.9 test case issues.
Comment 1 Jay Arthanareeswaran CLA 2016-06-07 02:54:19 EDT
First set of fixes released via:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA9&id=9155dfe5fa69fb2096196e5313e43eeabab82363

This addresses all the failures in org.eclipse.jdt.core.tests.eval.TestAll.

What's left now fall largely in two buckets:

1. Failures due to additional warning about deprecated stuff in Java 9.
2. Our JRT file system not considering javax.* modules, which are required by som tests.
Comment 2 Jay Arthanareeswaran CLA 2016-06-08 12:59:36 EDT
(In reply to Jay Arthanareeswaran from comment #1)
> 1. Failures due to additional warning about deprecated stuff in Java 9.

This is fixed via:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA9&id=aee2925515ff92b7466c70c1632476d226c7aba8
Comment 3 Jay Arthanareeswaran CLA 2016-06-09 01:59:25 EDT
These are the four tests currently failing:

test008 - (org.eclipse.jdt.core.tests.compiler.regression.SerialVersionUIDTests)
test009 - (org.eclipse.jdt.core.tests.compiler.regression.SerialVersionUIDTests)
test010 - (org.eclipse.jdt.core.tests.compiler.regression.SerialVersionUIDTests)
testBug386356_1 - (org.eclipse.jdt.core.tests.compiler.regression.AnnotationTest)

Totally there are 26 failures across all compliance levels.

Besides, the following two tests have been disabled for now and should be investigated:

CodeSnippetTest.testPackage()
NegativeCodeSnippetTest.testChangePackage()
Comment 4 Jay Arthanareeswaran CLA 2016-06-09 10:47:00 EDT
Looks like the SerialVersionUIDTests tests are failing because our effort in loading classes from non java.base modules through our extended class loader is failing. This class loader is redirecting all requests for system classes to findSystemClass(). But my guess is in classpath mode, non java.base modules are not available. Perhaps there's a way; will check with the Jigsaw experts.

I did a small experiment by commenting out the overridden loadClass in VerifyClassLoader and this makes all mentioned tests pass. But obviously this is done for some reason, proven by other failing tests.
Comment 5 Manoj N Palat CLA 2016-06-09 12:16:43 EDT
(In reply to Jay Arthanareeswaran from comment #2)
> (In reply to Jay Arthanareeswaran from comment #1)
> > 1. Failures due to additional warning about deprecated stuff in Java 9.
> 
> This is fixed via:
> 
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/
> ?h=BETA_JAVA9&id=aee2925515ff92b7466c70c1632476d226c7aba8

Jay I see the failure for ConditionalExpressionTest @ 8
Comment 6 Jay Arthanareeswaran CLA 2016-07-26 00:35:35 EDT
(In reply to Manoj Palat from comment #5)
> Jay I see the failure for ConditionalExpressionTest @ 8

This no longer fails. However, there are a bunch of failures in the following suites:

org.eclipse.jdt.compiler.apt.tests.BatchDispatchTests
org.eclipse.jdt.compiler.apt.tests.FilerTests
org.eclipse.jdt.compiler.apt.tests.NegativeTests

Causing being the same in all cases. the failing test is:

junit.framework.TestCase.assertEquals("Only one compiler available", 1, compilerCounter);

in BatchTestUtils line number 288.
Comment 7 Jay Arthanareeswaran CLA 2016-07-26 03:02:40 EDT
(In reply to Jay Arthanareeswaran from comment #6)

Looks like there is a change in behavior of java.util.ServiceLoader.load(Class<T>) in JRE. In JRE 8, we get only one instance of this after loading EclipseCompiler with a Class.forName(), but in JRE 9 we get both JavacTool and EclipseCompiler. The EclipseCompiler is retrieved even when I comment out the Class.forName(). I wonder if the test should worry about this at all.
Comment 8 Jay Arthanareeswaran CLA 2016-08-01 01:39:41 EDT
The following annotation related tests are failing:

testProcessorArgumentsWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.BatchDispatchTests)
testCompilerOneClassWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.BatchDispatchTests)
testTwoAnnotations(org.eclipse.jdt.compiler.apt.tests.BatchDispatchTests)
testElementWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.FilerTests)
testNegativeModel10WithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.NegativeTests)

The failure are caused by this commit:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA9&id=0171d3fa20410a884ea6de07ec393bc179fb73bd

The compiler is calling reset() on the annotation processing manager after each round (in the same session, though) of compilation, which results in loosing the loaded processors. Needs a closer look.
Comment 10 Jay Arthanareeswaran CLA 2016-09-09 03:13:33 EDT
One more failing test was found with JRE 9 and is fixed with:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA9&id=4853bfa39b4537870df1c5f0503a1078575fae46

and the failing tests @1.8 mentioned with comment #9 are disable for now:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA9&id=4853bfa39b4537870df1c5f0503a1078575fae46

At 1.9, there these failures:

testInheritedAnnosWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.BatchDispatchTests)
testProcessorArgumentsWithSystemCompiler(org.eclipse.jdt.compiler.apt.tests.BatchDispatchTests)
testTypeMirrorWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
testTypeMirrorWithSystemCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
testGenericsWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
testElementWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
testPackageAnnotationsWithJavac(org.eclipse.jdt.compiler.apt.tests.Java8ElementsTests)

And whole suite is unable to run with JRE 9: org.eclipse.jdt.compiler.tool.tests.AllTests
Comment 11 Jay Arthanareeswaran CLA 2016-09-28 03:11:16 EDT
The following tests are disabled for now:

AnnotationTest#testBug386356_1
SerialVersionUIDTests#test008
SerialVersionUIDTests#test009
SerialVersionUIDTests#test010

These don't fail when run alone, but fail when run as part of the suite.
Comment 12 Manoj N Palat CLA 2016-11-23 01:39:06 EST
note: RunAllJava9Tests (ref bug 506952) can be used run all Java 9 tests
Comment 13 Stephan Herrmann CLA 2017-05-28 10:22:43 EDT
Locally running tests.compiler.regression on 9-ea+164 I observe these regressions:

GenericTypeTest at 1.5+
- test1151, test1153, test1155
  -> reflection now prints inners with '$' instead of expected '.'

NullAnnotationBatchCompilerTest at 1.5+
- test440477, test440687b
  -> unexpected problems
- test490010NoEeaFile3
  -> different problem / severity

LambdaExpressionTest at 1.8+
- test039, test447119d, test449063, test449063a, test449063e
  -> various errors

SerializeLambdaTest at 1.8+
- testbug479119, testbug479119a
  -> InaccessibleObjectException

XLargeTest at 9
- test0016, test0017, test0018, test0019
  -> IllegalAccessError

GenericsRegressionTest_1_8 at 1.8+:
- testBug494733_comment0
  -> unexpected warning

EnumTest at 9
- test434442
  -> IllegalAccessError


Total: 51 failures

I don't see these in gerrit nor Y-builds (running on 1.8?).
Anybody else seeing these?
Comment 14 Stephan Herrmann CLA 2017-05-28 14:57:12 EDT
Locally running AllJavaModelTests on 9-ea+164 I see:

- 7 errors
- 520 failures

Is that expected? Reproduced by anybody else?
Comment 15 Stephan Herrmann CLA 2017-05-28 15:08:01 EDT
(In reply to Stephan Herrmann from comment #14)
> Locally running AllJavaModelTests on 9-ea+164 I see:
> 
> - 7 errors
> - 520 failures
> 
> Is that expected? Reproduced by anybody else?

FWIW, total # of tests was 7989 
(running off of https://git.eclipse.org/r/#/c/93798/14)
In a recent Y-build I see 7949, not sure if the diff is fully attributed to that gerrit change.
Of course some tests still seem to pass when the don't find the expected JRE.

Further note, that gerrit doesn't even tell us about AllJavaModelTests, since compiler tests already fail the build.
Comment 16 Manoj N Palat CLA 2017-05-29 11:35:53 EDT
I see 7976 tests - 5 failed , rest passed.
with ea 169, at commit bd6803034b95b7e0dd8c0cbcd0aead0a5c726f65 on windows
Comment 17 Stephan Herrmann CLA 2017-05-29 17:41:47 EDT
(In reply to Manoj Palat from comment #16)
> I see 7976 tests - 5 failed , rest passed.
> with ea 169, at commit bd6803034b95b7e0dd8c0cbcd0aead0a5c726f65 on windows

Alright, on HEAD of BETA_JAVA9 things look better indeed, same numbers here.

The problem exists only in https://git.eclipse.org/r/#/c/93798/
I checked if my #14 broke anything, but already at #13 I get 7 errors 520 failures.
Comment 18 Sasikanth Bharadwaj CLA 2017-06-05 04:19:39 EDT
(In reply to comment #13)
> Locally running tests.compiler.regression on 9-ea+164 I observe these
> regressions:
> 
> GenericTypeTest at 1.5+
> - test1151, test1153, test1155
> -> reflection now prints inners with '$' instead of expected '.'
> 
> NullAnnotationBatchCompilerTest at 1.5+
> - test440477, test440687b
> -> unexpected problems
> - test490010NoEeaFile3
> -> different problem / severity
> 
> LambdaExpressionTest at 1.8+
> - test039, test447119d, test449063, test449063a, test449063e
> -> various errors
> 
> SerializeLambdaTest at 1.8+
> - testbug479119, testbug479119a
> -> InaccessibleObjectException
> 
> XLargeTest at 9
> - test0016, test0017, test0018, test0019
> -> IllegalAccessError
> 
> GenericsRegressionTest_1_8 at 1.8+:
> - testBug494733_comment0
> -> unexpected warning
> 
> EnumTest at 9
> - test434442
> -> IllegalAccessError
> 
> 
> Total: 51 failures
> 
> I don't see these in gerrit nor Y-builds (running on 1.8?).
> Anybody else seeing these?
Yes, I believe we have bugs for each of these failures

(In reply to comment #17)
> (In reply to Manoj Palat from comment #16)
> > I see 7976 tests - 5 failed , rest passed.
> > with ea 169, at commit bd6803034b95b7e0dd8c0cbcd0aead0a5c726f65 on windows
> 
> Alright, on HEAD of BETA_JAVA9 things look better indeed, same numbers here.
> 
> The problem exists only in https://git.eclipse.org/r/#/c/93798/
> I checked if my #14 broke anything, but already at #13 I get 7 errors 520
> failures.
I'm on it, I'm sure it must be a result of my changes to JavaSearchNameEnvironment
Comment 19 Stephan Herrmann CLA 2017-06-05 04:41:07 EDT
(In reply to Sasikanth Bharadwaj from comment #18)
> (In reply to comment #13)
> > Locally running tests.compiler.regression on 9-ea+164 I observe these
> > regressions:
> > 
> > GenericTypeTest at 1.5+
> > - test1151, test1153, test1155
> > -> reflection now prints inners with '$' instead of expected '.'
> > 
> > NullAnnotationBatchCompilerTest at 1.5+
> > - test440477, test440687b
> > -> unexpected problems
> > - test490010NoEeaFile3
> > -> different problem / severity
> > 
> > LambdaExpressionTest at 1.8+
> > - test039, test447119d, test449063, test449063a, test449063e
> > -> various errors
> > 
> > SerializeLambdaTest at 1.8+
> > - testbug479119, testbug479119a
> > -> InaccessibleObjectException
> > 
> > XLargeTest at 9
> > - test0016, test0017, test0018, test0019
> > -> IllegalAccessError
> > 
> > GenericsRegressionTest_1_8 at 1.8+:
> > - testBug494733_comment0
> > -> unexpected warning
> > 
> > EnumTest at 9
> > - test434442
> > -> IllegalAccessError
> > 
> > 
> > Total: 51 failures
> > 
> > I don't see these in gerrit nor Y-builds (running on 1.8?).
> > Anybody else seeing these?
> Yes, I believe we have bugs for each of these failures

I have fixes for some of these. Could you point me to the individual bugs so I can put the fixes into the right buckets?

Question remains: are any gerrit or releng builds actually running on 9?
Comment 20 Sasikanth Bharadwaj CLA 2017-06-05 06:02:34 EDT
(In reply to comment #19)
> I have fixes for some of these. Could you point me to the individual bugs so I
> can put the fixes into the right buckets?
> 
bug 509123 for GTT and LET
bug 509753 for XLargeTest and EnumTest
bug 509754 for SLT

> Question remains: are any gerrit or releng builds actually running on 9?
no, I run RunAllJava9Tests locally using jdk9 and rely on gerrit for other regressions, but even that is not sufficient as you pointed out
Comment 21 Stephan Herrmann CLA 2017-06-05 11:17:18 EDT
(In reply to Sasikanth Bharadwaj from comment #20)
> (In reply to comment #19)
> > I have fixes for some of these. Could you point me to the individual bugs so I
> > can put the fixes into the right buckets?
> > 
> bug 509123 for GTT and LET

Fix released, but one issue remains (see bug 509123 comment 3):
- commit 85bb2bb3 (bug 480992) promises specialized error messages mentioning the involved modules, which, however, doesn't seem to be implemented? Should we just revert the test change or is anybody working on such improved error message?
Comment 22 Stephan Herrmann CLA 2017-07-06 16:33:56 EDT
(In reply to Stephan Herrmann from comment #21)
> (In reply to Sasikanth Bharadwaj from comment #20)
> > (In reply to comment #19)
> > > I have fixes for some of these. Could you point me to the individual bugs so I
> > > can put the fixes into the right buckets?
> > > 
> > bug 509123 for GTT and LET
> 
> Fix released, but one issue remains (see bug 509123 comment 3):
> - commit 85bb2bb3 (bug 480992) promises specialized error messages
> mentioning the involved modules, which, however, doesn't seem to be
> implemented? Should we just revert the test change or is anybody working on
> such improved error message?

Can anybody comment on the previous expected error message:
"X (in module: Unnamed Module) cannot be cast to I (in module: Unnamed Module)"

When writing bug 517808 comment 37 I was thinking of extending this message:

  The method serve(org.impl.SomeImpl) in the type Serve is not applicable for the arguments (org.impl.SomeImpl)

to

  The method serve(mod.one/org.impl.SomeImpl) in the type Serve is not applicable for the arguments (mod.two/org.impl.SomeImpl)

See also, that runtime stack traces already use the module/pack.age.Type syntax.

*If* ever we introduce such syntax in error messages, we probably want this only, if otherwise the names would be identical, right?
Comment 23 Stephan Herrmann CLA 2017-07-24 11:59:07 EDT
I again see failures in these when run on 9:

(In reply to Jay Arthanareeswaran from comment #10)
> At 1.9, there these failures:
> 
> [...]
> testElementWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
> testPackageAnnotationsWithJavac(org.eclipse.jdt.compiler.apt.tests.
> Java8ElementsTests)

Is this WIP, or s.t. I broke recently?
Comment 24 Jay Arthanareeswaran CLA 2017-07-24 12:22:51 EDT
(In reply to Stephan Herrmann from comment #23)
> I again see failures in these when run on 9:
> 
> (In reply to Jay Arthanareeswaran from comment #10)
> > At 1.9, there these failures:
> > 
> > [...]
> > testElementWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
> > testPackageAnnotationsWithJavac(org.eclipse.jdt.compiler.apt.tests.
> > Java8ElementsTests)
> 
> Is this WIP, or s.t. I broke recently?

I wouldn't worry about it for the moment. I mean, you needn't worry :)

I plan to tackle them with 517841.
Comment 25 Jay Arthanareeswaran CLA 2017-07-24 12:29:37 EDT
(In reply to Jay Arthanareeswaran from comment #24)
> I wouldn't worry about it for the moment. I mean, you needn't worry :)
> 
> I plan to tackle them with 517841.

I should have mentioned, I have been seeing them at least as far back as July 6th.
Comment 26 Stephan Herrmann CLA 2017-07-24 12:30:33 EDT
(In reply to Jay Arthanareeswaran from comment #24)
> (In reply to Stephan Herrmann from comment #23)
> > I again see failures in these when run on 9:
> > 
> > (In reply to Jay Arthanareeswaran from comment #10)
> > > At 1.9, there these failures:
> > > 
> > > [...]
> > > testElementWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
> > > testPackageAnnotationsWithJavac(org.eclipse.jdt.compiler.apt.tests.
> > > Java8ElementsTests)
> > 
> > Is this WIP, or s.t. I broke recently?
> 
> I wouldn't worry about it for the moment. I mean, you needn't worry :)

So I don't :)
Comment 27 Stephan Herrmann CLA 2017-08-19 15:04:09 EDT
Running on 9 I see NPE in these tests:

- org.eclipse.jdt.core.tests.eval.CodeSnippetTest.testFor89632()
- org.eclipse.jdt.core.tests.eval.NegativeCodeSnippetTest.testUnusedImport()

I'm looking into these.
Comment 28 Eclipse Genie CLA 2017-08-19 16:27:59 EDT
New Gerrit change created: https://git.eclipse.org/r/103333
Comment 29 Stephan Herrmann CLA 2017-08-19 16:30:40 EDT
(In reply to Eclipse Genie from comment #28)
> New Gerrit change created: https://git.eclipse.org/r/103333

Found the cause in the fix for bug 519723:

   Path("List.class").endsWith(".class") is not true in java.nio.file.

interesting API :-/
Comment 31 Stephan Herrmann CLA 2017-08-24 17:19:16 EDT
If seen failures in 
    ModuleCompilationTests.testBug519922
when run locally via RunAllJava9Tests. I haven't yet seen the problem when running only ModuleCompilationTests.

Difference is that in org.eclipse.SomeInterface we sometimes complain
  org cannot be resolved to a type
where we expect
  org.eclipse cannot be resolved to a type

I haven't investigated this, nor do I think this is severe, just wanted to let others know.
Comment 32 Stephan Herrmann CLA 2017-08-29 14:39:17 EDT
(In reply to Jay Arthanareeswaran from comment #24)
> (In reply to Stephan Herrmann from comment #23)
> > I again see failures in these when run on 9:
> > 
> > (In reply to Jay Arthanareeswaran from comment #10)
> > > At 1.9, there these failures:
> > > 
> > > [...]
> > > testElementWithEclipseCompiler(org.eclipse.jdt.compiler.apt.tests.ModelTests)
> > > testPackageAnnotationsWithJavac(org.eclipse.jdt.compiler.apt.tests.
> > > Java8ElementsTests)
> > 
> > Is this WIP, or s.t. I broke recently?
> 
> I wouldn't worry about it for the moment. I mean, you needn't worry :)
> 
> I plan to tackle them with 517841.

For once I saw Java8ElementsTests.testTypeAnnotations12Binary() on gerrit (using 1.8 of course):

java.lang.NullPointerException
	at org.eclipse.jdt.internal.compiler.apt.model.ElementsImpl.addMembers(ElementsImpl.java:216)
	at org.eclipse.jdt.internal.compiler.apt.model.ElementsImpl.getAllMembers(ElementsImpl.java:163)
	at org.eclipse.jdt.compiler.apt.tests.processors.elements.Java8ElementProcessor.tTypeAnnotations12(Java8ElementProcessor.java:605)
	at org.eclipse.jdt.compiler.apt.tests.processors.elements.Java8ElementProcessor.testTypeAnnotations12Binary(Java8ElementProcessor.java:641)

I couldn't reproduce this locally, though.
Comment 33 Sasikanth Bharadwaj CLA 2017-08-31 06:19:24 EDT
(In reply to comment #31)
> If seen failures in
> ModuleCompilationTests.testBug519922
> when run locally via RunAllJava9Tests. I haven't yet seen the problem when
> running only ModuleCompilationTests.
> 
> Difference is that in org.eclipse.SomeInterface we sometimes complain
> org cannot be resolved to a type
> where we expect
> org.eclipse cannot be resolved to a type
> 
> I haven't investigated this, nor do I think this is severe, just wanted to let
> others know.
I see an additional failure in CastTest that's probably related to the commit
http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=fce70ab5b1cb13a2f70c1b9062312a09892f1a96

Test expectation may need to be updated for  9
Comment 34 Eclipse Genie CLA 2017-08-31 07:06:31 EDT
New Gerrit change created: https://git.eclipse.org/r/104068
Comment 35 Stephan Herrmann CLA 2017-08-31 07:09:03 EDT
(In reply to Sasikanth Bharadwaj from comment #33)
> I see an additional failure in CastTest that's probably related to the commit
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=fce70ab5b1cb13a2f70c1b9062312a09892f1a96

Sorry, my bad. Must have missed to run the entire test class on 9 (and hudson doesn't catch this either).

(In reply to Eclipse Genie from comment #34)
> New Gerrit change created: https://git.eclipse.org/r/104068

> Test expectation may need to be updated for  9

The gerrit does exactly that.
Comment 37 Stephan Herrmann CLA 2017-09-05 15:31:40 EDT
Just to sync, this is the list of failures I get when running all JDT/Core tests on JRE 9 (total 82600):

org.eclipse.jdt.core.tests.compiler.regression.XLargeTest.test0016
org.eclipse.jdt.core.tests.compiler.regression.XLargeTest.test0017
org.eclipse.jdt.core.tests.compiler.regression.XLargeTest.test0018
org.eclipse.jdt.core.tests.compiler.regression.XLargeTest.test0019
org.eclipse.jdt.core.tests.compiler.regression.EnumTest.test434442
org.eclipse.jdt.core.tests.builder.GetResourcesTests.test005

IIRC this is the best result I ever achieved in BETA_JAVA9

(5x IllegalAccessError in enum, 1x binary names for anonymous types).
Comment 38 Sasikanth Bharadwaj CLA 2017-09-06 03:49:21 EDT
I see an additional failure in ModuleAttributeTests.testModuleCompile when running RunAllJava9Tests - appears to be a line separator issue
Comment 39 Sasikanth Bharadwaj CLA 2017-09-06 03:50:04 EDT
(In reply to comment #38)
> I see an additional failure in ModuleAttributeTests.testModuleCompile when
> running RunAllJava9Tests - appears to be a line separator issue
I meant path separator issue
Comment 40 Stephan Herrmann CLA 2017-09-06 08:40:14 EDT
(In reply to Sasikanth Bharadwaj from comment #39)
> (In reply to comment #38)
> > I see an additional failure in ModuleAttributeTests.testModuleCompile when
> > running RunAllJava9Tests - appears to be a line separator issue
> I meant path separator issue

Sorry, my bad.

As I can't test a fix (it's not broken on my machine), could you please try to fix this inside AbstractRegressionTest9.getCompilationUnits() by s.t. like
   fileNameString = fileNameString.replace(File.pathSeparator, '/')
?
Looks easier, than having to concat with File.pathSeparator in each and every test that uses associateToModule(), OK?

thanks
Comment 41 Eclipse Genie CLA 2017-09-07 16:45:15 EDT
New Gerrit change created: https://git.eclipse.org/r/104682
Comment 42 Stephan Herrmann CLA 2017-09-07 17:09:13 EDT
(In reply to Eclipse Genie from comment #41)
> New Gerrit change created: https://git.eclipse.org/r/104682

Fixes the failure in CompletionTests9.test486988_0015()

A call to nameEnvironment.findType() was not module-aware, nor did we have API for clients outside the compiler to actually follow reads edges through the module graph. Added such function to SearchableEnvironment.


Heads-up: CompletionTests9 is not yet listed below RunJDTCoreTests, only in RunAllJava9Tests! I do see Java9ElementsTest and ResolveTests9 in AllJavaModelTests, but this list isn't complete. Also missing: JavaSearchBugs9Tests. Anybody else?

compiler.regression suite looks "more complete".
Comment 44 Jay Arthanareeswaran CLA 2017-09-08 11:28:06 EDT
Is anybody else seeing failure in  ResolveTests.testInterfaceX()? I do, intermittently. Trouble shooting shows that the search gets cancelled since indexer is not ready. The call in question is searchAllTypeNames in SearchableEnvironment line number 572.

I can force the tests to pass if I change the argument from CANCEL_IF_NOT_READY_TO_SEARCH to WAIT_UNTIL_READY_TO_SEARCH.

Manoj, any idea what we can do here?
Comment 45 Manoj N Palat CLA 2017-09-10 22:45:04 EDT
(In reply to Jay Arthanareeswaran from comment #44)
> Is anybody else seeing failure in  ResolveTests.testInterfaceX()? I do,
> intermittently. Trouble shooting shows that the search gets cancelled since
> indexer is not ready. The call in question is searchAllTypeNames in
> SearchableEnvironment line number 572.
> 
> I can force the tests to pass if I change the argument from
> CANCEL_IF_NOT_READY_TO_SEARCH to WAIT_UNTIL_READY_TO_SEARCH.
> 
> Manoj, any idea what we can do here?

@Jay: On backtracking I found that this issue started coming up with commit 44c3522e5259e25f9b607e9ab760cd16382d194e. So you may want to start from these changes.
Comment 46 Sasikanth Bharadwaj CLA 2017-09-11 01:33:44 EDT
(In reply to comment #40)
> (In reply to Sasikanth Bharadwaj from comment #39)
> > (In reply to comment #38)
> > > I see an additional failure in ModuleAttributeTests.testModuleCompile when
> > > running RunAllJava9Tests - appears to be a line separator issue
> > I meant path separator issue
> 
> Sorry, my bad.
> 
> As I can't test a fix (it's not broken on my machine), could you please try to
> fix this inside AbstractRegressionTest9.getCompilationUnits() by s.t. like
> fileNameString = fileNameString.replace(File.pathSeparator, '/')
> ?
> Looks easier, than having to concat with File.pathSeparator in each and every
> test that uses associateToModule(), OK?
> 
The quick fix didn't solve the issue, I will dig further and make the necessary changes when I add more tests to ModuleAttributeTests
> thanks
Comment 47 Sasikanth Bharadwaj CLA 2017-09-25 02:03:47 EDT
(In reply to comment #46)
> (In reply to comment #40)
> > (In reply to Sasikanth Bharadwaj from comment #39)
> > > (In reply to comment #38)
> > > > I see an additional failure in ModuleAttributeTests.testModuleCompile when
> > > > running RunAllJava9Tests - appears to be a line separator issue
> > > I meant path separator issue
> >
> > Sorry, my bad.
> >
> > As I can't test a fix (it's not broken on my machine), could you please try to
> > fix this inside AbstractRegressionTest9.getCompilationUnits() by s.t. like
> > fileNameString = fileNameString.replace(File.pathSeparator, '/')
> > ?
> > Looks easier, than having to concat with File.pathSeparator in each and every
> > test that uses associateToModule(), OK?
> >
> The quick fix didn't solve the issue, I will dig further and make the necessary
> changes when I add more tests to ModuleAttributeTests
> > thanks
done via commit 26bb485688891a8824802195033c124ad5c82755. for bug 495967
Comment 48 Stephan Herrmann CLA 2017-11-04 10:46:32 EDT
Running builder tests with -Dcompliance=9 doesn't work.

I get 21 errors from TestingEnvironment.addProject() like:
  This test requires a 1.5 JRE

Obviously, a branch 
   if ("9".equals(compliance)) 
is missing.
Comment 49 Stephan Herrmann CLA 2017-11-04 10:47:50 EDT
Anybody else seeing the failure in
   org.eclipse.jdt.core.tests.builder.GetResourcesTests.test005
?
Comment 50 Eclipse Genie CLA 2017-11-05 17:50:30 EST
New Gerrit change created: https://git.eclipse.org/r/111029
Comment 51 Stephan Herrmann CLA 2017-11-05 17:55:53 EST
(In reply to Stephan Herrmann from comment #48)
> Running builder tests with -Dcompliance=9 doesn't work.
> 
> I get 21 errors from TestingEnvironment.addProject() like:
>   This test requires a 1.5 JRE
> 
> Obviously, a branch 
>    if ("9".equals(compliance)) 
> is missing.

that branch is missing, but that's not the cause for the exception. 

We seem to have different uses of -Dcompliance here. 
- Normally, in compiler tests this is used to select at which compliance the compiler should be tested.
- In TestingEnvironment.addProject() the same property is, however, used to detect, whether a project can be created at a requested compliance level, and instead of skipping tests with unmet requirements, we throw  RuntimeException.

(In reply to Eclipse Genie from comment #50)
> New Gerrit change created: https://git.eclipse.org/r/111029

this is the best & easiest fix I can think of right now:
- add the missing branch
- use java.specification.version rather than compliance
- check "<" instead of flag inclusion.

(Background: while gerrit jobs still run on Java 1.8 I need a way to locally perform all that is omitted, without waiting for results at all compliance levels).
Comment 53 Stephan Herrmann CLA 2017-11-07 16:01:40 EST
(In reply to Eclipse Genie from comment #52)
> Gerrit change https://git.eclipse.org/r/111029 was merged to [master].
> Commit:
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=922e76f860ca2f7ff1a9623fc6eb87b3b2728f15
 
Just as I was going to write a final comment about this I realize that now I'm comparing apples (like ClassFileConstants.JDK9) with oranges (like AbstractCompilerTest.F_9)

Additional fix coming soon ...
Comment 54 Stephan Herrmann CLA 2017-11-14 07:08:29 EST
Created attachment 271457 [details]
tracing output

Dumping an interesting section from a hudson console.

I particularly like this line:

06:58:16  	!!! ERROR: P/P was never deleted even after having waited 0ms!!!

Despite all that logging, the job seems to proceed alright, all this took roughly 2:30 minutes.

Do we have an open bug that waits for this tracing output?
Comment 55 Stephan Herrmann CLA 2017-11-14 11:28:55 EST
Updated list of problems (master):

ModuleBuilderTests.testAutoModule4()
- intermittent. I'll try to cover that via bug 522078

The 5 well-known IllegalAccessException re enum initialization

ModuleCompilationTests.testBug519922()
- random variation in error position, sometimes "org", sometimes "org.eclipse"

GenericsRegressionTest_9.testBug488663_005()
- random order in error message: Y<I & J> vs. Y<J & I>
Comment 56 Stephan Herrmann CLA 2017-11-14 17:12:05 EST
Locally running RunJDTCoreTests (to workaround problems on hundson), I see these failures:

- BuildPathTests.testIncompatibleJdkLEvelOnProject()
- BuildPathTests.testIncompatibleJdkLEvelOnWksp()
- GetResourcesTests.test005()

plus 221 failures in various ASTConverter tests.

Individually re-running these tests did not exhibit any failures, including RunConverterTests in its entirety.

Anybody else seeing s.t. like this?

All this is about running on 1.8.
Comment 57 Sasikanth Bharadwaj CLA 2017-11-29 12:40:51 EST
(In reply to comment #55)
> Updated list of problems (master):
> 
> ModuleBuilderTests.testAutoModule4()
> - intermittent. I'll try to cover that via bug 522078
> 
> The 5 well-known IllegalAccessException re enum initialization
> 
> ModuleCompilationTests.testBug519922()
> - random variation in error position, sometimes "org", sometimes "org.eclipse"
> 
> GenericsRegressionTest_9.testBug488663_005()
> - random order in error message: Y<I & J> vs. Y<J & I>

I'm seeing a new failure - CompletionTests9.testBug522164_jar, failure trace as follows. Fails consistently whether run via RunAllJava9Tests or individually

junit.framework.ComparisonFailure: 
----------- Expected ------------
test[METHOD_REF]{test(), Lp.a.Ifc;, ()V, test, 60}
------------ but was ------------

--------- Difference is ----------
 expected:<[test[METHOD_REF]{test(), Lp.a.Ifc;, ()V, test, 60}]> but was:<[]>
Comment 58 Manoj N Palat CLA 2017-11-30 04:43:21 EST
(In reply to Sasikanth Bharadwaj from comment #57)

> I'm seeing a new failure - CompletionTests9.testBug522164_jar, failure trace
> as follows. Fails consistently whether run via RunAllJava9Tests or
> individually

Test is failing from commit 2c4e46a161c87faff7d830d04cd9ce262b2a9e54 onwards
Comment 59 Manoj N Palat CLA 2017-12-03 19:56:31 EST
(In reply to Manoj Palat from comment #58)
> (In reply to Sasikanth Bharadwaj from comment #57)
> 
> > I'm seeing a new failure - CompletionTests9.testBug522164_jar, failure trace
> > as follows. Fails consistently whether run via RunAllJava9Tests or
> > individually
> 
> Test is failing from commit 2c4e46a161c87faff7d830d04cd9ce262b2a9e54 onwards

@Till: Would you be able to take a look on this?
Comment 60 Till Brychcy CLA 2017-12-04 02:51:19 EST
(In reply to Manoj Palat from comment #59)
> (In reply to Manoj Palat from comment #58)
> > (In reply to Sasikanth Bharadwaj from comment #57)
> > 
> > > I'm seeing a new failure - CompletionTests9.testBug522164_jar, failure trace
> > > as follows. Fails consistently whether run via RunAllJava9Tests or
> > > individually
> > 
> > Test is failing from commit 2c4e46a161c87faff7d830d04cd9ce262b2a9e54 onwards
> 
> @Till: Would you be able to take a look on this?

Fix is on the way, I'm adding it to bug 527576.
Comment 61 Manoj N Palat CLA 2017-12-04 04:24:34 EST
(In reply to Till Brychcy from comment #60)

> Fix is on the way, I'm adding it to bug 527576.

Thanks Till for the fast response!
Comment 62 Manoj N Palat CLA 2017-12-04 04:27:59 EST
(In reply to Manoj Palat from comment #61)
> (In reply to Till Brychcy from comment #60)
> 
> > Fix is on the way, I'm adding it to bug 527576.
> 
> Thanks Till for the fast response!

Fix by Till: 2d19e28eb87a29fea3b4dab9b0289a073ee6f30c

moving this out of M4 to address rest of test issues
Comment 63 Eclipse Genie CLA 2017-12-10 08:53:18 EST
New Gerrit change created: https://git.eclipse.org/r/113120
Comment 64 Stephan Herrmann CLA 2017-12-10 09:05:24 EST
(In reply to Eclipse Genie from comment #63)
> New Gerrit change created: https://git.eclipse.org/r/113120

Combines two fixes:

(1) Enable running jenkins tests on Java 9 (see also bug 527750) - only applied to o.e.j.c.tests.compiler atm.

(2) Avoid random order of interfaces in an intersection type that would cause random failures in GRT_9.testBug488663_005()
(We were already sorting in order to put classes to front, the fix just piggy-backs on that).
Comment 65 Stephan Herrmann CLA 2017-12-10 17:52:49 EST
(In reply to Stephan Herrmann from comment #64)
> (2) Avoid random order of interfaces in an intersection type that would
> cause random failures in GRT_9.testBug488663_005()
> (We were already sorting in order to put classes to front, the fix just
> piggy-backs on that).

Including interfaces in the existing sort() in LE.createIntersectionType18() unvealed several code locations / tests that depended on existing order, bogusly I think.

Since some of these changes are not obvious, here some notes on patch set #2:

- Adjusted test expectation, LMK if you see problems with changed messages

- Expression.computeConversion() accepted the well-known type Serializable as a legal boxing type leading to an illegal primitive cast instruction.

- ReferenceExpression.generateCode() used the wrong type (not the FI) as the return type in invokedynamic

A quick review would be great (https://git.eclipse.org/r/#/c/113120/)
Comment 67 Jay Arthanareeswaran CLA 2017-12-11 00:18:06 EST
(In reply to Eclipse Genie from comment #66)
> Gerrit change https://git.eclipse.org/r/113120 was merged to [master].
> Commit:
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=97508c79ae02c520c9c61b641f6ebc42e364ced7
> 

Only after merging did I notice that the jenkins still used JRE 8. What am I missing?
Comment 68 Stephan Herrmann CLA 2017-12-11 09:46:01 EST
(In reply to Jay Arthanareeswaran from comment #67)
> (In reply to Eclipse Genie from comment #66)
> > Gerrit change https://git.eclipse.org/r/113120 was merged to [master].
> > Commit:
> > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=97508c79ae02c520c9c61b641f6ebc42e364ced7
> > 
> 
> Only after merging did I notice that the jenkins still used JRE 8. What am I
> missing?

The trick is: the build itself runs on JRE 8 (otherwise we would run into bug 514119. But the test execution uses JDK 9, see these log snippets:

[INFO] ------------------------------------------------------------------------
[INFO] Building org.eclipse.jdt.core.tests.compiler 3.12.400-SNAPSHOT
[INFO] ------------------------------------------------------------------------
...
[INFO] --- maven-toolchains-plugin:1.1:toolchain (default) @ org.eclipse.jdt.core.tests.compiler ---
...
[INFO] Required toolchain: jdk [ id='JavaSE-9' ]
[INFO] Found matching toolchain for type jdk: JDK[/opt/public/common/java/oracle/jdk-9-ga_x64]
...
[INFO] --- tycho-surefire-plugin:1.1.0-SNAPSHOT:test (default-test) @ org.eclipse.jdt.core.tests.compiler ---
[INFO] Toolchain in tycho-surefire-plugin: JDK[/opt/public/common/java/oracle/jdk-9-ga_x64]
[INFO] Expected eclipse log file: /jobs/genie.jdt/eclipse.jdt.core-Gerrit/workspace/org.eclipse.jdt.core.tests.compiler/target/work/data/.metadata/.log
[INFO] Command line:
	[/opt/public/common/java/oracle/jdk-9-ga_x64/bin/java, -Dosgi.noShutdown=false, -Dosgi.os=linux, -Dosgi.ws=gtk, -Dosgi.arch=x86_64, -Dosgi.clean=true, -jar, /jobs/genie.jdt/eclipse.jdt.core-Gerrit/workspace/.repository/p2/osgi/bundle/org.eclipse.equinox.launcher/1.4.0.v20161219-1356/org.eclipse.equinox.launcher-1.4.0.v20161219-1356.jar, -data, /jobs/genie.jdt/eclipse.jdt.core-Gerrit/workspace/org.eclipse.jdt.core.tests.compiler/target/work/data, -install, /jobs/genie.jdt/eclipse.jdt.core-Gerrit/workspace/org.eclipse.jdt.core.tests.compiler/target/work, -configuration, /jobs/genie.jdt/eclipse.jdt.core-Gerrit/workspace/org.eclipse.jdt.core.tests.compiler/target/work/configuration, -application, org.eclipse.tycho.surefire.osgibooter.headlesstest, -testproperties, /jobs/genie.jdt/eclipse.jdt.core-Gerrit/workspace/org.eclipse.jdt.core.tests.compiler/target/surefire.properties]
...

Also see the increase in test numbers.

Cool, huh? :)
Comment 69 Jay Arthanareeswaran CLA 2017-12-11 09:55:55 EST
(In reply to Stephan Herrmann from comment #68)
> Also see the increase in test numbers.
> 
> Cool, huh? :)

Very cool, indeed!
Comment 70 Stephan Herrmann CLA 2017-12-11 19:38:20 EST
Meanwhile the first jenkins run with model tests on JDK 9 [1] shows these failures:

org.eclipse.jdt.core.tests.model.JavaProjectTests.testBug522554_2
org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testConvertToModule
org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug512053
org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282
org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282a

The first was expected, that's what drove me down the alley of testing on JDK 9 in the first place.

Anyone recognize any of the failures in ModuleBuilderTests?


[1] https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit/158/testReport/
Comment 71 Jay Arthanareeswaran CLA 2017-12-11 22:19:29 EST
(In reply to Stephan Herrmann from comment #70)
> org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testConvertToModule
> org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug512053
> org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282
> org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282a

They all seem to be about not being able to find the system classes. But need to dig deeper to find the cause.

Also interesting to note that we are getting the options, but don't seem to be using them anywhere.
Comment 72 Stephan Herrmann CLA 2017-12-12 08:46:41 EST
(In reply to Jay Arthanareeswaran from comment #71)
> (In reply to Stephan Herrmann from comment #70)
> > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testConvertToModule
> > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug512053
> > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282
> > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282a
> 
> They all seem to be about not being able to find the system classes. But
> need to dig deeper to find the cause.
> 
> Also interesting to note that we are getting the options, but don't seem to
> be using them anywhere.

I could not reproduce by invoking tests from Eclipse. So for good measure I retriggered the jenkins job (to check for cosmic interference) but the result was the same.
Comment 73 Stephan Herrmann CLA 2017-12-12 09:59:00 EST
(In reply to Stephan Herrmann from comment #72)
> (In reply to Jay Arthanareeswaran from comment #71)
> > (In reply to Stephan Herrmann from comment #70)
> > > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testConvertToModule
> > > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug512053
> > > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282
> > > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testBug518282a
> > 
> > They all seem to be about not being able to find the system classes. But
> > need to dig deeper to find the cause.
> > 
> > Also interesting to note that we are getting the options, but don't seem to
> > be using them anywhere.
> 
> I could not reproduce by invoking tests from Eclipse. So for good measure I
> retriggered the jenkins job (to check for cosmic interference) but the
> result was the same.

Phew, this one is tricky to debug, here's an approach that works:

We need to debug the test execution as it is invoked by maven/tycho, but due to the toolchains magic, tycho is forced to fork a new jvm for tests, hence running mvnDebug doesn't help. Instead we need to debug the target jvm launched by tycho:

In tests.model/pom.xml extend the command line like so:
      	  <argLine>-Djdt.default.test.compliance=1.8 -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=y</argLine>

Next, uncomment any test suites other than the one you want to debug.

Run maven inside git/eclipse.jdt.core:
$ mvn -o -P build-individual-bundles,bree-libs,test-on-javase-9 -pl org.eclipse.jdt.core.tests.model verify 

(-o to avoid waiting for "downloading the internet" ;p - requires a previous non-offline run)

At some point the build will pause, saying 
  Listening for transport dt_socket at address: 8000

Launch a remote debugging session in Eclipse (after having set relevant breakpoints of course).

=> Happy debugging.
Comment 74 Stephan Herrmann CLA 2017-12-12 10:14:56 EST
(In reply to Stephan Herrmann from comment #73)
> (In reply to Stephan Herrmann from comment #72)
> > (In reply to Jay Arthanareeswaran from comment #71)
> > > (In reply to Stephan Herrmann from comment #70)
> > > > org.eclipse.jdt.core.tests.model.ModuleBuilderTests.testConvertToModule

When debugging this one under maven etc. we are failing because in JavaModelManager.initializeContainer(IJavaProject, IPath) we don't find an initializer in path "org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-9"

Apparently, o.e.j.debug.lauching is not part of the SUT?

Right, to reproduce in Eclipse do:
- create launch config for ModuleBuilderTests
- under Plug-ins:
  - deselect all
  - select o.e.j.core.tests.model
- click "Add Required Plug-ins"
  - see that o.e.j.launching is not selected
- run / debug

Watch the same failures as reported above.

=> All those tests are bogus (when placed in o.e.j.core.tests.model), or we need to add a dependency on o.e.j.launching. Would the latter be legal?
Comment 75 Stephan Herrmann CLA 2017-12-12 10:58:05 EST
(In reply to Stephan Herrmann from comment #74)
> => All those tests are bogus (when placed in o.e.j.core.tests.model), or we
> need to add a dependency on o.e.j.launching. Would the latter be legal?

https://git.eclipse.org/r/#/c/113116/4 attempts to fix that problem by indeed just adding this one dependency to o.e.j.core.tests.model.

(That change additionally contains improvements re bug 522554, so it's booked on that bug again).
Comment 76 Jay Arthanareeswaran CLA 2017-12-12 11:03:40 EST
(In reply to Stephan Herrmann from comment #74)
> => All those tests are bogus (when placed in o.e.j.core.tests.model), or we
> need to add a dependency on o.e.j.launching. Would the latter be legal?

I was going to type out "No" then saw your next comment. So, will just wait for the tests. On the other hand, it would be okay since it is only a test project, right?
Comment 77 Stephan Herrmann CLA 2017-12-12 11:09:43 EST
(In reply to Jay Arthanareeswaran from comment #76)
> On the other hand, it would be okay since it is only a test
> project, right?

That's my hope.

Otherwise we'd need some clever new design for our tests, or move some tests over to JDT/Debug.

Let's see, but local maven build was happy with this change :)


BTW: what's the other bug where we would have needed the JRE container, and therefor couldn't create the test in core-land? Or was that one not about compilation but about execution?
Comment 78 Jay Arthanareeswaran CLA 2017-12-12 11:13:53 EST
(In reply to Stephan Herrmann from comment #77)
> BTW: what's the other bug where we would have needed the JRE container, and
> therefor couldn't create the test in core-land? Or was that one not about
> compilation but about execution?

Bug 521995, comment #11 and #12.
Comment 79 Stephan Herrmann CLA 2017-12-12 11:23:51 EST
(In reply to Jay Arthanareeswaran from comment #78)
> (In reply to Stephan Herrmann from comment #77)
> > BTW: what's the other bug where we would have needed the JRE container, and
> > therefor couldn't create the test in core-land? Or was that one not about
> > compilation but about execution?
> 
> Bug 521995, comment #11 and #12.

Thanks, your memory seems to work better than mine :)

So, would the pending addition of a new dependency also unblock you in that bug?
Comment 80 Jay Arthanareeswaran CLA 2017-12-12 11:30:38 EST
(In reply to Stephan Herrmann from comment #79)
> So, would the pending addition of a new dependency also unblock you in that
> bug?

I think so. I had earlier requested Sarika to help me in writing a test for that in debug, but if we can write such tests in our own test projects, then nothing like it.
Comment 81 Jay Arthanareeswaran CLA 2017-12-20 14:24:58 EST
(In reply to Stephan Herrmann from comment #68)
> The trick is: the build itself runs on JRE 8 (otherwise we would run into
> bug 514119. But the test execution uses JDK 9, see these log snippets:

Stephan, one of the tests is failing on the patch for bug 490103. This isn't failing locally. I put some logs and this is what I found:

Found matching toolchain for type jdk: JDK[/opt/public/common/java/oracle/jdk-9_x64-latest]
Getting JRT from : /opt/public/common/java/oracle/jdk-9-ga_x64

As you can see, there are two JDKs, the second one retrieved from System.getProperty("java.home")

I can't tell if the second is a proper JDK installation or not. But is there a way to get hold of the first JDK?
Comment 82 Stephan Herrmann CLA 2017-12-20 16:17:38 EST
(In reply to Jay Arthanareeswaran from comment #81)
> (In reply to Stephan Herrmann from comment #68)
> > The trick is: the build itself runs on JRE 8 (otherwise we would run into
> > bug 514119. But the test execution uses JDK 9, see these log snippets:
> 
> Stephan, one of the tests is failing on the patch for bug 490103. This isn't
> failing locally. I put some logs and this is what I found:
> 
> Found matching toolchain for type jdk:
> JDK[/opt/public/common/java/oracle/jdk-9_x64-latest]
> Getting JRT from : /opt/public/common/java/oracle/jdk-9-ga_x64
> 
> As you can see, there are two JDKs, the second one retrieved from
> System.getProperty("java.home")
> 
> I can't tell if the second is a proper JDK installation or not. But is there
> a way to get hold of the first JDK?

As a first quick guess I would say that one path is a symlink to the other ("-latest" being the constant name the will get redirected when new versions are installed).

FYI: if you need to inspect how things look in the filesystem on jenkins, adding a shell script as an additional build step on jenkins may come in handy :)
Comment 83 Stephan Herrmann CLA 2018-01-23 18:42:53 EST
*** Bug 530212 has been marked as a duplicate of this bug. ***
Comment 84 Ben Ferguson CLA 2018-01-24 10:18:21 EST
(In reply to Stephan Herrmann from comment #83)
> *** Bug 530212 has been marked as a duplicate of this bug. ***

Hello! Not trying to be a pain, but could you please explain how this bug is a duplicate? I don't see the error I was getting anywhere in the comments. Is my scenario covered in one of the unit tests that have failed? Thanks!
Comment 85 Stephan Herrmann CLA 2018-01-24 10:32:16 EST
(In reply to Ben Ferguson from comment #84)
> (In reply to Stephan Herrmann from comment #83)
> > *** Bug 530212 has been marked as a duplicate of this bug. ***
> 
> Hello! Not trying to be a pain, but could you please explain how this bug is
> a duplicate? I don't see the error I was getting anywhere in the comments.
> Is my scenario covered in one of the unit tests that have failed? Thanks!

I was surprised myself :)
Did you see bug 530212 comment 1 and bug 530212 comment 2?
Comment 86 Ben Ferguson CLA 2018-01-24 12:07:49 EST
(In reply to Stephan Herrmann from comment #85)
> (In reply to Ben Ferguson from comment #84)
> > (In reply to Stephan Herrmann from comment #83)
> > > *** Bug 530212 has been marked as a duplicate of this bug. ***
> > 
> > Hello! Not trying to be a pain, but could you please explain how this bug is
> > a duplicate? I don't see the error I was getting anywhere in the comments.
> > Is my scenario covered in one of the unit tests that have failed? Thanks!
> 
> I was surprised myself :)
> Did you see bug 530212 comment 1 and bug 530212 comment 2?

Ah, interesting. Gotta love it when issues accidentally fix themselves :). If I understand this correctly, this issue will be fixed in Oxygen v4.8?
Comment 87 Stephan Herrmann CLA 2018-01-24 12:23:52 EST
(In reply to Ben Ferguson from comment #86)
> (In reply to Stephan Herrmann from comment #85)
> > (In reply to Ben Ferguson from comment #84)
> > > (In reply to Stephan Herrmann from comment #83)
> > > > *** Bug 530212 has been marked as a duplicate of this bug. ***
> > > 
> > > Hello! Not trying to be a pain, but could you please explain how this bug is
> > > a duplicate? I don't see the error I was getting anywhere in the comments.
> > > Is my scenario covered in one of the unit tests that have failed? Thanks!
> > 
> > I was surprised myself :)
> > Did you see bug 530212 comment 1 and bug 530212 comment 2?
> 
> Ah, interesting. Gotta love it when issues accidentally fix themselves :).
> If I understand this correctly, this issue will be fixed in Oxygen v4.8?

Yes, "your" bug is closed with target 4.8 M5.
Comment 88 Eclipse Genie CLA 2018-02-04 14:12:35 EST
New Gerrit change created: https://git.eclipse.org/r/116680
Comment 90 Stephan Herrmann CLA 2018-02-04 16:50:31 EST
(In reply to Eclipse Genie from comment #89)
> Gerrit change https://git.eclipse.org/r/116680 was merged to [master].
> Commit:
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/
> ?id=dc9331a753b84c4de709d71cc785ab1fe2d10548

This should fix the issue of JavaProjectTests.testBug522554_2 and JavaProjectTests.testBug522554_2_unlimited where we had trouble with knowing the set of default modules. Previously I thought this to be a matter of JDK version, but the more likely explanation is that Oracle JDK and OpenJDK simply have different sets.
Comment 91 Stephan Herrmann CLA 2018-02-04 17:00:33 EST
(In reply to Stephan Herrmann from comment #90)
> (In reply to Eclipse Genie from comment #89)
> > Gerrit change https://git.eclipse.org/r/116680 was merged to [master].
> > Commit:
> > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/
> > ?id=dc9331a753b84c4de709d71cc785ab1fe2d10548
> 
> This should fix the issue of JavaProjectTests.testBug522554_2 and
> JavaProjectTests.testBug522554_2_unlimited where we had trouble with knowing
> the set of default modules. Previously I thought this to be a matter of JDK
> version, but the more likely explanation is that Oracle JDK and OpenJDK
> simply have different sets.

Only now I found bug 529981, which I had raised specifically for this issue. Let's wait for the next I-build and if successful, close the other bug as a duplicate of this one.
Comment 92 Stephan Herrmann CLA 2018-02-05 06:42:22 EST
*** Bug 529981 has been marked as a duplicate of this bug. ***
Comment 93 Jay Arthanareeswaran CLA 2018-02-08 03:28:04 EST
Finally, we have the missing test suites running with JRE 9:

http://download.eclipse.org/eclipse/downloads/drops4/I20180207-2000/testresults/html/org.eclipse.jdt.compiler.apt.tests_ep48I-unit-cen64-gtk3-java9_linux.gtk.x86_64_9.html

I see five failures in APT, which I can reproduce locally as well:

Java9ElementsTests#testRootElements1
Java9ElementsTests#testRootElements2
Java9ElementsTests#testRootElements1Javac
Java9ElementsTests#testRootElements2Javac
Java9ElementsTests#testAnnotations1

The fact that the first four fail with Javac as well might indicate that the tests are wrong. But I need to investigate more to find out how/why.
Comment 94 Stephan Herrmann CLA 2018-03-11 15:01:58 EDT
PackageInfoTest.test002 fails on 9 because:
  The method getPackage(String) from the type Package is deprecated
Comment 95 Stephan Herrmann CLA 2018-03-21 18:28:01 EDT
(In reply to Stephan Herrmann from comment #94)
> PackageInfoTest.test002 fails on 9 because:
>   The method getPackage(String) from the type Package is deprecated

Oops, there had been a version of the test that expected this warning at 9.

Then via bug 527569 this expectation (which was correct IMHO) was replaced by unconditional expectingNoProblems().

Next I found the check for "9" to be bogus, see  bug 531362 comment 4, fixed in BETA_JAVA_18_3.

With the merge back into master, the two above changes conflicted, and my manual resolution (?) the conditional has been restored - let's await tonights I-build for final result.
Comment 96 Eclipse Genie CLA 2018-05-08 01:46:47 EDT
New Gerrit change created: https://git.eclipse.org/r/122293
Comment 98 Jay Arthanareeswaran CLA 2018-05-10 12:11:03 EDT
We still have test issues to address. Will continue through RC1.
Comment 99 Jay Arthanareeswaran CLA 2018-05-17 10:22:56 EDT
We will continue the work through 4.9.
Comment 100 Stephan Herrmann CLA 2018-07-28 09:56:39 EDT
Failure in org.eclipse.jdt.apt.pluggable.tests.FilerTests.testCreateClass1

- Seen in gerrit-jenkins (#1078, #1080 and #1081) thus blocking contributions via gerrit.

- Some production builds:
  http://download.eclipse.org/eclipse/downloads/drops4/I20180727-2000/testresults/html/org.eclipse.jdt.apt.pluggable.tests_ep49I-unit-mac64_macosx.cocoa.x86_64_8.0.html
  http://download.eclipse.org/eclipse/downloads/drops4/I20180727-2000/testresults/html/org.eclipse.jdt.apt.pluggable.tests_ep49I-unit-cen64-gtk3_linux.gtk.x86_64_8.0.html

- Reproduced also locally (on linux).

Error is:

junit.framework.AssertionFailedError: file should have been overwritten
	at junit.framework.Assert.fail(Assert.java:57)
	at junit.framework.Assert.assertTrue(Assert.java:22)
	at junit.framework.TestCase.assertTrue(TestCase.java:192)
	at org.eclipse.jdt.apt.pluggable.tests.FilerTests.testCreateClass1(FilerTests.java:399)
Comment 101 Stephan Herrmann CLA 2018-07-28 09:59:20 EDT
(In reply to Stephan Herrmann from comment #100)
> Failure in org.eclipse.jdt.apt.pluggable.tests.FilerTests.testCreateClass1
> 
> - Seen in gerrit-jenkins (#1078, #1080 and #1081) thus blocking
> contributions via gerrit.
> 
> - Some production builds:
>  
> http://download.eclipse.org/eclipse/downloads/drops4/I20180727-2000/
> testresults/html/org.eclipse.jdt.apt.pluggable.tests_ep49I-unit-mac64_macosx.
> cocoa.x86_64_8.0.html
>  
> http://download.eclipse.org/eclipse/downloads/drops4/I20180727-2000/
> testresults/html/org.eclipse.jdt.apt.pluggable.tests_ep49I-unit-cen64-
> gtk3_linux.gtk.x86_64_8.0.html
> 
> - Reproduced also locally (on linux).
> 
> Error is:
> 
> junit.framework.AssertionFailedError: file should have been overwritten
> 	at junit.framework.Assert.fail(Assert.java:57)
> 	at junit.framework.Assert.assertTrue(Assert.java:22)
> 	at junit.framework.TestCase.assertTrue(TestCase.java:192)
> 	at
> org.eclipse.jdt.apt.pluggable.tests.FilerTests.testCreateClass1(FilerTests.
> java:399)

At a second look this has nothing to do with Java 9, so I filed bug 537459.
Comment 102 Eclipse Genie CLA 2022-05-26 20:18:45 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.