Bug 490103 - [1.9] JREContainer should work without any physical libraries
Summary: [1.9] JREContainer should work without any physical libraries
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.5.1   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Jay Arthanareeswaran CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks: 526331
  Show dependency tree
 
Reported: 2016-03-21 11:49 EDT by Jay Arthanareeswaran CLA
Modified: 2023-01-08 13:01 EST (History)
8 users (show)

See Also:


Attachments
JDT Launch changes (4.21 KB, patch)
2017-04-28 05:42 EDT, Jay Arthanareeswaran CLA
no flags Details | Diff
JDT UI Patch (3.41 KB, patch)
2017-04-28 05:44 EDT, Jay Arthanareeswaran CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jay Arthanareeswaran CLA 2016-03-21 11:49:24 EDT
The bug is same as reported in bug 489207. That bug provided a short-term fix by adding the jrt-fs.jar as the single library to the container. But ideally we should try to make the JREContainer work without referring to any physical library because there isn't any in case of JRE 9.
Comment 1 Jay Arthanareeswaran CLA 2016-03-22 05:10:43 EDT
I don't think we should fiddle with the IClasspathContainer#getClasspathEntries() and affect existing clients. Here's one idea that could work with minimal code changes:

I propose a new IClasspathEntry kind namely:

  IClasspathEntry#CPE_SYSTEM

The path could point to the root of the JRE, i.e. what is represented by the JREContainer. The JREContainer would be responsible for creating this (only in case of JRE 9) and making it available via getClasspathEntries().

And all we need to do in JDT Core is to look for this new kind and do what is required (i.e. create a ClasspathJimage, a kind of ClasspathLocation that specializes in reading from JRE 9 images).

Any suggestions?
Comment 2 Markus Keller CLA 2016-03-22 15:17:54 EDT
The problem with a new IClasspathEntry kind is that this will break every client that assumes the set given in IClasspathEntry#getEntryKind() is closed. The Javadoc of IClasspathEntry supports that viewpoint. I know that jdt.ui will throw exceptions if you do that, and I assume others won't like it either.

Can't we keep using CPE_LIBRARY and just use the root of the JRE as path? That probably needs some magic to distinguish the root of a JRE from a class folder, but I think that would cause less breakage.
Comment 3 Jay Arthanareeswaran CLA 2016-03-29 05:28:46 EDT
(In reply to Markus Keller from comment #2)
> Can't we keep using CPE_LIBRARY and just use the root of the JRE as path?
> That probably needs some magic to distinguish the root of a JRE from a class
> folder, but I think that would cause less breakage.

That magic could well be the presence of the jrt-fs.jar directly under the path. But we should think about the clients that are probably expecting this CPE_LIBRARY (a folder) to contain binary classes. They would be broken too.
Comment 4 Jay Arthanareeswaran CLA 2016-09-14 05:47:55 EDT
The suggestion from comment #1 was based on the assumption that there would be at least one entry from #getClasspathEntries(). But that's wrong. The JREContainer should clearly tell its clients that they are dealing with a JRT file system.

What we currently have is a crude way of such communication - i.e. the JREContainer#getClasspathEntries() returns exactly one entry pointing to jrt-fs.jar. This was clearly a stop-gap measure. So, I propose the new API goes to IClasspathContainer.

Once again IClasspathContainer#getKind() is closed. So, I propose this:

    /**
     * Answers whether or not this is a Java 9 runtime or later. The Java Run Time (shortly JRT)
     * is not represented by list of libraries but a format internal to the runtime. 
     * Therefore, for containers representing such a runtime, {@link #getClasspathEntries()}
     * returns an empty array. The kind of such a container, however, would remain one of
     * {@link #K_SYSTEM} or {@link #K_DEFAULT_SYSTEM} as applicable.
     *
     * @since 3.13 BETA_JAVA9
     * @see #getClasspathEntries()
     * @see #getKind() 
     */
    default boolean isJava9Runtime() {
    	return false;
    }
Comment 5 Jay Arthanareeswaran CLA 2016-09-14 06:49:30 EDT
(In reply to Jay Arthanareeswaran from comment #4)
> Once again IClasspathContainer#getKind() is closed. So, I propose this:

Note that this alone is not enough to make things work again. We would still require to tag the resolved IClasspathEntry with something to represent the JRT. It remains to be seen, though, how many clients are broken by using the incorrect CPE_LIBRARY as the entry kind that points to the root of the JDK/JRE.
Comment 6 Jay Arthanareeswaran CLA 2016-09-14 06:56:16 EDT
(In reply to Jay Arthanareeswaran from comment #4)
> Once again IClasspathContainer#getKind() is closed. So, I propose this:

Duh, the IClasspathContainer#getPath() returns the container path and not the VM install location.

Sarika, what are your thoughts?
Comment 7 Jay Arthanareeswaran CLA 2016-09-14 07:06:31 EDT
(In reply to Jay Arthanareeswaran from comment #6)
> Sarika, what are your thoughts?

If instead of providing a new API as I suggested, do you see problems in the JREContainer directly creating the special entry and supply via getClasspathEntries()? Perhaps, JavaCore can provide some methods for you to create such a special entry?
Comment 8 Sasikanth Bharadwaj CLA 2016-09-14 07:18:33 EDT
I feel the JRE container is redundant in a module setting. With java9, it is merely used to point to the JRE that needs to be used and for that, it does not need to be a container. With the target platform including system libraries as well and Module path container handling all module dependencies, we can probably do away with the JREContainer? Define an IModulePathEntry that can represent a module path entry and let it produce the necessary IClasspathEntries. With that, just an isModule() API in IClasspathEntry would be enough?
Comment 9 Jay Arthanareeswaran CLA 2016-09-14 07:39:30 EDT
(In reply to Sasikanth Bharadwaj from comment #8)
> Define an
> IModulePathEntry that can represent a module path entry and let it produce
> the necessary IClasspathEntries. With that, just an isModule() API in
> IClasspathEntry would be enough?

That will probably help us leave some of the baggages behind.

BTW, I also found some API in IJavaProject that might make our life bit difficult. For e.g:

IJavaProject#getPackageFragmentRoot(String externalLibraryPath)

Where the implementation is expected to come up with a Package fragment root by just looking at the path.
Comment 10 Jay Arthanareeswaran CLA 2016-09-15 00:25:27 EDT
I think I am hitting too many roadblocks to park this work. I guess I will wait to see how the target platform shapes and decide the course for this.

The existing solution works, except that ...

... if one adds the jrt-fs.jar to the build path as an individual JAR, that adds the whole system libraries to the build path :(
Comment 11 Eclipse Genie CLA 2017-04-21 04:59:26 EDT
New Gerrit change created: https://git.eclipse.org/r/95450
Comment 12 Jay Arthanareeswaran CLA 2017-04-21 05:06:01 EDT
(In reply to Eclipse Genie from comment #11)
> New Gerrit change created: https://git.eclipse.org/r/95450

This patch contains two sets of API changes:

1. Introduces a new IClasspathEntry kind and related JavaCore#newJrtEntry().
2. Removes all but the top level Java element for a module declaration.

The patch also contains all the implementation changes in jdt core to back up the API changes.

The first API change is essentially what I mentioned in comment #1. Had a discussion with Markus and this appears to be the simplest/cleanest solution we can think of yet. Yes, this will break some clients but those still using older Java runtimes should not be affected. The documentation needs improvement in certain areas and certain components in jdt launching should be updated to support this too.

The second set is a straight forward change that could probably be moved to a separate commit.
Comment 13 Jay Arthanareeswaran CLA 2017-04-24 13:16:48 EDT
As expected, this is turning out to be a highly disruptive change. I have addressed the issues on the launching side. But some things are broken in JDT Core and UI too. I will see if I can bring it to a decent shape within a reasonable amount of time (let's say this week).
Comment 14 Dani Megert CLA 2017-04-25 03:54:30 EDT
(In reply to Jay Arthanareeswaran from comment #13)
> As expected, this is turning out to be a highly disruptive change. I have
> addressed the issues on the launching side. But some things are broken in
> JDT Core and UI too. I will see if I can bring it to a decent shape within a
> reasonable amount of time (let's say this week).

What about other clients?
Comment 15 Jay Arthanareeswaran CLA 2017-04-25 05:08:55 EDT
(In reply to Dani Megert from comment #14)
> What about other clients?

They would be broken too, esp. those that assume that JREContainer is always resolved to a bunch of CPE_LIBRARY. I and Markus had a discussion and we agreed that this is by far the cleanest solution. And this will only affect when clients upgrade to a JRT based file system. What we are using (resolving the JREContainer to the bogus jrt-fs.jar) is a hack and don't look nice in the final product.

Couple of other options I suggested during the discussion:

1. Just as we do now, resolve the JREContainer to the list of JMOD that JRE provides. But Markus pointed out that that would defeat the purpose of not worrying about the JRE's implementation details.

2. Upgrade JDT's major version. Markus is of the opinion that we should rather break only the clients that depend on this API (and upgraded to JRT based JRE) than everybody.
Comment 16 Dani Megert CLA 2017-04-25 05:18:21 EDT
(In reply to Jay Arthanareeswaran from comment #15)
> (In reply to Dani Megert from comment #14)
> > What about other clients?
> 
> They would be broken too, esp. those that assume that JREContainer is always
> resolved to a bunch of CPE_LIBRARY. I and Markus had a discussion and we
> agreed that this is by far the cleanest solution. And this will only affect
> when clients upgrade to a JRT based file system. What we are using
> (resolving the JREContainer to the bogus jrt-fs.jar) is a hack and don't
> look nice in the final product.

OK. 

 
> 2. Upgrade JDT's major version. Markus is of the opinion that we should
> rather break only the clients that depend on this API (and upgraded to JRT
> based JRE) than everybody.

I agree with Markus.
Comment 17 Jay Arthanareeswaran CLA 2017-04-28 05:38:14 EDT
The latest patch fixes all but the following failures:

org.eclipse.jdt.core.tests.model.ExternalAnnotations18Test#test2
org.eclipse.jdt.core.tests.model.ExternalAnnotations18Test#test3
org.eclipse.jdt.core.tests.model.ExternalAnnotations18Test#testProjectDependencyReconcile2
org.eclipse.jdt.core.tests.model.ExternalAnnotations18Test#testProjectDependencyReconcile3
org.eclipse.jdt.core.tests.model.ExternalAnnotations18Test#testBug465296

This is because we don't yet know what kind of support we will need/provide for external annotations with a JRT. If these failures are not fixed with this, these should be added to bug 486013.
Comment 18 Jay Arthanareeswaran CLA 2017-04-28 05:42:12 EDT
Created attachment 268054 [details]
JDT Launch changes

This is the patch I used for my testing and nowhere close to completion. There are failures in JREContainer initialization tests. On a quick glance, doesn't look complicated.
Comment 19 Jay Arthanareeswaran CLA 2017-04-28 05:44:25 EDT
Created attachment 268056 [details]
JDT UI Patch

The patch I used in JDT UI to test classpath resolution and source attachment to system libraries. Again, nowhere close to completion. We should have a close look at all references to IClasspathEntry#CPE_LIBRARY.
Comment 20 Jay Arthanareeswaran CLA 2017-10-09 00:56:23 EDT
This fix should also include the following in the context of IClasspathEntry#CPE_JRT_SYSTEM:

adjust Javadoc of IClasspathEntry#getSourceAttachmentRootPath()

Brought from bug 509366.
Comment 21 Jay Arthanareeswaran CLA 2017-10-16 04:25:00 EDT
This would have been a good addition in 4.7.1a, but considering the seriousness of the API change, better not rush. I will re-test this and release early m4.
Comment 22 Eclipse Genie CLA 2017-12-20 03:51:30 EST
New Gerrit change created: https://git.eclipse.org/r/114457
Comment 23 Eclipse Genie CLA 2017-12-20 03:51:33 EST
New Gerrit change created: https://git.eclipse.org/r/114458
Comment 24 Eclipse Genie CLA 2017-12-20 03:51:46 EST
New Gerrit change created: https://git.eclipse.org/r/114459
Comment 25 Jay Arthanareeswaran CLA 2017-12-20 04:08:35 EST
Updated all three patches - but the debug and ui ones are bound to fail since they depend on the core's changes.
Comment 26 Jay Arthanareeswaran CLA 2017-12-20 08:38:01 EST
(In reply to Jay Arthanareeswaran from comment #25)
> Updated all three patches - but the debug and ui ones are bound to fail
> since they depend on the core's changes.

The following tests in core fail since they depend on the jdt.launching resolving the JRE container to the new CPE_JRT_SYSTEM library entry. With those projects checked out in my local environment, they pass.
Comment 27 Jay Arthanareeswaran CLA 2018-01-02 03:49:28 EST
Stephan, is it too much to ask for a review of this patch? :) Would be a nice addition in M5 well ahead of the API freeze.
Comment 28 Stephan Herrmann CLA 2018-01-05 08:21:03 EST
(In reply to Jay Arthanareeswaran from comment #27)
> Stephan, is it too much to ask for a review of this patch? :) Would be a
> nice addition in M5 well ahead of the API freeze.

Looking at the JDT/Core change now.
Comment 29 Stephan Herrmann CLA 2018-01-05 09:09:11 EST
The JDT/Core change looks quite solid to me, details commented in gerrit.

I haven't tried running this, though, so I'm curious what would be the user visible changes?
Comment 30 Stephan Herrmann CLA 2018-01-05 09:23:25 EST
From an adopter's perspective: I believe the javadoc of IClasspathContainer still needs updating, see e.g. slide #13 in https://www.eclipsecon.org/europe2017/sites/default/files/slides/JDT_Embraces_Java_9_ECE_2017.pdf

Do you have a suggested new wording? Just add CPE_JRT_SYSTEM to the list?
Comment 31 Jay Arthanareeswaran CLA 2018-01-10 01:17:26 EST
(In reply to Stephan Herrmann from comment #30)
> From an adopter's perspective: I believe the javadoc of IClasspathContainer
> still needs updating, see e.g. slide #13 in
> https://www.eclipsecon.org/europe2017/sites/default/files/slides/JDT_Embraces_Java_9_ECE_2017.pdf
> 
> 
> Do you have a suggested new wording? Just add CPE_JRT_SYSTEM to the list?

Sorry, missed this in the latest patch. Will consider this.

(In reply to Stephan Herrmann from comment #29)
> The JDT/Core change looks quite solid to me, details commented in gerrit.
> 
> I haven't tried running this, though, so I'm curious what would be the user
> visible changes?

In the package explorer and other places like the JRE definition dialog, the path will point to the java home instead of the jrt-fs.jar.
Comment 32 Jay Arthanareeswaran CLA 2018-01-10 10:41:36 EST
There was a new  test failure after I removed the jrt-fs.jar from AbstractJavaModelTests. Some tests still collect the system libraries from Util.getJavaClassLibs(), which still relies on "jrt-fs.jar" to identify a system library. The batch compiler and other compiler interfaces still pass the libraries as String.
Comment 33 Jay Arthanareeswaran CLA 2018-01-10 10:57:58 EST
Noopur/Sarika, can you please look at the respective patches and see if it's good enough for consumption. Because of some dependency between jdt core tests and launch, we should push all these changes together.
Comment 34 Jay Arthanareeswaran CLA 2018-01-10 23:41:35 EST
(In reply to Jay Arthanareeswaran from comment #33)
> Noopur/Sarika, can you please look at the respective patches and see if it's
> good enough for consumption. Because of some dependency between jdt core
> tests and launch, we should push all these changes together.

Can we do this today or latest this week please?
Comment 35 Sarika Sinha CLA 2018-01-11 01:16:07 EST
For JRE definition in Installed JRE, it will still continue to to show jrt-fs.jar?
Comment 36 Jay Arthanareeswaran CLA 2018-01-11 01:19:07 EST
(In reply to Sarika Sinha from comment #35)
> For JRE definition in Installed JRE, it will still continue to to show
> jrt-fs.jar?

I don't know how launch stores and retrieves the configurations. If the path has jrt-fs.jar, then yes. May be we can (for the time being), look for such containers and convert them to the java home path based entries?
Comment 37 Noopur Gupta CLA 2018-01-11 04:43:51 EST
I just tried the patches and I am seeing the following exception while hovering over any required module in a module-info.java/.class file:

java.lang.IllegalArgumentException: Entry must be of kind CPE_LIBRARY or CPE_VARIABLE
	at org.eclipse.jdt.internal.corext.javadoc.JavaDocLocations.getLibraryJavadocLocation(JavaDocLocations.java:176)
	at 
...

This should be checked and updated at all the locations with such checks.
Comment 38 Noopur Gupta CLA 2018-01-11 04:53:48 EST
Another issue:

Go to the Libraries tab in the Java Build Path dialog of a modular Java 9 project. Expand JRE System Library present under the Modulepath node. 

=> Modules are not shown. Instead, it shows an entry "unknown element". And we get the following exception in Error Log:

java.lang.NullPointerException
	at org.eclipse.jdt.internal.ui.wizards.buildpaths.CPListElement.equals(CPListElement.java:582)
...
Comment 39 Noopur Gupta CLA 2018-01-11 05:08:35 EST
Looks like it will break many locations where CPE_LIBRARY was expected and it can now receive CPE_JRT_SYSTEM. All such locations should be updated in the clients. 

Not sure but PDE could be another client in Eclipse SDK.
Comment 40 Jay Arthanareeswaran CLA 2018-01-16 00:12:57 EST
(In reply to Noopur Gupta from comment #39)
> Looks like it will break many locations where CPE_LIBRARY was expected and
> it can now receive CPE_JRT_SYSTEM. All such locations should be updated in
> the clients. 
> 
> Not sure but PDE could be another client in Eclipse SDK.

That's why I have been calling for some attention for a long time. And that's also why we should push this change in sooner than M6.
Comment 41 Noopur Gupta CLA 2018-01-16 00:37:22 EST
(In reply to Jay Arthanareeswaran from comment #40)
> (In reply to Noopur Gupta from comment #39)
> > Looks like it will break many locations where CPE_LIBRARY was expected and
> > it can now receive CPE_JRT_SYSTEM. All such locations should be updated in
> > the clients. 
> > 
> > Not sure but PDE could be another client in Eclipse SDK.
> 
> That's why I have been calling for some attention for a long time. And
> that's also why we should push this change in sooner than M6.
I think the same was discussed in comment #2. The non-SDK clients of JDT Core should also be considered. 

(In reply to Jay Arthanareeswaran from comment #33)
> Noopur/Sarika, can you please look at the respective patches and see if it's
> good enough for consumption. 
For JDT UI, a pass will be required over other usage locations as well to fix them if needed.
Comment 42 Jay Arthanareeswaran CLA 2018-01-16 00:57:53 EST
(In reply to Noopur Gupta from comment #41)
> I think the same was discussed in comment #2. The non-SDK clients of JDT
> Core should also be considered. 

If there was ever a need for a breaking change, this is it. None of our existing classpath entries satisfactorily cover the new kind introduced by the JRT. The alternatives discussed are not convincing to me:

1. JREContainer resolves to itself or '0" classpath entries?
2. The system libraries are represented by the jrt-fs.jar, like it is now? What if the clients want to have the jrt-fs.jar itself in the classpath?
3. CPE_LIBRARY pointing to the java.home? This will also break many clients who will simply try to look for classes directly under java.home.
Comment 43 Stephan Herrmann CLA 2018-01-16 09:24:33 EST
(In reply to Noopur Gupta from comment #41)
> (In reply to Jay Arthanareeswaran from comment #40)
> > (In reply to Noopur Gupta from comment #39)
> > > Looks like it will break many locations where CPE_LIBRARY was expected and
> > > it can now receive CPE_JRT_SYSTEM. All such locations should be updated in
> > > the clients. 
> > > 
> > > Not sure but PDE could be another client in Eclipse SDK.
> > 
> > That's why I have been calling for some attention for a long time. And
> > that's also why we should push this change in sooner than M6.
> I think the same was discussed in comment #2. The non-SDK clients of JDT
> Core should also be considered. 

Can you clarify what you mean by "should be considered"?

If you suggest we should find a solution that's guaranteed not to break any clients, I don't think that is possible, not with that Java 9 we have at hands.

BTW, I even announced at ECE that we have to break API contracts in this particular case one way or other.

IMHO, its important we have a precise description of the change and an I-build containing the change, and refer to both when sending an alert to cross-projects, ideally well before M5.
Comment 44 Noopur Gupta CLA 2018-01-16 09:48:31 EST
(In reply to Stephan Herrmann from comment #43)
> (In reply to Noopur Gupta from comment #41)
> > I think the same was discussed in comment #2. The non-SDK clients of JDT
> > Core should also be considered. 
> 
> Can you clarify what you mean by "should be considered"?
I mean to notify them (including PDE) in advance of this breaking change so that they can also take necessary actions.

> (In reply to Jay Arthanareeswaran from comment #33)
> > Noopur/Sarika, can you please look at the respective patches and see if it's
> > good enough for consumption. 
> For JDT UI, a pass will be required over other usage locations as well to
> fix them if needed.
Probably this comment wasn't clear. I am fine with the JDT Core change but the JDT UI patch needs more work before committing.
Comment 45 Stephan Herrmann CLA 2018-01-16 09:52:36 EST
(In reply to Noopur Gupta from comment #44)

Thanks, sounds good to me.
Comment 46 Stephan Herrmann CLA 2018-02-01 13:36:47 EST
Anything I can do to help completing this task?
Comment 47 Jay Arthanareeswaran CLA 2018-02-01 23:00:17 EST
(In reply to Stephan Herrmann from comment #46)
> Anything I can do to help completing this task?

I had a discussion with Noopur yesterday on this and she would like someone to handle the other locations in JDT UI where we switch on the classpath entry kind. I said I will take a stab at it this week. Perhaps, some review time later will be nice :)
Comment 48 Eclipse Genie CLA 2018-02-02 04:04:33 EST
New Gerrit change created: https://git.eclipse.org/r/116579
Comment 49 Jay Arthanareeswaran CLA 2018-02-02 05:56:52 EST
Noopue, the latest UI patchset contains some more changes. I went through some 160+ instances of getEntryKind() and IClasspathEntry#CPE_JRT*. Most locations don't need any change because they are dealing with raw classpath entries, which means nothing has changed. The CPE_JRT come into the picture only for resolved entries. For scenarios I wasn't sure of, like the image, label generation etc., I added the new kind in the same case block as CPE_LIBRARY. 

Just one question remains from me:

In the build path dialog, the container doesn't get child nodes (i.e. modules), like we used to get the rt.jar and all. Hence, there's no way to attach the Javadoc location to the JRE container.

Should we allow the Javadoc attribution to the container kinds as well?

Over to you Noopur. All tests pass with a JDK 9 locally.
Comment 50 Noopur Gupta CLA 2018-02-02 08:08:52 EST
I tried the new JDT Core Gerrit and the latest patch set in JDT UI Gerrit without checking any code. 

1) I still see the exception from comment #38 in Error log. Just the line number has changed now:
java.lang.NullPointerException
	at org.eclipse.jdt.internal.ui.wizards.buildpaths.CPListElement.equals(CPListElement.java:591)
...

2) On expanding the JRE 9 node in build path dialog, I see only java.activation module below "Is modular". It should show all the modules or none.

3) I am also getting the following exception in main Eclipse's Console quite frequently. I think one scenario is while trying to change the JRE in build path dialog and applying the changes:
java.lang.NegativeArraySizeException
	at org.eclipse.jdt.internal.core.builder.State.readRestriction(State.java:456)
	at org.eclipse.jdt.internal.core.builder.State.read(State.java:294)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.readState(JavaBuilder.java:151)
	at org.eclipse.jdt.internal.core.JavaModelManager.readState(JavaModelManager.java:4140)
	at org.eclipse.jdt.internal.core.JavaModelManager.getLastBuiltState(JavaModelManager.java:2288)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.getLastState(JavaBuilder.java:433)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:179)
...
org.eclipse.core.runtime.CoreException: Error reading last build state for project P9
	at org.eclipse.jdt.internal.core.JavaModelManager.readState(JavaModelManager.java:4148)
	at org.eclipse.jdt.internal.core.JavaModelManager.getLastBuiltState(JavaModelManager.java:2288)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.getLastState(JavaBuilder.java:433)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:179)
... 
Caused by: java.lang.NegativeArraySizeException
	at org.eclipse.jdt.internal.core.builder.State.readRestriction(State.java:456)
	at org.eclipse.jdt.internal.core.builder.State.read(State.java:294)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.readState(JavaBuilder.java:151)
	at org.eclipse.jdt.internal.core.JavaModelManager.readState(JavaModelManager.java:4140)
... 15 more

4) (In reply to Jay Arthanareeswaran from comment #49)
> In the build path dialog, the container doesn't get child nodes (i.e.
> modules), like we used to get the rt.jar and all. Hence, there's no way to
> attach the Javadoc location to the JRE container.
> 
> Should we allow the Javadoc attribution to the container kinds as well?
We are also missing the source attachment attribute like the Javadoc location. Other attributes like Native lib loc, access rules, and external annotations are available on the JRE container. I am not sure if individual modules can have different values for all these attributes. If not, I think we should allow the Javadoc and Source attributes on CPE_JRT_SYSTEM and not in general on all containers.
Comment 51 Noopur Gupta CLA 2018-02-02 08:30:53 EST
jdk-9.0.4+11

With these patches from core, ui and debug, when I am trying to run a simple Java program with main() in a Java 9 modular project (having module-info.java), I am getting the following exception and it does not run (same with JUnit tests also):

Error occurred during initialization of boot layer
java.lang.LayerInstantiationException: Package jdk.internal.jrtfs in both module java.base and module jrt.fs
Comment 52 Jay Arthanareeswaran CLA 2018-02-04 23:59:52 EST
(In reply to Noopur Gupta from comment #51)
> Error occurred during initialization of boot layer
> java.lang.LayerInstantiationException: Package jdk.internal.jrtfs in both
> module java.base and module jrt.fs

I suspect somehow the jrt-fs.jar ends as an automatic module in the runtime classpath. Will see what causes this.
Comment 53 Jay Arthanareeswaran CLA 2018-02-05 01:19:33 EST
I am going to have to request Sarika to take this further. I narrowed down to two places where the changes need to go:

1. StandardVMType.getDefaultLibraryLocations(File) - we should remove the code that adds the jrt-fs.jar to the library locations and a matching change in canDetectDefaultSystemLibraries() to allow "0" libraries.

2. JREContainer#computeClasspathEntries() should be made to work without the jrt-fs.jar in the getLibraryLocations(). We should somehow construct the source path and other classpath attributess right here.

Sarika, will you able to take this forward?
Comment 54 Sarika Sinha CLA 2018-02-05 02:04:31 EST
(In reply to Jay Arthanareeswaran from comment #53)
> I am going to have to request Sarika to take this further. I narrowed down
> to two places where the changes need to go:
> 
> 1. StandardVMType.getDefaultLibraryLocations(File) - we should remove the
> code that adds the jrt-fs.jar to the library locations and a matching change
> in canDetectDefaultSystemLibraries() to allow "0" libraries.
> 
> 2. JREContainer#computeClasspathEntries() should be made to work without the
> jrt-fs.jar in the getLibraryLocations(). We should somehow construct the
> source path and other classpath attributess right here.
> 
> Sarika, will you able to take this forward?

Will look into the feasibility of it.
Comment 55 Jay Arthanareeswaran CLA 2018-02-05 02:44:10 EST
(In reply to Sarika Sinha from comment #54)
> Will look into the feasibility of it.

Thanks Sarika.

I found a major bug in the jdt.core patch and fixed in the most recent iteration. The index was not considered for the new entry kind and this kept some features from working.
Comment 56 Sarika Sinha CLA 2018-02-05 04:05:06 EST
(In reply to Jay Arthanareeswaran from comment #0)
> The bug is same as reported in bug 489207. That bug provided a short-term
> fix by adding the jrt-fs.jar as the single library to the container. But
> ideally we should try to make the JREContainer work without referring to any
> physical library because there isn't any in case of JRE 9.

Sorry for going back to the basics.
Why do we say that there is no physical library in case of JRE 9. We have java.base modules ? Isn't it mandatory physical library for Java 9?
Comment 57 Jay Arthanareeswaran CLA 2018-02-05 04:16:07 EST
(In reply to Sarika Sinha from comment #56)
> Sorry for going back to the basics.
> Why do we say that there is no physical library in case of JRE 9. We have
> java.base modules ? Isn't it mandatory physical library for Java 9?

Did you mean java.base.jmod? Unlike JRE 8 or below, we can't say the system libraries are in a bunch of JAR files. The jmods are indeed physical libraries, but I don't think we will add them all to the container. On the other hand, it makes sense to change the container to work without any libraries and load the modules on a need basis.
Comment 58 Sarika Sinha CLA 2018-02-05 05:18:20 EST
Right now, I see two challenges. To compute the classpath attributes, 3 things are needed - java doc location, index location for the library and the external annotation location for the library.
So we can get the java doc location from vm, but what about other 2?
Comment 59 Jay Arthanareeswaran CLA 2018-02-05 05:21:58 EST
(In reply to Sarika Sinha from comment #58)
> Right now, I see two challenges. To compute the classpath attributes, 3
> things are needed - java doc location, index location for the library and
> the external annotation location for the library.
> So we can get the java doc location from vm, but what about other 2?

The other option is to let the library location be computed till the JREContainer is computed and later on removed. This could be a dumb suggestion, but I don't know the code much.
Comment 60 Sarika Sinha CLA 2018-02-05 05:35:18 EST
I think we need to think a little more on this, 4.7.3 does not seem feasible.
Comment 61 Jay Arthanareeswaran CLA 2018-02-05 05:54:11 EST
(In reply to Sarika Sinha from comment #60)
> I think we need to think a little more on this, 4.7.3 does not seem feasible.

I agree. But M6 is not very far away either. Early M6 would be nice.
Comment 62 Sarika Sinha CLA 2018-02-06 01:19:04 EST
Another question is, how should be the representation of libraries in "JRE Definition" dialog ?
How should we display the JRE system libraries ?
We still need a node to show the source, java doc and external annotation. But what should be it's label ?
And that also means there can be only one java doc location and one external annotation jar for all the modules of java 9.
Comment 63 Stephan Herrmann CLA 2018-02-06 08:36:59 EST
Trying to catch up on the traffic here ...

(In reply to Jay Arthanareeswaran from comment #57)
> (In reply to Sarika Sinha from comment #56)
> > Sorry for going back to the basics.
> > Why do we say that there is no physical library in case of JRE 9. We have
> > java.base modules ? Isn't it mandatory physical library for Java 9?
> 
> Did you mean java.base.jmod? Unlike JRE 8 or below, we can't say the system
> libraries are in a bunch of JAR files. The jmods are indeed physical
> libraries, but I don't think we will add them all to the container. On the
> other hand, it makes sense to change the container to work without any
> libraries and load the modules on a need basis.

If I understand correctly then jmod files are not relevant for compiling nor launching but provided as a basis for creating your own image (with jlink), right?

If true then I second the goal to not depend on those file. They might even stop shipping those in the default installation, couldn't they? Having jrt-fs.jar and the undocumented lib/modules would be sufficient for normal use.
Comment 64 Stephan Herrmann CLA 2018-02-06 08:44:01 EST
(In reply to Sarika Sinha from comment #62)
> Another question is, how should be the representation of libraries in "JRE
> Definition" dialog ?
> How should we display the JRE system libraries ?

Shouldn't that node have the library icon, just as it has in other locations?

> We still need a node to show the source, java doc and external annotation.
> But what should be it's label ?

A path just like now, just without the trailing jrt-fs.jar ?

> And that also means there can be only one java doc location and one external
> annotation jar for all the modules of java 9.

I think that's fair.

Going back to your previous question:

(In reply to Sarika Sinha from comment #58)
> Right now, I see two challenges. To compute the classpath attributes, 3
> things are needed - java doc location, index location for the library and
> the external annotation location for the library.
> So we can get the java doc location from vm, but what about other 2?

External annotation location is not automatically populated, so initially this will simply remain blank. Not a problem.

Remains javadoc location, how is that determined nowadays?
Comment 65 Jay Arthanareeswaran CLA 2018-02-06 09:13:45 EST
(In reply to Stephan Herrmann from comment #63)
> If I understand correctly then jmod files are not relevant for compiling nor
> launching but provided as a basis for creating your own image (with jlink),
> right?
> 
> If true then I second the goal to not depend on those file. They might even
> stop shipping those in the default installation, couldn't they? Having
> jrt-fs.jar and the undocumented lib/modules would be sufficient for normal
> use.

Spot on! Although expanding the container to individual jmod files will work (bug 500905, which is waiting for proper tests), that's an internal detail we don't want to worry about.
Comment 66 Jay Arthanareeswaran CLA 2018-02-06 21:46:03 EST
(In reply to Sarika Sinha from comment #62)
> 2) On expanding the JRE 9 node in build path dialog, I see only
> java.activation module below "Is modular". It should show all the modules or
> none.

I am going to need some help with this. I am looking at the code where we add the pseudo nodes to the container (CPListElement#addModuleNodes) and can see all modules being added to the children. But I can't pinpoint the location where only the [0] is picked. Can someone point me where I should be looking?
Comment 67 Noopur Gupta CLA 2018-02-07 04:30:02 EST
(In reply to Jay Arthanareeswaran from comment #66)
> (In reply to Sarika Sinha from comment #62)
> > 2) On expanding the JRE 9 node in build path dialog, I see only
> > java.activation module below "Is modular". It should show all the modules or
> > none.
> 
> I am going to need some help with this. I am looking at the code where we
> add the pseudo nodes to the container (CPListElement#addModuleNodes) and can
> see all modules being added to the children. But I can't pinpoint the
> location where only the [0] is picked. Can someone point me where I should
> be looking?

I don't think only [0] is being picked anywhere. This issue may be caused by the exception mentioned in (1) in comment #50. Check Error Log view when you expand JRE node. So, once this is fixed we should be able to see all modules as it was before the patch.
Comment 68 Jay Arthanareeswaran CLA 2018-02-07 05:45:31 EST
(In reply to Noopur Gupta from comment #67)
> I don't think only [0] is being picked anywhere. This issue may be caused by
> the exception mentioned in (1) in comment #50. Check Error Log view when you
> expand JRE node. So, once this is fixed we should be able to see all modules
> as it was before the patch.

Thanks Noopur! Are you beginning to wonder you might as well have written the patch? If so, it's still not too late :)

Anyway, the latest patch fixes the NPE and the missing modules. I don't see the other exception. Will play around bit more.
Comment 69 Noopur Gupta CLA 2018-02-07 05:54:18 EST
(In reply to Jay Arthanareeswaran from comment #68)
> (In reply to Noopur Gupta from comment #67)
> > I don't think only [0] is being picked anywhere. This issue may be caused by
> > the exception mentioned in (1) in comment #50. Check Error Log view when you
> > expand JRE node. So, once this is fixed we should be able to see all modules
> > as it was before the patch.
> 
> Thanks Noopur! Are you beginning to wonder you might as well have written
> the patch? 
Not yet! :)

> Anyway, the latest patch fixes the NPE and the missing modules. I don't see
> the other exception. Will play around bit more.
Thanks, I will also try out the new patch sets after we get the update in debug.
Comment 70 Sarika Sinha CLA 2018-02-07 07:40:19 EST
(In reply to Noopur Gupta from comment #69)
> Thanks, I will also try out the new patch sets after we get the update in
> debug.

Updated the JDT Debug patch.
Comment 71 Noopur Gupta CLA 2018-02-07 09:08:53 EST
I see the module nodes now but I can see only the "Javadoc location" attribute under each module. Other attributes like "Source attachment" etc. are missing. I don't think this is expected. I couldn't get any clue in a quick debugging session. Probably the new constant CPE_JRT_SYSTEM is still not handled somewhere.
Comment 72 Jay Arthanareeswaran CLA 2018-02-07 10:15:42 EST
The remaining issues required changes to both debug and ui. The gerrits are updated now.

Sarika, in patch set 6, I see you removed one of my changes in JREContainerInitializer#requestClasspathContainerUpdate(). Is this intentional? I have put that code back in. Let me know if it's wrong.
Comment 73 Sarika Sinha CLA 2018-02-07 22:57:17 EST
(In reply to Jay Arthanareeswaran from comment #72)
> The remaining issues required changes to both debug and ui. The gerrits are
> updated now.
> 
> Sarika, in patch set 6, I see you removed one of my changes in
> JREContainerInitializer#requestClasspathContainerUpdate(). Is this
> intentional? I have put that code back in. Let me know if it's wrong.

Looks fine!!
Comment 74 Noopur Gupta CLA 2018-02-12 07:31:52 EST
I tried the latest patch sets and the reported issues are resolved now.
Comment 78 Andrey Loskutov CLA 2018-02-13 12:10:32 EST
(In reply to Eclipse Genie from comment #75)
> Gerrit change https://git.eclipse.org/r/116579 was merged to [master].
> Commit:
> http://git.eclipse.org/c/jdt/eclipse.git/commit/
> ?id=1e07a149262601f724e34444775f518eaaad79c5

Since this patch Gerrit reports 12 failing tests, even the last patch set shows them. Any reason why the patch was still merged? Do we plan to fix tests later?
Comment 79 Jay Arthanareeswaran CLA 2018-02-13 22:23:51 EST
(In reply to Andrey Loskutov from comment #78)
> (In reply to Eclipse Genie from comment #75)
> > Gerrit change https://git.eclipse.org/r/116579 was merged to [master].
> > Commit:
> > http://git.eclipse.org/c/jdt/eclipse.git/commit/
> > ?id=1e07a149262601f724e34444775f518eaaad79c5
> 
> Since this patch Gerrit reports 12 failing tests, even the last patch set
> shows them. Any reason why the patch was still merged? Do we plan to fix
> tests later?

The Core test framework depends on the debug changes and these failures are expect to go away after we have the next I build. Of course, the debug changes can't go in first because they need core changes for compilation :)
Comment 80 Eclipse Genie CLA 2018-02-14 03:34:56 EST
New Gerrit change created: https://git.eclipse.org/r/117321
Comment 81 Eclipse Genie CLA 2018-02-14 04:47:02 EST
New Gerrit change created: https://git.eclipse.org/r/117335
Comment 82 Jay Arthanareeswaran CLA 2018-02-14 04:52:20 EST
(In reply to Eclipse Genie from comment #81)
> New Gerrit change created: https://git.eclipse.org/r/117335

Looks like some of the test changes got lost in merge or updating the gerrit.
This patch fixes the failures with JRE 8. I will post another gerrit for the failures in BuildpathTests with JRE 9.
Comment 84 Eclipse Genie CLA 2018-02-14 08:26:40 EST
New Gerrit change created: https://git.eclipse.org/r/117353
Comment 86 Sasikanth Bharadwaj CLA 2018-02-15 03:54:17 EST
I see lot of errors in my runtime since this change of the kind build path of project is incomplete, java.lang.Object cannot be resolved. Is any user action required like deleting the JRE and adding it again? ( I tired that, but it didn't work)
Comment 87 Sasikanth Bharadwaj CLA 2018-02-15 03:58:26 EST
And changing the default workspace JRE to 1.8 consistently causes OutOfMemoryError :-(
Comment 88 Dani Megert CLA 2018-02-15 04:23:39 EST
Something is fishy here. It looks like a race condition. In a fresh workspace I got the exceptions from comment 50 after adding a new project and a package!

Trying with another fresh workspace did not reproduce it.

NOTE: The exceptions are in the console that started the IDE, not in the .log or Eclipse Console view.
Comment 89 Dani Megert CLA 2018-02-15 05:07:35 EST
(In reply to Dani Megert from comment #88)
> Trying with another fresh workspace did not reproduce it.

Here are the steps to reproduce the issue:
1. Start a new workspace with 4.8 M5
2. Create Java project 'P'
3. Exit
4. Start with I20180214-2000
5. Create package 'p' in P
==> exceptions in console
Comment 90 Jay Arthanareeswaran CLA 2018-02-15 05:37:15 EST
I can reproduce some of the reported issues. I started investigating, but probably not going to be able to come around with a quick fix. So, for now I have reverted fixes to all the three repos:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=b970fee63ce679d80b7e665b70dcbd93d014a258

http://git.eclipse.org/c/jdt/eclipse.jdt.ui.git/commit/?id=aa56cd6109cd70ed2fcec454ddb6f149bfc66e05

http://git.eclipse.org/c/jdt/eclipse.jdt.debug.git/commit/?id=399251d24ca7750b72cc08f1d693cd6cc738d22e

I will let these get picked up by the next build.
Comment 91 Eclipse Genie CLA 2018-02-28 01:44:35 EST
New Gerrit change created: https://git.eclipse.org/r/118325
Comment 92 Jay Arthanareeswaran CLA 2018-02-28 07:59:45 EST
Okay, finally I nailed it down to the change in JavaRuntime. Patch set removed the following code:

if (!libraryPaths[i].toFile().isFile()) {
  libraryPaths[i] = Path.EMPTY;
}

But patch set brought that change back with a condition:

if (compareJavaVersions(vm, JavaCore.VERSION_9) < 0) {
  if (!libraryPaths[i].toFile().isFile()) {
    libraryPaths[i] = Path.EMPTY;
  }
}

Unfortunately, the check has no effect as at this point, the Java version in the vm is null.

Sarika, can you please look into this? Also, what was the effect if the code is removed altogether?
Comment 93 Sarika Sinha CLA 2018-03-01 00:10:42 EST
Having a check like this makes sense I guess instead of removing it -
if (!(libraryPaths[i].toFile().isFile() || libraryPaths[i].toFile().isDirectory())) {
	libraryPaths[i] = Path.EMPTY;
}
Comment 94 Eclipse Genie CLA 2018-03-01 01:40:27 EST
New Gerrit change created: https://git.eclipse.org/r/118394
Comment 95 Eclipse Genie CLA 2018-03-01 01:45:46 EST
New Gerrit change created: https://git.eclipse.org/r/118395
Comment 96 Dani Megert CLA 2018-03-01 08:51:40 EST
Moving to M7 as discussed with Jay.
Comment 97 Jay Arthanareeswaran CLA 2018-04-17 02:26:03 EDT
Since there's no pressing need for this and that there's some work to be done from the UI and launch front, I am forced to move this out of 4.8.
Comment 98 Jay Arthanareeswaran CLA 2018-09-05 05:47:42 EDT
Moving out of 4.9 again.
Comment 99 Jay Arthanareeswaran CLA 2018-11-27 01:11:28 EST
Despite spending lot of time on this already, I am not sure I am going to find in near future to tackle the remaining issues. Removing the target.
Comment 100 Eclipse Genie CLA 2020-11-17 13:35:49 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 101 Eclipse Genie CLA 2023-01-08 13:01:00 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.