Community
Participate
Working Groups
In order to expose to the resolver the packages provided by the JRE, p2 has been using a special IU called a.jre which is derived in multiple versions each representing a JRE level. This causes a number of issue like bug #341432. The current thinking is to allow for "capability providers" to provide a number of capabilities that would get automatically provided in the profile and used to resolve against. One of those capability providers would be a JRE provider that would exports all / some of the packages from the JRE as capabilities. Questions to think about: - When are those providers called to provide their list? Everytime a resolution occurs? - How do those providers work in the case of external provisioning. For example, I'm creating the EPP package on my machine to make available?
Marking it 3.8 for it to not fall of the radar. Answers to these questions greatly appreciated.
*** Bug 341432 has been marked as a duplicate of this bug. ***
Move all 3.8 bugs to Juno.
*** Bug 183884 has been marked as a duplicate of this bug. ***
As a followup on the last week discussion around JRE, here is a summary and initial breakdown of the problem: What p2 publishes as jre capabilities is currently based on an OSGi Java profile specified at publishing time (in fact the current implementation uses OSGi/J2SE-1.5 profile no matter what is specified and package versions are ignored). This creates an assumption that publish-time and runtime JRE packages would be same (or at least not conflicting) which is not always true. So we are looking for a way to publish a self-contained and consistent (resolvable) p2 repository without assuming what would be provided by the actual installation environment. A repo being self-contained is a former requirement that is worth discussing even if only to clarify why it is important. There are two aspects of the problem: - How to represent in p2 a requirement to an execution environment which to be satisfied by the actual environment; - How to deal with further adjustment of JRE packages set Some ideas that were discussed on the first one: - the a.jre could be created at installation time, based on inspection of which packages are actually provided by the target system. Extra attention should be paid here to handle cases where p2 installer and target environment require different JREs. - to define a requirement to the minimal runtime exec env (without capturing the actual packages set in an IU). For example if the product needs JRE 1.5, then have req on the IU 1.5. On the second aspect: There are several ways to configure packages provided by JRE. They are described in osgi.profile section of http://help.eclipse.org/indigo/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Freference%2Fmisc%2Fruntime-options.html. Configuration of that kind: - could happen at any time (during and/or after the initial installation) - it could be done either via p2 or manually - becomes effective after target system restart (because it's done via java system props) This would lead to inconsistent p2 profile if additional alignment doesn't take place. Example scenarios that may be interesting: - Installation requiring JavaSE 1.x contains an bundle configured to set an extra package when installed. Change of that kind would lead to unreliable/unexpected resolution on the next system restart since jre packages set is changed. - Java profile is changed explicitly – e.g. configured by an administrator. Previously installed artifacts may fail to resolve.
(In reply to comment #5) > - the a.jre could be created at installation time, based on inspection of which > packages are actually provided by the target system. Extra attention should be > paid here to handle cases where p2 installer and target environment require > different JREs. This comes down making the profile aware of the JRE it is supposed to be created for. However I don't believe that using a persisted IU for this is a good idea because that IU then may slip out of the profile. Remember, profiles are also p2 metadata repositories. > There are several ways to configure packages provided by JRE. They are described > in osgi.profile section of > http://help.eclipse.org/indigo/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Freference%2Fmisc%2Fruntime-options.html. > Configuration of that kind: > - could happen at any time (during and/or after the initial installation) > - it could be done either via p2 or manually I wouldn't try to support changes of the system.packages by anyone else but p2. > - Installation requiring JavaSE 1.x contains an bundle configured to set an > extra package when installed. Change of that kind would lead to > unreliable/unexpected resolution on the next system restart since jre packages > set is changed. You mean through a fragment to the system bundle? These fragments are installed through p2 so I don't see how this leads to instability. The more interesting question is if/how p2 should check if such a bundle can be installed.
(In reply to comment #6) > This comes down making the profile aware of the JRE it is supposed to be > created for. However I don't believe that using a persisted IU for this is a > good idea because that IU then may slip out of the profile. Remember, profiles > are also p2 metadata repositories. I agree, having as less hand-crafted IUs as possible in the repo is a good idea from my pov. On the other hand not having persisted IU for JRE packages in p2 profile would cut off the opportunity for installation on top of an "offline" system because there's no one to provide this required runtime information. > I wouldn't try to support changes of the system.packages by anyone else but p2. This means "disabling" of standard functionality provided by eqinox but I guess this sounds reasonable since such low-level stuff should be delegated to p2. > > - Installation requiring JavaSE 1.x contains an bundle configured to set an > > extra package when installed. Change of that kind would lead to > > unreliable/unexpected resolution on the next system restart since jre packages > > set is changed. > You mean through a fragment to the system bundle? These fragments are installed > through p2 so I don't see how this leads to instability. > The more interesting question is if/how p2 should check if such a bundle can be > installed. Fragments to the system bundle is not what bothers me here, let me give an example: Consider a simple product PA containing 1 bundle: - A is published with configure instruction addJVMArg ("system.packages += javax.persistence;version:1.3") - PA requires JavaSE1.5 Let's have also another repo containing bundle B which requires package javax.persistence;version:1.3 When trying to install PA + B, the resolution would fail because: - the product requirement to JavaSE1.5 would "provide" a capability "javax.persistence;version:0.0.0" (this is the current p2 behavior, and I use it just for illustration) - no other satisfies the req to 1.3. But if you install PA, configure instruction would be executed and after restart, the package javax.persistence;version:1.3 would be available so B could be installed. That's what I meant by inconsistency here and it's managed by p2. The problem comes from the fact that metadata(only for the special case of jre) could come after the initial resolution and thus could make it invalid. Does this make more sense now?
(In reply to comment #7) > (In reply to comment #6) > > This comes down making the profile aware of the JRE it is supposed to be > > created for. However I don't believe that using a persisted IU for this is a > > good idea because that IU then may slip out of the profile. Remember, profiles > > are also p2 metadata repositories. > I agree, having as less hand-crafted IUs as possible in the repo is a good idea > from my pov. On the other hand not having persisted IU for JRE packages in p2 > profile would cut off the opportunity for installation on top of an "offline" > system because there's no one to provide this required runtime information. I just wanted to say: don't fix this by just generating the "a.jre" IU by the p2 client, but have a special profile property for this with the same effect. I'm not entirely sure about the implementation though. > Fragments to the system bundle is not what bothers me here, let me give an > example: > Consider a simple product PA containing 1 bundle: > - A is published with configure instruction addJVMArg ("system.packages += > javax.persistence;version:1.3") The system packages need to be taken into account for planning, so they must neither be modified through touchpoints. I anyway don't understand why you'd want to do this - what's wrong with system bundle fragments for this use case?
(In reply to comment #8) > I just wanted to say: don't fix this by just generating the "a.jre" IU by the > p2 client, but have a special profile property for this with the same effect. > I'm not entirely sure about the implementation though. We discussed some options(including impl details) during the p2 call this week. I'll list the main points in the next comment so please share what you think. > The system packages need to be taken into account for planning, so they must > neither be modified through touchpoints. I anyway don't understand why you'd > want to do this - what's wrong with system bundle fragments for this use case? Nothing is wrong with system bundle fragments - it's one of the existing ways to change system packages and probably much better than changing them through touchpoints. My point here was to try to give an example how in one transaction (install operation) one can resolve against set of packages which would change later (before commit). Using touchpoints was the only thing that came to my mind as an illustration. So the problem is that such operation should be considered as "invalid" but there's no clean way to detect that. You're right, this may be a weird case so we don't need to worry about it. But I think it's good to have that stated clearly so thank you for helping me clarify it.
MM of this week's p2 call. Publish time: We should keep "a.jre" unit in the repository for backward compatibility. We're going to fix currenly existing limitations (first one with higher priority): - a.jre is always based on 1.5 profile since specified profile is ignored by the product publisher. There are several bugs opened on that: https://bugs.eclipse.org/bugs/show_bug.cgi?id=334519 https://bugs.eclipse.org/bugs/show_bug.cgi?id=364222 - a.jre provides all the packages listed in the profile with v 0.0.0; this currently causes issues when a consumer has explicitly specified version on package import. Challenges here: As Tobias mentioned it's not clear where to get from versions of these packages so this is still under investigation. Changing package versions from 0.0.0 to something concrete could break some of the existing users. Pascal is thinking of existing scenarios which could be affected. Install time: - Installation from scratch To support installation from scratch we should keep the fake a.jre unit in the profile. On one hand system packages are important for proper resolution, on the other in this case there's no way to guess or calculate the execution environment of the target system. Still, we can provide an option on installation to specify the version of expected env. This is not enough for a precise check and could be easily omitted but it will at least educate the end user that exec env is important. - Runtime: The fake a.jre will exist in the profile until the first start of the installation and would be a placeholder for the actually provided system packages. On every start, when profile registry is requested to load a profile, an exec env "capability provider" will be invoked to update this placeholder with these packages (ideally as seen and calculated by equinox since we don't want to duplicate existing logic). In the end there would be only one IU in the p2 profile providing system packages. This proposal doesn't cover all possible cases - it addresses currently exsisting use cases and offers a solution which could be extended if needed. (An example for not covered case - it would be still possible to have inconsistency when installing on top of not-yet-started installation which has customized list of system packages) From my point of view we have achieved a relatively clear vision what should be done so implementation could be started. If you have any concerns, questions, observations, please share.
I find this automatic adaptation to the running JRE problematic. What happens, if the JRE version is decreased and hence the installation becomes incomplete? The only solution I can think of is to ask the user. When do we want to do this? On startup? On the first provisioning operation? What do we do in headless mode?
(In reply to comment #11) Let me try to answer your questions in reverse order: When do we want to do > this? On startup? On the first provisioning operation? What do we do in > headless mode? Changes to jre packages will be available on system startup. So we have two choices here: - to update on startup; - to update on demand (when provisioning operation happens); As drawback in this case we lose offline update because the IU in the profile wouldn't be up-to-date if the system is shut down; What changes in headless mode? Why should we treat this case differently? > I find this automatic adaptation to the running JRE problematic. What happens, > if the JRE version is decreased and hence the installation becomes incomplete? Yes, there's a risk. We wouldn't cover all the cases since there's no way to guess whether user is going to change system configuration after initial installation. From my point of view, in the worst case we have the same behavior as current one. But this would be rather a rare case - up to now I'm not aware of a real use case where JRE version is decreased. The difference from now is that we'll make an attempt to protect system consistency and the "positive" scenarios (where user doesn't intend to hack the system) would be covered. I think this goal is worth the change. > The only solution I can think of is to ask the user. I can think of several options here: - Don't ask the user at all - thus he'll get what's published in the repository (this is the current behavior with the small exception that provider will be able to publish custom profile) - Ask for the version of JRE (1.5, 1.6, etc) - Ask for a whole profile with proper versioning of all the packages. I'm not in favor for having just the last option because: - it's too complex, so too error prone - consumer is supposed to know internals like what's profile, that it's used for generating fake IU capturing system packages of the target, etc. Also consumer should guarantee that the profile provider has used is compatible with what consumer would pass on installation. Install operation is a high level operation which is widely used. I think most of the end users wouldn't care about changing JRE packages so it's good to keep things simple for them. - my personal preference is to check what's really available in the system rather than strongly relying on user input. User would be more happy if when he passes wrong input the system is able to recover/calculate the right set of packages instead of blaming him and fail. In fact we can support all of these - my point here is to keep an eye on end user experience, not only on the technical part.
I'm moving this out. If someone plans on working on this please add it to kepler.
*** Bug 430474 has been marked as a duplicate of this bug. ***
*** Bug 429446 has been marked as a duplicate of this bug. ***
(In reply to Katya Todorova from comment #12) > In fact we can support all of these - my point here is to keep an eye on end > user experience, not only on the technical part. That sounds like an objective worth pursuing. The present situation means that a user with automatic updates enabled will discover that the product suddenly breaks. Bundles were installed that were incompatible with the EE. That's a totally unacceptable user experience. Let's not forget that one major objective for p2 is to create reliable solutions. The user should be able to trust that when p2 is able to create a solution successfully, then the install will also run OK. We can't have scenarios when this isn't true. I think the solution is simple. Produce the same kind of error that we do when there's a conflict of some kind and refuse to continue. That will give the user a hint that the JRE must be upgraded for the install to succeed.
Raising the priority on this to major since the current situation leads to a totally unacceptable user experience when automatic updates are enabled.
On the flip side there are several bundles I know of that provide optional support when using a later version of the VM. For example, org.eclipse.jdt.annotation in the Eclipse project that requires Java 8 but the overall platform still supports Java 6. Having p2 all of a sudden fail to install these types of bundles will break other folks. Not saying that means this issue cannot be addressed, but we also need a solution for this type of scenario.
(In reply to Thomas Watson from comment #18) > Having p2 all of a sudden fail to install these types of bundles will break > other folks. Not sure I understand. What will break if p2 reports an incompatibility instead of silently installing something that will not run?
(In reply to Thomas Hallgren from comment #19) > (In reply to Thomas Watson from comment #18) > > Having p2 all of a sudden fail to install these types of bundles will break > > other folks. > > Not sure I understand. What will break if p2 reports an incompatibility > instead of silently installing something that will not run? If I understand correctly this will prevent installing something that will run correctly when that something includes optional bundles that will only be enabled if running on the correct VM. If running on a lower VM version then the optional bundles will simply be unresolved, but the installed stuff still works correctly.
(In reply to Thomas Watson from comment #20) > If I understand correctly this will prevent installing something that will > run correctly when that something includes optional bundles that will only > be enabled if running on the correct VM. If the bundles are optional then p2 should still find a working solution that doesn't include them. I don't think it would prevent the install. It would just skip those bundles. One negative side effect would be that the bundles would be missing when the user switches to a more modern JRE. That is, until another attempt is made to install them. That's much less serious IMO than the current scenario where an existing install is rendered useless.
(In reply to Thomas Hallgren from comment #21) > (In reply to Thomas Watson from comment #20) > > If I understand correctly this will prevent installing something that will > > run correctly when that something includes optional bundles that will only > > be enabled if running on the correct VM. > > If the bundles are optional then p2 should still find a working solution > that doesn't include them. I don't think it would prevent the install. It > would just skip those bundles. I think that would depend on the IU created for the features that include the bundle. I'm not sure how these feature IUs are created, but it would seem that if the feature IU includes the bundle then it would need to be installed. Perhaps there is some way to mark the bundle as optional from the feature IU perspective.
(In reply to Thomas Watson from comment #22) > I think that would depend on the IU created for the features that include > the bundle. Yes. When I wrote "If the bundles are optional" what I really I meant was "if all requirements for a bundle are optional". If that's not true, then p2 will not find a satisfying solution. In other words, the bundle itself is never optional. That fact is always stated in the requirements. > Perhaps there is some way to mark the bundle as optional from the feature > IU perspective. Yes. A p2 requirement can always be made optional. If a feature.xml or MANIFEST.MF is not expressive enough then a p2.inf file can contain more details.
(In reply to Thomas Hallgren from comment #21) > (In reply to Thomas Watson from comment #20) > > If I understand correctly this will prevent installing something that will > > run correctly when that something includes optional bundles that will only > > be enabled if running on the correct VM. > > If the bundles are optional then p2 should still find a working solution > that doesn't include them. I don't think it would prevent the install. It > would just skip those bundles. > > One negative side effect would be that the bundles would be missing when the > user switches to a more modern JRE. That is, until another attempt is made > to install them. That's much less serious IMO than the current scenario > where an existing install is rendered useless. Don't forget to work though the various "build" and "packaging" scenarios. I don't know what the impact would be, off the top of my head, but seems like there would be a large one. It seems your main concern is a users installing/updating their own dev. environment, and my first thought would be to "solve with more UI" ... such as warn or ask the user what they'd like to do. I sort of see this like the "unsigned" workflow, where the user is given a warning, and can see a list of what's unsigned and decide if to continue or not. Perhaps if shown a list of bundles that would not run with their current VM, they might be able to decide if it was a minor case they didn't care about, cancel out of it all, or go ahead and update, and then, after not restarting, change their VM. Does that "UI approach" solve your concern, or are you thinking of something else?
(In reply to David Williams from comment #24) > Don't forget to work though the various "build" and "packaging" scenarios. > I don't know what the impact would be, off the top of my head, but seems > like there would be a large one. > I think build/packaging scenarios would need do what they currently do with os/ws/architecture, i.e. turn those filters off in cases where they don't apply to the target. A tool like the b3 aggregator would define the maximum allowed EE. This is actually a good argument for my suggested approach. If p2 will not install something that is incompatible with the current os/ws/archicture (and it doesn't), then why install something that is incompatible with the EE? > It seems your main concern is a users installing/updating their own dev. > environment My main concern is the users that report that their IDE no longer provides the functionality provided by our features after accepting suggested update. "I accepted the update and now my IDE is destroyed! WTF?". I completely understand the frustration but I'm unable to provide a good answer. I have to "blame" p2 and that doesn't feel good at all. > I sort of see this like the "unsigned" workflow, where the user is given > a warning, and can see a list of what's unsigned and decide if to continue > or not. Perhaps if shown a list of bundles that would not run with their > current VM, they might be able to decide if it was a minor case they didn't > care about, cancel out of it all, or go ahead and update, and then, after > not restarting, change their VM. > What you describe here sounds very similar to the remediation page that currently pops up in case of conflict. My concern would be addressed if that page also popped up in case of an EE conflict. The user would then be made aware of the problem and could be given an option to override the conflict.
(In reply to Thomas Hallgren from comment #25) > This is actually a good argument for my suggested approach. If p2 will not > install something that is incompatible with the current os/ws/archicture > (and it doesn't), then why install something that is incompatible with the > EE? I don't think we disagree on the real issue here, but wanted to make sure my point was clear. Just as you can use p2 on Linux to assemble a product for Windows (which for sure won't run on Linux) I'd hate to think I would have to increase the Java Version used in a build to the "maximum" used in a stack of plugins in order to assemble them all. (but, yes, if there was some explicit filter so a Java 6 build, could specify to include things up to "maximum" EE, then that's similar to the arch/ws/os filters. (But, it is hard for me to imagine a case where it would ever be important/desired to specify anything other than the maximum ... for builds ... but, the case you are rightfully concerned about is 'update'. Much different. And, yes, the "remediation page" is a better analogy of what to present to user, and letting them decide appropriate action. Interestingly, this problem will likely get "worse and worse" as more and more versions of Java come out, and as current "community of committers" are more and more willing to "up" the minimum EE required. Out of curiosity, can you give specifics of what was updated to what, that causes problems? I'm especially wondering if it was "across releases", such as "Juno to Luna" (which, we may be too permissive of allowing that, which can be restricted, with p2) ... or was it something smaller, like a "service release"? (which, would seem like a bad bug, to me). And, if it's a particular package, JEE? CDT? RAP RCP? it would be good if those packages get the feedback of one consequence of their "increased minimum constraints".
(In reply to David Williams from comment #26) > Out of curiosity, can you give specifics of what was updated to what, that > causes problems? I'm an employee at Puppet Labs nowadays and developing their Eclipse/Xtext based IDE named Geppetto. The problems I'm currently faced with are related to its features, not to the Eclipse release train or any particular Eclipse package. I totally agree though, the problem is likely to get worse as projects decide to take advantage of the cool features introduced in Java 1.7 and 1.8. It's probably a good idea to have a mitigation plan in place before it's time for the Mars release. Not so good if this becomes a blocker for projects that wants to move forward.
(In reply to comment #16) > I think the solution is simple. Produce the same kind of error that we do when > there's a conflict of some kind and refuse to continue. That will give the user > a hint that the JRE must be upgraded for the install to succeed. I agree. The core of the solution to this issue is to make the p2 resolver take execution environments into account. I can think of the following options: * We could encode the execution environment constraints as "requires" in the p2 metadata. In this way, the p2 resolver would be able to report the root cause of a resolution error, like "bundle xxx requires osgi.ee/JavaSE/1.6 but it could not be found". The implementation of this approach is related to bug 313553. It may be challenging because I am unsure if it is possible at all to express OR-requirements in the p2 metadata, i.e. when a bundle requires JavaSE-1.6 or JavaSE/compact1-1.8. Without changing the p2 metadata format, I can only think of an implementation with synthetic IUs (e.g. requires="JavaSE-1.6", provides="JavaSE-1.6-OR-JavaSE/compact1-1.8") * Another option is to encode the EE constraints as filters, similar to the platform filters. This would be straightforward, but the resolver would only find out _that_ a bundle requiring a higher EE is missing, and not _why_ it is missing. Also this implementation would be special to execution environments, and could not be applied to other Require-Capabilities. Once we have either of the solutions above, we'd need to make sure that the p2 profile (=installation metadata) either provides the JRE execution environment capabilities or sets the appropriate filter properties. As stated by Thomas in comment 21, p2 would need to check e.g. when triggering a new installation if these capabilities/properties still apply, or if the user is running his installation with a newer JRE in the meanwhile. Remediation support (comment #25), offering the option to use a newer JRE would be nice but IMHO optional. (In reply to comment #24) > Don't forget to work though the various "build" and "packaging" scenarios. Don't worry. Tycho already has more support for execution environment constraints than p2 alone [1], and your favorite setting to disregard them completely during the build is coming with the next release (bug 413116). Whatever will be implemented in p2 for this bug, it should be easy to adapt Tycho to do technically the same - but with its flexible configuration options. [1] http://wiki.eclipse.org/Tycho/Execution_Environments
(In reply to Tobias Oberlies from comment #28) > ... It may be challenging because I am unsure if it is possible > at all to express OR-requirements in the p2 metadata, i.e. when a bundle > requires JavaSE-1.6 or JavaSE/compact1-1.8. Without changing the p2 metadata > format, I can only think of an implementation with synthetic IUs (e.g. > requires="JavaSE-1.6", provides="JavaSE-1.6-OR-JavaSE/compact1-1.8") > It is possible. A requirement can be written as a p2 IMatchExpression which is quite powerful.
(In reply to comment #29) > It is possible. A requirement can be written as a p2 IMatchExpression which is > quite powerful. Great! I wasn't aware of this. Then this should in fact not be too hard to implement. Is there an example where these kind of expressions are written/published?
(In reply to Tobias Oberlies from comment #30) > Is there an example where these kind of expressions are written/published? Not out there in the wild (not that I can find anyway). But they are used internally. Here's one example: https://git.eclipse.org/c/equinox/rt.equinox.p2.git/tree/bundles/org.eclipse.equinox.p2.tests/src/org/eclipse/equinox/p2/tests/planner/ORTesting.java You will see how it's used in profile change requests if you navigate down to MetadataFactory.createRequirement(IMatchExpression...) and then do "Open call hierarchy" from there. There is an XML representation for such requirements as well where the attributes match and matchParameters can be used instead of the namespace, name, and range.
In Geppetto we mitigate this problem with a bundle that uses the org.eclipse.ui.startup extension point to pop up an error dialog that informs the user that some features have been disabled due to JVM incompatibilities. The code for this bundle can be viewed here: https://github.com/puppetlabs/geppetto/tree/master/com.puppetlabs.geppetto.startup
The problem here lies in the absence of representation in p2 of what is outside of p2's control. This includes things such as the JRE, packages exported by the app server (pb with the servlet bridge), OS version, etc. So far p2 has represented the JRE by the (in)famous a.jre IUs but those are problematic because they are not bound to the actual JRE running. They are just pure metadata that can be brought in by anybody as soon as they are made available from a repo. For example if tomorrow I had dependencies against Java 9 packages I could just create a a.jre IU for Java 9 and add it to my repo, and it would be provisioned thus satisfying the dependencies of my bundle but it would never cause Java 9 to be installed. A solution to this, which would also help in a number of other cases (such as servlet bridge, eclipse on fedora or equinox servlet bridge), is to have a representation of the outside-of-p2-control-bits be represented as provided capabilities only (not as IU, since IUs can be replaced) and have those capabilities be stored in the profile. As for how those capabilities come to life, I'm thinking of a model where the profile has capatiliby-providers (or system-capability-sniffer) whose responsibility is to explore the system on well-known topics and return corresponding capabilities. For example a JRE capability provider would return a set of capabilities for the JRE at hand. For the case where p2 provisions installs that are not being run (like at build time), I think we could use an approach where capabilities otherwise returned by the sniffers would be passed-in by the director or just rely on those that are already stored in the profile. Finally for the runtime behaviour, I think the sniffers should be evaluated when the profile is loaded and probably at most once per eclipse session.
(In reply to comment #33) > A solution to this, which would also help in a number of other cases (such as > servlet bridge, eclipse on fedora or equinox servlet bridge), is to have a > representation of the outside-of-p2-control-bits be represented as provided > capabilities only (not as IU, since IUs can be replaced) and have those > capabilities be stored in the profile. The approach to store capability sets outside of IUs is very problematic for build systems because every one of these would somehow need to be built into the build system. This may work for execution environments, but will certainly not scale for the other use cases you have mentioned. So I am strongly in favor to keep the system capability IUs like the a.jre IUs, and ensure that only the right one of these are installed through other means. The easiest way that I can think of is to just make the a.jre IUs require a profile property and simply make p2 automatically set/unset these properties according to the running system.
(In reply to Tobias Oberlies from comment #34) > (In reply to comment #33) > > A solution to this, which would also help in a number of other cases (such as > > servlet bridge, eclipse on fedora or equinox servlet bridge), is to have a > > representation of the outside-of-p2-control-bits be represented as provided > > capabilities only (not as IU, since IUs can be replaced) and have those > > capabilities be stored in the profile. > > The approach to store capability sets outside of IUs is very problematic for > build systems because every one of these would somehow need to be built into > the build system. This may work for execution environments, but will > certainly not scale for the other use cases you have mentioned. I agree that having build systems know about this is not an option. I think of this additional information as being part of the "environment". For example today we allow the user to say "I expect the os to be linux and the windowing system to be gtk", so saying "I expect Java 8" or "package javafx" are the same in that they define the environment in which the application is expected to run / be provisioned. So for example we could imagine adding to tycho a mechanism that let the user describe in the environment section additional capabilities it expects either in the form of just a couple packages, and/or as an IU/repo like it is done in other places of tycho (e.g. eclipserun application).
(In reply to comment #35) > So for example we could imagine adding to tycho a mechanism that let the user > describe in the environment section additional capabilities it expects (...) as an > IU/repo This is how Tycho works today: If you select and executionEnvironment which is not one of the standard ones, Tycho pulls the information about the execution environment from an "a.jre" IU. The important point is that the environment definition comes from a p2 repository. We had the whole discussion about where to store re-usable environment definitions when we implemented custom profile support in Tycho (bug 385930), and the only good solution was p2 repositories. We could also have stored the environment definition as Maven artifact, but this would have meant that we would have forced Tycho users to host their own Maven repository infrastructure - something which is not required as of today.
With modular JVMs which no longer mandate all packages possible for given Java version is available this becomes more important work.
Is this issue dead?
With Eclipse shipping with its own JRE, this feature request becomes less relevant since the IU of the actual JRE installed will be able to have a proper list of provided packages. Therefore the only cases where this could still be relevant are installation where a default JRE / JDK is provided by the system.
The OSGi framework will soon be able to describe all JRE modules at runtime as a special case of bundle, so it will be beneficial for p2 to do the same at provisioning time. There is a new use case where the goal is to jlink a minimal JRE based on the requirements of other UIs (e.g. OSGi bundles). Again OSGi R8 allows bundles to import java.* packages which will allow me to select the set of modules that my bundles need. Finally Eclipse is not the only user of p2 and the case is very relevant to the rest of us who do not always ship a JRE. Even when a JRE is shipped there is a need (as per the use cases above) for a JRe publisher that will scan the JRE and publish the java modules as IUs.
I have done work on a JRE publisher that scans a Java 9+ JRE and hope to contribute it at some point, so the issue is not dead. However I can't give a timeline on this.
Thanks for the update Todor
This would need to be an option for the ProfileChangeRequest
Is the a Java 8 compatible way to get Java packages on Java 9+ runtime? For Java 9+, I found `ModuleLayer.boot().modules().stream().flatMap(module -> module.getPackages().stream()).toArray()` does the job, but it uses APIs that aren't available in Java 8. Is there another possible approach to list all system packages in Java 8?
(In reply to Mickael Istria from comment #44) > Is the a Java 8 compatible way to get Java packages on Java 9+ runtime? > For Java 9+, I found `ModuleLayer.boot().modules().stream().flatMap(module > -> module.getPackages().stream()).toArray()` does the job, but it uses APIs > that aren't available in Java 8. > Is there another possible approach to list all system packages in Java 8? In the framework I resort to using reflection to do this in org.eclipse.osgi.storage.Storage.calculateVMPackages() https://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/bundles/org.eclipse.osgi/container/src/org/eclipse/osgi/storage/Storage.java#n1839 Eventually we will be able to use Java 11 (in 2020-09), but it is not clear to me that all parts of p2/equinox should freely move to Java 11 API if we want them to be usable on Java 8, which has extended life until 2030. For the time being you will need to use reflection. I will point out that your collection of the packages is not correct because it will include private packages and packages that are targeted to specific modules. As a result p2 will allow an import to get resolved to the JRE package, but at runtime the framework will not provide such a system package export for the private packages.
(In reply to Thomas Watson from comment #45) > Eventually we will be able to use Java 11 (in 2020-09), but it is not clear > to me that all parts of p2/equinox should freely move to Java 11 API if we > want them to be usable on Java 8, which has extended life until 2030. For > the time being you will need to use reflection. > While I about agree about OSGi framework implementation in Equinox I fail to see why p2 can not move on.
(In reply to Alexander Kurtakov from comment #46) > (In reply to Thomas Watson from comment #45) > > Eventually we will be able to use Java 11 (in 2020-09), but it is not clear > > to me that all parts of p2/equinox should freely move to Java 11 API if we > > want them to be usable on Java 8, which has extended life until 2030. For > > the time being you will need to use reflection. > > > While I about agree about OSGi framework implementation in Equinox I fail to > see why p2 can not move on. I refuse to hold you back :) I'm no p2 expert, but I was under the impression that p2 can be used outside the context of a fully running Eclipse IDE (help and all). It would seem that some users may want that to continue running on Java 8 for that.
(In reply to Thomas Watson from comment #47) > (In reply to Alexander Kurtakov from comment #46) > > (In reply to Thomas Watson from comment #45) > > > Eventually we will be able to use Java 11 (in 2020-09), but it is not clear > > > to me that all parts of p2/equinox should freely move to Java 11 API if we > > > want them to be usable on Java 8, which has extended life until 2030. For > > > the time being you will need to use reflection. > > > > > While I about agree about OSGi framework implementation in Equinox I fail to > > see why p2 can not move on. > > I refuse to hold you back :) > > I'm no p2 expert, but I was under the impression that p2 can be used outside > the context of a fully running Eclipse IDE (help and all). It would seem > that some users may want that to continue running on Java 8 for that. That's true but in theory. In practice I'm aware of only Tycho.
(In reply to Thomas Watson from comment #45) > In the framework I resort to using reflection to do this in > org.eclipse.osgi.storage.Storage.calculateVMPackages() > https://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/bundles/org. > eclipse.osgi/container/src/org/eclipse/osgi/storage/Storage.java#n1839 Thanks! But does this work when running on Java 8? The code seems to return an empty set in that case, is that so?
In contexts like the installer, where one specifies a JRE/JDK to be used for the installer (different from the one running the installer) one would need something that can return the list of packages of that JRE/JDK. Of course one could run some library/jar in that JRE, i.e., launch to compute results, as Thomas suggests is possible. We already do stuff like that in Oomph to determine the version of the JRE, but something that runs on Java 8 would be needed... This Bugzilla is not a place for sociopolitical discussions, but I expect we will kill off another portion of our consumer base (freeloads as they may be) by shoving Java 11 down everyone's throats. :-(
(In reply to Mickael Istria from comment #49) > Thanks! > But does this work when running on Java 8? The code seems to return an empty > set in that case, is that so? Ok, got it: for Java 8 and lower, we use the .profile like the JREAction does to generate the IU for the current Execution Environment and resolve against it, for further versions, we build the IU and package capabilities by introspecting the modules (using reflection code).
(In reply to Mickael Istria from comment #51) > (In reply to Mickael Istria from comment #49) > > Thanks! > > But does this work when running on Java 8? The code seems to return an empty > > set in that case, is that so? > > Ok, got it: for Java 8 and lower, we use the .profile like the JREAction > does to generate the IU for the current Execution Environment and resolve > against it, for further versions, we build the IU and package capabilities > by introspecting the modules (using reflection code). Right, I should have been clear on that. When running on Java 8 we always resorted on hard coded lists of packages. Finally, in Java 9 we got API to discover it.
