Community
Participate
Working Groups
The various Guava versions are not compatible therefore com.google.guava MUST be a singleton.
Ed, can you help me to better understand this requirement? What are the compatibility issues? Can the consumer define version ranges to only import compatible versions? I'm afraid that making this bundle a singleton will prevent running different versions in parallel, which is a supported use-case under OSGi.
The problem is that Guava is useful so many projects use it has clever reflection so incompatibilities appear only at run-time has a highly unstable API (at least compared to Eclipse stability) I am forever encountering problems with the left hand (e.g. Xtext) having loaded version X and then bombing because the right hand (e.g Papyrus) wants version Y but finds that the loader already has an X.
A specific example: xtext.common.types.ui 2.5.0 has a version 10 only dependency (fixed in 2.5.1). Papyrus 1.0.0M5 has a >= version 11 dependency. Papyrus uses Xtext, but Eclipse does not detect that Papyrus has a dependency conflict; it is an obscure downstream ClassLoader problem. If guava was a singleton, this conflict would be visible to the Papyrus developers.
Is it possible to re-produce the issue without Xtext and Papyrus, i.e. with just two small bundles? I'd like to understand the root cause. It could be missing usage clauses on the exported packages of the Guava bundle. I doubt that marking the bundle as singleton will solve the issue. It will mask the issue but may result into other obscure unresolved or not-starting bundle errors.
(In reply to Gunnar Wagenknecht from comment #4) > Is it possible to re-produce the issue without Xtext and Papyrus, Sorry, no idea. I find the whole thing mysterious in particular my recent Xtext/Papyrus conflict only arose after uninstalling an re-installing Papyrus. But fundamentally, I thought the whole principle of singletons was that any given class loader could only even think about 1 version. Currently we allow it to try and fail. My best guess is that because each plugin has its own class loader, each can make its own chouice about which Guava to use. The problem arises when a Guava class is passed between plugins. My most recent problem was with Bug 427861 where com/google/common/base/Optional was the wrong one. --- Yes making Guava a singleton will be painful, but better for committers to have P2-time pain than users to have run-time pain. That's what singletons are for. Do it in good time for M6.
Sorry but we can't make that bundle a singleton. Guava is such a fundamental library that it must be able run in multiple versions on the framework and it does run in multiple versions in parallel. Having said that, I'm here to spent time analyzing the problem and improve the bundle quality. The problem you linked looks like a specific setup issue that (according to your comment) solved itself by re-installing. It might be related to missing or wrong metadata in Guava. But very likely it can also be a problem with the metadata of the consumers.
Java doesn't allow multiple Java's to co-exist; a rela pain that Eclipse doesn't enforce this better. Eclipse doesn't allow most Eclipse plugins to co-exist. Why should Guava be allowed to? Or rather if Guava is allowed to, it must be prohibited fropm participating in API interfaces. Guava is such a fundamental library that we must ensure that it works reliably. My suggestion on cross-project dev, was that all plugins must include Vx in their accepted range and EPP should ship Vx. Users may try something different iff all the plugins they use have open enough ranges. I leave the choice of Vx to consensus. I'll start the discussion at 12, where Java 6 kicked in. --- You mis-read; re-installing created the problem!
Had some more pain. My MWE2 scripts refused to run aftef Payrus was installed. It seems to me that the problem is: Team TA ship a plugin PA built with Guava version GA. Team TB ship a plugin PB built with Guava version GB. An application uses both PA and PB. When some Guava object is passed between PA and PB, we hit a problem if that Guava class has had any method added or removed between GA ang GB. The PB class loader reports that the incoming GA is a bad class if the GB version has already been loaded. So even if PA works with GB and PB works with GA, the incompatible builds cause a run-time failure.
FWIW, we ran into issues with different guava versions being available some time ago too. We solved our issues by using "uses" directives in our manifests and strictly requiring package imports to be in a certain range. Not all Guava packages prior version 15 where exported with a package version but AFAICT Guava 15 (now in Orbit) fixes this. IMHO: the root cause of your problems are unconstrained imports (and exports) of bundles depending on Guava. It's in the responsibility of the plugin providers (xtext, papyrus etc.) to ensure their compatibility with each other. I agree with Gunnar that restricting the whole release train to use Guava in a certain version is no option.
(In reply to Marcel Bruch from comment #9) > FWIW, Worth a lot. > we ran into issues with different guava versions being available some time > ago too. We solved our issues by using "uses" directives in our manifests > and strictly requiring package imports to be in a certain range. Not all > Guava packages prior version 15 where exported with a package version but > AFAICT Guava 15 (now in Orbit) fixes this. Can you be clearer please? What is "uses"? I'm only familiar with "RequireBundle" and "ImportPackage". What does "strictly requiring mean", who/what is requiring what where? Seems like we should all make a big effort to move to Guava 15.
(In reply to Ed Willink from comment #10) > Can you be clearer please? What is "uses"? This may be a good resource: http://spring.io/blog/2008/10/20/understanding-the-osgi-uses-directive/ > I'm only familiar with > "RequireBundle" and "ImportPackage". What does "strictly requiring mean", We use package imports only and ever package import is qualified with a version range [14.0.0,15.0.0) > who/what is requiring what where? Maybe this manifest.mf helps to shed some light on it: http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.utils/META-INF/MANIFEST.MF
Thanks Marcel for the additional info. I'm not sure if the Orbit Guava bundle export packages with the proper uses constraints. But I was suspecting that this is part of the problem.
(In reply to Gunnar Wagenknecht from comment #12) > Thanks Marcel for the additional info. I'm not sure if the Orbit Guava > bundle export packages with the proper uses constraints. But I was > suspecting that this is part of the problem. FYI, Guava contains proper uses directives from version 12 on (generated with the bundle-maven-plugin). Versions 10 and 11 (which are in Orbit) do *not* contain proper uses directives. Luckily, both Guava 12 and 15 are in Orbit now. One thing to be aware of, however, is that the bundle org.eclipse.m2e.maven.runtime 1.5.0 exports Guava packages in version 14 *without* proper uses directives. So, using Import-Package: com.google.common.base;version="[14.0.0,15.0.0)" *might* cause problems when another Guava 14 bundle is around. (This is one of the reasons Code Recommenders moves to Guava 15.)
Thanks Andreas for confirming this. Given that we do have recent versions with usage constraints in Orbit and we do not want to make this bundle a singleton, I'm closing this as WONTFIX.
(In reply to Gunnar Wagenknecht from comment #14) > Thanks Andreas for confirming this. > > Given that we do have recent versions with usage constraints in Orbit and we > do not want to make this bundle a singleton, I'm closing this as WONTFIX. Should we ask on cross-project, if anyone NEEDS older one, for Luna, and if not aggressively remove them? That is, is there still risk of "interference", or if someone encounters interference, the burden is on them to move up to most recent? (Sorry if obvious, I'm just skim reading)
(In reply to David Williams from comment #15) > Should we ask on cross-project, if anyone NEEDS older one, for Luna, and if > not aggressively remove them? That is, is there still risk of > "interference", or if someone encounters interference, the burden is on them > to move up to most recent? > > (Sorry if obvious, I'm just skim reading) Not obvious. There is a clear problem. Guava needs to be a singleton but making it one is too frightening. Therefore we need to approximate a singleton by as many methods as possible. - Just one version in Orbit would be really good. Fixes an all-Eclipse installation. - Plugins re-exporting just that one major version range would be good. Avoids goodies from third parties causing chaos.
(In reply to Ed Willink from comment #16) > Guava needs to be a singleton but making it one is too frightening. > Therefore we need to approximate a singleton by as many methods as possible. Pretty sure this has already been explained already ... which I got from skim reading :) Something does not need to be a singleton just because it has its own class loader. Only needs to be a singleton if it is a "front end" to a resource (or class) for which itself there should just be one of, such as "Locale", or "Plugin Extensions Registry".
A new scenario appears via Bug 428921. Consider a (non-Guava) plugin (e.g. JDT UI) that provides an extension point. The extension by Application A causes Guava class G version A to be loaded by JDT UI. (e.g. an operation signature mentions a Guava class G). The extension by Application B requires Guava class G version B to be loaded by JDT UI. Oops this fails because version A is already loaded. (In reply to David Williams from comment #17) > Only needs to be a singleton if it is a "front end" to a resource (or class) > for which itself there should just be one of, such as "Locale", or "Plugin > Extensions Registry". Therefore Guava is a front end to its own classes for which there must be only one version, if we are to avoid conflicts between extension point extenders. Reopening because, while we might get away without making Guava a singleton, we must have - only one Guava version in Orbit - or some other way of guaranteeing that all Simultaneous Release participants use only one Guava version - or some way of prohibiting Guava references in APIs Just one Guava version in Orbit should be easy. David's release scripts could verify that all participants used an agreed version. Prohibiting Guava in APIs seems unrealistic.
IMHO you should get all users of Guava on this bug (Xtext, Xtend, Papyrus, ...). Otherwise they won't be aware of this discussion going on and we don't get any step further.
(In reply to Ed Willink from comment #18) > A new scenario appears via Bug 428921. This specific issue happened because we had inconsistent guava dependencies in Xtext 2.5.0 (different version constraints). It won't happen with later versions. Still some Xtext bundles reexport guava, as there are guava types in API signatures. So the potential conflict described by Ed could happen is some other bundle reexports a different version.
We also run into trouble with Xtext/Xtend reexporting Guava some time ago. Do you see any chance that we coordinate upgrading to the same Guava versions, say every 3-6 months (if anyone requests a newer version)? AFAICT this affects Xtext, Xtend, some other modeling projects, Mylyn, and Code Recommenders. Additional projects may be there too.
(In reply to Marcel Bruch from comment #21) > Do you see any chance that we coordinate upgrading to the same Guava > versions, say every 3-6 months (if anyone requests a newer version)? Sure, currently the versions constraint is [10.0.1,16.0.0). But we plan to change that to 12 minimum, as we also plan to end support for Java 5. I think providing a single version of guava in the release train as well as coordinating between the projects to ship the newest version available through orbit is the best we can do for now. For a possible future major release we would remove reexports and usage of guava in APIs.
(In reply to Ed Willink from comment #18) > - or some other way of guaranteeing that all Simultaneous Release > participants use only one Guava version (In reply to Marcel Bruch from comment #21) > Do you see any chance that we coordinate upgrading to the same Guava > versions, say every 3-6 months (if anyone requests a newer version)? The policy of "guaranteeing that all Simultaneous Release participants use only one Guava version" together with a legitimate desire to use a reasonably recent version of Guava may have unintended side-effects: As the Guava API may change between major versions (which are released every 6 months), the proposed policy may force Eclipse projects to maintain different code bases for Simultaneous Release N and Simultaneous Release N+1. A real-world example: In our case (Eclipse Code Recommenders), we maintain a single codebase that works with Kepler and will work with Luna. However, if the "one Guava per Simultaneous Release" were enforced we will have to maintain some compatibility fragments merely to cover up differences in Guava's API between the version picked for Kepler and Luna (the LoadingCache classes changed, for example). Note that this problem is there regardless of whether Guava is part of Code Recommenders public API or used only internally (like the cache classes). If Orbit's Guava were a singleton, we are forced to maintain multiple codebases, one for each Simultaneous Release we still want to support with the latest version of Code Recommenders. (Of course, this policy doesn't apply to Kepler and Luna yet; this is merely an example of the problems the policy may cause.)
(In reply to Marcel Bruch from comment #21) > Do you see any chance that we coordinate upgrading to the same Guava > versions, say every 3-6 months (if anyone requests a newer version)? Informal co-ordination between long-standing projects is a bit fragile but could work. However new incubation projects tend to be more innovative in their dependencies and less experienced in their release engineering. We need a policy that will inherently work.
Within Papyrus, we started developing with Guava 10. In Guava 11, some methods became deprecated (Actually, moved to another class), and in Guava 12, they were removed (And still present in the other class). We tried to use only a common subset of Guava, and avoid strong version dependency, but when a method is moved, we need to choose either one or the other (Or reimplement an equivalent method in Papyrus...) So now, Papyrus relies on Guava 11, and doesn't use any deprecated API. This works fine at least until Guava 14; I haven't tried with Guava >= 15. However, at runtime, we sometimes (often?) have one of our dependencies which loads Guava 10 (XText, or even lower-level components; not sure which ones). And at this point, everything becomes really chaotic. I agree for one of these solutions: - Guava >= 12 for everyone (I'm not sure whether there are as many API conflicts between Guava 12 and 15 as there were between Guava 10 and Guava 12) - A single instance of Guava in Orbit For Papyrus at least, this should be fine.
(In reply to Camille Letavernier from comment #25) > I agree for one of these solutions: > > - Guava >= 12 for everyone (I'm not sure whether there are as many API > conflicts between Guava 12 and 15 as there were between Guava 10 and Guava > 12) +1 Guava 12+ has proper uses directives and we at Code Recommenders haven't had any issues with it (or rather Guava 14, using Import-Package). > - A single instance of Guava in Orbit -1 If I want to depend on Guava and it is not even part of my public API, why should I not be free to use whatever Guava version I want (provided its manifest is in order, which is the case for Guava 12+)?
(In reply to Andreas Sewe from comment #26) > > - Guava >= 12 for everyone A multi-version range does not avoid the class loader conflict. > > - A single instance of Guava in Orbit > > If I want to depend on Guava and it is not even part of my public API, why > should I not be free to use whatever Guava version I want (provided its > manifest is in order, which is the case for Guava 12+)? Reasonable, which means that we need to 'enforce' re-exporting only an agreed single major version. But how do we solve the problem with downstream consumers installing alternative versions of Guava and recreating this chaos for themselves? Can P2 / ManifestBuilder infer a per-extension point permitted version so that third parties are aware that their usage introduces a conflict? How could this inference work if not all Simultaneous Release plugins are installed? It seems to me that the hazards and workarounds that allow Guava to be a non-singleton are much worse than the reasonable discipline of using a singleton. Can we have both singletons and non-singletons in Orbit? Re-exporting plugins use the singleton. Internal-only applications can use whatever version they like?
I feel your pain, I really do. But the solution is absolutely NOT to make Guava a singleton, in fact that would cause much more harm. The solution is to use version ranges properly. Unfortunately Google does not apply proper semantic versioning to its exports so this is harder than it should be. Anyhow, I'm not even sure why this bug is raised here. Guava is not an Eclipse project, the fix would need to be supplied by Google, so raise a bug on their site. However, I would again reiterate that the fix is not to make Guava a singleton. The fix I would like to request is that the Guava developers go and read about semantic versioning, e.g. at semver.org or the OSGi whitepaper on the subject. I know already that they have rejected the idea, however. They want their versions to be "simple", which really means simple for *them* but not for their users. Le sigh.
What about rebuilding the versions of guava on orbit and add proper versions (ie semantic versions) on their exported packages? The bundle version would stay identical to the guava release's one, but at least we will be able to rely on import package to create a valid configuration for a simrel.
Hi All, I'll keep that bug a WONTFIX. It may even be a NOT_ECLIPSE as Neil suggested. Sorry folks, but we can't make *any* bundle in Orbit a singleton just because some projects don't get their versioning right. I totally admit that this is a brutal case of a 3rd-party library using counters as versions. As for shipping a single version of Guava in the release train, I suggest raising that issue with your PMC's planning council representative. AFAIK we do not like shipping different versions in the release train and there are activities to reduce them. But this requires buy-in from every project participating in the release train. It may be possible to make this a strict rule that all projects in the release train repo must follow. But this is up to the planning council to decide. Thanks! -Gunnar
Hi Gunnar, What about my proposal to version guava properly on Orbit? Should I raise a bug against orbit?
Even that's potentially problematic because I know people have asked for package imports instead of bundle imports so they can resolve them against whatever bundles they ultimately want to deploy. If we depart from the version numbers provided by Guava itself, which does distribute real OSGi bundles, albeit not proper semantically version, that seems like another source of problems or certainly cause for complaints, i.e., why must I use what's redistributed by Eclipse?
(In reply to Gunnar Wagenknecht from comment #30) > I'll keep that bug a WONTFIX. It may even be a NOT_ECLIPSE as Neil > suggested. Gunnar: This is a genuine problem with options under active discussion. Surely it cannot yet be WONTFIX? > As for shipping a single version of Guava in the release train, Perhaps the Bug needs moving to Cross-Project if the solution has to be a cross-project policy rather than an Orbit change. (In reply to Ed Willink from comment #27) > (In reply to Andreas Sewe from comment #26) > > If I want to depend on Guava and it is not even part of my public API, why > > should I not be free to use whatever Guava version I want (provided its > > manifest is in order, which is the case for Guava 12+)? > > Reasonable, which means that we need to 'enforce' re-exporting only an > agreed single major version. Is it reasonable? If any implemented extension point is capable of invoking code that invokes your 'internal' Guava is it sufficiently 'internal'? It seems that for the Simultaneous Release we must enforce a single version. For third parties, we need to produce a Manifest warning if a different Guava version is required (? and if any transitive plugin dependency has extension point implementations). Otherwise the developers of WizzyEditor will have no clue that their wonderful product can cause chaos when used with CoolStudio. Ultimately this will lead to a lot of inaccurate finger pointing as to which product is buggy.
Ed, so I'll move the bug then.
Ed, please update the bug summary to express the problem to solve. I've made a first guess.
I don't understand how semantic versioning the packages would help. The conflict as I understood is the following: We have two bundles A and B which use different versions of guava. Those versions are reexported. Bundle C which uses both A and B runs into runtime issues because of mismatching types, etc. If bundle A and B were using package exports and proper uses clauses, than bundle C still couldn't solve that issue, could it? Or is it just that these problems would occur less often as we now have more fine grained units?
(In reply to Mikael Barbero from comment #31) > Hi Gunnar, > > What about my proposal to version guava properly on Orbit? Should I raise a > bug against orbit? As others have implied, we in Orbit don't fix others bugs ... we simply provide OSGi versions of their third party bundles. If the third party has an bug (or less than ideal code) you need to work with them to fix.
(In reply to Sven Efftinge from comment #36) > I don't understand how semantic versioning the packages would help. > > The conflict as I understood is the following: > > We have two bundles A and B which use different versions of guava. > Those versions are reexported. > Bundle C which uses both A and B runs into runtime issues because of > mismatching types, etc. > If bundle A and B were using package exports and proper uses clauses, than > bundle C still couldn't solve that issue, could it? > > Or is it just that these problems would occur less often as we now have more > fine grained units? It will occurs less often but even more important, it could be detected during the development/TP validation/provisoning, and not only at runtime. That is a big win.
David, that's fine to me. I was trying to find alternative solutions.
(In reply to Ed Merks from comment #32) > Even that's potentially problematic because I know people have asked for > package imports instead of bundle imports so they can resolve them against > whatever bundles they ultimately want to deploy. If we depart from the > version numbers provided by Guava itself, which does distribute real OSGi > bundles, albeit not proper semantically version, that seems like another > source of problems or certainly cause for complaints, i.e., why must I use > what's redistributed by Eclipse? I understand your concerns and they are legitimate. I was actually quite active on the addition of OSGi manifest entries to guava (https://code.google.com/p/guava-libraries/issues/detail?id=688) and I'm ready to do the work to support semantic versions of exported packages of Guava (as soon as I figured out how to do that from the comments on my request for advice on OSGi https://www.osgi.org/bugzilla/show_bug.cgi?id=161). It will not change anything about this bug, but at least will reduce the pain in the future. Does it sound more reasonable?
(In reply to Mikael Barbero from comment #38) > It will occurs less often but even more important, it could be detected > during the development/TP validation/provisoning, and not only at runtime. > That is a big win. That's what I thought, too. But at least with equinox and PDE there's neither a compile-time problem nor is anything at runtime failing before the linkage errors.
(In reply to Mikael Barbero from comment #40) > I was actually quite active on the addition of OSGi manifest entries to > guava (https://code.google.com/p/guava-libraries/issues/detail?id=688) and > I'm ready to do the work to support semantic versions of exported packages > of Guava (as soon as I figured out how to do that from the comments on my > request for advice on OSGi > https://www.osgi.org/bugzilla/show_bug.cgi?id=161). It will not change > anything about this bug, but at least will reduce the pain in the future. > Does it sound more reasonable? I'm not clear what you're suggesting. Given your greater insight can you outline your recommendation for a short term Luna solution and a long term Mars solution?
As I understand it, "using the latest version" in Orbit (version 15) helps solve the problem not merely because then everyone uses the same version, but also because it introduces the use of "uses clauses", which helps OSGi get all the right classes wired up correctly ... right? I am promoting an Orbit I-build now that will have "version 15" in it ... so would encourage everyone in Luna to see if you can move to that version ... but I _think_ that from then on, it should be not quite so sensitive to "exact version" -- but, I have only enough understanding of this issue to be dangerous, not authoritative (so, please, report the issues to the original third party, so they can improve their OSGi friendliness). = = = = = = = While a bit off-topic, does this issue also demonstrates the problems of exposing some "third party" code as part of your API ... once you do, its yours! Is this a case of people ignoring the "Law of Demeter" [1] (or, information hiding, or encapsulation)? And, I know it can not always be avoided ... having spent half my career dealing with "Xerxes compatibility problems" I understand its not easy to solve. [1] https://en.wikipedia.org/wiki/Law_of_Demeter
(In reply to Ed Willink from comment #42) > I'm not clear what you're suggesting. Given your greater insight can you > outline your recommendation for a short term Luna solution and a long term > Mars solution? I don't have any great idea. Pushing for a single 15.0 version is a real showstopper for people who still want to provide bundle with a Java 5 BREE (eg. EMF Compare) as Guava 12 and + are Java 6 only. That's why we (EMF Compare) are using Guava 11. We are using import packages with a broader range [11,15) because we tested that we are (source) compatible with all of these versions (we now have to test against v15 to broaden the range even more). Binary compatibility should be checked, but we did not had the time yet. And obviously we are very careful not to expose any guava class in our API. So maybe we a solution for Luna would to check every exporter of Guava'types in the simrel and check if we find an overlapping range and restrict Luna to that range. That would be better than a single version. For Mars, I hope that Java 5 compatibility will not be necessary anymore so that everyone could go to a very recent version of guava with proper semantic versions on exported packages ;)
m2e won't be able to move to guava 15 until this guava version works with sisu and maven (it currently does not, guava 14 is the highest we could get). It maybe possible to stop re-exporting guava packages from m2e maven runtime, but I will not have time to look into this in foreseeable future.
FYI https://code.google.com/p/guava-libraries/issues/detail?id=1682
(In reply to Igor Fedorenko from comment #45) > m2e won't be able to move to guava 15 until this guava version works with > sisu and maven (it currently does not, guava 14 is the highest we could > get). It maybe possible to stop re-exporting guava packages from m2e maven > runtime, but I will not have time to look into this in foreseeable future. So this may be the upper bound for the simrel, this is a start. Igor, do you know your lower bound?
(In reply to Mikael Barbero from comment #44) > Pushing for a single 15.0 version is a real > showstopper for people who still want to provide bundle with a Java 5 BREE > (eg. EMF Compare) as Guava 12 and + are Java 6 only. Java 5 BREE actually died about 3 release ago, but it is theOSGI migration to Java 6 that really drives it home. IMHO Java 5 should have no influence on this discussion. > So maybe we a solution for Luna would to check every exporter of Guava'types > in the simrel and check if we find an overlapping range and restrict Luna to > that range. That would be better than a single version. But it has to be a single version to avoid divergent class loading. Though certainly we need a simrel check to 'enforce' what we're not letting P2 enforce.
(In reply to Ed Willink from comment #48) > (In reply to Mikael Barbero from comment #44) > > Pushing for a single 15.0 version is a real > > showstopper for people who still want to provide bundle with a Java 5 BREE > > (eg. EMF Compare) as Guava 12 and + are Java 6 only. > > Java 5 BREE actually died about 3 release ago, but it is theOSGI migration > to Java 6 that really drives it home. IMHO Java 5 should have no influence > on this discussion. > > > So maybe we a solution for Luna would to check every exporter of Guava'types > > in the simrel and check if we find an overlapping range and restrict Luna to > > that range. That would be better than a single version. > > But it has to be a single version to avoid divergent class loading. Though > certainly we need a simrel check to 'enforce' what we're not letting P2 > enforce. Actually that's not true. You only need to maintain class loading consistency for types communicated between bundles. Keep guava types out of your exported interfaces and there is no issue. This is essentially what Mikael suggested.
(In reply to Igor Fedorenko from comment #45) > m2e won't be able to move to guava 15 until this guava version works with > sisu and maven (it currently does not, guava 14 is the highest we could > get). It maybe possible to stop re-exporting guava packages from m2e maven > runtime, but I will not have time to look into this in foreseeable future. Not re-exporting Guava 14 from org.eclipse.m2e.maven.runtime 1.5.0 would be great. While your runtime bundle properly versions all exported Guava packages, it unfortunately forgets to use "uses" directives. This has bitten Eclipse Code Recommenders in the past (and been one reason for us moving from Guava 14 to 15): What can happen is that you obtain package com.google.commons.collect from bundle org.eclipse.m2e.maven.runtime and com.google.commons.base from com.google.guava (or vice versa). This will then cause LinkageErrors at runtime, when your bundle, expecting a com.google.guava com.google.commons.base.Optional to be returned from com.google.commons.collect.Iterables.tryFind is faced with a org.eclipse.m2e.maven.runtime Optional. Uses directives (which the "real" Guava 14 employs) prevent this kind of faulty wiring. Shall I open a bug with m2e?
(In reply to Neil Bartlett from comment #49) > Keep guava types out of your exported interfaces and there is no issue. This might be the least bad solution, but it seems hard to expect programmer's to understand that they can pass Map but not MultiMap in a public API. But if API tooling gives a warning maybe we're home and dry.
FWIW, we have Guava classes in our public API and properly declare versioned package imports, exports and uses. We, however, are not the major bone of contention ;-) Anyways, I think we go around in circles. I'm trying to summarize the discussion (bear with me if I got a part wrong): If not needed, m2e should not export all Guava packages in their runtime - or at least version their exported guava packages as quickly as possible (whatever definition of quick is possible for m2e). If possible Xtend/Xtext should not enforce a particular Guava version and allow the runtime to select their own version (not sure that Xtext currently causes that kind of trouble atm). We, however, run into trouble when we were using these plugins in our code in the past until we clearly separated the data flow. Plugins that have Guava types in the public API should use uses directives to make the runtime aware of problems and help it to tie the best possible bundles together. All these things *do not prevent* us from having trouble but may reduce the amount of issues. Personally, I'd be happy if (i) any of the major eclipse projects (xtext, mylyn, me2,...) let others know on cross-project-issues when they update their Guava dependencies and (ii) as many as possible now upgrade to Guava 15.
(In reply to Mikael Barbero from comment #47) > (In reply to Igor Fedorenko from comment #45) > > m2e won't be able to move to guava 15 until this guava version works with > > sisu and maven (it currently does not, guava 14 is the highest we could > > get). It maybe possible to stop re-exporting guava packages from m2e maven > > runtime, but I will not have time to look into this in foreseeable future. > > So this may be the upper bound for the simrel, this is a start. Igor, do you > know your lower bound? m2e maven runtime comes bundled with guava 14.0.1, so there is no "lower bound". (In reply to Andreas Sewe from comment #50) > (In reply to Igor Fedorenko from comment #45) > > m2e won't be able to move to guava 15 until this guava version works with > > sisu and maven (it currently does not, guava 14 is the highest we could > > get). It maybe possible to stop re-exporting guava packages from m2e maven > > runtime, but I will not have time to look into this in foreseeable future. > > Not re-exporting Guava 14 from org.eclipse.m2e.maven.runtime 1.5.0 would be > great. While your runtime bundle properly versions all exported Guava > packages, it unfortunately forgets to use "uses" directives. > > This has bitten Eclipse Code Recommenders in the past (and been one reason > for us moving from Guava 14 to 15): What can happen is that you obtain > package com.google.commons.collect from bundle org.eclipse.m2e.maven.runtime > and com.google.commons.base from com.google.guava (or vice versa). This will > then cause LinkageErrors at runtime, when your bundle, expecting a > com.google.guava com.google.commons.base.Optional to be returned from > com.google.commons.collect.Iterables.tryFind is faced with a > org.eclipse.m2e.maven.runtime Optional. Uses directives (which the "real" > Guava 14 employs) prevent this kind of faulty wiring. > > Shall I open a bug with m2e? There bug 403243 already.
> > Not re-exporting Guava 14 from org.eclipse.m2e.maven.runtime 1.5.0 would be > > great. While your runtime bundle properly versions all exported Guava > > packages, it unfortunately forgets to use "uses" directives. > > > > This has bitten Eclipse Code Recommenders in the past (and been one reason > > for us moving from Guava 14 to 15): What can happen is that you obtain > > package com.google.commons.collect from bundle org.eclipse.m2e.maven.runtime > > and com.google.commons.base from com.google.guava (or vice versa). This will > > then cause LinkageErrors at runtime, when your bundle, expecting a > > com.google.guava com.google.commons.base.Optional to be returned from > > com.google.commons.collect.Iterables.tryFind is faced with a > > org.eclipse.m2e.maven.runtime Optional. Uses directives (which the "real" > > Guava 14 employs) prevent this kind of faulty wiring. > > > > Shall I open a bug with m2e? > > There bug 403243 already. Never mind, this is about Guice, not Gauava. Feel free to open a bug, but as I already said, I won't have time to look into this any time soon.
(In reply to Marcel Bruch from comment #52) > If not needed, m2e should not export all Guava packages in their runtime - > or at least version their exported guava packages as quickly as possible > (whatever definition of quick is possible for m2e). I actually think that m2e exporting guava packages is one source of problems, as some projects use import-package while others use require-bundle, the latter never resolve against m2e. So it would be great if m2e could stop doing that. > If possible Xtend/Xtext should not enforce a particular Guava version and > allow the runtime to select their own version (not sure that Xtext > currently causes that kind of trouble atm). Xtext uses require-bundle with a huge version range [10.0.1,16.0.0). And we tested against those versions. So I don't think there's a limitation here. The real bad thing we do is that we reexport that bundle, which in turn puts this dependency on all our clients. We would love to fix that, but it would break clients, so we can't do it without incrementing the major version number.
(In reply to Sven Efftinge from comment #55) > ... We would love to fix that, but it would > break clients, so we can't do it without incrementing the major version > number. My first reaction was "what's so bad with that"? :) Re-exporting a bundle is breaking other people so ... something has to give ... And, even knowing Neil Bartlett is reading this bug, :) I'll dare say, I think you can sometimes "bend the (semantics) rules" ... if you think it is the less evil, (i.e. less breaking) to increment the minor version and just make very clear to clients who use that bundle they don't need to change their code, but they will need to "require bundle" themselves. [And, that's just my personal opinion ... in your situation, with your clients, they might well prefer the "major" version change" as they'd be happy sticking with a lower level version of your code where they did not need to change a thing ... maybe you could poll a few?].
(In reply to David Williams from comment #56) > (In reply to Sven Efftinge from comment #55) > > ... We would love to fix that, but it would > > break clients, so we can't do it without incrementing the major version > > number. > > My first reaction was "what's so bad with that"? :) > Re-exporting a bundle is breaking other people so ... something has to give > ... > > And, even knowing Neil Bartlett is reading this bug, :) I'll dare say, I > think you can sometimes "bend the (semantics) rules" ... if you think it is > the less evil, (i.e. less breaking) to increment the minor version and just > make very clear to clients who use that bundle they don't need to change > their code, but they will need to "require bundle" themselves. No way. We cannot force everyone to update their manifests and recompile with a minor release. Bumping up the major version is also not something we would do lightly, because some Xtext bundles are singeltons which means that it's not possible to have multiple versions running in the same IDE. Finally, although a lot of people join the discussion on this bugzilla there are very few real problems happening and they are rare. Most of them were caused by a bug we have already fixed.
(In reply to Sven Efftinge from comment #57) > No way. We cannot force everyone to update their manifests and recompile > with a minor release. But it's new for Luna, so only affects Luna releases that have time to adapt. > Finally, although a lot of people join the discussion on this bugzilla there > are very few real problems happening and they are rare. Most of them were > caused by a bug we have already fixed. Rare because there are few Luna users and even fewer cross-project Luna users. Xtext by itself is fine, Xtext plus Papyrus is survivable. It's when I add Mylyn Wikitext that my nightmares really begin. First it was Guava 12 causing Java 5 failures, now its Guava 14 confusing everything else. Once Luna is in use by serious third parties, the rare may trend into "Luna doesn't work".
(In reply to Ed Willink from comment #58) > > Finally, although a lot of people join the discussion on this bugzilla there > > are very few real problems happening and they are rare. Most of them were > > caused by a bug we have already fixed. > > Rare because there are few Luna users and even fewer cross-project Luna > users. > > Xtext by itself is fine, Xtext plus Papyrus is survivable. It's when I add > Mylyn Wikitext that my nightmares really begin. First it was Guava 12 > causing Java 5 failures, now its Guava 14 confusing everything else. > > Once Luna is in use by serious third parties, the rare may trend into "Luna > doesn't work". Has anybody really understood what exactly causes the problems you have? I see many guesses in this bugzilla and even more possible solutions, but has anybody really debugged through a concrete scenario and can explain what causes the problem in detail? I would like to fully understand the problem before trying to fix it.
> Xtext by itself is fine, Xtext plus Papyrus is survivable. It's when I add Mylyn Wikitext that my nightmares really begin. First it was Guava 12 causing Java 5 failures, now its Guava 14 confusing everything else. One of the issues might be that org.eclipse.mylyn.docs.intent.markup (+ markup.gen) requires exactly Guava 11, and other Mylyn components require Guava >= 12. With Papyrus (Nightly) + XText (Nightly) + Mylyn (M5, Wikitext + Bugzilla task list), I always have this kind of exception: org.osgi.service.resolver.ResolutionException: Uses constraint violation. Unable to resolve resource org.eclipse.xtext.xtext.ui.graph [osgi.identity; osgi.identity="org.eclipse.xtext.xtext.ui.graph"; type="osgi.bundle"; version:Version="2.6.0.v201402250520"; singleton:="true"] because it is exposed to package 'javax.annotation' from resources org.eclipse.osgi [osgi.identity; osgi.identity="org.eclipse.osgi"; type="osgi.bundle"; version:Version="3.10.0.v20131210-2136"; singleton:="true"] and javax.annotation [osgi.identity; osgi.identity="javax.annotation"; type="osgi.bundle"; version:Version="1.1.0.v201209060031"] via two dependency chains. Chain 1: org.eclipse.xtext.xtext.ui.graph [osgi.identity; osgi.identity="org.eclipse.xtext.xtext.ui.graph"; type="osgi.bundle"; version:Version="2.6.0.v201402250520"; singleton:="true"] require: (&(osgi.wiring.bundle=org.eclipse.core.runtime)(bundle-version>=3.5.0)) | provide: osgi.wiring.bundle: org.eclipse.core.runtime org.eclipse.osgi [osgi.identity; osgi.identity="org.eclipse.osgi"; type="osgi.bundle"; version:Version="3.10.0.v20131210-2136"; singleton:="true"] Chain 2: org.eclipse.xtext.xtext.ui.graph [osgi.identity; osgi.identity="org.eclipse.xtext.xtext.ui.graph"; type="osgi.bundle"; version:Version="2.6.0.v201402250520"; singleton:="true"] require: (&(osgi.wiring.bundle=org.eclipse.xtext)(bundle-version>=1.0.0)) | provide: osgi.wiring.bundle; osgi.wiring.bundle="org.eclipse.xtext"; bundle-version:Version="2.6.0.v201402250520"; singleton:="true" com.google.guava [osgi.identity; osgi.identity="com.google.guava"; type="osgi.bundle"; version:Version="12.0.0.v201212092141"] import: (osgi.wiring.package=javax.annotation) | export: osgi.wiring.package: javax.annotation javax.annotation [osgi.identity; osgi.identity="javax.annotation"; type="osgi.bundle"; version:Version="1.1.0.v201209060031"]
Camille : Next Mylyn Intent milestone (M6) will require Guava >= 12 as all other Mylyn components
See Bug 429658 for an example of how 'helpful' releng can be really unhelpful: Installing Xtext explicitly installs Guava 10.
What's the best way for deploying Orbit bundles? I've read somewhere that we are supposed to embed a specific version of the plug-in in our own features. For Papyrus, we do that with Guava 11; it is included in one of our features. However, this results in Guava 11 being installed along with Papyrus, even if another version of Guava (> 11) is already available.
(In reply to Camille Letavernier from comment #63) > What's the best way for deploying Orbit bundles? > > I've read somewhere that we are supposed to embed a specific version of the > plug-in in our own features. > > For Papyrus, we do that with Guava 11; it is included in one of our > features. However, this results in Guava 11 being installed along with > Papyrus, even if another version of Guava (> 11) is already available. Include it in you "build" feature and import it in your SDK/Runtime feature
(In reply to Mikael Barbero from comment #44) > (In reply to Ed Willink from comment #42) > > I'm not clear what you're suggesting. Given your greater insight can you > > outline your recommendation for a short term Luna solution and a long term > > Mars solution? > > ... Pushing for a single 15.0 version is a real > showstopper for people who still want to provide bundle with a Java 5 BREE > (eg. EMF Compare) as Guava 12 and + are Java 6 only. > Where's your requirement for Java 5 BREE coming from? And, just because one 3rd party bundle uses 1.6, does not mean the rest of your BREEs have to change, and in fact, If you have clients that need 1.5, perhaps they are using it in such a way they don't need Guava? (I'm being hypothetical of course, not knowing much about your code or Guava ... ) So you currently use 11? Do you re-export it? (and if so, can you stop doing that!) Is it part of your API? (Granted, not time to stop doing that ... but ... as a general rule, 3rd party code should not be part of anyone's API).
(In reply to Camille Letavernier from comment #63) > What's the best way for deploying Orbit bundles? > > I've read somewhere that we are supposed to embed a specific version of the > plug-in in our own features. > > For Papyrus, we do that with Guava 11; it is included in one of our > features. However, this results in Guava 11 being installed along with > Papyrus, even if another version of Guava (> 11) is already available. Even if its installed, in a perfect world it would not be used at runtime. My guess is that it's a problem because Guava uses it own class loader, (instead of relying on OSGi). The whole point of "coordinated release" is that you'd include the higher/highest version in your feature, once a new one comes along. Have you tried that? Any reason not to "include" Guava 15 instead?
(In reply to Igor Fedorenko from comment #45) > m2e won't be able to move to guava 15 until this guava version works with > sisu and maven (it currently does not, guava 14 is the highest we could > get). It maybe possible to stop re-exporting guava packages from m2e maven > runtime, but I will not have time to look into this in foreseeable future. Not re-exporting is the key (as that is nearly always "harmful to others"). It's unfortunate you've not had time to look at it over the past several months, since now that there is only a week or two left, I'm sure you or no one on your team or community would either. Did you open a bug for this?
(In reply to Sven Efftinge from comment #57) > ... there > are very few real problems happening and they are rare. Most of them were > caused by a bug we have already fixed. What bug (number) was that?
(In reply to David Williams from comment #67) > (In reply to Igor Fedorenko from comment #45) > > m2e won't be able to move to guava 15 until this guava version works with > > sisu and maven (it currently does not, guava 14 is the highest we could > > get). It maybe possible to stop re-exporting guava packages from m2e maven > > runtime, but I will not have time to look into this in foreseeable future. > > Not re-exporting is the key (as that is nearly always "harmful to others"). > It's unfortunate you've not had time to look at it over the past several > months, since now that there is only a week or two left, I'm sure you or no > one on your team or community would either. Did you open a bug for this? m2e does not re-export guava packages, see bug 430714, and does not force installation of any specific guava version, see bug 435616.
(In reply to Igor Fedorenko from comment #69) > (In reply to David Williams from comment #67) > > (In reply to Igor Fedorenko from comment #45) > > > m2e won't be able to move to guava 15 until this guava version works with > > > sisu and maven (it currently does not, guava 14 is the highest we could > > > get). It maybe possible to stop re-exporting guava packages from m2e maven > > > runtime, but I will not have time to look into this in foreseeable future. > > > > Not re-exporting is the key (as that is nearly always "harmful to others"). > > It's unfortunate you've not had time to look at it over the past several > > months, since now that there is only a week or two left, I'm sure you or no > > one on your team or community would either. Did you open a bug for this? > > m2e does not re-export guava packages, see bug 430714, and does not force > installation of any specific guava version, see bug 435616. Excellent! Thanks for the update.
See Bug 437107 for suggestions for a future policy.
I ran into pretty much the same problem as reported in comment #60 when trying to install GEF4 MVC bundles into an Mars M1 Eclipse classic. All bundles that depended on org.eclipse.osgi as well as com.google.guava failed with a 'Uses constraint violation.'. As a workaround I had to specify an unconstrained package-import for javax.annotation (i.e. without version constraint) in all those bundles (I did not find another way) and it looks like this shouldn't actually be required.
(In reply to Alexander Nyßen from comment #72) > As a workaround I had to specify an > unconstrained package-import for javax.annotation (i.e. without version > constraint) in all those bundles (I did not find another way) and it looks > like this shouldn't actually be required. This sounds like the start of a new round of chaos. The sooner we agree on a policy in Bug 437107, the sooner we can get tooling support and debug any further consequences. I have not seen any Eclipse plugin that I use with 'uses' directives.
(In reply to Dennis Huebner from comment #64) > (In reply to Camille Letavernier from comment #63) > > What's the best way for deploying Orbit bundles? > > > > I've read somewhere that we are supposed to embed a specific version of the > > plug-in in our own features. > > > > For Papyrus, we do that with Guava 11; it is included in one of our > > features. However, this results in Guava 11 being installed along with > > Papyrus, even if another version of Guava (> 11) is already available. > > Include it in you "build" feature and import it in your SDK/Runtime feature What to do if the dependency is on an orbit fragment. We have a depedency on com.google.inject.multibindings. I cannot specify a dependency in my runtime feature in this case, only a package import in my bundles. Any recommendations?
(In reply to Alexander Nyßen from comment #74) > What to do if the dependency is on an orbit fragment. We have a depedency on > com.google.inject.multibindings. I cannot specify a dependency in my runtime > feature in this case, only a package import in my bundles. Any > recommendations? For Code Recommenders we use feature/requires/import [1]. PDE flags a warning for fragments used as //import@plugin, but aside from that this approach works well for us. Hope this helps. [1] <http://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/features/org.eclipse.recommenders.rcp.feature/feature.xml?id=v2.1.14#n30>
I see both com.google.guava_18.0.0.201507031715.jar and com.google.guava_15.0.0.v201403281430.jar in http://download.eclipse.org/releases/staging/plugins Is it going to cause trouble? (Sorry for the noise!)
Sure to be a problem. We have been agreed on Guava 15 for Luna and Mars, so it is probably time to move on. In the absence of a tested and tooled solution using perhaps uses directives, we probably need a cross-project announcement that all projects should use primarily and at least Guava 18 in Neon, but primarily and at least Guava 15 for Mars sub-releases.
(In reply to Marc-Andre Laperle from comment #76) > I see both com.google.guava_18.0.0.201507031715.jar and > com.google.guava_15.0.0.v201403281430.jar in > http://download.eclipse.org/releases/staging/plugins > > Is it going to cause trouble? > > (Sorry for the noise!) Interestingly, the current Orbit integration build for Neon (http://download.eclipse.org/tools/orbit/downloads/drops/I20150910053315/) does not seem to contain Guava 18. Where did it come from?
(In reply to Alexander Nyßen from comment #78) > (In reply to Marc-Andre Laperle from comment #76) > > I see both com.google.guava_18.0.0.201507031715.jar and > > com.google.guava_15.0.0.v201403281430.jar in > > http://download.eclipse.org/releases/staging/plugins > > > > Is it going to cause trouble? > > > > (Sorry for the noise!) > > Interestingly, the current Orbit integration build for Neon > (http://download.eclipse.org/tools/orbit/downloads/drops/I20150910053315/) > does not seem to contain Guava 18. Where did it come from? It seems http://download.eclipse.org/releases/staging/plugins does no longer contain Guava 18.
Do we already have an agreement on which Guava version to settle on for Neon? It seems up to now, we are still bound to Guava 15 (and I am fine with that).
Guava 18 is in Orbit and is in use / tolerated by some projects. In the absence of an explicit agreement, I consider that the latest version in Orbit constitutes an implicit agreement.
(In reply to Ed Willink from comment #81) > Guava 18 is in Orbit and is in use / tolerated by some projects. > > In the absence of an explicit agreement, I consider that the latest version > in Orbit constitutes an implicit agreement. No, it isn't. The Orbit build for Neon M3 (http://download.eclipse.org/tools/orbit/downloads/drops/S20151016170251/) does not contain it, and as I mentioned in comment #79, also simrel does no longer seem to contain it. Thats the reason I ask...
You are right, and it's not in the latest Orbit I-build either. I'll try to identify which SimRel project is distributing a non-Orbit Guava 18 and so confusing me. https://dev.eclipse.org/ipzilla/show_bug.cgi?id=10367 shows an awaiting_pmc for iot.hawkbit
(In reply to Ed Willink from comment #81) > Guava 18 is in Orbit and is in use / tolerated by some projects. Guava 18 is not in Orbit but it is in aggregated builds. See Bug 483590.
I think the majority of comments is in agreement that we will not mark Guava bundles as singletons. Instead we need to identify projects with trouble using Guava and helping them applying correct import/export/uses constraints. Also, if anyone thinks their are issues with a specific Guava version in Orbit please raise them. FWIW, I think the better resolution here is "worksforme" as we do have projects successfully co-existing in the same instance of Eclipse consuming different Guava versions.
