Bug 351117 - Indigo Update Site Causes Circular Dependency Issue
Summary: Indigo Update Site Causes Circular Dependency Issue
Status: VERIFIED FIXED
Alias: None
Product: Objectteams
Classification: Tools
Component: Releng (show other bugs)
Version: 2.0   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: 2.0.1   Edit
Assignee: Stephan Herrmann CLA
QA Contact:
URL:
Whiteboard: Also released in 2.1 M1
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-04 16:02 EDT by Chris Eagan CLA
Modified: 2011-08-21 11:13 EDT (History)
4 users (show)

See Also:


Attachments
Set of Projects which cause the issue (115.67 KB, application/x-zip-compressed)
2011-07-04 16:03 EDT, Chris Eagan CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Eagan CLA 2011-07-04 16:02:25 EDT
Build Identifier: 20110615-0604

When using an RMap which depends on both the eclipse 3.7 update site and the indigo release update site within the same search path, the dependency resolution via buckminster resulted in a a circular dependency resolution issue. The workaround is to include the 3.7 update site is it's own search path first like so:

<rm:locator searchPathRef="eclipse.updates" failOnError="false"/>
<rm:locator searchPathRef="eclipse.indigo" failOnError="false"/>


Reproducible: Always

Steps to Reproduce:
1. Import the existing plug-ins from the attached zip
2. Select com.example.sample.releng/local.mspec and Run Buckminster | Import
3. The error should appear in the resulting dialog.
Comment 1 Chris Eagan CLA 2011-07-04 16:03:57 EDT
Created attachment 199071 [details]
Set of Projects which cause the issue

This is the archive from which to import the existing projects into a workspace in order to see the issue.
Comment 2 Chris Eagan CLA 2011-07-04 16:04:53 EDT
There is a related discussion on the forums at http://www.eclipse.org/forums/index.php/t/217899/
Comment 3 Carsten Reckord CLA 2011-07-13 13:06:59 EDT
The problem here is the following cycle in the indigo repository's jdt.core version:

org.eclipse.jdt.core -> org.eclipse.jdt.ui -> org.eclipse.jdt.launching -> org.eclipse.jdt.core

Thomas was actually wrong: The version of jdt.core does differ between the 3.7 Platform site and the Indigo site: Indigo has org.eclipse.jdt.core:3.7.0.v_OTDT_r200_201106070730 with the offending dependency to jdt.ui, whereas Platform has org.eclipse.jdt.core:3.7.0.v_B61 without this dependency. 

So bucky is really working as intended here and I guess this might be a bug for the JDT guys then!?
Comment 4 Thomas Hallgren CLA 2011-07-13 16:19:23 EDT
(In reply to comment #3)
> So bucky is really working as intended here and I guess this might be a bug for
> the JDT guys then!?

You're right. I find two versions of org.eclipse.jdt.core in the Indigo repository. The culprit with the dependency to org.eclipse.jdt.ui has a lexically greater version qualifier. This is really bad. I'm reassigning this to jdt.
Comment 5 Stephan Herrmann CLA 2011-07-13 17:32:13 EDT
Moving on to Object Teams which contributes one of the plugins in question:
org.eclipse.jdt.core:3.7.0.v_OTDT_r200_201106070730
Comment 6 Stephan Herrmann CLA 2011-07-13 17:49:33 EDT
Some quick explanations of what you are seeing:

The mentioned plugin org.eclipse.jdt.core:3.7.0.v_OTDT_r200_201106070730
is a variant of the original jdt.core plugin. Although this plugin has
a lexically higher qualifier than its original it also has a special
dependency that prevents accidental installation via the UI: a non-greedy
non-optional dependency on the corresponding patch feature through which
it is brought in (see bug 330534 comment 8).

Question to the buckminster experts: how come the OT variant of this plugin
is selected even if the patch feature is not selected? This looks wrong
to me.

Another bit of explanation: in anticipation of the BETA_JAVA7 patch feature 
I tried to make the OT patch and the BETA_JAVA7 patch incompatible
(both patch feature replace the same plugin) by placing a dependency on the
exact version of jdt.ui that is part of Indigo, knowing that BETA_JAVA7 
brings a different version of that plugin.
Unfortunately, by the time of the Indigo release I had no chance to test
against the BETA_JAVA7 patch feature, which was not available at that time.

Equinox and p2 don't seem to have a problem with this circular dependency.
Which part of the tooling breaks on circular dependencies? Why?
Comment 7 Thomas Hallgren CLA 2011-07-14 02:05:17 EDT
(In reply to comment #6)
> Question to the buckminster experts: how come the OT variant of this plugin
> is selected even if the patch feature is not selected? This looks wrong
> to me.
> 
Buckminster cannot use the p2 planner to gather material for the target platform. It's a catch 22 situation. The planner must have knowledge of all candidates in order to come up with a plan. The job for the Buckminster resolver is to find the candidates.

In essence, we use the same approach as the slicer. We simply look at each requirement in turn and try to select the best match using a fairly rudimentary conflict resolution. This process is of course not as complete as the exhaustive analyze that is performed by the SAT resolver but as I said before, there's no way we can use that. A lot of the stuff that we resolve is in source form and incomplete (no generated version ranges, no qualifiers, etc.) so the resolution is always an approximation.

I can think of three possible solutions for Buckminster users:
1. Put a cap on the upper range on any requirement to org.eclipse.jdt.core using a cquery advice node.
2. Tweak the rmap so that sensitive components are selected from the eclipse/updates-3.7 repository rather than from releases/indigo.
3. Use a feature that explicitly appoints the correct version of that plug-in.

If you want more info on this, please ask on the buckminster forums.

> Equinox and p2 don't seem to have a problem with this circular dependency.
> Which part of the tooling breaks on circular dependencies? Why?

I think that depends on the use-case. Any traversal that encounters non-exact version ranges and uses the slicer will get both bundles. Any tooling that uses a p2 query to select the latest 'org.eclipse.jdt.core' from Indigo will get the OT version.

I'm guessing here but I think that a solution to all of the above would be to have the qualifier of the OT version lexically less than the JDT version. That way, the JDT would be considered the default at all times and I think the OT patch would guarantee that the OT version would be selected when needed.
Comment 8 Stephan Herrmann CLA 2011-07-14 05:48:08 EDT
We have (correct me if I'm wrong):

The indigo repository is sound and works OK when accessed using the p2 planner

When used as a target platform (?) during buckminster builds (which I'm
not an expert in), bundle selection works in unexpected ways. Comment 7
suggests how to help buckminster to pick the desired variant.

Circular dependencies are a normal thing in OSGi, I'm surprised that this
breaks some tool. If circularity should be avoided in the indigo repository
I'd strongly suggest to add corresponding checks to the b3 aggregator.

I'd be happy to further improve our deployment if that can help other tools.
Using a lesser version number for the OT variant has been investigated
in bug 330534 comment 5. However, this solution would require bug 330763
in PDE/Build to be fixed, which currently is RESOLVED WONTFIX.
Comment 9 Thomas Hallgren CLA 2011-07-14 06:08:26 EDT
(In reply to comment #8)
> Circular dependencies are a normal thing in OSGi, I'm surprised that this
> breaks some tool.

"Some tools" are not limited to OSGi ;-). Besides,  the breakage is a default behavior that can be overridden.

While I agree that circularity is a normal thing in OSGi, they are most often introduced when using package dependencies. In this case, we have a circular, non-optional, Require-Bundle dependency. IMO, that's wrong and should be avoided at all times.

> I'd be happy to further improve our deployment if that can help other tools.

I'm surprised that a dependency has been introduced from the core to the ui component. This will invalidate headless deployment of the launcher and I know people that invested a significant effort to make that possible in Helios. Would it be possible to do some refactoring to avoid this?

> Using a lesser version number for the OT variant has been investigated
> in bug 330534 comment 5. However, this solution would require bug 330763
> in PDE/Build to be fixed, which currently is RESOLVED WONTFIX.

OK, I wasn't aware of that.
Comment 10 Stephan Herrmann CLA 2011-07-14 06:49:03 EDT
(In reply to comment #9)
> > I'd be happy to further improve our deployment if that can help other tools.
> 
> I'm surprised that a dependency has been introduced from the core to the ui
> component. This will invalidate headless deployment of the launcher and I know
> people that invested a significant effort to make that possible in Helios.
> Would it be possible to do some refactoring to avoid this?

I agree that this was an unfortunate (and even useless) effort. I've added
this dependency to declare an incompatibility between independent patch
features for the same base bundle (to avoid killing one enhancement by
instaling another independent enhancement). I've just asked in the p2 forum
for advice on better ways for declaring this kind of conflict.

For future releases I will *not* specify the core->ui dependency.

Should we reopen bug 330763 to get a new assessment and perhaps revive
the strategy of using a lesser qualifier? Is this critical for anybody?

Otherwise I'm inclined to marking as resolved by removing the core->ui
dependency for future releases.
Comment 11 Stephan Herrmann CLA 2011-08-02 05:56:19 EDT
(In reply to comment #10)
> > I'm surprised that a dependency has been introduced from the core to the ui
> > component. This will invalidate headless deployment of the launcher and I know
> > people that invested a significant effort to make that possible in Helios.
> > Would it be possible to do some refactoring to avoid this?
> 
> I agree that this was an unfortunate (and even useless) effort. I've added
> this dependency to declare an incompatibility between independent patch
> features for the same base bundle (to avoid killing one enhancement by
> instaling another independent enhancement). I've just asked in the p2 forum
> for advice on better ways for declaring this kind of conflict.

Unfortunately, this message yielded no answer :(
http://www.eclipse.org/forums/index.php/mv/msg/220579/696581/#msg_696581


> For future releases I will *not* specify the core->ui dependency.

I have removed that hapless dependency.
 
> Should we reopen bug 330763 to get a new assessment and perhaps revive
> the strategy of using a lesser qualifier? Is this critical for anybody?

I read no answer as not critical to anybody on this bugs Cc.

> Otherwise I'm inclined to marking as resolved by removing the core->ui
> dependency for future releases.

Marking indeed as resolved.
Comment 12 Stephan Herrmann CLA 2011-08-17 12:32:29 EDT
Verified for 2.0.1 using build 201108160141.