Bug 381699 - 'org.eclipse.equinox.concurrent' disappeared from distributions
Summary: 'org.eclipse.equinox.concurrent' disappeared from distributions
Status: VERIFIED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 3.8   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 3.8 RC4   Edit
Assignee: John Arthorne CLA
QA Contact:
URL:
Whiteboard:
Keywords: readme
Depends on:
Blocks:
 
Reported: 2012-06-05 09:03 EDT by Dani Megert CLA
Modified: 2013-05-28 08:05 EDT (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dani Megert CLA 2012-06-05 09:03:43 EDT
3.8-I20120604-2100 and 4.2-I20120604-1900.

API Breakage: the bundle 'org.eclipse.equinox.concurrent' disappeared from the build. It was there up to RC2.

I assume this is collateral damage from the fix for bug 381500.
Comment 1 Dani Megert CLA 2012-06-05 09:06:09 EDT
>  It was there up to RC2.
And also in RC3.
Comment 2 Dani Megert CLA 2012-06-05 09:11:49 EDT
The bundle is part of the SDK since 3.5.
Comment 3 Dani Megert CLA 2012-06-05 09:13:25 EDT
FYI: I found this out when checking the API Tools report:
http://download.eclipse.org/eclipse/downloads/drops/I20120604-2100/apitools/apitoolsverifications/html/index.html
Comment 4 John Arthorne CLA 2012-06-05 09:16:34 EDT
Just to be clear, we never wanted this bundle in our build. Its unwanted appearance was one of the use cases that prompted all these greedy/non-greedy changes in the first place (bug 268178). Also the API is still marked experimental/x-internal (see bug 309247). Having said that, RC4 obviously isn't a nice time to be removing it. Maybe we all should have avoided making this "greedy" fix so late in the end-game.
Comment 5 Thomas Watson CLA 2012-06-05 09:23:41 EDT
So what is the fix?  I don't think we go back to ECF and ask them to make the dependency optional again.  I guess we could hack it back in, but I don't suggest we add it to any of our features.  Since the API is marked internal I think it is fine to simply add a note to the migration document and move on.
Comment 6 Dani Megert CLA 2012-06-05 09:32:13 EDT
(In reply to comment #5)
> So what is the fix?  I don't think we go back to ECF and ask them to make the
> dependency optional again.  I guess we could hack it back in, but I don't
> suggest we add it to any of our features.  Since the API is marked internal I
> think it is fine to simply add a note to the migration document and move on.

That could work if we talk about the SDK, but the bundle was also in the Platform Runtime Binary since 3.5. Removing it during RC4 is a no go for me.
Comment 7 John Arthorne CLA 2012-06-05 09:32:30 EDT
One option is to add it back to the platform feature but only in the 3.8 stream. We already have quite a few bundles being deleted in the 4.2 stream so one more isn't the end of the world. If any feature gets installed that actually requires it, then it would pulled in by virtue of being in the juno release train repository.
Comment 8 David Williams CLA 2012-06-05 09:43:05 EDT
It is still in the repository, judging from the file system: 

.../eclipse/updates/4.2-I-builds/I20120604-1900/plugins

$ ls -1 org.eclipse.equinox.concurrent*
org.eclipse.equinox.concurrent_1.0.300.v20120522-2049.jar
org.eclipse.equinox.concurrent_1.0.300.v20120522-2049.jar.pack.gz
org.eclipse.equinox.concurrent.source_1.0.300.v20120522-2049.jar
org.eclipse.equinox.concurrent.source_1.0.300.v20120522-2049.jar.pack.gz

So consumers can get/add to their feature, if they need it. 

IMHO, if "we never wanted this bundle in our build" then it shouldn't be there. 

Yes, we could leave in 3.8 but not in 4.2 ... but, IMHO, we are in "cut-corners mode" here ... every minute we spend on fixing "just in case" problem is another minute not fixing a known problem. 

And sounds like this would be an improvement, that downstream consumers could easily work around .. "the right way" .. now that we've fixed the greedy business. 

But, that's for catching in Dani. Very much appreciated. 

I am fine living with what every you PMC members deem is critical for the success of Eclipse.
Comment 9 David Williams CLA 2012-06-05 09:46:40 EDT
> 
> But, that's for catching in Dani. Very much appreciated. 
> 

But, THANKS for catching IT Dani 

... m u s t   l e a r n  t o  p r o o f r e a d
Comment 10 Dani Megert CLA 2012-06-05 09:48:36 EDT
(In reply to comment #8)
> So consumers can get/add to their feature, if they need it. 

I'm not especially worried about consumers but simple users that have some additional bundles (no update site) that work fine against 3.7 or 4.1 and then simply download 3.8 or 4.2, add their extra bundles, and are doomed if one of the bundles uses 'org.eclipse.equinox.concurrent'.
Comment 11 Thomas Watson CLA 2012-06-05 09:56:21 EDT
(In reply to comment #10)
> (In reply to comment #8)
> > So consumers can get/add to their feature, if they need it. 
> 
> I'm not especially worried about consumers but simple users that have some
> additional bundles (no update site) that work fine against 3.7 or 4.1 and then
> simply download 3.8 or 4.2, add their extra bundles, and are doomed if one of
> the bundles uses 'org.eclipse.equinox.concurrent'.

But in such cases they would be using an x-internal package.  This does not seem like an API breaking change to me to remove some x-internal package.
Comment 12 Dani Megert CLA 2012-06-05 09:58:40 EDT
(In reply to comment #11)
> (In reply to comment #10)
> > (In reply to comment #8)
> > > So consumers can get/add to their feature, if they need it. 
> > 
> > I'm not especially worried about consumers but simple users that have some
> > additional bundles (no update site) that work fine against 3.7 or 4.1 and then
> > simply download 3.8 or 4.2, add their extra bundles, and are doomed if one of
> > the bundles uses 'org.eclipse.equinox.concurrent'.
> 
> But in such cases they would be using an x-internal package.  This does not
> seem like an API breaking change to me to remove some x-internal package.

If everything in that bundle was internal, then I agree.
Comment 13 Dani Megert CLA 2012-06-05 10:04:27 EDT
(In reply to comment #12)
> (In reply to comment #11)
> > (In reply to comment #10)
> > > (In reply to comment #8)
> > > > So consumers can get/add to their feature, if they need it. 
> > > 
> > > I'm not especially worried about consumers but simple users that have some
> > > additional bundles (no update site) that work fine against 3.7 or 4.1 and then
> > > simply download 3.8 or 4.2, add their extra bundles, and are doomed if one of
> > > the bundles uses 'org.eclipse.equinox.concurrent'.
> > 
> > But in such cases they would be using an x-internal package.  This does not
> > seem like an API breaking change to me to remove some x-internal package.
> 
> If everything in that bundle was internal, then I agree.

Verified that this is the case since 3.5:

Export-Package: org.eclipse.equinox.concurrent.future;version="1.0";x-
 internal:="true"
Comment 14 David Williams CLA 2012-06-06 00:44:29 EDT
So, it sounds like title would more accurately be: 

'org.eclipse.equinox.concurrent' disappeared from distributions

instead of 

API Breakage: 'org.eclipse.equinox.concurrent' disappeared from build

In that it is in the repo and has never been API? 

(I would have changed it myself, but was afraid I was mis-reading something ... and didn't want to seem to be down-playing the problem ... it is unfortunate to make the change so late ... but, if I'm reading it right, not quite as bad as "API Breakage" and "disappearing".
Comment 15 Dani Megert CLA 2012-06-06 02:53:49 EDT
(In reply to comment #14)
> So, it sounds like title would more accurately be: 
> 
> 'org.eclipse.equinox.concurrent' disappeared from distributions
> 
> instead of 
> 
> API Breakage: 'org.eclipse.equinox.concurrent' disappeared from build

Yes, true because the bundle contains no API packages. Otherwise it would clearly be an API breakage because it also disappeared from the Platform Runtime binary build, not just the SDK.
Comment 16 Thomas Watson CLA 2012-06-06 08:46:36 EDT
Changing title and reducing severity.
Comment 17 Dani Megert CLA 2012-06-06 10:52:28 EDT
We discussed this in the PMC call and decided to leave it removed, but add an entry to the porting guide and the readme, explaining exactly how/where the bundle can be fetched.
Comment 19 Scott Lewis CLA 2012-06-06 15:12:38 EDT
ECF consumes/requires concurrent in our build (to provide OSGi remote service/RSA impl).  If asked, we would vote -1 on removing from Equinox.

It seems a little strange that we haven't been included in this discussion (i.e. not asked).

Concurrent is x-internal (still) since the Equinox team has not made it non-provisional as per another bug that I'm still looking up (i.e. it should have been made non x-internal some time ago).
Comment 20 John Arthorne CLA 2012-06-06 15:21:54 EDT
(In reply to comment #19)
> ECF consumes/requires concurrent in our build (to provide OSGi remote
> service/RSA impl).  If asked, we would vote -1 on removing from Equinox.

The bundle hasn't been removed from Equinox. It is still in the Equinox SDK, still in our p2 repository and the Juno release repository.

> It seems a little strange that we haven't been included in this discussion
> (i.e. not asked).

Well, we didn't actually make any change. ECF changed its dependencies which caused it to no longer be installed when provisioning the platform. This was not an expected change and it was just discovered yesterday. You should have been CC'd when we discovered it - sorry for missing that.

> Concurrent is x-internal (still) since the Equinox team has not made it
> non-provisional as per another bug that I'm still looking up (i.e. it should
> have been made non x-internal some time ago).

See comment #4.
Comment 21 Thomas Watson CLA 2012-06-06 15:29:40 EDT
(In reply to comment #20)
>You should have been CC'd when we discovered it - sorry for missing that.

True, but this is also discussed in ECF bug 381478 where the ECF repo was changed to not use greedy optional dependencies.  That bug lead to this side effect.
Comment 22 Scott Lewis CLA 2012-06-06 15:32:57 EDT
(In reply to comment #20)
> (In reply to comment #19)
> > ECF consumes/requires concurrent in our build (to provide OSGi remote
> > service/RSA impl).  If asked, we would vote -1 on removing from Equinox.
> 
> The bundle hasn't been removed from Equinox. It is still in the Equinox SDK,
> still in our p2 repository and the Juno release repository.

I see.

> 
> > It seems a little strange that we haven't been included in this discussion
> > (i.e. not asked).
> 
> Well, we didn't actually make any change. ECF changed its dependencies which
> caused it to no longer be installed when provisioning the platform. This was
> not an expected change and it was just discovered yesterday. You should have
> been CC'd when we discovered it - sorry for missing that.

Np...thanks for the explanation.

> 
> > Concurrent is x-internal (still) since the Equinox team has not made it
> > non-provisional as per another bug that I'm still looking up (i.e. it should
> > have been made non x-internal some time ago).

From ECF's perspective (along with previous equinox committers), we would disagree with you about the 'we never wanted this bundle in our build'.  

Since it's in the equinox namespace (and so is managed/maintained by equinox...not us) we would rather not have to compile, package, p2ify and distribute as part of ECF...in order to be available to support OSGi remote services within Eclipse.  If forced to do that we will have to...but we would rather not have to build and distribute code that we can't maintain/modify.
Comment 23 Scott Lewis CLA 2012-06-06 15:38:37 EDT
In case it's not obvious from the discussion so far...I understand that concurrent is (still) in the Equinox SDK, but because Eclipse's distribution 

a) already has had concurrent in sdk until now
b) is much more heavily used by consumer (e.g. we use Eclipse as target platform for ECF build...because we need it for other parts of ECF);  

I would strongly prefer to have the concurrent API continue to be included with Eclipse.  Otherwise it complicates both ECF's build as well as any/all consumers of ECF's impl of OSGi remote services (now a part of OSGi core...as I'm sure you know).
Comment 24 John Arthorne CLA 2012-06-06 16:26:37 EDT
(In reply to comment #17)
> We discussed this in the PMC call and decided to leave it removed, but add an
> entry to the porting guide and the readme, explaining exactly how/where the
> bundle can be fetched.

I struggled with how to document this in the readme. The problem we were describing this morning about a previously dropped in bundle failing to resolve is actually fairly general. For example it might depend on an older version range of any changed bundle in the platform, or any other bundle that we removed such as org.mortbay.jetty. Anyway after several attempts here is what I came up with:

http://git.eclipse.org/c/platform/eclipse.platform.releng.git/commit/?h=R4_HEAD&id=4ea38919c45c045f2b238d0bea7a8bf19f6ec3b9
Comment 25 Scott Lewis CLA 2012-06-06 16:51:24 EDT
>We discussed this in the PMC call and decided to leave it removed

Just to be clear, ECF votes -1 on removing concurrent from SDK

If we had known that complying with bug 381478 meant that it would break our own future builds, and possibly break our remote service and RSA consumer's systems/products as a result, we would not have removed the greedy requirement.  Rather, I would have added x-greedy or whatever that flag is instead.

The optional resolution in org.eclipse.ecf was only put there originally because the concurrent future api was x-internal (provisional)...since we have been waiting on bug 309247.
Comment 26 Scott Lewis CLA 2012-06-06 17:03:43 EDT
(In reply to comment #10)
> (In reply to comment #8)
> > So consumers can get/add to their feature, if they need it. 
> 
> I'm not especially worried about consumers but simple users that have some
> additional bundles (no update site) that work fine against 3.7 or 4.1 and then
> simply download 3.8 or 4.2, add their extra bundles, and are doomed if one of
> the bundles uses 'org.eclipse.equinox.concurrent'.

Yes, this is exactly the problem consumers of ECF remote services/RSA will encounter if concurrent is removed from SDK.  Worse...they will likely blame ECF for this breakage.

Has anyone considered simply adding org.eclipse.equinox.concurrent to the set of bundles that ECF filetransfer contributes to Eclipse/p2?
Comment 27 Andrew Overholt CLA 2012-06-06 17:08:39 EDT
(In reply to comment #24)
> The problem we were
> describing this morning about a previously dropped in bundle failing to resolve
> is actually fairly general. For example it might depend on an older version
> range of any changed bundle in the platform, or any other bundle that we
> removed such as org.mortbay.jetty. Anyway after several attempts here is what I
> came up with:
> 
> http://git.eclipse.org/c/platform/eclipse.platform.releng.git/commit/?h=R4_HEAD&id=4ea38919c45c045f2b238d0bea7a8bf19f6ec3b9

This looks good to me but I may be a poor judge of it since I'm all
too familiar with this problem :)
Comment 28 John Arthorne CLA 2012-06-06 17:20:43 EDT
Your arguments aren't entirely convincing Scott. Bundles aren't added to the Platform simply to "make builds simpler". ECF's build must have many bundles it relies on that are not in the platform and this would just be another case. Since the concurrent bundle is still present in all the p2 repositories it was in last release, I don't know what kind of build would be impacted. A PDE/Tycho/Buckminster build would typically be fetching required bundles from a repository rather than extracting them from a particular zip like the Platform zip.

I also don't understand the ECF/remote services failure case. In bug 381474 comment 8 you said the concurrent bundle was explicitly included in your ECF features. So, anyone installing those features would be fine.
Comment 29 Thomas Watson CLA 2012-06-06 17:21:51 EDT
(In reply to comment #26)
> (In reply to comment #10)
> > (In reply to comment #8)
> > > So consumers can get/add to their feature, if they need it. 
> > 
> > I'm not especially worried about consumers but simple users that have some
> > additional bundles (no update site) that work fine against 3.7 or 4.1 and then
> > simply download 3.8 or 4.2, add their extra bundles, and are doomed if one of
> > the bundles uses 'org.eclipse.equinox.concurrent'.
> 
> Yes, this is exactly the problem consumers of ECF remote services/RSA will
> encounter if concurrent is removed from SDK.  Worse...they will likely blame
> ECF for this breakage.

I'm confused by this since in you stated in bug 381478 comment 8 that you include the concurrent bundle directly anyway.  So if someone is targeting one of your features, wouldn't they get the concurrent bundle?
Comment 30 Scott Lewis CLA 2012-06-06 17:43:17 EDT
(In reply to comment #28)
> Your arguments aren't entirely convincing Scott. 

I disagree.  When you consider it from ECF's perspective...and ECF's consumers...which is what my concern is...I think my arguments are entirely convincing John :).
Comment 31 Dani Megert CLA 2012-06-07 02:40:12 EDT
I'm marking this bug fixed since I verified that the porting guide and the readme are updated in 3.8-I20120606-2100 and 4.2-I20120606-1900.

Scott, if you disagree with the resolution, then please open a new bug.
Comment 32 Scott Lewis CLA 2012-06-07 10:38:10 EDT
(In reply to comment #31)
> I'm marking this bug fixed since I verified that the porting guide and the
> readme are updated in 3.8-I20120606-2100 and 4.2-I20120606-1900.
> 
> Scott, if you disagree with the resolution, then please open a new bug.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=382011
Comment 33 Igor Fedorenko CLA 2012-06-24 07:29:01 EDT
I just want to confirm I understand the outcome of this bugfix correctly -- org.eclipse.ecf bundle is not resolvable if installed as part of org.eclipse.sdk.ide feature. At least when using .target file with includeMode="slicer", have not checked other cases yet. Did I get this right?
Comment 34 John Arthorne CLA 2012-06-25 08:56:11 EDT
(In reply to comment #33)
> I just want to confirm I understand the outcome of this bugfix correctly --
> org.eclipse.ecf bundle is not resolvable if installed as part of
> org.eclipse.sdk.ide feature. At least when using .target file with
> includeMode="slicer", have not checked other cases yet. Did I get this right?

The final state in Juno is that org.eclipse.equinox.concurrent was added back as a non-optional dependency (see bug 382103). So org.eclipse.ecf should resolve fine as part of org.eclipse.sdk.ide.
Comment 35 Igor Fedorenko CLA 2012-06-25 14:25:07 EDT
(In reply to comment #34)
> (In reply to comment #33)
> > I just want to confirm I understand the outcome of this bugfix correctly --
> > org.eclipse.ecf bundle is not resolvable if installed as part of
> > org.eclipse.sdk.ide feature. At least when using .target file with
> > includeMode="slicer", have not checked other cases yet. Did I get this right?
> 
> The final state in Juno is that org.eclipse.equinox.concurrent was added back
> as a non-optional dependency (see bug 382103). So org.eclipse.ecf should
> resolve fine as part of org.eclipse.sdk.ide.

What feature is supposed to include org.eclipse.equinox.concurrent? I only it referenced by import-package in org.eclipse.ecf bundle, but not included in any feature that is part of org.eclipse.sdk.ide.
Comment 36 John Arthorne CLA 2012-06-25 14:33:53 EDT
(In reply to comment #35)
> What feature is supposed to include org.eclipse.equinox.concurrent? I only it
> referenced by import-package in org.eclipse.ecf bundle, but not included in any
> feature that is part of org.eclipse.sdk.ide.

Yes that is the current state. It is not in any feature (and never was in the past either).