Bug 538737 - Remove org.eclipse.equinox.ds from products
Summary: Remove org.eclipse.equinox.ds from products
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 4.8   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 4.10   Edit
Assignee: Platform-Releng-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: Documentation, noteworthy
Depends on:
Blocks: 539655
  Show dependency tree
 
Reported: 2018-09-06 10:15 EDT by Mickael Istria CLA
Modified: 2020-08-05 10:52 EDT (History)
13 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mickael Istria CLA 2018-09-06 10:15:53 EDT
It seems like org.eclipse.equinox.ds is no more useful and is mostly a proxy for org.apache.felix.scr. So we should probably consider it removing it from the product. To do so, it's mostly a matter of removing it from the org.eclipse.e4.rcp feature.
Comment 1 Thomas Watson CLA 2018-09-06 10:20:01 EDT
(In reply to Mickael Istria from comment #0)
> It seems like org.eclipse.equinox.ds is no more useful and is mostly a proxy
> for org.apache.felix.scr. So we should probably consider it removing it from
> the product. To do so, it's mostly a matter of removing it from the
> org.eclipse.e4.rcp feature.

And also making sure nothing else is depending on it.  This has to be announced to the community that we intend to remove the bundle I think.
Comment 2 Alexander Kurtakov CLA 2018-09-06 10:23:17 EDT
IMHO, initially we should remove it from our features but keep publishing it to the repo. That should be a good sign to the community.
Comment 3 Alexander Kurtakov CLA 2018-09-06 10:25:28 EDT
Actual removal of equinox.ds can happen after 2018-12.
Comment 4 Eclipse Genie CLA 2018-09-06 10:36:26 EDT
New Gerrit change created: https://git.eclipse.org/r/128845
Comment 5 Eclipse Genie CLA 2018-09-07 03:34:46 EDT
New Gerrit change created: https://git.eclipse.org/r/128882
Comment 6 Mickael Istria CLA 2018-09-07 04:01:54 EDT
(In reply to Thomas Watson from comment #1)
> And also making sure nothing else is depending on it.  This has to be
> announced to the community that we intend to remove the bundle I think.

Right.
I identified one place where adopters need to apply a change if Platform changes it and adopters adopt newer platform: the .product files usually configure the auto-start ane start-level for org.eclipse.equinox.ds. There are such .product files in Tycho, in PDE templates, and in other projects. Surprisingly, it seems like the .product files for EPP don't have this declaration and still work well...
So indeed we need to announce if we plan to remove it from feature, and document the .product must be updated to reference org.apache.felix.scr in place of org.eclipse.equinox.ds (technically, products could already add the line about org.apache.felix.scr).
I suggest we wait after SimRel 4.9 release is done, and then we'll send the warning and recommendations.
Comment 7 Thomas Watson CLA 2018-09-07 09:05:40 EDT
(In reply to Mickael Istria from comment #6)
> (In reply to Thomas Watson from comment #1)
> > And also making sure nothing else is depending on it.  This has to be
> > announced to the community that we intend to remove the bundle I think.
> 
> Right.
> I identified one place where adopters need to apply a change if Platform
> changes it and adopters adopt newer platform: the .product files usually
> configure the auto-start ane start-level for org.eclipse.equinox.ds. There
> are such .product files in Tycho, in PDE templates, and in other projects.
> Surprisingly, it seems like the .product files for EPP don't have this
> declaration and still work well...
> So indeed we need to announce if we plan to remove it from feature, and
> document the .product must be updated to reference org.apache.felix.scr in
> place of org.eclipse.equinox.ds (technically, products could already add the
> line about org.apache.felix.scr).
> I suggest we wait after SimRel 4.9 release is done, and then we'll send the
> warning and recommendations.

For sure after 4.9 (2018-09), but probably a release or two after.  I recommend after 2018-12 and we announce as soon as we agree.
Comment 10 Andrey Loskutov CLA 2018-09-28 05:12:34 EDT
This caused 169 test fails in equinox.http.servlet.tests, see
http://download.eclipse.org/eclipse/downloads/drops4/I20180927-1800/testresults/html/org.eclipse.equinox.http.servlet.tests_ep410I-unit-cen64-gtk3_linux.gtk.x86_64_8.0.html

Basically something like

Multiple Failures (2 failures) Failed to find bundle: org.eclipse.equinox.ds <no message> in java.lang.NullPointerException 

org.opentest4j.MultipleFailuresError: Multiple Failures (2 failures)
Failed to find bundle: org.eclipse.equinox.ds
<no message> in java.lang.NullPointerException
Comment 11 Eclipse Genie CLA 2018-10-01 04:34:11 EDT
New Gerrit change created: https://git.eclipse.org/r/130229
Comment 13 Eclipse Genie CLA 2018-10-01 06:17:21 EDT
New Gerrit change created: https://git.eclipse.org/r/130235
Comment 15 Thomas Watson CLA 2018-10-01 14:42:51 EDT
*** Bug 539655 has been marked as a duplicate of this bug. ***
Comment 16 Ed Willink CLA 2018-10-01 15:51:10 EDT
Removing a bundle is an unannounced API breakage.

This appears to be an unannounced API breakage.

Strictly speaking this requires a major version change, but more pragmatically surely it requires an announcement and a two year delay before removal?

As it is existing builds fail on 4.10 I-builds since an existing dependency is missing.

CRITCAL since the breakage has already been committed.
Comment 17 Mickael Istria CLA 2018-10-01 16:25:25 EDT
(In reply to Ed Willink from comment #16)
> Removing a bundle is an unannounced API breakage.

The bundle is still available in the p2 repo for those who still need it in their target-platform, and its API didn't change.
It's just not included by default in feature/products any more. But there are alternatives and workaround.
What is the actual issue you're seeing as an adopter? Please a separate bug about it and link it to this one.
Comment 18 Thomas Watson CLA 2018-10-01 16:48:35 EDT
(In reply to Mickael Istria from comment #17)
> What is the actual issue you're seeing as an adopter? Please a separate bug
> about it and link it to this one.

I dupp'ed Ed's bug 539655 to this one.  I have now reopened that one and linked it to this bug.
Comment 19 Christian Dietrich CLA 2018-10-03 11:00:16 EDT
See https://github.com/eclipse/xtext-eclipse/issues/841

We currently use it as extra requirement in tycho
Comment 20 Mickael Istria CLA 2018-10-03 11:30:37 EDT
(In reply to Christian Dietrich from comment #19)
> See https://github.com/eclipse/xtext-eclipse/issues/841
> We currently use it as extra requirement in tycho

why do you have an explicit requirement on org.eclipse.equinox.ds?
Comment 21 Christian Dietrich CLA 2018-10-03 12:52:55 EDT
I don’t know. The code is longer there than I am committer. Of course the commit messages has no clues. I assume the tests need that to have a proper ide started (don’t know which services are used in one of our gazillions of tests)
Comment 22 Eclipse Genie CLA 2018-10-04 11:46:25 EDT
New Gerrit change created: https://git.eclipse.org/r/130417
Comment 24 Mickael Istria CLA 2018-10-04 16:37:45 EDT
About the missing "org.eclipse.equinox.ds.tests" results in the "production" build, I think we have 2 choices:
* Either we just remove them from the official build (remove lines 1661-1665 from eclipse.platform.releng.aggregator/production/testScripts/configuration/sdk.tests/testScripts/test.xml, or
* We keep the test in, and we need to include org.eclipse.equinox.ds in the test feature eclipse.platform.releng.aggregator/eclipse.platform.releng/features/org.eclipse.sdk.tests/feature.xml so it can be resolved.
Comment 25 Eclipse Genie CLA 2018-10-05 01:38:40 EDT
New Gerrit change created: https://git.eclipse.org/r/130443
Comment 27 Alexander Kurtakov CLA 2018-10-05 02:13:05 EDT
(In reply to Eclipse Genie from comment #26)
> Gerrit change https://git.eclipse.org/r/130443 was merged to [master].
> Commit:
> http://git.eclipse.org/c/equinox/rt.equinox.bundles.git/commit/
> ?id=b51f5bad64d73179bbdd073c398428383e1302ee

That's better approach so we still have tests verifying dynamic services work. Only single test verifying old felix annotations had to be removed.
Comment 28 Alexander Kurtakov CLA 2018-10-05 02:14:27 EDT
Mickael, please make sure there is N&N entry and it's properly documented in migration docs (http://git.eclipse.org/c/platform/eclipse.platform.common.git/tree/bundles/org.eclipse.platform.doc.isv/porting - you would have to create the 4.10 stub pages first).
Comment 29 Eclipse Genie CLA 2018-10-05 06:38:42 EDT
New Gerrit change created: https://git.eclipse.org/r/130458
Comment 30 Thomas Watson CLA 2018-10-05 08:30:26 EDT
(In reply to Alexander Kurtakov from comment #27)
> (In reply to Eclipse Genie from comment #26)
> > Gerrit change https://git.eclipse.org/r/130443 was merged to [master].
> > Commit:
> > http://git.eclipse.org/c/equinox/rt.equinox.bundles.git/commit/
> > ?id=b51f5bad64d73179bbdd073c398428383e1302ee
> 
> That's better approach so we still have tests verifying dynamic services
> work. Only single test verifying old felix annotations had to be removed.

Yes, I think that is the right thing to do.  No need keeping tests for the old stuff that we will be getting rid of entirely in the future.
Comment 32 Thomas Watson CLA 2018-10-05 08:34:44 EDT
(In reply to Eclipse Genie from comment #29)
> New Gerrit change created: https://git.eclipse.org/r/130458

Thanks Mickael, I merged the N&N entries.
Comment 33 Ed Willink CLA 2018-10-05 08:55:44 EDT
A user's perspective on the necessary workarounds:

0) Originally o.e.equinox.ds was magically provided and started.

1) The o.a.felix.scr rationalization meant that o.e.equinox.ds/o.a.felix.scr was not started breaking Tycho 1.1.0 / 1.2.0 test startup.

- workaround add an explicit Require-Bundle for o.e.equinox.ds to every test bundle

2) The o.e.equinox.ds 'removal' in terms of magic provision (it seems never to have been in a feature) causes Tycho to fail to resolve bundle requirements. This seems almost designed to break the previous workaround.

- workaround add <unit id="org.eclipse.equinox.ds" version="0.0.0"/> to ensure it is fetched by the platform definition.
Comment 34 Eclipse Genie CLA 2018-10-06 02:07:09 EDT
New Gerrit change created: https://git.eclipse.org/r/130526
Comment 36 Simeon Andreev CLA 2018-10-30 09:21:00 EDT
When trying our product with 4.10 as platform, we ran into bug 540605 (resp. bug 510673).
Comment 37 Mickael Istria CLA 2018-11-19 16:04:36 EST
I think we're done here.
Comment 38 Ed Willink CLA 2018-11-20 01:48:24 EST
(In reply to Mickael Istria from comment #37)
> I think we're done here.

Sorry. I don't. See Bug 539655. Tycho 1.3.0 is not yet available, so to build with current tools I have had to create my own dummy org.eclipse.equinox.ds to make many warnings go away.

org.eclipse.equinox.ds is a referenceable entity. It is therefore API. The earlier changes for 4.9 mandated that users reference it. It should therefore remain available for two years after announcement of deprecation/removal.

Today with Tycho 1.3.0 not even available, org.eclipse.equinox.ds is a necessary name, even if it does nothing.
Comment 39 Mickael Istria CLA 2018-11-20 02:24:21 EST
Ed, this change was monitored and approved by project leads and even PMC I believe. I assume this means it's all clear in term of contract since those entities re careful experts of the topic.
No concern about API was raised because package content aren't part of an API constraint, and the bundle is still referenceable and resolvable from p2 repo.
Please use other tickets to track this on Tycho or other integrations, but unless there is something wrong inside Platform about it,please don't reopen.
Comment 40 Ed Willink CLA 2018-11-20 03:29:02 EST
The bundle is no longer part of the ZIP, which is what some/most users use to install Eclipse. I certainly don't install the Eclipse platform from a repo so my Eclipse installation is broken by this instant API breakage. Total violation of policy.

Where on https://help.eclipse.org/2018-09/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fporting%2Fremovals.html is the mention that org.eclipse.equinox.ds is being removed or even changed?

Tycho is an irrelevance since Tycho is not a synchronized to the platform. It just demonstrates the instancy of the breakage.
Comment 41 J Clausius CLA 2020-08-05 09:23:50 EDT
(In reply to Ed Willink from comment #40)
> The bundle is no longer part of the ZIP, which is what some/most users use
> to install Eclipse. I certainly don't install the Eclipse platform from a
> repo so my Eclipse installation is broken by this instant API breakage.
> Total violation of policy.
> 
> Where on
> https://help.eclipse.org/2018-09/index.jsp?topic=%2Forg.eclipse.platform.doc.
> isv%2Fporting%2Fremovals.html is the mention that org.eclipse.equinox.ds is
> being removed or even changed?
> 
> Tycho is an irrelevance since Tycho is not a synchronized to the platform.
> It just demonstrates the instancy of the breakage.


So, what is the actual fix / instructions if one is coming from an older version, and is hitting this problem now?  

The app.products file includes <plugin id="org.apache.felix.scr"... /> in the <configurations> section.

A) If I re-add the app.product file to include <plugin id="org.eclipse.equinox.ds...> (and use it in the .product file's <configurations> section), Tycho 1.7.0 produces an error of : Missing requirement: toolingcocoa.macosx.x86_64org.eclipse.equinox.ds 10.1.0.1128- requires 'osgi.bundle; org.eclipse.equinox.ds

However, I remove this, still leaving the org.apache.felix.scr plugin, Tycho 1.7.0 errors with : "Unable to locate feature 'org.eclipse.equinox.executable'. This feature is required for native product launchers..."

Is this a Tycho bug?  [ https://bugs.eclipse.org/bugs/show_bug.cgi?id=565799#c2 ]

I cannot find solutions to migrating from a .product's use of "org.eclipse.equinox.ds" to "org.apache.felix.scr" AND having Tycho product a build target which includes native launchers.  Any and all suggestions welcome.

TIA
-jclausius
Comment 42 Christian Dietrich CLA 2020-08-05 09:40:06 EDT
our product works fine with 2019-06 / 4.12
thus i wonder if you can provide a small reproducible example
Comment 43 Christian Dietrich CLA 2020-08-05 09:42:09 EDT
(we use useFeatures=true though)
Comment 44 J Clausius CLA 2020-08-05 10:30:30 EDT
(In reply to Christian Dietrich from comment #42)
> our product works fine with 2019-06 / 4.12
> thus i wonder if you can provide a small reproducible example

Absolutely.  Would you like it to target  Linux, Mac, Windows (or any combination)?
Comment 45 Christian Dietrich CLA 2020-08-05 10:31:13 EDT
best would be all 3
Comment 46 J Clausius CLA 2020-08-05 10:52:42 EDT
(In reply to Christian Dietrich from comment #43)
> (we use useFeatures=true though)


Our RCP app is plugin based ( useFeatures = false ).  Seen any examples on how to convert to features based?