Bug 340546 - It should be possible to use fragments classes during development time
Summary: It should be possible to use fragments classes during development time
Status: NEW
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows XP
: P3 enhancement with 7 votes (vote)
Target Milestone: ---   Edit
Assignee: PDE-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 341738 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-03-21 04:30 EDT by Krzysztof Daniel CLA
Modified: 2014-05-23 03:02 EDT (History)
12 users (show)

See Also:


Attachments
Mock of the property page (29.75 KB, image/png)
2011-03-21 04:39 EDT, Krzysztof Daniel CLA
no flags Details
Our problem in diagrams (49.24 KB, application/pdf)
2013-05-04 02:35 EDT, Eike Stepper CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Krzysztof Daniel CLA 2011-03-21 04:30:48 EDT
Fragments exports are generally ignored during plugin development, unless the hosting plugin has the eclipse extensible api entry in the header. This does not apply to OSGi plugins. During runtime they will be merged anyway with the host plugins.

I'd like to see a preference in the PluginDevelopmentPage property page, where you could define if a particular plugin/bundle should or should not include the contents of the fragment during development time.
Comment 1 Krzysztof Daniel CLA 2011-03-21 04:39:54 EDT
Created attachment 191595 [details]
Mock of the property page
Comment 2 Krzysztof Daniel CLA 2011-04-04 03:21:33 EDT
*** Bug 341738 has been marked as a duplicate of this bug. ***
Comment 3 Eike Stepper CLA 2013-05-03 03:16:40 EDT
Maybe I don't know about all the reasons for this strange Eclipse-ExtensibleAPI header, but I also found out about its existence after hours of debugging the PDE source code, just to see that I can't use fragments because I don't control the host plugin. This is very frustrating. More details about our use case in https://bugs.eclipse.org/bugs/show_bug.cgi?id=400388#c10 .

Why don't fragments at development time behave by default in the same way as at run time?
Comment 4 Eike Stepper CLA 2013-05-03 03:29:53 EDT
BTW. I'd be happy with a new global preference that disables (ideally by default) the hasExtensibleAPI() check in RequiredPluginsClasspathContainer.addDependency():

  BundleDescription[] fragments = hasExtensibleAPI(desc) ? desc.getFragments() : new BundleDescription[0];
Comment 5 Curtis Windatt CLA 2013-05-03 10:14:53 EDT
I'm not familiar with the reasoning behind the ExtensibleAPI header.  It does feel like the header should be ignored when doing OSGi, non-Eclipse, development.  I will mark this bug for consideration in 4.4.  Controlling the behaviour by preference doesn't help a developer figure out why they cannot add new packages to a fragment.
Comment 6 Eike Stepper CLA 2013-05-03 12:20:10 EDT
(In reply to comment #5)
> I'm not familiar with the reasoning behind the ExtensibleAPI header.  It
> does feel like the header should be ignored when doing OSGi, non-Eclipse,
> development.  I will mark this bug for consideration in 4.4.  

If I quickly attach a high-quality patch, is there a chance to have it included in Kepler?

> Controlling
> the behaviour by preference doesn't help a developer figure out why they
> cannot add new packages to a fragment.

I don't understand this assertion.

Firstly I don't have issues with adding new packages and I see nobody complaining about that. Why do you mention "new packages" explicitely?

Secondly why do you think that a global preference (e.g., "USE_EXTENSIBLE_API_HEADER") doesn't solve the problem that we were talking about? Please see the code in my comment #4, which could then bewritten as:

  BundleDescription[] fragments = !USE_EXTENSIBLE_API_HEADER || hasExtensibleAPI(desc) ? desc.getFragments() : new BundleDescription[0];

As long as nobody can explain the reasoning behind thise mysterious header, which causes enormous problems, it should either be removed entirely or a preference should be provided to ignore it.
Comment 7 Curtis Windatt CLA 2013-05-03 12:36:30 EDT
(In reply to comment #6)
> If I quickly attach a high-quality patch, is there a chance to have it
> included in Kepler?

No, with M7 finished no more feature work is planned for Kepler.

> > Controlling
> > the behaviour by preference doesn't help a developer figure out why they
> > cannot add new packages to a fragment.
> 
> I don't understand this assertion.

From the header description in Eclipse help:
The Eclipse-ExtensibleAPI is used to specify whether a host bundle allows fragment bundles to add additional API to the host. This header should be used if a host bundle wants to allow fragments to add additional packages to the API of the host.

> Secondly why do you think that a global preference (e.g.,
> "USE_EXTENSIBLE_API_HEADER") doesn't solve the problem that we were talking
> about? Please see the code in my comment #4, which could then bewritten as:

I'm not saying it doesn't solve the problem.  I'm saying that there needs to be a way to drive users to find that preference.  You say you spent hours debugging the PDE source code.  When you first encountered the problem, how would you know that there was a preference available to deal with the issue?

> As long as nobody can explain the reasoning behind thise mysterious header,
> which causes enormous problems, it should either be removed entirely or a
> preference should be provided to ignore it.

I cc'd Tom in case he can chime in on the reasoning behind that header.  Depending on why the header exists, it may make sense to remove support for the header, or at least change the default value to true (so only adding the header with false would restrict fragment API).
Comment 8 Eike Stepper CLA 2013-05-03 12:43:55 EDT
(In reply to comment #7)
> (In reply to comment #6)
> > If I quickly attach a high-quality patch, is there a chance to have it
> > included in Kepler?
> 
> No, with M7 finished no more feature work is planned for Kepler.
> 
> > > Controlling
> > > the behaviour by preference doesn't help a developer figure out why they
> > > cannot add new packages to a fragment.
> > 
> > I don't understand this assertion.
> 
> From the header description in Eclipse help:
> The Eclipse-ExtensibleAPI is used to specify whether a host bundle allows
> fragment bundles to add additional API to the host. This header should be
> used if a host bundle wants to allow fragments to add additional packages to
> the API of the host.
> 
> > Secondly why do you think that a global preference (e.g.,
> > "USE_EXTENSIBLE_API_HEADER") doesn't solve the problem that we were talking
> > about? Please see the code in my comment #4, which could then bewritten as:
> 
> I'm not saying it doesn't solve the problem.  I'm saying that there needs to
> be a way to drive users to find that preference.  You say you spent hours
> debugging the PDE source code.  When you first encountered the problem, how
> would you know that there was a preference available to deal with the issue?
> 
> > As long as nobody can explain the reasoning behind thise mysterious header,
> > which causes enormous problems, it should either be removed entirely or a
> > preference should be provided to ignore it.
> 
> I cc'd Tom in case he can chime in on the reasoning behind that header. 
> Depending on why the header exists, it may make sense to remove support for
> the header, or at least change the default value to true (so only adding the
> header with false would restrict fragment API).
Comment 9 Eike Stepper CLA 2013-05-03 12:44:36 EDT
Sorry!! Wanted to resize the text field and hit Save Changes instead ;-(
Comment 10 Eike Stepper CLA 2013-05-03 12:51:53 EDT
(In reply to comment #7)
> > > Controlling
> > > the behaviour by preference doesn't help a developer figure out why they
> > > cannot add new packages to a fragment.
> > 
> > I don't understand this assertion.
> 
> From the header description in Eclipse help:
> The Eclipse-ExtensibleAPI is used to specify whether a host bundle allows
> fragment bundles to add additional API to the host. This header should be
> used if a host bundle wants to allow fragments to add additional packages to
> the API of the host.

I see. Then it strikes me that I would rather like to report a bug, not an enhancement. With the PDE code in my comment #4 it's not possible to add ANYTHING to the classpath (i.e., not even new classes to the existing packages) of a host bundle unless it declares this manifest header. Would you agree?

> > Secondly why do you think that a global preference (e.g.,
> > "USE_EXTENSIBLE_API_HEADER") doesn't solve the problem that we were talking
> > about? Please see the code in my comment #4, which could then bewritten as:
> 
> I'm not saying it doesn't solve the problem.  I'm saying that there needs to
> be a way to drive users to find that preference.  You say you spent hours
> debugging the PDE source code.  When you first encountered the problem, how
> would you know that there was a preference available to deal with the issue?

If you agree that I should file a bug (as opposed to this enhancement request) that addresses the inability to add new types to the existing packages, then I wouldn't need a preference anymore.

> > If I quickly attach a high-quality patch, is there a chance to have it
> > included in Kepler?
> 
> No, with M7 finished no more feature work is planned for Kepler.

Bug fixes could still be applied to Kepler?
Comment 11 Thomas Watson CLA 2013-05-03 13:52:24 EDT
See an old bug 45767 discussion on why bundle fragments are not automatically added to the class path of bundles that depend on the host.

In short, from the PDE perspective fragments do not provide API by default.  The Eclipse-ExtensibleAPI was a way to signal to PDE that a host wants to allow its fragments to provide API.

(In reply to comment #10)
> I see. Then it strikes me that I would rather like to report a bug, not an
> enhancement. With the PDE code in my comment #4 it's not possible to add
> ANYTHING to the classpath (i.e., not even new classes to the existing
> packages) of a host bundle unless it declares this manifest header. Would
> you agree?
> 

I agree it is not possible in PDE without the Eclipse-ExtensibleAPI header.

But this needs more thought, and I don't think there is enough time for Kepler.

The issue for me is I don't see a good way for the client bundles to depend on these extra classes that may come from the fragment.  The client either uses Import-Package or Require-Bundle for the host and will resolve if only the host is present.  At build time you will fail to build if the fragment is not available, but that is fine because you can control your target and include the fragment to build.  But at runtime the client bundle will have an declared requirement that the fragment is present otherwise they will get class loading errors.  I think this is the main reason PDE does not add fragments to the classpath by default, because fragments do not provide API that can be declared as a requirement.
Comment 12 Thomas Watson CLA 2013-05-03 13:54:59 EDT
(In reply to comment #11)
> But at runtime the client bundle will have an declared requirement that the 
> fragment is present otherwise they will get class loading errors.

That should read ".. will have an UNDECLARED requirement ..."
Comment 13 Curtis Windatt CLA 2013-05-03 15:23:43 EDT
Thanks for the information Tom.  Even if the restriction protects the user from a bad code pattern, PDE provides no indication about what is happening.  The header isn't even available in the manifest content assist.  Instead of silently ignoring the fragment, we could add an error marker on the fragment project and possibly a warning on the project using an API type from a fragment.
Comment 14 Eike Stepper CLA 2013-05-04 02:35:59 EDT
Created attachment 230475 [details]
Our problem in diagrams
Comment 15 Eike Stepper CLA 2013-05-04 02:55:10 EDT
(In reply to comment #11)
> See an old bug 45767 discussion on why bundle fragments are not
> automatically added to the class path of bundles that depend on the host.

That's interesting because Wassim's comment #6 ( https://bugs.eclipse.org/bugs/show_bug.cgi?id=45767#c6 ) is exactly my point:

Fragments have to be on the classpath, if PDE plans on maintaining its claim 
that it provides a development environment that is as close as possible to the 
runtime environment.  This is not negotiable.

> In short, from the PDE perspective fragments do not provide API by default. 

From an OSGi (runtime) perspective they certainly do and I would expect PDE to support this perspective by default.

> The Eclipse-ExtensibleAPI was a way to signal to PDE that a host wants to
> allow its fragments to provide API.

Please have a look at the diagrams I just attached to this bugzilla (attachment 230475 [details]). They show how we wanted to use OSGi fragments to make newer CDO versions compatible with older EMF versions (as per user demands). We've created a fragment that "injects" the new EMF class into older EMF bundles. It works perfectly at runtime but the generated models of our users do not compile at development time because PDE does not include the new EMF class into teir older EMF bundle. Please see https://bugs.eclipse.org/bugs/show_bug.cgi?id=400388#c10 for a more detailed description.

We don't want this artificial PDE perspective on fragments and their host bundles. Especially not if we don't have control over these host bundles.

If there are special cases (as they seem to be discussed in this long bug 45767) then there should/could be exceptional or optional handling of those particular special cases. But it should not be the default as long as it hinders the development of the normal OSGi behaviour.