Bug 140745 - Headless build does not resolve inter-feature dependencies
Summary: Headless build does not resolve inter-feature dependencies
Status: ASSIGNED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: Build (show other bugs)
Version: 3.2   Edit
Hardware: PC Windows XP
: P3 normal with 9 votes (vote)
Target Milestone: ---   Edit
Assignee: pde-build-inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
: 182171 192292 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-05-09 03:10 EDT by jan.koehnlein CLA
Modified: 2022-11-20 13:16 EST (History)
17 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description jan.koehnlein CLA 2006-05-09 03:10:55 EDT
It seems like the dependencies between features are not resolved at all. As a result the build order of features and their contained plugins is wrong, and I get lots of
    <CLASSNAME> cannot be resolved
errors, because required plugins are have not been build yet.

My setting is as follows

product <--- featurex <---+--- feature1 <------- plugin1
                          |        |                |
                          |        |<--dependency-->|
                          |        |                |
                          |        v                v
                          +--- feature2 <------- plugin2

So plugin1 depends on plugin2 and that's why feature1 also depends on feature2.

When I build the product from eclipse using the Eclipse product export wizard on the product, everything works fine. I think that's because the classes are already compiled in the bin folders. In the headless build the plugin1 is compiled first and build fails, because referenced classes from plugin2 cannot be found.

It seems like the build order of features is always alphabetically. Inter-feature dependencies are completely ignored. Everything works fine when I rename feature2 to x.
Comment 1 Pascal Rapicault CLA 2006-05-09 08:33:59 EDT
This is correct, feature dependencies are not used to decide of the compilation order. However I think the order in which features are built is based on the order in which features are included in the containing feature.
Comment 2 jan.koehnlein CLA 2006-05-09 09:19:28 EDT
Unfortunately, the order is alphabetical.

(In reply to comment #1)
> This is correct, feature dependencies are not used to decide of the compilation
> order. However I think the order in which features are built is based on the
> order in which features are included in the containing feature.
> 

Comment 3 Prince Singh CLA 2006-06-15 09:04:12 EDT
Make sure that you are including every required plugin and features in feature.xml.

Also, are you using cvs repo to get the projects or you have them locally already.
In any case, after the build fails, you must check and see if all the required plugins and features are there. If anything is missing then try to find why.

Please reply about your analysis and whether it resolved your problems. You may also try with simple setup first. i.e. one or two interdependent features and plugins and copy them to a seperate location leaving the bin alone. Then build and see. Also, try to use eclipse 3.2, since it has better documentation in the Online Help > Plugin Dev Environment > Tasks > Building Features.
Comment 4 Pascal Rapicault CLA 2006-06-15 17:42:58 EDT
More importantly than better doc, it now log messages like a good ant citizen.
Comment 5 Leo Bayer CLA 2006-08-08 13:04:06 EDT
I have also encountered this issue and found that it is indeed true that the features are ordered alphabetically.  Is there any plan to address this?
Comment 6 Ketan Padegaonkar CLA 2006-12-14 02:44:42 EST
We're having to modify the generated build files manually to get around this problem. Are there any plans to resolve this in 3.3 ?
Comment 7 Pascal Rapicault CLA 2006-12-14 11:50:16 EST
I have not been able to reproduce the problem. I created a feature1 and feature2 included in a featureX and the compile order is the order in which the features are listed in featureX.
Please attach a simple test allowing me to reproduce the problem.
In case of a bug, I would really appreciate if you guys could provide a patch. Of course we would give you help as to where to start, etc.
Comment 8 Bill Kayser CLA 2007-02-16 14:35:01 EST
I have encountered the same problem but with respect to features listed in the product definition.  When two features are listed in the product--say, feature A and feature B, and B requires a to be built first, the dependency is not accounted for in generating the build files as part of the headless build.  It builds the features in the order they appear in the product file.  It could infer the dependency on the basis of B listing A as a required feature.  In this case B also lists the specific plugins in A which are required, so it captures the dependency on multiple levels (to no avail).

Two workarounds for this: 1) Change the order of the features in the product definition.  There's no UI for this and I'm not sure if this change will be sticky.  2) List feature A as embedded in B, and remove A from the product definition.
Comment 9 Pascal Rapicault CLA 2007-04-24 09:47:30 EDT
*** Bug 182171 has been marked as a duplicate of this bug. ***
Comment 10 Andrew Niefer CLA 2007-06-13 08:38:15 EDT
*** Bug 192292 has been marked as a duplicate of this bug. ***
Comment 11 Nick Boldt CLA 2007-06-13 10:38:02 EDT
(In reply to comment #7)
> Please attach a simple test allowing me to reproduce the problem.
> In case of a bug, I would really appreciate if you guys could provide a patch.

See bug 192292 for a simple example where source feature is listed before runtime feature in SDK's feature.xml, and therefore source feature is built, but is empty (because source doesn't yet exist). This was found yesterday when debugging the RAP project's build. Shouldn't be too hard to set up an SDK feature, source feature, and runtime feature with minimal plugin content just for the sake of testing. 
Comment 12 Alejandro Bascuas CLA 2008-03-27 11:44:23 EDT
Not sure if this is still being paid attention to, but I'm running into this problem now. Manually ordering the features in the product configuration file works, however, this must be done using the text editor. After setting the order, if you open the file later using the Product Configuration Editor, it will sort the features alphabetically. Here is the simple test case:

1) Edit a product file as text, and reorder the features in a non-alphabetical order.

2) Open the product file in the Product Configuration Editor

3) Note that features are now alphabetical

When editing a feature.xml file, included features are also always displayed alphabetically, however if you change the order manually, that sticks even after adding new features via the UI. Not sure how this affects build order since the first workaround works.

It seems like many headaches can be avoided by not sorting the features, and providing appropriate buttons to change the order in the UI of the Product Configuration Editor (new bug?). However, an ideal fix would be to add logic to determine the correct order to build features.

(In reply to comment #8)
> Two workarounds for this: 1) Change the order of the features in the product
> definition.  There's no UI for this and I'm not sure if this change will be
> sticky.  2) List feature A as embedded in B, and remove A from the product
> definition.
> 

Comment 13 Nick Boldt CLA 2008-04-23 14:33:30 EDT
> However, an ideal fix would be to add logic to
> determine the correct order to build features.

Considering that this resolution can be done at the plugin level (ie., figure out what order to build plugins for a given feature) I'd expect the same to be true for the recursive case of a feature containing a feature which in turn contains other plugins (and/or more features).

If you need a complex use case for testing, look at emf.all {emf.sdk {emf.* features}, xsd.sdk {xsd.* features}, sdo.sdk {sdo.* features}}... and each feature, naturally, contains 1 or more plugins. Or, GEF, with its .all { *.sdk { others }} nesting.

While this isn't as painful and in-your-face as recently squashed bug 109137, it's a very hard to diagnose problem for all us lowly releng folks. Frequently encountered enough that it lives in my releng FAQ here:

http://wiki.eclipse.org/Modeling_Project_Releng/Building#SomeBundleOrClass_cannot_be_resolved_--_compilation_fails
Comment 14 Micah Hainline CLA 2008-05-22 17:54:57 EDT
Currently PDE/Build tries to build each feature that is included one at a time, and then builds any included plugins after.  We have a problem in that it doesn't know which feature to build first, but I think the underlying problem is that we are building the features one at a time rather than aggregating them.  Anyone that doesn't need to be convinced of this, skip the next two paragraphs.

Take a feature Foo that represents a nice stand-alone product which incorporates org.eclipse.rcp along with some other plugins.  Plugin A adds some extra functionality, and can be bundled with Foo.  Feature Bar adds some additional functionality to A and lists it as a dependency, but A is not included in feature Bar.  Feature FooBar includes features Foo and Bar, as well as plugin A. 

FooBar can't be built with headless PDE/Build because Bar can't be built without building A, and A won't be built until after all the included features are built.  This is just one fairly simple scenario, but there are many others.

The solution then is to do something like this:
Collect all included features, recursing down the hierarchy.  Grab all plugins out of all the features, and then build them together, using standard plugin dependency rules to determine which to build first.  This way including the a feature in another feature would be no different than including each of the plugins from the feature individually, and I am fairly certain that is what we want.

Comment 15 Micah Hainline CLA 2009-05-20 16:08:50 EDT
As we pass the one year mark since the last action on this bug, I was wondering if there are any updates, or if anyone has a comment on the solution I proposed last year.
Comment 16 Nick Boldt CLA 2009-05-20 16:43:49 EDT
(In reply to comment #15)
> As we pass the one year mark since the last action on this bug, I was wondering
> if there are any updates, or if anyone has a comment on the solution I proposed
> last year.

Have you tried these in your build.properties (works with PDE and basebuilder after 3.5M5)?

flattenDependencies=true
parallelCompilation=true

http://aniefer.blogspot.com/2008/10/sorting-bundles-and-parallel.html

There's also an option to allow binary cycles but I can't recall the property for that.
Comment 17 Lars Vogel CLA 2018-12-03 09:10:50 EST
Currently we are not actively enhancing PDE build anymore. Therefore, I close this bug as WONTFIX. 

Please reopen, if you plan to provide a fix.
Comment 18 Lars Vogel CLA 2018-12-03 09:11:45 EST
Currently we are not actively enhancing PDE build anymore. Therefore, I close this bug as WONTFIX. 

Please reopen, if you plan to provide a fix.
Comment 19 Eclipse Genie CLA 2020-11-29 14:01:51 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 20 Eclipse Genie CLA 2022-11-20 13:16:21 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.