Bug 570760 - [product editor] allow an option for feature based product to include required bundles
Summary: [product editor] allow an option for feature based product to include require...
Status: VERIFIED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.18   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 4.24 M3   Edit
Assignee: Hannes Wellmann CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 340350 539637 544838
Blocks:
  Show dependency tree
 
Reported: 2021-01-29 11:22 EST by Christoph Laeubrich CLA
Modified: 2022-05-17 18:53 EDT (History)
3 users (show)

See Also:


Attachments
Enhanced Product Editor (36.88 KB, image/png)
2022-03-28 17:27 EDT, Hannes Wellmann CLA
no flags Details
Simple m2e product (2.71 KB, application/x-zip-compressed)
2022-03-30 05:03 EDT, Hannes Wellmann CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Christoph Laeubrich CLA 2021-01-29 11:22:42 EST
Assume I have a target that resolves well in "planner-mode", that target provides me with some features. Now I add some features to a feature-based product.

What now can happen that the features require additional bundles that are not covered by my current feature set (e.g. add required do not find any feature that includes them) but my target platform does include these.

Instead of define a "pseudofeature" that includes all these bundles, I'd like to have a checkbox next to the "the product configuration is based on" that reads like 
[x] include required plugins to the product

This will then behave like the planner-mode in the target, if PDE can find these required bundles they will silently be included.

This feature can greatly help in complex deployment setups (e.g. eclipse packages itself) and prevent one from either define the product based on bundles or define additional features just for the sake of add necessary dependencies.
Comment 1 Hannes Wellmann CLA 2021-07-26 14:43:18 EDT
This is a great feature suggestion, which would have saved me many hours if not even days of monotonous work finding out which feature includes a required plug-in or package.
It would be really helpful if we could avoid this! Tycho automatically includes required bundles into a product and has an option to auto-include dependencies into the product repository ("update-site") as well, so both the product and its repository can be made self-contained automatically.
So the only reason I have to make my product self contained product is to be able to launch it from the product-editor's overview page without further adjustments of the launch config.

Therefore I really would like to make this happen and offer to implement it.

My first thought was to combine this bug with Bug 544838. Then the check-box "include required plugins to the product" would only cause a corresponding boolean flag in the launch config respectively a corresponding check-box in the Launch-Editor to be set/checked. When launched all required plug-ins would silently be added to the launching product.

However when starting to prototype a solution I noticed that if one selects in the Launch-Editor's "Plug-ins" tab of an Eclipse-Application "Launch with: 'features selected below'" and you select only the features of your product (plus the auto-started plug-ins) PDE already includes all required bundles into the launching product.

The only change for that is setting the boolean attribute "useCustomFeatures=true". When changing to a feature based launch in the editor the string attributes "featureDefaultLocation" and "featurePluginResolution" are also set to "workspace" (but this is the default, so it is not necessary).

In addition I think it could be considered to only set the "selected_features" to the features included in the product and leave "selected_workspace_bundles" and "selected_target_bundles" empty.
This avoids an over-specification of the launch-config, which is bad when the launch-config should be committed into version control.
Furthermore I get validation errors, when I change a launch-config created from a product to be feature based, because fragments for other platforms are included (maybe the Eclipse-Platform filter is not considered).

Actually I would expect a feature based product to be specified by features in a launch-config by default any ways. But this would change the default behaviour to include all dependencies.

So in order to implement this I suggest to change to set the attribute "useCustomFeatures=true" and additionally to include only the features included into the product, when a product is launched.
Comment 2 Julian Honnen CLA 2021-07-27 03:02:56 EDT
(In reply to Hannes Wellmann from comment #1)
In bug 548677 I actually started this, but not as default because we didn't want to change the existing behavior back then.

> Furthermore I get validation errors, when I change a launch-config created
> from a product to be feature based, because fragments for other platforms
> are included (maybe the Eclipse-Platform filter is not considered).
Feature based launch configs also have issues when multiple versions of a bundle exist and the features don't specify exact versions (bug 539637).

Before we can switch the default like you suggest, that launch mode must be stable. Real-world problems tend to surface only after switching the default though ... 

But I agree that this is the correct approach to solve this bug's usecase.
Comment 3 Julian Honnen CLA 2021-07-27 03:13:32 EDT
(In reply to Julian Honnen from comment #2)
> But I agree that this is the correct approach to solve this bug's usecase.
Sorry, it seems I got lost in Hannes analysis and ignored your request, Christoph.

You actually want to focus on build time, not on launching from the IDE right?
IMO there are no constraints from PDE to modify the product definition, because we would only have to touch the editor UI (as PDE *build* is not maintained anymore).
So the heavy lifting would be on tycho.

I'm also open to just making this planner-mode to be the default or even to just change the behavor without a configuration option because PDE's launch config already behaves this way.
But again, this is a question the tycho project should answer.
Comment 4 Christoph Laeubrich CLA 2021-07-27 04:56:52 EDT
If Hannes likes to implement this it would be great, just one clarification

(In reply to Hannes Wellmann from comment #1)
> So the only reason I have to make my product self contained product is to be
> able to launch it from the product-editor's overview page without further
> adjustments of the launch config.

I know there is some great demand to make things "magically happen" and my request foes in this direction, but I like to clarify, because I feel this is a common misunderstanding, that a product / osgi-launch is not *required* to be self-contained and this is an absolute valid use-case, so this should *always* be possible and the auto-magic be an option.

While in an update-side it is (maybe) acceptable to include even optional items, in a product this might not be desired and even to qualify what is optional or not can't be determined reliable.

Beside that, I always would advocate for writing better, self-describing (!) features as degrading them to just a container to loosely group some arbitrary bundles really does not reveal its power. Where a well-crafted manifest and dependency management makes a good bundle, a well-crafted and good sliced features makes using them much more profitable.
Comment 5 Christoph Laeubrich CLA 2021-07-27 05:17:09 EDT
(In reply to Julian Honnen from comment #3)
> (In reply to Julian Honnen from comment #2)
> > But I agree that this is the correct approach to solve this bug's usecase.
> Sorry, it seems I got lost in Hannes analysis and ignored your request,
> Christoph.
> 
> You actually want to focus on build time, not on launching from the IDE
> right?

I have learned that changes are better done from UI->Tycho so this is specifically about PDE supporting this.

I'd like to invite everyone with a little experiment (maybe even a good test-case):

- create an empty target with current release [1] and include: SDK, JDT, M2E and Target Components
- set it as a target and create a new product definition
- now try to create an feature based eclipse-product that is an IDE with m2e support (don't cheat and copy the EPP packages ;-))

This is nearly impossible to get a satisfiable solution (at least for me)! E.g. one would probably start with adding the m2e-core feature and add the SDK feature but sooner or later one is completely lost as there are required bundles/packages that are not part of any (obvious) feature and even PDE fails to help here with adding required stuff!

It works a bit better with a launch config, by using the current-installation (!) as part of the target (${eclipse-home})...

So what I would expect is something like:
- I add m2e-core + ... (what ever I'd like to have from a use-case(!) perspective e.g. m2e.editor feature)
- I'd like to press the "Engage!" button and starting my product with everything "planned" (even though it might take some more time as the engines need to heat up) whether or not required stuff has features or not.

Of course this still could fail to find a solution, but it won't require me to find out in what dam bundle a package resides, and if I found that what crazy feature (if any) includes that bundle and the starting all over again because there is now another unmeet requirement...


[1] http://download.eclipse.org/releases/2021-06/
Comment 6 Hannes Wellmann CLA 2021-07-27 07:15:26 EDT
(In reply to Christoph Laeubrich from comment #5)
> 
> I'd like to invite everyone with a little experiment (maybe even a good
> test-case):

I made this experiment already by myself when setting up my companies product, so I know the frustrating work to get a result. Therefore I thought about a possible solution and found this bug. And I fully agree, that you should be able to specify a feature based product, hit the launch button and everything should work (maybe you have to check that you want to include required dependencies).


(In reply to Christoph Laeubrich from comment #4)
> I know there is some great demand to make things "magically happen" and my
> request foes in this direction, but I like to clarify, because I feel this
> is a common misunderstanding, that a product / osgi-launch is not *required*
> to be self-contained and this is an absolute valid use-case, so this should
> *always* be possible and the auto-magic be an option.

I agree on that it should always be possible to choose if required plug-ins/features should be automatically included or not. And it can be a valid use-case to not include e.g. optional dependencies. However if a dependency is required and used then it has to be included. Otherwise your execution just fails with a ClassNotFoundException or similar.


> Beside that, I always would advocate for writing better, self-describing (!)
> features as degrading them to just a container to loosely group some
> arbitrary bundles really does not reveal its power. Where a well-crafted
> manifest and dependency management makes a good bundle, a well-crafted and
> good sliced features makes using them much more profitable.

I agree that features should be crafted wisely and not arbitrarily. However I cannot include all required plug-ins and features into a feature. Pretend a plug-in requires PDE, the I had to include (PDE, JDT and the Platform). If I build an update site for that I would just unnecessarily mirror a lot from Eclipse that is usually present in a installation any ways. But that was probably not what you meant with self-describing (compared to self-contained).

But let's put that general discussion aside and focus on the goal of this bug. :)  

(In reply to Julian Honnen from comment #3)
> You actually want to focus on build time, not on launching from the IDE
> right?

That's right. Launching from the IDE shouldn't make things more complicated.

> IMO there are no constraints from PDE to modify the product definition,
> because we would only have to touch the editor UI (as PDE *build* is not
> maintained anymore).
> So the heavy lifting would be on tycho.
> 
> I'm also open to just making this planner-mode to be the default or even to
> just change the behavor without a configuration option because PDE's launch
> config already behaves this way.
> But again, this is a question the tycho project should answer.

As far as I can see Tycho already provides everything I need (Christoph please correct me if I'm wrong). When building products Tycho collects all included and required(!) plug-ins and features (included features definitely but I'm not 100% sure about required features) like in the PDE planner mode. Besides that the "tycho-p2-repository-plugin" can be configured to "includeAllDependencies" so the product repository is also self-contained. At least I am satisfied with that.
Comment 7 Hannes Wellmann CLA 2021-07-27 07:18:09 EDT
(In reply to Julian Honnen from comment #2)
> (In reply to Hannes Wellmann from comment #1)
> In bug 548677 I actually started this, but not as default because we didn't
> want to change the existing behavior back then.
> 
> > Furthermore I get validation errors, when I change a launch-config created
> > from a product to be feature based, because fragments for other platforms
> > are included (maybe the Eclipse-Platform filter is not considered).
> Feature based launch configs also have issues when multiple versions of a
> bundle exist and the features don't specify exact versions (bug 539637).
> 
> Before we can switch the default like you suggest, that launch mode must be
> stable. Real-world problems tend to surface only after switching the default
> though ... 
> 
> But I agree that this is the correct approach to solve this bug's usecase.

Thanks for the link to that bug. I also encountered that problem. I can have a look at that too, but we should discuss it there.
And yes I agree that it should definitely work before we change any default.

Furthermore I got the impression (only from looking at the code, so maybe I missed something), that features included in other features are not considered in BundleLauncherHelper.getMergedBundleMapFeatureBased() and their content is therefore not added to the launch? Is there already an existing bug for that?

From the launching perspective I think most of the relevant logic is in:

- org.eclipse.pde.internal.core.DependencyManager
- org.eclipse.pde.internal.launching.launcher.BundleLauncherHelper
- org.eclipse.pde.launching.EclipseApplicationLaunchConfiguration

Once all bugs are fixed I think "Eclipse-Application" launch configurations could work the following:
- They can based either on
  - features, then the set of included features is specified in the launch-config (plus the additional_plugins)
  - plug-ins, then the set of included plug-ins specified in the launch-config
This is already implemented (even tough I find the boolean switch "useCustomFeatures" not verify clear, but that's not a real issue.

- It can be chosen if required dependencies should be included or not:
  - for plug-in based launches this is requested with Bug 544838
  - for feature based launches required plug-ins of directly contained plug-ins are always already added automatically, 
    but nested features seem not be considered as well as dependencies to other features.
I think this should be the following way:
  - if dependencies should NOT be included only the plug-ins or features (and their content) that are explicitly specified should be included
    (for features this includes nested features IMO).
  - if dependencies should be added automatically, all plug-ins and features that a contained plug-in or feature has a dependency to should be included as well.
    - all optional dependencies can be excluded if the corresponding checkbox is unchecked, if checked they are included if present.
This probably requires a new boolean attribute for launch-configs.

The product editor should then work the following way:
  - for a plug-in based product a plug-in based launch is created that contains exactly those plug-ins specified in the product
  - for a feature based product a feature based launch is created that contains exactly those features specified in the product
  + (in both cases) the additional plug-ins that are specified in the configuration tab of the product (this seems to be already the case).
  - there should be the checkbox requested initially to include dependencies automatically, whose value is just translated to the corresponding boolean attribute of the launch config.

Once the described improvements for launches are implemented exposing this to the product editor should straight forward and easy to implement.
Furthermore we could think about if and how automatically added plug-ins and features are displayed in the launch-config editor and if validation is still required or if adding required dependencies is "self-validating"
Comment 8 Julian Honnen CLA 2021-07-27 07:38:08 EDT
(In reply to Hannes Wellmann from comment #7)
> Furthermore I got the impression (only from looking at the code, so maybe I
> missed something), that features included in other features are not
> considered in BundleLauncherHelper.getMergedBundleMapFeatureBased() and
> their content is therefore not added to the launch? Is there already an
> existing bug for that?
I haven't tried it, but I also don't see any handling for either included features or feature dependencies.
I didn't find any existing bug for that earlier.

> - It can be chosen if required dependencies should be included or not:
IMO we don't need any further configuration option for feature-based launches and the point above is simply a bug.
AFAIK, feature-based launches were intended to be an easier-to-use alternative to plugin-based launches that just works without fiddling.
In that spirit, always including required dependencies seems like the correct behavior.
Comment 9 Christoph Laeubrich CLA 2021-07-27 09:19:33 EDT
(In reply to Hannes Wellmann from comment #6)
> However if a  dependency is required and used then it has to be included.

Nope thats what I mean, wit "common misunderstanding" that a product only has to do *one* job and if not is "broken", OSGi is not designed that way, it is meant to be a platform for cooperative modules. It can offer a variety of functions, that might be available from start on, later installed, enhanced at any time. 

> Otherwise your execution just fails with a ClassNotFoundException or similar.

Nope, see above, a bundle in INSTALLED state is absolutely valid (and because it do not get RESOLVED will never throw CNFE as no classloader exits for it!). Beside that there are a variety of 'requirements' or more specifically Capabilities. These can be packages, extenders, services, configurations or  hardware or JVM itself.

So I can build an OSGi product that is not satisfied and certain functionality is not available until a specific event happens, e.g. one might install a bundle that is licensed in a special way, so I simply not allowed to include it in my product (Oracle has some strange licensing term for some of there JDBC drivers AFAIK). Maybe it would require to have some extender in place think about fragment.e4xmi that offers some Part to the e4 model:

- If I put that bundle inside an e4 app it will be loaded and part is shown
- If I put it in a framework not running e4 it will be ignored

Still the bundle itself would have some value because it may register services, or offers extension-points for traditional e3 apps...

So is e4 required? is it optional? Is e3 required? Should a full blown e4 dependency chain be included just because I'd like to use the services it offers? Should maybe e3 *and* e4 be included (even though both do not play well together).

> As far as I can see Tycho already provides everything I need (Christoph
> please correct me if I'm wrong). When building products Tycho collects all
> included and required(!) plug-ins and features (included features definitely
> but I'm not 100% sure about required features) like in the PDE planner mode.

You best try it out but from my knowledge and working with tycho in the past years this is not true. There is a special 'validate-product' mojo that do check if the product is 'complete' but without it only the features and its declared dependencies are included but not any additional (bundle or feature) dependencies.

> Besides that the "tycho-p2-repository-plugin" can be configured to
> "includeAllDependencies" so the product repository is also self-contained.
> At least I am satisfied with that.

includeAllDependencies works for most cases but only for "repositories" and it sometimes includes more than desired.
Comment 10 Hannes Wellmann CLA 2021-07-27 11:41:25 EDT
(In reply to Julian Honnen from comment #8)
> (In reply to Hannes Wellmann from comment #7)
> > Furthermore I got the impression (only from looking at the code, so maybe I
> > missed something), that features included in other features are not
> > considered in BundleLauncherHelper.getMergedBundleMapFeatureBased() and
> > their content is therefore not added to the launch? Is there already an
> > existing bug for that?
> I haven't tried it, but I also don't see any handling for either included
> features or feature dependencies.
> I didn't find any existing bug for that earlier.

Me neither. Alright. So I will first handle Bug 539637 and then open another bug for the mentioned problem.

> 
> > - It can be chosen if required dependencies should be included or not:
> IMO we don't need any further configuration option for feature-based
> launches and the point above is simply a bug.
> AFAIK, feature-based launches were intended to be an easier-to-use
> alternative to plugin-based launches that just works without fiddling.
> In that spirit, always including required dependencies seems like the
> correct behavior.

Most of the time (until now actually always), I want that behaviour and I agree with you. However I made the experience that not everybody has the same use cases as I do and as Christoph pointed out, it can be perfectly valid to have an application that does not satisfy all dependencies. And since a checkbox to configure that behaviour does not cost us much, I suggest to add one.
However I suggest to make it the default to include required dependencies for feature based products/launches.
Maybe we should shift that discussion to Bug 544838?

I think the main logic should go to the launching facility and the product editor should just be adjusted to create corresponding 'pure' feature or plug-in based launch configs depending on the product and should copy the attribute to include dependencies automatically or not.
Comment 11 Eclipse Genie CLA 2021-10-22 12:49:15 EDT
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/186834
Comment 12 Hannes Wellmann CLA 2022-03-27 07:35:02 EDT
The implementation is more or less ready. Please see the attached Gerrit.
@Christoph does this satisfy your needs/intention?

I'm a bit unhappy about the fact that the option if requirements should be added automatically is stored in the product file although it only affects the created launch configuration and is nothing that is relevant when when the product is build. But I think it should be persisted so that one does not have to (un)check the box if the default (checked for feature based products and unchecked for plug-in based products) is not desired.

I just got the idea that instead of storing the option in the product model the Overview-Page could be searched for an existing associated launch-configuration and the corresponding attribute value stored there could be taken.

@Christoph, @Julian what do you think? Do you have a better idea?

Furthermore do you know if the UI changes in the OverviewPage could be done better? I have not yet worked with Eclipse UI forms and my current approach seems to be cumbersome.
Comment 13 Christoph Laeubrich CLA 2022-03-27 09:48:41 EDT
I think it should be persisted in the product and clearly belongs there as it influence if/how the product needs to be handled. We most probably would also need tycho to handle it in some way as well...
Comment 14 Hannes Wellmann CLA 2022-03-27 14:33:15 EDT
(In reply to Christoph Laeubrich from comment #13)
> I think it should be persisted in the product and clearly belongs there as
> it influence if/how the product needs to be handled. We most probably would
> also need tycho to handle it in some way as well...

If this can considered by Tycho then I agree that it belongs into the product file, but I wonder if the P2-Director can not include required bundles/feature when installing/assembling a product? At least in the installation UI I never noticed such an option, but actually I never looked for it because I usually want to install all requirements.

The only thing I know in this area that can decide to include requirements or not is the 'assemble-repository' Mojo of the 'tycho-p2-repository-plugin' (https://www.eclipse.org/tycho/sitedocs/tycho-p2/tycho-p2-repository-plugin/assemble-repository-mojo.html#includeAllDependencies)

If this attribute is stored in the product file and also affects how the product is build by Tycho, then the check-box should not be placed in the Testing section, probably the Product Definiton section next to the "plug-ins" or "feature" radio would be better.

And then the attribute should always be written into the product file. At the moment it is only written when different from the default to modify existing products less.
Comment 15 Christoph Laeubrich CLA 2022-03-28 00:53:42 EDT
I think it doesn't mater much what Tycho supports *now* as there will always be a small (time) gap between Tycho <-> PDE and I found it most useful to first have a feature in PDE and then in Tycho. Anyways it might be also that one will add support to the P2-ProductAction that tycho uses internally so tycho has only to update to latest P2 then...

So I think the attribute should always stored inside the product.
Comment 16 Hannes Wellmann CLA 2022-03-28 17:27:26 EDT
Created attachment 288307 [details]
Enhanced Product Editor
Comment 17 Hannes Wellmann CLA 2022-03-28 17:30:02 EDT
(In reply to Christoph Laeubrich from comment #15)
> I think it doesn't mater much what Tycho supports *now* as there will always
> be a small (time) gap between Tycho <-> PDE and I found it most useful to
> first have a feature in PDE and then in Tycho. Anyways it might be also that
> one will add support to the P2-ProductAction that tycho uses internally so
> tycho has only to update to latest P2 then...
> 
> So I think the attribute should always stored inside the product.

My concern is not the timely gap between PDE and Tycho, that's totally fine. I just wonder if it is in general possible at all (at least without major effort) to assemble a product with P2 that does not satisfy all requirements of all installed Plug-ins?

However I'm fine to store it in the product. Using a Launch-Config would be difficult to handle in case multiple launch-configs exists for the same product. And maybe this could also influence the default value of the assemble-repository-mojo's includeAllDependencies property (although this is not exactly what the new product attribute describes).

The latest patch-set act accordingly and I moved the checkbox to the "Product Definition" section. I choose the labels based on the discussion in Bug 544838.

Please find attached a screenshot with an excerpt of the new page.
Comment 18 Christoph Laeubrich CLA 2022-03-28 22:04:50 EDT
(In reply to Hannes Wellmann from comment #17)
> My concern is not the timely gap between PDE and Tycho, that's totally fine.
> I just wonder if it is in general possible at all (at least without major
> effort) to assemble a product with P2 that does not satisfy all requirements
> of all installed Plug-ins?

I'm a bit confused maybe we are talking about different things here? At the moment, you can create a tycho/p2 product that has "missing" plugins without a problem, thats the default and as discussed earlier also a valid configuration.

So the new option/flag should in contrast create a product that has all requirements satisfied and that should be possible with p2 as well.

> However I'm fine to store it in the product. Using a Launch-Config would be
> difficult to handle in case multiple launch-configs exists for the same
> product. And maybe this could also influence the default value of the
> assemble-repository-mojo's includeAllDependencies property (although this is
> not exactly what the new product attribute describes).

I think the flag should be handled by the ProductAction of P2 then we probably don't need to do anything in tycho.
Comment 19 Julian Honnen CLA 2022-03-29 02:21:53 EDT
(In reply to Christoph Laeubrich from comment #18)
> I think the flag should be handled by the ProductAction of P2 then we
> probably don't need to do anything in tycho.
Could you please try that in a prototype, Hannes? I'd rather not add some API-like option to PDE if it's only going to be used for launch config creation.
Comment 20 Hannes Wellmann CLA 2022-03-30 05:03:18 EDT
(In reply to Christoph Laeubrich from comment #18)
> (In reply to Hannes Wellmann from comment #17)
> > My concern is not the timely gap between PDE and Tycho, that's totally fine.
> > I just wonder if it is in general possible at all (at least without major
> > effort) to assemble a product with P2 that does not satisfy all requirements
> > of all installed Plug-ins?
> 
> I'm a bit confused maybe we are talking about different things here? At the
> moment, you can create a tycho/p2 product that has "missing" plugins without
> a problem, thats the default and as discussed earlier also a valid
> configuration.
> 
> So the new option/flag should in contrast create a product that has all
> requirements satisfied and that should be possible with p2 as well.

From my observation it is possible to define a product based on feature's that has "missing" Plug-ins, i.e. where not all dependencies (Require-Bundle, Import-Package, etc.) of the Plug-ins (transitively) contained in the specified features (or the features required by them). But when such Product is build by Tycho those missing Plug-ins, although they are not contained in any feature, are also installed into the product.

I have created a simple product based on your experiment from comment 5 (btw. it is now very simple in the IDE :) ).
Please find it attached. The target platform for it contains only the eclipse.sdk, the equinox.executable and the m2e.sdk and the product only consist of the two features
- org.eclipse.sdk
- org.eclipse.m2e.feature

As expected this results in an Eclipse-IDE where you can work with Maven, Plug-in or plain Java projects, for both cases when you build the product with Tycho and use the build product or when you launch it from within the IDE (because including requirements is the default for Feature based launches for a long time this already works even without this change with the current nightly build, but you can now also launch only with the included plug-ins).

I compared the "bundles.info" file of the simple.configurator for both cases and they are (almost) identical. The only difference is that "biz.aQute.bndlib" and "biz.aQute.bnd.util" are also present in the launch from the IDE. I'm not exactly sure why.
However when I launch this product from the IDE I can observe that missing requirements are added in the end by computing the bundles requirement closure, which adds for example "org.apache.xml.resolver" or "com.google.gson". So those bundles seem to be not included in any feature but are still installed into the product (when launching from the IDE and by Tycho/P2).

So what I'm trying to say is, that from my observations Tycho respectively P2 already installs all required dependencies (features or Plug-ins) although they are not specified explicitly.
So when we add a flag to control this behaviour it looks like the part for includeRequirements=false has to be implemented.

And I cannot say if this is a simple task or not. Actually this should be the simplar part than the other way round, but I suspect that other parts could rely on a fully satisfied installation.

> > However I'm fine to store it in the product. Using a Launch-Config would be
> > difficult to handle in case multiple launch-configs exists for the same
> > product. And maybe this could also influence the default value of the
> > assemble-repository-mojo's includeAllDependencies property (although this is
> > not exactly what the new product attribute describes).
> 
> I think the flag should be handled by the ProductAction of P2 then we
> probably don't need to do anything in tycho.

That's a reasonable suggestion.


(In reply to Julian Honnen from comment #19)
> (In reply to Christoph Laeubrich from comment #18)
> > I think the flag should be handled by the ProductAction of P2 then we
> > probably don't need to do anything in tycho.
> Could you please try that in a prototype, Hannes? I'd rather not add some
> API-like option to PDE if it's only going to be used for launch config
> creation.
I agree and I can try to do that, but I'm not yet familiar with the P2 code so it will likely take some time and I won't be able to complete that before the move to GitHub (which is IMO not a problem, I will then just migrate the change).
Comment 21 Hannes Wellmann CLA 2022-03-30 05:03:47 EDT
Created attachment 288331 [details]
Simple m2e product
Comment 22 Christoph Laeubrich CLA 2022-03-30 05:17:10 EDT
(In reply to Hannes Wellmann from comment #20)
> So what I'm trying to say is, that from my observations Tycho respectively
> P2 already installs all required dependencies (features or Plug-ins)
> although they are not specified explicitly.

Then this is a bug in tycho/p2 as you have proven PDE do not handle it that way and it should be possible.

Anyways I also noticed cases where incomplete products are produced, so maybe its already configurable.

> So when we add a flag to control this behaviour it looks like the part for
> includeRequirements=false has to be implemented.

as mentioned above this should be a property of the product, not the pom configuration, but we don't need to care from PDE side about this.

Anyways, having an integration test for this in tycho later with the various options would be the most suitable place to discuss/change this.
Comment 23 Hannes Wellmann CLA 2022-03-31 18:35:32 EDT
(In reply to Christoph Laeubrich from comment #22)
> (In reply to Hannes Wellmann from comment #20)
> > So what I'm trying to say is, that from my observations Tycho respectively
> > P2 already installs all required dependencies (features or Plug-ins)
> > although they are not specified explicitly.
> 
> Then this is a bug in tycho/p2 as you have proven PDE do not handle it that
> way and it should be possible.

It should be possible, yes. Naively said the bundles.info for the SimpleConfigurator just have to contain the Plug-ins transitivly included in the features.

> 
> Anyways I also noticed cases where incomplete products are produced, so
> maybe its already configurable.

I also wonder if P2 and PDE could share the code to compute the content of a Product? I.E. PDE would then have to use the code of P2 and maybe the code in P2 would have to be adjusted to handle primary and secondary sources of PDE.
At least my primary goal was to create Product launches from the IDE that behave as much as possible like the product build by Tycho.

> 
> > So when we add a flag to control this behaviour it looks like the part for
> > includeRequirements=false has to be implemented.
> 
> as mentioned above this should be a property of the product, not the pom
> configuration, but we don't need to care from PDE side about this.

I was referring to the property of the product "autoIncludeRequirements" and was to lazy to fully type it and was therefore unclear. :)

> 
> Anyways, having an integration test for this in tycho later with the various
> options would be the most suitable place to discuss/change this.

I agree with you. Actually the attached example is not too bad, but I think this takes to long for an integration test.

Besides that I wonder if we autoIncludeRequirements=true should be used as default for Plug-in and Feature based launches, because this is the ProductActions default for both cases, isn't it? I did not yet check it.
Comment 24 Hannes Wellmann CLA 2022-05-16 14:43:30 EDT
It was a long (and maybe not the fastest) way but this is finally completed. :)
Many aspects of launching Products and applications and Eclipse applications were improved on that way.
The last step was done with PR https://github.com/eclipse-pde/eclipse.pde/pull/71
Comment 25 Vikas Chandra CLA 2022-05-17 00:57:29 EDT
(In reply to Hannes Wellmann from comment #24)
> It was a long (and maybe not the fastest) way but this is finally completed.
> :)
> Many aspects of launching Products and applications and Eclipse applications
> were improved on that way.
> The last step was done with PR
> https://github.com/eclipse-pde/eclipse.pde/pull/71

Do we want to add a N&N now?
Comment 26 Hannes Wellmann CLA 2022-05-17 01:53:42 EDT
Verified with Eclipse SDK

Version: 2022-06 (4.24)
Build id: I20220516-1800

I'm working on a N&N entry.
Comment 27 Hannes Wellmann CLA 2022-05-17 18:53:22 EDT
PR for the N&N entry:
https://github.com/eclipse-platform/www.eclipse.org-eclipse-news/pull/24