Bug 236719 - [plan] [p2] Leverage API for installed bundles and source bundles
Summary: [plan] [p2] Leverage API for installed bundles and source bundles
Status: RESOLVED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.5   Edit
Hardware: PC Windows XP
: P3 enhancement (vote)
Target Milestone: 3.5 M6   Edit
Assignee: Curtis Windatt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 250126
Blocks: 240705
  Show dependency tree
 
Reported: 2008-06-11 15:42 EDT by Darin Wright CLA
Modified: 2009-03-19 10:21 EDT (History)
9 users (show)

See Also:


Attachments
Quick patch using example code (7.29 KB, patch)
2008-09-22 15:24 EDT, Curtis Windatt CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Darin Wright CLA 2008-06-11 15:42:50 EDT
PDE (JDT, and others) would like an API that provides installed bundles and source bundles for a given profile. 

PDE also needs an API to obtain the current/active profile such that we can create a "target plaform with attached source" that corresponds to the running SDK.

Ideally we could use the same API for the running profile vs. other p2 provisioned installs - such that target platforms can be built by pointing to other install locations corresponding to different versions of the SDK, RCP apps, etc.

If such an API already exists, please point us in the right direction.
Comment 1 John Arthorne CLA 2008-06-11 16:13:48 EDT
We have no API now, but we should be able to do this in 3.5.
Comment 2 Pascal Rapicault CLA 2008-06-11 22:37:45 EDT
We will have to be extremely careful as how we do this, since p2 only reason in terms of IU and does not know it is installing bundles, source bundles, etc and it should not.
We may need to add additional adaptation / introspection mechanism bridging the two worlds (bundles and IUs). This also fit into a more general class of problem usually relating to system status, etc.
Comment 3 Sudarsha Wijenayake CLA 2008-07-24 13:11:14 EDT
Is there a chance for this proposed API to be made available in 3.4.1 instead of 3.5?
Comment 4 Jeff McAffer CLA 2008-07-29 22:32:50 EDT
seems unlikely
Comment 5 John Arthorne CLA 2008-08-20 15:14:06 EDT
Here is some information on the current provisional API we have in p2 1.0. For querying profiles, the entry point is the IProfileRegistry service.  Obtain the registry like you would any OSGi service (BundleContext#getService...). From IProfileRegistry you can get an IProfile for a given profile id. For the currently running profile, use IProfileRegistry.getProfile(IProfileRegistry.SELF).

IProfile can be queried for the IUs contained within it. For example to query for all OSGi bundles in a profile, you would do this:

RequiredCapability bundleCapability = MetadataFactory.createRequiredCapability("org.eclipse.equinox.p2.eclipse.type", "bundle", VersionRange.emptyRange, null, false, false);
Query query = new CapabilityQuery(bundleCapability);
Collection result = profile.query(query, new Collector(), null).toCollection();

This wad of code basically says, "give me the installable units in this profile that provide the 'bundles' capability".  The variable "result" contains the IUs that provide OSGi bundles.  For source bundles, the string "bundle" above would be replaced with "source".

This is of course provisional API, subject to change, and relies on some magic strings that aren't well documented, but it gives you an idea of what can be done.
Comment 6 Sudarsha Wijenayake CLA 2008-09-03 13:34:38 EDT
All of the following classes used in the above code snippet are kept internal to their corresponding plug-in (at least in Eclipse 3.4). 

org.eclipse.equinox.internal.provisional.p2.engine.IProfile
org.eclipse.equinox.internal.provisional.p2.engine.IProfileRegistry
org.eclipse.equinox.internal.provisional.p2.metadata.MetadataFactory
org.eclipse.equinox.internal.provisional.p2.metadata.RequiredCapability
org.eclipse.equinox.internal.provisional.p2.metadata.query.CapabilityQuery
org.eclipse.equinox.internal.provisional.p2.query.Collector
org.eclipse.equinox.internal.provisional.p2.query.Query

I understand that this is a provisional API and that must be the reason why that it has been kept inaccessible to external plug-ins. A public API similar to this would certainly allow us to extract information about source plug-ins.
Comment 7 Darin Wright CLA 2008-09-16 10:30:51 EDT
Moving to PDE - we need to validate the provisional API provided by p2. We should use this API to do bundle discovery, rather than parsing bundles.info ourselves.
Comment 8 Darin Wright CLA 2008-09-18 09:22:28 EDT
I used this code, as suggested by John in comment#5:

BundleContext bundleContext = PDECore.getDefault().getBundleContext();
ServiceReference reference = bundleContext.getServiceReference(IProfileRegistry.class.getName());
if (reference != null) {
	IProfileRegistry registry = (IProfileRegistry) bundleContext.getService(reference);
	if (registry != null) {
		IProfile profile = registry.getProfile(IProfileRegistry.SELF);
		if (profile != null) {
			RequiredCapability bundleCapability = MetadataFactory.createRequiredCapability("org.eclipse.equinox.p2.eclipse.type", "bundle", VersionRange.emptyRange, null, false, false);
			Query query = new CapabilityQuery(bundleCapability);
			Collection result = profile.query(query, new Collector(), null).toCollection();
			Iterator it = result.iterator();
			while (it.hasNext()) {
				IInstallableUnit unit = (IInstallableUnit) it.next();
				System.out.println(unit);
			}
		}
	}
	bundleContext.ungetService(reference);
}


However, the 'result' returned by the runnig the query is always empty. I was debugging this in a target workbench - I recall that there might be some issues with the default profile in a target? would that be the cause of the empty list, or is there something wrong with the query?
Comment 9 Sudarsha Wijenayake CLA 2008-09-18 10:10:22 EDT
Hi Darin, I too saw the same behavior when tried the above provisional P2 API.  

This particular API is still internal. Does this mean that you would request the P2 Team to make it public if the particular API was suitable for the PDE?
Comment 10 Darin Wright CLA 2008-09-18 10:13:50 EDT
(In reply to comment #9)
> This particular API is still internal. Does this mean that you would request
> the P2 Team to make it public if the particular API was suitable for the PDE?

Although I'd love a stable API, that is up to the p2 team to decide :-)
Comment 11 Chris Aniszczyk CLA 2008-09-18 10:15:46 EDT
There's also the possibility of us being x-friended to p2
Comment 12 Chris Aniszczyk CLA 2008-09-18 23:34:13 EDT
I think this is because you are launching the thing you are running.  Since PDE does not do "self hosting" (in the sense of feature-based selfhosting), the launched configuration does not have a p2 profile "SELF".  So you either need to export your bundle and do whatever the Zurich folks do or the p2 self hosting support needs to be updated.  The latter is preferred :-)  (BTW, this is Jeff typing in Chris' account)
Comment 13 Darin Wright CLA 2008-09-19 09:12:42 EDT
The _SELF_ profile does exist. However, when running the query, it is try to match against top level features (mostly), and none of the features match "bundle".
Comment 14 Jeff McAffer CLA 2008-09-19 21:55:56 EDT
I suspect that that profile is not the right one. Since PDE is launching there isn't anyone creating the profile. You might want to look at the p2 data area in the running system and see what's up. 
Comment 15 Pascal Rapicault CLA 2008-09-20 14:41:15 EDT
Do you code that we can get to see directly what is going on?
Comment 16 Curtis Windatt CLA 2008-09-22 15:24:44 EDT
Created attachment 113178 [details]
Quick patch using example code

Patch applies to PDE Core.  Add a breakpoint in the new method scanRuntimeBundles() in P2Util.  When PDE tries to create a target platform on startup, this method will be called.  I tried searching for different required capabilities and never obtained any results.
Comment 17 Pascal Rapicault CLA 2008-09-23 12:13:39 EDT
The problem you are encountering comes from the fact that when you are selfhosting, the self profile representing the running system is "partial". This means that it only contains the things that have been installed by the user (aka the roots) and not all the IUs resulting from the complete expansion of the profile. The complete expansion is only done once a provisioning operation has occurred on the system.
If you were to export this plugin and run with it this problem would not occur.

That said the problem would still be encountered in shared install mode.
Comment 18 Chris Aniszczyk CLA 2008-09-23 12:15:08 EDT
When we launch, we simply hack out a bundles.info and go. It seems that during launch, we should be doing a provisioning operation almost to have things generated for us. Is that the path you're thinking Pascal?
Comment 19 Darin Wright CLA 2008-09-23 14:01:26 EDT
After talking with Pascal, I discovered we'd have to do the following for this to work (since we need to do target platform discovery in any level of a nested target workbench...)

* make a profile out of what is in the target platform + the workspace
* generate metadata from the content of what's in the workspace

There was old code that started/attempted this - look at the selfhosting bundle in the RT repo:

org.eclipse.equinox/p2/bundles/org.eclipse.equinox.p2.selfhosting
Comment 20 Pascal Rapicault CLA 2008-09-23 14:35:21 EDT
The idea behind the selfhosting bundle was that the instance that is started by PDE was resulting from a provisioning operation and that in this scenario PDE would not have to write a bundle.info / config.ini or eclipse.ini. This whole story is "ideal" and is more targeted to be a replacement of the "workspace features"  functionality, however I'm not sure if we need to go all way on this to address the problem at hand.
Comment 21 Chris Aniszczyk CLA 2008-09-23 15:12:03 EDT
(In reply to comment #20)
> The idea behind the selfhosting bundle was that the instance that is started by
> PDE was resulting from a provisioning operation and that in this scenario PDE
> would not have to write a bundle.info / config.ini or eclipse.ini. This whole
> story is "ideal" and is more targeted to be a replacement of the "workspace
> features"  functionality, however I'm not sure if we need to go all way on this
> to address the problem at hand.

I would prefer if we go all the way as we want to make sure that we mirror the reality of things. In the past, we have had problems with subtle differences the way PDE, PDE Build and Update did things. If we all use the same mechanism, we should have less issues and less code to maintain hopefully.

Having a lot of code that serves the same purpose (ie., writing out bundles.info, config.ini, eclipse.ini is bad).

The only problem I can foresee is backwards compatibility. PDE may have to keep its current code paths for handling <=3.3 and use p2 for >=3.4 as I don't think p2 handles older releases.
Comment 22 Curtis Windatt CLA 2008-10-08 13:02:42 EDT
Making this a higher level plan item for 3.5.  There are multiple ways in which PDE can leverage p2 APIs.

See bug 250126 for work towards p2 self-hosting.

Comment 23 Darin Wright CLA 2009-03-19 10:21:04 EDT
PDE is using the provisional SimpleConfiguratorManipulator API to derive installed bundles/source. Marking fixed.