Bug 204347 - [prov] Extension to the PDE Target provisioner
Summary: [prov] Extension to the PDE Target provisioner
Status: RESOLVED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.4 M7   Edit
Assignee: Curtis Windatt CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
Depends on: 207501 225154
Blocks:
  Show dependency tree
 
Reported: 2007-09-21 22:56 EDT by Pascal Rapicault CLA
Modified: 2008-04-14 14:37 EDT (History)
9 users (show)

See Also:


Attachments
mylyn/context/zip (956 bytes, application/octet-stream)
2007-09-23 09:56 EDT, Chris Aniszczyk CLA
no flags Details
Patch for equinox.p2.target (55.12 KB, patch)
2007-11-07 09:44 EST, Curtis Windatt CLA
no flags Details | Diff
Screenshot of provisioner (23.31 KB, image/png)
2008-04-03 10:28 EDT, Curtis Windatt CLA
no flags Details
Patch against p2 incubator project (63.12 KB, patch)
2008-04-03 10:51 EDT, Curtis Windatt CLA
no flags Details | Diff
Patch to add target provisioner to new plugin (77.12 KB, patch)
2008-04-08 16:02 EDT, Curtis Windatt CLA
no flags Details | Diff
Updated patch (72.33 KB, patch)
2008-04-09 15:27 EDT, Curtis Windatt CLA
no flags Details | Diff
Icon (585 bytes, image/gif)
2008-04-09 15:29 EDT, Curtis Windatt CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pascal Rapicault CLA 2007-09-21 22:56:52 EDT
If we want to develop with an eclipse that have been provisioned with p2, we need a target provisioner that shapes the target based on the bundles.txt.
Comment 1 Jeff McAffer CLA 2007-09-22 20:38:28 EDT
seems like something for the p2 pde support?
Comment 2 Chris Aniszczyk CLA 2007-09-22 20:43:50 EDT
bundles.txt == platform.xml?
Comment 3 Chris Aniszczyk CLA 2007-09-23 09:56:49 EDT
Darin, tell your boys to look into this (or we can at the Summit). If they can read a txt file and return an array of File objects, they're good :)

IProvisionerWizard is of interest here. I'm attaching a Mylyn context for people that love Mylyn.
Comment 4 Chris Aniszczyk CLA 2007-09-23 09:56:54 EDT
Created attachment 79042 [details]
mylyn/context/zip
Comment 5 Chris Aniszczyk CLA 2007-09-24 10:33:55 EDT
Jeff, direction from the provisioning team like what APIs to use would be fantastic.
Comment 6 John Arthorne CLA 2007-09-24 10:46:10 EDT
I don't think there is API for this right now. See SimpleConfiguratorUtils.readConfiguration in bundle org.eclipse.equinox.simpleconfigurator.
Comment 7 Pascal Rapicault CLA 2007-09-24 16:56:04 EDT
I have released an initial implementation in a project called org.eclipse.equinox.p2.target (under equinox-incubator/provisioning).

It does not work because PDE is trying to be smart and does not use the File provided by the wizard as is, but it actually looks in a subfolder of the given file.
Comment 8 Pascal Rapicault CLA 2007-09-24 20:55:27 EDT
CC'ing Brian for input.
Comment 9 Brian Bauman CLA 2007-10-04 02:52:11 EDT
Darin is going to take a look at this when he gets some time in the near future.
Comment 10 Pascal Rapicault CLA 2007-10-18 16:21:57 EDT
Is there anything happening on that front?
Comment 11 Darin Wright CLA 2007-10-18 16:28:35 EDT
Jeff, Curtis and I are meeting tomorrow afternoon re: p2 tooling.
Comment 12 Pascal Rapicault CLA 2007-10-24 09:30:49 EDT
I'm reiterating my request on this bug. This is fairly important to us for selfhosting. 
Comment 13 Curtis Windatt CLA 2007-10-25 16:21:14 EDT
Wiki page describing evolution of this (and other PDE related) p2 features is at
http://wiki.eclipse.org/Equinox_p2_PDE_Integration#Target_Provisioning
Comment 14 Chris Aniszczyk CLA 2007-10-25 16:26:56 EDT
As part of this work, you need to consider refactoring out Update related code into another plug-in in PDE. This way when the time comes, it will be easier to drop support for features. Plus, the goal of componentizing PDE better is always nice.
Comment 15 Darin Wright CLA 2007-10-29 22:42:33 EDT
Status update: The IProvisionerWizard interface, implemented by clients such as the p2 target provisioner is required to return a collection of files representing directories that contain plug-ins to be added to the target platform. For p2, we'd prefer to return a collection of files representing the plug-ins (bundles) themselves. However, as pointed out in bug 207503, this means it is not easy tell the difference between a plug-in in a folder and a folder containing nested plug-ins.

Since p2 could suppport a configuration where plug-ins are distributed in many folders, but only a few plug-ins are taken from each folder, I propose that an immediate solution would be to have the p2 provisioning wizard return a collection of folders containing plug-ins (as per the original spec), but in addition allow for an IProvisionerWizard2 interface specifying the set of enabled plug-ins from those directories. This way it will be easier to manage/automatically set the collection of enabled plug-ins.

In the longer term, I think we need to look at more flexible APIs/models for supporting multiple target definitions in the workspace. However the immediate goal is to enable p2, rather than to expose all of its power.
Comment 16 Curtis Windatt CLA 2007-11-07 09:44:40 EST
Created attachment 82330 [details]
Patch for equinox.p2.target

Alternative solution.

This patch modifies the p2 target bundle to use it's own UI elements rather than trying to integrate with PDE's preference page and extension point.  Target provisioning is accomplished by creating a 'target definition' file that describes the folders where bundles can be found and which bundles to include in the target.  

This patch provides a new main menu entry that allows target definitions to be managed.  There is an option that will use a wizard to create a definition file from a p2 install directory.  After creating it, the target described by the file can be loaded as the target platform in several ways 1) A 'load' entry under the new menu, 2) The 'Load as Target Platform' link in the target definition editor, 3) On the target platform preference page, browsing to your file and pressing load.

Known issues:
- You must create the definition file in your workspace (so you must have a project to put it in).
- The target platform preference page will still add every bundle in every directory referred to in the bundles.txt to the target, but only the specified bundles will be enabled.  Changing this will require significant changes to PDE.
- The code uses a lot of internal PDE code, could probably be optimized and needs more documentation.
Comment 17 Curtis Windatt CLA 2007-12-06 17:51:24 EST
This bug can probably be closed, as we are taking other approaches to integrating p2 with PDE, but I'll let Darin decide.

Removing dependency on bug 207503 because we don't need any changes to IProvisionerWizard.
Comment 18 Curtis Windatt CLA 2008-01-22 11:46:02 EST
This will be worked on for M5.  We are going to replace the current update site provisioner with one that supports both p2 and old update sites.  The UI will reuse components already available in p2 to browse repositories, select groups/features, download the associated artifacts/bundles and add them to the   platform.

We will try to use PDE's existing API/extension point, though it only allows directories to be returned.

The p2/PDE integration wiki page has been updated at http://wiki.eclipse.org/Equinox_p2_PDE_Integration
Comment 19 Pascal Rapicault CLA 2008-02-08 10:53:42 EST
Darin, should this bug be moved to the PDE bucket?
Comment 20 Curtis Windatt CLA 2008-02-08 11:21:16 EST
The actual code changes will be part of p2, using the extension point provided by PDE, so this is not a PDE bug.
Comment 21 John Arthorne CLA 2008-03-17 10:45:34 EDT
Can this be closed?
Comment 22 Darin Wright CLA 2008-03-17 16:38:37 EDT
We still plan on enhancing the the target plaform pref page to allow downloading from a p2 site (similar to downloading from an update site). Should be moved to PDE.
Comment 23 Chris Aniszczyk CLA 2008-03-23 16:49:58 EDT
FYI, this should be a p2 bug not a PDE bug. They should have ownership of the extension IMHO.

Let's try this for M7.
Comment 24 Jeff McAffer CLA 2008-03-26 20:34:59 EDT
how about we stop moving it around and just implement the function.  I assume that Curtis or Darin is doing the work...  I agree with having the p2 tooling distinct from the main body of pde code but it is not clear that the bundles should not have pde in the name.  Lets get the function and worry about naming the bundle later.

Can someone take ownership of this
Comment 25 Darin Wright CLA 2008-03-26 22:14:19 EDT
Curtis will work on this in M7.
Comment 26 Curtis Windatt CLA 2008-04-01 15:31:03 EDT
I am getting close to having a patch ready, but I need to know where it is going.  I'm using enough p2 UI code that it should really belong to Equinox p2, not PDE.  However, it will have to be in a plugin that depends on PDE to access the extension point.

Currently I am working in the org.eclipse.equinox.p2.target plugin, which is part of the incubator.

Can someone from p2 comment on where the code should go?
Comment 27 Chris Aniszczyk CLA 2008-04-01 15:34:52 EDT
I'm fine as it being a separate plug-in.

I would actually love to move the Update target provisioner in another plug-in also.

We can use this as an opportunity to do both.
Comment 28 Pascal Rapicault CLA 2008-04-02 09:54:59 EDT
As I'm not sure to know what are you after, here are 3 answers to the question
- committer ownership? Curtis :-)
- bugzilla component? PDE
- feature inclusion? PDE because of the dependency on the PDE extension.
- plugin name? I'm with the current name

As for moving the update target into a separate plugin, I don't see the immediate value since that will not remove the dependency that PDE has on update anyway, but I'm not the owner.
Comment 29 Darin Wright CLA 2008-04-03 10:12:52 EDT
The target provisioner requires PDE and p2. PDE will own the code. I propose to make a new plug-in called something like "org.eclipse.pde.ui.p2". This will allow us to keep the new function under the PDE umbrella but still separated from PDE proper. This gives us flexibility when producing nob-p2-ized builds, etc. In the future PDE will likely depend on p2 directly, but we don't want to make the hard dependency yet.
Comment 30 Curtis Windatt CLA 2008-04-03 10:28:53 EDT
Created attachment 94728 [details]
Screenshot of provisioner

Attached is a screenshot showing what my current work looks like.

You can select a directory to download bundles to (default is a new directory inside the pde.core metadata (the same place runtime configuration data is stored).  There is an option to clear the folder before downloading (because any bundles in the folder will end up in your target).

The UI to select an IU is based on provisional p2 API.  I had to hack a few things in to get the site management and properties buttons to work properly.

When you hit finish, we run the provisioning in a separate thread and report progress.
Comment 31 Curtis Windatt CLA 2008-04-03 10:51:38 EDT
Created attachment 94733 [details]
Patch against p2 incubator project

This patch contains my most recent code.  To get it to apply, you will need org.eclipse.equinox.p2.target from the Equinox incubator and org.eclipse.equinox.p2.ui.sdk.  It currently uses internal p2 code.

I'm looking forward to having a new pde plugin to put this code in.
Comment 32 Curtis Windatt CLA 2008-04-07 17:12:27 EDT
The new plugin org.eclipse.pde.ui.p2 has been created.  I will have a patch against it in the near future, but I am currently waiting on some API changes from p2 ui.
Comment 33 Curtis Windatt CLA 2008-04-08 16:02:14 EDT
Created attachment 95262 [details]
Patch to add target provisioner to new plugin

Patch against the new org.eclipse.pde.p2.ui plugin.  Needs an icon (I was borrowing one from p2).  This is pretty much ready to go.
Comment 34 Curtis Windatt CLA 2008-04-08 16:04:49 EDT
I am encountering a bug in the available IU group with my latest work.  Sometimes when expanding categories, the "pending..." placeholder shows up, but the fetch job never runs so it just says pending... forever.
Comment 35 Susan McCourt CLA 2008-04-08 16:39:01 EDT
>Sometimes when expanding categories, the "pending..." placeholder shows up, but
>the fetch job never runs so it just says pending... forever.

Do you know that it never runs?  Or runs and then gets cancelled?  Or just runs forever?  If you run with the ProgressView showing, you'll see those fetch jobs in the view and you can see if they appear, get cancelled, etc.  

I had some similar problems in this area with item filtering on, when it switched the content provider to a synchronous mode, but I thought I had fixed this.  I'm not seeing it in the regular UI.

I'll take a look at the patch.  
Comment 36 Susan McCourt CLA 2008-04-08 18:42:22 EDT
Curtis, I can reproduce this and see the nature of the problem. 

I'm getting done in by DeferredTreeContentManager (again...sigh...).  Its implementation of addUpdateCompleteListener is actually replacing the listeners, so my tracking of which fetches are complete is not right.  I'm still debugging to figure out exactly the conditions when this happens (why the main UI doesn't have this problem.)

Suffice to say that I'll provide a fix.  I am done looking at it for today...In the meantime, a workaround is to type in the filter text, that will fully realize the tree. 
Comment 37 Curtis Windatt CLA 2008-04-09 10:05:46 EDT
Thanks Susan.

Chris, when you have time, please review and commit the patch.  We can talk about what icon to use.  Also, because p2 can handle old update sites, we can consider getting rid of the current update site provisioner.
Comment 38 Susan McCourt CLA 2008-04-09 14:06:57 EDT
The p2 UI side is fixed.  I didn't have the right default pattern filter set up for you.  The update UI code was creating one explicitly, that is why there was a difference.  (The default PatternFilter does unnecessary expansions of the tree even when there is no filtering to be done and that conflicts with the timing of the deferred fetch.)

Fixed in HEAD.

Also, since you are using the "view by site" mode, you probably want to use the bold fonts for filter matches.  Add this line of code after you create the available IU group:

   fAvailableIUGroup.setUseBoldFontForFilteredItems(true);


>We can talk
>about what icon to use.  Also, because p2 can handle old update sites, we can
>consider getting rid of the current update site provisioner.

My 2 cents:
I found it a bit weird to see both provisioners in the wizard.  You probably only want the update site provisioner active when the capability for "Classic Update" is active.

As for the icon to use:  I think you should use the same (site) icon for the provisioner rather than the IU icon, since you are showing the view by site and the wizard is called "Repository or Update Site".

Also, a terminology comment.  The description for the wizard says:
"This provisioner adds new plug-ins from repositories and update sites into your target platform."  But the wizard page uses the "Installable Unit" terminology everywhere.  We don't use "Installable Unit" anywhere in the update UI, so I suggest you use "plug-ins" in the context of the PDE target provisioner (or feature if that is a more appropriate word).  In the main UI we are trying to be agnostic by calling it "Available Software".
Comment 39 Curtis Windatt CLA 2008-04-09 14:56:27 EDT
(In reply to comment #38)
> The p2 UI side is fixed.  I didn't have the right default pattern filter set up
> for you.  The update UI code was creating one explicitly, that is why there was
> a difference.  (The default PatternFilter does unnecessary expansions of the
> tree even when there is no filtering to be done and that conflicts with the
> timing of the deferred fetch.)

Works great, thank you.

> 
> My 2 cents:
> I found it a bit weird to see both provisioners in the wizard.  You probably
> only want the update site provisioner active when the capability for "Classic
> Update" is active.

See bug 226334.
 
> Also, a terminology comment.  The description for the wizard says:
> "This provisioner adds new plug-ins from repositories and update sites into
> your target platform."  But the wizard page uses the "Installable Unit"
> terminology everywhere.  We don't use "Installable Unit" anywhere in the update
> UI, so I suggest you use "plug-ins" in the context of the PDE target
> provisioner (or feature if that is a more appropriate word).  In the main UI we
> are trying to be agnostic by calling it "Available Software".
> 

Yes, I wasn't sure what terminology to use and never went back to make it better fit in with the main UI.  I'll improve the wording shortly.

I prefer the IU icon over the update site one because when they open up the target provisioner they'll see a list of IU icons in the table, but this is not a major concern.

Comment 40 Curtis Windatt CLA 2008-04-09 15:27:28 EDT
Created attachment 95410 [details]
Updated patch

Updated the terminology.  Fixed up the translation files while I was there.
Comment 41 Curtis Windatt CLA 2008-04-09 15:29:31 EDT
Created attachment 95411 [details]
Icon

This is the icon I would prefer to use.  It would need to be put in the project at icons/obj/iu_obj.gif to work with the most recent patch.
Comment 42 Chris Aniszczyk CLA 2008-04-10 00:23:45 EDT
Thanks Curtis. I will commit the initial code soon and I am targetting the 4/15 i-build for this.
Comment 43 Chris Aniszczyk CLA 2008-04-10 19:18:38 EDT
committed code to org.eclipse.pde.p2.ui, made some version number changes and aesthetic changes.

need to open a bug against the wonderful releng team to get this added to the map file / feature

Thanks Curtis.
Comment 44 Curtis Windatt CLA 2008-04-11 11:29:01 EDT
Thanks for getting this committed Chris.  Opened Bug 226711 with releng.
Comment 45 Curtis Windatt CLA 2008-04-14 14:37:08 EDT
Provisioner is available in HEAD, marking as FIXED.  Please file separate bugs against PDE UI for any problems.