Bug 292996 - provide API for installing features
Summary: provide API for installing features
Status: VERIFIED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: 3.6 M5   Edit
Assignee: Susan McCourt CLA
QA Contact:
URL:
Whiteboard:
Keywords: api
Depends on:
Blocks: 293959
  Show dependency tree
 
Reported: 2009-10-22 03:10 EDT by Steffen Pingel CLA
Modified: 2010-01-26 13:12 EST (History)
6 users (show)

See Also:


Attachments
code that drives P2 (15.00 KB, text/plain)
2009-10-22 03:14 EDT, Steffen Pingel CLA
no flags Details
code that drives PreselectedIUInstallWizard (3.23 KB, text/plain)
2009-10-22 03:16 EDT, Steffen Pingel CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steffen Pingel CLA 2009-10-22 03:10:32 EDT
Mylyn provides a simple UI for installing extensions (bug 272621, bug 291094) that uses P2 internals to drive installation of single features. The process works as follows:

* Add required metadata and artifact repositories
* Create a ProfileChangeRequest to install selected features
* Resolve the provisioning plan using a PlannerResolutionOperation
* Use results to drive PreselectedIUInstallWizard

I have attached the code that does the main chunk of the work. The sources of the discovery plug-ins are here: http://www.eclipse.org/mylyn/doc/dev/mylyn-discovery.psf .

The user interaction of the Mylyn discovery install is currently not optimal since multiple wizards are currently required (discovery wizard and P2 license wizard) and the progress reporting is not optimal. We didn't find a good way to hook into the actual install jobs to provide feedback to the user when provisioning is complete.

In a perfect world the P2 API would provide the following:

* Adding of metadata and artifact repositories
* A job for computing if a feature can be installed into the current profile
* Reusable UI for license confirmation that can be embedded in an existing wizard
* A job for doing the actual install that allows to attach listeners for progress reporting etc.
* A hook to automatically apply changes (i.e. activate installed features) instead of prompting the user to apply or restart

Our goal is to streamline the installation of extensions so it's possible to embed the whole process into existing workflows, e.g.:

* A user selects to connect to a JIRA repository
* If the JIRA connector is not installed the user is prompted for the license in the same wizard that is used to create the repository
* The installation proceeds as part of the wizard workflow and the JIRA connector is enabled automatically
* The wizard proceeds with the standard repository creation
Comment 1 Steffen Pingel CLA 2009-10-22 03:14:40 EDT
Created attachment 150201 [details]
code that drives P2
Comment 2 Steffen Pingel CLA 2009-10-22 03:16:22 EDT
Created attachment 150202 [details]
code that drives PreselectedIUInstallWizard
Comment 3 Pascal Rapicault CLA 2009-10-22 09:58:56 EDT
What is the relationship with the requirements expressed in bug #292907 ?
Comment 4 Susan McCourt CLA 2009-10-22 13:37:27 EDT
(In reply to comment #0)

One idea we've been kicking around is refactoring the UI operations to expose what is now in org.eclipse.equinox.internal.provisional.p2.ui.operations in a non-UI level package as a set of helper API that simplifies the interaction with p2 core.  So that "casual" clients traffic in IUs, repos, statuses, etc. but not engines and operands.  And the things that we do traffic in would be rather opaque.  These operations (along with ProvisioningOperationRunner) have evolved ad hoc and are rather low level as we deal with problems of progress reporting, sync vs async running, etc. 

I don't see that this API would literally traffic in "features" but I do think we could provide something higher level than PlannerResolutionOperation that combines what is today in wizard code, PlannerResolutionOperation, ProfileModificationOperation, and in static methods in InstallAction.  So you could define an InstallOperation, UninstallOperation, UpdateOperation, etc. and then work with those.

We could also define some kind of "context" that would let you define the profile and repositories you care about, how you want to run the operation, so that you aren't handing around things like QueryableMetadataRepositoryManager and so forth.  It has some aspects of IRunnableContext and what ProvisioningOperationRunner does.

I've given no thought to real names, but something like:
ProvisioningOperationContext opContext = new ProvisioningOperationContext(String someProfileID, URI [] repos, boolean cancelable, boolean userVisible, boolean runInBackground, int restartPolicy); 
InstallOperation op = new InstallOperation(opContext, IInstallableUnit [] ius);

Status status = op.resolve(IProgressMonitor);
if (status.isOK()) {
  op.run(IProgressMonitor);
}
// if status is not ok, figure out what to do, report to the user, 
// make another InstallOperation with same context


The idea of specifying synchronous vs. asynchronous and also getting to provide your own progress monitor is not trivial given current platform API.  Today the p2 UI decides whether these things are synchronous or asynchronous depending on where the operation is hosted and what kind of progress is needed.  For example, we resolve in the background on "Check for Updates" but we resolve in the context of a wizard when we want local feedback.  As it stands now, we can only report local wizard progress when we are running synchronously in the UI thread, otherwise we can't report the progress locally (see bug 279385).  This has to be figured out to support the idea above.

Today we use ProvisioningOperationRunner to manage post-operation behavior (restart/apply, etc.)  (work is happening right now for restart hooks in bug 274876).  

> In a perfect world the P2 API would provide the following:
> 
> * Adding of metadata and artifact repositories
Already provided, correct?  But the example above makes it contextual with your operation.

> * A job for computing if a feature can be installed into the current profile
PlannerResolutionOperation but simplified

> * Reusable UI for license confirmation that can be embedded in an existing
> wizard
I would expect the p2 wizard framework to become more opaque if these higher level operation constructs are defined, but AcceptLicensesWizardPage is pretty simple already.

> * A job for doing the actual install that allows to attach listeners for
> progress reporting etc.
This requires some platform API, we are working this in bug 279385.  

> * A hook to automatically apply changes (i.e. activate installed features)
> instead of prompting the user to apply or restart
This is happening in bug 274876.  
I could also imagine the need for some listeners that are called when the operation is done so that other custom behavior could be plugged in.  Maybe these listeners should be defined at the operation level so they can be used in and out of jobs.  Today that complexity is forced on the caller.

> 
> Our goal is to streamline the installation of extensions so it's possible to
> embed the whole process into existing workflows, e.g.:
> 
> * A user selects to connect to a JIRA repository
> * If the JIRA connector is not installed the user is prompted for the license
> in the same wizard that is used to create the repository
> * The installation proceeds as part of the wizard workflow and the JIRA
> connector is enabled automatically
> * The wizard proceeds with the standard repository creation

This is a *great* use case description.

If the installation of the JIRA connector proceeds as part of the wizard workflow, then it's not so much that you want an install job for asynchronous workflow, it's just you don't need the UI thread and don't want to block the UI while it's happening.  What *could* the user do while this job was running, just move back or cancel?  Or is there other user interaction you would expect to happen?  Understanding this scenario will help me to formulate a platform UI bug as discussed in bug 279385.
Comment 5 Steffen Pingel CLA 2009-10-22 14:54:14 EDT
(In reply to comment #3)
> What is the relationship with the requirements expressed in bug #292907 ?

Bug 292907 describes the requirement for programmatically installing a feature without any user interaction which is a requirement for the Tasktop RCP application. 

This bug has additional requirements for Mylyn's discovery UI which is potentially a superset of bug 292907.
Comment 6 Robert Elves CLA 2009-10-22 15:49:34 EDT
(In reply to comment #4)
> I've given no thought to real names, but something like:
> ProvisioningOperationContext opContext = new
> ProvisioningOperationContext(String someProfileID, URI [] repos, boolean
> cancelable, boolean userVisible, boolean runInBackground, int restartPolicy);
> InstallOperation op = new InstallOperation(opContext, IInstallableUnit [] ius);
> 
> Status status = op.resolve(IProgressMonitor);
> if (status.isOK()) {
> op.run(IProgressMonitor);
> }
> // if status is not ok, figure out what to do, report to the user,
> // make another InstallOperation with same context


Susan, this level of abstraction would be excellent!  I've written code to drive both the old update manager and p2 to do semi headless installs and an api like you describe would save a lot of time. Just learning p2's constructs is no small undertaking let alone making it perform what you want.
 

> > Our goal is to streamline the installation of extensions so it's possible to
> > embed the whole process into existing workflows, e.g.:
> >
> > * A user selects to connect to a JIRA repository
> > * If the JIRA connector is not installed the user is prompted for the license
> > in the same wizard that is used to create the repository
> > * The installation proceeds as part of the wizard workflow and the JIRA
> > connector is enabled automatically
> > * The wizard proceeds with the standard repository creation
> 
> This is a *great* use case description.
> 
> If the installation of the JIRA connector proceeds as part of the wizard
> workflow, then it's not so much that you want an install job for asynchronous
> workflow, it's just you don't need the UI thread and don't want to block the UI
> while it's happening.  

Right, we would just want to show progress as part of the current wizard context.

>What *could* the user do while this job was running,
> just move back or cancel?  
Yes, while the progress is indicated in the wizard the user would be able to cancel or go back.

> Or is there other user interaction you would expect
> to happen?  

There could be other interactions where the wizard continues in parallel and/or information is presented to the user on the current/next page. Practically speaking though, we'd probably want to keep things simple by first ensuring the installation succeeded, and only then allow moving on to subsequent pages.
Comment 7 Susan McCourt CLA 2009-10-22 16:42:06 EDT
> Susan, this level of abstraction would be excellent!  I've written code to
> drive both the old update manager and p2 to do semi headless installs and an
> api like you describe would save a lot of time. Just learning p2's constructs
> is no small undertaking let alone making it perform what you want.
> 

Good.  The thing to keep in mind is that the abstraction stays nice and simple when everything goes perfectly.  (And maybe for installing connectors the dependencies are easy to manage and most things successfully resolve).

You'll still get dragged down into the details when an installation request doesn't resolve, depending on what you want to do to recover.  In the SDK p2 UI, we report problems and let the user change the selections based on the problems.  In your case, you want to do something automatically, so you might end up having to understand the ProvisioningPlan API and how it communicates resolution problems.

ProvisioningPlan.getRequestStatus()
ProvisioningPlan.getRequestStatus(IInstallableUnit)
ProvisioningPlan.getSideEffectChanges()

because your end goal is to produce a successful InstallOperation and isolate the error reporting from the user.  This is something we can't guess for you (ie, whether you want to find different versions, try other repositories, etc.)  But ideally this can be buried in one place in your code.

I'll keep in touch in this bug with what's happening in this space.  

I'm taking ownership of the bug and marking M4.
Comment 8 Susan McCourt CLA 2009-11-04 11:53:48 EST
Status update:

I'm in the middle of a pretty big refactoring in my workspace, but I wanted to give you a taste of what the client code looks like.  Some of the detail exposed here (rootMarkerKey, profileId, using an external ProvisioningOperationRunner) is not ideal, but is needed so that the operations can run in a headless environment.  I don't want to bury any of the UI helper code inside the operations, so clients have to feed some of that info into the operations and use UI code to actually run the jobs.

The principle is still good, I think.  If everything goes according to plan (no pun intended), then you don't have to know about provisioning plans and such.  If the install won't resolve, then you have to know more.

// Create an install operation.  
// provisioningSession can be obtained from a helper, it has all the p2 services
// instantiated
// rootMarkerKey is the property key that is used to mark install roots 
// iusToInstall is an array of ius.

InstallOperation op = new InstallOperation(provisioningSession, profileId, rootMarkerKey, provisioningContext, iusToInstall);

// to resolve in a wizard context
op.resolveModal(progressMonitor);

// to resolve in the background and then do something
ProvisioningJob job = op.getResolveJob(progressMonitor);
job. addListener(new JobChangeAdapter() {
   public void done(JobChangeEvent event) {
     // do something after the resolve
   }
};

// to check the results.
IStatus status = op.getResolutionResult();
if (status.getSeverity() == IStatus.CANCEL)
  return;

if (status.getSeverity() != IStatus.ERROR) {
  // everything's cool
  ProvisioningOperationRunner.schedule(op.getProvisioningJob(progressMonitor);
}

// You must get into the details at this point because something is wrong
// To tell the user something
String description = op.getResolutionDetails();

// Or decide to do something else
Provisioning plan = op.getProvisioningPlan();
// now you are back in the lower level code analyzing what happened and
// in the end you either need to pick a different set of IUs to install 
// or give up.

I still have to work out some issues as to operation life cycle (can you reseed it with IUs and try again?  do you need a new operation?) but this is the general direction.

It's possible that we could have helpers on the UI side so you only have to feed the relevant info.
InstallOperation op = uiHelper.createInstallOperation(iusToInstall, URI [] reposToUse);

and the helper could get the right provisioning session, the correct profile id, create the correct provisioning context, set the root marker key, etc.

A really nice side effect of this refactoring is that plan validation code that has been scattered across the UI (actions, wizard pages, wizards, etc.) is now pushed into the headless operation.  This will make writing test cases much easier as well.
Comment 9 Robert Elves CLA 2009-11-04 18:45:57 EST
(In reply to comment #8)
> I still have to work out some issues as to operation life cycle (can you reseed
> it with IUs and try again?  do you need a new operation?) but this is the
> general direction.

This api is looking great.  Just fyi, we generally have an explicit idea of what needs to be installed and if a failure happens we don't anticipate there being a plan B other than simply informing the user (ideally in some form that is remotely understandable to humans :) ).  I can however understand how this would be useful for more involved use cases.

> It's possible that we could have helpers on the UI side so you only have to
> feed the relevant info.
> InstallOperation op = uiHelper.createInstallOperation(iusToInstall, URI []
> reposToUse);
> and the helper could get the right provisioning session, the correct profile
> id, create the correct provisioning context, set the root marker key, etc.

This would be excellent if feasible!
Comment 10 Susan McCourt CLA 2009-11-09 19:48:29 EST
I've committed some code to the R3_6_api_cleanup branch.  There are still some details to work out as to what helper methods remain public, but here's the current state of the world.

// Create an install operation.  
// provisioningSession can be obtained from a helper, it has all the p2
services instantiated

InstallOperation op = new InstallOperation(provisioningSession, iusToInstall);
// to resolve modally
op.resolveModal(progressMonitor);

// to resolve in the background and then do something
ProvisioningJob job = op.getResolveJob(progressMonitor);
job. addListener(new JobChangeAdapter() {
   public void done(JobChangeEvent event) {
     // do something after the resolve
   }
};

// to check the results.
IStatus status = op.getResolutionResult();
if (status.getSeverity() == IStatus.CANCEL)
  return;

if (status.getSeverity() != IStatus.ERROR) {
  // everything's cool
  op.getProvisioningJob(progressMonitor).schedule
}

If you are working with the UI, there are helpers to launch wizards or create operations.
InstallOperation op = ProvisioningUI.getDefaultUI().getInstallOperation(iusToInstall, reposToUse);
ProvisioningUI.getDefaultUI().openInstallWizard(shell, null, op, null);

Currently the SDK UI is able to run using only API which was certainly not the case prior to this refactoring.
Comment 11 Susan McCourt CLA 2009-11-23 20:00:28 EST
I'm going to mark this as fixed for my record keeping, as the work is done.
Note we are responding to specific issues in new bugs.
Of course it won't appear in an SDK build until the branch is merged back into HEAD.  At that time I'll figure out what milestone to mark it, etc...
Comment 12 Susan McCourt CLA 2009-12-01 12:57:01 EST
changing milestone to M5.  The p2 api branch will be merged back into HEAD early in M5
Comment 13 Susan McCourt CLA 2010-01-26 13:12:19 EST
verified in I20100126-0100 via source inspection.
If there are additional needs from the operations API, please open new bugs (note that M6 is API freeze.)