Bug 241477 - [eclipse] better support for framework extensions via eclipse touchpoint
Summary: [eclipse] better support for framework extensions via eclipse touchpoint
Status: VERIFIED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.4   Edit
Hardware: PC Linux
: P2 enhancement with 1 vote (vote)
Target Milestone: 3.5 M7   Edit
Assignee: Simon Kaegi CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 220944 248629 (view as bug list)
Depends on: 257178
Blocks: 262295
  Show dependency tree
 
Reported: 2008-07-20 09:37 EDT by Stephan Herrmann CLA
Modified: 2009-04-17 17:37 EDT (History)
7 users (show)

See Also:


Attachments
Patch that adds new touchpoint action installExtension (28.69 KB, patch)
2009-01-28 19:54 EST, Andrew Eisenberg CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Stephan Herrmann CLA 2008-07-20 09:37:11 EDT
I'm spawning this from resolved bug 235006 for an issue that
has not been addressed yet (see bug 235006 comment 23):

In order to install a framework extension via p2 it
seems necessary to include exact version numbers,
because otherwise reverting is not possible.
(Currently, when I try to revert in the presence of
a framework extension, without changing the set of
installed features/plugins, no error is reported,
but also reverting does _not_ succeed to perform any changes).

Thus I suggest to integrate pdebuild and p2 such that
p2.inf can use the ".qualifier" notation which should
then be expanded during build. As an alternative,
the meta data generator could perform this expansion, too.

Also note, that after simulating the above by manually editing
config.ini, the StorageManager crashed as reported in
bug 235006 comment 23.

As good citizens, we would also need a way to incrementally 
add/remove items from the list of framework extensions ;-)
Comment 1 Pascal Rapicault CLA 2008-07-21 07:03:02 EDT
*** Bug 220944 has been marked as a duplicate of this bug. ***
Comment 2 Pascal Rapicault CLA 2008-09-25 13:57:52 EDT
*** Bug 248629 has been marked as a duplicate of this bug. ***
Comment 3 Jeff McAffer CLA 2008-09-28 13:38:58 EDT
At least some of this supprot could be added via the publisher in the BundlesAction.
Comment 4 Thomas Watson CLA 2009-01-20 18:01:59 EST
To fix this bug I think we need to enhance the osgi.framework.extensions to allow for relative and absolute file:reference: URLs.  See bug 257178.
Comment 5 Miles Sabin CLA 2009-01-21 10:34:07 EST
I'm happy to help all I can, but I really need some guidance here ... I'm not at all familiar with the internals of p2.
Comment 6 Pascal Rapicault CLA 2009-01-21 22:32:45 EST
To fix this you will need to reach into two layers of p2. One is framework admin (bundle org.eclipse.equinox.frameworkadmin.equinox) and the other is the eclipse touchpoint and optionally the metadata generator / publisher.

I recommend a bottom up approach.
Framework admin is the actual place where the work of reading and writing the config.ini file happens (see EquinoxFwConfigFileParser) and is the place where files are made relative, etc. This is the place where a key part of the work will happen. You will also need to find a way on ConfigData or Manipulator to set which bundles is an extension bundle.

Once this will be done, you will need to create a new eclipse touchpoint action that will allow for the p2 metadata to invoke this action you created (for example see InstallBundleAction)

Eventually you may want to modify the metadata generation to automatically generate this action as part of the metadata generation.

To get started with the p2 code: http://wiki.eclipse.org/Equinox_p2_Getting_Started_for_Developers
Comment 7 Andrew Eisenberg CLA 2009-01-27 15:22:01 EST
Hi all,

Here is a plan for how to implement this feature.  Can someone please look this over and see if this is reasonable or if I've missed anything?

1. Add OSGi extension awareness to the config reader and writer:

a. in the ConfigData class, add a new field called 
extensionsList type is LinkedHashSet and it is treated similarly to bundlesList.


Create these 3 methods:
addExtension
getExtensions
setExtensions

Add appropriate handling of the field here:
initialize
toString

b. in the EquinoxFwConfigFileParser class, create 2 new methods:
writeExtensionsList (based on writeBundlesList)
readExtensionsList (based on readBundlesList)
In both cases, the methods are somewhat simpler because there is no need to check for org.eclipse.osgi, etc)

These methods should be called appropriately where writeBundlesList and readBundlesList are called.

2. Add the touchpoint actions

a. Create class InstallExtensionAction similar to InstallBundleAction
update the execute and undo methods appropriately and create the installExtension(Map) method based on InstallBundleAction.installBundle(Map) with some differences:

the bundle must be a fragment, so raise return if it is not a fragment

make a call to ConfigData.addExtension() at the end of the method


b. Create class UninstallExtensionAction similar to UninstallBundleAction
as above

make a call to ConfigData.removeExtension() at the end of the method


c. Add the actions as extension points to the plugin.xml.  Should look something like this:

 <extension
       point="org.eclipse.equinox.p2.engine.actions"]]
    <action
          class="org.eclipse.equinox.internal.p2.touchpoint.eclipse.actions.InstallExtensionAction"
          name="installExtension"
          touchpointType="org.eclipse.equinox.p2.osgi"
          touchpointVersion="1.0.0"
          version="1.0.0"]]
    </action>
 </extension> 

<extension
       point="org.eclipse.equinox.p2.engine.actions"]]
    <action
          class="org.eclipse.equinox.internal.p2.touchpoint.eclipse.actions.UninstallExtensionAction"
          name="uninstallExtension"
          touchpointType="org.eclipse.equinox.p2.osgi"
          touchpointVersion="1.0.0"
          version="1.0.0"]]
    </action>
 </extension>


Not sure if this is sufficient.  Would this mean that adding a line like this to your p2.inf file is sufficient to add the fully qualified extension to the right place?

instructions.configure = \
    installExtension(bundle:${artifact});

and 

instructions.unconfigure = \
    uninstallExtension(bundle:${artifact});
Comment 8 Andrew Eisenberg CLA 2009-01-28 19:54:40 EST
Created attachment 124117 [details]
Patch that adds new touchpoint action installExtension

This patch is implemented exactly how I said in the previous comment.  It also includes 2 test cases.  The test cases are copied from InstallBundleActionTest and UninstallBundleActionTest and changed where appropriate.
Comment 9 Pascal Rapicault CLA 2009-02-02 22:07:37 EST
Assigning to M6 so it does not fall off. I'll review in the next few days.
Comment 10 Andrew Eisenberg CLA 2009-02-13 12:27:14 EST
Any progress on this bug?  I would very much like to see this make it into M6.  Thanks.
Comment 11 Pascal Rapicault CLA 2009-02-13 15:43:53 EST
It is on my list
Comment 12 Stephan Herrmann CLA 2009-02-21 14:37:00 EST
Just one more cross-reference: please see bug 231557 comment 15.
Comment 13 Miles Sabin CLA 2009-03-02 18:00:38 EST
What's the current status of this bug? Is there anything I can do to help make it more likely that this patch will be included in M6?
Comment 14 Pascal Rapicault CLA 2009-03-02 20:17:03 EST
The chances are very slim as we still have some feature work to finish up for M6 that has significant impact. This is small enough that it can go in M7.
Comment 15 Stephan Herrmann CLA 2009-03-03 12:12:44 EST
While the progress in this bug looks promising (I really hope it will
make it into 3.5!!), I'm wondering whether (part of) the solution could
be applied to other properties beyond osgi.framework.extensions.

In our case the reason behind defining framework extensions is so
we can install adaptor hooks. This means, to be good citizens we would
need to _incrementally_ set and unset osgi.hook.configurators.include, too.
Is there anything from installExtension/uninstallExtension that could
be reused to yield actions like installHookConfigurator? Hm, to avoid a 
myriad of individual actions, would a generic "addToProgramProperty" 
and "removeFromProgramProperty" make sense?

(This was originally discussed in bug 231557 comment 15,
 where, however, no action is planned).
Comment 16 Andrew Eisenberg CLA 2009-03-03 12:42:50 EST
(In reply to comment #15)
> Is there anything from installExtension/uninstallExtension that could
> be reused to yield actions like installHookConfigurator? Hm, to avoid a 
> myriad of individual actions, would a generic "addToProgramProperty" 
> and "removeFromProgramProperty" make sense?

Yes.  The UninstallExtensionAction and InstallExtensionAction classes are largely copied from Un/InstallBundleAction.  There is not much that is specific to framework extensions.  It could be generified.

However, I would prefer to see the patch be applied without generification, rather than risk missing inclusion into 3.5.
Comment 17 Miles Sabin CLA 2009-03-03 14:17:38 EST
I'd like to second comment #16 ... this patch is a huge improvement over the status quo as it stands and I don't want to see scope creep lead to it missing the 3.5 release deadline.
Comment 18 Simon Kaegi CLA 2009-03-28 15:12:13 EDT
I've checked support for framework extensions into HEAD but am going to leave this bug open until we have better regression tests in place. The areas this fix touched are incredibly sensitive.

The approach I took was to not add any new actions and just make the guts underlying install/uninstall bundle smarter. So in a nutshell, the Equinox Framework Admin implementation recognizes a system bundle fragment when reading and writing the config.ini and does the right thing under the covers.

Thanks Andrew, looking at your patch really helped me do the change more quickly that I otherwise would have been able to.

Comment 19 Stephan Herrmann CLA 2009-03-31 10:28:11 EDT
(In reply to comment #18)

> I've checked support for framework extensions into HEAD  ...

Great news!
 
> The approach I took was to not add any new actions and just make the guts
> underlying install/uninstall bundle smarter. So in a nutshell, the Equinox
> Framework Admin implementation recognizes a system bundle fragment when reading
> and writing the config.ini and does the right thing under the covers.

I was going to look at your patch, but not being a p2 developer I didn't 
see an easy way to find the patch within the myriad of p2 projects in CVS.
(JDT/Core, e.g., always attach the patch to the bug, which make development
very transparent :).

Could you give some more words on how your implementation can be consumed?
Currently I use 
  instructions.install = \
	setProgramProperty(propName:osgi.framework.extensions,propValue:my.extension)

(and uninstall correspondingly).

Are you saying I can now completely drop this line from p2.inf and any 
regular installation of any framework extension will automatically add 
the full versioned bundle name to the osg.framework.extensions property?

Comment 20 Simon Kaegi CLA 2009-03-31 12:01:02 EDT
(In reply to comment #19)
> Are you saying I can now completely drop this line from p2.inf and any 
> regular installation of any framework extension will automatically add 
> the full versioned bundle name to the osg.framework.extensions property?

That's right. We should just handle this under the covers. The fact that we use some property in config.ini is an implementation detail that should now be hidden away.

Comment 21 Simon Kaegi CLA 2009-04-05 19:00:30 EDT
Marking FIXED.

Andrew, Stephan -- it would be much appreciated if you guys could verify with your setups. Thanks.
Comment 22 Andrew Eisenberg CLA 2009-04-05 23:32:05 EDT
Thanks for incorporating the fix.  I'm happy to try it out.  Is this available from a build somewhere, or do I have to get it from cvs?
Comment 23 Thomas Watson CLA 2009-04-06 08:46:35 EDT
(In reply to comment #22)
> Thanks for incorporating the fix.  I'm happy to try it out.  Is this available
> from a build somewhere, or do I have to get it from cvs?
> 

The latest I-Build should contain this fix.
Comment 24 Andrew Eisenberg CLA 2009-04-08 17:07:00 EDT
(In reply to comment #23)
> The latest I-Build should contain this fix.

I am currently using the latest N build and I have this version of the bundle:
org.eclipse.equinox.p2.touchpoint.eclipse_1.0.100.N20090406-2000.jar

But I do not see the class InstallExtensionAction (whereas InstallBundleAction is there).  Has this patch made it into the build yet?
Comment 25 Simon Kaegi CLA 2009-04-08 17:16:18 EDT
(In reply to comment #24)
> (In reply to comment #23)
> > The latest I-Build should contain this fix.
> I am currently using the latest N build and I have this version of the bundle:
> org.eclipse.equinox.p2.touchpoint.eclipse_1.0.100.N20090406-2000.jar
> But I do not see the class InstallExtensionAction (whereas InstallBundleAction
> is there).  Has this patch made it into the build yet?

Yes, however there is no separate installextensionaction. The installbundleaction is smart enough to do the right thing for framework extensions now.
Comment 26 Andrew Eisenberg CLA 2009-04-08 19:12:11 EDT
(In reply to comment #25)
> Yes, however there is no separate installextensionaction. The
> installbundleaction is smart enough to do the right thing for framework
> extensions now.
> 

I guess I'm just a little confused in that I don't see any evidence of this fix.  In fact, I still see this line in InstallBundleAction:

if (iu.isFragment()) {
	System.out.println("What is a fragment doing here!!! -- " + iu); //$NON-NLS-1$
	return Status.OK_STATUS;
}


If InstallBundleAction is now handling framework extensions as well, wouldn't this line make the install fail (since all framework extensions are bundle fragments)?

I'm sure that I'm missing something here.  I am going to try it out anyway as soon as I can mock up an install site with the proper (lack of) metadata.
Comment 27 Pascal Rapicault CLA 2009-04-08 22:03:26 EDT
This check is unrelated to bundle fragments. What is being checked in the snippet you show is whether the IU being processed is an IUfragment which should never be treated alone.
Please give it a try.
Comment 28 Andrew Eisenberg CLA 2009-04-09 15:00:58 EDT
Finally got this all working in my set up and everything looks great.

Thanks everyone for putting this fix in!
Comment 29 Stephan Herrmann CLA 2009-04-17 17:37:16 EDT
(In reply to comment #21)
> Marking FIXED.
> 
> Andrew, Stephan -- it would be much appreciated if you guys could verify with
> your setups. Thanks.

I finally started joining the 3.5 train, and: YES it works for me too.

The real good news: for the first time ever, I can uninstall our feature
containing the framework extension, revert to older version etc.

Thanks everybody