Bug 232356 - [GC] Option to remove the plugins automatically when uninstall feature
Summary: [GC] Option to remove the plugins automatically when uninstall feature
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 enhancement with 2 votes (vote)
Target Milestone: Juno M1   Edit
Assignee: John Arthorne CLA
QA Contact:
URL:
Whiteboard:
Keywords: helpwanted
: 243621 272154 329472 350124 (view as bug list)
Depends on:
Blocks: 361591
  Show dependency tree
 
Reported: 2008-05-15 13:31 EDT by Samuel Wu CLA
Modified: 2011-10-20 16:27 EDT (History)
16 users (show)

See Also:


Attachments
GC app (3.58 KB, application/java-archive)
2008-09-20 14:30 EDT, Pascal Rapicault CLA
no flags Details
Unzip and drop plugin jar into dropins folder (2.84 KB, application/zip)
2010-12-01 10:04 EST, Catherine Griffin CLA
no flags Details
Fix v01 (5.13 KB, patch)
2011-07-19 14:11 EDT, John Arthorne CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Samuel Wu CLA 2008-05-15 13:31:28 EDT
Build ID: I20080513-0817

Steps To Reproduce:
1. Download the update site zip file from the following URL and upload it to an http server
ftp://ftp.emea.ibm.com/fromibm/pmr/83938,49R,000/update.zip
(the file will be removed after 05/22/2008)
2. Add this site to the available software site
3. Select and install feature com.ibm.test.parent
4. Restart workbench and uninstall the feature com.ibm.test.parent
5.  When feature com.ibm.test.parent was uninstalled, the feature and the included ones were removed from the file system. But the plugins were not. The feature and included features contains a couple of plugins. They should be removed from the file system.

More information:
Comment 1 Pascal Rapicault CLA 2008-05-15 22:17:49 EDT
This works as expected. When the system is running the plug-ins can not be removed because they could be running or they could be necessary to close.
Plug-ins are removed the next time an install or an uninstallation is performed implicitly or explicitly.
This also allows the user to change is mind and reinstall something that he could have accidentally uninstalled.
Comment 2 Samuel Wu CLA 2008-05-16 10:09:03 EDT
Well, I can't see how this helps a user who uninstalls something by mistake. The user has to reinstall the feature anyway. And the common sense of the installation would tell the user that the plugin would be downloaded again from the update site. It would be quite a surprise that the plugins are not downloaded again when the feature is reinstalled. 
I think it mixes up to disable a feature and uninstall a feature here. A user can disable a feature without removing it. But uninstall means remove the feature physically.
We haven't started timestamping our plugins yet. Our corporate customers need the forth digit of the version to provide corporate wide support. If the plugins are not downloaded again from the update site during the installation, I can't see how we are going to pick up the updated plugins.
I would think this is a bug more than a feature.
As to unable to remove a running plugin, it's just an implementation issue. There is always a way if there is a will. They can be removed when the workbench restarts itself for instance.
Comment 3 John Arthorne CLA 2008-05-16 10:23:23 EDT
Samuel, the plugins *are* deleted, they just aren't deleted immediately after uninstall. I can't recall exactly when the garbage collection runs, but the plugins will disappear at some point when the garbage collection is next triggered. It will either be during the next session, or the next time a change is made to the profile (the next install/uninstall/update of that configuration).

I may have misunderstood your comment, but if you are delivering different plugin contents later on with the exact same four part plugin qualifier, this is a bug in your delivery process. This makes it impossible to audit exactly what users have installed because there could be multiple versions of the plugin with the same version number.
Comment 4 Samuel Wu CLA 2008-05-16 12:21:28 EDT
John, thank you for your concern on our deployment process. As long as p2 can remove the uninstalled plugins as promised and redownload it when reinstalling, we are fine.
Comment 5 Samuel Wu CLA 2008-05-16 13:59:47 EDT
John,
The plugins are not removed as claimed. 
1. I uninstalled the feature which left behind the plugins.
2. I rebuilt the feature so that the same plugins can have new content.
3. I reinstall the feature 
But the plugins were not updated during the installation. They still contained the old contents.
Comment 6 Pascal Rapicault CLA 2008-05-16 15:37:44 EDT
p2, like update is based on the assumption that 
  "one bundle id" + "one version (including qualifier)" == one set of bytes for ever.

As John indicated, failing to do this is just a recipe for disaster and management nightmare. PDE Build has full support to generate a qualifier.
Comment 7 John Arthorne CLA 2008-05-16 15:43:04 EDT
And, the plugins are eventually garbage collected, but just not as soon as you would expect. Pascal, we may want to look at improving the gc triggers. I think the gc doesn't currently happen until after a subsequent uninstall. I.e.:

- Install A
- Install B
- Uninstall A
- Uninstall B -> Now A is gc'd
- Uninstall C -> Now B is gc'd

This is not tragic, but may take users by surprise and lead them to think gc is broken.
Comment 8 Samuel Wu CLA 2008-05-16 16:19:40 EDT
If we do need keep the uninstalled plugins around, which I don't see a reason, we'd better let the user know and we have to provide a mean for the user to remove them. It's just way too much that you have to uninstall another feature to remove the uninstalled plugins.
Comment 9 Steve Francisco CLA 2008-05-16 16:48:03 EDT
Would it make sense to have an install action trigger a gc before it starts? This would flush out deletions and let the install not just be an undelete.
Comment 10 Pascal Rapicault CLA 2008-05-17 16:04:39 EDT
The GC can programmatically be disabled and in the future we want to have it managed by the user (see bug 212078). Relying on the removal of the artifact will only get you so far.
Comment 11 Steve Francisco CLA 2008-05-18 14:33:31 EDT
I'm curious - what is the purpose of putting the deleted plugins in a deletion queue instead of simply deleting them?  If the user was performing the delete to reclaim disk space they would be disappointed.
Comment 12 Andreas Goetz CLA 2008-06-02 06:56:40 EDT
Regarding comment 3, is it possible to trigger the plugin deletion and to influence how many and for how long older plugin versions are kept? Also- is deleting also triggered when installing a newer feature version without prior explicit uninstallation of older version?
When working with nightly update sites during development the file space can become an issue, e.g. my plugins folder is at 600mb already with not too many plugins installed.
Comment 13 John Arthorne CLA 2008-06-02 09:41:14 EDT
Re comment #11 - plug-ins are not deleted immediately because they may be still be running at the time of uninstall and deletion will typically fail. Also, a user may decide to revert (undo) the uninstall shortly after doing it, if it was done by accident or they uninstalled the wrong feature.
Comment 14 Pascal Rapicault CLA 2008-06-02 09:46:37 EDT
> Is it possible to trigger the plugin deletion and to influence how many and for how long older plugin versions are kept?
 This is not possible in 1.0 but it is planned for future releases (bug 212078). 

>is deleting also triggered when installing a newer feature version without prior explicit uninstallation of older version?
  Yes. It is triggered any time a plug-in is updated or uninstalled as a result of some more general installation operation, whatever that one is.
Comment 15 Samuel Wu CLA 2008-06-24 13:28:57 EDT
In addition to the automatic removement of the old features, p2 should provide a way so that the user can manually remove them.
Comment 16 Ashish Patel CLA 2008-07-17 08:29:43 EDT
I find that this is also happening when I have a custom install handler in my feature.  So after my uninstallInitiated method completes the plugins for my features aren't removed.  I set a break point in the p2.garbagecollector.GCActivator.refreshGCTrigger() and noticed that its never fired after uninstalling.
Comment 17 John Arthorne CLA 2008-08-13 10:13:42 EDT
*** Bug 243621 has been marked as a duplicate of this bug. ***
Comment 18 Gabriele Garuglieri CLA 2008-09-16 04:34:16 EDT
Excuse me, would it be possible as an option to have back the old UM behavior?
With it the plugins were deleted immediately or at first restart if in use.
Comment 19 Andreas Goetz CLA 2008-09-16 07:56:33 EDT
..or have an option to trigger the P2 garbage collector, even if per command line?
Comment 20 Pascal Rapicault CLA 2008-09-16 21:49:20 EDT
In 3.5, we added a GC application. Gabriele, Andreas, what is the main motivation to eagerly remove the bundles? I'm trying to understand the use case that requires it.
Comment 21 Gabriele Garuglieri CLA 2008-09-17 03:18:53 EDT
Hi Pascal,
i appreciate the fact that the bundle pool act as trash can from which i can recover the plugins should i change my mind. But once i delete something i'd like to have also the option to reclaim immediately the space if i want and not condition it to the next installation or uninstallation, that could not happen for a long time.

If we can manually trigger GC that's enough but i'd like to see it implemented in 3.4 too and not wait another year or more.

Excuse me, this is general consideration, perhaps a bit OOT, but if you pospose any functional enhancement to P2 to 3.5, the 3.4 release, at least for my group, will be completely skipped.

As a matter of fact here we are about fifty developers and several of us attempted to use P2 to reach something similar to the installation env we used successfully for several years, but we could not come even near to it.
I'm sure we didn't grasp yet the new P2 paradigm, model documentation is somewhat sparse, but the sense of loss of control over the installation was such that we gave up.
I'll continue, time allowing it, to experiment with 3.4, but we felt P2 is still too unstable and functionally incomplete for production and we decided to stick to 3.3 until P2 is enhanced enough to give us the complete CONTROL and KNOWLEDGE of WHAT is installed, uninstalled and WHERE, comparable to what we have now with UM.

If anything over there may sound offensive to someone, i apologize in advance, that was not my intention.
Comment 22 Pascal Rapicault CLA 2008-09-20 14:30:36 EDT
Created attachment 113065 [details]
GC app

Gabriele, please find attached a quickly written GC application to run on 3.4.x. It is not tested, but please give it a try and let me know if this helps.

As for your other points, p2 takes a complete different approach to things than UM which may be what throw you off. That said I don't want to discuss them here because it is not the appropriate forum but I would really want to understand your scenarios, so feel free to drop me a note or comment on the mailing list. Thx.
Comment 23 Gabriele Garuglieri CLA 2008-09-23 01:59:28 EDT
Thank you Pascal, pls be patient for a few days, i'll try to feedback you both on the GC app and on the mailing list as soon as i can.
Comment 24 Gabriele Garuglieri CLA 2008-09-26 05:55:53 EDT
Hi Pascal,
i tested your garbage collector and it looks to be working.
I didn't do an in depth test, i tried to uninstall some features, update some others and the uninstalled plugins and old version plugins are removed immediately with no apparent side effect.

Now if you can make this behavior optional, so that one can choose if he wish to reclaim immediately the disk space or keep uninstalled plugins for later use, or just launch the garbage collector at wish, that would be perfect.

I must say that there is one case i didn't verify. I tested the behavior on a shared bundle pool installation, but the uninstalled  plugins were not used by any other profiles, too much work to build such configuration  and too little time available, but i guess you have considered the case that when plugins are uninstalled from one profile they must not be deleted if in use by another profile.

Regards,  Gabriele
Comment 25 Pascal Rapicault CLA 2008-09-29 23:23:57 EDT
> I tested the behavior on a shared bundle pool installation, but the uninstalled  plugins were not used by
any other profiles,
  This works properly as long as all the profiles using the bundle pool are all managed from the same profile registry (same p2/org.eclipse.equinox.p2.engine).
Comment 26 Pascal Rapicault CLA 2009-04-16 21:57:11 EDT
*** Bug 272154 has been marked as a duplicate of this bug. ***
Comment 27 John P. Petrakis CLA 2010-01-14 10:17:10 EST
It has been my rather unfortunate experience with uninstalling features that the plugins (even in 3.5.1 GA) never go away.  I installed a feature, then I updated that feature.  I uninstalled the update, restarted, then I reverted the config to the previous one which had the first version of the feature installed, and the updated plugins did NOT get removed.  So I am still seeing this behavior.
Comment 28 jlundgren CLA 2010-08-12 12:16:48 EDT
(In reply to comment #27)
> It has been my rather unfortunate experience with uninstalling features that
> the plugins (even in 3.5.1 GA) never go away.  I installed a feature, then I
> updated that feature.  I uninstalled the update, restarted, then I reverted the
> config to the previous one which had the first version of the feature
> installed, and the updated plugins did NOT get removed.  So I am still seeing
> this behavior.

I can confirm that this issue remains open in Helios 3.6 (20100617-1415).
I have yet to find a situation where the GC actually kicks in. I have also tried manually to force the gc with:
./eclipse -application org.eclipse.equinox.p2.garbagecollector.application
but am getting fileNotFound exceptions, although that might be specific to my system. 
I really hope that you chose to physically remove all features and plugins artifacts when the user uninstalls. 

Thank you,
Jens
Comment 29 Pascal Rapicault CLA 2010-11-05 06:48:44 EDT
*** Bug 329472 has been marked as a duplicate of this bug. ***
Comment 30 Samuel Wu CLA 2010-11-05 08:48:49 EDT
Pascal,
Any plan to fix this problem? It has been pending for more than two years...
Comment 31 Pascal Rapicault CLA 2010-11-09 20:53:39 EST
This is not a problem since this is working as expected. At best this is an enhancement request. 
Fixing it means scheduling an application / job to run next time the system is rebooted since there is at the time of the uninstall the plugins can still be in use.
Comment 32 jlundgren CLA 2010-11-10 04:51:03 EST
(In reply to comment #31)
> This is not a problem since this is working as expected. At best this is an
> enhancement request. 
> Fixing it means scheduling an application / job to run next time the system is
> rebooted since there is at the time of the uninstall the plugins can still be
> in use.

I still fail to understand why this is not a bug. If I want to delete a feature, I do not expect the feature to work next time I start my Eclipse instance. It is extremely frustrating trying to develop features and testing them out by installing them "properly" when you afterwards know you have to nuke your entire Eclipse instance to make sure there's no residue. And what about updating a feature where the old jar remains physically in your plugins directory - A nightmare.
Comment 33 Catherine Griffin CLA 2010-11-10 07:09:34 EST
(In reply to comment #22)
The GC app attached here does not work - the plugin.xml has the wrong class name for the application.  After fixing that up I was able to run it and it worked well.  Would it be possible to have a corrected version attached here, together with instructions for running it ?  Thanks.
Comment 34 Samuel Wu CLA 2010-11-10 09:18:30 EST
(In reply to comment #31)
> This is not a problem since this is working as expected. At best this is an
> enhancement request. 
> Fixing it means scheduling an application / job to run next time the system is
> rebooted since there is at the time of the uninstall the plugins can still be
> in use.

It's not a common practice to uninstall a software without removing the parts it has installed. I don't know how many people expect it to work this way, but it is definitely not as expected by our customers. Our customers want to clean up and reuse the disk space by uninstalling the feature. 
The uninstalled plugins might be still in use when uninstallation happens. It only means that a better way needs to be found to address this issue. It can never be used as an excuse that those plugins will not be removed.
Comment 35 Pascal Rapicault CLA 2010-11-21 20:57:29 EST
Catherine since you have fixed up the app, please attach your fixed up version. Thx.
Comment 36 Catherine Griffin CLA 2010-12-01 10:04:02 EST
Created attachment 184266 [details]
Unzip and drop plugin jar into dropins folder

The attachment GCapp_csg is the same as GC app except for a corrected plugin.xml file.
Comment 37 DJ Houghton CLA 2011-01-31 16:03:54 EST
Pascal, do you still have the source for your app? What exactly does it do? Just run the GC when the bundle starts?
Comment 38 DJ Houghton CLA 2011-01-31 16:29:34 EST
If we do make changes in this area we need to be careful to play nice with the changes proposed in bug 227547. We don't want bundles deleted on startup (or any other time) that the reconciler wants to try and install.
Comment 39 DJ Houghton CLA 2011-02-07 17:45:59 EST
I've talked to Pascal and the bundle just called the GC app on startup. 

Changing the current GC behaviour in the Eclipse base is not something that we want to do. 

That being said, we could potentially provide a preference that products could set if they want the GC to run. I'll have to check but depending on how we do this, the GC bundle maybe still need to be referenced in order to kick it to start (because it is lazy-start).
Comment 40 DJ Houghton CLA 2011-06-23 08:12:22 EDT
*** Bug 350124 has been marked as a duplicate of this bug. ***
Comment 41 John Arthorne CLA 2011-07-19 14:08:27 EDT
(In reply to comment #39)
> That being said, we could potentially provide a preference that products could
> set if they want the GC to run. I'll have to check but depending on how we do
> this, the GC bundle maybe still need to be referenced in order to kick it to
> start (because it is lazy-start).

A preference that products can set seems like a reasonable approach.
Comment 42 John Arthorne CLA 2011-07-19 14:11:26 EDT
Created attachment 199926 [details]
Fix v01

With this patch, adding the following to a product's plugin_customization.ini will cause a GC to occur on each startup (after reconciler has run):

org.eclipse.equinox.p2.ui.sdk.scheduler/gcOnStartup=true
Comment 45 Samuel Wu CLA 2011-09-09 16:15:53 EDT
Hi John,
We are still on Eclipse 3.6. Wonder whether this fix can be ported back to 3.6. Thanks.