Bug 189725 - Lots of memory allocated by SaveManager
Summary: Lots of memory allocated by SaveManager
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 3.3   Edit
Hardware: PC Windows XP
: P2 critical (vote)
Target Milestone: 3.3 RC3   Edit
Assignee: John Arthorne CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
: 75427 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-05-29 13:16 EDT by Martin Aeschlimann CLA
Modified: 2007-06-22 11:07 EDT (History)
4 users (show)

See Also:
dj.houghton: review+
Michael.Valenta: review+


Attachments
Fix (5.19 KB, patch)
2007-05-30 14:40 EDT, John Arthorne CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Aeschlimann CLA 2007-05-29 13:16:52 EDT
3.3 RC2

I made a memory dump of my active development workspace and found that 10MB are kept by the SaveManager.savedStates

There are 2 entries,
org.eclipse.hyades.probekit, retained size 10 MB
org.eclipse.jdt.core, retained size 400 Bytes

Interestingly, I haven't org.eclipse.hyades.probekit installed anymore, actually for quite a long time.

I'm not sure if I correctly understood what ISavedState is for, but is it collecting all resource deltas until Hyades will be installed again?!

I'm not sure if this is related, but I also think shutting down takes very long on my machine.
Comment 1 John Arthorne CLA 2007-05-29 15:22:07 EDT
Yes, it is essentially collecting resource deltas until that save participant asks for it again.  The default retention policy is to keep deltas for 30 days. There is a preference org.eclipse.core.resources/delta.expiration that controls the expiration date.  It can also be cleared by the API IWorkspace#forgetSavedTree. I'm assuming you haven't touched that preference - is it possible that you had that plugin installed within the last 30 days?

We could improve this by adding a check to see if the plugin is still installed. Note to myself: see SaveManager.isOldPluginTree.
Comment 2 Martin Aeschlimann CLA 2007-05-30 04:01:09 EDT
I think I didn't use that plug-in for almost a year.

I just looked in About > Configuration Details that I haven't set 'org.eclipse.core.resources/delta.expiration', so I assume the default is used.

I debugged into SaveManager.getDeltaExpiration(..) and for "org.eclipse.hyades.probekit" I got 
'Wed May 30 09:27:32 CEST 2007', which was just half an hour ago.

Later I saw that the hyades timestamp is written on shutdown with currentTimeMillis . So that seems to be the bug.

So testing if a plug-in is still in the install would be a good idea, as 30 days is still a long time.

I hope you can do something for 3.3, as this is, for my workspace, a major startup/shutdown and runtime performance penalty.
Comment 3 John Arthorne CLA 2007-05-30 10:18:22 EDT
You're right about the source of the bug.  It's incredible, it looks like this bug has existed forever (since 1.0). Investigating a fix for the timestamp problem...

By the way, the 10MB size might be incorrect - I don't know how the value was computed.  All resource trees are linked together, and the oldest tree will have pointers to all newer trees. So, following the references from savedStates will pull in the trees for every builder, and the current tree, with all of the various properties and data attached to it. Still, this is a potentially severe memory if a workspace is held onto for a long time.
Comment 4 Martin Aeschlimann CLA 2007-05-30 12:24:07 EDT
The 10MB is what yourkit said is the retained size. So that might not be fully true, you're right.
Comment 5 John Arthorne CLA 2007-05-30 14:16:11 EDT
I have a fix prepared for both updating expiration timestamp correctly, and discarding deltas for uninstaled plugins.

From my understanding of the endgame rules, I need two additional committers to review the patch, and three committers to approve the fix.  DJ, Michael: can you stop by and review the patch with me?  Martin, Dani: can one of you add fix approval as well?

My large self-hosting workspace was affected by this as well.  I had a stale delta being maintained from when I installed CDT a long time ago.  After restarting with the fix, my allocated memory dropped by 16MB (as reported by Windows). This is an extreme case - this has been my main development workspace for five years.
Comment 6 Dani Megert CLA 2007-05-30 14:18:42 EDT
This is a good one. Approving to fix.
Comment 7 John Arthorne CLA 2007-05-30 14:40:51 EDT
Created attachment 69355 [details]
Fix
Comment 8 John Arthorne CLA 2007-05-30 15:26:57 EDT
Fix released.
Comment 9 John Arthorne CLA 2007-06-22 11:07:29 EDT
*** Bug 75427 has been marked as a duplicate of this bug. ***