Bug 182930 - JavaModelCache's size grows when displaying type hierarchy
Summary: JavaModelCache's size grows when displaying type hierarchy
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.3   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.3 M7   Edit
Assignee: Jerome Lanneluc CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2007-04-18 07:34 EDT by Eric Jodet CLA
Modified: 2007-05-03 11:02 EDT (History)
0 users

See Also:


Attachments
screen shot (149.71 KB, image/jpeg)
2007-04-18 07:36 EDT, Eric Jodet CLA
no flags Details
Proposed fix (6.77 KB, patch)
2007-04-23 04:45 EDT, Jerome Lanneluc CLA
no flags Details | Diff
Other improvement (24.87 KB, patch)
2007-04-23 17:03 EDT, Jerome Lanneluc CLA
no flags Details | Diff
Other improvement v2 (10.18 KB, patch)
2007-04-24 04:29 EDT, Jerome Lanneluc CLA
no flags Details | Diff
Other improvement v2 (15.13 KB, patch)
2007-04-24 05:51 EDT, Jerome Lanneluc CLA
no flags Details | Diff
Patch against R3_2_maintenance (18.37 KB, patch)
2007-05-03 10:04 EDT, Jerome Lanneluc CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Jodet CLA 2007-04-18 07:34:42 EDT
3.3 M6 + org.eclipse.jdt.core_3.3.0.v_749.jar
sun jre 1.6.0_02-ea

- installed europa on top M6
- imported all plugins / features in a new wks (1000 + projects)
- show view --> Progress
- Open Type Hierarchy --> open type core.runtime.Plugin
- click ok and wait for the hierarchy to be displayed

--> noted that allocated memory grew from 128 to 332 Mo

Took several Yourkit MEM snapshots and found that the increase of allocated memory was due to JavaModelCache (see to be attached screen shot)

wonder if this is expected behavior
Comment 1 Eric Jodet CLA 2007-04-18 07:36:13 EDT
Created attachment 64162 [details]
screen shot

delta observed between 2 yourkit snapshots
Comment 2 Eric Jodet CLA 2007-04-18 07:58:18 EDT
had another test run with yourkit.
once hierarchy displayed, forced a gc: used memory decreased from 240 to 190 Mo.
Still, according to the snapshot taken right after the gc, the JavaModel retained size is still 68 Mo.
Comment 3 Eric Jodet CLA 2007-04-18 07:59:23 EDT
please read: (...) the JavaModelCache retained size is (...)
Comment 4 Philipe Mulet CLA 2007-04-18 20:37:36 EDT
This feels a lot. Are hierarchy binary types still being used ?
Comment 5 Jerome Lanneluc CLA 2007-04-23 04:45:47 EDT
Created attachment 64576 [details]
Proposed fix

This fix consists in removing the 'isPackage' set from the ProjectCache as it takes a huge space in a workspace with lots of roots.
Comment 6 Jerome Lanneluc CLA 2007-04-23 06:47:51 EDT
Patch released for 3.3M7 in HEAD.

Note that with this patch the retained size of the ProjectCache goes from 45MB to 27MB on my monster workspace (after computing the hierarchy on org.eclipse.core.runtime.Plugin).

Keep investigating if more memory can be gained.
Comment 7 Jerome Lanneluc CLA 2007-04-23 17:03:39 EDT
Created attachment 64656 [details]
Other improvement

This improvement consists in having on package fragment cache per root and iterating through these caches.

This reduces the memmory used by the ProjectCache in the monster workspace to 6MB. However this slows reconciling down.
Comment 8 Jerome Lanneluc CLA 2007-04-24 04:29:36 EDT
Created attachment 64691 [details]
Other improvement v2

This improvement still shares the package fragment caches, but brings back the flattened package fragment cache per project. This brings performances back to normal and memory of the ProjectCache for the monster workspace is now 23MB.
Comment 9 Jerome Lanneluc CLA 2007-04-24 05:51:17 EDT
Created attachment 64707 [details]
Other improvement v2

Uses a HashSetOfArray to store package names instead of a HashtableOfArrayToObject.
Comment 10 Jerome Lanneluc CLA 2007-04-24 07:19:22 EDT
Other improvement v2 released for 3.3M7 in HEAD.

To conclude: 
1. the JavaModelCache's size will still grow the first time the type hierarchy is computed (as the ProjectCaches are initialized). But it will remain stable the same that.
2. we gained 22MB on the monster workspace scenario
Comment 11 Eric Jodet CLA 2007-04-27 04:46:14 EDT
Running the exact same scenario with a recent build, allocated mem never exceeded 184 Mo (compared to the 332 Mo without the fix)!
Also took a Yourkit snapshot, where JavaModelManager is now 38 Mo compared to the 68 Mo with previous version.

Verified for 3.3 M7 using build I20070427-0010
Comment 12 Jerome Lanneluc CLA 2007-05-03 10:04:42 EDT
Created attachment 65764 [details]
Patch against R3_2_maintenance

This patch:
- merges both "Proposed fix" patch and "Other improvement v2" patch
- includes the fix for bug 183833 (which was a regression of "Other improvement v2" patch)

Also note that this patch also fixes bug 161175 (since children of package fragment roots are computed only once).
Comment 13 Jerome Lanneluc CLA 2007-05-03 11:02:19 EDT
Patch released in R3_2_maintenance