| RE: [platform-core-dev] How much memory can a Workspace take? |
Hi Martin,
Could we move the discussion to the bug? I think the one mentioned by James
is fine.
This way anyone would be able to track and join the discussion.
Thanks
--
Szymon Brandys
"Oberhuber,
Martin"
<Martin.Oberhuber To
@windriver.com> "Eclipse Platform Core component
Sent by: developers list."
platform-core-dev <platform-core-dev@xxxxxxxxxxx>
-bounces@eclipse. cc
org
Subject
RE: [platform-core-dev] How much
2009-09-29 15:40 memory can a Workspace take?
Please respond to
"Eclipse Platform
Core component
developers list."
<platform-core-de
v@xxxxxxxxxxx>
Hi again,
With some expert help from the MAT people, I think we've been
able to detect a suspect responsible for the problem in my case.
Here is what we did:
1. In memory analyzer, allow the default report ("wizard") to run.
2. Expand details of first problem suspect.
3. On first item (AbstractDataTreeNode[174] @ 0x97887040 :
- click > Show retained set.
--> 209000 objects of type char[] take 66MB
4. On char[]: right-click > Java Basics > Group by Value
--> See 209000 Strings, all of which have the same prefix
-->
"/net/pek-wb-build1/buildarea/kcao/182581/build/linux/wrs/patches"
At this point, it seems clear that there is room for optimization.
Why does the workspace retain such long Prefixes? An ElementTree
in the Workspace should always only hold the basename of an element
in each node.
5. Pick the first long char[].
Right-click > List Objects > With incoming references
At this point, I am getting attached screenshot: It looks like the
long char[] array is held by a String with value
"fix-swapped-nibbles-in-tracer"
So although the value of that String is only a basename, that String
still retains a very long char[] array with the entire path!
It looks like this is a peculiarity of the Java String.substring()
operation, which retains the entire original char[] array although
the String now only references the trailing part of it.
Unfortunately the memory analyzer cannot tell us who performs the
substring() operation that still retains the large char[] array.
But it looks like it should be possible to optimize away my memory
problem by replacing
fullPath.substring(lastSlashIdx);
by
new String(fullPath.substring(lastSlashIdx);
by explicitly performing a new String() operation at this point, the
original substring which retains the char[] array will be released
for garbage collection, and the new String will only retain the
desired substring.
Fellow committers, do you have an idea where that substring() operation
could be? I suspect somewhere in UnifiedTree from the refresh operation,
but I am not sure...
Cheers,
--
Martin Oberhuber, Senior Member of Technical Staff, Wind River
Target Management Project Lead, DSDP PMC Member
http://www.eclipse.org/dsdp/tm
> -----Original Message-----
> From: platform-core-dev-bounces@xxxxxxxxxxx
> [mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of
> Oberhuber, Martin
> Sent: Tuesday, September 29, 2009 2:12 PM
> To: Eclipse Platform Core component developers list.
> Subject: RE: [platform-core-dev] How much memory can a Workspace take?
>
> Hi Szymon, James -
>
> my hprof.zip is at
> http://dl.getdropbox.com/u/1975868/java_pid11610.zip
>
> The ZIP is 68MB large (the 200MB had also included the entire
> .metadata with some large CDT pdom's).
>
> As mentioned this was with Eclipse-3.5.0 on Linux-GTK RHEL4 (32-bit).
> The problem happened directly after launching, during a workspace
> refresh that was initiated because a previous shutdown was unclean.
>
> Here are some probably interesting bits from the .log file. It looks
> like the machine was very busy at that time -- it took 6 minutes
> from initial Launch till refreshing the workspace, and it took 19
> minutes from starting the WS refresh till running into the
> OutOfMemoryError:
>
> !SESSION 2009-09-28 12:19:25.891
> -----------------------------------------------
> eclipse.buildId=unknown
> java.version=1.5.0_11
> java.vendor=Sun Microsystems Inc.
> BootLoader constants: OS=linux, ARCH=x86, WS=gtk, NL=en_US
> Command-line arguments: -os linux -ws gtk -arch x86 -data
> /folk/hxu2/WindRiver/workspace2
>
> !ENTRY org.eclipse.core.resources 2 10035 2009-09-28 12:25:34.028
> !MESSAGE The workspace exited with unsaved changes in the previous
> session; refreshing workspace to recover changes.
>
> !ENTRY org.eclipse.core.resources 4 1 2009-09-28 12:44:01.498
> !MESSAGE Internal Error
> !STACK 0
> java.lang.OutOfMemoryError: Java heap space
>
> Cheers,
> --
> Martin Oberhuber, Senior Member of Technical Staff, Wind River
> Target Management Project Lead, DSDP PMC Member
> http://www.eclipse.org/dsdp/tm
>
>
>
> > -----Original Message-----
> > From: platform-core-dev-bounces@xxxxxxxxxxx
> > [mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of
> > Szymon Brandys
> > Sent: Tuesday, September 29, 2009 1:28 PM
> > To: Eclipse Platform Core component developers list.
> > Subject: Re: [platform-core-dev] How much memory can a
> Workspace take?
> >
> > Hi Martin,
> > Could I get the .hprof file?
> > --
> > Szymon
> >
> >
> >
> >
> > "Oberhuber,
> >
> > Martin"
> >
> > <Martin.Oberhuber
> > To
> > @windriver.com> "Eclipse Platform Core
> > component
> > Sent by: developers list."
> >
> > platform-core-dev
> > <platform-core-dev@xxxxxxxxxxx>
> > -bounces@eclipse.
> > cc
> > org
> >
> >
> > Subject
> > [platform-core-dev]
> > How much memory
> > 2009-09-29 12:30 can a Workspace take?
> >
> >
> >
> >
> >
> > Please respond to
> >
> > "Eclipse Platform
> >
> > Core component
> >
> > developers list."
> >
> > <platform-core-de
> >
> > v@xxxxxxxxxxx>
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Hi fellow committers,
> >
> >
> > I got a report about an OutOfMemoryError generated during a
> Workspace
> > Refresh operation. When looking at the .hprof file with the
> > MemoryAnalyzer,
> > it turned out that pretty much all the memory was taken by
> > core/Workspace
> > only:
> > 100 MB (53%) in org.eclipse.core.internal.resources.Workspace
> > (occupied by ElementTree).
> > 52 MB (27%) in
> > org.eclipse.core.internal.dtree.DeltaDataTree (this is
> > a different additional DeltaDataTree)
> > 37 MB all the rest.
> >
> >
> > The workspace was arguably very large, but still I am surprised
> > 1. that the numbers are that large
> > 2. that in addition to the Workspace, there is such a
> > large additional
> > DeltaDataTree,
> > 3. that I see an OutOfMemoryError reported with 189MB
> > total where I used
> > -vmargs -Xmx320m ... what happened to the extra 131MB ?
> >
> >
> > Do my numbers sound reasonable, or could there be a bug
> that leads the
> > Workspace to take an excessive amount of memory? How much
> memory do we
> > expect a workspace to take for, say 50000 files? How much
> *additional*
> > memory may a workspace delta take that is generated during a
> > Refresh? Would
> > any additional investigation using the Coretools make sense
> > at this point?
> >
> >
> > I can make the .hprof file available on request (it's a 200MB
> > ZIP file).
> >
> >
> > Many thanks for any pointers!
> >
> >
> > Cheers,
> > --
> > Martin Oberhuber, Senior Member of Technical Staff, Wind River
> > Target Management Project Lead, DSDP PMC Member
> > http://www.eclipse.org/dsdp/tm
> >
> > _______________________________________________
> > platform-core-dev mailing list
> > platform-core-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/platform-core-dev
> >
> >
> > _______________________________________________
> > platform-core-dev mailing list
> > platform-core-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/platform-core-dev
> >
> _______________________________________________
> platform-core-dev mailing list
> platform-core-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/platform-core-dev
>
(See attached file: wsOome.gif)
_______________________________________________
platform-core-dev mailing list
platform-core-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-core-dev
Attachment:
wsOome.gif
Description: GIF image
