[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [platform-core-dev] How much memory can a Workspace take?
|
By taking a quick pass with the Call Hierarchy view. It looks like the
main entry point for creating the DataTreeNodes is
ElementTree#createElement(IPath key, Obj data).
This calls tree.createChild(parent, key.lastSegment(), data); which
could be the source of the issue. IPath.lastSegment() returns the last
segment the segments having been created by substring...
As each IResource has it's fullPath as a full IPath, the result is
that the DataTree contains yet another handle on the Resource's
FullPath. If the Resource is re-created the DataTree still has a
handle to the old fullpath string, for every item in the tree.
Cheers,
James
2009/9/29 Oberhuber, Martin <Martin.Oberhuber@xxxxxxxxxxxxx>:
> Hi again,
>
> With some expert help from the MAT people, I think we've been
> able to detect a suspect responsible for the problem in my case.
>
> Here is what we did:
> 1. In memory analyzer, allow the default report ("wizard") to run.
> 2. Expand details of first problem suspect.
> 3. On first item (AbstractDataTreeNode[174] @ 0x97887040 :
> - click > Show retained set.
> --> 209000 objects of type char[] take 66MB
> 4. On char[]: right-click > Java Basics > Group by Value
> --> See 209000 Strings, all of which have the same prefix
> -->
> "/net/pek-wb-build1/buildarea/kcao/182581/build/linux/wrs/patches"
>
> At this point, it seems clear that there is room for optimization.
> Why does the workspace retain such long Prefixes? An ElementTree
> in the Workspace should always only hold the basename of an element
> in each node.
>
> 5. Pick the first long char[].
> Right-click > List Objects > With incoming references
>
> At this point, I am getting attached screenshot: It looks like the
> long char[] array is held by a String with value
> "fix-swapped-nibbles-in-tracer"
> So although the value of that String is only a basename, that String
> still retains a very long char[] array with the entire path!
>
> It looks like this is a peculiarity of the Java String.substring()
> operation, which retains the entire original char[] array although
> the String now only references the trailing part of it.
>
> Unfortunately the memory analyzer cannot tell us who performs the
> substring() operation that still retains the large char[] array.
> But it looks like it should be possible to optimize away my memory
> problem by replacing
>
> fullPath.substring(lastSlashIdx);
>
> by
>
> new String(fullPath.substring(lastSlashIdx);
>
> by explicitly performing a new String() operation at this point, the
> original substring which retains the char[] array will be released
> for garbage collection, and the new String will only retain the
> desired substring.
>
> Fellow committers, do you have an idea where that substring() operation
> could be? I suspect somewhere in UnifiedTree from the refresh operation,
> but I am not sure...
>
>
> Cheers,
> --
> Martin Oberhuber, Senior Member of Technical Staff, Wind River
> Target Management Project Lead, DSDP PMC Member
> http://www.eclipse.org/dsdp/tm
>
>
>
>> -----Original Message-----
>> From: platform-core-dev-bounces@xxxxxxxxxxx
>> [mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of
>> Oberhuber, Martin
>> Sent: Tuesday, September 29, 2009 2:12 PM
>> To: Eclipse Platform Core component developers list.
>> Subject: RE: [platform-core-dev] How much memory can a Workspace take?
>>
>> Hi Szymon, James -
>>
>> my hprof.zip is at
>> http://dl.getdropbox.com/u/1975868/java_pid11610.zip
>>
>> The ZIP is 68MB large (the 200MB had also included the entire
>> .metadata with some large CDT pdom's).
>>
>> As mentioned this was with Eclipse-3.5.0 on Linux-GTK RHEL4 (32-bit).
>> The problem happened directly after launching, during a workspace
>> refresh that was initiated because a previous shutdown was unclean.
>>
>> Here are some probably interesting bits from the .log file. It looks
>> like the machine was very busy at that time -- it took 6 minutes
>> from initial Launch till refreshing the workspace, and it took 19
>> minutes from starting the WS refresh till running into the
>> OutOfMemoryError:
>>
>> !SESSION 2009-09-28 12:19:25.891
>> -----------------------------------------------
>> eclipse.buildId=unknown
>> java.version=1.5.0_11
>> java.vendor=Sun Microsystems Inc.
>> BootLoader constants: OS=linux, ARCH=x86, WS=gtk, NL=en_US
>> Command-line arguments: -os linux -ws gtk -arch x86 -data
>> /folk/hxu2/WindRiver/workspace2
>>
>> !ENTRY org.eclipse.core.resources 2 10035 2009-09-28 12:25:34.028
>> !MESSAGE The workspace exited with unsaved changes in the previous
>> session; refreshing workspace to recover changes.
>>
>> !ENTRY org.eclipse.core.resources 4 1 2009-09-28 12:44:01.498
>> !MESSAGE Internal Error
>> !STACK 0
>> java.lang.OutOfMemoryError: Java heap space
>>
>> Cheers,
>> --
>> Martin Oberhuber, Senior Member of Technical Staff, Wind River
>> Target Management Project Lead, DSDP PMC Member
>> http://www.eclipse.org/dsdp/tm
>>
>>
>>
>> > -----Original Message-----
>> > From: platform-core-dev-bounces@xxxxxxxxxxx
>> > [mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of
>> > Szymon Brandys
>> > Sent: Tuesday, September 29, 2009 1:28 PM
>> > To: Eclipse Platform Core component developers list.
>> > Subject: Re: [platform-core-dev] How much memory can a
>> Workspace take?
>> >
>> > Hi Martin,
>> > Could I get the .hprof file?
>> > --
>> > Szymon
>> >
>> >
>> >
>> >
>> > "Oberhuber,
>> >
>> > Martin"
>> >
>> > <Martin.Oberhuber
>> > To
>> > @windriver.com> "Eclipse Platform Core
>> > component
>> > Sent by: developers list."
>> >
>> > platform-core-dev
>> > <platform-core-dev@xxxxxxxxxxx>
>> > -bounces@eclipse.
>> > cc
>> > org
>> >
>> >
>> > Subject
>> > [platform-core-dev]
>> > How much memory
>> > 2009-09-29 12:30 can a Workspace take?
>> >
>> >
>> >
>> >
>> >
>> > Please respond to
>> >
>> > "Eclipse Platform
>> >
>> > Core component
>> >
>> > developers list."
>> >
>> > <platform-core-de
>> >
>> > v@xxxxxxxxxxx>
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > Hi fellow committers,
>> >
>> >
>> > I got a report about an OutOfMemoryError generated during a
>> Workspace
>> > Refresh operation. When looking at the .hprof file with the
>> > MemoryAnalyzer,
>> > it turned out that pretty much all the memory was taken by
>> > core/Workspace
>> > only:
>> > 100 MB (53%) in org.eclipse.core.internal.resources.Workspace
>> > (occupied by ElementTree).
>> > 52 MB (27%) in
>> > org.eclipse.core.internal.dtree.DeltaDataTree (this is
>> > a different additional DeltaDataTree)
>> > 37 MB all the rest.
>> >
>> >
>> > The workspace was arguably very large, but still I am surprised
>> > 1. that the numbers are that large
>> > 2. that in addition to the Workspace, there is such a
>> > large additional
>> > DeltaDataTree,
>> > 3. that I see an OutOfMemoryError reported with 189MB
>> > total where I used
>> > -vmargs -Xmx320m ... what happened to the extra 131MB ?
>> >
>> >
>> > Do my numbers sound reasonable, or could there be a bug
>> that leads the
>> > Workspace to take an excessive amount of memory? How much
>> memory do we
>> > expect a workspace to take for, say 50000 files? How much
>> *additional*
>> > memory may a workspace delta take that is generated during a
>> > Refresh? Would
>> > any additional investigation using the Coretools make sense
>> > at this point?
>> >
>> >
>> > I can make the .hprof file available on request (it's a 200MB
>> > ZIP file).
>> >
>> >
>> > Many thanks for any pointers!
>> >
>> >
>> > Cheers,
>> > --
>> > Martin Oberhuber, Senior Member of Technical Staff, Wind River
>> > Target Management Project Lead, DSDP PMC Member
>> > http://www.eclipse.org/dsdp/tm
>> >
>> > _______________________________________________
>> > platform-core-dev mailing list
>> > platform-core-dev@xxxxxxxxxxx
>> > https://dev.eclipse.org/mailman/listinfo/platform-core-dev
>> >
>> >
>> > _______________________________________________
>> > platform-core-dev mailing list
>> > platform-core-dev@xxxxxxxxxxx
>> > https://dev.eclipse.org/mailman/listinfo/platform-core-dev
>> >
>> _______________________________________________
>> platform-core-dev mailing list
>> platform-core-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/platform-core-dev
>>
>
> _______________________________________________
> platform-core-dev mailing list
> platform-core-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/platform-core-dev
>
>
