[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-dev] Memory consumed by indexer in big projects

Actually to be clear, those are AST objects, not indexer. In the past, we've made sure they don't stay around long (thus the index :)). It would be interesting to see what's keeping them from being gc'ed after the parse is completed.

Do they see the same behavior off of master (i.e., it's not your changes causing this is it?)

On Fri, Jan 20, 2012 at 10:45 AM, Andrew Gvozdev <angvoz.dev@xxxxxxxxx> wrote:
Hi,
We are testing the new scanner discovery with mozilla folks on firefox project. In general, it seems to be pretty good improvement but we run into huge consumption of heap memory by indexer objects.
Eclipse is given 2Gb of heap memory in eclipse.ini but still Out Of Memory exception is thrown. It seems that most of the memory is used by indexer objects (see summary below and attached report).

I want to be clear, this is the scanner discovery branch sd90 and the project is big and settings for the project are very elaborate so pretty much every file has slightly different settings and likely could use different variants of headers with different set of macros.
So the question is if it is conceivable to indexer to use up to that much memory, or we are looking at a memory leak? (Jonathan also tried to give it 8Gb of heap with similar result, I did not look at the dump tho.)

Scanner discovery itself takes constant 0.5M for this project and does not grow, but maybe I am using indexer incorrectly and induce the leak somehow? I created a small class org.eclipse.cdt.internal.core.pdom.LanguageSettingsChangeListener to listen to changes in language settings and trigger reindexing (on branch sd90).

Thanks,
Andrew

  Problem Suspect 1

One instance of"org.eclipse.cdt.internal.core.dom.parser.cpp.CPPASTTranslationUnit" loaded by"org.eclipse.cdt.core" occupies 856,848,368 (31.40%) bytes. The memory is accumulated in one instance of "org.eclipse.cdt.core.dom.ast.IASTDeclaration[]"loaded by "org.eclipse.cdt.core".

Keywords
org.eclipse.cdt.core
org.eclipse.cdt.internal.core.dom.parser.cpp.CPPASTTranslationUnit
org.eclipse.cdt.core.dom.ast.IASTDeclaration[]

  Problem Suspect 2

One instance of "org.eclipse.cdt.internal.core.parser.scanner.LocationMap" loaded by "org.eclipse.cdt.core" occupies 812,268,408 (29.77%) bytes. The memory is accumulated in one instance of "java.lang.Object[]" loaded by "<system class loader>".

Keywords
org.eclipse.cdt.core
java.lang.Object[]
org.eclipse.cdt.internal.core.parser.scanner.LocationMap

  Problem Suspect 3

12,818 instances of "org.eclipse.cdt.internal.core.parser.scanner.CharArray", loaded by "org.eclipse.cdt.core" occupy 381,328,568 (13.97%) bytes. These instances are referenced from one instance of "java.lang.Object[]", loaded by "<system class loader>"

Keywords
org.eclipse.cdt.core
org.eclipse.cdt.internal.core.parser.scanner.CharArray
java.lang.Object[]

 Hint 1

The problem suspects 1, 2 and 3 may be related, because the reference chains to them have a common beginning.


Thanks,
Andrew


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev