[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[cdt-dev] Memory consumed by indexer in big projects
- From: Andrew Gvozdev <angvoz.dev@xxxxxxxxx>
- Date: Fri, 20 Jan 2012 10:45:57 -0500
- Delivered-to: email@example.com
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=VwOxd5ueV4v+CXJshwICbibp5GblzO2OkzKwpv0J0+0=; b=gSginsp8PlHDTkJ8tMPnTQUEV3GLhpVqOlkA/ODNhfiDMfI34Org+YvLbXhy+GZHx8 pNy8WHVTxu4zbx26+vse2yX9hh88E7ykau+fRXP+3cI8cM481PrHfWaMEVtU31bE1hVd HJrGUzEb2d4xvefTGpR0f4owwlwTY72NUDaRk=
We are testing the new scanner discovery with mozilla folks on firefox project. In general, it seems to be pretty good improvement but we run into huge consumption of heap memory by indexer objects.
Eclipse is given 2Gb of heap memory in eclipse.ini but still Out Of Memory exception is thrown. It seems that most of the memory is used by indexer objects (see summary below and attached report).
I want to be clear, this is the scanner discovery branch sd90 and the project is big and settings for the project are very elaborate so pretty much every file has slightly different settings and likely could use different variants of headers with different set of macros.
So the question is if it is conceivable to indexer to use up to that much memory, or we are looking at a memory leak? (Jonathan also tried to give it 8Gb of heap with similar result, I did not look at the dump tho.)
Scanner discovery itself takes constant 0.5M for this project and does not grow, but maybe I am using indexer incorrectly and induce the leak somehow? I created a small class org.eclipse.cdt.internal.core.pdom.LanguageSettingsChangeListener to listen to changes in language settings and trigger reindexing (on branch sd90).
One instance of"org.eclipse.cdt.internal.core.dom.parser.cpp.CPPASTTranslationUnit" loaded by"org.eclipse.cdt.core" occupies 856,848,368 (31.40%) bytes. The memory is accumulated in one instance of "org.eclipse.cdt.core.dom.ast.IASTDeclaration"loaded by "org.eclipse.cdt.core".
One instance of "org.eclipse.cdt.internal.core.parser.scanner.LocationMap" loaded by "org.eclipse.cdt.core" occupies 812,268,408 (29.77%) bytes. The memory is accumulated in one instance of "java.lang.Object" loaded by "<system class loader>".
12,818 instances of "org.eclipse.cdt.internal.core.parser.scanner.CharArray", loaded by "org.eclipse.cdt.core" occupy 381,328,568 (13.97%) bytes. These instances are referenced from one instance of "java.lang.Object", loaded by "<system class loader>"
The problem suspects 1, 2 and 3 may be related, because the reference chains to them have a common beginning.
Description: Zip archive