Have other people been noticed that more memory is being used during indexing in recent builds? I would expect some
extra memory and runtime usage since the parser/indexer are able to recognize more things. However, the tests I've done
show only a small increase in what is found to put into the index at a cost of much more runtime and memory.
We initially noticed the problem on internal projects, but I've been able to reproduce similar results using boost as
the test project. In order to get repeatable results I've been using a project that I created from boost release tag
1.55.0:
$ svn co http://svn.boost.org/svn/boost/tags/release/Boost_1_55_0 boost-1.55.0
I imported this into Eclipse using "Import - Existing code as Makefile Project". I choose only the boost folder (within
the boost-1.55.0 checkout folder). Trying to index the complete source runs into significant problems when it gets to:
boost-1.55.0/libs/preprocessor/doc/example/delay.c
Using CDT 8.1.0 as a baseline I was able to index the complete source using a peak of just over 1Gb of memory:
CDT 8.1.0:
C/C++ Indexer: Project 'boost-1.55.0' (12 sources, 13,156 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=true, unusedHeaders=useCPP, skipReferences=false,
skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
Database: 461152256 bytes
Timings: 1377747 total, 701404 parser, 157978 resolution, 215679 index update.
Errors: 152 internal, 189 include, 154 scanner, 4261 syntax errors.
Names: 1641458 declarations, 9460002 references, 912516(7.60%) unresolved.
Cache[64MB]: 855744592 hits, 19246137(2.20%) misses.
Indexer: completed PDOMRebuildTask[1377840ms]
Using the current master (at commit 23c2fd) I ran out of memory and crashed during indexing. I raised my max memory
settings to 2Gb and was able to index this project using a peak of just under 1.5Gb:
CDT master (commit: 23c2fd):
C/C++ Indexer: Project 'boost-1.55.0' (13 sources, 12,796 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=true, unusedHeaders=useCPP, skipReferences=false,
skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
Database: 496119808 bytes
Timings: 3200221 total, 906062 parser, 639984 resolution, 220217 index update.
Errors: 22 internal, 184 include, 195 scanner, 4625 syntax errors.
Names: 1593995 declarations, 9480500 references, 882830(7.38%) unresolved.
Cache[64MB]: 1832837748 hits, 201870349(9.92%) misses.
Indexer: completed PDOMUpdateTask[3200948ms]
A few interesting things in this comparison:
- 8.1.0 indexed more headers (but fewer sources)
- master has fewer unresolved symbols (although it found fewer declarations, so I'm not sure if that is an improvement)
- master finds more references
- 8.1.0 was more than *twice* as fast as master
I've attached the jconsole heap usage graphs for these two cases.
I don't have peak memory use for 8.2.0 (since I wasn't using jconsole yet when I did that test). This release found
more headers, but is otherwise sort of between the previous 8.1.0 and master:
CDT 8.2.0:
C/C++ Indexer: Project 'boost-1.55.0' (13 sources, 13,393 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=true, unusedHeaders=useCPP, skipReferences=false,
skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
Database: 477392896 bytes
Timings: 2358690 total, 1244299 parser, 263195 resolution, 310024 index update.
Errors: 12 internal, 184 include, 168 scanner, 4302 syntax errors.
Names: 1666695 declarations, 9633150 references, 901119(7.39%) unresolved.
Cache[64MB]: 724158478 hits, 19846710(2.67%) misses.
Indexer: completed PDOMRebuildTask[2359837ms]
-Andrew
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
