Community
Participate
Working Groups
I have been trying the OpenType dialog with a small to midsize project (1300 cpp files). (Using CDT Head - from this morning, and Eclipse 3.0M9) Here is some background on the project: 1. The project has been previously indexed (Image grew from 48M to 116M, took about 9 minutes to index), and 2. Code assist and search work reasonably well (although sometimes quite slow). 3. Closing eclipse and restarting it causes no re-indexing at all. (image is now 48M) 4. I have the "cache types in background" flag off 5. The first time I use the OpenType dialog (after restarting eclipse), it takes a minute or so to "Updating Type Cache", 6. Each subsequent invocation is instant. This looks great! Now here is the problem: The memory requirements on step 5 have jumped by 160M , I think that this will not scale well.... There was no parsing being done (I have a breakpoint on Parser.translationUnit) Has anybody else measured something similar?
I've tried my own test case for this problem. Starting with a project with ~8000 files: 1. The Indexer took ~25 minutes. Memory usage peaked at about 272MB after 47% done, at which point the system slowed to a crawl and became unusable (ie task manager shows "not responding"). However I let it continue running and after a few minutes it became responsive again. Memory usage levelled off to around 270MB for the remaining 10 minutes. 2. The .index file on disk was ~991KB, but the .log file was 82MB, 99% of which was this error: MESSAGE Parser: Unexpected exception in simpleDeclaration:createSimpleTypeSpecifier:org.eclipse.cdt.core.parser.ast.ASTSemanticException::null. 3. Quit & restart eclipse. Memory usage now ~48M. Indexer not running. 4. Click Open Type. Progress dialog "Updating Type Cache" appears for ~2 seconds and then the type selection dialog is displayed. Memory usage now ~54MB. 5. Subsequent invocations are instantaneous. Bogdan, can you investigate? Seems like the indexer is doing something heavy. This same project had no trouble being indexed the last time I tried (maybe 2 weeks ago).
Hmmm...it sounds like we might be getting stuck on a couple of files and having the indexer timeout kick the parser along. Adding JohnC.
OK - I looked into this and I can reproduce what Dave is seeing: i) background type caching off ii) import a project/let it index iii) open types -> at this point memory shoots up to reach ~320MB (or throw an OutOfMemory error if the VM doesn't have enough memory assigned to it) Looking at the code, if I comment out the last two lines of the insert function in TypeCache - where we add the new type to the collection and set the cache in the new type - there is no memory growth (ie. the indexer isn't significantly contributing to the memory growth). Let me know if you need the project we've been seeing this behaviour on... (Also, maybe I can help out with the project that you saw the indexer getting stuck on - it's possible that there's a file in there that is hurting the parser...)
I get an out of memory exception while building the type cache for the for a project containing GCC-3.3 on solaris. The IEntryResult [] in IndexerTypesJob.updateTypes has 19874 elements, I made it to 6034 before getting the out of memory exception. Indexing was completed and not running when we started building the type cache. The size of the index on disk is 2.7 M
Chris is out on Vacation 'til Monday. If this is urgent, let me know, I'll take a second look. thanks.
Bogdan has set the default of the "calculate in background" flag to false by default (now that the user cannot see the flag). This will get us (and any potential user) into a working state. But, if anybody uses the navigate to type, on a project similar to andrews or mine, they will run out of memory (Quickly). I don't see a need to fix it today or tomorrow, but I think that it should keep its P1 priority and be fixed before release.
> Bogdan has set the default of the "calculate in background" flag to false by > default (now that the user cannot see the flag). hmm .. It should have been set to false by default since this was work in progress. Allright thanks for the heads up.
We (Doug, myself, Dave and Andrew) are working at making the parse as quick as possible. If this becomes the case (if not for 2.0, for sure for 2.0.1) then Open-Type/Browser need not maintain a global table, perhaps just using a LRU cache for recent navigations. The fear from my standpoint is that once we start encountering OutOfMemoryErrors from enabling the browsing perspective, there is little the user can do about it as the cache gets rebuilt @ starting up anew.
> Let me know if you need the project we've been seeing this behaviour Can you attach the test case please. Or if it's too big email. thanks.
I have sent my project to Alain, Chris, Bogdan, and Andrew via private e-mail.
I think we can consider this one fixed.
