Community
Participate
Working Groups
Created attachment 262199 [details] stack trace (i1) Hi all, I'm experiencing some issues with the indexer in a rather large project (chromium). It manifests itself through (i) frequent "Waiting for exclusive index access" messages of the indexer in the progress view (similar to bug 494084), and through (ii) occasional UI freezes in the editor. Both situations resolve themselves after waiting for approx. 20-60s. I took a few stack traces of when a situation like this occurred (two for i, one for ii; will attach them all). They show multiple threads waiting on a lock inside org.eclipse.cdt.internal.core.pdom.db.Database.getChunk(Database.java:270). In each stack trace, one of the threads seems to have acquired the lock and is in RUNNABLE state, but doesn't seem to have progressed after the lock was acquired, e.g.: "org.eclipse.cdt.internal.ui.text.CReconciler" #2915 daemon prio=1 os_prio=0 tid=0x00007f2807a53800 nid=0x1a578 runnable [0x00007f24f02bd000] java.lang.Thread.State: RUNNABLE at (C/C++) 0x00007f280bc24232 (Unknown Source) at (C/C++) 0x00007f280a869ee7 (Unknown Source) at (C/C++) 0x00007f280ad67bed (Unknown Source) at (C/C++) 0x00007f280a7240d2 (Unknown Source) at (C/C++) 0x00007f280a8d0a02 (Unknown Source) at org.eclipse.cdt.internal.core.pdom.db.Database.getChunk(Database.java:270) - locked <0x0000000606a19360> (a org.eclipse.cdt.internal.core.pdom.db.ChunkCache) at org.eclipse.cdt.internal.core.pdom.db.Database.getRecPtr(Database.java:486) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPSpecialization.getSpecializedBinding(PDOMCPPSpecialization.java:74) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPClassSpecialization.getSpecializedBinding(PDOMCPPClassSpecialization.java:112) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPClassInstance.getTemplateDefinition(PDOMCPPClassInstance.java:73) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMInstanceCache.populate(PDOMInstanceCache.java:89) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMInstanceCache.getCache(PDOMInstanceCache.java:41) Can someone shed light on whether this could be the underlying issue (i.e. that the RUNNABLE thread doesn't progress), and if so, what the reason for this might be? My setup/configuration: I'm using CDT version 9.0.0.201604211004 in Eclipse 4.5.2. Indexer cache is configured to use up to 896MB. Thanks! Eric
Created attachment 262200 [details] stack trace (i2)
Created attachment 262201 [details] stack trace (ii)
(In reply to Eric Seckler from comment #0) > I took a few stack traces of when a situation like this occurred (two for i, > one for ii; will attach them all). They show multiple threads waiting on a > lock inside > org.eclipse.cdt.internal.core.pdom.db.Database.getChunk(Database.java:270). > In each stack trace, one of the threads seems to have acquired the lock and > is in RUNNABLE state, but doesn't seem to have progressed after the lock was > acquired This happens in (i1) and (ii), but not in (i2). In (i2), the runnable thread has progressed to Database.java:284, i.e. it's in the code block protected by the lock. As getChunk() is a pretty frequent operation, without a larger sample size I'm inclined to think that you just happened to catch the thread right after it acquires the lock in two of your three samples. (In reply to Eric Seckler from comment #0) > It manifests itself through (i) frequent "Waiting for exclusive > index access" messages of the indexer in the progress view (similar to bug > 494084) Does this have any user impact beside the messages themselves? > and through (ii) occasional UI freezes in the editor The hang in trace (ii) is happening because the formatter, which runs on the UI thread, is waiting on the index database cache lock. A possible solution might be to move the formatter off the UI thread.
(In reply to Nathan Ridge from comment #3) > This happens in (i1) and (ii), but not in (i2). In (i2), the runnable thread > has progressed to Database.java:284, i.e. it's in the code block protected > by the lock. > > As getChunk() is a pretty frequent operation, without a larger sample size > I'm inclined to think that you just happened to catch the thread right after > it acquires the lock in two of your three samples. You're right, if getChunk() is called very frequently during one of these situations, that's a realistic explanation. > Does this have any user impact beside the messages themselves? Not directly, but if a UI action that requires index access is started at the same time, it may be blocked. > The hang in trace (ii) is happening because the formatter, which runs on the > UI thread, is waiting on the index database cache lock. > > A possible solution might be to move the formatter off the UI thread. Okay, makes sense. Other UI actions that may hang are: hover-tooltips on source code entities, auto-complete, mark occurrences. I played around with larger index sizes (now using 4GB max), which helped a little bit - typically these hangs resolve themselves after about 10-20s now. Is there anything I can do to help figure out what exactly takes this long? Thanks!
(In reply to Eric Seckler from comment #4) > I played around with larger index sizes (now using 4GB max), which helped a > little bit - typically these hangs resolve themselves after about 10-20s > now. Is there anything I can do to help figure out what exactly takes this > long? Thanks! I think it's just a combination of a large codebase (==> large index) with many different threads providing different features (indexer, semantic highlighting, code analysis, hovers, mark occurrences, etc.) running in the background, and others initiated by user action (code completion, formatter, open declaration, etc.) all competing for access to the index. An obvious mitigation is to disable some of these features if you don't use them. Other approaches are to exclude folders you don't use from the project, or create several smaller projects each for a subfolder of the codebase rather than having one for the whole. A more proper solution would be to design the index cache in a way that all these threads don't need to wait on the same lock. I don't have much expertise in concurrent programming, so I don't know exactly what that would look like (a lock-free data structure perhaps?), but I expect the CDT community would welcome ideas, and especially, contributions.
