Community
Participate
Working Groups
In my project I'm getting the message "Corrupted database" every time I run the indexer. I'm using the 2019/03 release. I actually spent quite some time debugging this problem. Its triggered by use after free inside PDOMNamedNode. I added a bit of code to track calls to Database free and find use-after-free scenarios. The release occurs here: https://git.eclipse.org/c/cdt/org.eclipse.cdt.git/tree/core/org.eclipse.cdt.core/parser/org/eclipse/cdt/internal/core/pdom/dom/cpp/PDOMCPPFunction.java#n171 oldParams.delete(linkage); The use after free occurs here: java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Thread.java:1333) at org.eclipse.cdt.internal.core.pdom.db.Database.checkUseAfterFree(Database.java:513) at org.eclipse.cdt.internal.core.pdom.db.Database.getString(Database.java:632) at org.eclipse.cdt.internal.core.pdom.dom.PDOMNamedNode.getDBName(PDOMNamedNode.java:75) at org.eclipse.cdt.internal.core.pdom.dom.PDOMNamedNode.getDBName(PDOMNamedNode.java:70) at org.eclipse.cdt.internal.core.pdom.dom.PDOMNamedNode.getNameCharArray(PDOMNamedNode.java:82) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPParameter.getNameCharArray(PDOMCPPParameter.java:179) at org.eclipse.cdt.internal.core.dom.parser.cpp.CPPSpecialization.getNameCharArray(CPPSpecialization.java:93) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPSpecialization.<init>(PDOMCPPSpecialization.java:50) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPParameterSpecialization.<init>(PDOMCPPParameterSpecialization.java:54) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPFunctionSpecialization.setParameters(PDOMCPPFunctionSpecialization.java:140) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPFunctionSpecialization.initData(PDOMCPPFunctionSpecialization.java:99) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage$ConfigureFunctionSpecialization.run(PDOMCPPLinkage.java:367) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.handlePostProcesses(PDOMCPPLinkage.java:1301) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.addBinding(PDOMCPPLinkage.java:633) at org.eclipse.cdt.internal.core.pdom.dom.PDOMFile.createPDOMName(PDOMFile.java:526) at org.eclipse.cdt.internal.core.pdom.dom.PDOMFile.addNames(PDOMFile.java:492) at org.eclipse.cdt.internal.core.pdom.WritablePDOM.addFileContent(WritablePDOM.java:158) at org.eclipse.cdt.internal.core.index.WritableCIndex.setFileContent(WritableCIndex.java:89) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.storeFileInIndex(PDOMWriter.java:679) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.storeSymbolsInIndex(PDOMWriter.java:329) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.addSymbols(PDOMWriter.java:287) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.writeToIndex(AbstractIndexerTask.java:1307) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseFile(AbstractIndexerTask.java:1107) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseLinkage(AbstractIndexerTask.java:910) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.runTask(AbstractIndexerTask.java:572) at org.eclipse.cdt.internal.core.pdom.indexer.PDOMIndexerTask.run(PDOMIndexerTask.java:164) at org.eclipse.cdt.internal.core.pdom.indexer.PDOMRebuildTask.run(PDOMRebuildTask.java:94) at org.eclipse.cdt.internal.core.pdom.PDOMIndexerJob.run(PDOMIndexerJob.java:160) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63) Sadly I have no way of reproducing this issue with a minimal example.
Thanks for filing. Do you have a stack trace for the delete as well?
Stacktrace for delete: java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Thread.java:1333) at org.eclipse.cdt.internal.core.pdom.db.Database.setFreed(Database.java:506) at org.eclipse.cdt.internal.core.pdom.dom.PDOMNamedNode.delete(PDOMNamedNode.java:115) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPParameter.flatDelete(PDOMCPPParameter.java:245) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPParameter.delete(PDOMCPPParameter.java:236) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPFunction.update(PDOMCPPFunction.java:174) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPMethod.update(PDOMCPPMethod.java:70) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.addImplicitMethods(PDOMCPPLinkage.java:1031) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.addBinding(PDOMCPPLinkage.java:637) at org.eclipse.cdt.internal.core.pdom.dom.PDOMFile.createPDOMName(PDOMFile.java:526) at org.eclipse.cdt.internal.core.pdom.dom.PDOMFile.addNames(PDOMFile.java:492) at org.eclipse.cdt.internal.core.pdom.WritablePDOM.addFileContent(WritablePDOM.java:158) at org.eclipse.cdt.internal.core.index.WritableCIndex.setFileContent(WritableCIndex.java:89) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.storeFileInIndex(PDOMWriter.java:679) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.storeSymbolsInIndex(PDOMWriter.java:329) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.addSymbols(PDOMWriter.java:287) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.writeToIndex(AbstractIndexerTask.java:1307) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseFile(AbstractIndexerTask.java:1107) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseLinkage(AbstractIndexerTask.java:910) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.runTask(AbstractIndexerTask.java:572) at org.eclipse.cdt.internal.core.pdom.indexer.PDOMIndexerTask.run(PDOMIndexerTask.java:164) at org.eclipse.cdt.internal.core.pdom.indexer.PDOMRebuildTask.run(PDOMRebuildTask.java:94) at org.eclipse.cdt.internal.core.pdom.PDOMIndexerJob.run(PDOMIndexerJob.java:160) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Based on the stacks, it looks like while updating a function in the index, we are replacing old parameter bindings with new ones, and deleting the old ones, but there exist specializations of the function which have parameter specializations still referring to the old parameters. Unfortunately, it's not clear from the available information whether this is a scenario that needs to be handled in general (which doesn't seem trivial to do), or if we could avoid getting into this situation in the first place.
I spent the last few days debugging this problem and now have a fix. I would love to make the commit myself, but I can't fully proof that my change is correct in all scenarios. I will post a minimal patch later.
When parsing include directives that turn out to be pragma-once'd headers CPreprocessor::executeInclude might encounter a SKIP_FILE kind of InternalFileContent instance. In that case it will generate an ASTInclusionStatement that only provides an empty SignificantMacros. That header version with no macros will then be written to the database alongside the correct version. Subsequent include lookups will then fail and nested headers might possibly get reparsed and indexed. There might be more side effects to this bug. Does that all make sense? :D Here is the patch: https://gist.github.com/nidefawl/3270f31ebb56cc5d261462c7ddbe459e
Thank you for investigating! (In reply to Michael Hept from comment #5) > Does that all make sense? :D I don't have a good understanding of this part of the code, so it's hard for me to offer an informed opinion, but based on a surface-level reading, your explanation sounds plausible, yes. > Here is the patch: > https://gist.github.com/nidefawl/3270f31ebb56cc5d261462c7ddbe459e Could you upload the patch to gerrit? Instructions can be found here: https://wiki.eclipse.org/CDT/git#Using_Gerrit_for_CDT
New Gerrit change created: https://git.eclipse.org/r/142818
https://git.eclipse.org/r/c/cdt/org.eclipse.cdt/+/172503/9#message-da07129fe3cd9724e6c6829480fe40e4d7b8bf2f which is on a fix for Bug 568957 says: after looking at Bug 547491, I have found out that the given fix [to Bug 568957] there also fixes my Reporter Bug and also makes my JUnit test pass. The conflicting fix seems to me quite similar (but less clear). So I am cross referencing these bugs.
