Community
Participate
Working Groups
Summary: In a simple program "could not be resolved" semantic errors occur for a type that should be known (and it's fields). Other types included via the same parent are resolved successfully. Both the range and buffer types are included via a parent which includes their header files: #include <CL/sycl.hpp> The contents of sycl.hpp include: #include <CL/sycl/range.hpp> #include <CL/sycl/buffer.hpp> range is resolved successfully. buffer is not resolved. Please note that the steps to reproduce seem complicated, but I do not know how to create a simple reproducer. That being said, it doesn't take very long to complete the steps. I'm willing to troubleshoot this if I can get some pointers about how to setup the environment and where to set breakpoints. Steps to reproduce: 1. Install Eclipse for C/C++ Developers 2020-09 2. Install Intel oneAPI beta09: https://dynamicinstaller.intel.com/oneapi/toolkits/base-kit/linux/ 3. Follow the Get Started with the Intel® oneAPI Base Toolkit for Linux* (Beta) https://software.intel.com/content/www/us/en/develop/documentation/get-started-with-intel-oneapi-base-linux/top.html 4. Follow Build and Run a Sample Project Using Eclipse* https://software.intel.com/content/www/us/en/develop/documentation/get-started-with-intel-oneapi-base-linux/top/run-a-sample-project-using-an-ide.html 5. Open vector-add-buffers.cpp in the editor and observe the semantic errors
Hi Tony. I started replying to you but now I think it's better to write a small guide to help investigate this kind of issue in general. It's a work in progress here: https://wiki.eclipse.org/CDT/User/HowToTroubleshootCDTIndexing
Thanks Marc-André! I just read over what you have written so far and it is already very helpful.
Created attachment 284395 [details] parser log 1 I have some questions and comments after working through the "Diagnosing the index" section of https://wiki.eclipse.org/CDT/User/HowToTroubleshootCDTIndexing. 1. Here is the output from a manual rebuild of the index: Indexed 'Vector_Add' (2 sources, 421 headers) in 23.4 sec: 57,637 declarations; 113,995 references; 0 unresolved inclusions; 47 syntax errors; 99 unresolved names (0.058%) Disabling the indexer preference that were suggested did not change these counts. 2. When I look through the Include Browser, I see all of the files I am looking for. 3. "or enabling the equivalent in the Tracing tab you you are setup for CDT development". Can you elaborate on this a bit in the wiki page? I thought I was setup for CDT development (I am running CDT from sources, launching from Eclipse For Committers) but I cannot find a Tracing tab. 4. "inspect the headers to see if the unresolved symbols of interest are not in inactive preprocessor regions". Please elaborate on this a bit. Does the editor assist me in understanding which preprocessor regions are inactive? 5. As suggested, I generated a parser log file and attached it. Some of the relevant files listed in the Include Browser are listed in the "Unresolved includes (from headers in index)" section of the parser log as "not indexed", for example, "file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/buffer.hpp is not indexed". Why is that file not indexed? 6. The unresolved names section of the parser log is exactly as expected. There are 6 line items that match the 6 errors shown in the Problems view. In the editor, if I right-click on the 'buffer' symbol and pick 'Open Declaration', the buffer.hpp file is opened, so some kind of association between the symbol and the header file exists. Is that association part of a data structure that the indexer does not use? With regard to the Important Limitations section, perhaps this issue is due to language support not being complete? I'll attach a copy of buffer.hpp in case you can give feedback on this.
Created attachment 284396 [details] buffer.hpp
(In reply to Tony Homer from comment #3) > Created attachment 284395 [details] > parser log 1 > > I have some questions and comments after working through the "Diagnosing the > index" section of > https://wiki.eclipse.org/CDT/User/HowToTroubleshootCDTIndexing. > > 1. Here is the output from a manual rebuild of the index: > Indexed 'Vector_Add' (2 sources, 421 headers) in 23.4 sec: 57,637 > declarations; 113,995 references; 0 unresolved inclusions; 47 syntax errors; > 99 unresolved names (0.058%) Seems like fairly good numbers to start with, syntax errors might hide more unresolved symbols though (maybe not important for this bug). > Disabling the indexer preference that were suggested did not change these > counts. It's more useful in case you have a project that has very distinct configurations. For example if you have source files that are only compiled on Windows but you are developing on Linux, it will trigger many unresolved includes and names. It sounds like your use case is not in this situation. > 3. "or enabling the equivalent in the Tracing tab you you are setup for CDT > development". Can you elaborate on this a bit in the wiki page? I thought I > was setup for CDT development (I am running CDT from sources, launching from > Eclipse For Committers) but I cannot find a Tracing tab. Sorry I ran out of time and it was very late :) I added more information in this section. Look for the tab in your launch configuration of the Eclipse Application. > 4. "inspect the headers to see if the unresolved symbols of interest are not > in inactive preprocessor regions". > Please elaborate on this a bit. Does the editor assist me in understanding > which preprocessor regions are inactive? Yes. In the editor, if a preprocessor region is inactive in an #ifdef for example, it will have a grey background. I added a screenshot on the page. If an unresolved symbol is in a grey region, then this needs to be investigated. > 5. As suggested, I generated a parser log file and attached it. Some of the > relevant files listed in the Include Browser are listed in the "Unresolved > includes (from headers in index)" section of the parser log as "not > indexed", for example, > "file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/ > buffer.hpp is not indexed". Why is that file not indexed? Good question. The include paths (-I) look OK based on the parser log, unless I'm missing something obvious. The method CPreprocessor.executeInclude is where the CDT preprocessor acts on #include so you could set a breakpoint there to make sure the header gets included or find out why it's not. Is seems odd that they would not get indexed, maybe we are misinterpreting the log. > 6. The unresolved names section of the parser log is exactly as expected. > There are 6 line items that match the 6 errors shown in the Problems view. > > In the editor, if I right-click on the 'buffer' symbol and pick 'Open > Declaration', the buffer.hpp file is opened, so some kind of association > between the symbol and the header file exists. Is that association part of > a data structure that the indexer does not use? So it looks like the indexer knows about the symbol but there is a problem with it, so it probably generates a ProblemBinding (CDT class) with type IProblemBinding.SEMANTIC_INVALID_TYPE which outputs in the log "Invalid type encountered in..." which should get reported in the editor hover (and Problem view) as "Type 'buffer' could not be resolved", is that right? Then all attempts to use members will result in a "Symbol 'foo' could not be resolved". If that's the case, I feel like the message "type could not be resolved" is misleading and we should consider changing it. > With regard to the Important Limitations section, perhaps this issue is due > to language support not being complete? I'll attach a copy of buffer.hpp in > case you can give feedback on this. The part with #ifdef __cpp_deduction_guides seems like could be the culprit, it's worth a look I think. Support for deduction guides was added in CDT 10.0 but was just reverted because it caused major regressions (revert not released yet but very soon aka 10.0.1). See bug 567261. You could either try to compile your code in C++14 mode if possible (which will disable the definition of __cpp_deduction_guides) or disable this preprocessor region with the built-in CDT macro __CDT_PARSER__ (of just comment it all out) as a temporary work around to investigate and confirm this is the problem. BTW, I tried to do the setup but the Intel download seems incredibly slow and was not able to do it in reasonable time for this reply.
(In reply to Marc-André Laperle from comment #5) Thanks for the continued updates to the wiki page. The addition of the screenshots is really helpful. >> I cannot find a Tracing tab. > > I added more information in this section. Look for the tab in your launch configuration of the Eclipse Application. This is very helpful, I was not aware of this feature! I didn't use it yet but I will try it ASAP. >> Does the editor assist me in understanding which preprocessor regions are inactive? > Yes. In the editor, if a preprocessor region is inactive in an #ifdef for > example, it will have a grey background. Thanks for explaining and adding the screenshot. It's a helpful tip but it is not the issue in my case. >> "file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/buffer.hpp is not indexed". Why is that file not indexed? > The method CPreprocessor.executeInclude is where the CDT preprocessor acts on #include... Is seems odd that they would not get indexed, maybe we are misinterpreting the log. I spent a lot of time with breakpoints in and around CPreprocessor.executeInclude. I will describe my preliminary findings from that and from exploring log generation in another comment. > So it looks like the indexer knows about the symbol but there is a problem > with it, so it probably generates a ProblemBinding (CDT class) with type > IProblemBinding.SEMANTIC_INVALID_TYPE which outputs in the log "Invalid type > encountered in..." which should get reported in the editor hover (and > Problem view) as "Type 'buffer' could not be resolved", is that right? Then > all attempts to use members will result in a "Symbol 'foo' could not be > resolved". If that's the case, I feel like the message "type could not be > resolved" is misleading and we should consider changing it. This is all correct. I hope I will be able to make a small reproducible case to illustrate this. > The part with #ifdef __cpp_deduction_guides seems like could be the culprit, it's worth a look I think. Thanks for the suggestion. I commented this section out but it did not make a difference. > BTW, I tried to do the setup but the Intel download seems incredibly slow > and was not able to do it in reasonable time for this reply. Thanks for trying. I will try to make a small example that depends on public repos only.
I'd like to share some progress that I made along 3 lines. A. Investigating the parser log output. B. Debugging around CPreprocessor.executeInclude C. A seemingly unrelated error that is likely related ********************************************************************* A. ********************************************************************* I set a breakpoint in CreateParserLogAction.outputUnresolvedIncludes. There are two functions with this name. If I understand correctly, the first one loops over the top-level includes. The second one recurses through the tree of includes from each include at the top-level. So if a.h includes b.h which includes c.h, a.h is inspected in the first function, which calls the second function for each include in a.h, such as b.h. The second function inspects b.h and finds c.h, so it calls itself with c.h, etc. I am about 7 levels deep and I found an example from the "Unresolved includes (from headers in index):" section of the log, but I don't understand it. The tree, starting with the source file which I am generating the parse log for is: 1. /home/tony/development/oomph-cdt-master/runtime-cdt-development/Vector_Add/src/vector-add-buffers.cpp 2. /opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl.hpp 3. <CL/sycl/accessor.hpp> in file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl.hpp resolved to file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/accessor.hpp 4. <CL/sycl/atomic.hpp> in file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/accessor.hpp resolved to file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/atomic.hpp 5. <CL/sycl/access/access.hpp> in file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/atomic.hpp resolved to file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/access/access.hpp 6. <CL/sycl/detail/common.hpp> in file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/access/access.hpp resolved to file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/detail/common.hpp 7. <CL/sycl/detail/export.hpp> in file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/detail/common.hpp resolved to file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/detail/export.hpp Please refer to this line: https://git.eclipse.org/r/plugins/gitiles/cdt/org.eclipse.cdt/+/refs/heads/master/core/org.eclipse.cdt.ui/src/org/eclipse/cdt/internal/ui/actions/CreateParserLogAction.java#392 I reached here from the recursive call here: https://git.eclipse.org/r/plugins/gitiles/cdt/org.eclipse.cdt/+/refs/heads/master/core/org.eclipse.cdt.ui/src/org/eclipse/cdt/internal/ui/actions/CreateParserLogAction.java#400 Because next/ifile is null, writeUnresolvedTitle is called, resulting in this message in the log: " file:/opt/intel/oneapi/compiler/2021.1-beta09/linux/include/sycl/CL/sycl/detail/export.hpp is not indexed" This seems like it might be a bug. Maybe I am misunderstanding the code, but I think net/ifile is null because export.hpp does not include any files, so it is the leaf node of the recursive tree. This seems like a normal exit condition to the include inspection, but it is being treated as an error. I'm not sure, so if you can take a look that would be great. ********************************************************************* B. ********************************************************************* Please refer to the attachment "CoreException-ExitCondition" which is a set of screen captures of an interesting state I encountered which I am still exploring. It might be behaving as expected but from my learning perspective it seems strange. In ASTTranslationUnit.parsingFile there is an empty CoreException handler: https://git.eclipse.org/r/plugins/gitiles/cdt/org.eclipse.cdt/+/refs/heads/master/core/org.eclipse.cdt.core/parser/org/eclipse/cdt/internal/core/dom/parser/ASTTranslationUnit.java#420 I noticed that sometime the call from the for loop inside of ASTTranslationUnit.parsingFile: https://git.eclipse.org/r/plugins/gitiles/cdt/org.eclipse.cdt/+/refs/heads/master/core/org.eclipse.cdt.core/parser/org/eclipse/cdt/internal/core/dom/parser/ASTTranslationUnit.java#414 into IndexBasedFileContentProvider.findIndexFiles into CIndex.getFiles returns IIndexFile.EMPTY_FILE_ARRAY, which causes a CoreException that is handled by the handler mentioned above. I suppose this is an expected exit condition that occurs when the end of a recursive level is reached (no more includes in the current file), but the comment inside the empty exception handler ("Ignore, tracking of replaced files fails.") was really confusing. ********************************************************************* C. ********************************************************************* I verified that buffer.hpp (which contains the definition of buffer, the missing symbol in the example code I using to explore this) is correctly handled inside of CPreprocessor.nextToken loop, at least through the CPreprocessor.executeInclude phase. I haven't explored farther into the parsing yet, but I stepped through enough to see that buffer.hpp is evaluated, all of its includes are evaluated and the recursion unwinds to get to the next include after buffer.hpp with no errors. However, I noticed that there is an error (in a different indirectly included file )which is displayed in the console. I wonder if this error causes the indexer to bail out, resulting in some set of symbols not being indexed properly, include "buffer"? See below for the full stack trace, but the error message is "ParamTy is not a member of id". id is defined in id.hpp, which you can see here: https://github.com/KhronosGroup/SYCL_Reference/blob/main/ref/sycl-intel/CL/sycl/id.hpp I have a couple questions about this. 1. Could this error cause indexing to stop, resulting in unrelated errors? 2. Why is this an error? Are the constructs in id.hpp "new C++" which CDT does not know how to support (sorry, I am not very proficient in C++ so I am not sure). !ENTRY org.eclipse.cdt.core 4 0 2020-10-12 17:44:50.871 !MESSAGE Error: ParamTy is not a member of id !STACK 0 java.lang.IllegalArgumentException: ParamTy is not a member of id at org.eclipse.cdt.internal.core.dom.parser.cpp.ClassTypeHelper.invalidMember(ClassTypeHelper.java:1155) at org.eclipse.cdt.internal.core.dom.parser.cpp.ClassTypeHelper.getVisibility(ClassTypeHelper.java:1081) at org.eclipse.cdt.internal.core.dom.parser.cpp.CPPClassTemplate.getVisibility(CPPClassTemplate.java:281) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.getVisibility(PDOMCPPLinkage.java:923) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.addChild(PDOMCPPLinkage.java:933) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.createBinding(PDOMCPPLinkage.java:885) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.addBinding(PDOMCPPLinkage.java:706) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.addTypeBinding(PDOMCPPLinkage.java:1606) at org.eclipse.cdt.internal.core.pdom.dom.TypeMarshalBuffer.marshalBinding(TypeMarshalBuffer.java:92) at org.eclipse.cdt.internal.core.dom.parser.cpp.semantics.CPPDependentEvaluation.marshalTemplateDefinition(CPPDependentEvaluation.java:98) at org.eclipse.cdt.internal.core.dom.parser.cpp.semantics.EvalBinary.marshal(EvalBinary.java:426) at org.eclipse.cdt.internal.core.pdom.dom.TypeMarshalBuffer.marshalEvaluation(TypeMarshalBuffer.java:169) at org.eclipse.cdt.internal.core.dom.parser.cpp.semantics.EvalUnary.marshal(EvalUnary.java:387) at org.eclipse.cdt.internal.core.pdom.dom.TypeMarshalBuffer.marshalTemplateArgument(TypeMarshalBuffer.java:225) at org.eclipse.cdt.internal.core.dom.parser.cpp.CPPDeferredClassInstance.marshal(CPPDeferredClassInstance.java:237) at org.eclipse.cdt.internal.core.pdom.dom.TypeMarshalBuffer.marshalType(TypeMarshalBuffer.java:135) at org.eclipse.cdt.internal.core.dom.parser.cpp.CPPUnknownMember.marshal(CPPUnknownMember.java:68) at org.eclipse.cdt.internal.core.pdom.dom.TypeMarshalBuffer.marshalType(TypeMarshalBuffer.java:135) at org.eclipse.cdt.internal.core.dom.parser.cpp.CPPFunctionType.marshal(CPPFunctionType.java:170) at org.eclipse.cdt.internal.core.pdom.dom.TypeMarshalBuffer.marshalType(TypeMarshalBuffer.java:135) at org.eclipse.cdt.internal.core.pdom.dom.PDOMLinkage.storeType(PDOMLinkage.java:468) at org.eclipse.cdt.internal.core.pdom.dom.PDOMLinkage.storeType(PDOMLinkage.java:462) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPFunction.setType(PDOMCPPFunction.java:218) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPFunction.initData(PDOMCPPFunction.java:117) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage$ConfigureFunctionTemplate.run(PDOMCPPLinkage.java:452) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.handlePostProcesses(PDOMCPPLinkage.java:1302) at org.eclipse.cdt.internal.core.pdom.dom.cpp.PDOMCPPLinkage.addBinding(PDOMCPPLinkage.java:634) at org.eclipse.cdt.internal.core.pdom.dom.PDOMFile.createPDOMName(PDOMFile.java:522) at org.eclipse.cdt.internal.core.pdom.dom.PDOMFile.addNames(PDOMFile.java:488) at org.eclipse.cdt.internal.core.pdom.WritablePDOM.addFileContent(WritablePDOM.java:158) at org.eclipse.cdt.internal.core.index.WritableCIndex.setFileContent(WritableCIndex.java:89) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.storeFileInIndex(PDOMWriter.java:679) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.storeSymbolsInIndex(PDOMWriter.java:329) at org.eclipse.cdt.internal.core.pdom.PDOMWriter.addSymbols(PDOMWriter.java:287) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.writeToIndex(AbstractIndexerTask.java:1295) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseFile(AbstractIndexerTask.java:1107) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.parseLinkage(AbstractIndexerTask.java:910) at org.eclipse.cdt.internal.core.pdom.AbstractIndexerTask.runTask(AbstractIndexerTask.java:572) at org.eclipse.cdt.internal.core.pdom.indexer.PDOMIndexerTask.run(PDOMIndexerTask.java:164) at org.eclipse.cdt.internal.core.pdom.indexer.PDOMRebuildTask.run(PDOMRebuildTask.java:94) at org.eclipse.cdt.internal.core.pdom.PDOMIndexerJob.run(PDOMIndexerJob.java:160) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Created attachment 284439 [details] CoreException-ExitCondition
I'll leave it to Marc-André to answer your specific questions, but I wanted to chime in with a general suggestion for an alternative strategy for tracking down indexing problems that I like to use: 1. Reduce the project to a minimal example demonstrating the error by iteratively "inlining" included headers and removing unused declarations / simplifying the code in other ways. Note that, due to the difference in how CDT processes code in included header files vs. code in the file that's open in the editor, a minimal example doesn't necessarily mean a single-file example -- but you often can minimize a problem to a single source-header pair. 2. Debug CDT's processing of the minimal example. I find this to be much more tractable than trying to debug an unreduced project. Another advantage of this approach is that if you do step (1) -- even if you don't get all the way to a *minimal* example, but at least to an example that is free of third-party dependencies, you might find others are more willing to step in and do the remaining reduction and debugging.
(In reply to Tony Homer from comment #7) > > This seems like it might be a bug. Maybe I am misunderstanding the code, > but I think net/ifile is null because export.hpp does not include any files, > so it is the leaf node of the recursive tree. This seems like a normal exit > condition to the include inspection, but it is being treated as an error. > I'm not sure, so if you can take a look that would be great. I only have vague understanding of how this all works, but I think each file (.h, .cpp, etc) is represented as an IIndexFile stored in the index. Each IIndexFile contains information about which files it includes (along with all symbol declarations, references within that file, etc). So to me it sounds like this code is OK and the IIndexFile was not saved to the index, possibly because it threw an exception before (aborted PDOMWriter.storeFileInIndex from your stack below?). So the parser log message "buffer.hpp is not indexed" seems useful and accurate: the include is resolved but the file was not stored in the index. > I suppose this is an expected exit condition that occurs when the end of a > recursive level is reached (no more includes in the current file), but the > comment inside the empty exception handler ("Ignore, tracking of replaced > files fails.") was really confusing. Sorry I'm not sure about the exception. It seems to me that the empty array would be for the case where the header file location is resolved but it hasn't been indexed. > I have a couple questions about this. > 1. Could this error cause indexing to stop, resulting in unrelated errors? I don't know how error handling is done. Is id.hpp included indirectly by buffer.hpp? Maybe the errors are related if the buffer class definition depends on id definition. > 2. Why is this an error? Are the constructs in id.hpp "new C++" which CDT > does not know how to support (sorry, I am not very proficient in C++ so I am > not sure). The only obvious thing I see is the deduction guides which are not supported by CDT. On the page, I have only written information about improving general indexing results by looking at setup, and some preprocessor behavior (missing inclusions and macro definitions) but not down to a single error in parsing. Nathan's suggestion to create a minimal example is best. I sometimes preprocess the file (gcc or clang -E) and start trimming it down and verifying at each step that I still get the error (also make sure CDT editor is not in scalability mode). If you don't get the error on the single preprocessed file initially, it usually means that you have to split it like Nathan said.
(In reply to Marc-André Laperle from comment #10) > The only obvious thing I see is the deduction guides which are not supported > by CDT. That reminds me, it's worth updating to the recently released CDT 10.0.1 -- which contains a fix for an important regression where the presence of deduction guides caused problems analyzing code using that class -- and seeing if the error still occurs.
I was able to complete the setup. I'm pretty sure the error is because of lack of deduction guide support. If I comment the deduction guides in buffer.hpp, the sample doesn't compile. Moreoever, if I change the usages of buffer to buffer<int> in the sample, CDT parses the code correctly.
Thanks again Marc-Andre and Nathan. I am still working on an isolated example that would be easier to troubleshoot, but that work is paused for now. I will get back to this ASAP to try to confirm Marc-Andre's theory that the issue is lack of deduction guide support. The information you already provided has been very helpful. Sorry for the continuing delay in follow-up on my side.
