Bug 178061 - [Help Wanted] Standard library in Sun C++ compilers messes up CDT parser/indexer
Summary: [Help Wanted] Standard library in Sun C++ compilers messes up CDT parser/indexer
Status: NEW
Alias: None
Product: CDT
Classification: Tools
Component: cdt-core (show other bugs)
Version: 3.1.2   Edit
Hardware: Sun Solaris
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Jonah Graham CLA
URL:
Whiteboard:
Keywords: helpwanted
Depends on:
Blocks:
 
Reported: 2007-03-19 10:17 EDT by Alexey Danilcenko CLA
Modified: 2020-09-04 15:22 EDT (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Danilcenko CLA 2007-03-19 10:17:51 EDT
I am running CDT 3.1.2 on Eclipse 3.2.2 on Solaris 10. The compiler is Sun C++ Pro compiler 6.2. It comes with the standard C++ library implementation that uses RogueWave version of STL (RW v.7). The problem I have is as soon as I add include directory for the standard library from that Sun C++ compiler to my C++ project include paths all parsing seems to be messed up completely. This manifests itself in a way that all the symbols from outline(or underlying file in a project) disappear for the open file - like it does not have a structure at all. As soon as I remove that include  from the include paths - everything goes back to normal. 

It would not be such a big deal but for one problem: a lot of symbols from stanadard C++ library (like ostream) are not getting recognised and results in a lot of errors that makes it difficult to distinguish from reall errors.

This error was also observed on CDT 3.0.2 on Eclipse 3.1.2 on Solaris - only there the path was produced by auto discovery (CDT 3.1.2 does not do it automatically)
Comment 1 Chris Recoskie CLA 2007-06-13 13:18:01 EDT
It's hard to say what the problem is here.  There is likely a problem with the indexer not knowing what the include paths and macros are due to a lack of a scanner configuration provider for Sun Pro.  It may also be that the dialect of C++ they support may or may not be different than what our parser supports.

In any case none of the committers are currently looking at anything related to Sun's tools, so help from the community will be required if this is to be addressed.
Comment 2 Alexey Danilcenko CLA 2007-06-13 17:37:25 EDT
(In reply to comment #1)
> There is likely a problem with the
> indexer not knowing what the include paths and macros are due to a lack of a
> scanner configuration provider for Sun Pro.

This is an STL and as such the directory is self-contained. If the parser really parses C++ code then this should be no problem for it. As I said I tried to add directory  with STL source manually but it messed up the parsing. To me it looks like the parser that you have does not recover from the errors gracefully (and what is more worrying - does not report the errors - for I didn't find any in the logs).


> It may also be that the dialect of
> C++ they support may or may not be different than what our parser supports.

Sun C++ Pro compiler is pretty standard and supports all the C++ features properly. All the code that we have compilable on both Sun C++ Pro and GNU C++ for Solaris and Linux so that addresses that should eliminate suspicion on dialect difference. Besides it's the most recent version (updated quarterly) so it's pretty much up to date.

I now gave up trying tomake it work and have to put up with parser suggesting errors in source files everytime I use STL function.

Funny enough the problem with outline messed up is still manifests on some classes. It seems that it has to do with number of elements in outline because I was developing one class in the course of the last week and it was fine until after one moment when I added new method and all outline disappeared. This only seems to happen with source files not header files.


Comment 3 Chris Recoskie CLA 2007-06-14 07:50:14 EDT
(In reply to comment #2)
> This is an STL and as such the directory is self-contained. If the parser
> really parses C++ code then this should be no problem for it. As I said I tried
> to add directory  with STL source manually but it messed up the parsing. To me
> it looks like the parser that you have does not recover from the errors
> gracefully (and what is more worrying - does not report the errors - for I
> didn't find any in the logs).

The parser still needs context.  RogueWave provides their STL for more platforms than just the Sun compiler.  From experience, unless something has changed drastically in the past few years, this means the code is riddled with #ifdefs.  If you don't have the proper macro context setup with the same #defines that the compiler will provide, then the code will not be parsed the same.  This can lead to errors.  It doesn't make sense to log these errors in the logs because they are not due to erroneous operation (e.g. "parser crashed").

Also if it can't find all the header files in the entire include chain, then this can cause parsing to fail due to undefined macros, declarations, etc.  So, you need to properly point the parser at where all the includes live, or if you're going to copy them somewhere, you must make sure you get them all and preserve the directory structure exactly.
Comment 4 Alexey Danilcenko CLA 2007-06-14 08:11:50 EDT
(In reply to comment #3)
> The parser still needs context.  RogueWave provides their STL for more
> platforms than just the Sun compiler.  From experience, unless something has
> changed drastically in the past few years, this means the code is riddled with
> #ifdefs.  If you don't have the proper macro context setup with the same
> #defines that the compiler will provide, then the code will not be parsed the
> same.  This can lead to errors.  It doesn't make sense to log these errors in
> the logs because they are not due to erroneous operation (e.g. "parser
> crashed").

Ok I realise all that and now now why there are no errors in a log - but why is the rest fails? Why is improper include results in the failing of the outline for the whole source file? It would make a sense to have an option to say not follow all includes (like system includes or manually added ones) or an option that will allow to ignore failed includes.

The error you meantion give me a thought that CDT parsing is trying to imitate compiler and does it in the same way but this is a bit wrong because there will be plenty of situations like the one I have with this STL and to avoid it all preprocessor stuff needs to be duplicated in makefiles and CDT project preferences. I don't see why CDT needs the full blown C++ parser mimicking compiler. The purpouse as I unederstand it is to build a cros-referenced symbol table. Imho it could be done by just parsing the file with proper C++ parser building symbol table and then go through all includes with the same parser trying to resolve unresolved symbols build on a first stage. Not exactly the way C++ compiler does it but avoids breaking the whole parsing because of one broken header.


Comment 5 Alexey Danilcenko CLA 2007-06-14 08:15:42 EDT
(In reply to comment #4)
> Imho it could be done by just parsing the file with proper C++ parser
> building symbol table and then go through all includes with the same parser
> trying to resolve unresolved symbols build on a first stage. 

By that I mean do a flat parsing on a first stage (not following any includes) and parsing all the includes to resolve symbols on a second stage.

Comment 6 Chris Recoskie CLA 2007-06-14 08:46:15 EDT
It is not that simple.  A full abstract syntax tree is built from the code.  Some clients need this much information for analyzing the structure of the code.

The existence of the preprocessor also means that there is lots of code out there for which you cannot parse even declarations without properly setting up the preprocessor context.  People like to do evil things with macros to save typing, and without properly expanding these macros you are going to miss constructs in the code.

At any rate, we're off topic.  Things are working as they are intended and designed.  The root cause of this problem is that the Sun Pro tools are not directly supported by CDT.  We currently have no plans to do so.  If someone in the community is interested in doing the work we will gladly accept a contribution.
Comment 7 Alexey Danilcenko CLA 2007-06-14 09:03:59 EDT
(In reply to comment #6)
> At any rate, we're off topic.  Things are working as they are intended and
> designed.  The root cause of this problem is that the Sun Pro tools are not
> directly supported by CDT.  We currently have no plans to do so.  If someone in
> the community is interested in doing the work we will gladly accept a
> contribution.
> 

It seems that in that case the CDT should really be called GCC DT because it does not support other compilers ;-)

On a serious note - what kind of compiler support are you talking about? What does it involve to develop support for Sun C++ or any other compiler for that matter?

And I still think you are missing the second part of what I said - outliner still does messes up when complexity of the code is growing. I have removed all compiler specific includes and it's only our projects now. Our code does not have macroses (we have a strict policy to use inlines and typedefs - no macro) so it should not happen according to what you said - yet it still does in some of the files I work on (even happened as I worked on file).
Comment 8 Chris Recoskie CLA 2007-06-15 13:57:41 EDT
(In reply to comment #7)
> And I still think you are missing the second part of what I said - outliner
> still does messes up when complexity of the code is growing. I have removed all
> compiler specific includes and it's only our projects now. Our code does not
> have macroses (we have a strict policy to use inlines and typedefs - no macro)
> so it should not happen according to what you said - yet it still does in some
> of the files I work on (even happened as I worked on file).


If you have a specific example you can provide, then I'd suggest filing a separate bug for that.