Re: [cdt-dev] Questions on CDT's indexer/parser internals

[Nathan Ridge <zeratul976@xxxxxxxxxxx>]

> The Issues we're talking about are ones that are already documented
> for other larger projects like Mozilla
> (https://developer.mozilla.org/en-US/docs/Eclipse_CDT#Known_Issues).
>
> What we really what to know is what's happening going forward on the indexer.
> Is it as bad as the original CDT documentation said or has it had
> enough recent work to be mostly in shape.
>
> If someone who knows more about the indexer's recent history could
> catch us we we'd really appreciate it. And I don't mind taking the
> conversation offline so everyone doesn't get spammed on it. :-)

I'm not a CDT developer, but I can share my experience as a CDT user.

I have been using CDT since Ganymede (which was released in 2008).
Initially the imperfections in the parser/indexer were very 
noticeable (red underlines weren't introduced until Indigo, but the 
imperfections were apparent in the incorrect syntax highlighting and 
code navigation). I have filed over 50 bugs in the parser/indexer
since then, most of which have been fixed. I think that overall the 
CDT parser/indexer has come a long way since Ganymede. There are 
still false positive errors, but their frequency is much lower than 
before.

The following bugs have been/are responsible for the greatest number
of false positive errors in our codebase:


1. https://bugs.eclipse.org/bugs/show_bug.cgi?id=197989

This bug relates to storing multiple variants of a header file in 
the index. In the presence of #ifdef/#endif blocks, the parts of
a header file that are seen by the compiler can depend on what 
macros are defined (and what values they have) when the header
file is included (the "macro dictionary"). Often the same header
file can be included in two different places with different
macro dictionaries. If only one of the variants is stored in the
index, then code that relies on declarations present only in the
other variants is not indexed accurately.

This bug is fixed in Juno, but the fix is only partial. Multiple
variants are only stored for header files which have no include
guards, and which are therefore meant to be included multiple
times *in the same translation unit*. This issue, however, can
also affect header files which have include guards, if different
variants of them are included in different translation units.
Storing multiple variants of all header files, however, leads to
a combinatorial explosion of variants, which is why it wasn't
implemented.

Proposals to mitigate this remaining problem can be found here:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=381601
and here:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=380511


2. https://bugs.eclipse.org/bugs/show_bug.cgi?id=299911

This bug relates to parsing "dependent expressions", which are
expressions of the form 'sizeof(e)' (or, in C++11, 
'decltype(e)') that appear in a template and where e is
dependent on some of the template parameters. CDT cannot store
these correctly in the index, leading to problems when they
are used in a header file.

Code that uses the Boost libraries is particularly suspectible
to this bug because Boost type traits metafunctions such as
boost::is_convertible<T, U>, which are used throughout Boost,
use dependent expressions of the form 'sizeof(e)'.

This bug is not yet fixed, though a recent comment indicates
that most of the work has been done and outlines the remaining
steps. This bug causes the great majority of the remaining
false positive errors in our codebase, which uses Boost
extensively.


3. https://bugs.eclipse.org/bugs/show_bug.cgi?id=315964

This bug relates to header files which are meant to be included
at a non-global scope. CDT cannot parse these at all, because
it expects that all header files contain declarations in the
global scope.

Admittedly, header files are seldom used like this, but some
projects use them to e.g. generate members of an enumeration
using an external tool, and then include the generated file
like so:

enum E
{
   #include "members.autogenerated.h"
};

An approach for fixing this bug has been outlined in a comment
to the bug, but no one has implemented it yet.


I believe that once these 3 issues are fixed, there will be
very few false positive errors given by CDT (at least in our
codebase).

Regards,
Nate