Re: [cdt-dev] Help with Indexer Bug

[Guy.Bonneau@xxxxxxxxxxx]

What draw me irresistibly to Eclipse out of the Visual Studio World was the
content assist and the indexer capability which set it apart from Microsoft
Visual Studio IDE. I remember the first time I used it. Wow what a great
feature...but then...hey what wrong with this symbol. So Eclipse CDT
strength was paradoxically it weakness. As soon as you jump into moderate
to complex project from time to time I cringe because the indexer would
complains of unresolved symbols. I have some coworkers using Eclipse mostly
for Java development and when they need to switch from time to time to a
CDT project I always have them come to me and ask why do the indexer
complains of unresolved symbols. Coming from the Eclipse Java world is hard
to understand for them when you almost never worked with C++.

Shouldn't some option could be provided to let the user choose between a
full indexing at the expense of processing time vs an heuristic indexing.
May be multi-treading indexing if possible would help. At least we would be
able to choose according to the project size and feature. We shouldn't need
to jump into the CDTcode source package and have to modify the indexer code
like Joseph did.

Thanks
Guy



From:	Joseph Henry <Joseph.Henry@xxxxxxx>
To:	CDT General developers list. <cdt-dev@xxxxxxxxxxx>,
Date:	2013-10-03 09:17
Subject:	Re: [cdt-dev] Help with Indexer Bug
Sent by:	cdt-dev-bounces@xxxxxxxxxxx



For my purposes, forcing the Indexer to always track significant macros
seems to work very well.

I have not seen much performance degradation yet.

I will continue to look into this when I have time, because I feel that
content assist is the single most important feature in an IDE and if it
does not work correctly, the IDE wont be as widely used as it could be.

Joseph.

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
Behalf Of Schorn, Markus
Sent: Thursday, October 03, 2013 3:55 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

We have tried to provide a complete solution, which worked for smaller
projects but did not work for projects with a few thousands of files. You
can study what we have tried and you will need an idea beyond that to come
up with a scalable solution:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=197989

To make indexing work for your code, I think your best chance is to exempt
files that need multiple variants from CDTs pragma once treatment.
This can for instance be done by one of the following means:
* Change your source code (intentionally disturb CDT's pragma once
detection)
* Provide some preference mechanism in CDT, where you can explicitly force
multiple versions for some files
* Identify a pattern to automatically detect files for which we should
force multiple versions (although they have
    pragma once semantics)

Markus.

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
Behalf Of Joseph Henry
Sent: Wed, 02. 10. 2013 18:47
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

Adding a comment that Brandon Philips sent to me directly:

Could CDT store in the database the preprocessor symbols actually
referenced by each header it parses?  Then CDT can check if those specific
values have changed to decide if re-parsing is necessary.  Maybe in the
realm of "not a simple change" though. :)

(Sorry to jump in randomly.)

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
Behalf Of Andrew Eidsness
Sent: Wednesday, October 02, 2013 12:34 PM
To: cdt-dev@xxxxxxxxxxx
Subject: Re: [cdt-dev] Help with Indexer Bug

On 13-10-02 12:01 PM, Joseph Henry wrote:
> What was the fairly larger project that you tested?
>
> I ran a test of my own. The first one is I added a
ctx.trackSignificantMacros(); statement in the detectIncludeGuard function
in CPreprocessor even if an include guard was found. This fixes all of my
issues. This is the output of the log:
>
> !MESSAGE Indexed 'Index1' (10 sources, 242 headers) in 3.45 sec:
> 31,198 declarations; 65,703 references; 0 unresolved inclusions; 0
> syntax errors; 0 unresolved names (0.00%)
>
> Now this is how the output without my change:
>
> !MESSAGE Indexed 'Index1' (10 sources, 221 headers) in 3.18 sec:
> 26,224 declarations; 56,348 references; 0 unresolved inclusions; 101
> syntax errors; 323 unresolved names (0.39%)
>
> I do not see a significant performance increase, yet I do see that my
project resolves perfectly.

The sample projects are just from internal development.  I'm not at all
familiar with the structure of the projects, so I can't make any guesses at
why one was so much worse.  However, it also happened to be the largest of
my three samples.

I don't have the full results from the test anymore, but here is the
summary of the indexer (ran just now) from the project that was 7x slower:

C/C++ Indexer: Project 'seven_times_slower' (504 sources, 5844 headers)
    Options: indexer='PDOMFastIndexer', parseAllFiles=true,
unusedHeaders=useCPP, skipReferences=false, skipImplicitReferences=false,
skipTypeReferences=false, skipMacroReferences=false.
    Database: 119873536 bytes
    Timings: 1342211 total, 319891 parser, 9345 resolution, 51686 index
update.
    Errors: 0 internal, 0 include, 0 scanner, 0 syntax errors.
    Names: 399371 declarations, 1735121 references, 350(0.02%) unresolved.
    Cache[64MB]: 4872643790 hits, 7259(0.00%) misses.
Indexer: completed PDOMRebuildTask[1342246ms]

I wouldn't call this project large, but it was bigger the other toy
projects that I was using for testing.

Also, I would caution against the change you described.  I was not able to
make that variation work without introducing new problems in other
projects.

Finally, a technique that I found useful was to put print statements in the
constructors for PDOMFile, PDOMMacro, and PDOMInclude.  I used the record
id as a unique identifier.  In PDOMInclude I printed both the containing
and the target files.  Printing the lists of significant and
internallyModified macros generated ALOT of output, so I used filtering and
the focused on only one or two at a time.

While looking for these results I found notes on a related problem that I
had meant to raise as a bug.  I've done that now (418533).  From what I
remember, the description of that bug was one of the things that wound up
introducing new bugs when I made the change you described.

-Andrew
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev




DISCLAIMER:
Privileged and/or Confidential information may be contained in this
message. If you are not the addressee of this message, you may not
copy, use or deliver this message to anyone. In such event, you
should destroy the message and kindly notify the sender by reply
e-mail. It is understood that opinions or conclusions that do not
relate to the official business of the company are neither given
nor endorsed by the company.
Thank You.