Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-dev] Help with Indexer Bug 
For my purposes, forcing the Indexer to always track significant macros seems to work very well.

I have not seen much performance degradation yet.

I will continue to look into this when I have time, because I feel that content assist is the single most important feature in an IDE and if it does not work correctly, the IDE wont be as widely used as it could be.

Joseph.

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Schorn, Markus
Sent: Thursday, October 03, 2013 3:55 AM
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

We have tried to provide a complete solution, which worked for smaller projects but did not work for projects with a few thousands of files. You can study what we have tried and you will need an idea beyond that to come up with a scalable solution:  https://bugs.eclipse.org/bugs/show_bug.cgi?id=197989

To make indexing work for your code, I think your best chance is to exempt files that need multiple variants from CDTs pragma once treatment.
This can for instance be done by one of the following means:
* Change your source code (intentionally disturb CDT's pragma once detection)
* Provide some preference mechanism in CDT, where you can explicitly force multiple versions for some files
* Identify a pattern to automatically detect files for which we should force multiple versions (although they have
    pragma once semantics)

Markus.

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Joseph Henry
Sent: Wed, 02. 10. 2013 18:47
To: CDT General developers list.
Subject: Re: [cdt-dev] Help with Indexer Bug

Adding a comment that Brandon Philips sent to me directly:

Could CDT store in the database the preprocessor symbols actually referenced by each header it parses?  Then CDT can check if those specific values have changed to decide if re-parsing is necessary.  Maybe in the realm of "not a simple change" though. :)

(Sorry to jump in randomly.)

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Andrew Eidsness
Sent: Wednesday, October 02, 2013 12:34 PM
To: cdt-dev@xxxxxxxxxxx
Subject: Re: [cdt-dev] Help with Indexer Bug

On 13-10-02 12:01 PM, Joseph Henry wrote:
> What was the fairly larger project that you tested?
> 
> I ran a test of my own. The first one is I added a ctx.trackSignificantMacros(); statement in the detectIncludeGuard function in CPreprocessor even if an include guard was found. This fixes all of my issues. This is the output of the log:
> 
> !MESSAGE Indexed 'Index1' (10 sources, 242 headers) in 3.45 sec: 
> 31,198 declarations; 65,703 references; 0 unresolved inclusions; 0 
> syntax errors; 0 unresolved names (0.00%)
> 
> Now this is how the output without my change:
> 
> !MESSAGE Indexed 'Index1' (10 sources, 221 headers) in 3.18 sec: 
> 26,224 declarations; 56,348 references; 0 unresolved inclusions; 101 
> syntax errors; 323 unresolved names (0.39%)
> 
> I do not see a significant performance increase, yet I do see that my project resolves perfectly.

The sample projects are just from internal development.  I'm not at all familiar with the structure of the projects, so I can't make any guesses at why one was so much worse.  However, it also happened to be the largest of my three samples.

I don't have the full results from the test anymore, but here is the summary of the indexer (ran just now) from the project that was 7x slower:

C/C++ Indexer: Project 'seven_times_slower' (504 sources, 5844 headers)
    Options: indexer='PDOMFastIndexer', parseAllFiles=true, unusedHeaders=useCPP, skipReferences=false, skipImplicitReferences=false, skipTypeReferences=false, skipMacroReferences=false.
    Database: 119873536 bytes
    Timings: 1342211 total, 319891 parser, 9345 resolution, 51686 index update.
    Errors: 0 internal, 0 include, 0 scanner, 0 syntax errors.
    Names: 399371 declarations, 1735121 references, 350(0.02%) unresolved.
    Cache[64MB]: 4872643790 hits, 7259(0.00%) misses.
Indexer: completed PDOMRebuildTask[1342246ms]

I wouldn't call this project large, but it was bigger the other toy projects that I was using for testing.

Also, I would caution against the change you described.  I was not able to make that variation work without introducing new problems in other projects.

Finally, a technique that I found useful was to put print statements in the constructors for PDOMFile, PDOMMacro, and PDOMInclude.  I used the record id as a unique identifier.  In PDOMInclude I printed both the containing and the target files.  Printing the lists of significant and internallyModified macros generated ALOT of output, so I used filtering and the focused on only one or two at a time.

While looking for these results I found notes on a related problem that I had meant to raise as a bug.  I've done that now (418533).  From what I remember, the description of that bug was one of the things that wound up introducing new bugs when I made the change you described.

-Andrew
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

Back to the top