Re: [cdt-dev] Help with Indexer Bug

[Doug Schaefer <dschaefer@xxxxxxx>]

Be careful what you ask for. Most often the reason why users on complex
projects see unresolved symbols errors is because the include path and
toolchain config isn't set up correctly. Even with a full indexer, you'll
still see those errors.

And we need to remember where we came from. 8 years ago when we had a full
indexer, on complex projects, the index took over an hour to build. It was
a pretty horrible experience. Machines are much faster now, but user
expectations are probably still higher too. When I create one of our Qt
projects and it takes a minute to index just the template code, I get
annoyed. Especially since we show semantic errors until it's done (we
really need to do something about that).

I like to idea of tweaking the pattern to make sure we can detect these
things properly. As Andrew E hinted at earlier, we get the same issue with
the QNX/BB10 header files. It's been tough coming up with the right
solution. Maybe a config to force the multiple header versions on certain
files makes sense. But really, I would never expect the user to figure out
what files to put there. It would need to be done on an SDK basis. Which
then leads us to the issue that we don't model SDKs in the CDT...

Doug.

On 2013-10-03 10:36 AM, "Guy.Bonneau@xxxxxxxxxxx"
<Guy.Bonneau@xxxxxxxxxxx> wrote:

>What draw me irresistibly to Eclipse out of the Visual Studio World was
>the
>content assist and the indexer capability which set it apart from
>Microsoft
>Visual Studio IDE. I remember the first time I used it. Wow what a great
>feature...but then...hey what wrong with this symbol. So Eclipse CDT
>strength was paradoxically it weakness. As soon as you jump into moderate
>to complex project from time to time I cringe because the indexer would
>complains of unresolved symbols. I have some coworkers using Eclipse
>mostly
>for Java development and when they need to switch from time to time to a
>CDT project I always have them come to me and ask why do the indexer
>complains of unresolved symbols. Coming from the Eclipse Java world is
>hard
>to understand for them when you almost never worked with C++.
>
>Shouldn't some option could be provided to let the user choose between a
>full indexing at the expense of processing time vs an heuristic indexing.
>May be multi-treading indexing if possible would help. At least we would
>be
>able to choose according to the project size and feature. We shouldn't
>need
>to jump into the CDTcode source package and have to modify the indexer
>code
>like Joseph did.
>
>Thanks
>Guy
>
>
>
>From:	Joseph Henry <Joseph.Henry@xxxxxxx>
>To:	CDT General developers list. <cdt-dev@xxxxxxxxxxx>,
>Date:	2013-10-03 09:17
>Subject:	Re: [cdt-dev] Help with Indexer Bug
>Sent by:	cdt-dev-bounces@xxxxxxxxxxx
>
>
>
>For my purposes, forcing the Indexer to always track significant macros
>seems to work very well.
>
>I have not seen much performance degradation yet.
>
>I will continue to look into this when I have time, because I feel that
>content assist is the single most important feature in an IDE and if it
>does not work correctly, the IDE wont be as widely used as it could be.
>
>Joseph.
>
>-----Original Message-----
>From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
>Behalf Of Schorn, Markus
>Sent: Thursday, October 03, 2013 3:55 AM
>To: CDT General developers list.
>Subject: Re: [cdt-dev] Help with Indexer Bug
>
>We have tried to provide a complete solution, which worked for smaller
>projects but did not work for projects with a few thousands of files. You
>can study what we have tried and you will need an idea beyond that to come
>up with a scalable solution:
>https://bugs.eclipse.org/bugs/show_bug.cgi?id=197989
>
>To make indexing work for your code, I think your best chance is to exempt
>files that need multiple variants from CDTs pragma once treatment.
>This can for instance be done by one of the following means:
>* Change your source code (intentionally disturb CDT's pragma once
>detection)
>* Provide some preference mechanism in CDT, where you can explicitly force
>multiple versions for some files
>* Identify a pattern to automatically detect files for which we should
>force multiple versions (although they have
>    pragma once semantics)
>
>Markus.
>
>-----Original Message-----
>From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
>Behalf Of Joseph Henry
>Sent: Wed, 02. 10. 2013 18:47
>To: CDT General developers list.
>Subject: Re: [cdt-dev] Help with Indexer Bug
>
>Adding a comment that Brandon Philips sent to me directly:
>
>Could CDT store in the database the preprocessor symbols actually
>referenced by each header it parses?  Then CDT can check if those specific
>values have changed to decide if re-parsing is necessary.  Maybe in the
>realm of "not a simple change" though. :)
>
>(Sorry to jump in randomly.)
>
>-----Original Message-----
>From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On
>Behalf Of Andrew Eidsness
>Sent: Wednesday, October 02, 2013 12:34 PM
>To: cdt-dev@xxxxxxxxxxx
>Subject: Re: [cdt-dev] Help with Indexer Bug
>
>On 13-10-02 12:01 PM, Joseph Henry wrote:
>> What was the fairly larger project that you tested?
>>
>> I ran a test of my own. The first one is I added a
>ctx.trackSignificantMacros(); statement in the detectIncludeGuard function
>in CPreprocessor even if an include guard was found. This fixes all of my
>issues. This is the output of the log:
>>
>> !MESSAGE Indexed 'Index1' (10 sources, 242 headers) in 3.45 sec:
>> 31,198 declarations; 65,703 references; 0 unresolved inclusions; 0
>> syntax errors; 0 unresolved names (0.00%)
>>
>> Now this is how the output without my change:
>>
>> !MESSAGE Indexed 'Index1' (10 sources, 221 headers) in 3.18 sec:
>> 26,224 declarations; 56,348 references; 0 unresolved inclusions; 101
>> syntax errors; 323 unresolved names (0.39%)
>>
>> I do not see a significant performance increase, yet I do see that my
>project resolves perfectly.
>
>The sample projects are just from internal development.  I'm not at all
>familiar with the structure of the projects, so I can't make any guesses
>at
>why one was so much worse.  However, it also happened to be the largest of
>my three samples.
>
>I don't have the full results from the test anymore, but here is the
>summary of the indexer (ran just now) from the project that was 7x slower:
>
>C/C++ Indexer: Project 'seven_times_slower' (504 sources, 5844 headers)
>    Options: indexer='PDOMFastIndexer', parseAllFiles=true,
>unusedHeaders=useCPP, skipReferences=false, skipImplicitReferences=false,
>skipTypeReferences=false, skipMacroReferences=false.
>    Database: 119873536 bytes
>    Timings: 1342211 total, 319891 parser, 9345 resolution, 51686 index
>update.
>    Errors: 0 internal, 0 include, 0 scanner, 0 syntax errors.
>    Names: 399371 declarations, 1735121 references, 350(0.02%) unresolved.
>    Cache[64MB]: 4872643790 hits, 7259(0.00%) misses.
>Indexer: completed PDOMRebuildTask[1342246ms]
>
>I wouldn't call this project large, but it was bigger the other toy
>projects that I was using for testing.
>
>Also, I would caution against the change you described.  I was not able to
>make that variation work without introducing new problems in other
>projects.
>
>Finally, a technique that I found useful was to put print statements in
>the
>constructors for PDOMFile, PDOMMacro, and PDOMInclude.  I used the record
>id as a unique identifier.  In PDOMInclude I printed both the containing
>and the target files.  Printing the lists of significant and
>internallyModified macros generated ALOT of output, so I used filtering
>and
>the focused on only one or two at a time.
>
>While looking for these results I found notes on a related problem that I
>had meant to raise as a bug.  I've done that now (418533).  From what I
>remember, the description of that bug was one of the things that wound up
>introducing new bugs when I made the change you described.
>
>-Andrew
>_______________________________________________
>cdt-dev mailing list
>cdt-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/cdt-dev
>
>
>_______________________________________________
>cdt-dev mailing list
>cdt-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/cdt-dev
>_______________________________________________
>cdt-dev mailing list
>cdt-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/cdt-dev
>
>
>_______________________________________________
>cdt-dev mailing list
>cdt-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/cdt-dev
>
>
>
>
>DISCLAIMER:
>Privileged and/or Confidential information may be contained in this
>message. If you are not the addressee of this message, you may not
>copy, use or deliver this message to anyone. In such event, you
>should destroy the message and kindly notify the sender by reply
>e-mail. It is understood that opinions or conclusions that do not
>relate to the official business of the company are neither given
>nor endorsed by the company.
>Thank You.
>_______________________________________________
>cdt-dev mailing list
>cdt-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/cdt-dev