Hi
Achim,
There
are a bunch of questions, I respond to the overall performance of the
indexer in your project.
My
guess is, that in your specific project the resolution of user-defined
operators takes quite some time. Your tracing macro adds 12 of them
each time you use it. I had created a few bugs on the performance issue
and also worked on improvements, nevertheless it remains to be a rather
expensive computation.
To
speed up the indexer you can try to disable indexing of user-defined
operators (Indexer option ‘Skip implicit references’). It may also
help to update CDT to version 7.0. There are no further performance
improvements for resolving user-defined operators after that.
Markus.
Hello,
currently I'm in the process of digging into the indexing process for
our huge legacy project. I enabled the indexer statistics, but I'm
stuck with some figures:
C/C++ Indexer: Project 'IDXTST'
(2154 sources, 2888 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=false,
unusedHeaders=skip, skipReferences=false, skipImplicitReferences=false,
skipTypeReferences=false, skipMacroReferences=false.
Database: 108953600 bytes
Timings: 1127585 total, 314138 parser, 58757 resolution,
244067 index update.
Errors: 0 internal, 21 include, 204 scanner, 137 syntax
errors.
Names: 492543 declarations, 2420523 references, 24306(0.83%)
unresolved.
Cache[88mb]: 264220629 hits, 948(0.00%) misses.
As you see, it takes 1127 seconds to rebuild the index, almost 20
minutes. If I add up the values for parser, resolution and index
update, I get only 617 seconds. What happens in the remaining 511
seconds?
Well, now it gets really weird: Indexing the same project, but changing
one preprocessor macro which enables some internal tracing code, and
the result is as follows:
C/C++ Indexer: Project 'IDXTST'
(2153 sources, 2891 headers)
Options: indexer='PDOMFastIndexer', parseAllFiles=false,
unusedHeaders=skip, skipReferences=false, skipImplicitReferences=false,
skipTypeReferences=false, skipMacroReferences=false.
Database: 122183680 bytes
Timings: 8242961 total, 460663 parser, 70149 resolution,
308092 index update.
Errors: 0 internal, 21 include, 204 scanner, 140 syntax
errors.
Names: 498832 declarations, 2870298 references, 25360(0.75%)
unresolved.
Cache[88mb]: 652516848 hits, 2077(0.00%) misses.
The total time explodes, it is 8243 seconds now, 2 hours and 17
minutes. All the other numbers increase in a reasonable amount, but
total time is way off what I would expect. Here is a table that shows
it clearly:
NOTRACES
with Traces Factor
sources 2154 2153 1,000
headers 2888 2891 1,001
DB bytes 108953600 122183680 1,121
Timing
total (seconds) 1128 8243 7,310
parser 314 461 1,466
resolution 59 70 1,194
index upd. 244 308 1,262
parser+res+idxupd 617 839 1,360
Diff to total 511 7404
45% 90%
Names:
declarations 492543 498832 1,013
references 2420523 2870298 1,186
unresolved 24306 25360 1,043
In this case, 90% of the total time are not accounted for in the
parser, resolution and index updates figures. Why does it increase so
dramatically?
Our tracing macro is defined as follows:
#ifdef
NOTRACES
#define COCOUT(X,Y,Z)
#else
#define COCOUT(_INDEX,_MASK,_PARS) \
((void)((COTraceFlags[_INDEX]&_MASK) && \
(cout << coutPrep()) _PARS ))
#endif
It is used 34,000 times in the code. Additionally, there are 3,800
places with conditional code inside #ifndef NOTRACES, which effectively
is formatting stuff for the trace facility, like this:
#ifndef NOTRACES
COCOUT(TraceDpa,DPA_AC_CONNECTION,
<< "sdDptVarResCallbackFct: "
<< " PDU. len = " << pduLen
<< " Data: " << hex);
for (int i = 0; i < pduLen; i++)
COCOUT(TraceDpa,DPA_AC_CONNECTION, << " 0x" <<
(int)s7PduP[i]);
COCOUT(TraceDpa,DPA_AC_CONNECTION, << dec <<
endl);
#endif // !NOTRACES
Any insight into these figures (Markus maybe??).
For reference: We are running Eclipse 3.5.1 with CDT 6.0.1 (yes, there
is a very conservative deployment policy in our department).
Thanks, Achim
