Re: [cdt-dev] Proofing the CDT w/ large projects

[Mathieu Lapensee <lapensee@xxxxxxxxxx>]

Thomas,

Thanks for the observations.  
We will take note of these indexer times for CDT 1.2 .

We will get more benchmarks in CDT 1.2
and we will compare them to the ones with CDT 2.0 as we are planning to
execute lots of scalability/performance tests.
We will concentrate our efforts on indexer
related features like Content Assist and Search.

Mathieu Lapensée
Software Verification - IBM Rational

"Thomas Fletcher" <thomasf@xxxxxxx>
Sent by: cdt-dev-admin@xxxxxxxxxxx

02/02/2004 04:49 AM

Please respond to
cdt-dev@xxxxxxxxxxx

To	<cdt-dev@xxxxxxxxxxx>
cc
Subject	[cdt-dev] Proofing the CDT w/ large projects

Folks,

  Just thought you would be interested to hear a bit about the environment
that one of our customers is working with using CDT 1.2:

- Project tree contains 5.68 GB of content when fully built
- Distributed over 38,601 files in 3,783 directories
- Development file breakdown:
  27   *.cxx
  464  *.c
  6039 *.cpp  
  
  3    *.hxx
  2453 *.hpp              
  
  9061 *.h

  2    *.so
  35   *.dll
  45   *.lib
  75   *.exe
  994  *.a
  6206 *.so

The time it takes to load this project into the IDE initially (on a 3Ghz
machine) is ~3m, but subsequent IDE start-ups are in the 7s timeframe.
 
Compared to previous versions of CDT, this is pretty awesome!

The big bottleneck right now seems to be the indexer.  Initially it
takes
>5m to index the 15,000+ source files and then >2m to pull out all
of the
classes and types from a search (I'm testing via the Open Type action 
Chris Wiebe implemented).  (Note: It is also killer for the code completion
perfomance it seems since the user doesn't get any feedback about what
is
happening).

Some things I've noticed and am looking into:
- There are a number of scanner and parser errors (ranging from failures
  to outright exceptions).  This is likely due to an improperly
configured
  source base (no #defines, #includes set), but shouldn't affect the
  re-indexing operation.

- There are a couple of "expected exceptions" which then return
null, for
  example in FileListBlock.getFile(int fileNum) we hit the exception
  condition very often.  Checking the range and returning null
improves
  the performance

- There are a couple of mystery exceptions I still haven't figured out.
  For example WordEntry.mapRefs(int [] mappings) is often throwing
an
  ArrayIndexOutOfBounds exception ... this is trouble I believe and
  leads to:

- The fact that in the IndexManager.getIndex(IPath, boolean, boolean)
  we (in this particular project) are never retrieving an index which
  has a known state (it is always UNKNOWN_STATE) which means we go
and
  rebuild the index for everything.

I think that with Chris Wiebe's "Open Type" contributed plugin
and a 
moderately large project you should be able to see this kind of churn.
I'm looking into it, but thought that the indexer guy (Hi Bogdan!) might
have some thoughts about what is going on.

Also ... it takes a fair amount of time to add in 15,000+ entries using
the IndexAllProject.execute() -> IndexManager.addSource() method, several
seconds at least in my environment.  

Just passing along some observations.  Unfortunately I can't pass
along the
source base, but I will certainly try and pull out relevant information
as
requested/required and dig into this while I can.

Thanks,
 Thomas
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/cdt-dev