Re: [cdt-dev] Proofing the CDT w/ large projects

[Chris Songer <songer@xxxxxxxxxxxxx>]

Thomas,

What speed / class of machine were you running on? Was this a 1g/256m 
machine or 3g/1g machine? I'm curious only because this kind of information 
will be helpful as we look to set expectations with the AEs and customers.

Thanks!
-Chris

At 09:49 AM 2/2/2004 +0000, Thomas Fletcher wrote:



> Folks,
>
>   Just thought you would be interested to hear a bit about the environment
> that one of our customers is working with using CDT 1.2:
>
> - Project tree contains 5.68 GB of content when fully built
> - Distributed over 38,601 files in 3,783 directories
> - Development file breakdown:
>   27   *.cxx
>   464  *.c
>   6039 *.cpp
>
>   3    *.hxx
>   2453 *.hpp
>   9061 *.h
>
>   2    *.so
>   35   *.dll
>   45   *.lib
>   75   *.exe
>   994  *.a
>   6206 *.so
>
> The time it takes to load this project into the IDE initially (on a 3Ghz
> machine) is ~3m, but subsequent IDE start-ups are in the 7s timeframe.
> Compared to previous versions of CDT, this is pretty awesome!
>
> The big bottleneck right now seems to be the indexer.  Initially it takes
> >5m to index the 15,000+ source files and then >2m to pull out all of the
> classes and types from a search (I'm testing via the Open Type action
> Chris Wiebe implemented).  (Note: It is also killer for the code completion
> perfomance it seems since the user doesn't get any feedback about what is
> happening).
>
> Some things I've noticed and am looking into:
> - There are a number of scanner and parser errors (ranging from failures
>   to outright exceptions).  This is likely due to an improperly configured
>   source base (no #defines, #includes set), but shouldn't affect the
>   re-indexing operation.
>
> - There are a couple of "expected exceptions" which then return null, for
>   example in FileListBlock.getFile(int fileNum) we hit the exception
>   condition very often.  Checking the range and returning null improves
>   the performance
>
> - There are a couple of mystery exceptions I still haven't figured out.
>   For example WordEntry.mapRefs(int [] mappings) is often throwing an
>   ArrayIndexOutOfBounds exception ... this is trouble I believe and
>   leads to:
>
> - The fact that in the IndexManager.getIndex(IPath, boolean, boolean)
>   we (in this particular project) are never retrieving an index which
>   has a known state (it is always UNKNOWN_STATE) which means we go and
>   rebuild the index for everything.
>
> I think that with Chris Wiebe's "Open Type" contributed plugin and a
> moderately large project you should be able to see this kind of churn.
> I'm looking into it, but thought that the indexer guy (Hi Bogdan!) might
> have some thoughts about what is going on.
>
> Also ... it takes a fair amount of time to add in 15,000+ entries using
> the IndexAllProject.execute() -> IndexManager.addSource() method, several
> seconds at least in my environment.
>
> Just passing along some observations.  Unfortunately I can't pass along the
> source base, but I will certainly try and pull out relevant information as
> requested/required and dig into this while I can.
>
> Thanks,
>  Thomas
> _______________________________________________
> cdt-dev mailing list
> cdt-dev@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/cdt-dev