Re: [cdt-dev] LLVM Project Blog: A path forward for an LLVM toolchain on

[Sergey Prigogin <eclipse.sprigogin@xxxxxxxxx>]

On Wed, Sep 11, 2013 at 7:34 PM, Nathan Ridge <zeratul976@xxxxxxxxxxx> wrote:

> Clang's parsing is indeed significantly slower than the CDT one, on
> order of 5-6 seconds for a typical source file of few hundred lines.

Are you referring to CDT or clang? I find that clang (or any other
compiler) can often take 30-45 seconds to compile a source file,
especially if the code in it is template-heavy.

The 5-6 second number was for clang-check (no code generation involved) on a file using only the basic template classes like string and vector. 

> Indexing would be particularly sensitive to parsing slowness since it
> would bring us back to the days of full indexer. One possible solution
> to this problem is parsing of many source files in parallel.

If we assume that the amount of time taken to parse/compile a file
or translation unit is proportional to the number of lines of code
in it, then the difference betwen indexing by running the compiler
on every translation unit (the hypothetical clang way) versus
indexing by parsing each file (the current CDT way) is roughly the
ratio of the number of lines in the average translation unit (T) to
the number of lines in the average source file (F).

I'd say typical numbers are T being in the tens of thousands of
lines, and F being hundreds of lines, so we are looking at a
factor of ~100 difference. You'd need a lot of parallelism to
overcome that.

If we assume that Clang's parser is faster than CDT by a factor of 5, we only need cover the remaining ~20 times difference :-).

Think of a large C++ project you worked on recently. How long
did it take to do a full (clean), parallel build of it? (That's
about how long clang-based indexing would take. Not quite as
much since clang wouldn't be doing the optimization and code
generation phases, but on that order of magnitude). How long
did it take for CDT to index it the first time? In my experience,
indexing is much faster - and that's with a non-parallel indexer
written in Java, compared to a highly runtime- and memory-
optimized C++ compiler being run in parallel.

More importantly, even if parallelism brings down the amount
of time required to index all translation units in a project,
it doesn't bring down the amount of time required to index a
single translation unit. That means that the amount of time
between you writing a line of code, and the IDE underlining
errors in it (or semantically coloring it, or allowing you
to perform an 'Open Declaration' or a refactoring on it, or
whatever else) will be the amount of time it takes to
recompile the translation unit - a lot more time than it
does now.

Agree. Our attempt to use clang-check to display errors didn't have much success due to slowness. Some work is being done in the Clang land on incremental parsing. The first parsing is still slow, but subsequent parses don't have to reparse the header files unless they or the include statements changed.

Regards,
Nate

-sergey 

_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev