RE: [cdt-dev] Indexing conf call - summary

["Schorn, Markus" <Markus.Schorn@xxxxxxxxxxxxx>]

Hi Jason,
it will be fairly easy, at least possible to isolate the parser + dom,
it is harder up to impossible to do the same for the index + indexer.

I have created https://bugs.eclipse.org/bugs/show_bug.cgi?id=162172
which should give some insight, why indexing needs the resources plugin.

Markus.

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Jason Montojo
Sent: Dienstag, 24. Oktober 2006 00:17
To: CDT General developers list.
Subject: Re: [cdt-dev] Indexing conf call - summary

Hi everyone,

My apologies for the sudden exit during today's call.  I had to run out
to
attend a training session and it overlapped with the call.

Markus:  Bug 151846 (and its dependencies) is about building a JAR for
the
parser.  I believe you are already on the CC list.

As I mentioned during the call, I'm currently trying to find alternative
solutions to building a parser JAR from HEAD.  My original goal was to
isolate the smallest number of source files from the source folder
"org.eclipse.cdt.core/parser" that would allow the construction of an
AST
via an ISourceCodeParser that could be compiled standalone. This was not
possible for a number of reasons.  Here are the major ones:

- DOMException extends CoreException [1]
- IIndex and IIndexFragment reference IPath [1]
- IASTTranslationUnit references ILanguage [2]
- IASTTranslationUnit references IIndex which references ICElement [2]
- Many classes use CCorePlugin for Exception logging [3]

[1] CoreException, IPath is part of the Equinox common JAR.  Even though
this JAR is only about 80k  on disk, at runtime, it consumes roughly 1M
of
RAM (See [4] for details of my testing methodology), even without
parsing
anything.  Our general rule of thumb is to multiply that by 500
concurrent
users.  That works out to a minimum added footprint of 500M.  By
substituting a JAR containing stubs for the classes needed at runtime
(e.g.
IPath.java would only contain "public interface IPath {}"), the memory
footprint was reduced to about 500k.  That's 250M worth of savings for
us.

[2] ILanguage and ICElement are part of the C Model.  At the moment,
they
are not absolutely required to build an AST using any of the existing
ISourceCodeParser implementations.  These interfaces can also be stubbed
out like in [1] to allow the parser classes to function independently at
runtime.  This shouldn't be a problem because the stubs are replacements
for classes that would not normally be present at runtime when running
standalone (i.e. C Model elements are not available unless Eclipse is
running).

[3] Using the Plugin-based logging system requires the Eclipse Platform
runtime JAR.  I haven't started considering workarounds for this issue
yet.

So without further changes to the current architecture, it is possible
to
build a JAR containing the parser classes (which is a subset of the
classes
in org.eclipse.core/parser).  At runtime, this JAR depends on the
Equinox
common JAR, or a hacky set of stubs (See [4] for details).  I verified
using the parser test suite that both alternatives produce functional
parsers.  What I don't like about this approach is that the source files
for the parser JAR cannot be built independently, but I'm not sure if we
can avoid that without major changes to the architecture.  On the other
hand, it's possible to build a JAR without any runtime dependencies on
Eclipse-specific classes.

Thoughts?

Jason Montojo
IBM CDT Team
IBM Toronto Lab
905-413-5228
jmontojo@xxxxxxxxxx

-----------8<--------------------

[4] To measure the memory footprint of the Equinox JAR versus the hacked
JAR of stubs, I used this:

import org.eclipse.core.runtime.CoreException;
import org.eclipse.core.runtime.IAdaptable;
import org.eclipse.core.runtime.PlatformObject;

public class SpinSpinSpin {
      public static void main(String[] args) {
            Foo foo = new Foo();
            foo.getAdapter(Foo2.class);
            while (true);
      }

      static class Foo extends PlatformObject {}
      static class Foo2 implements IAdaptable {
            public Object getAdapter(Class arg0) {
                  return null;
            }

            public int foo() throws Throwable {
                  throw new CoreException(null);
            }
      }
}

I ran SpinSpinSpin first with the Equinox common JAR, let the memory
usuage
settle, then measured the memory consumption in Windows Task Manager.
Then
I did the same thing with a JAR full of empty stubs for ILanguage,
CoreException, IAdaptable, and PlatformObject, and measured the
difference.

_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev