Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [platform-core-dev] [perf] org.eclipse.core.runtime.Path (computeSegments) : is cache possible ?

My feeling is you are looking too far down the stack. If there are 110GB of Path strings being allocated, it likely means something stupid is being done much higher up the call chain and it is exploding outwards to these hotspots at the leaf level. We could possibly optimize Path further, although adding caching of course means a  speed/space tradeoff. But optimizing at the leaf level often just masks a deeper algorithmic problem higher up.  I suggest you dig into the details much higher up the chain. Which builder is taking all the time, what phase of the build, etc. Some common examples of problems that could cause excess path creation:

 - traversals of the resource tree. There is a heavily optimized IResourceProxyVisitor class that will only instantiate Path object lazily when requested, which can make a huge difference. Of course minimizing/avoiding deep traversals altogether is even better.
- Resource changes occurring outside of builds or other batching operations, which triggers excess resource change events
- Build cycles that cause builders to build and then have to throw away and build again (especially if you have a builder that generates Java source code)

John




From:        Martin Kočí <martin.kocicak.koci@xxxxxxxxx>
To:        "Eclipse Platform Core component developers list." <platform-core-dev@xxxxxxxxxxx>,
Date:        11/12/2013 02:52 PM
Subject:        [platform-core-dev] [perf] org.eclipse.core.runtime.Path (computeSegments) : is cache possible ?
Sent by:        platform-core-dev-bounces@xxxxxxxxxxx




Hi,

hier is some background for this performance-problem: http://dev.eclipse.org/mhonarc/lists/platform-core-dev/msg01707.html (topic [perf] AbstractDataTreeNode.simplifyWithParent creates 100 mil instances during one build)

I've elimitated m2e and maven and now I have a 15min build in eclipse.

UseCase is: Build -> Clean -> Clean all projects.

The most allocations comes from  org.eclipse.core.runtime.Path.computeSegments and  org.eclipse.core.runtime.Path instance self : total 110GB (!) was allocated from org.eclipse.core.runtime.Path classes.

My quuestion is: is a cache for Path or segments or both possible ? Was this already discussed ?

A path is "/a/immutable/string":
As a quick proof of concept I've implemeted cache in org.eclipse.core.runtime.Path.computeSegments (key is String path) and for some invocation of new Path() (new method createPath(path)). The build time drops to 8 minutes with this patch = 2x faster.

Thank you for your answers


Martin_______________________________________________
platform-core-dev mailing list
platform-core-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-core-dev


Back to the top