Community
Participate
Working Groups
When a user installs a feature, we add it to the appropriate site in the platform.xml file. When a user installs 2300 features, we do the same. Unfortunately we don't do any caching so we parse this file every time. We should investigate: - caching the platform.xml in memory - batching operations - conversion of the parsing from DOM to SAX Check out the Configuration class in the Eclipse touchpoint.
Created attachment 153934 [details] eclipse.touchpoint JAR for Eclipse 3.4.2 This is a new org.eclipse.equinox.p2.touchpoint.eclipse JAR for Eclipse 3.4.2 which contains code which caches the Configuration file. In a local test it reduced the number of times we parse the platform.xml file from n+1 to 1. (where n is the number of features that we are trying to install via the drop-ins) Note we are still writing the file n+1 times.
Released platform.xml caching change into api_branch.
Is it possible to detect that you are in a state where you do not need to write the configuration file (because a follow on step will cause it to be written)?
This should be possible since there are some lifecycles. However this change was originally developed to be applied in 3.4 where we wanted to be conservative and the writing of the file was not an issue (parsing was). For 3.6, this could be revised.
Created attachment 154326 [details] patch for 3.4.x and 3.5.x
DJ and I investigated this yesterday, and we found at least three other places where we are n^2 or worse where n is the number of features being installed. Here they are in order of decreasing impact on performance: 1) PublisherResult#query. On each query this method copies all the existing results from a map into a list, and then creates an iterator and checks each one. For InstallableUnitQuery we can make this O(1) because we already store a map keyed by installable unit id. This is the most common (only?) query on PublisherResult during publishing (at least it's the only one that showed up under the profiler). 2) LocalMetadataRepository#query. This uses a linear search each time a feature is requested. 3) PublisherInfo#getAdvice(). This does a linear search over all advice in the info object. Each feature and bundle that gets installed contributes at least on BundleShapeAdvice, so this search is O(n) and is called many times during publishing. I have patches ready for 1) and 2) and they make a big improvement using our test data. 3) is a little bit more complicated and may not be worth the optimization.
Created attachment 154591 [details] Patch against 3.6 HEAD This patch contains: - New implementation of DJ's platform configuration caching on read - Fixes for issues 1) and 2) in my list of O(n^2) performance issues during reconcile.
After more investigation, it turns out none of the problems in comment #6 apply to the 3.4.x stream. Those are performance regressions introduced by moving to the publisher in 3.5. I have opened bug 298216 to track those problems in 3.6 (and possibly 3.5.2), and will leave this bug to track problems in the 3.4.x stream.
Created attachment 154808 [details] Fix for platform.xml caching only This is the most up to date version of DJ's patch to cache reading of platform.xml. That is the only fix that seems to have any impact on reconciler performance in the 3.4.x stream.
Created attachment 155529 [details] Updated patch for 3.4.x stream that removes visibility changes
DJ and I have done some further testing on the patch this morning and we're satisfied with it. Released into 3.4.x stream.
