Community
Participate
Working Groups
There is a 25% to 40% degradation in M5 of the headless startup perfromance comparing to the Eclipse 3.1 (the exact number seems to depend on a day of the week). It is not clear why we have such a drop in performance. In UI startup test, reversing split runtime back into onle plugin (thus eliminating split packages and reducing number of plugins) did not result in a measurable improvement in performance. Could it be that as the headless startup time is lower, those factors play bigger role in the headless startup time? The difference in time of the startup is not that significant itself (we are speaking about 0.3 sec), but it would be nice to at least understand where it is coming from.
I suspect part of the slowdown is from the hookable adaptor. There needs to be a measurement of the time to when the first bundle is activated to accurately measure and slowdown from the adaptor.
Created attachment 35555 [details] Patch for the org.eclipse.core.runtime and org.eclipse.equinox.registry The patch should improve performance of the headless startup by about 6% by changing the registry offset table to be read and written as an array of bytes (native I/O methods are called) rather than a large numbers of integers. Also the patch removes the dependency of the AdapterManagerListenet on Platform. The Platform is still used by the update configuratior so this change won't have immediate consequence, but once the update configurator dependency is removed, it can produce about 2% improvement in startup by not requiring Platform class to be initialized.
Note: merging back {registry, common, runtime, preferences, jobs, contenttype} into runtime produced about 3% improvement in the headless startup time. Thus, while addition of the split packages in the runtime and 5 extra plugins caused slight slowdown, it is rather minor comparing to the 40% we currently see in the performance tests.
Some more data to chew on. 1. Turns out, the headless startup time significantly depends on the number of plugins installed. (We are speaking here about the warm startup and plugins that are not activated.) The difference is substantial - around 35% decrease in the startup time if I run install with 11 plugins required vs. 108 plugins normally installed. Moreover, additional experiments show that, indeed, adding plugins (that are not used by the headless RCP application) increases startup time. 2. After quite a bit of experimentation, the differences between M3 and M5 startup time falls into the following groups (note that numbers below are about +/-3%): - New non-runtime plugins, possibly differences in scalability on the number of plugins: 7% - Runtime split (5 new plugins, split packages): 5% - Other runtime changes (inc. service registration and tracking): 9% - OSGi changes (out of those, at least 5% belongs to the HookRegistry): 7% Total change in my experiments: 28% (Note that my measurements were slightly different from what happens on the testing machine. In particular, I measures startup/shutdown cycle rather then startup itself and used profiler.) Potential areas for performance improvement: - Why startup time depends so much on the number of plugins installed? - ServiceTracker optimization - Service registration optimization - HookRegistry optimization - the patch above for the registry initialization
HookRegistry is spending a majority of its time classloading the various hooks. I'm not sure how to improve this by much without collapsing our hook implementations. There are currently 10 independant hooks which are loaded by the hook registry currently. The hook registry does call ClassLoader.getResources("hookconfigurators.properties") This is relatively expensive because every jar on boot needs to be searched. If I use reflection on to call ClassLoader.findResources instead this can be improved. But it will not be the big savings we are looking for.
the design also implies that many hooks get called but do nothing. For example, alot of the pre and post hooks are never implemented and return immediately. While somehow these might get JIT'd out eventually, that likely does not happen for a while. Perhaps there is another way we can arrange things. For example, suppose every hook identified the mehtods it actaully wanted called. Then at any given hook point the framework would just get a list of the hooks that registered for that point and call them. while it was not my intention, this might allow for all the hook classes to be collapsed into one. The framework could provide one abstract class that had dummy implementations for all methods but registered for none. Hook implementors then extend and implement only the methods they need and supply a mask that identifies those methods. I've not thought this through completely...
Created attachment 35856 [details] Patch for the org.eclipse.osgi The patch improves the headless startup time by about 4% in my tests. The patch is independent from the registry patch above. This patch improves performance of creation of bundles from the cached information by: 1. Bundle version is saved in the cache as (int, int, int, String) eliminating the need to parse version string on the load 2. The actual file name for the base file is cached along with install location. As long as install location stays the same, the cached file path is used (eliminates need to re-create paths from the install location and relative path to the bundle on load). The dirty flag is going to be set if install location changed. 3. The type of the bundle file is cached (file or directory) eliminating File.isDirectory() calls on the load. (Not sure if the BaseStorageHook is the right place for the FILE_TYPE_* enumeration.) Note that I incremented BaseStorage.BUNDLEDATA_VERSION, but not BaseStorageHook.STORAGE_VERSION. Not sure if both versions need to be incremented. I am not familiar with the OSGi code so please check the proposed modifications and feel free to update the patch.
please pay particular attention to the relative path issues. Previously we had the abiliity to run a preinitialized eclipse of a CD without writing any data at all. This implied that the cached data could just be loaded directly and everything remained relative. Having said that, I'm not sure how well that worked or if we have any test cases for it. Rafael was the man on that at the end of 3.1
I spent some time experimenting with Jeff's idea in comment 6. I found no change in startup performance when I avoid calling hook methods that do nothing. With the added complexity and no performance gain of this solution I do not recommend persuing it further. I will look at Olegs patch, now ...
Created attachment 35910 [details] updated perf patch to org.eclipse.osgi Updated patch. Oleg's original patch was storing three strings for each bundle in the .bundledata - previous install path - full path - relative path This updated patch moves the previous install path string to a shared string in BaseStorage. The relative path is no longer stored in the bundle data. Instead, if the install path has changed from the previous install path then we make the full path relative to the previous install path (which would have been the relative path that got stored) and then make that absolute to the new installpath. I don't think we should make the bundledata dirty if the install path has changed. We did not do this before and I don't want to force a save in the case where eclipse is on a CD. Oleg can run performance tests on this patch? Thanks.
Forgot to mention that the new patch combines the bundle file type (jar, dir etc.) with the reference flag into one bit-masked int "flags".
The registry patch did not make any difference in the tests. It turns out that profiler I've been using (YourKit) is not very accurate. The patch was optimizing the CPU time for the HashtableOfInt.load. Here is the percentage of the headless startup time for this method as reported by different profilers: YourKit 5.7% Quantify 0.1% OptimizeIt 0.4% From the direct measurements, the actual run time of this method is close to the time reported by Quantify. As such, the method doesn't contribute much to the startup time and trying to optimize it was pointless. The changes in the HashtableOfInt should probably be removed at some point as they introduce more complex code without any tangible benefit.
Created attachment 36151 [details] Improves headless startup time by about 10% The patch improves headless startup performance by: - removes JobManager service from being registered by Jobs plugin (creation of this service causes excessive class loading on startup. Service was added to provide connectivity between the org.eclipse.core.runtime and org.eclipse.core.jobs and was later replaced by Job.getJobManager(). At present, this service is not used and was not advertised.) [3% of startup time] - Eliminating activation of Preferences plugin in the headless startup - unless required by the application. The Preferences plugin, while specified as the "LazyStart" was always activated on the runtime startup as it was referred to by the Plug class, had a split package and was in the wrong place in the dependency order, and contained 2 interface description that were registred by the runtime. The patch modifies the Plugin class to hide dependencies on the Preferences; renames internal package to avoid split package problem and modifies dependency order; and moves interfaces ILegacyPreferences and IProductPreferencesService into the common plugin. [4%] - Makes AdapterManager to be lazily filled with the information from the registry rather than fill it in with all the information on the startup [3%] Note that this patch suggest moving public interface IProductPreferencesService into an internal package (API change). The interface was added in the M4 to facilitate the runtime split, and, in the hindside, should not have been made public at this point. The functionality offered by the interface is likely to be affected by the planned changes to the application startup; it is very likely that it would be deprecated or modified in the next release. The patch affects the following plugins: org.eclipse.core.jobs org.eclipse.core.runtime org.eclipse.equinox.common org.eclipse.equinox.preferences
At this point in the release cycle, the fewer the changes the better. There seems to be some refactoring changes here. (package renaming) If they aren't required, then I would recommend submitting a new patch without them. It makes less changes to look at when reviewing the patch.
Also note that there are compile errors in the test suites when the latest patch is loaded.
Created attachment 36154 [details] Improves headless startup - Runitme Same patch as the "36151: Improves headless startup time by about 10%" but includes modication to the Junits test (PreferenceForwarder moved to a different package). As for the package renaming, it eliminates split package org.eclipse.core.internal.preferences and I would prefer to keep it (in fact, at some later date I would like to rename org.eclipse.core.internal.runtime packages to further reduce the number of split packages and, hence, the startup time :-)).
I un-applied the registry part of the very first patch since it had no effect.
Jeff: we need a PMC member to comment on the API change in the patch. If having split packages are a performance hit, are there any other (internal) packages that we can address?
Is removing the IJobManager service registration a breaking API change that we need to get approval for?
Created attachment 37564 [details] Improves headless startup - Runitme Updated version of the runtime activation patch. Affected plugins: org.eclipse.core.jobs org.eclipse.core.runtime org.eclipse.core.tests.runtime org.eclipse.equinox.common org.eclipse.equinox.preferences This patch adds a switch "eclipse.activateRuntimePlugins" which, if set to "false", allows runtime to lazily activate preference plugin. This way preference plugin is normally activated by the runtime (allowing it to register its OSGi sources), but this can be turned off for performance reasons if preference services are not required. The rest of the patch is the same as above. Note that in M6 due to the new "shorted" version of the osgi.bundle list, the Jobs plugin is not activated anymore, so removing JobManager service will actually be good to reduce confusion (it might not be available overwise). The patch provides about 8% improvement in the "actual" headless startup time, but due to the way the JUnits tests work, provides little improvement for the JUnit performance test.
This needs to be reviewed in detail as we are starting to lock things down...
Marking milestone to review patch, its now or never for 3.2 ...
Created attachment 38437 [details] Patch updates runtime The patch is very similar to the previous patch. The main difference is that ILegacyPreferences and IProductPreferenceServices are left in the preferences plugin and specified in the exceptions list of the AutoStart tag of the preferences plugin. The patch also merged with the latest from HEAD. On my computer this patch provides about 8% improvement in the headless startup performance.
Created attachment 38672 [details] Patch updates runtime Same patch as before adapted to the latest changes in CVS. To recap: - The patch cleans up service registration in Jobs plugin (the service is currently registered only after an attempt ot use JobManager) - Adds an option "eclipse.activateRuntimePlugins". The preferences plugin is activated by default, code to activate Jobs plugin is commented for now. If option is set to "false", preferences plugin won't be activated by runtime. [It really is an improvement from what we have now as with the short osgi.bundles line Preferences plugin is activated just by "coincedence"]. So far, Preferences is the only split runtime plugin that registeres services, but approach suggested can easily accomodate other plugins being activated as well. - Makes AdapterManager to be lazily initialized with the information from extension registry. The overall gain of the headless startup performance test from this patch should be 7 to 12%.
Created attachment 38729 [details] Patch updates runtime Similar patch to the above plus: - Simplified code in the AdapterManager - Added eclipse.service.jobs, eclipse.service.pref options to supress crestion of OSGi services by Jobs and Preferences - Checks to see if compatibility bundle is present before registering ILegacyPreferences - Adds new properties to platform.doc.isv Plugins affected: org.eclipse.core.jobs org.eclipse.core.runtime org.eclipse.core.tests.runtime org.eclipse.equinox.common org.eclipse.equinox.preferences org.eclipse.platform.doc.isv
Released the latest patch. Will keep this report open until we resolve: 1). InternalPlatform#startServices - should we be calling #getProduct and only registerin the product pref cust service if we have a product? How expensive is this? 2). The preferences field in Plugin was changed from a PreferenceForwarder instance to a Preferences instance and then all the PreferenceForwarder code was put into an inner class to avoid classloading. We need to determine if this is really necessary.
moving milestone to RC2 for rest of the cleanup.
Aren't we done with that now?
At present it doesn't look like we could do more without major code changes so I am closing this as "Fixed". Summary: The "true" headless startup time on a minimum target increased by about 16% or 0.1 sec. The scalability of Eclipse is not affected as the "per plugin" startup time stays about the same. The increase of the startup time can be attributed to new functionality provided in 3.2. The headless startup time turns out to be very sensitive to class loading. The JUnit performance test adds a number of extra variables and can't be used as an accurate measure of the startup time. ----------------------------------------------------------------------------- Few details that might be useful in future: A. What got slower? Sources of slowdown comparing to 3.1 in % increase of the startup time that we influence (in brackets, I added times for reference as measured on WinXP 2GHz 2Gb): PlatformActivator.startServices - 2% (11 ms) Extra class loading for split runtime Activator's - 4% (28 ms) Registry - 2% (11ms) - mostly increased class loading Split plugis - 2% (15 ms) - split packages, extra I/O, extra data structures OSGi - 4% (24ms) - HookRegistry.initialize() Unknown / measurement error - 2% (13ms) Total: 16% (102 ms). The measurements were done for the time period between entering startup.jar and the time a headless application starts running. The System.nanoTime() was used, the numbers are medians of 30 consecutive runs. The standard deviation was typically about 10% of the result. B. The existing performance test measures something else :-) The existing performance test runs as a JUnit application and introduces number of extra variables unrelated to the performance of our code. For starters, the test is run with "-dev" option that adds about 30% to the time. The test measures the time period from the start of the Eclipse process rather from the time we get into startup.jar introducing another 10 - 20 % difference. The test causes excessive class loading of JUnits and establishes communication with the host process that also skews the "startup" time we get. As a result, the actual startup time we can influence is, at best, half of the time displayed. On another hand, a number of extra variables are added to the test that are beyond our control and effects of which is rather non-trivial to quantify. I am going to create another bug to address the inadequacy of the JUnits headless startup test. C. The "per plugin" time stays the same The startup time for an application can be approximated as a "base time" + "per plugin time" * "number of plugins" The base time, as indicated above, increased by about 16%. The "per plugin" time, however, stayed about the same (about 1ms per plugin on WinXP 2GHz 2Gb). This means that scalability of Eclipse was maintained in 3.2.
Created bug 138927 to address the JUnit issues.
This was a huge amount of work Oleg. Thanks! Bet it felt good to close that one...
