Bug 545179 - Improve CapabilityIndex performance for arbitrary namespaces
Summary: Improve CapabilityIndex performance for arbitrary namespaces
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 4.11   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 4.12 M1   Edit
Assignee: Sebastian Ratz CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2019-03-07 09:53 EST by Sebastian Ratz CLA
Modified: 2019-03-30 12:15 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastian Ratz CLA 2019-03-07 09:53:29 EST
Bug 531345 added a special handling for the 'osgi.ee' namespace to improve candidate lookup performance.

While this helped a lot, the situation can be much improved in general by expanding the cache for arbitrary namespaces.

Dependencies that do not just look for "java.package" or "osgi.bundle" via the "name" property are more-and-more common in recent versions of Eclipse and suffer from bad performance.

Example:
"osgi.service", "(objectClass=org.osgi.service.event.EventAdmin)"
Comment 1 Eclipse Genie CLA 2019-03-07 09:56:32 EST
New Gerrit change created: https://git.eclipse.org/r/138273
Comment 2 Sebastian Ratz CLA 2019-03-07 10:02:56 EST
I added the 2018-12 p2 repo as test data and also added a new test org.eclipse.equinox.p2.tests.ql.PerformanceTest.testCapabilityQueryPerformanceOsgiService().

2018-12 has a lot more non-trivial dependencies such as "osgi.service".

Test results of org.eclipse.equinox.p2.tests.ql.PerformanceTest:

Before:
MatchFilter took: 119 milliseconds
IteratorFilter took: 65 milliseconds
MatchQuery took: 177 milliseconds

100 * Slicing took: 1524
100 * Indexed Traverse expression took: 402

ExpressionQuery took: 521 milliseconds
MatchQuery took: 134 milliseconds

CapabilityQuery took: 123 milliseconds
PredicateQuery took: 106 milliseconds

10 * Slicing took: 11203
10 * Indexed Traverse expression took: 473

Parse of 50000 expressions took: 113 milliseconds
IndexedExpressionQuery took: 2 milliseconds
MatchQuery took: 134 milliseconds

CapabilityQuery took: 859 milliseconds

1000 * CapabilityQuery for osgi.service took: 6656 milliseconds

IUPropertyQuery took: 190 milliseconds
PredicateQuery took: 182 milliseconds

IndexedExpressionQuery took: 8 milliseconds
MatchQuery took: 495 milliseconds



After:
MatchFilter took: 110 milliseconds
IteratorFilter took: 63 milliseconds
MatchQuery took: 176 milliseconds

100 * Slicing took: 1503
100 * Indexed Traverse expression took: 418

ExpressionQuery took: 548 milliseconds
MatchQuery took: 132 milliseconds

CapabilityQuery took: 116 milliseconds
PredicateQuery took: 108 milliseconds

10 * Slicing took: 11961
10 * Indexed Traverse expression took: 515

Parse of 50000 expressions took: 115 milliseconds
IndexedExpressionQuery took: 0 milliseconds
MatchQuery took: 106 milliseconds

CapabilityQuery took: 9 milliseconds

1000 * CapabilityQuery for osgi.service took: 462 milliseconds

IUPropertyQuery took: 207 milliseconds
PredicateQuery took: 200 milliseconds

IndexedExpressionQuery took: 15 milliseconds
MatchQuery took: 650 milliseconds


The numbers are fluctuating a bit but the new test shows the significant improvement:
1000 * CapabilityQuery for osgi.service took: 6656 milliseconds
vs.
1000 * CapabilityQuery for osgi.service took: 462 milliseconds


This cache is obviously still not optimal, it would be nice if we could somehow index stuff done in an LDAPFilter.

However, the patch should help a lot in current scenarios. As can be seen, the additional namespace cache reduces the result set a lot. namespaceMap for 2018-12 p2 contains:

osgi.implementation 3
java.package 24878
osgi.bundle 5436
osgi.ee 29
org.eclipse.equinox.p2.localization 5692
toolingorg.eclipse.platform.ide 12
osgi.extender 3
org.eclipse.equinox.p2.iu 7877
org.eclipse.rap 1
org.eclipse.update.feature 1121
osgi.identity 2630
osgi.service 33
osgi.fragment 91
toolingorg.eclipse.sdk.ide 12
toolingorg.eclipse.platform.sdk 12
org.eclipse.equinox.p2.flavor 32
toolingorg.eclipse.rcp.sdk.id 8
org.eclipse.equinox.p2.eclipse.type 6551
Comment 4 Mickael Istria CLA 2019-03-30 12:15:24 EDT
Thanks Sebastian!