Bug 239890 - RepositoryLister.synchronizeArtifactRepository is slow in large installs
Summary: RepositoryLister.synchronizeArtifactRepository is slow in large installs
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.5 M1   Edit
Assignee: P2 Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks: 238312
  Show dependency tree
 
Reported: 2008-07-07 17:41 EDT by Simon Kaegi CLA
Modified: 2008-08-08 16:20 EDT (History)
0 users

See Also:
john.arthorne: review+


Attachments
patch (5.31 KB, patch)
2008-07-09 17:52 EDT, Simon Kaegi CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Kaegi CLA 2008-07-07 17:41:45 EDT
When checking for changes synchronizeArtifactRepository ends up calling ...

artifactRepository.getArtifactDescriptors(key);

This is an O(n) call and when done once for each individual artifact being scanned is O(n^2). We should store a hash index of the artifactrepository keys to allow us to do the lookup checks in constant time.

See:
RepositoryLister.synchronizeArtifactRepository (line 307)
Comment 1 Simon Kaegi CLA 2008-07-08 23:33:17 EDT
After digging around here I noticed that we don't have an efficient way to iterate over the artifact descriptors in an artifact repo. The current iteration approach of getting the artifact keys and then getting the related artifactdescriptors using the key is unfortunately O(n^2) because the getArtifactDescriptor(IArtifactKey key) call does a linear search of the contained descriptors.

To fix this bug it looks like we need to consider allowing proper iteration or possibly query of an artifact repository.
Comment 2 Simon Kaegi CLA 2008-07-09 13:09:39 EDT
Also see bug 238669 as if we're removing watched bundles this is going to get really slow.
Comment 3 Simon Kaegi CLA 2008-07-09 17:52:26 EDT
Created attachment 107018 [details]
patch

This patch adds an ArtifactKey index to the SimpleArtifactRepository to allow us to retrieve ArtifactDescriptors in constant time.
Comment 4 John Arthorne CLA 2008-07-10 10:00:43 EDT
Can you give some timing results on this patch so we can decide if it is worth putting in 3.4.1?
Comment 5 Simon Kaegi CLA 2008-07-14 17:45:50 EDT
I've put timing results up in bug 238312

I've committed to the 3.5 stream to allow further testing.
Comment 6 Simon Kaegi CLA 2008-08-08 01:45:35 EDT
Marking fixed along with bug 238312