Bug 329313 - [repository] Make more use of batching artifact repo operations
Summary: [repository] Make more use of batching artifact repo operations
Status: CLOSED WONTFIX
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.7   Edit
Hardware: PC Mac OS X - Carbon (unsup.)
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Ian Bull CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: performance
: 335454 (view as bug list)
Depends on: 338987
Blocks:
  Show dependency tree
 
Reported: 2010-11-02 22:27 EDT by Jeff McAffer CLA
Modified: 2019-05-01 08:42 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jeff McAffer CLA 2010-11-02 22:27:13 EDT
Repo2Runnable indirectly calls SimpleArtifactRepo.getArtifacts() to populate the destination repo.  By default this results in a repo save() being done after each artifact is added. That writes out the artifacts.xml and, if the repo is compressed, proceeds to JAR it. 

SimpleArtifactRepo supports then notion of batch operation using #executeBatch but Repo2Runnable (and more generally AbstractApplication) do not use it.  The win is not huge but the effort to use batching is minimal.  

I'm working on a patch for this and will attach soon.
Comment 1 Jeff McAffer CLA 2010-11-03 12:49:47 EDT
I'm broadening this a little as more investigation is done.  

Turns out that the executeBatch mechanism is synchronous.  The net result is that you can't use it during a provisioning operation because the download manager spawns multiple jobs each of which uses things like contains() which is also synchronous.  Deadlock.  Given that there is some sync code we seem to be interested in having the in memory version of the repo be threadsafe. If so, why not allow the executeBatch to work with/for multiple threads?

Related, it also seems that running the engine does not allow for batching.  The collect phase can result in any number of download requests being made and each could be to a different repo.  Perhaps the answer is to have the download manager "start a batch" on each *destination* (if it is IBatchable whatever) and then when it starts and then "stop batch" when it ends?
Comment 2 DJ Houghton CLA 2011-01-27 07:25:09 EST
*** Bug 335454 has been marked as a duplicate of this bug. ***
Comment 3 DJ Houghton CLA 2011-01-27 07:26:20 EST
Another simple use case (covered by bug 335454) is when you call the mirror application to mirror an artifact repository. The repository index file is written for each artifact that you are mirroring.
Comment 4 DJ Houghton CLA 2011-01-27 07:27:04 EST
Marking as 3.7 so we don't lose track of it.
Comment 5 Ian Bull CLA 2011-04-21 14:12:08 EDT
This can't be addressed until bug#338987 is fixed. I'm moving this out.
Comment 6 Thomas Watson CLA 2011-06-08 11:30:20 EDT
Move all 3.8 bugs to Juno.
Comment 7 Eclipse Genie CLA 2019-05-01 08:42:51 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.