Summary: | [CVS Core] Putting resources under version control creates lots of garbage | ||
---|---|---|---|
Product: | [Eclipse Project] Platform | Reporter: | Tim Ellison <t.p.ellison> |
Component: | Resources | Assignee: | John Arthorne <john.arthorne> |
Status: | RESOLVED INVALID | QA Contact: | |
Severity: | normal | ||
Priority: | P2 | CC: | bshingar |
Version: | 2.0 | Keywords: | performance |
Target Milestone: | --- | ||
Hardware: | PC | ||
OS: | Windows 2000 | ||
Whiteboard: | |||
Attachments: |
Description
Tim Ellison
2002-06-25 13:05:33 EDT
For bug completeness, which vm/jdk are you using Tim? post 2.0 investigation I'm running with IBM JDK 1.3.1 reopening for 2.1 performance work Profiling memory shows that these allocations occur during combining deltas in core. Transferring to core for further investigation. I will investigate The allocations are actually occurring in CVS code. It is asking each resource if it is ignored, which creates StringMatcher objects (which contain strings, vectors, StringBuffers, etc). Sharing a project with 12,000 files created 110MB of garbage, most of which is allocated within isIgnored. Will attach some allocation back traces from OptimizeIt. Michael thinks the following code (in EclipseFolder#members) is the culprit (note that it checks ignore state even if it is not interested in processing it): boolean isManaged = cvsResource.isManaged(); boolean isIgnored = cvsResource.isIgnored(); if ((isManaged && includeManaged)|| (isIgnored && includeIgnored) || ( ! isManaged && ! isIgnored && includeUnmanaged)) { boolean exists = cvsResource.exists(); if ((includeExisting && exists) || (includePhantoms && !exists)) { result.add(cvsResource); } } Created attachment 3451 [details]
allocations of Object[] while sharing a project
Created attachment 3452 [details]
Allocations of StringMatcher while sharing a project
Solution: 1. keep a static list of StringMatchers, one for every ignore pattern 2. lazy init the list from Team.getAllTypes() 3. clear list from Team.setAllIgnores() 4. in Team.isIgnored, iterate through the StringMatchers instead of the patterns Created attachment 3641 [details]
Implementation of the solution described in Kevin's comment
This does speed up the first stage. The problem remains with the part where we
combine deltas. I am investigating and will attach the exact breakup.
We should do the same fix for the default cvs ignore list (see Michael V). The fix from Boris has been applied. It turns out that CVS doesn't even use the API from Team. The reason is that the API takes IFile but CVS needs to check IResource. We've updated the API (any objections Kevin?) and modified the CVS code to use it and cache it's internal ignores as StringMatchers as well. Hmm, since one would want to ignore folders, taking IFile seems like it was
bogus.
>>We've updated the API (any objections Kevin?)
So you've widened the type of the field from being IFile to IResource? I guess
this can't break anyone.
I re-profiled it with the last code (i.e., with StringMatchers fixed for core and cvs). While this does speed up the adding stage, it does not affect the original behavior mentioned by Tim, that is grabbing progressively bigger memory chunks. As I pointed out a few comments above: the memory gets allocated in AbstractDataTreeNode.assembleWith() during the cvs sync info update phase. Created attachment 3851 [details]
Allocation backtrace while updating syncinfo
Reassigning to core, for comment on assembleWith(). Stale bug. Closing. |