Community
Participate
Working Groups
Build 20020624 (GM1) Bringing a large project (11K files, 620 folders) under version control is very slow, and, watching the VM output on verbose GC shows eclipse is trying to allocate increasing blocks of memory (see below). About every ten seconds Eclipse is grabbing 5Mb -> 6Mb (increasing) of memory, which is subsequently collected (along with much more memory not shown). Just a heads up in case there is some problem with building a synch tree, or such like. ... <AF[117]: Allocation Failure. need 5664216 bytes, 2684 ms since last AF> <AF[118]: Allocation Failure. need 5707720 bytes, 10256 ms since last AF> <AF[119]: Allocation Failure. need 5750192 bytes, 10730 ms since last AF> <AF[120]: Allocation Failure. need 5788056 bytes, 9529 ms since last AF> <AF[121]: Allocation Failure. need 5823928 bytes, 9324 ms since last AF> <AF[122]: Allocation Failure. need 5859544 bytes, 9836 ms since last AF> <AF[123]: Allocation Failure. need 5893744 bytes, 8909 ms since last AF> <AF[124]: Allocation Failure. need 5928424 bytes, 9111 ms since last AF> <AF[125]: Allocation Failure. need 5954712 bytes, 7209 ms since last AF> <AF[126]: Allocation Failure. need 5989344 bytes, 9113 ms since last AF> <AF[127]: Allocation Failure. need 6027272 bytes, 10119 ms since last AF> <AF[128]: Allocation Failure. need 6062616 bytes, 10122 ms since last AF> <AF[129]: Allocation Failure. need 6091000 bytes, 7888 ms since last AF> <AF[130]: Allocation Failure. need 6119640 bytes, 8888 ms since last AF> <AF[131]: Allocation Failure. need 6146864 bytes, 7714 ms since last AF> <AF[132]: Allocation Failure. need 6173464 bytes, 7701 ms since last AF> <AF[133]: Allocation Failure. need 6200784 bytes, 8286 ms since last AF> <AF[134]: Allocation Failure. need 6228112 bytes, 7657 ms since last AF> <AF[135]: Allocation Failure. need 6253448 bytes, 8051 ms since last AF> <AF[136]: Allocation Failure. need 6285536 bytes, 9237 ms since last AF> <AF[137]: Allocation Failure. need 6318144 bytes, 9736 ms since last AF> <AF[138]: Allocation Failure. need 6341000 bytes, 13332 ms since last AF> ...
For bug completeness, which vm/jdk are you using Tim?
post 2.0 investigation
I'm running with IBM JDK 1.3.1
reopening for 2.1 performance work
Profiling memory shows that these allocations occur during combining deltas in core. Transferring to core for further investigation.
I will investigate
The allocations are actually occurring in CVS code. It is asking each resource if it is ignored, which creates StringMatcher objects (which contain strings, vectors, StringBuffers, etc). Sharing a project with 12,000 files created 110MB of garbage, most of which is allocated within isIgnored. Will attach some allocation back traces from OptimizeIt. Michael thinks the following code (in EclipseFolder#members) is the culprit (note that it checks ignore state even if it is not interested in processing it): boolean isManaged = cvsResource.isManaged(); boolean isIgnored = cvsResource.isIgnored(); if ((isManaged && includeManaged)|| (isIgnored && includeIgnored) || ( ! isManaged && ! isIgnored && includeUnmanaged)) { boolean exists = cvsResource.exists(); if ((includeExisting && exists) || (includePhantoms && !exists)) { result.add(cvsResource); } }
Created attachment 3451 [details] allocations of Object[] while sharing a project
Created attachment 3452 [details] Allocations of StringMatcher while sharing a project
Solution: 1. keep a static list of StringMatchers, one for every ignore pattern 2. lazy init the list from Team.getAllTypes() 3. clear list from Team.setAllIgnores() 4. in Team.isIgnored, iterate through the StringMatchers instead of the patterns
Created attachment 3641 [details] Implementation of the solution described in Kevin's comment This does speed up the first stage. The problem remains with the part where we combine deltas. I am investigating and will attach the exact breakup.
We should do the same fix for the default cvs ignore list (see Michael V).
The fix from Boris has been applied. It turns out that CVS doesn't even use the API from Team. The reason is that the API takes IFile but CVS needs to check IResource. We've updated the API (any objections Kevin?) and modified the CVS code to use it and cache it's internal ignores as StringMatchers as well.
Hmm, since one would want to ignore folders, taking IFile seems like it was bogus. >>We've updated the API (any objections Kevin?) So you've widened the type of the field from being IFile to IResource? I guess this can't break anyone.
I re-profiled it with the last code (i.e., with StringMatchers fixed for core and cvs). While this does speed up the adding stage, it does not affect the original behavior mentioned by Tim, that is grabbing progressively bigger memory chunks. As I pointed out a few comments above: the memory gets allocated in AbstractDataTreeNode.assembleWith() during the cvs sync info update phase.
Created attachment 3851 [details] Allocation backtrace while updating syncinfo
Reassigning to core, for comment on assembleWith().
Stale bug. Closing.
