Bug 33156 - commiting many new resources takes a long time...
Summary: commiting many new resources takes a long time...
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Team (show other bugs)
Version: 2.1   Edit
Hardware: PC Windows 2000
: P2 normal (vote)
Target Milestone: 2.1 RC2   Edit
Assignee: Kevin McGuire CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
: 35242 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-02-25 17:17 EST by Jeff McAffer CLA
Modified: 2003-03-19 08:59 EST (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jeff McAffer CLA 2003-02-25 17:17:41 EST
I created a new project with 6500 resources in an empty workspace.  I then 
committed the new project to a CVS repo.  Processing seemed to go in three 
steps:

- 2 minutes where it seemed to create the directory structure
- 2 minutes where it was pushing about 65M of content
- 5 minutes of updating CVS sync information

(Note: all times are very approximate but show rough scale)

The last time is the most surprising since the CPU was at 100% (i.e., we did 
not seem to be hitting the server) and the CVS info was brand new so there 
should not have been too much comparing etc going on.
Comment 1 Jean-Michel Lemieux CLA 2003-02-25 20:33:14 EST
I too have noticed that the saving of sync information is taking much longer
than you would expect. If 65MB was transfered over the wire in approx. 2 minutes
than there is no reason that saving the sync info to disk would take any longer.

IMHO, this is pretty important to investigate for RC2.
Comment 2 Kevin McGuire CLA 2003-02-25 23:07:39 EST
Gee, Jeff gets on other people's cases for not marking performance 
bugs "performance" :)
Comment 3 Jean-Michel Lemieux CLA 2003-02-26 10:13:27 EST
Pretty sure this is related to Bug 28343. Which will be fixed by the UI team in RC2.

Basically the decorator was over-updating in the wrong thread. Tod will be
fixing this. As a result, all CVS operations take a whil to complete. The reason
"CVS saving sync info" is shown is because this is when the cvs state change
events are broadcasted and the decorator is a listener.
Comment 4 Jeff McAffer CLA 2003-02-26 11:15:52 EST
Note that CVS decorators were NOT on during any of this work.  This is a fresh 
biuld, fresh workspace and the decorators are off by default.

More information/additional scenario...

After the initial commit of 6500 resources we deleted all in our workspace and 
got new ones with slightly different content.  Using the same technique as 
before, we shared them with the same repo.  That initial sharing took 5 
minutes.  Progress was reported well but the overall time seemed long.

Once we got the sync view up, everything was a conflict (this is expected 
because we had deleted everything locally including the sync info).  We 
selected all and did "override and commit".  The progress dialog came up 
saying "Operation in progress" and there was no other feedback for over 7 
minutes.  I then left for about 45 minutes.  When I returned the operation had 
completed (so I don't know how long the operation really took).  During the 
initial 7 minute period, the CPU was hovering around 2-6% with most of that 
time going to Eclipse.

Side note:  The exact scenario we are doing is to make a copy of several 
versions of the platfrom from one repo to another.  Here are the steps:

- grab all the projects related to a particular version of the platform (M5 RC1 
in our case)
- disconnect all
- shutdown eclipse
- in the filesystem, move all the projects under the dir structure 
org.eclipse.equinox/plugins in the workspace dir.  
- delete the .metadata dir
- restart Eclipse and make a project called org.eclipse.equinox
- follow the steps outlined for sync and commit etc.
Comment 5 Jean-Michel Lemieux CLA 2003-02-26 14:45:42 EST
some profile results that point to some bad performance problems:

- on initial commit 28% of time is spent updating the CVS console that isn't
even visible or created yet. Worse yet, on initial add from the sync view
console updating is taking 60% is operation.

- on initial share isIgnore() is taking over 40% of operation time and I suspect
a large amount of memory for operation scoped vars. I don't even think this test
is important on share.

Haven't had a chance to test the override and commit yet.
Comment 6 Jean-Michel Lemieux CLA 2003-02-27 17:06:35 EST
Kevin wanted to take a crack at this one. 
Comment 7 Kevin McGuire CLA 2003-03-05 19:18:18 EST
Buffer console and don't do async unless the console is open.  Also ensure 
only one async at a time.  Document no longer buffers since console now does 
this.

Policy.java
  new revision: 1.6; previous revision: 1.5
ConsoleDocument.java
  new revision: 1.5; previous revision: 1.4
Console.java
  new revision: 1.21; previous revision: 1.20

Still to fix:
- remove preference for "auto open console"
Comment 8 Kevin McGuire CLA 2003-03-06 17:14:47 EST
cvs ci -m "33156
	- removed "auto open console" pref
	- console font now updates when pref 

ICVSUIConstants.java
  new revision: 1.49; previous revision: 1.48
Console.java
  new revision: 1.24; previous revision: 1.23
messages.properties
  new revision: 1.248; previous revision: 1.247


There should be a seperate bug report for isIgnored().
Comment 9 Jean-Michel Lemieux CLA 2003-03-19 08:59:26 EST
*** Bug 35242 has been marked as a duplicate of this bug. ***