Bug 187187 - Server tools do not scale when there are a large number of projects
Summary: Server tools do not scale when there are a large number of projects
Status: CLOSED FIXED
Alias: None
Product: WTP ServerTools
Classification: WebTools
Component: wst.server (show other bugs)
Version: 1.5.4   Edit
Hardware: PC Windows XP
: P2 major (vote)
Target Milestone: 1.5.5 M155   Edit
Assignee: Tim deBoer CLA
QA Contact: Tim deBoer CLA
URL:
Whiteboard: PMC_approved
Keywords: performance
Depends on:
Blocks:
 
Reported: 2007-05-16 02:47 EDT by Gary Karasiuk CLA
Modified: 2017-10-11 16:14 EDT (History)
1 user (show)

See Also:
david_williams: pmc_approved+
raghunathan.srinivasan: pmc_approved+
naci.dai: pmc_approved+
jlanuti: pmc_approved? (deboer)
neil.hauge: pmc_approved+
jgarms: pmc_approved+

Attachments
Switch ArrayList to a HashSet (868 bytes, patch)
2007-05-16 02:53 EDT, Gary Karasiuk CLA 		no flags Details | Diff
Patch to inline getModules() (2.35 KB, patch)
2007-05-17 14:23 EDT, Tim deBoer CLA 		no flags Details | Diff
Released patch (3.07 KB, patch)
2007-05-22 17:45 EDT, Tim deBoer CLA 		no flags Details | Diff
Show Obsolete (2) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Gary Karasiuk CLA 2007-05-16 02:47:47 EDT 
Build ID: M20070212-1330

Steps To Reproduce:
1.Start with an application with lots of projects. The application I was looking at had 230 projects.
2.Try to create a new server, WTP will "hang" for minutes
3.Try to add a project to an existing server, WTP will "hang" for minutes


More information:
Comment 1 Gary Karasiuk CLA 2007-05-16 02:52:31 EDT 
On my fastest system (AMD Athlon 64 x2 Dual Core 4200+) it took over three minutes to add a project to the server.

Just changing the ArrayList to a HashSet as I've done in the supplied patch, dropped that time to 51 seconds. (51 seconds isn't acceptable either but at least it is a big step in the right direction).
Comment 2 Gary Karasiuk CLA 2007-05-16 02:53:26 EDT 
Created attachment 67354 [details]
Switch ArrayList to a HashSet
Comment 3 Tim deBoer CLA 2007-05-17 14:23:13 EDT 
Created attachment 67700 [details]
Patch to inline getModules()

Improved patch - keeps improvement to getModules(), but inlines the content of getModules() into getModule(IProject) as well so that an additional performance improvement can be made to avoid the list/set completely and exit processing as soon as an appropriate module is found. This results in an additional ~4x performance improvement, critical because in Gary's testcase this method is being called over 230000 times.
Comment 4 Tim deBoer CLA 2007-05-17 14:24:08 EDT 
Raising for PMC approval for WTP 1.5.5.
Comment 5 Gary Karasiuk CLA 2007-05-17 16:11:34 EDT 
I tested the new patch on my system, and the dialog came up in 38 seconds.
Comment 6 Tim deBoer CLA 2007-05-17 16:32:54 EDT 
Unfortunate it's not better, but I'll still take 38s over 51s over 'minutes'. The remaining performance hit does not appear to be in server tools, so I've passed the info over to Chuck to investigate via a different bug.
Comment 7 Naci Dai CLA 2007-05-18 11:07:32 EDT 
This is important to fix.  Patch looks ok
Comment 8 Jess Garms CLA 2007-05-18 16:23:01 EDT 
Approved.
Comment 9 Tim deBoer CLA 2007-05-22 17:45:12 EDT 
Created attachment 68221 [details]
Released patch

In the interest of full disclosure and public humiliation :), I'll admit that further testing found a single character bug in the above patch. On ServerUtil.java, line 60:
   if (project.equals(modules[j].getProject(...
should be:
   if (project.equals(modules[i].getProject(...

Although the patch would still works in many cases (including the ones I originally tested with...), this change avoids regression and actually improves performance by another significant margin.

Attaching the patch as released to 1.5.5, including plugin version bump.
Comment 10 Tim deBoer CLA 2007-05-22 17:47:41 EDT 
Fix released to WTP 1.5.5.

Please see bug 188484 for adopter's request to provide a patch on 1.5.4.
Comment 11 Tim deBoer CLA 2007-08-30 14:23:23 EDT 
Verified, closing.
Comment 12 Eclipse Genie CLA 2017-10-11 16:14:51 EDT 
New Gerrit change created: https://git.eclipse.org/r/108294