Bug 211578 - [build path] Dialog to extend classpath variable slow
Summary: [build path] Dialog to extend classpath variable slow
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.3.1   Edit
Hardware: PC Windows XP
: P3 major with 3 votes (vote)
Target Milestone: 3.4.1   Edit
Assignee: Dani Megert CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks: 239647
  Show dependency tree
 
Reported: 2007-11-30 05:43 EST by Daniel Hirscher CLA
Modified: 2008-10-05 09:33 EDT (History)
8 users (show)

See Also:
markus.kell.r: review+


Attachments
proposed fix for 3.4.1 (965 bytes, patch)
2008-07-02 12:30 EDT, Benno Baumgartner CLA
no flags Details | Diff
Fix (7.75 KB, patch)
2008-07-09 03:58 EDT, Dani Megert CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Hirscher CLA 2007-11-30 05:43:22 EST
Build ID: M20071023-1652

Steps To Reproduce:
1. get a big 3rd party repository on the hard disk >400 folders
2. define a jdt classpath variable to point to the base folder 
3. configure the classpath on a java project, on the libraries tab click "Add Variable..."
4. double click or select and click "Extend..." the classpath variable defined before
5. -> eclipse hangs some seconds (sometimes >10 sec.) until the dialog opens


More information:
Subsequent uses are somewhat faster but still too slow.
Maybe it is possible to optimize it on subsequent openings.

We are currently editing the .classpath file direct with a text editor. This is faster for us.

Maybe this is related to the windows file system, however I cannot prove this on a e.g. linux machine (I don't have one at work). Our development computers are up-to-date hardware.
Comment 1 Benno Baumgartner CLA 2007-12-10 03:51:06 EST
reproduced in I20071204-1547

1. Create new variable 'foo' pointing to 'c:\'
2. Add variable to project class path
3. Press 'Extend...'
Is:
 It takes several seconds to open dialog, hardisk is on heavy use.
Comment 2 Benno Baumgartner CLA 2007-12-10 05:04:12 EST
The extend dialog only shows folder containing archive files. To determine which folders contain an archive, all folders need to be scanned, which is expensive. 
Options:
- Show all folders
- Show progress
- Use lazy tree
- Ignore this problem
Comment 3 Daniel Hirscher CLA 2007-12-10 05:09:04 EST
How would the lazy tree work? Would it also show all folders?

I would like to see at least progress. This would also force that the calculation is not running in the event dispatcher thread any more.
Comment 4 Geert Buelens CLA 2008-06-05 15:33:40 EDT
We use a classpath variable that points to a huge ClearCase repository.
In our situation, the scenario let Eclipse hang for more then 15 minutes!!
Because it works fine in Eclipse 3.2 (we use currently IBM Rational Application Developer 7.0.0.4), this problem blocks us from using Eclipse 3.3 (or will block us if there is a new RAD version based on it). 
Can the priority to fix this problem be increased?
Comment 5 David Nouls CLA 2008-06-06 02:28:46 EDT
I have the same situation with clearcase and a large repository.

I use Eclipse 3.3 and I had to workaround it by defining a project in which I create links to clearcase directories.

If I use the steps as in this case, I never had the patience to wait for the dialog to show up (waiting longer than 15 minutes). The CPU is not doing anything at all, but on clearcase we noticed that he was trying to walk the entire directory tree in all our vobs. This is in the order of thousands of directories with hunderthousands of files.... I'm sure this can be improved!
Comment 6 Rassaerts Bruno CLA 2008-06-06 03:06:50 EDT
Indeed this problems makes GUI-based classpath modifications imposible on large source repositories. We use a classpath variable to defined on which mapped drive the source repository is linked and it contains ALL projects. Expanding this to select a JAR is nearly impossible, waiting a lot of time to add a single JAR.
Comment 7 Sean Kelly CLA 2008-06-06 03:27:16 EDT
<quote>It takes several seconds to open dialog</quote>

I think you're lucky, it has taken up to ten minutes (sometimes even longer) here for each and every variable extension attempted to return anything.

Right now this feature is unuseable for us until a solution to this bug is found. Our workaround is not to use this 'extend variable' feature at all :(
Comment 8 Benno Baumgartner CLA 2008-06-06 03:45:12 EDT
(In reply to comment #4)
> We use a classpath variable that points to a huge ClearCase repository.
> In our situation, the scenario let Eclipse hang for more then 15 minutes!!
> Because it works fine in Eclipse 3.2 (we use currently IBM Rational Application
> Developer 7.0.0.4), this problem blocks us from using Eclipse 3.3 (or will
> block us if there is a new RAD version based on it). 
> Can the priority to fix this problem be increased?
> 

Hi Geert

The reason why this has not been fixed yet is that comment #0 speaks about several seconds, not minutes. 15 minutes is of course not acceptable and also that it use to work in 3.2 definitely increases the severity of this bug. The reason is the filtered tree selection dialog which needs to be populated beforehand.

But it is too late now to fix this for 3.4, I'll see what I can do for 3.4.1

In the meantime I hope that you can use this workaround
> I use Eclipse 3.3 and I had to workaround it by defining a project in which I
> create links to clearcase directories.

Benno
Comment 9 Benno Baumgartner CLA 2008-06-06 03:48:46 EDT
See org.eclipse.jdt.internal.ui.wizards.buildpaths.JARFileSelectionDialog
Comment 10 Martin Aeschlimann CLA 2008-06-06 05:28:52 EDT
Have you tried 3.4 M7?
While adding the support for archives other than *.zip and *.jar I changed the 'extend variable' dialog to populate in the background.
Comment 11 Martin Aeschlimann CLA 2008-06-06 05:58:24 EDT
Sorry, we just tested it again, and the situation is not much improved even if we use the new selection dialog. We have to improve that in 3.4.1
Comment 12 Benno Baumgartner CLA 2008-07-02 12:30:25 EDT
Created attachment 106328 [details]
proposed fix for 3.4.1

This is a very simple fix for 3.4.1: The dialog opens fast and is fast as long as you do not type into the filter field. As soon as you do type into the filter field you're toast.
Comment 13 Geert Buelens CLA 2008-07-02 14:16:01 EDT
(In reply to comment #12)
> The dialog opens fast and is fast as long as you do not type into the filter
> field. As soon as you do type into the filter field you're toast.

This would most likely fix it for me, because I don't recall using the filter field a lot: I start opening the folders untill I reach the folder with the required library in. Maybe at that last folder a filter could be usefull if the folder contained a lot of libraries.


Comment 14 Benno Baumgartner CLA 2008-07-03 04:36:00 EDT
I don't know how to fix this for real: The FilteredTree calls TreeViewer#refresh in the UI thread and this is a long running operation. The correct fix would be to  initialize the filters cache in a non UI Job first and then do the refresh. But this basically requires a redesign of the FilteredTree. I have a working prototype implemented which can handle long running filter operations to fix bug 149110. If we decide to fix bug 149110 we can reuse this code, if not I opt to remove the filter field from this dialog.

Martin? 
Comment 15 Dani Megert CLA 2008-07-07 07:13:34 EDT
>If we decide to fix bug 149110 we
>can reuse this code, if not I opt to remove the filter field from this dialog.
If you have working code ready we might use that code even if we don't fix bug 149110. Removing the filter will make it quite hard for users to add JARs as they have to manually drill into the folders.
Comment 16 Benno Baumgartner CLA 2008-07-08 03:30:27 EDT
(In reply to comment #15)
> >If we decide to fix bug 149110 we
> >can reuse this code, if not I opt to remove the filter field from this dialog.
> If you have working code ready we might use that code even if we don't fix bug
> 149110. Removing the filter will make it quite hard for users to add JARs as
> they have to manually drill into the folders.

It's a prototype, not product quality code, more work is required. I also think that a lazy tree filter dialog would be of use for many clients and should be provided by platform/ui. 
Comment 17 Dani Megert CLA 2008-07-08 10:55:03 EDT
David and Geert, just to clarify: performance is only a problem for the 'Add Variable...' > 'Extend...' dialog and the performance is acceptable when using 'Add JARs...'?
Comment 18 Dani Megert CLA 2008-07-09 03:58:04 EDT
Created attachment 106921 [details]
Fix
Comment 19 David Nouls CLA 2008-07-09 04:26:14 EDT
(In reply to comment #17)
> David and Geert, just to clarify: performance is only a problem for the 'Add
> Variable...' > 'Extend...' dialog and the performance is acceptable when using
> 'Add JARs...'?

Hi,

I use Eclipse 3.4 ganymede and using "Add JARs ..." is fast. I just have to stay away from variables. What I currently do is create a linked folder in one big java project using a variable that points to ClearCase and then I can manually drill down to the correct project jar files.

But as soon as I use Add Variable... and Extend ... I can fetch a coffee, do a big walk around the lake, ... etc... it helps to relax a bit!

Also maybe important to differentiate: I do not store the project files in clearcase, just the sources. I don't know how the performance will be if I would put everything in clearcase and then use Add JARs. I guess Geert can give an answer to that question.

David
Comment 20 Geert Buelens CLA 2008-07-09 14:04:18 EDT
> David and Geert, just to clarify: performance is only a problem for the 'Add
> Variable...' > 'Extend...' dialog and the performance is acceptable when using
> 'Add JARs...'?

We currently use David's suggestion to have a folder within the project that holds ClearCase symbolic links to the ClearCase JARs that we need.
Because that folder only contains a couple of JARs there is no performance problem.  I didn't test it to link the whole ClearCase root folder within the project and then do 'Add JARs...' if that is what you are suggesting.
I'll give it a try tomorrow and let you know...

Geert
Comment 21 Markus Keller CLA 2008-07-10 10:31:08 EDT
(In reply to comment #18)
> Created an attachment (id=106921) [details]
> Fix

+1. Please add a comment like this to FilteredTreeWithFilter.doCreateRefreshJob():
// This is a copy of the super method, but without auto-expansion.
Comment 22 Dani Megert CLA 2008-07-10 11:07:26 EDT
Fixed in HEAD (>N20080709-2000) and R3_4_maintenance.

Filed bug 240347 to track reworking that code once a better FilteredTree is available.
Comment 23 Dani Megert CLA 2008-08-06 05:45:45 EDT
Verified in build input for M20080808-0800.
Comment 24 Geert Buelens CLA 2008-10-05 09:33:34 EDT
Fix confirmed in Eclipse 3.4.1
Thanks!