Bug 233958 - Translation lookup is very slow
Summary: Translation lookup is very slow
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 major (vote)
Target Milestone: 3.4 RC3   Edit
Assignee: P2 Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
: 234180 (view as bug list)
Depends on:
Blocks: 229069 233761
  Show dependency tree
 
Reported: 2008-05-26 10:08 EDT by Martin Oberhuber CLA
Modified: 2008-05-28 20:42 EDT (History)
4 users (show)

See Also:
pascal: review+
susan: review+


Attachments
Thread dump #1 (30.78 KB, text/plain)
2008-05-26 19:55 EDT, Martin Oberhuber CLA
no flags Details
Thread dump #2 (61.99 KB, text/plain)
2008-05-26 19:56 EDT, Martin Oberhuber CLA
no flags Details
Thread dump #3 (31.23 KB, text/plain)
2008-05-26 19:56 EDT, Martin Oberhuber CLA
no flags Details
Thread dump #4 (31.24 KB, text/plain)
2008-05-26 19:57 EDT, Martin Oberhuber CLA
no flags Details
Patch to limit locale lookup to current profile (3.29 KB, patch)
2008-05-27 16:24 EDT, John Arthorne CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Oberhuber CLA 2008-05-26 10:08:44 EDT
Build Id: I20080523-0100 (3.4rc2)

Launch on an existing workspace, 11 extension locations linked in via links/ folder. Choose Help > Software Updates...

Switch on "Installed Software" tab.

"Pending" nodes stays there for at least > 3 minutes. Progress indicator shows "Fetching install information". I'd expect that all information about installed software is local, so there should be no slow network transfer necessary to fill the "installed software" tab.

Marking "Major" since this makes the P2 feature unusable for me.
Comment 1 John Arthorne CLA 2008-05-26 10:40:24 EDT
Is the long delay only on the very first startup, or after launching with -clean? Or, does it happen every session? There is some initial cost on a new configuration to compute and cache metadata for legacy update sites. This cost will either be hit when expanding items in the "Available Software" tab or when first opening the "Installed Software" tab (because it is looking for translated strings which may be stored in any repository).
Comment 2 Pascal Rapicault CLA 2008-05-26 10:57:45 EDT
Martin would it be possible that you be running with -clean?
Comment 3 Martin Oberhuber CLA 2008-05-26 11:03:16 EDT
After finally waiting for "Installed Software" to populate, I installed an
update to CDT. This asked me to restart Eclipse when done (which I did). I the
new session, clicking on "Installed Software" is slow again.

After manual quit & restart (without -clean), it is slow again. After waiting for dialog to populate, then doing close dialog / open dialog, it is slow again -- measured 5:40 minutes.

Does this answer your question? For me it looks like it's slow on every dialog
invocation; if there is any cache, then checking its state is too slow. I'm
definitely not running with -clean, here is how I start Eclipse (on WinXP SP2):

SET EHOME=D:\PDE\eclipse3.4rc2_p2\eclipse
SET ECLIPSE=%EHOME%\eclipse.exe
SET ECLWS=D:/WS/RSE.new/ws
SET ECLOPT= -keyring D:\PDE\prefs\ring\.keyring -eclipse.keyring
D:\PDE\prefs\ring\.secure_ring -vmargs -Xmx512m -XX:MaxPermSize=128m
-Dcom.sun.management.jmxremote
start %ECLIPSE% -data %ECLWS% %ECLOPT%


I don't think it's a good idea contacting repositories for translated Strings,
when already installed stuff is queried. Installed stuff should be
self-contained without any more requests to the outside.
Comment 4 John Arthorne CLA 2008-05-26 13:43:05 EDT
> Installed stuff should be self-contained without any more requests to the outside.

Agreed. We're just not sure if we have time to do any differently for translation lookup in 3.4. Our safe and easy options right now are: 

 1) Don't show translated strings in this tab
 2) Leave it as it is and live with the slowness
 3) Possibly have some restriction that we only consult local repositories for translated string lookup.

Having said that, I haven't been able to reproduce such a slowdown except for on the first startup after install. Can you produce a couple of stack dumps during this 3 minute period so we can see what is going on? It is possible you are encountering some other bug, such as in repository caching, that is making this seem much worse.
Comment 5 Martin Oberhuber CLA 2008-05-26 19:55:43 EDT
Created attachment 102079 [details]
Thread dump #1
Comment 6 Martin Oberhuber CLA 2008-05-26 19:56:26 EDT
Created attachment 102080 [details]
Thread dump #2
Comment 7 Martin Oberhuber CLA 2008-05-26 19:56:47 EDT
Created attachment 102081 [details]
Thread dump #3
Comment 8 Martin Oberhuber CLA 2008-05-26 19:57:22 EDT
Created attachment 102082 [details]
Thread dump #4
Comment 9 John Arthorne CLA 2008-05-27 16:14:49 EDT
The good news is that these stack dumps don't reveal a caching problem. These stack dumps are all during checking of cache consistency. This requires a single network call to each site to find out what last modified date, so we can figure out if our local cache is up to date.
Comment 10 Martin Oberhuber CLA 2008-05-27 16:18:34 EDT
Ok... I've tried to make the dumps during different phases of the computation, approx after 30 seconds, 1:30 minutes, 2 minutes and 4 minutes. Looks like just checking cache consistency takes almost as long as populating the cache.
Comment 11 John Arthorne CLA 2008-05-27 16:23:13 EDT
Now for the bad news... despite various optimizations (bug 233523) and workarounds in the UI (bug 232413), we are still seeing this problem hitting us that network activity occurs in the UI thread. See bug 234180 for the most recent example. Every attempt to handle this in the UI is resulting in the hit from loading the repository to happen somewhere else.

We've decided for 3.4.0 we're going to have to disable the ability to browse repositories in the user's current locale. For installed software, we will limit the locale lookup to the current profile, so no remote repositories are consulted. This reduces us to the same state as UM - locale data is only available after you have installed a feature that contains localization data.
Comment 12 John Arthorne CLA 2008-05-27 16:24:27 EDT
Created attachment 102240 [details]
Patch to limit locale lookup to current profile
Comment 13 John Arthorne CLA 2008-05-27 16:34:02 EDT
*** Bug 234180 has been marked as a duplicate of this bug. ***
Comment 14 John Arthorne CLA 2008-05-27 17:38:41 EDT
Dave, can you also look at this patch if you have time? I have been running with it for a couple of hours, and it definitely eliminates the performance problems. What I'm not sure of is how to test that localization data in the profile will still be found.
Comment 15 Susan McCourt CLA 2008-05-27 18:05:31 EDT
The patch looks okay.  I ran it along with my patch to bug #233761 and verified that the installed list comes up fast, and that I see the proper properties for Eclemmma Java Code Coverage in the installed software list.  (At one time last week this feature was coming up as %featureName in the installed software list).

I give it +1 with conditions. 
- I would like to run this patch along with my patches for the rest of today and some of tomorrow to ensure that it runs fine.
- I'm not sure that looking at the EclEmma properties verifies that localized strings were found, so like John in comment #14, I would want to know from Dave what the test scenario is for verifying that the strings for an externalized IU get installed in the profile.
Comment 16 Pascal Rapicault CLA 2008-05-27 21:40:32 EDT
Patch released.
Comment 17 Dave Stevenson CLA 2008-05-28 20:42:13 EDT
I verified that the translation lookup works correctly with the IUs in the profile. I found a couple of 3.4 bundles from an early translation drop that had correct tolerances. I switched my locale to zh_HK, installed the bundles, and got the translated description strings.