Bug 329288 - Fetching parameter names literally hangs on a class with a lot of methods
Summary: Fetching parameter names literally hangs on a class with a lot of methods
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.6.1   Edit
Hardware: All All
: P3 major (vote)
Target Milestone: 3.6.2   Edit
Assignee: Ayushman Jain CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
: 332311 346012 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-11-02 14:44 EDT by Mike CLA
Modified: 2011-07-21 22:27 EDT (History)
10 users (show)

See Also:
daniel_megert: pmc_approved+
Olivier_Thomann: review+


Attachments
CodeAssit Advanced Settings page (106.31 KB, image/jpeg)
2010-11-02 14:50 EDT, Mike CLA
no flags Details
CodeAssist main mage (98.38 KB, image/jpeg)
2010-11-02 14:50 EDT, Mike CLA
no flags Details
Thread dump (30.20 KB, text/plain)
2010-11-02 15:23 EDT, Mike CLA
no flags Details
illustrates how it hangs (300.20 KB, text/plain)
2010-11-02 15:25 EDT, Mike CLA
no flags Details
illustrates how it hangs (300.20 KB, image/jpeg)
2010-11-02 15:27 EDT, Mike CLA
no flags Details
illustrates how it hangs (298.82 KB, image/jpeg)
2010-11-02 15:31 EDT, Mike CLA
no flags Details
Eclipse runtime settings (49.16 KB, image/jpeg)
2010-11-02 15:36 EDT, Mike CLA
no flags Details
thread dump in new fresh workspace (35.74 KB, text/plain)
2010-11-02 15:53 EDT, Mike CLA
no flags Details
test jar with the class with 700 methods (19.41 KB, application/java-archive)
2010-11-04 17:42 EDT, Mike CLA
no flags Details
java doc jar (57.86 KB, application/java-archive)
2010-11-04 17:43 EDT, Mike CLA
no flags Details
proposed fix v1.0 (3.76 KB, patch)
2010-11-09 11:31 EST, Ayushman Jain CLA
no flags Details | Diff
proposed fix v1.0 (4.01 KB, patch)
2010-11-16 14:46 EST, Ayushman Jain CLA
no flags Details | Diff
Proposed fix + regression tests (6.24 KB, patch)
2010-11-18 10:20 EST, Olivier Thomann CLA
no flags Details | Diff
fix for 3.6.2 (6.18 KB, patch)
2011-01-17 06:44 EST, Ayushman Jain CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mike CLA 2010-11-02 14:44:32 EDT
Build Identifier: Build id: 20100917-0705


Typing . shows a list of methods, but attempts to scroll it pretty much hang Eclipse. Similarly, typing 3 letters after the . causing Eclipse to hang. I have Code Assist wit all default settings. 
Eclipse Java EE IDE for Web Developers.

Version: Helios Service Release 1
Build id: 20100917-0705

Please, describe how Code Assist works? Where it's looking for packages and etc? I need a guideline to track down the issue. Similar installation of Ganymed doesn't have this issue...

Reproducible: Always

Steps to Reproduce:
Exactly as it described ealier
Comment 1 Mike CLA 2010-11-02 14:46:04 EDT
Not sure how to track this down - what affects the performance of Code Assist? What configuration settings to check and etc?
Comment 2 Mike CLA 2010-11-02 14:50:11 EDT
Created attachment 182229 [details]
CodeAssit Advanced Settings page
Comment 3 Mike CLA 2010-11-02 14:50:40 EDT
Created attachment 182230 [details]
CodeAssist main mage
Comment 4 Remy Suen CLA 2010-11-02 14:53:44 EDT
Try turning off your JAX-WS and/or your JPA proposals. See bug 317979.
Comment 5 Mike CLA 2010-11-02 14:58:32 EDT
(In reply to comment #4)
> Try turning off your JAX-WS and/or your JPA proposals. See bug 317979.
I've seen it - it didn't help. Leaving Java Proposals only causes the same issue.
Comment 6 Remy Suen CLA 2010-11-02 15:02:07 EDT
Get a thread dump when Eclipse is hung.
http://wiki.eclipse.org/How_to_report_a_deadlock

Also, please try to reproduce the problem in a new workspace.
Comment 7 Mike CLA 2010-11-02 15:23:35 EDT
Created attachment 182237 [details]
Thread dump

Here is thread dump from jvisualvm
Comment 8 Mike CLA 2010-11-02 15:25:40 EDT
Created attachment 182238 [details]
illustrates how it hangs
Comment 9 Mike CLA 2010-11-02 15:27:03 EDT
Created attachment 182239 [details]
illustrates how it hangs
Comment 10 Mike CLA 2010-11-02 15:31:56 EDT
Created attachment 182241 [details]
illustrates how it hangs
Comment 11 Mike CLA 2010-11-02 15:36:24 EDT
Created attachment 182242 [details]
Eclipse runtime settings
Comment 12 Mike CLA 2010-11-02 15:53:28 EDT
Created attachment 182244 [details]
thread dump in new fresh workspace
Comment 13 Dani Megert CLA 2010-11-03 03:52:47 EDT
The problem comes from fetching the parameter names from Javadoc. This can be very slow if
- the Javadoc is fetched from the Web via slow connection (not local)
- the attached Javadoc file is very big

It would be great if you could attach the JAR and its Javadoc to this bug, so that we can take a closer look.

In the meanwhile you can either
- make sure the Javadoc is local if it isn't already
- attach source instead of Javadoc (best solution)
- choose a smaller timeout in Java > Content Assist > Advanced
Comment 14 Dani Megert CLA 2010-11-03 03:53:33 EDT
> - choose a smaller timeout in Java > Content Assist > Advanced
Java > Editor > Content Assist > Advanced
Comment 15 Olivier Thomann CLA 2010-11-03 08:35:34 EDT
Could you please provide a test case that shows the performance issue?
Comment 16 Mike CLA 2010-11-03 10:46:25 EDT
(In reply to comment #13)
> The problem comes from fetching the parameter names from Javadoc. This can be
> very slow if
> - the Javadoc is fetched from the Web via slow connection (not local)
> - the attached Javadoc file is very big
> 
> It would be great if you could attach the JAR and its Javadoc to this bug, so
> that we can take a closer look.
> 
> In the meanwhile you can either
> - make sure the Javadoc is local if it isn't already
> - attach source instead of Javadoc (best solution)
> - choose a smaller timeout in Java > Content Assist > Advanced

- Java doc is, definitely, local...
- not sure, what do you mean by "attach" - it hangs on the classes from the libraries that I import (they're on a build path). The Java docs are also there - it's jarred into .jar file. Unfortunately, I'm not aware how Code Assist does the job, I'd imagine it does a reflection...
- changing timeout does not make any difference...
Comment 17 Dani Megert CLA 2010-11-03 11:14:08 EDT
> - not sure, what do you mean by "attach" - it hangs on the classes from the
> libraries that I import (they're on a build path). The Java docs are also >there
See the properties of your JAR. There you see the attached source and the Javadoc (location).

> - changing timeout does not make any difference...
Strange. Even if you set it to 0?

Can you attach the JAR and its Javadoc?
Comment 18 Mike CLA 2010-11-03 17:19:56 EDT
(In reply to comment #17)
> > - not sure, what do you mean by "attach" - it hangs on the classes from the
> > libraries that I import (they're on a build path). The Java docs are also >there
> See the properties of your JAR. There you see the attached source and the
> Javadoc (location).
> 
> > - changing timeout does not make any difference...
> Strange. Even if you set it to 0?
> 
> Can you attach the JAR and its Javadoc?
Well, I was not entirely accurate.Decreasing timeout does speed insignificantly the method list building. The dialog with the list of methods pops-up a bit faster. But, first it's still can go, sometimes for 1 min or more... Second, my biggest issue is a scrolling and code assist automatic kick-off while typing - it's practically non-responsive on scrolling and, even worse, it's literally hangs on context search  for typed letter(s)....
Comment 19 Mike CLA 2010-11-03 17:34:44 EDT
(In reply to comment #15)
> Could you please provide a test case that shows the performance issue?
Unfortunately, I cannot provide our jar file with the class causing Helios to hang on code assist (sorry, company policy)...
But, I can try to simulate the issue. This class has 700 public methods. I believe, that is the problem - too many methods for Helios's Code Assist to handle. People, in my company saying that only Helios SR1 has this issue. Previous release of Helios work fine for the same classes...
Comment 20 Mike CLA 2010-11-03 18:11:51 EDT
(In reply to comment #19)
> (In reply to comment #15)
> > Could you please provide a test case that shows the performance issue?
> Unfortunately, I cannot provide our jar file with the class causing Helios to
> hang on code assist (sorry, company policy)...
> But, I can try to simulate the issue. This class has 700 public methods. I
> believe, that is the problem - too many methods for Helios's Code Assist to
> handle. People, in my company saying that only Helios SR1 has this issue.
> Previous release of Helios work fine for the same classes...
I take the last statement back - I checked Helios Release Build id: 20100617-1415 has the same issue...
Comment 21 Mike CLA 2010-11-03 18:38:27 EDT
(In reply to comment #20)
> (In reply to comment #19)
> > (In reply to comment #15)
> > > Could you please provide a test case that shows the performance issue?
> > Unfortunately, I cannot provide our jar file with the class causing Helios to
> > hang on code assist (sorry, company policy)...
> > But, I can try to simulate the issue. This class has 700 public methods. I
> > believe, that is the problem - too many methods for Helios's Code Assist to
> > handle. People, in my company saying that only Helios SR1 has this issue.
> > Previous release of Helios work fine for the same classes...
> I take the last statement back - I checked Helios Release Build id:
> 20100617-1415 has the same issue...

Finally, I found latest eclipse release that doesn't have this issue: Eclipse Java EE IDE for Web Developers.

Build id: 20100218-1602
Comment 22 Dani Megert CLA 2010-11-04 02:58:29 EDT
Are you using the Android tooling?
Comment 23 Olivier Thomann CLA 2010-11-04 08:12:25 EDT
(In reply to comment #21)
> Finally, I found latest eclipse release that doesn't have this issue: Eclipse
> Java EE IDE for Web Developers.
> Build id: 20100218-1602
Could you please provide the version of the bundle org.eclipse.jdt.core?
Thanks.
Comment 24 Mike CLA 2010-11-04 10:10:51 EDT
(In reply to comment #22)
> Are you using the Android tooling?
No
Comment 25 Mike CLA 2010-11-04 10:28:41 EDT
(In reply to comment #23)
> (In reply to comment #21)
> > Finally, I found latest eclipse release that doesn't have this issue: Eclipse
> > Java EE IDE for Web Developers.
> > Build id: 20100218-1602
> Could you please provide the version of the bundle org.eclipse.jdt.core?
> Thanks.

Helios (doesn't work): 3.6.1v_A68_R36x
Galileo (works): 3.5.2v_981_R35x
Comment 26 Dani Megert CLA 2010-11-04 12:40:08 EDT
Might be related to bug 325829.
Comment 27 Mike CLA 2010-11-04 13:17:23 EDT
(In reply to comment #26)
> Might be related to bug 325829.
There is definitely more to it... 
1.My project doesn't have sources attached, only jar itself and jarred javadoc
2. To test and isolate the problem, I've created a test project with my jars containing classes causing the problem with Helios specified in the User library. I have Javadoc jar specified there as well (validated the path via Helios button)
3. Galileo has no code assist problem with the same project at all...
Comment 28 Mike CLA 2010-11-04 17:42:48 EDT
Created attachment 182438 [details]
test jar with the class with 700 methods

Someone was asking for a test case. Please find attached test jar containing the class with 700 methods. In addition, in the next attachment you'll find jarred java docs for this class... When I had test.jar Helios would handle it with no problem... Then I created a user library and attached the  jared java doc. 
And that is where I start getting 2 - 3 min delays on code assist context search, scrolling or "Ctrl+Space" command... Note, it's happening only at beginning of playing with code assist - i guess, that is time when some sort of cache is built... I wasn't able to make it hang for as long as it was in case of my real java doc jar (it is 4.7 Mb), but still it was hanging intolerable 2-3 min on the doc jar that is just 58Kb...
Comment 29 Mike CLA 2010-11-04 17:43:31 EDT
Created attachment 182439 [details]
java doc jar

see previous comments
Comment 30 Mike CLA 2010-11-05 10:26:17 EDT
(In reply to comment #26)
> Might be related to bug 325829.
There is definitely more to it... 
1.My project doesn't have sources attached, only jar itself and jarred javadoc
2. To test and isolate the problem, I've created a test project with my jars containing classes causing the problem with Helios specified in the User library. I have Javadoc jar specified there as well (validated the path via Helios button)
3. Galileo has no code assist problem with the same project at all...
Comment 31 Ayushman Jain CLA 2010-11-05 11:51:02 EDT
I could reproduce the problem using the attached jar by just adding it into my project (i didnt need to create a user library with the jar). It doesnt cause a minute long hang, but yes it does hold up the UI for 4-5 seconds only when we try to scroll through the list. This is normal in Galileo. I see the following culprit here:

The fix for bug 293955 changed the way we calculate ranges i.e. in Galileo we used the String.startsWith() method, but for Helios we changed it to CharOperation.indexOf(). This example shows one case where the CharOperation.indexOf method's algorithm (lines 2126 onwards) takes too much time. We should either change implementation of indexOf, or use a more efficient method.
Comment 32 Ayushman Jain CLA 2010-11-09 11:31:55 EST
Created attachment 182731 [details]
proposed fix v1.0

The problem in the fix for bug 293955 was that to find whether a given word begins with a given prefix, CharOperation.indexOf was used, which tries to find the index of a particular prefix. The original intension there was to find whether the given string starts with a given prefix, starting from a given offset. Thats why String.startsWith was originally used.

With this patch, CharOperation.prefixEquals is used instead, with a new overloaded instance which takes in a starting offset.
I'll add a perf test for this.
Comment 33 Markus Keller CLA 2010-11-09 13:25:23 EST
(In reply to comment #32)
Wouldn't it be better to avoid code duplication and just replace the existing implementation of prefixEquals(char[], char[], boolean) with this:
    return prefixEquals(prefix, name, isCaseSensitive, 0);
Comment 34 Ayushman Jain CLA 2010-11-16 14:46:43 EST
Created attachment 183264 [details]
proposed fix v1.0

(In reply to comment #33)
> (In reply to comment #32)
> Wouldn't it be better to avoid code duplication and just replace the existing
> implementation of prefixEquals(char[], char[], boolean) with this:
>     return prefixEquals(prefix, name, isCaseSensitive, 0);

Done.

This case cannot really be tested through CompletionTests in the org.eclipse.jdt.core.tests.model, since its basically only a slowness/fastness issue, and adding a performance test doesnt seem like a  very good option, because this test involves referencing an external library and adding that library to the classpath may also affect other perf measurements. Also, the perf tests rely on a full eclipse 3.0 source, and even has a filter to make sure we only deal with eclipse projects. So the best way to test this fix is by naked eye. Try pressing CTRL-SPACE where indicated in the following, and then scroll through the list of suggestions. There shouldnt be any delay in scrolling.

import testgen.Test;
public class Bug {
   public void foo() {
      Test t = new Test();
      t.met[CTRL-SPACE]
   }
}

(make sure the attached test jar is on the classpath and its javadoc location points to the attached javadoc).


Olivier, please review. Thanks!
Comment 35 Olivier Thomann CLA 2010-11-18 10:20:48 EST
Created attachment 183385 [details]
Proposed fix + regression tests

Same patch with a new test in CharOperationTests.
Comment 36 Olivier Thomann CLA 2010-11-18 10:21:15 EST
Performance improvement is impressive.
+1.
Comment 37 Ayushman Jain CLA 2010-11-19 14:10:01 EST
Released in HEAD for 3.7M4.

Thanx for the test Olivier!
Comment 38 Srikanth Sankaran CLA 2010-12-07 04:43:25 EST
Verified for 3.7M4 using build id I20101205-2000
Comment 39 Sebastian CLA 2010-12-16 06:13:43 EST
*** Bug 332311 has been marked as a duplicate of this bug. ***
Comment 40 Ayushman Jain CLA 2011-01-15 14:08:00 EST
Dani, with reference to bug 325289 (comments 29 and 30), do you think its ok to backport this fix to 3.6.2? Android isnt fully compatible with 3.7 yet and 3.6 is the only choice for Android developers. Because of the new javadoc package, this might cause frequent problems without this fix.
Comment 41 Markus Keller CLA 2011-01-16 10:23:25 EST
(In reply to comment #40)
> Dani, with reference to bug 325289 (comments 29 and 30), do you think its ok to
> backport this fix to 3.6.2?

The bug reference above is wrong, should be bug 325829.

+1 from my side for backporting to 3.6.2, since this bug is a major regression that has been introduced in 3.6.
Comment 42 Kostya Vasilyev CLA 2011-01-16 10:58:57 EST
(In reply to comment #41)
> (In reply to comment #40)
> > Dani, with reference to bug 325289 (comments 29 and 30), do you think its ok to
> > backport this fix to 3.6.2?
> 
> The bug reference above is wrong, should be bug 325829.
> 
> +1 from my side for backporting to 3.6.2, since this bug is a major regression
> that has been introduced in 3.6.

Wunderbar.

Especially since this affects not only Android, but possibly other Java projects that have classes with a large number of methods.
Comment 43 Ayushman Jain CLA 2011-01-17 03:23:25 EST
Reopening for 3.6.2
Comment 44 Dani Megert CLA 2011-01-17 03:30:12 EST
(In reply to comment #40)
> Dani, with reference to bug 325289 (comments 29 and 30), do you think its ok to
> backport this fix to 3.6.2? Android isnt fully compatible with 3.7 yet and 3.6
> is the only choice for Android developers. Because of the new javadoc package,
> this might cause frequent problems without this fix.

+1 from my side. Please attach a patch for 'R3_6_maintenance' for the required
review.
Comment 45 Ayushman Jain CLA 2011-01-17 06:44:11 EST
Created attachment 186898 [details]
fix for 3.6.2

This is the fix for R3_6_maintenance branch.The new API CharOperation#prefixEquals introduced for 3.7 is moved to org.eclipse.jdt.internal.core.util.Util.prefixEquals(char[], char[], boolean, int) for 3.6 and the corresponding regression test has also been moved to UtilTests.
All tests pass
Comment 46 Olivier Thomann CLA 2011-01-17 09:34:31 EST
+1. Please release for next M-build.
Comment 47 Ayushman Jain CLA 2011-01-17 12:14:46 EST
Released in R3_6_maintenance for 3.6.2
Comment 48 Jay Arthanareeswaran CLA 2011-01-20 04:24:46 EST
Verified for 3.6.2 using build M20110119-0834
Comment 49 Sergey Prigogin CLA 2011-07-21 22:27:46 EDT
*** Bug 346012 has been marked as a duplicate of this bug. ***