Bug 73400 - [implementation] Reuse scanners
Summary: [implementation] Reuse scanners
Status: RESOLVED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Text (show other bugs)
Version: 3.0   Edit
Hardware: PC Windows 2000
: P3 normal (vote)
Target Milestone: 3.3 M4   Edit
Assignee: JDT-Text-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2004-09-07 15:36 EDT by John Arthorne CLA
Modified: 2006-11-11 09:54 EST (History)
5 users (show)

See Also:


Attachments
Allocation backtraces for PublicScanner (49.55 KB, text/html)
2004-09-07 15:37 EDT, John Arthorne CLA
no flags Details
Performance Results (17.12 KB, text/plain)
2006-11-11 09:54 EST, Dani Megert CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description John Arthorne CLA 2004-09-07 15:36:35 EDT
Build: Release 3.0.0

Jerome reported slowness while opening a Java Editor on Parser.java in JDT Core.
Although he reported against 3.1 M1, I did some profiling with the 3.0.0
release. Using a native profiler, I found a lot of time (15%) was spent in the
garbage collector. In a Java profiler, I then took a look at the big sources of
object creations. In total, over 800,000 garbage objects (>60 MB) are created
when opening this editor once. This is mostly Object[] (212,000 instances),
StyleRange (160,000 instances), char[] (117,069), and Region (97,000 instances).

One example I found that might be easy to improve: 272 instances of
jdt.internal.core.util.PublicScanner are created (see first attachment for
backtraces). This in turn creates 51,238 Object[] objects. If a single parser
could somehow be reused, rather than creating many parser instances, it would
save many object creations.

Steps:

1. Start Eclipse.
2. Accept path to workspace
3. Close Welcome view
4. Window > Open Perspective > Java
5. File > Import External Plug-ins and Fragments > Next > 
6. Project with source folders > Next > 
7. Select org.eclipse.jdt.core > Add > Finish
8. Navigate > Open Type... > Type in 'Parser' > Select the class (C) Parser >
Qualifier: org.eclipse.jdt.internal.compiler.parser > OK
9. Close the editor
10. Repeat step 8 (all classes are now loaded)
Comment 1 John Arthorne CLA 2004-09-07 15:37:53 EDT
Created attachment 14434 [details]
Allocation backtraces for PublicScanner
Comment 2 Christof Marti CLA 2004-09-08 06:14:34 EDT
Is this regression consistent across different VMs?

For example the impact of bug 63681, bug 63718 and Folding scales with the size
of the CU.
Comment 3 Dani Megert CLA 2004-09-08 06:20:02 EDT
We also observed that editing feels slower.
Needs to be investigated
Comment 4 John Arthorne CLA 2004-09-08 11:21:32 EDT
I only did the profiling on one build (3.0.0), so I can't comment on any
regression. The profiler measurements were taken with Sun 1.4.2-b28 VM.
Comment 5 John Arthorne CLA 2004-09-08 18:18:08 EDT
I did some profiling of this scenario in relation to a J9 bug I am tracking, but
I thought the results would be interesting here. All results are from opening
and then closing Parser.java in the Java perspective. I have a simple plugin
that starts a timer, opens and closes the editor, then stops the timer. I always
discarded the first ten runs (proportionally similar results, but slower), and
then timed the average of 40 runs. The first four are with out-of-box preference
settings. I am using the most recent IBM JRE (20040908). Here are the results:

Build M20040908: 1.11 seconds
Build 3.1 M1: 1.35 seconds
Build 3.1 M1, code folding and quick diff disabled: 0.85 seconds

So, roughly 20% regression in time to open the editor. Code folding and quick
diff add a 60% hit.

Comment 6 Christof Marti CLA 2004-09-09 03:43:01 EDT
John, just to be sure, M20040908 does not show a regression? It is 3.1M1 that
shows it?
Comment 7 Dani Megert CLA 2004-09-09 13:12:03 EDT
re comments 3: Is was comparing 3.1 builds with R3.0. I do not see such a
regression when using 3.0.1 builds.

A typing areas which is slower in 3.1 builds is for example the Ctrl+Click
feature. I filed a new bug 73589 to investigate this.
Comment 8 John Arthorne CLA 2004-09-09 13:42:45 EDT
My initial profiler information is for the 3.0.0 release build.

The measurements I did later were strictly on M20040908 and 3.1 M1. I did not
produce performance numbers comparing 3.0.0 to any other build. The short
summary is: this scenario creates a lot of garbage in 3.0.0, and is slower in
3.1 M1 than in 3.0.1.
Comment 9 Dani Megert CLA 2004-09-13 10:13:37 EDT
We measured the performance of opening a text and Java editor and can confirm
the regression for the Java editor from 3.0 to 3.1 but not from 3.0 to 3.0.1.
Comment 10 Dani Megert CLA 2004-09-13 10:14:35 EDT
Jerome, would there be less object creations (not counting the scanners ;-) if
we reuse the scanner and call setSource instead of creating a scanner each time?
Comment 11 Jerome Lanneluc CLA 2004-09-13 10:58:02 EDT
Yes, much less objects would be created. However we just discovered that the 
public scanner cannot be reused with different sources (see bug 73762). Do you 
call setSource(...) with different sources (when opening one editor) ? I guess 
not ...
Comment 12 Dani Megert CLA 2004-09-13 11:27:58 EDT
You guess wrong: if we change to reuse the parser we would set different sources
(the source of an individual Java element).
Comment 13 Jerome Lanneluc CLA 2004-09-13 11:44:36 EDT
Trying to guess again :-) All Java elements are in the same CU and for each 
Java element you use setSource(...) passing the source obtained using 
ISourceReference#getSource() ? If this is true, you should set the source of 
the whole CU, and use resetTo(...) with the positions of each Java element 
obtained using ISourceReference#getSourceRange(). Of course the line ends 
would be relative to the whole CU, but that would be even easier to map in the 
Java editor, right ?
Comment 14 Dani Megert CLA 2004-09-14 04:32:20 EDT
We have a working copy not a CU. The code changes while editing. As a result we
need to parse the changed java elements. We would still need to create a new
parser for each java element delta if setSource does not work, right?
Comment 15 Jerome Lanneluc CLA 2004-09-14 05:38:07 EDT
That's correct.
Comment 16 Jerome Lanneluc CLA 2004-09-20 11:08:05 EDT
In today's build, bug 73762 is fixed so the public scanner can be reused with 
different sources.
Comment 17 Dani Megert CLA 2004-09-22 11:39:56 EDT
The regression in the 3.1 stream seems to be gone in latest builds (changes in J
Core I assume), e.g. I200409210010. The open Java editor performance test
results are in the R3.0 range again. Also, bug 73589 seems fixed.

Adapting summary and severity.
Comment 18 Tod Creasey CLA 2005-03-07 11:57:30 EST
Adding my name to the cc list as we are now tracking performance issues more
closely. Please remove the performance keyword if this is not a performance bug.
Comment 19 Mike Wilson CLA 2005-04-25 11:07:00 EDT
Are there still outstanding work items here? From the comments above, it sounds like this could be 
closed.
Comment 20 Dani Megert CLA 2005-04-25 11:18:02 EDT
Nothing has been done yet. What made you think that this got fixed?
Comment 21 Mike Wilson CLA 2005-04-25 11:59:19 EDT
"regression in the 3.1 stream seems to be gone"
Comment 22 Dani Megert CLA 2005-05-26 04:24:52 EDT
Deferred.
Comment 23 Dani Megert CLA 2005-06-16 06:23:14 EDT
Resetting priority to P3. Will be reassessed for the next release.
Comment 24 Dani Megert CLA 2006-11-11 09:48:08 EST
Fixed in HEAD.
Available in builds > N20061111-0010.
Comment 25 Dani Megert CLA 2006-11-11 09:54:31 EST
Created attachment 53687 [details]
Performance Results

The attached performance tests of opening the Java editor shows a performance gain up to 10% depending on the use case. As expected testOpenEditor4 and testOpenEditor6 show no improvement as folding is disabled there. The biggest performance gains comes with enabled folding and especially with large files (testOpenEditor3 and testOpenEditor5).

Of course the fix also improves the performance of updating the folding structure while typing (during reconcile).