Bug 65626 - OutOfMemory error when compiling simple file, plus serious performance degradation
Summary: OutOfMemory error when compiling simple file, plus serious performance degrad...
Status: RESOLVED WORKSFORME
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.0   Edit
Hardware: PC Windows XP
: P2 critical (vote)
Target Milestone: 3.0 RC3   Edit
Assignee: Philipe Mulet CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2004-06-03 16:30 EDT by Victor Costan CLA
Modified: 2004-06-14 15:43 EDT (History)
3 users (show)

See Also:


Attachments
The file causing the trouble. (103.42 KB, text/java)
2004-06-03 17:07 EDT, Victor Costan CLA
no flags Details
Java code formatter settings file (19.04 KB, text/plain)
2004-06-14 11:27 EDT, Victor Costan CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Victor Costan CLA 2004-06-03 16:30:58 EDT
I'm working on a Java file for the MIT Lapis project 
(http://graphics.csail.mit.edu/lapis), and I used the "organize imports" 
feature. The operation took about 2 minutes. Then I tried saving the file, and 
I got a OutOfMemory error in the compiler. Also, I had to close the Overview 
pane when working with the file.

I consider this to be different from the similar bug report that you received 
some time ago because I'm working on a file that's part of a real system, and 
not a synthetic 1000-method class.

I'm using Eclipse 3.0RC1, with the default startup settings (didn't use -Xmx 
or some related option). My computer has a HyperThreaded Pentium4 2.8Ghz, 
WindowsXP SP1a, and 512Mb of RAM (I think the last part is irrelevant, but I 
wanted to be as complete as possible).
Comment 1 Olivier Thomann CLA 2004-06-03 16:58:06 EDT
The default startup settings are 64M of RAM. This is clearly not enough when you
compile a large system.
Comment 2 Victor Costan CLA 2004-06-03 17:05:22 EDT
The performance issues still stands.
Comment 3 Victor Costan CLA 2004-06-03 17:07:04 EDT
Created attachment 11570 [details]
The file causing the trouble.

I don't think the file is too big. Processing it slows down the build and
everything, even with 384Mb of memory enabled.
Comment 4 Philipe Mulet CLA 2004-06-09 04:39:13 EDT
How did you set the build path ?
Comment 5 Victor Costan CLA 2004-06-09 09:58:50 EDT
The file is part of a bigger project - about 5mb of source code - I can attach 
it here, if it helps. As far as I know, Eclipse only loads files that I touch, 
or that are dependent on what I touch... that should be about 110 files, 1.3Mb.

Also, having more memory (I set the limit to 384Mb) on newer builds seems to 
do the trick - the slowdown is less significant, no more exceptions.
Suggestion: you should instruct the user to give Eclipse more memory for 
medium to large projects or, even better, have shortcuts / batch files that do 
that.
Comment 6 Philipe Mulet CLA 2004-06-11 06:11:19 EDT
Is the source for the project available at 
http://graphics.csail.mit.edu/lapis ?

Need to reproduce first.
Comment 7 Laurent Duperval CLA 2004-06-13 10:00:49 EDT
I  would like to add that I've got a similar problem. My project is not nearly
as big as Lapis. however, moving from M8 to RC1 shows serious degradation in
performance. Every time I save a file (even if it only contains 20 lines of
code), Eclipse runs the CPU to 100% for about 15-20 seconds.

When I used to do a Project Clean in M8, it took about 60 seconds to complete
the operation. Now, it takes up to 15 minutes then I have an OutOfMemory error.
I increased memory use to 256M, but to me that seems wrong since it worked very
well in M8, without increasing memory allocation.

Other symptoms I've found is that Project Clean does *way* too much. It goes
through all the HTML files of my Javadoc directory. The Messages view then
reports more than 100K problems in my project. With M8, it only reported about
300. I doubt that I added that many errors myself... :-)

I'm not sure how much of this is similar to what Victor is seeing and whether or
not I should open a new bug report for this.

Currently, RC1 is almost unusable for me. :-(
Comment 8 Philipe Mulet CLA 2004-06-14 05:31:53 EDT
Laurent: pls provide exact steps to reproduce. Likely we would need copy of 
sources involved. 
Comment 9 Philipe Mulet CLA 2004-06-14 06:56:01 EDT
Victor - I cannot reproduce the out of memory. 

I did start Eclipse with VM defaults (no -Xmx arg), using JDK1.4.2_05 on winXP.
I did setup a workspace with one project: Lapis. Configured the project to use 
xerces + jakarta-regexp + junit on its classpath (first 2 libs were part of 
Lapis delivery). I did repeat combinations of organize imports and clean 
builds, memory peaked at ~100MB. Using profiling tool:
1- Startup with Lapis fully build:  14MB retained memory size
2- Organize imports of Browser:     26MB
3- Clean build:                     35MB
4- Clean build:                     32MB
5- Organize imports:                26MB
6- Clean build:                     33MB
Comment 10 Philipe Mulet CLA 2004-06-14 07:41:26 EDT
Interestingly, when organizing imports for entire project at once (with -
Xmx256M), I observed the VM peaking at ~500MB.Huge memory increase. No 
apparent leak at the end. Organize imports looks very memory intensive.
Comment 11 Philipe Mulet CLA 2004-06-14 07:42:00 EDT
CC'ing Dirk for further assessment of organize imports memory consumption.
Comment 12 Victor Costan CLA 2004-06-14 09:25:23 EDT
I will re-test with RC2 today, and provide complete codebase if bug persists.
Comment 13 Philipe Mulet CLA 2004-06-14 09:37:20 EDT
I did try using Lapis downladable sources.
Comment 14 Laurent Duperval CLA 2004-06-14 10:02:00 EDT
Unfortunately, I am not a t liberty to share the source code since I don't own it.

A simple way for me to recreate the problem is to do a Project | Clean. It can
also happen if I do a project refresh I am also using my Eclipse IDE, if that
can have an effect.

Basically, what happens is that my code is recompiled but on top of that,
Eclipse goes through every single HTML javadoc file, to find problems in them.
That's when I get the error. The only way I found to circumvent the problem was
to clean up the my Javadoc output directory. 

I will try RC2 today. I wil also try it first, without MyEclipseIDE and then I
will add MEI do see if that changes anything.

L
Comment 15 Philipe Mulet CLA 2004-06-14 10:33:25 EDT
Laurent - HTML processing isn't part of JDT. May I suggest you instead log a 
defect against the provider of this HTML tool ?
Comment 16 Victor Costan CLA 2004-06-14 11:26:10 EDT
I tested the current codebase under RC2 - default options, I tried to do nasty 
things to it (deleting and reorganizing imports, modifying some interface 
everything depends on), and Eclipse is still responsive.

However, I think the bug is still somewhere there - I tried formatting 
Browser.java, and I got the JVM to crash. I'm using the same exact computer as 
before, with Sun's JDK 1.4.2_04

Will immediatly attach the code formatter preferences, and current codebase 
(it is not the one on the website... this may or may not matter).
Comment 17 Victor Costan CLA 2004-06-14 11:27:23 EDT
Created attachment 12032 [details]
Java code formatter settings file
Comment 18 Victor Costan CLA 2004-06-14 11:52:30 EDT
I cannot attach the codebase. You can find it here, for a limited time.

http://web.mit.edu/costan/www/bug65626/lapis.zip
Comment 19 Martin Aeschlimann CLA 2004-06-14 12:14:12 EDT
Some additional info about organize import:
Organize imports creates an AST for each CU that is to optimized. This is a AST
with bindings but it is released again after the corresponding file is processed
(I just verified this).
Besides that, jdt.ui's all type cache is used to find types for unresolved type
references. 
Comment 20 Philipe Mulet CLA 2004-06-14 12:36:17 EDT
Good to hear that we have improved in RC2, as we took several measures towards 
reducing memory consumption. Now formatting is still a memory issue, but it is 
a different problem (on same testcase); will investigate anyway.

Did you format exactly one file or entire project ?
Comment 21 Victor Costan CLA 2004-06-14 12:49:28 EDT
I only formatted Browser.java. I fired the command from the context menu that 
I get when right-clicking inside the editor window.

I chose to submit this issue on this thread and not as a separate bug because 
I think it may reach the same (or similar) code as the crashing RC1 
incremental compiler did. (hope I'm making sense... I'm still new to this)
Comment 22 Dirk Baeumer CLA 2004-06-14 14:08:23 EDT
This could be related to bug 64002, which is marked as a dup of bug 62662. 
Victor, to verify this, can you please do the following:

- select Browser.java and another small file in the same package
- make sure that Browser.java isn't opened in the editor
- execute format from the Source menu

This should format both files without opening Browser.java in the editor. 
Since Browser.java seems to be large the format could take a while, but 
shouldn't cause an OutOfMemory exception.
Comment 23 Victor Costan CLA 2004-06-14 14:41:30 EDT
Worked, with mixed performance results - first time, it worked immediatly. The 
second time, it took ~ 2-3 seconds, and it worked. I'm getting a "quick diff" 
related error, but I suppose that's not yours.
Comment 24 Philipe Mulet CLA 2004-06-14 15:43:05 EDT
Thanks Victor. I believe thus we can close this defect, as remaining 
formatting issue is a different issue, and logged in bug 62662.