Bug 289439 - Eclipse 3.5 out of memory error
Summary: Eclipse 3.5 out of memory error
Status: RESOLVED WORKSFORME
Alias: None
Product: WTP Source Editing
Classification: WebTools
Component: jst.jsp (show other bugs)
Version: 3.1   Edit
Hardware: PC Windows XP
: P3 critical (vote)
Target Milestone: Future   Edit
Assignee: Nick Sandonato CLA
QA Contact: Nitin Dahyabhai CLA
URL:
Whiteboard:
Keywords: needinfo
Depends on:
Blocks:
 
Reported: 2009-09-15 05:44 EDT by Serhiy CLA
Modified: 2009-10-29 14:08 EDT (History)
5 users (show)

See Also:


Attachments
eclipse.ini file. (320 bytes, application/octet-stream)
2009-09-15 11:25 EDT, Serhiy CLA
no flags Details
Eclipse runtime configuration (586.38 KB, text/plain)
2009-09-15 11:31 EDT, Serhiy CLA
no flags Details
Out Of memory error (10.58 KB, image/x-png)
2009-10-02 05:30 EDT, Serhiy CLA
no flags Details
Eclipse error log (6.53 KB, text/plain)
2009-10-02 05:35 EDT, Serhiy CLA
no flags Details
Heap Size (40.59 KB, image/x-png)
2009-10-08 12:57 EDT, Serhiy CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Serhiy CLA 2009-09-15 05:44:47 EDT
User-Agent:       Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.2; MS-RTC LM 8; .NET CLR 2.0.50727)
Build Identifier: 20090621-0832

When trying to do some actions (searching of references and indexing) of really big projects in Eclipse 3.5 I get out of memory error. My project has few Mbs of jars (3-rd partylibraries) and a lot of source files. 

Reproducible: Always

Steps to Reproduce:
It happens very often when I am trying to find references of some variables in large project. Try to find references (right click->references) to some variable and repeat this for another variables.



I've worked with exactly same project with Eclipse 3.4 and never had this out of memory error!
Comment 1 Olivier Thomann CLA 2009-09-15 09:15:50 EDT
Could you please provide your command line and your eclipse.ini file ?
Comment 2 Serhiy CLA 2009-09-15 11:25:45 EDT
Created attachment 147211 [details]
eclipse.ini file.

Attached eclipse.ini
Comment 3 Serhiy CLA 2009-09-15 11:31:16 EDT
Created attachment 147212 [details]
Eclipse runtime configuration

Attached my eclipse runtime configuration.

I forgot to mention in initial issue that I am using j2ee bundle of 3.5.
Comment 4 Olivier Thomann CLA 2009-09-15 11:36:20 EDT
Are you talking about a java project or some other kinds of project ?
Comment 5 Serhiy CLA 2009-09-17 04:28:02 EDT
Yes, I am talking about java project.
Comment 6 Olivier Thomann CLA 2009-09-18 11:19:49 EDT
We definitely need more information about your use case. Do you get the same problem without j2ee bundles installed?
Comment 7 Serhiy CLA 2009-10-02 05:30:15 EDT
Created attachment 148618 [details]
Out Of memory error
Comment 8 Serhiy CLA 2009-10-02 05:35:36 EDT
Created attachment 148620 [details]
Eclipse error log

Attached screenshot and error log related to out of memory exception.

Error with JSP index occured just after I started Eclipse and clicked "F5" on project directory structure to refresh sources (I use external tool to work with source control system). It seems that this occures when a lot of files were updated in the file system and Eclipse needs to reindex all of them.

This is example of out of memory error which heppned on JSP index. About 3 out of 4 such out of memory errors happen during jsp index and only 1 happens for indexing java files. 

Also similar out of memory error happens with my co-worrkers who use same build of Eclipse.

Since such error happens not only for jsp indeing but also for java indexing I don't think it is something related to j2ee bundle.

I hope that attached error log can help.

Thanks,
Serhiy
Comment 9 Olivier Thomann CLA 2009-10-02 09:30:37 EDT
Can you get OOM error when j2ee bundles are not used at all?
Comment 10 Serhiy CLA 2009-10-06 06:24:51 EDT
My initial info was not corrent. Instead of "has few Mbs of jars" it should be few Gbs of gars. In fact the actual size of project directory is about 4 Gbs. Most of this space is occupied by jars and zip archives.

Most of the time error happens on "JSP" actions but sometimes it happens on serching and indexing java files. I had no chance to test this without j2ee  bundles installed because was very busy at work.

Currently I've installed eclipse 3.5.1 and this problem still exists. Since I cn't easily reproduce this problem for something else than "JSP" I can assume that this is something related to j2ee packages.
Comment 11 Dani Megert CLA 2009-10-06 06:53:05 EDT
OOME happens in  org.eclipse.jst.jsp.*
Comment 12 Nick Sandonato CLA 2009-10-06 17:36:59 EDT
Hi,

Without having your particular workspace at hand, it might be difficult to reproduce your situation. I don't recall offhand anything being changed between 3.4 and 3.5 that would have resulted in some kind of massive increase in memory usage in the stages you provided in the stacktrace.

One thing that stood out to me in the stacktrace though is:
at org.eclipse.jst.jsp.core.taglib.ProjectDescription.updateJAR(ProjectDescription.java:2083)

At this point, the code is collecting the names of all of the entries from that particular JAR that's been updated. One guess that I have is that there may be so many entry names being added to the list that we cause the OOME. But I'm not sure if the size of your JARs is from thousands upon thousands of entries or just a few really big entries in the archive.

In 3.6, we've changed how we update and this list is no longer maintained. If it's not too much trouble, would you mind trying your workspace in our most recent Milestone?

1. Go to http://build.eclipse.org/webtools/committers/wtp-R3.2.0-S/20090924194346/S-3.2.0M2-20090924194346/
2. Download the files
 a. eclipse-SDK-3.6M2-win32.zip
 b. emf-runtime-2.6.0M2.zip
 c. xsd-runtime-2.6.0M2.zip
 d. GEF-SDK-3.6.0M1.zip
 e. wtp-sdk
Comment 13 Nick Sandonato CLA 2009-10-06 17:39:21 EDT
Whoops, hit Commit too soon.

3. Extract all of those archives to the same location.
4. Navigate to that [location]\eclipse and click on Eclipse.exe (If you want to be consistent, make sure your configurations are the same as with your current installation)
Comment 14 Serhiy CLA 2009-10-07 06:58:11 EDT
Unfortunately I can't provide you with a copy of my workspace because it is commercial closed source project.
We really have a huge number of jars. We have about 3 Gbs of them. Not all of them are used and there are a lot of duplicated jars in diffrenet locations but the total size of project when it is built is about 4 Gbs. So I believe this can cause this problem. But I have worked on the same project with older versions of Eclipse without any problems. And my co-vorkers also started to experience this OOM error only after update to 3.5.

I will try 3.6 and update this ticket with status.

Thanks,
Serhiy
Comment 15 David Williams CLA 2009-10-07 07:41:19 EDT
The 750m heap size in eclipse.ini is the low end of normal working values (sort of a minimum-required for typical work). 

Have you tried increasing it? Say to 1g, or 1.5g if your machine can suffer it. 

I myself always use 1g and sometimes 2g!

The typical way to add/change the Xmx directive is to include it at the end of your eclipse invocation command (e.g. in a shortcut, or bat file). That'd be -vmargs -Xmx1g or similar. 

I know part of what you are saying is there may be a memory requirement regression from 3.4 to 3.5 ... and could be ... as far as I know we haven't tested much for that ... but I'm just giving you some advise in case it improves your day to day work. It sounds like you are working with some pretty substantial stuff and you may have been close before and just got pushed over the edge. 

Or, perhaps you are uncovering some (new) memory leak, in which case increasing -Xmx would not help ... may just increase how long it takes to leak up to that amount of memory. 

HTH
Comment 16 Serhiy CLA 2009-10-08 12:57:12 EDT
Created attachment 149140 [details]
Heap Size

(In reply to comment #15)
> The 750m heap size in eclipse.ini is the low end of normal working values (sort
> of a minimum-required for typical work). 
> Have you tried increasing it? Say to 1g, or 1.5g if your machine can suffer it. 
> I myself always use 1g and sometimes 2g!
> The typical way to add/change the Xmx directive is to include it at the end of
> your eclipse invocation command (e.g. in a shortcut, or bat file). That'd be
> -vmargs -Xmx1g or similar. 
> I know part of what you are saying is there may be a memory requirement
> regression from 3.4 to 3.5 ... and could be ... as far as I know we haven't
> tested much for that ... but I'm just giving you some advise in case it
> improves your day to day work. It sounds like you are working with some pretty
> substantial stuff and you may have been close before and just got pushed over
> the edge. 
> Or, perhaps you are uncovering some (new) memory leak, in which case increasing
> -Xmx would not help ... may just increase how long it takes to leak up to that
> amount of memory. 
> HTH

I had no chance to test this with snapshot of Eclipse 3.6.
I've increased my heap size to be more than 1 gig but I still have this problem.
If you look at attached eclipse_oome.png you can see that the eclipse process does not take more than 350 MBs of heap space so even 700 MBs should be enough.
So it looks that the problem is not with heap size. I am going to test this with Eclipse 3.6 in a next couple of days.
Comment 17 Nick Sandonato CLA 2009-10-22 13:12:06 EDT
Hi, just pinging to see if you've had any luck with trying WTP 3.2?
Comment 18 Nitin Dahyabhai CLA 2009-10-29 14:08:10 EDT
Resolving pending feedback from the reporter.