Comment 1 John Arthorne 2002-05-29 10:41:27 EDT

Moving to UI, as this error occurred opening an editor.  The other stack trace 
also hints that the editor was in an invalid state.  Henrik, please add as much 
detail as possible:

1) How much RAM does your computer have?
2) How big was the project that you loaded?
3) What type of file did you try to open when the error happened?
4) Is this problem reproducible?
5) Does it happen when you try to open any file, or just one particular file?  
If it's one particular file, it would be nice to attach a copy to the bug 
report.
6) What editor-related preferences, if any, did you change?

Comment 2 Nick Edgar 2002-05-29 11:14:45 EDT

Reassigning to Kai for the NPE.
Note that the NPE occurred only 20 seconds after the OutOfMemoryError, so this 
might have left the editor in an unexpected state.

We will be doing further leak testing on today's build.

Comment 3 Henrik Hjoellund Hansen 2002-05-30 03:14:25 EDT

1) How much RAM does your computer have?
I got the error on more computers, one having 256 MB RAM and two others having 
512 MB Ram

2) How big was the project that you loaded?

The project have 651 source files in 111 folders. This size of the source files 
is 2 MB
3) What type of file did you try to open when the error happened?
I opened a *.java file, it happens with any of the *.java files. I am not sure 
it is directly connected to an open of a file. 
4) Is this problem reproducible?
yes, on more PC with either 256 MB or 512 MB RAM. I have seen it for both the 
F1 and the 28may integration build.
5) Does it happen when you try to open any file, or just one particular file?  
any file
If it's one particular file, it would be nice to attach a copy to the bug 
report.
6) What editor-related preferences, if any, did you change?
I did not change any preferences when changing from M5 to F1.

Comment 4 Claude Knaus 2002-05-31 12:31:20 EDT

fixed NPE >= 20020530.

Passing back to Nick.

Comment 5 Nick Edgar 2002-05-31 19:51:49 EDT

Need to leak check again.

Comment 6 Nick Edgar 2002-06-06 21:13:27 EDT

Closing since the original bug has been fixed.
Should still do further leak checking.

Comment 7 Henrik Hjoellund Hansen 2002-06-14 08:49:23 EDT

I tried the project in F3, and I got the same behaviour. 

log lines*******************

!SESSION jun 14, 2002 14:44:36.472 ---------------------------------------------
java.version=1.3.1_03
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=da_DK
Command-line arguments: -os win32 -ws win32 -arch x86 -install 
file:C:/download/eclipse/eclipse-SDK-20020612-win32/eclipse/
!ENTRY org.eclipse.jdt.ui 4 1 jun 14, 2002 14:44:36.472
!MESSAGE Internal Error
!STACK 0
java.lang.reflect.InvocationTargetException: java.lang.OutOfMemoryError
	<<no stack trace available>>
!SESSION jun 14, 2002 14:47:43.58 ----------------------------------------------
java.version=1.3.1_03
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=da_DK
Command-line arguments: -os win32 -ws win32 -arch x86 -install 
file:C:/download/eclipse/eclipse-SDK-20020612-win32/eclipse/
!ENTRY org.eclipse.ui 4 4 jun 14, 2002 14:47:43.58
!MESSAGE Unhandled exception caught in event loop.
!ENTRY org.eclipse.ui 4 0 jun 14, 2002 14:47:43.68
!MESSAGE java.lang.OutOfMemoryError
!STACK 0
java.lang.OutOfMemoryError
	<<no stack trace available>>

Comment 8 Nick Edgar 2002-06-14 09:35:37 EDT

What kind of editor are you opening?
Can you provide steps to reproduce this?

Comment 9 Nick Edgar 2002-06-14 16:13:23 EDT

Can you supply a workspace which exhibits this problem?

Comment 10 Nick Edgar 2002-06-14 17:09:58 EDT

Can't commit this for F4 until we have reproduceable steps and a proposed fix.

Comment 11 Henrik Hjoellund Hansen 2002-06-17 02:52:54 EDT

I can reproduce the error the following way:

Start from scratch, by setting up the project, see this defect for project size 
(point to location, include external jars (40),) click finish and the project 
is scrubbing output folder, compiling and it is still OK. In the project there 
is a compile warning, ( a method is deprecated)I click on the entry in the task 
window, and after 20 second the editor is showing the class with the deprecated 
method. The next time I ask for an action ex. clicking on another java file in 
the package view, the out of memory error happens. It really does not matter 
what I do after compiling, the out of memory error will occur anyway.
I have a reproduced on 256 and 512 MB RAM Windows PC's

I am sorry I cannot give a workspace for this, as I regard the project as being 
confidential.

Comment 12 Nick Edgar 2002-06-17 09:07:34 EDT

To be clear, you have:
- one project
- 651 source files in 111 folders
- project classpath has 40 external jars

Does the file that you are opening refer to many types in these external jars?
Does it make a difference if you turn off all settings under Workbench > 
Preferences > Java / Editor / Problem Indication?

Comment 13 Nick Edgar 2002-06-17 16:03:23 EDT

Please investigate with a similar setup using a memory profiling tool.

Comment 14 Henrik Hjoellund Hansen 2002-06-18 08:34:35 EDT

I also get the error in a smaller project. Another project has 135 files, but 
is still using the 40 external jars. 
If I turn of the "all settings under Workbench > 
Preferences > Java / Editor / Problem Indication" it in way does make a 
difference. I do not get the OutOfMemory at once, but I get a responce time of 
every click of about 5 sec, and after say 5 click opening java files in the 
editor I get the same OutOfMemory error. (I have checked the log file). I do 
not get the error using the M5. 

Comment 15 Henrik Hjoellund Hansen 2002-06-20 07:41:19 EDT

I have installed a profiler tool. It requires me to start eclipse by the jar 
file. I have tried to start eclipse by entering 

C:\download\eclipse\eclipse-SDK-20020612-win32\eclipse>java org.eclipse.core.lau
ncher.Main 

but it does not work. So the question is: How can I start eclipse by pointing 
to a jar file, and also have the right classpath. 

Comment 16 Kai-Uwe Maetzel 2002-06-20 07:49:27 EDT

You can start it with the following command line:

java -ms30M -mx200M -verify -cp startup.jar org.eclipse.core.launcher.Main 
-data <your workspace>

This also tells you how to set up the profiler.

Comment 17 Chris McLaren 2002-06-23 23:22:04 EDT

Henrik:

Respecting the confidentiality of your workspace, I constructed a single 
project of similar size: 71 jars, 50 source files (each 2000 lines long). I 
created these source files automatically with Test.java (attached). As such, 
they don't represent any sort of real world application. The intention was only 
to tax Eclipse's parser/editor a bit.

1. Windows 2000, PIII 866, 512 MB, IBM JVM, Clean 20020620 integration build 
(in C:\eclipse-SDK-20020620-win32\eclipse), Start Eclipse with no command line 
parameters

2. Window|Preferences -> Turn off "Perform build automatically.."

3. Open Java perspective and create a new project "Test"

4. To get a large number of useful jars into the project, I took the complete 
set of Eclipse jars. (In explorer, C:\eclipse-SDK-20020620-win32
\eclipse\plugins, F3 to search for *.jar. Select All & Copy to C:\eclipse-SDK-
20020620-win32\eclipse\workspace\Test. Overwrite any duplicates. You should end 
up with 71 unique jars.

5. Select project Test, Right click Properties, Java build path, External jars, 
Add external jars.

6. Copy Test.java to C:\eclipse-SDK-20020620-win32\eclipse\workspace\Test.
(When run, this application will open all 71 jars in the Test project, and 
produce the complete set of 7814 outer class names across all jars. It then 
produces 50 Java source files (Output0.java - Output49.java), each containing 
2000 declarations to random outer classes within that list.)

7. Select project Test, right click Refresh. Run Test as a new Java 
application. Refresh again to add Output??.java to the project.

8. Start opening Output0.java - Output49.java in order.

I opened them all successfully, though my system was virtually unresponsive.
Then I started to click between a number of these open editors and voila:

!SESSION Jun 21, 2002 14:08:52.554 ---------------------------------------------
java.fullversion=J2RE 1.3.1 IBM Windows 32 build cn131-20020403 (JIT enabled: 
jitc)
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments: -os win32 -ws win32 -arch x86 -install file:C:/eclipse-
SDK-20020620-win32/eclipse/
!ENTRY org.eclipse.ui 4 4 Jun 21, 2002 14:08:52.554
!MESSAGE Unhandled exception caught in event loop.
!ENTRY org.eclipse.ui 4 0 Jun 21, 2002 14:08:52.554
!MESSAGE java.lang.OutOfMemoryError
!STACK 0
java.lang.OutOfMemoryError

The javaw process had peak mem usage of 331,816K and it page faulted an 
outstanding ~2.481M times.

I'm actually showing you the last test first - I did a number of others before 
running out of memory. In fact, the test above was the only one that ever 
caused the VM to run out of memory. Usually, the VM becomes so unresponsive 
that it is hard to continue driving it toward overflow. I suppose this is a 
form of self-preservation.

I was unable to produce the OutOfMemoryException upon opening a file 
specifically, but I certainly think that it is possible. Perhaps it would be 
helpful to know the number of lines in the source file you are trying to open. 
For instance, is it very high.. > 10,000?

In another test I tried 10 files, each 8000 lines long. Opening Output0.java, 
Output1.java, and Output2.java took about 30 seconds. At this point, the javaw 
process memory usage has jumped up to about 190,000 K. Opening Output3.java 
causes the javaw process memory usage to leap by nearly 100MB - now a 287,000K.
Open Output4.java. javaw mem usage about 306,000K. At this point, things are 
nearly unresponsive, but eventually all 10 open without an OutOfMemoryException.

During these tests I did encounter a few odd exceptions over the course of a 
few runs: see e1.txt and e2.txt attached.

I've tried to get more information by profiling (in OptimizeIt) with the 
following config:
	working dir: C:\eclipse-SDK-20020620-win32\eclipse
	main class: org.eclipse.core.launcher.Main
	java parameters: -ms30M -mx200M -verify
	classpath: C:\eclipse-SDK-20020620-win32\eclipse\startup.jar;.

It is impossibly slow to do this. (It takes 5-10 min just to open Output0.java, 
the 8000 line version). After it opened and Eclipse settled, I had a look at 
where all the memory was gone.

Of the single largest number of instances, char[], I looked at the allocation 
backtrace to get an idea where they were coming from:

Of the 100934 instances at the time of the snapshot,
eight methods allocate more than 1% of these instances each,
combined these methods total more than 92.4% of all char[] allocations
these eight methods are:

String.String() 			[~28460]
Scanner.getCurrentIdentifierSource(); 	[~22320]
CharOperation.concatWith(); 		[~15450]
Scanner.optimizedCurrentTokenSource5(); [~13640]
StringBuffer.StringBuffer()		[~9210]
Scanner.optimizedCurrentTokenSource4()  [~1890]
ClassLoader.defineClass()		[~1230]
StringBuffer.expandCapacity()		[~1070]

The Scanner methods, at least, originate in Parser, from methods such as 
Parser.pushIdentifier().

I clicked on the Eclipse desktop to bring it to the foreground and after 
waiting for about 5 minutes for a response, the allocated instances of char[] 
jumped up to nearly 1M. I got control of the profiler when the char[] dropped to
about 400K, and found on allocation backtrace that SearchableEnvironment.find() 
and SearchableEnvironment.findType() combined to allocate approximately 40% of 
these.

I can't draw a lot of conclusions from this. The memory problems I have seem to 
occur 'naturally' in the sense I am demanding a lot of Eclipse in a controlled 
manner. 

I'm not sure where to go from here, if anywhere. Any suggestions?

Comment 18 Chris McLaren 2002-06-23 23:23:11 EDT

Created attachment 1563 [details]
Test.java

Comment 19 Chris McLaren 2002-06-23 23:23:31 EDT

Created attachment 1564 [details]
e1.txt

Comment 20 Chris McLaren 2002-06-23 23:23:42 EDT

Created attachment 1565 [details]
e2.txt

Comment 21 Henrik Hjoellund Hansen 2002-06-24 07:28:27 EDT

I have run my project with Optimizeit using the 0416 version and F3. I have 
attached the results. I am still able to run the 0416 version without the 
OutOfMemory error. Then I run the F3 version the OutOfMemory error shows up 
during compilation of the project. (When I set up the project and then compile) 
I have also seen a difference in starting the F3 with the Eclipse.exe and the 
command line:

java -ms30M -mx200M -verify -cp startup.jar org.eclipse.core.launcher.Main 
-data <your workspace>

I do not get the OutOFMemory error when I start with the java command line. 
Does it allocate more memory and is this the solution, or maybe a coincidence

Chris: it seems like the same error you are getting, and even though you write 
this is expected with aproject at that size I still think the size of my 
project is not that big, (it is a real world example) so I think other users 
will run into this problem also.

Comment 22 Henrik Hjoellund Hansen 2002-06-24 07:29:35 EDT

Created attachment 1569 [details]
report from Optimizeit version 04162002

Comment 23 Henrik Hjoellund Hansen 2002-06-24 07:30:04 EDT

Created attachment 1570 [details]
report from Optimizeit version F3

Comment 24 Henrik Hjoellund Hansen 2002-06-27 04:21:32 EDT

I will out of office the next 3 weeks, and will not be able to repoond to
questions. Do you have any news about this defect, it is preventing us from
using the F1 to F3 builds. Thanks

Comment 25 Nick Edgar 2002-07-03 11:50:13 EDT

The high number of WordEntry and related objects needs to be investigated 
further.
I don't see a big difference in the shape of the heap dumps provided by Henrik 
between 20020416 and F3.  The F3 dump just has higher counts.  This could occur 
if the 20020416 dump was using an older workspace where some files had already 
been indexed.

Comment 26 Nick Edgar 2002-07-03 11:51:47 EDT

Moving to JDT Core.  The Indexer should merge the index periodically when 
indexing a large jar.

Comment 27 Nick Edgar 2002-07-03 11:52:18 EDT

Henrik, it would help to know the sizes of the jars you're dealing with.

Comment 28 Henrik Hjoellund Hansen 2002-07-22 04:32:00 EDT

Hi 
The size of the jars: 27 MB in 48 files.

Just to make sure: I start from "zero" both when using april 16. version and 
the F3 version. I do not get the NPE in the april 16 version.  

Comment 29 Henrik Hjoellund Hansen 2002-07-31 02:51:54 EDT

This might narrow it down...
I downloaded the R2 (eclipse-SDK-2.0-win32.zip) version. I tried it on 2 
projests. The projects differ in size. One project has 127 files (913 KB) in 15 
folders, the other project has 287 files (971 KB) in 51 folders. Both project 
har using the same external jars. The size of the jars is 27 MB in 49 files. 
The R2 version works well on the project with the 127 files, but I get the 
reported NPE when running on the project with the 287 files.
 

Comment 30 Kent Johnson 2002-07-31 11:43:25 EDT

Henrik, what does your normal command line look like?

Mine is:
c:\aR2.0\eclipse\eclipse.exe -data c:\aR2.0\workspace\plugins -vm c:\jdk1.3.1_01
\bin\javaw.exe -vmargs -Xmx150mb

Sun's VM has a couple startup options which you can configure:
-Xms<size>        set initial Java heap size
-Xmx<size>        set maximum Java heap size

The VM defaults are usually not sufficient to run Eclipse.

Comment 31 Henrik Hjoellund Hansen 2002-07-31 16:11:00 EDT

hi

I start eclipse by creating a shortcut to eclipse.exe. I have seen this
recomended on the Eclipse web site. In the properties for the shortcup I specify
a working directory. And thats it. I will try to use the command line you
specified, and report back. 

Should't the eclipse.exe specify parameters sufficient for even bigger projects?

thanks

Comment 32 Henrik Hjoellund Hansen 2002-08-01 04:12:29 EDT

hi 

I have specified the -vm arguments as you proposed, and it seem to have solved 
the problem :-)

thanks
Henrik.

Comment 33 Nick Edgar 2002-08-01 10:11:27 EDT

Just to clarify, when you were getting this before increasing the heap size, 
was it an NPE or an OutOfMemoryError?
If it was an NPE, there's still another problem besides the heap being too 
small.

Comment 34 Kent Johnson 2002-08-01 12:14:32 EDT

Henrik, I believe eclipse.exe doesn't alter the default VM args but I'll check. 
It could get tricky considering the number of possible VMs and platforms.

Most users are specifying where their VM lives and as a result increase the 
defaults.

Comment 35 Henrik Hjoellund Hansen 2002-08-02 03:00:22 EDT

Hi

The question about OutOfMemory or NPE:
At the time this defect was entered on F1 ( may 29, 2002 ) I had both a NPE and
OutOfMemory error. (See log in the defect) But it seems like the NPE has been
solved, so now on R2 only a OutOfMemory error occurs. And the specified vm args
has solved this. 

Now that the solution is so obvious (Allowing more memory) I think it would be a
good idea to make this more clear in on Readme doc. Now I can see it is there
but I think it would be good to say like: If you are working on projects larger
than (XXX) you should specify (-vmargs more memory) otherwise you can get an
OutOfMemmory error. 

I also think this should be added to the FAQ list. 

But thanks for your help. I'm pleased that I am now able to run R2 :-)

Henrik.

Comment 36 Kent Johnson 2002-09-03 14:01:25 EDT

Forwarding to RelEng to see if they can provide config options/pointers on the 
download page.

Comment 37 Dean Roberts 2002-11-18 11:11:49 EST

Closing this bug since the actual error was fixed long ago.  Opening a new 
issue with regards to documenting eclipse arguments somewhere useful like the 
readmes