Bug 457489 - [IDE] Default heap max (512MB) is too small
Summary: [IDE] Default heap max (512MB) is too small
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 4.4.2   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 4.5 M6   Edit
Assignee: David Williams CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
Depends on:
Blocks: 459596
  Show dependency tree
 
Reported: 2015-01-14 13:35 EST by David Williams CLA
Modified: 2015-03-16 10:25 EDT (History)
10 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2015-01-14 13:35:45 EST
+++ This bug was initially created as a clone of Bug #456585 +++

See bug 456585 for other details. But, we did discuss at today's (1/14/2015) arch. and planning meeting for the Platform and agreed it could use more research and discussion at the "Platform level" (which, might must mean the "SDK", not literally the "Platform" ... and, that too would be one of the things to discuss in this bug). And, we do think each EPP package needs to evaluate their own needs and audience and might easily have different, higher values that the Platform has. But, the originator of bug 456585 raises good points that deserve attention and research. 

One of the hardest issues, it that the VM's differ quite a bit based on their "bitness" and architecture, and even which vendor the VM is from, so its unclear is we can easily find "the best" minimum value to set ... that still satisfies most users. 

Note: I've put this "releng" for now, since typically mx is set as part of each "product build" ... but, in theory, could be changes in launcher to do some sort of fancy stuff.
Comment 1 Dani Megert CLA 2015-02-10 08:26:29 EST
512 is definitely too small for the SDK / IDE. I think we should raise it to 1024 for now.
Comment 2 Lars Vogel CLA 2015-02-10 09:04:42 EST
(In reply to Dani Megert from comment #1)
> 512 is definitely too small for the SDK / IDE. I think we should raise it to
> 1024 for now.

+1
Comment 3 David Williams CLA 2015-02-10 12:46:28 EST
I am taking comment 1 to mean "SDK only", not "Platform Binary". 

And here is the fix for that: 
http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=857a416a30dcdfb830ef04b77af16a5285c97a4e

In making this change, it made me think that we should *not* set these value in the p2.inf file (for "custom touch point"). During an "update", I believe the effects of that *might* be to change what someone had already customized (such as, if they had already already changed it to "1.5G" we might set it back to 1G? Should test after this is in a build. It will be in "nightly" of N20150210-2000. 


In the "Platform binary" we do not have those custom settings for mx value. But, only set in the product's definition file which get "written to" the eclipse.ini.
Comment 4 David Williams CLA 2015-02-10 13:48:08 EST
Added Markus Knauer to CC, in case those with EPP packages want to make similar changes (if they have not already).
Comment 5 David Williams CLA 2015-02-10 17:36:41 EST
(In reply to David Williams from comment #3)

> In making this change, it made me think that we should *not* set these value
> in the p2.inf file (for "custom touch point"). During an "update", I believe
> the effects of that *might* be to change what someone had already customized
> (such as, if they had already already changed it to "1.5G" we might set it
> back to 1G? Should test after this is in a build. It will be in "nightly" of
> N20150210-2000. 

I think it's correct to *not* set in p2.inf files ... but, for the opposite reason of what I first thought. 

From looking at code, and reading Paul Webster's blog on the topic (http://pweclipse.blogspot.com/2012_02_01_archive.html) p2 won't "change" a users value. At least, not one that is different than what we are working with. and since we are add 1024m during config, and remove 1024m during config, I don't think this would ever be useful. 

It might be useful, for example, if in a service release, we decided we did have to change to 1.5g, because then we could remove the known value of 1024m during "unconfigure' and during "configure" addJvmArg to 1536m. (This still wouldn't effect users who had "set themselves" but would change the value is users had left at previously shipped value. 

See also AddJVMArgumentAction.java (There's not a lot of JavaDoc, but skimming the code, appears to behave as Paul described.)

If anyone has any strong case to "configure" the JVM argument, other than "leave it as it has been :) then let me know. Otherwise, plan to remove it from the p2.inf file (for SDK or other "products").
Comment 6 David Williams CLA 2015-02-10 17:52:29 EST
This commit remove the "setJVMarg" touch points. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=18846cdc6cc840bef57ed288ad4303bd022d59f0
Comment 7 David Williams CLA 2015-02-10 22:42:39 EST
The other thing that a change in "max heap size" effects, are the settings used during unit and performance tests. There, in the past, we have said we want to use "the same values as used in the Eclipse SDK ini file. Hence, where we had been specifying "512", we now want to specify "1024". This changes three spots in "library.xml" file. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=783b638b3a45a7290513f5d9e8a90cf77943d6bd

To cross reference, some may recall we need to specify mx in library.xml at all, because (long ago) the test framework focused on using "java -jar o.e.launcher*, instead of using eclipse executable directly. (hence "ini file is not consulted). 
See bug 387638.
Comment 8 Lars Vogel CLA 2015-02-18 16:41:30 EST
Shall we also increase the current -Xms value? Currently it is set to -Xms40m.

I personally set -Xms512m together with -Xmx1024m, see http://www.vogella.com/tutorials/Eclipse/article.html#eclipse_memorysettings
Comment 9 Andrey Loskutov CLA 2015-02-23 11:09:17 EST
+1000 for increasing heap, 1.5-2 GB would be better, we are using 3 GB for example, but 1 GB is good starting point.

BTW would be nice to increase ReservedCodeCacheSize as requested in 459267.
Comment 10 David Williams CLA 2015-03-05 23:01:22 EST
(In reply to Andrey Loskutov from comment #9)
> +1000 for increasing heap, 1.5-2 GB would be better, we are using 3 GB for
> example, but 1 GB is good starting point.
> 
> BTW would be nice to increase ReservedCodeCacheSize as requested in 459267.

This will need to be addressed via bug 459267.
Comment 11 David Williams CLA 2015-03-05 23:13:25 EST
(In reply to Lars Vogel from comment #8)
> Shall we also increase the current -Xms value? Currently it is set to
> -Xms40m.
> 
> I personally set -Xms512m together with -Xmx1024m, see
> http://www.vogella.com/tutorials/Eclipse/article.html#eclipse_memorysettings

I think -Xms256m would be better ... and, consistent with what at least one Vendor's VM computes for itself, if nothing specified. See 

https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gc-ergonomics.html
and/or 
https://docs.oracle.com/javase/7/docs/technotes/guides/vm/gc-ergonomics.html

If we had defaults of "initial heap" of 256m and "max heap" of 1024m, then we are basically providing "the defaults" for a 4 Gig machine. 

I think that's a reason to not specify *any* (ms or mx) and let the VM work it out for themselves. I suspect "by Java 7" most VMs do a pretty good job of it. 

Plus, I suspect the reason we ever even started specifying ms and mx, was that the defaults for ?Java 5? and Java 4 were extremely small, thus "too small" for a large program such as Eclipse. The above references say " Before Java SE 5.0, the default maximum heap size was 64MB". 

The only problem with "leaving it out" is I do not know how every VM handles it, on all Platforms. But, again, my guess is defaults these days would be fine, for Eclipse. 

In conclusion, I suggest we "leave them out" ... and if/when someone complains that "Eclipse won't start" on some architecture, then consider putting them back in, for those architectures. 

If that is too radical, I suggest 256m for "initial heap", and count this bug "done".
Comment 12 Lars Vogel CLA 2015-03-08 00:37:36 EST
(In reply to David Williams from comment #11)

> In conclusion, I suggest we "leave them out" ... and if/when someone
> complains that "Eclipse won't start" on some architecture, then consider
> putting them back in, for those architectures. 

+1, I suggest to close this bug for now.
Comment 13 David Williams CLA 2015-03-08 11:24:30 EDT
(In reply to Lars Vogel from comment #12)
> (In reply to David Williams from comment #11)
> 
> > In conclusion, I suggest we "leave them out" ... and if/when someone
> > complains that "Eclipse won't start" on some architecture, then consider
> > putting them back in, for those architectures. 
> 
> +1, I suggest to close this bug for now.

+1 and "close" sound contradictory. I was suggesting we remove ms/mx entirely, or change ms to 256m.
Comment 14 Andrey Loskutov CLA 2015-03-08 11:32:15 EDT
(In reply to David Williams from comment #13)
> (In reply to Lars Vogel from comment #12)
> > (In reply to David Williams from comment #11)
> > 
> > > In conclusion, I suggest we "leave them out" ... and if/when someone
> > > complains that "Eclipse won't start" on some architecture, then consider
> > > putting them back in, for those architectures. 
> > 
> > +1, I suggest to close this bug for now.
> 
> +1 and "close" sound contradictory. I was suggesting we remove ms/mx
> entirely, or change ms to 256m.

Yes, removing sounds good.
Comment 15 Toby Thain CLA 2015-03-08 12:31:12 EDT
(In reply to David Williams from comment #13)
> 
> If we had defaults of "initial heap" of 256m and "max heap" of 1024m, then
> we are basically providing "the defaults" for a 4 Gig machine. 
> 
> I think that's a reason to not specify *any* (ms or mx) and let the VM work
> it out for themselves. I suspect "by Java 7" most VMs do a pretty good job
> of it. 
> ...
> The only problem with "leaving it out" is I do not know how every VM handles
> it, on all Platforms. But, again, my guess is defaults these days would be
> fine, for Eclipse. 
> 
> In conclusion, I suggest we "leave them out" ... and if/when someone
> complains that "Eclipse won't start" on some architecture, then consider
> putting them back in, for those architectures. 

Surely we don't need to guess? We can just *test* on a set of common, supported platforms. And add to the manual so that users know what to expect memory wise.
Comment 16 David Williams CLA 2015-03-08 12:54:51 EDT
Testing is good ... and, in fact, I just now found 

http://stackoverflow.com/questions/2915276/what-is-the-default-maximum-heap-size-for-suns-jvm-from-java-se-6

which shows some ways to "query" what max heap, etc., are. 

And, seems for IBM's VM, it does not take into account "physical memory" (at least for "client" class machines, like my Linux laptop) and always sets default mx to 512m and default ms to 4m! 

So ... based on that testing alone, I think we'd be better off with the proposed ms=256m and mx=1024m as defaults. 

I also want to point out, to those initially initially saying "defaults are too small" we do now, specify, by default, --launcher.appendVmargs. So, just in case you didn't know it, it should be pretty easy for anyone to specify different values, for "each" invocation of Eclipse (in your "short cuts" or eclipse startup scripts ... i.e. you do not have to modify "eclipse.ini" like in the old days, to get the values to "take effect". 

In other words, 
./eclipse -vmargs -Xms1024m -Xmx4g should "give you want you want" 
... if that makes things any easier. 
Just sharing the knowledge. :) 

Thanks for everyone's (continuing) thoughts on the matter, but, will go with 
ms == 256m and mx == 1024m for the SDK (again, I think I'll leave "binary Platform" as is).
Comment 17 David Williams CLA 2015-03-08 13:20:57 EDT
Adding Arun and Tom to CC, in case you know of any esoteric architectures or VMs for which these new defaults (ms == 256m, mx == 1024m) would be inappropriate. 

Thanks,
Comment 18 David Williams CLA 2015-03-08 13:28:24 EDT
For completeness, I search our "docs" repositories for "512m" (looking for cases where we might have documented the previous default values) and only found one hit, in a file named gcj.html but it seemed to be about another topic, and not really about memory settings, so I don't think it would have to be changed. 

It did, though, point to the wiki, at https://wiki.eclipse.org/Eclipse.ini
and that page, and some of what it links to, might deserve some slight polish, but, again, is already contain "wiggle word" that "the values shown are just examples and what is in your eclipse.ini may vary".
Comment 20 Lars Vogel CLA 2015-03-16 10:25:51 EDT
Added a short entry to N&N M6, feel free to extend it.