Community
Participate
Working Groups
It is easy to detect when the job manager becomes idle (no background jobs running). This may be a good place to periodically force garbage collection. Investigate.
Created attachment 20311 [details] Patch to org.eclipse.core.runtime Patch to runtime to add IJobManager#isIdle() method
Created attachment 20312 [details] Patch to org.eclipse.ui.ide Patch to IDE adds filters on KeyUp and MouseUp, and performs garbage collection if all of the following are true: - Mouse and keyboard have been idle at least five seconds - JobManager is idle - No gc has occurred in the past minute. I have been running with this on Windows and it feels good. It certainly keeps used Java heap size to a minimum. I'm attaching this patch so it can be tested on other platforms/OSes.
I approve the addition of the new IJobManager API.
I have released the IJobManager.isIdle() API. Note that this API release is unrelated to whether or not we actually go ahead with the garbage collection. This API adds a single boolean method that returns true if the job manager is "idle", where "idle" means there are currently no running or waiting jobs.
Just an implementation thought that may not be that important in practice. You say "Mouse and keyboard have been idle at least five seconds", but that doesn't seem to rule out really graphic intensive diplays that might be keeping the display thread busy for a while (e.g. a small movie type graphic). Is there any way to include "Display thread not busy" as one of the rules to fire the .gc? Just curious. I do not know of any immediate use case that does this type of graphics in Eclipse ... just seems like an obvious one.
My first cut was to only gc when the event loop is idle. Unfortunately, on most platforms there are events that fire even when nothing interesting is happening at all. You are correct that there may be other cases where non keyboard and mouse events continue to fire but you don't want to gc. The problem is I don't know of any way to filter "interesting" UI events from the noise of events that is firing all the time.
Explicit gc code released to HEAD, off by default. It uses the following system properties: ide.gc - gc enabled if this property is "true" (default: false) ide.gc.interval - the minimum time interval between explicit gcs (default: 60000) ide.gc.max - the maximum gc duration before the feature is turned off (default: 8000) It also uses a multiple of the previous gc duration to determine the minimum time before the next gc. Currently this multiple is set to 60, so a gc taking two seconds will result in no gcs occurring for at least two minutes. This ensures that in the worst case, gc will be running for no more than 1.7% of total elapsed time.
Having it off by default makes sense. My vote would be to turn it on (by default) immediately after R3.1 ships and run with it for the entire R3.2 development cycle. If we haven't seen any pathological behavior by the time the R3.2 release comes around, we'll leave it on. Also, should we build a new and noteworthy to describe the new capability?
How's this interact with the fix for bug 85072?
> How's this interact with the fix for bug 85072? It's unrelated.
I see them as trying to solve the same root problem in two different ways: poor interaction between Windows VM paging and Java Garbage collection. Doing gc's during idle was one of the proposed solutions to bug 85072 but ultimately Steve put in something with the Minimize message as suggested by Microsoft. So I'm just wondering if one or the other fix is sufficient on its own, or if the two help each other, or if the two hinder each other. Each can have negative effects so if one can solve it by itself then maybe you don't need both.
bug 85072 discusses a Windows only solution, whereas performing gc when idle benefits all platforms. The main benefit of doing a gc when idle is that it reduces the occurrence of garbage collections when things are not idle (ie., when the user is typing). It does have the side benefit of compacting the used portion of the Java heap, and keeping used memory pages swapped in. This has some overlap with the fix in bug 85072.
Instead of doing this with display.timerExec() and IJobManager.isIdle(), wouldn't it be a better design to have the concept of "Idle jobs" in the job manager that run only if there is nothing better to do? Then you could use the normal job scheduling and conflict handling, and have more than one idle handler. The GC job would be nothing special, just an Idle System job that happens to do a GC. Other projects might be able to take advantage of Idle jobs too. For example you could do some search indexing while idle, maybe some CVS synchronization, or perhaps perform some long running computation (like a Seti@Eclipse kind of thing). In contrast, this isIdle() API seems to be of limited use.
I tried an approach using only the job manager. The problem is, the job manager has no way of determining whether the user interface is idle. This is much more important a factor than whether there are any background jobs running. For example, if the user is typing in an editor, there are typically no jobs running, but performing a full gc at that time would be very bad, since it can lock the process for a second or more. When I tried running with this for a day I found it often doing gc when I didn't want it to as a user. Therefore, the solution needs input from two places: 1) Whether jobs are idle, 2) Whether UI is idle. Since SWT knows nothing about jobs and IJobManager knows nothing about a UI, the solution needed to live in the Eclipse UI that has knowledge of both. I considered a more general mechanism where idle jobs could be added to the job manager (such as the original Unix idle process that computed Pi to infinite decimal places when no other processes were active). However, given that we were close to the API freeze and had no other compelling uses for it, I opted for the minimal API instead (the isIdle method). This allows the gc algorithm to work without releasing unneeded API. As it turns out, isIdle() has become a very useful method for test suites that want to ensure no other jobs are running before they start a test.
3.1I20050509-2010 I added -Dide.gc=true to the eclipse command line after -vmargs. How can I tell it's working? In the patch there was a println but I don't see any output in the console window. I started with -debug.
These .options values will print info to standard output when gc occurs: org.eclipse.ui.ide/debug=true org.eclipse.ui.ide/debug/gc=true And these options will turn on the heap usage view in the status line: org.eclipse.core.runtime/perf=true org.eclipse.ui/perf/showHeapStatus=true
(In reply to comment #15) > -Dide.gc=true Anybody could point me to a comprehensive list of eclise.exe command line options? Thanks
Shouldn't the idle gc wait until after Eclipse is not busy for a while? I just saw it run seconds after I deleted a project with the mouse.
Re comment #17: There is no comprehensive list of command line arguments and system properties. This is mainly due to the modular nature of Eclipse. Each plugin in the system has its own list of command line arguments and system properties that changes over time, which makes it difficult to create a complete list. The best reference I can give is Help > Help Contents > Workbench User Guide > Tasks > Running Eclipse. This lists the main supported command line arguments. Undocumented properties such as "ide.gc" should be treated just like the -X arguments in java - these are not API and might change over time. In this case the property was added so people can experiment with it, in order to decide if it makes sense to turn it on by default in the next release. Re comment #18: it will currently GC if the keyboard and mouse have been idle for a sufficient time, but it doesn't wait for extra time after a long running job completes. If you don't click or hit any keys, as soon as a long running background job completes it might GC. This seems reasonable, since after a long running job completes there is likely to be garbage that needs collecting. I'm open to arguments for a different heuristic if that doesn't make sense.
I'm pretty sure I was moving the mouse and clicking on things, just not doing keyboard input. I was cleaning up my workspace, deleting projects and moving them around, so I thought it was odd that it did an idle gc in the middle of that.
Just so I'm clear on this: You are sure it was an idle GC and not just a naturally occuring one?
Yes, I have the option turned on to print to the DOS console when the idle gc runs and that's how I knew it ran one. It looks like this: Explicit GC took: 797 Next GC to run in: 60000 I had it happen again just now while I was scrolling through the Eclipse console window with the mouse. I noticed a little pause, and one of those messages blipped up in the DOS console. It's acting like the 'idleness' test doesn't take into account mouse events. Even with this problem I think it's a nice feature. Between this, the working set trim on minimize thing (bug 85072), the increased jvm memory sizes (bug 89059) and Eclipse memory optimizations (various), this version *flies* compared to 3.0.
To be precise, the idle test only takes into account mouse up (releasing a mouse button) and key up. I didn't want to impose the extra overhead of filtering all mouse events.
Explicit GC has been turned on by default for the workbench IDE application (3.2 M2).
(In reply to comment #23) > To be precise, the idle test only takes into account mouse up (releasing a mouse > button) and key up. I didn't want to impose the extra overhead of filtering all > mouse events. MouseUp events are not frequent enough. If the user is interacting graphically via native Drag-and-Drop, you will never see *ANY* mouse up events even though the mouse is released to perform the DROP. The same is true for the mousewheel. You should also investigate whether accelerators are follwed by "regular" keyup events.
I would like to see this running in the wild for awhile to get more user feedback before tweaking the definition of "idle" we currently use. I have entered a new bug report to track this: bug 109760.
Verified in I20050920-0010.
