Community
Participate
Working Groups
I have a simple test SWT application which demonstrates that the memory allocated for a SWT browser can not be properly garbage collected after the browser is disposed. The test application has a simple browser in it, a text control to enter url and a button to dispose the browser widget. The observed behavior is as follows: 1. the creation of the browser widget costs around 3M 2. upon going to about:blank page, memory increases by another 2M 3. upon going to google, another 4M is added 4. upon navigating to a few pages from google, the memory easily goes up another 10-15MB 5. disposing the browser at this point releases only 6M of memory, leaving a large amount of memory un-reclaimed. Can anyone help me identify where the memory goes and is there a way to garbage collect it?
Where is the code? Please attach it to this bug report.
Created attachment 37656 [details] A simple swt browser test application Attached is the source code for a simple browser test app. The test steps are: 0. start the app 1. click the 'create' button to create a browser 2. navigate to a few sites (e.g. google news, cnn, etc), monitor the growth of memory in task manager 3. click the 'dispose' button, see the drop of memory (~6M on my windows XP) 4. repeat step 1 through 3 multiple times and observe the steady growth of memory (up to 50M in about 10 minutes of such interation) Suspecting the cause due to natural growth of VM heap size, I added the following vm args to my test run: -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=10 -Xmx10m, but it doesn't seem to help at all. Being fully aware of IE potential memory leak, I still can't understand why after the browser is disposed, the leaked memory of the browser cannot be re-claimed. Maybe the memory is released, but stays as dark matter for some reason? I will set up some monitoring tool that allows me to monitor the breakdown of what's on the heap and post my findings here. One thing I did observe is that the memory growth rate is proportional to the number of distinct web pages I navigate to. Returning to the same web page doesn't increase memory consumption. This leads me to think: is it possible that IE caches large content in memory and adds that to the memory space of the application and stays there forever? Is there a way to completely flush out the cached in-memory content as if a true IE browser instance is killed? Thanks in advance for your support.
In attempt to identify where the memory went, I used JRockit 5 jconsole to monitor the memory usage of the test application posted in the comment above. It turns out that while the heap memory is totally behaving, the non-heap memory accounts for the continuously growing memory consumption. with an initial size of 24M, it easily climbs up to the 40s and 50s zone by a few clicks from google news homepage. when the browser is disposed, the non-heap memory drops ~6M while the heap size stays stable. I hope this info may help diagnose the isse at hand.
Created attachment 38083 [details] Snapshot of non-heap memory growth using JConsole Using Jconsole, I was able to look into the non-heap memory growth pattern. Interesting enough, there seems an average raise of 1 mb on the non-heap between 2 consecutive disposals of a browser widget. Is that normal? Your feedback and support is greatly appreciated.
Thank-you for looking into this! No memory leak is ever 'expected', so you have likely found a bug. I have never done an analysis like this on the Browser before though, so I don't know what the proper amount of memory usage should be. At this point in the release cycle our focus is on fixing crashes, so I don't have a chance to investigate this right now. Please continue to append your findings though, and this will be looked at when time permits. Also note that it would be even more helpful if your investigation was based on a recent 3.2-stream version of the Browser.
Grant, Thanks for getting back to me on this. This potential memory leak really slows down our dev cycle significantly. After several days of research, I have yet found a profiling or memory leak detection tool that can look into the non-heap space and provides meaningful data as for what is really growing in that space. Do you happen to know such a tool? Also, you mentioned using 3.2 stream browser version, can you please direct where I may get it? I'm currently using eclipse 3.2M6 stream stable build. Thanks!
Sorry, I don't know of a good tool to suggest for this. 3.2M6 is new enough.
Hello, can it be that the problem is inside the Internet Explorer and not inside the browser widget which just wraps the Internet Explorer? I normally come from C++ and there I have the same problem. Each time I navigate to a further URLs my application occupies more and more memory, even if I delete the empedded Internet Explorer object after each URL again. I found out that every new URL causes several additional threads in my application, which Internet Explorer is creating. But if I delete the browser object, these threads were not killed and still keep in memory and may this causes that also no memory is given back to OS. I further tested just with the Internet Explorer itself. If I navigate to different URLs the memory used by Internet Explorer gets bigger and bigger. (I just used the taskmanager's virtual memory and view the pagefile getting bigger and bigger). So it seems the Internet Explorer causes the problem. Is this information helpful to anyone?
The snippet below uses just a few lines of OLE to embed IE, and it shows a very similar growth in memory usage to the previous snippet that uses the Browser. So it seems that the leak source must be one or more of: - the embedded IE control (nothing we can do about this) - swt's OLE implementation, which the Browser uses (DN can you investigate this with the snippet below?) - some missing implementation detail that's needed at dispose time (I'll look into this, though if anyone else is able to make the snippet below stop leaking then please follow up here) public class ModifiedSnippet123 { static OleFrame frame; static OleControlSite controlSite; static OleAutomation webBrowser; public static void main(String[] args) { final Display display = new Display(); final Shell shell = new Shell(display); shell.setBounds(10,10,800,800); Button createButton = new Button(shell, SWT.PUSH); createButton.setText("Create"); createButton.setBounds(10,10,100,30); Button disposeButton = new Button(shell, SWT.PUSH); disposeButton.setText("Dispose"); disposeButton.setBounds(150,10,100,30); final Text urlText = new Text(shell, SWT.SINGLE); urlText.setLayoutData(new GridData(GridData.FILL_HORIZONTAL)); urlText.setBounds(300,10,400,30); // start of interesting section createButton.addListener(SWT.Selection, new Listener() { public void handleEvent(Event event) { if (frame != null) { System.out.println("there's already a browser so run away"); return; } try { frame = new OleFrame(shell, SWT.BORDER); frame.setBounds(10,100,700,600); controlSite = new OleControlSite(frame, SWT.NONE, "Shell.Explorer"); controlSite.doVerb(OLE.OLEIVERB_INPLACEACTIVATE); webBrowser = new OleAutomation(controlSite); } catch (SWTError e) { System.out.println("Unable to open activeX control"); return; } } }); disposeButton.addListener(SWT.Selection, new Listener() { public void handleEvent(Event event) { if (frame == null) { System.out.println("there's no browser so run away"); return; } webBrowser.dispose(); controlSite.dispose(); frame.dispose(); webBrowser = null; controlSite = null; frame = null; } }); urlText.addListener(SWT.DefaultSelection, new Listener() { public void handleEvent(Event event) { int[] ids = webBrowser.getIDsOfNames(new String[]{"Navigate", "URL"}); Variant variant = new Variant(urlText.getText()); webBrowser.invoke(ids[0], new Variant[] {variant}, new int[]{ids[1]}); variant.dispose(); } }); // end of interesting section shell.open(); while (!shell.isDisposed()) { if (!display.readAndDispatch()) display.sleep(); } display.dispose(); } }
I investigated similar to mr. burns (#8) that the Java process keeps handles to files that are used for caching by the IE control and GDI handles (I used the Sysinternals Process Explorer), even after the Browser widget was disposed. The number of handles increases with every new instance of Browser and every new page that is displayed. So I asume it is a bug in the IE control.
There are pages that leak a lot (6MB), and that causes a big problem because we keep adding and removing those (piloted web browser). I built a test case and noticed something interesting: refresh() causes a big drop of memory. It turns out that if I refresh and defer disposal (asyncexec), the memory does not grow. The problem is that disposal is defered which is not practical in several of our situations. Note that I don't know if the refresh method works when there are pages in the history. Would someone be kind enough to have a look at my test case, and see if something along those lines could be integrated within the Browser implementation? In that test case, a browser points to the FCKeditor online demo page (big leak there). A button allows to recreate that browser, and a check box on the side allows to switch to the optimized mode (no leak).
Created attachment 99356 [details] A test case showing a big leak, with a method to avoid the leak
Here is more info about the test case and its workaround. I open the task manager and I look for the memory used by the Java process. Then I click on the "recreate" button which disposes the browser and recreates a new one, setting its URL to the page that produces a big leak. When the button is pressed and when I don't use my workaround, I see the memory slightly going down and then going up. Here is the memory between each "recreate": 35MB 32MB 41MB 39MB 46MB 44MB 52MB 50MB 58MB I noticed that I was not disposing my temporary Shell in my test case. Fixing it gives: browser.addLocationListener(new LocationAdapter() { @Override public void changing(LocationEvent e) { // Does not seem to be called, but let's play it safe. e.doit = false; } }); browser.refresh(); // Simulate some sort of dispose by removing it from its original parent. final Shell shell = new Shell(); browser.setParent(shell); // Defer disposal so that the effect of refresh in cleaning resources happens. browser.getDisplay().asyncExec(new Runnable() { public void run() { browser.dispose(); shell.dispose(); } }); I thought that I could just dispose the shell and that it would dispose the browser, but there is still a memory leak if I simply dispose the Shell, hence the 2 dispose() calls. As I mentioned, we can't use that workaround in several cases because the dispose() calls are deferred. Nevertheless, I hope it helps bringing new ideas for a deeply needed fix in the SWT Browser code. --- Here is also some information on how we use the Browser and why it is important to get a fix. In fact I believe the severity of this bug should be increased at least to MAJOR. We embed the FCKeditor, piloted through Javascript. This allows rich HTML editing from within the application. Some users have many screens with one or more such HTML editors (to see and edit notes on their clients, etc.). Each time they visit and leave those screens, it means a creation and disposal. If we consider a 6MB leak per instance, and that a new instance is created quite often, this becomes a real problem pretty fast.
Christopher, I work on a similar component using the TinyMCE and found another workaround: I created a pool for Browser instances. Each Browser is initialized only once with the TinyMCE-Code when it is originally created. Later on, I set only the content that is to be edited by calling the TinyMCE API via Browser.execute(). For the unused Browser instances, I use an invisible Shell as parent. This works in my special case, because the actual page in the Browser instance never changes, and I have only some (< 10) of this Browser instances that are visible at the same time. So I can afford to keep them in memory for reuse.
(In reply to comment #14) Florian, I had instance caching in mind as well, but it is not applicable in our case, which is why we need a clean dispose. I think my test case shows that IE can dispose of its resources properly, let it be with the refresh() method or some IE-specific API to be found, and I hope it will lead to a fix in the Browser class for everyone's benefit.
Thanks for these findings Christopher! I see the same results you see, and I also see them when used with an OLE snippet like the one in comment 9. At this point I'm not sure how to turn this into a fix, but I'm hoping to sit with someone next week to investigate further.
I found that the code can be simplified by navigating to "about:blank" or setting the HTML content to an empty string rather than using refresh(). I guess the refresh method was working because it was first clearing the page and my dispose() was coming before the new page got the opportunity to reconstruct. We can even make that somehow synchronous. It has a few drawbacks, so still not a proper fix, but it could give you some ideas: browser.addLocationListener(new LocationAdapter() { public void changed(LocationEvent event) { browser.dispose(); } }); browser.setText(""); while(!browser.isDisposed()) { browser.getDisplay().readAndDispatch(); } browser.dispose(); There still seems to be a small leak, probably the size of that empty page. That would need to be confirmed though. Still, a few hundred bytes is much better than megabytes. And if the native interfaces allow for removing the page/DOM, that may have a similar cleanup effect but without any leak. The drawbacks are that listeners are not disconnected and would see the navigation event, potentially vetoing it. Also, it pumps the events and may steal that pumping from higher level pumping which may not be desirable.
This is quite serious for us as well. It would be great if some investigation could also spent into why the IE requires more and more memory as soon as you start browsing websites. I noticed increased memory consumption after every setUrl()-call, with no decrease at all. It seems the IE holds to visited websites in memory, which is quite bad.
(In reply to comment #18) > I noticed increased memory consumption after every > setUrl()-call, with no decrease at all. It seems the IE holds to visited > websites in memory, which is quite bad. Isn't that because of the browsing history so that back/forward are faster? If this is the case then I don't think it is a bug. For me it is a bug if all this memory is not released when disposing the component. In any case, I think a good enhancement (maybe we should open this bug...) would be to allow a setUrl(s) to clear the history in case we don't need it. Something like: Browser.setUrl(String url, boolean clearHistory); ... and with a similar functionality for the setText(String html).
The bug is even worse: plugins that are loaded by a page seems to not be closed properly. Here is a comment from a user: "I am using the webbrowser component to view word documents by loading the word plugin whithin IE. However when closing the application the word process is left behind. Making it impossible to re-open the application. This is because the shutdown not seems to take to account the status of the shutdown of IE." The about:blank page trick solves the issue, so it seems to be the same bug. As I mentioned earlier, deferring disposal is not always possible. Moreover, many users don't know yet that they have this bug and they are likely to find it the hard way. Isn't it possible to push the workaround to the Browser's internal? i.e. disconnect the listeners, have the isDisposed() method return true but internally navigating to the blank page before releasing the resources?
Yes it is. We'd like to find a better fix or understand more what is happening first. Unfortunately, 3.4 is shipping soon and we can't fix this bug for that release.
Regarding the FCKeditor case, it looks like ~ 5/6 of the leak has been fixed in FCKeditor v2.6.2, probably as a result of the fix for http://dev.fckeditor.net/ticket/2018 . It appears that this is the same memory that was being reclaimed by the strategy in comment 17, because my hacked implementation of comment 17, which significantly reduced the leak with FCKeditor pre-v2.6.2, is now giving ~ the same result as the current dispose implementation with FCKeditor v2.6.2.
(In reply to comment #22) So it seems this was an FCKeditor/IE issue. Nevertheless, are there plans to add a fix so that badly coded website don't eat any memory after a browser disposal?
The active X control has a flag for NavNoHistory, but SWT.Browser only sets the URL on the Navigate operation. Below is sample code that navigates to a given URL bypassing the browser's history: Browser browser = getBrowserWidget(); try { Object webBrowser = getField(browser, "webBrowser"); OleAutomation auto = (OleAutomation) getField(webBrowser, "auto"); int[] nameIds = auto.getIDsOfNames( new String[] { "Navigate", "Flags", "URL"} ); short NavNoHistory = 0x2; auto.invoke( nameIds[0], new Variant[] { new Variant(NavNoHistory), new Variant(url) }, new int[] { nameIds[1], nameIds[2] } ); } catch (Exception e) { browser.setUrl(url); } I am doing this from outside the widget source, using reflection, as I don't want to change the SWT browser source and ship it myself. Here is the reflection code I use to get to the private webBrowser/auto field: Object getField(Object object, String fieldName) throws SecurityException, NoSuchFieldException, IllegalArgumentException, IllegalAccessException { Field field = object.getClass().getDeclaredField(fieldName); field.setAccessible(true); return field.get(object); } Maybe my fix can be added to the SWT browser as API like this: setUrl(String url, boolean noHistory) Then we can call browser.setUrl(url, false) to keep the history from growing.
I saw bug #84620, so I decided to test this memory leak again with a recent build. I base my tests on the leak test in this report, where I compare a normal dispose and setting empty content before disposing. I look at the memory usage after each component re-creation, with an old build and a recent nightly (N20091022-2000-win32-win32-x86). Memory in MB, as seen in the Windows Task Manager: Old build, normal dispose: 40, 49, 54, 58, 63, 68 Old build, disposal trick: 40, 45, 50, 54, 59, 64 New build, normal dispose: 40, 45, 50, 55, 59, 64 New build, disposal trick: 40, 45, 50, 54, 59, 64 The first thing we can see is that the disposal trick (setting an empty content) is not necessary anymore. The second thing we can see is that we still leak about 5MB everytime we visit the page from the test case. I recreated the browser components several times more to have a big memory usage in the Task Manager (125MB). I then looked at the Java heap size and it indicates 1MB used for a heap of 5MB. Perm Gen indicates 6MB used out of 12MB. I tend to think there is a leak in the native code, and this is very annoying because it happens for a page containing a popular Javascript-based HTML editor, which is clearly something people like to embed in their applications. I also tried with SWT.MOZILLA, and I noticed the same trend! But there is something strange that I noticed: if I dispose the browser, wait a second, and then re-create it, memory remains constant with Mozilla, whereas with IE it keeps growing. To sum up the issues: - Disposing IE does not reclaim all used memory. - Disposing/re-creating Mozilla too fast does not reclaim all used memory.
> - Disposing/re-creating Mozilla too fast does not reclaim all used memory. Further tests show that it does reclaim from time to time so overall memory is constants with Mozilla. IE on the other hand does seem to keep growing...
(In reply to comment #26) > > - Disposing/re-creating Mozilla too fast does not reclaim all used memory. > > Further tests show that it does reclaim from time to time so overall memory is > constants with Mozilla. IE on the other hand does seem to keep growing... What IE version are you using? I found IE8 to be much better than IE7. I won't even spend time to discuss my annoyances with IE6. Even with IE8, we have to be very careful not to leak. I actually tear down the current document (by removing all elements from the DOM) in the Browser widget before changing its URL. Using Mozilla resolves many issues, but is a pain for clients to use due to the extra installation/setup work that needs to be done. The problems is _really_ cause, I think, by IE sticking around in memory. Even when you dispose all browser widgets, the IE ActiveX Control appears to stay connected to the process. What we found works best is to use a browser widget Pool and ask each application to clear its DOM before reloading it with a new application.
> What IE version are you using? IE6, because many users still use it so this is the one I use for testing. > The problems is _really_ cause, I think, by IE sticking around in memory. Even > when you dispose all browser widgets, the IE ActiveX Control appears to stay > connected to the process. The problem I have is not that some memory is retained by the Active X control, but that this memory keeps growing. > What we found works best is to use a browser widget Pool and ask each > application to clear its DOM before reloading it with a new application. I think my test case shows that this would not even work, because I suspect disposal should clear the DOM. Isn't that the case? And if it isn't and clearing the DOM helps, why not have that performed internally to the SWT IE browser? In any case, what I tried to highlight when I revisited this bug is that the situation has improved. What I would like to know now is whether the remaining leak can be fixed within SWT or should be resolved as WONTFIX/NOT_ECLIPSE.
I'm experimenting with clearing the document's content. It seems to make no difference to the memory usage on IE6, but on IE8 it reduces the leak for the fckeditor demo to < 1M, so there's potential for improvement here. The Browser wouldn't clear document content between page navigations because Back/Forward functionality needs to be preserved, but it's probably doable during the Browser disposal process. Will continue investigating.
It looks like clearing the document content only helps if it's done while the document is current, so clearing them at Browser dispose time hasn't proven to be helpful yet.
I still see the memory leak in Mozilla Browser in Eclipse 3.6, here is my scenario: 1) Double click on a file, then my Editor with Mozilla Browser in it will be opened 2) The memory increased 15M 3) Close the Editor, but the memory is not released by the Browser widget Another scenario: 1) Double click on a file, then my Editor with Mozilla Browser in it will be opened 2) The memory increases 15M 3) Refresh the page, and the memory increases 10M for each refresh 3) Close the Editor, but the memory is not released by the Browser widget I'm sure the memory leak is cuased by Browser: the Java heap size of Eclips is atable at 200M, but the memory size in Task Manager is keeping increasing to about 800M. When the memory size reaches over 900M, then Eclipse will crash. It's appreciated for any idea/fix/workarounk. Thanks a lot.
I also encounter the same problem. I'm working on a screenshot generator that works through a list of urls ... opens them, scrolls to the right position, makes a screenshot and closes them. The memory each screenshot cycle consumes is about is some MB. I'm working on Mac OS X 10.7.1 and just switched to 10.7.2. The internally used browser is Safari.
I completely forget to mention that I checked the memory twice: 1st I checked the heap with MAT (Memory Analyzer Tool). The heap looked completely healthy and I couldn't find a single org.eclipse.swt.browser.Browser instance. 2nd I used Sleak (SWT analysis tool) and couldn't find any unused allocation After about 1500-2000 screenshots the java process consumes about 3GB memory whereas the heap is only around 50-60 MB.
This leak still happens with the latest 4.3 version of SWT. Version 4.3 fixes part of the leak, but the process size still manages to grow at about 4-6 MB per page visit. This process size can be seen using the Windows Task Manager. Does anyone have an ideas how to fix or find what is been leaked?
Does anyone have found a solution for this issue?
This leak still happens with the latest 4.7 version of SWT using Windows 7 64bit and IE11. The problem in my case is, that the target javascript-heavy web page itself creates memory leaks and the browser as well (known IE11 issue which won't be fixed due to edge). As our rcp based application uses the SWT browser widget quite a lot, the memory steadily increases and is never released even when the rcp-part containing the browser is closed. Therefore it slows down up to an unbearable performance. The only solution is to close the application entirely which is obviously a nuisance for any user. This problem can be easily reproduced in eclipse (I used Luna Service Release 2 (4.4.2)) by following steps: 1 set internal browser to IE 2 open internal browser 3 navigate to a JS-heavy page like http://ckeditor.com/ 4 monitor memory usage 5 repeatedly press F5 in the internal browser -> steady increase of the used memory but heap stays approximately the same 6 close internal browser 7 some memory is release, most of it remains I found a "workaround" for non rcp-applications by creating threads and use multiple Display objects which is not recommended. But when display.dispose() is called all memory is released. Sadly, rcp based applications only use a single SWT Display object which puts me again in a problematic situation as calling display.dispose() will shut down the entire rcp application. Due to the memory leak, my only chance at this point is to create another Display object in an own thread and somehow manage to get it to work with rcp (not knowing what other problems may arise due to multiple display objects). I see this as a major problem as applications using the SWT browser widget as well as users of the internal browser in eclipse will eventually run into this problem with no other workaround but to close the application. There is another issue https://bugs.eclipse.org/bugs/show_bug.cgi?id=405711 where the leak can be found in the Browser class which may help fixing this problem.
I've been playing around with some minimal SWT-Browser-application (Shell + Browser) and I'm pretty confident that (at least for IE) the problem is closely related to the caching the component performs. The JVM was allowed to use a maximum of 4 MB of heap memory, but the task manager came up with ~70 MB for the java process after loading the first page. This memory won't be freed after closing and disposing the Shell. On the other hand, the memory consumption doesn't increase when looping through the following cyle: new Shell -> new Browser -> setUrl() -> close window -> dispose Shell -> (repeat) When cycling through a small set of mostly static web pages from the same site (like wikipedia) or reloading the content by repeatedly clicking the site's logo, the memory consumption will clip to a reasonable maximum. When doing the same with an ad/banner-loaded site, paging though search results or scrolling through a map service, the memory consumption of the whole process constantly increases to about 630 MB before clipping. At the same time a single CPU core runs into saturation and the browser becomes partially unresponsive. As long as the SWT-wrapper doesn't provide the browser component with an memory allocation service which bypasses the JVM I'm pretty confident the problem lies within the component itself. Is there a known (and possibly hacky) way to clear the browsers cache to verify this? As an alternative it might help to forbid building a navigation history - maybe that's a trigger for the browser to cache at all.
Moving to inbox.
