Comment 1 Oliver Meier 2018-04-24 12:13:25 EDT

Created attachment 273749 [details]
1 - GTK2 - new dummy(shell, SWT.NONE, false,false)

Comment 2 Oliver Meier 2018-04-24 12:13:43 EDT

Created attachment 273750 [details]
2 - GTK2 - new dummy(shell, SWT.NONE, true,false)

Comment 3 Oliver Meier 2018-04-24 12:14:05 EDT

Created attachment 273751 [details]
3 - GTK2 - new dummy(shell, SWT.NONE, true,true)

Comment 4 Oliver Meier 2018-04-24 12:14:32 EDT

Created attachment 273752 [details]
4 - GTK3 - new dummy(shell, SWT.NONE, false,false)

Comment 5 Oliver Meier 2018-04-24 12:14:54 EDT

Created attachment 273753 [details]
5 - GTK3 - new dummy(shell, SWT.NONE, true,false)

Comment 6 Oliver Meier 2018-04-24 12:15:16 EDT

Created attachment 273754 [details]
6 - GTK3 - new dummy(shell, SWT.NONE, true,true)

Comment 7 Eric Williams 2018-04-24 12:44:51 EDT

Hello Oliver,

Would you be able to try with a 4.8 I-build? The internal color background/foreground machinery has drastically changed in 4.8. 4.8 M6 or anything newer would be a good one to try.

Comment 8 Leo Ufimtsev 2018-04-24 13:29:03 EDT

(In reply to Eric Williams from comment #7)
> Hello Oliver,
> 
> Would you be able to try with a 4.8 I-build? The internal color
> background/foreground machinery has drastically changed in 4.8. 4.8 M6 or
> anything newer would be a good one to try.

As a note, the latest integration builds are here:
http://download.eclipse.org/eclipse/downloads/

Comment 9 Oliver Meier 2018-04-24 15:12:54 EDT

(In reply to Eric Williams from comment #7)
> Hello Oliver,
> 
> Would you be able to try with a 4.8 I-build? The internal color
> background/foreground machinery has drastically changed in 4.8. 4.8 M6 or
> anything newer would be a good one to try.

Hi Eric,

I am currently running the test with 4.8M6.
It uses significantly less memory on startup (130MB instead of 190MB) which is very nice!
Is there a way to enable the dark theme again (see "Screenshot 1.png")? I still have this hack to the SWTResourceManager ("Cheap hack.java") so it is not extremely important to me, but it would be nice to have "native" dark theme support.

Comment 10 Oliver Meier 2018-04-24 15:16:11 EDT

Created attachment 273758 [details]
Screenshot 1 - Wrong colors

Comment 11 Oliver Meier 2018-04-24 15:18:58 EDT

Created attachment 273759 [details]
Cheap hack.java

Comment 12 Oliver Meier 2018-04-24 15:21:31 EDT

While uploading i noticed, that the file dialogue in firefox also had the wrong button and background colors, so it may be a problem of the theme I am using on my laptop. Is there a way to force the dark theme like exporting GTK_THEME=Adwaita:dark?

Comment 13 Eric Williams 2018-04-24 15:28:57 EDT

(In reply to Oliver Meier from comment #12)
> While uploading i noticed, that the file dialogue in firefox also had the
> wrong button and background colors, so it may be a problem of the theme I am
> using on my laptop. Is there a way to force the dark theme like exporting
> GTK_THEME=Adwaita:dark?

GTK_THEME=Adwaita:dark should work for any GTK app, not just SWT.

As for the SWT snippet, I think you may have to keep your hack for now. SWT does load the system colors from the GTK theme, however it doesn't seem to detect that the dark variant is specified. This is a feature request I would definitely be interested in investigating for 4.9 though. :)

As for the memory leaks, please do let us know if the issue occurs on your 4.8M6 setup.

Comment 14 Oliver Meier 2018-04-24 16:29:08 EDT

Hmm, unfortunately the leak still seems to be there. The memory goes up straight in "7 - GTK3 - new dummy(shell, SWT.NONE, true,true) - SWT-4.8M6".
I still hope it will stop at some point, but visualvm tells me the heap size is 110MB and the usage stays between 2MB and 25MB.
From the current rate it looks like the leak got worse :(

Comment 15 Oliver Meier 2018-04-24 16:30:08 EDT

Created attachment 273761 [details]
7 - GTK3 - new dummy(shell, SWT.NONE, true,true) - SWT-4.8M6

Comment 16 Oliver Meier 2018-04-24 16:34:40 EDT

BTW, I filed a feature request for the theme stuff in bug 534007.

I will also repeat this test with a slower redraw rate of 50ms instead of 5ms tomorrow...

Comment 17 Oliver Meier 2018-04-24 17:12:14 EDT

Arghh! It keeps looking worse..
The initial memory consumption is probably only related to the java-vm reserving space related to available memory. With GTK2, where the theming is also not working anymore, the initial rss size drops to 80MB from 130MB.. (But at least rss stays constant at around 115MB (just below the heap size of 124MB) after 20min).

In the mean time the gtk3 instances rss rose straight up to 350MB, so it definitely looks like a memory leak now, tja.

Comment 18 Eric Williams 2018-04-24 17:13:21 EDT

Interesting. You say there might be some relation to the dark theme, does this memory leak not happen with the regular theme?

Comment 19 Andrey Loskutov 2018-04-24 17:22:18 EDT

Yes, interesting.

Can you create a memory dump (jmap) from the case with leak and attach it here? jmap also allows to print some statistics about memory, may be we can see there something interesting. Beside this, you can also use Yourkit to trace memory allocation/leaks, if you have it installed.

Comment 20 Oliver Meier 2018-04-24 17:24:46 EDT

(In reply to Eric Williams from comment #18)
> Interesting. You say there might be some relation to the dark theme, does
> this memory leak not happen with the regular theme?

I do not think so since it seems that the theme is kind of ignored anyway. I will set up a fedora 27 vm running gnome 3 without any changes tomorrow to check this. I will also test if the leak only occurs when using system colors. It is now 23:23h here, time to do something else (eg. sleep :P).

Comment 21 Oliver Meier 2018-04-24 17:26:00 EDT

(In reply to Andrey Loskutov from comment #19)
> Yes, interesting.
> 
> Can you create a memory dump (jmap) from the case with leak and attach it
> here? jmap also allows to print some statistics about memory, may be we can
> see there something interesting. Beside this, you can also use Yourkit to
> trace memory allocation/leaks, if you have it installed.

Thanks for the suggestions, I will try them tomorrow.

Comment 22 Oliver Meier 2018-04-24 17:32:36 EDT

BTW, over all this I forgot to mention, that I use java-1.8.0-openjdk-1.8.0.171-1.b10.fc27.x86_64, maybe it is related to the java-vm?

Comment 23 Eric Williams 2018-04-25 10:07:36 EDT

(In reply to Oliver Meier from comment #22)
> BTW, over all this I forgot to mention, that I use
> java-1.8.0-openjdk-1.8.0.171-1.b10.fc27.x86_64, maybe it is related to the
> java-vm?

Could be. MATE might also have something to do with it, not sure.

Oliver could you provide the exact steps you used to profile this issue? I.e. which tool you used to monitor the memory, etc. I'm going to let my machine run overnight and see whether I can reproduce this issue.

Comment 24 Oliver Meier 2018-04-25 11:47:32 EDT

Hi Andrey,

the jmap memory dump is 100MB. So i just added the "8 - GTK3 - new dummy(shell, SWT.NONE, true,true) - SWT-4.8M6 - RSS image" and "8 - GTK3 - new dummy(shell, SWT.NONE, true,true) - SWT-4.8M6 - jmap heap summary".

I could not figure out (in 5minutes) how to detect native memory leaks using YourKit, but I acquired a test license and set it up with eclipse.


PS: Also I figured out that I have to use the Adwaita-Dark theme in the window manager for the elements to turn dark. No other way with other theme or export GTK_THEME=Adwaita:dark

Comment 25 Oliver Meier 2018-04-25 11:47:50 EDT

Created attachment 273778 [details]
8 - GTK3 - new dummy(shell, SWT.NONE, true,true) - SWT-4.8M6 - RSS image

Comment 26 Oliver Meier 2018-04-25 11:48:52 EDT

Created attachment 273779 [details]
8 - GTK3 - new dummy(shell, SWT.NONE, true,true) - SWT-4.8M6 - jmap heap summary

Comment 27 Oliver Meier 2018-04-25 11:51:56 EDT

Created attachment 273780 [details]
RSS Log script

This python script requires python3-matplotlib-gtk3.

Usage:
 ./monitor_native_memory.py PID [optional rate in seconds]

Only X11:
 ./monitor_native_memory.py ? [optional rate in seconds]
 -> click window to monitor

Comment 28 Oliver Meier 2018-04-25 11:55:49 EDT

Eric,

I used my own cheap "RSS Log script".

I tested the application today on a fresh fc27 installation in Gnome 3 (Wayland). It showed the same leaking behaviour as on X11 Mate.

What may be useful is, that the leak does not occur when the window is minimized or the screen is locked.

Comment 29 Oliver Meier 2018-04-26 05:53:06 EDT

Created attachment 273796 [details]
9 - GTK3 - new dummy(shell, SWT.NONE, false,false) - Not leaking - Jemalloc

Test run for 10min
RSS was constant from 2minutes on at ~400MB

Jemalloc options:
 MALLOC_CONF=prof_leak:true,lg_prof_sample:0,prof_final:true
 jeprof --show_bytes `which java` *.heap

Comment 30 Oliver Meier 2018-04-26 05:56:27 EDT

Created attachment 273797 [details]
10 - GTK3 - new dummy(shell, SWT.NONE, true,true) - Leaking - Jemalloc

Test run for 10min
RSS was rising linearly up to ~1600MB after 10min

Jemalloc options:
 MALLOC_CONF=prof_leak:true,lg_prof_sample:0,prof_final:true
 jeprof --show_bytes `which java` *.heap

Comment 31 Oliver Meier 2018-04-26 05:58:41 EDT

I added some jemalloc analysis. Not sure what to see here.

Comment 32 Oliver Meier 2018-04-26 08:38:58 EDT

Created attachment 273805 [details]
9 - GTK3 - new dummy(shell, SWT.NONE, false,false) - Not leaking - RSS

Comment 33 Oliver Meier 2018-04-26 08:39:25 EDT

Created attachment 273806 [details]
10 - GTK3 - new dummy(shell, SWT.NONE, true,true) - Leaking - RSS

Comment 34 Eric Williams 2018-05-01 12:29:51 EDT

So I've been investigating further and here is what I've discovered so far.

The memory leak occurs on SWT for all GTK3.14+. I compiled on the GTK3.14 minus one commit (before the version bump) and no leak happens, so I am positive this is SWT. GTK3.14 is when we start using CSS for background colors so I suspected this to be the cause.

The memory leak doesn't happen at all if I comment out the contents of the method Control.gtk_css_provider_load_from_css(). I've tried a few theories as to why this is, but with no luck. I'll continue to investigate further.

Comment 35 Eclipse Genie 2018-05-01 15:54:39 EDT

New Gerrit change created: https://git.eclipse.org/r/121981

Comment 36 Eric Williams 2018-05-01 15:56:53 EDT

Created attachment 273877 [details]
GTK3.22 on Wayland with the patch applied

The patch from comment 35 fixes the bug. Basically, the string from gtk_css_provider_to_string() was not being freed. 

Attached is the profiler screenshot with the patch applied.

Comment 37 Eclipse Genie 2018-05-01 16:50:09 EDT

Gerrit change https://git.eclipse.org/r/121981 was merged to [master].
Commit: http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=574f2721edfda6c62b9b58519cb07216753dc055

Comment 38 Eric Williams 2018-05-01 16:51:10 EDT

(In reply to Eclipse Genie from comment #37)
> Gerrit change https://git.eclipse.org/r/121981 was merged to [master].
> Commit:
> http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/
> ?id=574f2721edfda6c62b9b58519cb07216753dc055

Patch is in master now.

Comment 39 Oliver Meier 2018-05-03 04:13:50 EDT

Thanks for the fast patch Eric!

I ran the test-application with the integration build I20180502-0100 (linux, 64-bit) over the night.
The RSS memory was stable during this time.

My real application was also running with the same swt build in over night, not showing any (obvious) memory leaks.

Oli

Comment 40 Eric Williams 2018-05-03 11:14:42 EDT

(In reply to Oliver Meier from comment #39)
> Thanks for the fast patch Eric!
> 
> I ran the test-application with the integration build I20180502-0100 (linux,
> 64-bit) over the night.
> The RSS memory was stable during this time.
> 
> My real application was also running with the same swt build in over night,
> not showing any (obvious) memory leaks.
> 
> Oli

Glad to hear the issue is resolved on your end. :)

Comment 41 Eclipse Genie 2018-05-04 10:21:30 EDT

New Gerrit change created: https://git.eclipse.org/r/122190

Comment 42 Eclipse Genie 2018-05-04 10:46:33 EDT

Gerrit change https://git.eclipse.org/r/122190 was merged to [master].
Commit: http://git.eclipse.org/c/www.eclipse.org/eclipse/news.git/commit/?id=ee4c296787649b3b4fec306a30da6fba7c7a38f2

Comment 43 Alexander Kurtakov 2018-07-13 04:18:13 EDT

*** Bug 531319 has been marked as a duplicate of this bug. ***

Comment 44 Eric Williams 2018-08-16 14:58:09 EDT

*** Bug 474104 has been marked as a duplicate of this bug. ***