Bug 354978 - Use Cairo as backend for graphics operations
Summary: Use Cairo as backend for graphics operations
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 3.8   Edit
Hardware: PC Linux-GTK
: P3 normal (vote)
Target Milestone: 3.8   Edit
Assignee: Arun Thondapu CLA
QA Contact:
URL:
Whiteboard:
Keywords: plan
Depends on:
Blocks: 340067 368543
  Show dependency tree
 
Reported: 2011-08-17 12:06 EDT by Silenio Quarti CLA
Modified: 2012-06-07 18:09 EDT (History)
12 users (show)

See Also:


Attachments
Changes to GC.copyArea() to use Cairo image surfaces (5.62 KB, patch)
2011-11-08 09:40 EST, Arun Thondapu CLA
no flags Details | Diff
Fix GC.copyArea() variant for USE_CAIRO and update CAIRO_OPERATOR_SRC (6.18 KB, patch)
2011-11-28 12:54 EST, Arun Thondapu CLA
no flags Details | Diff
Possible patch for Tree/Table drag source effects (3.17 KB, patch)
2012-02-02 13:16 EST, Arun Thondapu CLA
no flags Details | Diff
Eclipse Toolbar problem with Cairo enabled (29.40 KB, image/png)
2012-02-03 06:09 EST, Arun Thondapu CLA
no flags Details
WindowBuilder pallette renders incorrectly. (135.39 KB, image/png)
2012-06-07 18:09 EDT, Alexander Mitin CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Silenio Quarti CLA 2011-08-17 12:06:08 EDT
In order to keep current with GTK3, we need to rewrite parts of SWT graphics to stop using GdkGC, GdkPixmap, GdkBitmap, and GdkRegion for GTK 2.22 and greater.
Comment 1 Remy Suen CLA 2011-08-17 13:08:10 EDT
I don't think we are starting a Win32/GTK port. :)
Comment 2 Silenio Quarti CLA 2011-08-17 13:54:32 EDT
Just to clarify: this work is intended to get our Linux GTK port running better on recent versions of GTK where the old X based graphics API has been deprecated. This is a move towards supporting GTK3 but, for this release, we are not planning a full GTK3 port.
Comment 3 Silenio Quarti CLA 2011-10-12 10:56:27 EDT
The work for this bug has stared and is being done on this branch:

http://git.eclipse.org/c/platform/eclipse.platform.swt.git/log/?h=Bug354978
Comment 4 Bogdan Gheorghe CLA 2011-10-12 11:31:18 EDT
Some context into this work:

Currently. GTK GC operations are a combination of gdk Xlib calls and Cairo calls. The Cairo calls are currently used when advanced graphics are needed. In the most recent versions of GTK (2.22 and higher), they have stopped using the Xlib graphics backend in favour of Cairo. We already have Cairo calls for GC, but our Image class is still written in terms of GdkPixmap. This needs to be rewritten to use Cairo surfaces.

As  mentioned previously, we have created a branch in both repos (eclipse.platform.swt and eclipse.platform.swt.binaries) called Bug354978 that contains a first pass at this. The work done so far:

http://git.eclipse.org/c/platform/eclipse.platform.swt.git/log/?h=Bug354978
http://git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/log/?h=Bug354978

- Added an OS flag USE_CAIRO_SURFACE which is true if the GTK version > 2.18 (might need to change version to 2.22, we can decide later)

- In GC, we added 2 checks to initCairo and setCairoClip to see if a drawable exists. (A GdkDrawable can be either a GdkWindow or a GdkPixmap.) Since we are using cairo surfaces, we no longer have a drawable for Images. In places where drawable was used, an alternative must be found. There are still places we have not fixed (for example: GC.copyArea()). A good way to see what still needs to be done is to do a search on all references to Image.pixmap and GCData.drawable

- rewrote Image to use cairo image surfaces everywhere (http://cairographics.org/manual/cairo-Image-Surfaces.html). This approach might not perform well given that images are stored in the client side and all the pixels need to the be transferred over the wire when the image is drawn (in case of remote connections).   This has to be confirmed still.   An alternative is to create all images that are opaque with the gdk_window_create_similar_surface().   This should create a cairo surface with a XLIB backend. These images are stored in the server side.

http://developer.gnome.org/gdk/stable/gdk-Cairo-Interaction.html#gdk-window-create-similar-surface

If we decide to use gdk_window_create_similar_surface(), we will have to find a way to implement Image.getImageData(), Display.createPixbuf(), etc.  These methods are currently relying on the fact that the Image.surfaceData is always set, but this is not going to be the case anymore.    I believe we can create a temporary image surface and copy the XLIB surface data into the temporary image using cairo_paint(). I am not sure it is the only (or best) option.
Comment 5 Arun Thondapu CLA 2011-11-08 09:40:12 EST
Created attachment 206598 [details]
Changes to GC.copyArea() to use Cairo image surfaces

I'm still testing the changes with the various possible scenarios.
It works with a couple of snippets I tried with.
Will update after doing more testing.
Comment 6 Arun Thondapu CLA 2011-11-09 07:08:44 EST
(In reply to comment #5)
> Will update after doing more testing.

I have tested all the code flows in this patch with various use cases like copying from one image surface to another and copying from a display/window/widget to an image surface. I also tested with different versions of GTK from 2.16 to 2.24 and everything seems to work fine for me.
Comment 8 Silenio Quarti CLA 2011-11-14 11:25:36 EST
Review:

1) Need to add a NO_MORE_HANDLES check after cairo_create()

2) I am not able to take a screenshot using Snippet215.  The GTK version I am running is gtk2-2.18.9-6.el6.i686 (RHEL6).  Does it work for you?  I believe this is happening because we do not have the equivalent of 

OS.gdk_gc_set_subwindow(gdkGC, OS.GDK_INCLUDE_INFERIORS);

for cairo.

3) Shouldn't we set the cairo operation to CAIRO_OPERATOR_SOURCE. I do not think alpha blending should happen in copyArea(). Please check to other platforms.

4) Need a special case for gc on Printer since there is no data.image nor data.drawable in that case. For now, just avoid the warnings/crash. The previous code (GdkGC) does not work either.
Comment 9 Arun Thondapu CLA 2011-11-14 13:13:06 EST
(In reply to comment #8)
> Review:
> 
> 1) Need to add a NO_MORE_HANDLES check after cairo_create()

Will add the check.

> 2) I am not able to take a screenshot using Snippet215.  The GTK version I am
> running is gtk2-2.18.9-6.el6.i686 (RHEL6).  Does it work for you?  I believe
> this is happening because we do not have the equivalent of 
> 
> OS.gdk_gc_set_subwindow(gdkGC, OS.GDK_INCLUDE_INFERIORS);
> 
> for cairo.

I had tested with this screenshot snippet many times and it was working for me. I'm fairly sure I tested with GTK versions 2.16, 2.18, 2.18.7 and 2.24.
Is it crashing for you or are you getting a blank image?

I'll confirm about points 3 & 4 and then provide an update again.
Thanks for the feedback!
Comment 10 Silenio Quarti CLA 2011-11-14 13:37:28 EST
(In reply to comment #9)
> I'm fairly sure I tested with GTK versions 2.16, 2.18, 2.18.7 and 2.24.
> Is it crashing for you or are you getting a blank image?

blank image.
Comment 11 Arun Thondapu CLA 2011-11-15 07:48:27 EST
(In reply to comment #8)
> 1) Need to add a NO_MORE_HANDLES check after cairo_create()
Added the check.

> 2) I am not able to take a screenshot using Snippet215.  The GTK version I am
> running is gtk2-2.18.9-6.el6.i686 (RHEL6).  Does it work for you?  I believe
> this is happening because we do not have the equivalent of 
> 
> OS.gdk_gc_set_subwindow(gdkGC, OS.GDK_INCLUDE_INFERIORS);
> 
> for cairo.
Confirmed again that it works for me. I'm not sure if GDK_INCLUDE_INFERIORS is a problem since the doc says that Cairo will always use this setting on sources and masks and GDK_CLIP_BY_CHILDREN on targets. There may be some other problem on your setup.

> 3) Shouldn't we set the cairo operation to CAIRO_OPERATOR_SOURCE. I do not
> think alpha blending should happen in copyArea(). Please check to other
> platforms.
I have checked the Mac and Windows code and you're right that they use operators which directly copy from source to destination without any alpha blending at all. I have made the necessary change.

> 4) Need a special case for gc on Printer since there is no data.image nor
> data.drawable in that case. For now, just avoid the warnings/crash. The
> previous code (GdkGC) does not work either.
Is it okay in that case to just have a check for the Printer device and return without doing anything?
Comment 12 Arun Thondapu CLA 2011-11-28 12:54:38 EST
Created attachment 207614 [details]
Fix GC.copyArea() variant for USE_CAIRO and update CAIRO_OPERATOR_SRC

This patch includes changes referred in comment 11 and further changes in GC.copyArea() variant method.

In my testing, using the same Cairo surface as source and destination for copying via cairo_set_source_surface() is working without having to use cairo_push_group()/cairo_pop_group_to_source().
However, this might require more in-depth testing. As of now, I have tested with some modified sample snippets as I could not find any existing usage scenarios for this copyArea() method. The paint event handling is not fixed yet.
Comment 13 Silenio Quarti CLA 2012-01-17 17:57:44 EST
(In reply to comment #12)
> Created attachment 207614 [details]
> Fix GC.copyArea() variant for USE_CAIRO and update CAIRO_OPERATOR_SRC
> 
> This patch includes changes referred in comment 11 and further changes in
> GC.copyArea() variant method.
> 
> In my testing, using the same Cairo surface as source and destination for
> copying via cairo_set_source_surface() is working without having to use
> cairo_push_group()/cairo_pop_group_to_source().
> However, this might require more in-depth testing. As of now, I have tested
> with some modified sample snippets as I could not find any existing usage
> scenarios for this copyArea() method. The paint event handling is not fixed
> yet.

The changes in GC.copyArea(Image, int, int) and the renaming of CAIRO_OPERATOR_SRC were fine and have being released to the branch.

The implementation of GC.copyArea(int, int, int, int, int, int, boolean) had some issues:

- calling gdk_x11_drawable_get_xid(drawable) forces GTK to create the native X Window for the drawable. We do not want this because it defeats GTK and it also causes an expose event for the whole widget.

- the GC handle needs to be used to do the copy bits instead of a new cairo instance otherwise the GC state (for example, clipping) is not used.

When we removed the call to gdk_x11_drawable_get_xid() (use gdk_cairo_set_source_window instead) and the creation of the new cairo instance, we had to add the cairo_push_group()/cairo_pop_group_to_source() otherwise it would not work for the drawable case, but it still worked for the image case.

Arun, please do some testing. I will attach the testcase we used.

By the way, we have decided that USE_CAIRO will be true only when running with gtk 2.24 (cairo 1.10) for a couple of reasons:

- there are bugs in cairo 1.8.8 with CAIRO_OPERATOR_DIFFERENCE
- gdk_cairo_set_source_window is only available in 2.24
Comment 14 Silenio Quarti CLA 2012-01-17 18:00:57 EST
Testcase for GC.copyArea:


import org.eclipse.swt.graphics.GC;
import org.eclipse.swt.widgets.*;
import org.eclipse.swt.*;

public class PR90087 {
	public static void main(String[] a) {
		final Display display = new Display();
		final Shell shell = new Shell(display);
		shell.setSize(800, 800);

		Composite c = new Composite(shell, SWT.NONE);
		c.setBackground(display.getSystemColor(SWT.COLOR_GREEN));
		c.setBounds(25, 25, 50, 50);

		Composite c1 = new Composite(shell, SWT.NONE);
		c1.setBackground(display.getSystemColor(SWT.COLOR_BLUE));
		c1.setBounds(35, 35, 500, 500);

		final Composite comp = c1;
		Listener listener = new Listener() {
			boolean paint = true;

			public void handleEvent(Event e) {
				switch (e.type) {
				case SWT.Paint: {
					if (paint) {
						e.gc.setBackground(display.getSystemColor(SWT.COLOR_BLUE));
						e.gc.setForeground(display.getSystemColor(SWT.COLOR_RED));
						e.gc.fillGradientRectangle(0,  0, 500, 500, false);
						e.gc.drawLine(0, 0, 1000, 1000);
					} else {
						System.out.println("PAINT");
						e.gc.setBackground(display.getSystemColor(SWT.COLOR_BLACK));
						e.gc.fillRectangle(-100, -100, 1000, 1000);
					}
					break;
				}
				case SWT.MouseDown:
					paint = false;
					GC gc = new GC(comp);
					gc.copyArea(-10, -10, 110, 110, 100, 0, true);
//					gc.copyArea(250, 250, 250, 250, 250, 0, true);
					gc.dispose();
					break;
				}
			}
		};
		comp.addListener(SWT.Paint, listener);
		comp.addListener(SWT.MouseDown, listener);
		shell.open();
		while (!shell.isDisposed()) {
			if (!display.readAndDispatch())
				display.sleep();
		}
		display.dispose();
	}
}
Comment 15 Silenio Quarti CLA 2012-01-17 18:06:51 EST
Arun, please work next in the senders of Image.gtk_new(Device, int, int pixmap, int mask). There are a couple of cases in DND. 

We need to convert the pixmap and mask into a cairo surface.
Comment 16 Bogdan Gheorghe CLA 2012-01-19 16:00:10 EST
We have merged the Cairo changes into master and have added a system property flag to enable the new code. By default, the code does not run.

For those who want to try out the new code (and help us out with testing), you need to:

1) start eclipse with this vm arg (can be added to the end of the eclipse.ini):

-Dorg.eclipse.swt.internal.gtk.cairoGraphics=true

2) Ensure that your GTK version >= 2.24, otherwise the code is not enabled.

We plan to exercise this code as much as possible during the next milestone and make it the default for M6.

There are still some outstanding items on the "to do" list to be able to fully support GTK 3 (on the graphics side), but these should not impact GTK 2 behavior. 

The most prominent items are:

i)  Not able to draw directly to the screen - new GC(display). 

ii) No drag source effects for tree and table


These changes will be in the upcoming M5 build.
Comment 17 Paul Webster CLA 2012-01-20 08:46:54 EST
(In reply to comment #16)
> 1) start eclipse with this vm arg (can be added to the end of the eclipse.ini):
> 
> -Dorg.eclipse.swt.internal.gtk.cairoGraphics=true
> 
> 2) Ensure that your GTK version >= 2.24, otherwise the code is not enabled.
> 

Does it not make sense to support cairo prior to 2.24?  My RHEL 6 based system (on which I've had some of these errors) only has:
bash-4.1$ rpm -q gtk2
gtk2-2.18.9-4.el6.x86_64
gtk2-2.18.9-4.el6.i686

PW
Comment 18 Arun Thondapu CLA 2012-02-02 13:16:02 EST
Created attachment 210467 [details]
Possible patch for Tree/Table drag source effects

Instead of replicating the code to convert the drag source effect image from pixmap(s) to a Cairo surface in both TreeDragSourceEffect and TableDragSourceEffect, I'm reusing the existing Image.createSurface() method in Image.gtk_new() for creating the image surface for the drag icon.

From GTK 3.x, the gtk_tree_view_create_row_drag_icon() method itself returns a Cairo image surface directly and so we can get rid of the pixmap completely then.

I have tested this code with a few DND use cases like DNDExample and Snippet91 and it is working well for me. Please review and let me know if this is okay or if you think it is a better idea to include the code for creating the image surface from pixmap(s) directly in TreeDragSourceEffect and TableDragSourceEffect.

I have also removed the code which creates an Xlib surface in GC.copyArea(Image, int, int) as it will remain dead code if USE_CAIRO is only enabled for GTK versions  2.24 and above (since gdk_cairo_set_source_window() is available from GTK 2.24).
Comment 19 Arun Thondapu CLA 2012-02-03 06:09:03 EST
Created attachment 210504 [details]
Eclipse Toolbar problem with Cairo enabled

I have been testing 4.2 M5 with Cairo graphics enabled and there seems to be a problem with the toolbar. As shown in the screenshot, there is some lingering shadow/highlighting effect on the icons whenever I hover and move away. This doesn't happen when I launch Eclipse with -Dorg.eclipse.swt.internal.gtk.cairoGraphics=false.
Comment 20 Silenio Quarti CLA 2012-02-06 15:03:24 EST
(In reply to comment #18)
> Created attachment 210467 [details]
> Possible patch for Tree/Table drag source effects
> 

Thanks Arun. I committed the patch:

http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=d7e72ee1edd674fa1cd0705e281c907c9694b440

Note that I removed the changes in GC. I am still not convinced that we need GTK 2.24.   If it turns out to be the case, we can remove that code later. We might be able to run the code with GTK 2.18 as long as we have cairo 1.10 (or we work around the bugs we found in cairo 1.8 (ie. CAIRO_OPERATOR_DIFFERENCE)).
Comment 21 Silenio Quarti CLA 2012-02-06 15:35:48 EST
(In reply to comment #19)
> Created attachment 210504 [details]
> Eclipse Toolbar problem with Cairo enabled
> 
> I have been testing 4.2 M5 with Cairo graphics enabled and there seems to be a
> problem with the toolbar. As shown in the screenshot, there is some lingering
> shadow/highlighting effect on the icons whenever I hover and move away. This
> doesn't happen when I launch Eclipse with
> -Dorg.eclipse.swt.internal.gtk.cairoGraphics=false.

Fixed (it is just a typo...)

http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=8d9180667f57a1f516b2b01d133df4e4d98f8466
Comment 22 Silenio Quarti CLA 2012-06-04 12:09:04 EDT
Marking this as fixed since we have made all the changes we planned.

For further work on GTK3 support see bug#340067.
Comment 23 Alexander Mitin CLA 2012-06-07 18:07:51 EDT
It seems that it still has drawing issue (tried with latest integration build). See https://bugs.eclipse.org/bugs/show_bug.cgi?id=368543#c10
Comment 24 Alexander Mitin CLA 2012-06-07 18:09:29 EDT
Created attachment 217057 [details]
WindowBuilder pallette renders incorrectly.