Bug 543796 - GC.drawImage does not use correct image on Mac
Summary: GC.drawImage does not use correct image on Mac
Status: NEW
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.10   Edit
Hardware: Macintosh Mac OS X
: P3 major with 2 votes (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact: Sravan Kumar Lakkimsetti CLA
URL:
Whiteboard:
Keywords:
: 564977 564978 (view as bug list)
Depends on:
Blocks: 567722
  Show dependency tree
 
Reported: 2019-01-24 12:01 EST by Phil Beauvoir CLA
Modified: 2022-04-14 07:14 EDT (History)
10 users (show)

See Also:


Attachments
Snippet demonstrating the problem (1.20 KB, application/octet-stream)
2019-01-24 12:01 EST, Phil Beauvoir CLA
no flags Details
Improved Snippet (1.07 KB, application/octet-stream)
2019-01-24 12:46 EST, Phil Beauvoir CLA
no flags Details
Simplified Snippet (1.36 KB, text/plain)
2019-01-24 17:31 EST, Phil Beauvoir CLA
no flags Details
Fixed Snippet (1.48 KB, text/plain)
2019-01-24 17:37 EST, Phil Beauvoir CLA
no flags Details
Simplified Snippet that will crash on Big Sur (980 bytes, text/plain)
2020-10-08 13:26 EDT, Phil Beauvoir CLA
no flags Details
Crash log (86.93 KB, text/plain)
2020-10-08 13:55 EDT, Phil Beauvoir CLA
no flags Details
Crash log in different context (90.12 KB, text/plain)
2020-10-08 14:04 EDT, Phil Beauvoir CLA
no flags Details
Custom Thumbnail impl with workaround for crash on BigSur (11.94 KB, application/octet-stream)
2020-10-09 06:30 EDT, Justin Dolezy CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Phil Beauvoir CLA 2019-01-24 12:01:03 EST
Created attachment 277272 [details]
Snippet demonstrating the problem

Attached is a snippet that demonstrates this problem.

It draws an image of a coloured box onto a GC with drawImage() twice in two x, y positions.

The first time the image is filled with a red colour, the second the image is filled with a blue colour.

On Windows this works as expected. On Mac it does not. Both image renders are red.
Comment 1 Phil Beauvoir CLA 2019-01-24 12:46:52 EST
Created attachment 277273 [details]
Improved Snippet

Made a new Snippet. The old one had some Draw2D code in it. This one is pure SWT.
Comment 2 Phil Beauvoir CLA 2019-01-24 17:31:55 EST
Created attachment 277276 [details]
Simplified Snippet

Edited Snippet to be clearer and with comments.
Comment 3 Phil Beauvoir CLA 2019-01-24 17:37:50 EST
Created attachment 277277 [details]
Fixed Snippet

Fixed Snippet
Comment 4 Phil Beauvoir CLA 2019-03-05 07:51:39 EST
Note - works as expected on Linux. So it's a Mac problem.
Comment 5 Lakshmi P Shanmugam CLA 2019-04-03 07:31:35 EDT
Sravan, can you please investigate?
Comment 6 Sravan Kumar Lakkimsetti CLA 2019-04-03 23:18:43 EDT
This snippet is crashing on Linux at 200%. 

@niraj,
Can you check on windows?
Comment 7 Sravan Kumar Lakkimsetti CLA 2019-04-04 00:24:31 EDT
This is reproducible in 4.10. 

After testing the attached snippet, I found if I comment first drawImage call in the paint method I can see blue color box. the second draw was working correctly. 

I feel this might be related to retina display. Can someone check on non retina display?
Comment 8 Phil Beauvoir CLA 2019-04-04 04:53:42 EDT
I checked on non-retine display on Mac. Same problem.

This snippet was created to try to narrow down a problem that appears in a Draw2d class org.eclipse.draw2d.parts.ScrollableThumbnail that uses image caching by using more than one GC.

My theory is that MacOS is caching image data.
Comment 9 Sravan Kumar Lakkimsetti CLA 2019-04-04 05:00:50 EDT
(In reply to Phil Beauvoir from comment #8)
> I checked on non-retine display on Mac. Same problem.
> 
> This snippet was created to try to narrow down a problem that appears in a
> Draw2d class org.eclipse.draw2d.parts.ScrollableThumbnail that uses image
> caching by using more than one GC.
> 
> My theory is that MacOS is caching image data.

I am thinking in the same lines too. Need to look into drawInRect call to flush the cache
Comment 10 Phil Beauvoir CLA 2020-05-18 08:59:44 EDT
(In reply to Sravan Kumar Lakkimsetti from comment #9)
> (In reply to Phil Beauvoir from comment #8)
> > I checked on non-retine display on Mac. Same problem.
> > 
> > This snippet was created to try to narrow down a problem that appears in a
> > Draw2d class org.eclipse.draw2d.parts.ScrollableThumbnail that uses image
> > caching by using more than one GC.
> > 
> > My theory is that MacOS is caching image data.
> 
> I am thinking in the same lines too. Need to look into drawInRect call to
> flush the cache

I wonder if there is some way that one flush this cache?
Comment 11 Sravan Kumar Lakkimsetti CLA 2020-06-30 02:42:20 EDT
Investigated further and found that NSImage caches the image. To clear this we need to call recache()(https://developer.apple.com/documentation/appkit/nsimage/1519890-recache). Still working on where to call this.
Comment 12 Phil Beauvoir CLA 2020-06-30 06:41:39 EDT
(In reply to Sravan Kumar Lakkimsetti from comment #11)
> Investigated further and found that NSImage caches the image. To clear this
> we need to call
> recache()(https://developer.apple.com/documentation/appkit/nsimage/1519890-
> recache). Still working on where to call this.

Thanks for looking at this. :-)
Comment 13 Mario Marinato CLA 2020-07-06 15:19:32 EDT
I'm facing the same problem.  

Plus, I discovered the same problem happens if you save the image to a file. My code, simplified:

    Image myImg = new Image( Display.getDefault(), 100, 100 );
    GC mygc = new GC( myImg );

    // draw something

    // Saves the image to a file.
    // The image on the file is correct.
    ImageLoader iLoader = new ImageLoader();
    iLoader.data = new ImageData[] { image.getImageData() };
    iLoader.save( "aName.png", SWT.IMAGE_PNG );

    // draw something else on the same image.

    // Saves the image to another file.
    // The image on the file is the original one.
    ImageLoader iLoader = new ImageLoader();
    iLoader.data = new ImageData[] { image.getImageData() };
    iLoader.save( "anotherName.png", SWT.IMAGE_PNG );

    myImg.dispose();
    mygc.dispose();

I have just discovered that it happens only when DPIUtil.getDeviceZoom() returns 200.  When it returns 100, the file is saved correctly.
Unfortunately, the snippet Phil provided still faces the problem, even when getDeviceZoom() return 100.
Comment 14 Andrey Loskutov CLA 2020-07-06 15:35:30 EDT
(In reply to Mario Marinato from comment #13)
> I'm facing the same problem.  
> 
> Plus, I discovered the same problem happens if you save the image to a file.

So bug 564977 is duplicate of this one?
Comment 15 Mario Marinato CLA 2020-07-06 15:46:40 EDT
(In reply to Andrey Loskutov from comment #14)
> (In reply to Mario Marinato from comment #13)
> > I'm facing the same problem.  
> > 
> > Plus, I discovered the same problem happens if you save the image to a file.
> 
> So bug 564977 is duplicate of this one?

I think so.  Bug 564978 is also a duplicate (I have no idea why two bugs were created, I haven't even noticed 564977 before you modified it.)
Comment 16 Andrey Loskutov CLA 2020-07-06 15:59:45 EDT
*** Bug 564978 has been marked as a duplicate of this bug. ***
Comment 17 Andrey Loskutov CLA 2020-07-06 16:00:13 EDT
*** Bug 564977 has been marked as a duplicate of this bug. ***
Comment 18 Sravan Kumar Lakkimsetti CLA 2020-08-03 06:14:20 EDT
Looks like this is a very old problem see https://bugs.eclipse.org/bugs/show_bug.cgi?id=339132#c42

When I applied the same fix here. I am getting a crash. calling recache() did not help here. still investigating
Comment 19 Phil Beauvoir CLA 2020-10-08 13:21:09 EDT
Sravan, I tested this snippet again using Big Sur public beta 5 (20A5384c). 

Running this snippet and resizing the shell window to force a repaint crashes completely with a EXC_BAD_ACCESS (SIGABRT)

This snippet actually represents what goes on in some Draw2d code used in diagrams, so the crash can happen in Eclipse Ecore editors.
Comment 20 Phil Beauvoir CLA 2020-10-08 13:26:45 EDT
Created attachment 284390 [details]
Simplified Snippet that will crash on Big Sur

Simplified snippet that crashes on Big Sur.

Run Snippet and resized the window to force a repaint.
Comment 21 Phil Beauvoir CLA 2020-10-08 13:55:54 EDT
Created attachment 284392 [details]
Crash log

Crash log attached.
Comment 22 Phil Beauvoir CLA 2020-10-08 14:04:38 EDT
Created attachment 284393 [details]
Crash log in different context

Here's a crash log where the code is similar to the snippet but used in a different context.
Comment 23 Justin Dolezy CLA 2020-10-08 19:17:45 EDT
We've just dealt with a similar crash in our own GEF Outline view impl - seems Big Sur doesn't like cached GC's - we moved creation to be local to the point of use as a workaround. You could try moving the gcImage into the PaintListener, may well not crash anymore.

We actually had a similar crash happening in Nebula Grid a while back, might be of interest..

- https://bugs.eclipse.org/bugs/show_bug.cgi?id=529920
- https://github.com/eclipse/nebula/commit/08f1d8cbefa850253d4220282a4d3755c1213566#diff-fcf16e0f3b1ffac849ceac8449de48dc
Comment 24 Phil Beauvoir CLA 2020-10-08 19:29:05 EDT
Thanks for the info, Justin.

We see the crash in a Draw2d class - "org.eclipse.draw2d.parts.Thumbnail"

Here's our hacked version of that class (contains workarounds for the original issue here of cached GC):

https://github.com/archimatetool/archi/blob/master/com.archimatetool.editor/src/org/eclipse/draw2d/archi/parts/Thumbnail.java

It crashes at line 220 on Big Sur:

thumbnailGC.drawImage(tileImage, 0, 0, sx2 - sx1, sy2 - sy1, sx1,
                    sy1, sx2 - sx1, sy2 - sy1);
Comment 25 Justin Dolezy CLA 2020-10-09 04:44:19 EDT
Yes, that's what we had also. The workaround I mentioned in the previous post works for us - moving thumbnailGC to not be allocated as a class attribute but allocated locally on the stack directly in the run() method and disposed every time.

We found that tileGC then caused the same crash so we had to move that locally also (& associated tileGCGraphics, tileGraphics).

Presumably this has performance implications (haven't measured the difference), but it's better than a crash!

Moving the GC instance from the heap to the stack feels like a memory management issue in the native code, someone with Cocoa knowledge & access to Big Sur would need to investigate..

Perhaps the crash should be raised as a separate bug to the original image caching problem?

BTW as you've a GEF app also, once you stop the crashing do let me know if you're also seeing strange rendering issues on Big Sur! For example, switching between editors does not repaint the Outline and revealed editor - their client areas are simply black! Seems there's a missing paint event, replicated in the IDE's editor also, see here:

- https://bugs.eclipse.org/bugs/show_bug.cgi?id=567615
Comment 26 Phil Beauvoir CLA 2020-10-09 04:54:04 EDT
(In reply to Justin Dolezy from comment #25)
> Yes, that's what we had also. The workaround I mentioned in the previous
> post works for us - moving thumbnailGC to not be allocated as a class
> attribute but allocated locally on the stack directly in the run() method
> and disposed every time.
> 
> We found that tileGC then caused the same crash so we had to move that
> locally also (& associated tileGCGraphics, tileGraphics).
> 
> Presumably this has performance implications (haven't measured the
> difference), but it's better than a crash!
> 
> Moving the GC instance from the heap to the stack feels like a memory
> management issue in the native code, someone with Cocoa knowledge & access
> to Big Sur would need to investigate..
> 
> Perhaps the crash should be raised as a separate bug to the original image
> caching problem?
> 
> BTW as you've a GEF app also, once you stop the crashing do let me know if
> you're also seeing strange rendering issues on Big Sur! For example,
> switching between editors does not repaint the Outline and revealed editor -
> their client areas are simply black! Seems there's a missing paint event,
> replicated in the IDE's editor also, see here:
> 
> - https://bugs.eclipse.org/bugs/show_bug.cgi?id=567615

Is your source code available for the Thumbnail?

Yes, I sometimes saw a black GEF/Draw2d editor client area when switching.

Another things is white banding on toolbar buttons - Bug 567613

Sometimes even opening a GEF/Draw2d editor crashed - See also - Bug 567722

I've removed the Big Sur beta from my partition for now, I'll wait for final release before spending time on it.
Comment 27 Justin Dolezy CLA 2020-10-09 06:30:53 EDT
Created attachment 284399 [details]
Custom Thumbnail impl with workaround for crash on BigSur

My product's not open source but I've attached our custom Thumbnail implementation - take a look at ThumbnailUpdaterAdapter (extends ThumbnailUpdater) where you'll see the changes I described.

There's definitely a bunch of SWT issues on Big Sur, glad to hear you've repro'd a lot of them and I'm not going mad :)

I very much doubt anything will change on the Apple side to magically fix this stuff but would be nice of course lol. Looks like we'll have to tell our customers to avoid upgrading to Big Sur if they're dependant on our app..
Comment 28 Phil Beauvoir CLA 2020-10-15 15:07:55 EDT
(In reply to Justin Dolezy from comment #25)
> Yes, that's what we had also. The workaround I mentioned in the previous
> post works for us - moving thumbnailGC to not be allocated as a class
> attribute but allocated locally on the stack directly in the run() method
> and disposed every time.
> 
> We found that tileGC then caused the same crash so we had to move that
> locally also (& associated tileGCGraphics, tileGraphics).
> 
> Presumably this has performance implications (haven't measured the
> difference), but it's better than a crash!
> 
> Moving the GC instance from the heap to the stack feels like a memory
> management issue in the native code, someone with Cocoa knowledge & access
> to Big Sur would need to investigate..
> 
> Perhaps the crash should be raised as a separate bug to the original image
> caching problem?
> 

I applied this workaround to our hacked Thumbnail class and this works on Big Sur Beta 10. Without the workaround it's a total crash to desktop.

But the underlying cause is something to do with GC.

(@Justin what we ideally need is a re-worked Thumbnail class that doesn't use GCs like this)
Comment 29 Phil Beauvoir CLA 2020-10-29 15:57:05 EDT
I have better news.

MacOS Big Sur 11.0.1 beta was released today and I tested this issue. The original problem still exists but it doesn't crash to desktop.
Comment 30 Sravan Kumar Lakkimsetti CLA 2020-11-04 08:21:06 EST
Actually root cause for this problem is same as bug 339132. 

I did look into the using recache, but it turned out we actually set CacheMode as never. So recache did not work here.

As a workaround we need to recreate gc for the temp image every time  like this

        shell.addListener(SWT.Paint, event -> {
            GC gc = event.gc;

            // Draw red background on image
            GC gcImage = new GC(image);
            gcImage.setBackground(red);
            gcImage.fillRectangle(0, 0, 128, 128);
            gc.drawImage(image, 0, 0);
            gcImage.dispose();
    
            // Draw blue background on image and move x position
            GC gcImage = new GC(image);
            gcImage.setBackground(blue);
            gcImage.fillRectangle(0, 0, 128, 128);
            gc.drawImage(image, 128, 0);
            gcImage.dispose();
        });
Comment 31 Justin Dolezy CLA 2020-11-05 11:15:05 EST
We've rebased our SWT fork to catch up with the great work Lakshmi has done :)

Just wanted to query this thing I mentioned before:

> For example, switching between editors does not repaint the Outline and revealed editor - their client areas are simply black!

With the latest code (we've rebased to Y20201102) the draw2d editor area is always repainted and no longer is black.

However the thumbnails in the outline view are!

It only seems to be occur after switching perspectives - doing so and then changing to different editor makes the Outline view client area black..

@Phil - are you seeing this also with the out-of-the-box implementation?
Comment 32 Matthias Becker CLA 2021-10-29 02:17:34 EDT
What's the state of this issue?
Comment 33 Nikita Nemkin CLA 2022-04-14 05:54:54 EDT
I've investigated the issue and I think using a work-around is preferable to fixing it.

# Why it happens

The basic image primitive on macOS is CGImage. It wraps a raw image buffer in a given format and it is _immutable_. There is no such thing as drawing directly to an image on macOS. Graphics context always draws to its own freshly allocated buffer. For drawing the result, the graphics context buffer is copied to a fresh CGImage. This copying only happens _once_, the resulting CGImage is cached and there is no API to force a refresh. (NSGraphicsContext.flush does an entirely different thing). All this machinery is hidden inside NSImage/NSBitmapImageRep.

# What can be done

SWT GC could switch to a lower level API (CGBitmapContext) and manage CGImage creation manually. Unfortunately, SWT Image is already a mess and this addition won't improve things.

# Why this use case is troublesome

An Image that's simultaneously written to and read from is a problem in itself. Native graphics APIs don't allow this. At the very least, some king of explicit flush is required, but SWT GC has no such API. SWT could add this API or it could synchronize before every drawImage or it could track image modification and flush conditionally. None of the options seem appealing to me compared to simply disallowing the drawing of Images which have an open GC.

Allocating a local GC for every drawing session should be relatively cheap, compared to the drawing itself. Is it not the case for you? Are GEF graphics contexts so heavy as to require caching?
Comment 34 Phil Beauvoir CLA 2022-04-14 07:14:01 EDT
Thanks very much for this analysis, Nikita. Your advice is very valuable. For the GEF Thumbnail case I think it will be up to the committers there to decide what to do.