Bug 568234 - [linux][gtk3] support scaled image instances
Summary: [linux][gtk3] support scaled image instances
Status: NEW
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.14   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-10-26 00:51 EDT by Christoph Laeubrich CLA
Modified: 2020-10-28 02:57 EDT (History)
2 users (show)

See Also:


Attachments
image snippet (1.78 KB, text/plain)
2020-10-26 03:37 EDT, Sravan Kumar Lakkimsetti CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Christoph Laeubrich CLA 2020-10-26 00:51:08 EDT
SWT-GTK3 currently supports loading of SVG files through ImageLoader.

The problem is that
- org.eclipse.swt.graphics.Image.getImageData(int zoom)
- org.eclipse.swt.graphics.ImageData.scaledTo(int, int)

uses rastered variants and thus the images are blurred, gtk support loading a scaled variant of a file [1], it would be really usefull if the follwoing could be added:

- when loading an ImageData from a file also determine (and save) the file-type in ImageData
- if any scaling is requested (either by Image.getImageData or ImageData.scaledTo) and the ImageData is a file-base svg a new, scaled variant is loaded.


[1] https://developer.gnome.org/gdk-pixbuf/stable/gdk-pixbuf-File-Loading.html#gdk-pixbuf-new-from-file-at-scale
Comment 1 Sravan Kumar Lakkimsetti CLA 2020-10-26 03:37:00 EDT
Created attachment 284561 [details]
image snippet

I did find we had native svg support in GTK during the development of hidpi.

Can you try the attached snippet with a svg image and see how it works? We can identify exact use cases and add necessary api.
Comment 2 Christoph Laeubrich CLA 2020-10-26 03:48:05 EDT
Thanks for the feedback I'm using the following code for testing:

public static void main(String[] args) throws IOException {

	ImageData[] imageDatas = new ImageLoader().load(new FileInputStream(args[0]));
	Display display = new Display();
	Shell shell = new Shell(display);
	shell.setText("SVG");
	shell.setLayout(new FillLayout());
	Label label = new Label(shell, SWT.NONE);
	label.setImage(new Image(display, new Image(display, imageDatas[0]).getImageData(3125)));
	// or one can use
	// label.setImage(new Image(display, imageDatas[0].scaledTo(500, 500)));
	shell.open();
	shell.pack();
	while(!shell.isDisposed()) {
		if(!display.readAndDispatch())
			display.sleep();
	}
	display.dispose();
}

For example one can use git/eclipse.platform.images/org.eclipse.images/eclipse-svg/org.eclipse.ui/icons/full/elcl16/backward_nav.svg

That shows a magnified version of the 16x16 size SVG and its clearly blurred. The same effect can be seen with the snippet you have provided, also here I think we need to take the (intial) source svg data to rerender it in different size.
Comment 3 Christoph Laeubrich CLA 2020-10-26 03:57:12 EDT
What I have forgot to mention: Currently there is no constant for the IMAGE_SVG type maybe this can be added as a first step? (currently if i print out the type it reads as 8 what is not a valid IMAGE_ constant at all).

Currently the image itself is already loaded in a byte array in  getImageDataArrayFromStream(...) so this array can be stored in the ImageData together with type = 8 so then scaling methods can call back with that array to the pix_buff methods to created scaled versions of the image.
Comment 4 Sravan Kumar Lakkimsetti CLA 2020-10-26 04:33:11 EDT
There are two implementations in SWT. loading an image using ImageData class and creating an Image object using Image class.

ImageData class has multiple loaders implemented in java itself. This does not handle svg. this is platform agnostic. We would need help in implementing SVG support here.

In case of Image class, the image class uses native api. In gtk case pixbuf/cairo. Some tweaking will be required to handle SVG. This implementation of platform specific. we need implementation from other platforms to go forward. 

Using Image class will have good performance as the image operations are delegated to native layer. But we will need to modify native build systems for this.
Comment 5 Christoph Laeubrich CLA 2020-10-26 04:37:53 EDT
(In reply to Sravan Kumar Lakkimsetti from comment #4)
> There are two implementations in SWT. loading an image using ImageData class
> and creating an Image object using Image class.
> 
> ImageData class has multiple loaders implemented in java itself. This does
> not handle svg. this is platform agnostic. We would need help in
> implementing SVG support here.

Is it possible for a plugin and/or fragment to provide an own loader? What needs to be done here?

> In case of Image class, the image class uses native api. In gtk case
> pixbuf/cairo.

That's what I'm trying to actually address with this ticket, as loading SVGs already works (I know its platform dependent here) but not scaling them (but there is pixbuf API for that as far as I can see), this would at least provide nice scaling with gtk backend.
Comment 6 Christoph Laeubrich CLA 2020-10-27 00:56:09 EDT
I have now checked and it seems what you mean(In reply to Sravan Kumar Lakkimsetti from comment #4)
> ImageData class has multiple loaders implemented in java itself. 
I think I have found the code in org.eclipse.swt.internal.image but it seems it is not used for gtk backend even worse it seems to only support a fixed set defined in FORMATS static field so it seems there is no generic way to extend supported file formats.

It seems that ImageLoader is in fact the only public entry point to load images from file/streams, do you see any chance in getting some kind of way to inject support for additional filetypes here?
Comment 7 Sravan Kumar Lakkimsetti CLA 2020-10-27 01:35:48 EDT
(In reply to Christoph Laeubrich from comment #6)
> I have now checked and it seems what you mean(In reply to Sravan Kumar
> Lakkimsetti from comment #4)
> > ImageData class has multiple loaders implemented in java itself. 
> I think I have found the code in org.eclipse.swt.internal.image but it seems
> it is not used for gtk backend even worse it seems to only support a fixed
> set defined in FORMATS static field so it seems there is no generic way to
> extend supported file formats.

This needs to be investigated. If we can add SVG format to  org.eclipse.swt.internal.image that will help all platforms. But We still need to modify ImageData class to handle scaling with SVG images.
> 
> It seems that ImageLoader is in fact the only public entry point to load
> images from file/streams, do you see any chance in getting some kind of way
> to inject support for additional filetypes here?

Please take a look at private method initNative() in Image class in gtk port. This method creates pixbuf from file. I believe this can be extended to handle streams as well.

(PS: We should get rid of pixbuf instead use cairo surfaces in gtk port. this would help in handling SVG images)
Comment 8 Christoph Laeubrich CLA 2020-10-27 02:17:06 EDT
I have a working example for loading/converting svg into ImageData including scaling support. The "problem" is that this requires apache-batik as dependency an I don't know if that is suitable.

Another idea would be to first try read the native implemented formats and the fall back to ImageIO with a conversion from BufferedImage -> Imagedata. This would not require additional dependencies and image formats (like SVG) can be plugged in with standard Java ImageIO API (and the scaling can be handled via scaledInstance(...)).

BTW I think I found a slight inconsistency in the native pixbuff loading:

> if (data_buffer.length == 0) SWT.error(SWT.ERROR_UNSUPPORTED_FORMAT);	// empty stream

and a bit below 

> if (pixbuf_animation == 0) SWT.error(SWT.ERROR_INVALID_IMAGE);

according to the description in the javadoc the error codes should be swapped here.
Comment 9 Sravan Kumar Lakkimsetti CLA 2020-10-27 03:42:41 EDT
(In reply to Christoph Laeubrich from comment #8)
> I have a working example for loading/converting svg into ImageData including
> scaling support. The "problem" is that this requires apache-batik as
> dependency an I don't know if that is suitable.
> 
> Another idea would be to first try read the native implemented formats and
> the fall back to ImageIO with a conversion from BufferedImage -> Imagedata.
> This would not require additional dependencies and image formats (like SVG)
> can be plugged in with standard Java ImageIO API (and the scaling can be
> handled via scaledInstance(...)).
> 

This is exactly what we are doing in GTK(this happens only when you use Image class with Imagefilenameprovider interface). first we try to load the Image using native implementation. If not successful we try to create ImageData using our own implementation in org.eclipse.swt.internal.image. 

We try to avoid third party dependencies in SWT. SWT needs to be a standalone jar.
Comment 10 Christoph Laeubrich CLA 2020-10-27 04:02:42 EDT
But all of this ultimately leads to a call to ImageDataLoader.load(...) that at least on GTK seem to never call back to the FileFormat implementation.

Would using ImageIO (as its pure jdk, no additional dependencies) be an option to consider? That would then be cross-platform and extendable.
Comment 11 Alexander Kurtakov CLA 2020-10-27 04:08:27 EDT
(In reply to Christoph Laeubrich from comment #10)
> But all of this ultimately leads to a call to ImageDataLoader.load(...) that
> at least on GTK seem to never call back to the FileFormat implementation.
> 
> Would using ImageIO (as its pure jdk, no additional dependencies) be an
> option to consider? That would then be cross-platform and extendable.

In theory it is if ImageIO supports svg natively. One of the principles in SWT is to not have additional dependencies as playing with classpath puts core functionality under risk.
Comment 12 Christoph Laeubrich CLA 2020-10-27 04:19:57 EDT
(In reply to Alexander Kurtakov from comment #11)
> > Would using ImageIO (as its pure jdk, no additional dependencies) be an
> > option to consider? That would then be cross-platform and extendable.
> 
> In theory it is if ImageIO supports svg natively. One of the principles in
> SWT is to not have additional dependencies as playing with classpath puts
> core functionality under risk.

I'm not sure if I understand right, but ignoring svg support for now I think it would be good to have (plug-able) support for additional format without the need to always implement them in swt-core.

ImageIO allows different deployment scenarios (e.g. you can supply a classloader, of an SWT class that would allow fragments to provide additional readers through standard SPI interface), one could supply one on-demand (that would allow using plain plugins) and you can modify the systemclasspath of your installation (e.g. for SWT standalone deployments), nothing I would consider "playing with classpath" nor risky. And nothing of this requires any additional effort from SWT-side.

If this does not succeeds, one could still throw unsupported format exception :-)
Comment 13 Christoph Laeubrich CLA 2020-10-27 04:25:37 EDT
SVG support can the be provided by some "3rdparty" bundle independent from swt.
Comment 14 Christoph Laeubrich CLA 2020-10-28 02:57:25 EDT
I think it makes sense to split this up, so I created Bug 568325 for the cross-platfrom part so we can focus here on the *native scaling* in GTK pixbuf available.