Community
Participate
Working Groups
Build ID: N/A Steps To Reproduce: For images that are millions of pixels, it is not always practical to use ImageLoader and ImageData to load such a large image into memory. We would like these classes to support tile-based image computation so that things like loading, scaling, rotating, saving, and other similar operations can be computed in a progressive series of tiles. Image operations such as scaling, rotating, and filtering should be supported in terms of generalized image operators. An image operator plugin registry should be supported so that new image processing operations may be added as desired that also can take advantage of tile-based image computation. One facet of this is that ImageLoaders should also be image operators and should accept plugin-able types (see bug 44779). Native acceleration of image operations should also be supported when the given platform makes this available. More information:
This is something that we might consider tackling in the 3.5 timeframe. Mark, can you recommend some of the API that you would envision working with? For example, ImageLoader already has the concept of an array of ImageData. Would one tile == one ImageData? And then Image could have a new constructor that takes an array of ImageData to create a single Image? Would an ImageData need to know that it was a "tile", or would it not care? In short, anything you can think of from the point of view of an API consumer would help us get started. At this point, though, please note that we will not be able to start until after 3.4 ships.
Carolyn, I will give this some deep thought for a couple of weeks and get back to you with a detailed recommendation for design and API.
Please bounce your ideas off of us as you go, to keep us in the loop! Thanks!
(Sorry to take longer on this than I had anticipated...it's been a crazy last couple of months.) > Would an ImageData need to know that it was a "tile", or would it not care? I think that an ImageData may not care if it is a tile. I think that each tile will need coordinates that identify where it is positioned within the tiled image. This will allow us to define API for a tiled image that can get and set tiles on-demand. However, I think that a particular tile need not care about what its own tile coordinates are...that can be the concern of a peer class that coordinates tile getting and setting. > Would one tile == one ImageData? And then Image could have a new constructor > that takes an array of ImageData to create a single Image? I think sometimes yes and sometimes no. More specifically, I think that many times you just want to load a (small) image as just one tile, and in that case you may have many tiles that comprise one ImageData that becomes one Image (see getImageData() below). Other times, you may want to load, render, or save very large Images as a set of Images, each with its own ImageData (see getTile(x,y) below). For example, if an image is much bigger than the screen, a very efficient way to render it is to only load and render the tiles that are visible at any one time (a la Google Maps). This would be a case where each 256x256 pixel tile == one ImageData => one Image. > Mark, > can you recommend some of the API that you would envision working with? I think part of the API in the AWT interfaces called RenderedImage and WritableRenderedImage are a good model to look at as an example for how this feature could be designed. Obviously, the AWT classes such as Raster, SampleModel, ColorModel, and DataBuffer would have to be made into SWT equivalents. I will attempt to stub out some interfaces here: public interface TiledImageData { /*** based on RenderedImage ***/ /** * The collection of source TiledImageData such as might * come from an ImageLoader or an collection of Images that * were drawn on by a GC. */ Collection<TiledImageData> getSources(); /** get methods about image properties */ Object getProperty(String name); String[] getPropertyNames(); /** get methods about the image coordinate system */ int getWidth(); int getHeight(); int getMinX(); int getMinY(); /** get methods about the tile coordinate system */ int getNumXTiles(); int getNumYTiles(); int getMinTileX(); int getMinTileY(); int getTileWidth(); int getTileHeight(); int getTileGridXOffset(); int getTileGridYOffset(); /** * Returns tile (tileX, tileY). Note that tileX and tileY are indices * into the tile array, not pixel locations. * @param tileX the X index of the requested tile in the tile array * @param tileY the Y index of the requested tile in the tile array * @return the tile given the specified indices. */ ImageData getTile(int tileX, int tileY); /** * @return the entire image as one large tile. */ ImageData getImageData(); /** * Computes and returns an arbitrary region of the TiledImageData. * The ImageData returned is a copy of the image data and will not * be updated if the image is changed. * @param rect the region of the TiledImageData to be returned. * @return the region of the <code>TiledImageData</code> * indicated by the specified <code>Rectangle</code>. */ ImageData getData(Rectangle rect); /** * Computes an arbitrary rectangular region of the TiledImageData * and copies it into a caller-supplied ImageData. The region * to be computed is determined from the bounds of the supplied * ImageData. * @param imagedata an ImageData to hold the returned portion of the * image, or null. * @return a reference to the supplied or created ImageData. */ ImageData copyData(ImageData imageData); /*** based on WritableRenderedImage ***/ /** * Sets a rect of the image to the contents of the ImageData imageData, * which is * assumed to be in the same coordinate space as the TiledImageData. * The operation is clipped to the bounds of the TiledImageData. * @param imageData the specified <code>ImageData</code> */ public void setData(ImageData imageData); /*** There are TileObservers and methods for locking tiles for writing *** in WritableRenderedImage that I don't know whether we need. ***/ } A good question to think about would be whether we would try to implement this TiledImageData interface by subclassing ImageData, or just writing a new class to do it. I would try to steer us toward writing a new class because one of the other things that we would like this feature to support would be to model images that are based on other primitive types than byte[] arrays. For digital photography we frequently work with >8 bits per color channel and we would strongly desire to support images with bands of byte, short, int, float, and double arrays. Also, the maximum number of bands should be unlimited. Certainly, if you want to render an image on the screen, the usual constraints of 32 bits per pixel or less and 1-3 color bands plus alpha apply...so it is the job of the programmer to convert any image data that doesn't meet these requirements into a version that does so that it can be rendered.
Please take a look at the animated GIF support in SWT. Particularly, these 3 fields in class ImageLoader: /** * the array of ImageData objects in this ImageLoader. * This array is read in when the load method is called, * and it is written out when the save method is called */ public ImageData[] data; /** * the width of the logical screen on which the images * reside, in pixels (this corresponds to the GIF89a * Logical Screen Width value) */ public int logicalScreenWidth; /** * the height of the logical screen on which the images * reside, in pixels (this corresponds to the GIF89a * Logical Screen Height value) */ public int logicalScreenHeight; Snippet141, Snippet194, and the ImageAnalyzer example all have sample code for reading and writing animated GIFs. I am wondering if we can "shoehorn" image tiling into the same API somehow, using the ImageData array to store the tiles, and the "logicalScreenWidth/Height" to store the size of the full image. Thoughts?
I would say at this point that this feature request is no longer needed. Better to spend time on other things. :)
Mark, how did you work around this? Do you convert to awt images and use jai api? Did you roll your own tiff encoder/decoder?
Yes, we've gone with a hybrid JAI/AWT solution in our app where we use JAI for all the heavy lifting and SWT to build any image we need to actually display. I'd close this requirement and plan for more important things.
Thanks, Mark. I'm reluctant to close the requirement because it would still be a useful addition to the SWT toolkit, and would save folks having to load up AWT and SWT at the same time if all they need from AWT/JAI is tile capability for large images. I guess you are using JAI for more than just tiling? Does JAI support working with more than 8 bits per color channel (bands)? i.e. even if we provided tiles and a way to encode/decode to/from TIFF, would you still need to use AWT/JAI for other things? Are you still storing tiles in TIFF, or do you use JPEG tile capability as well?
Glad to hear you're still open to providing a tiled image framework! For encoding/decoding, we're using mainly JPEG2000 for normal image data (8-bit & 16 bit) and also TIFF for floating point image data. We're also using JAI as a pipeline of image operations that are chained together with an in-memory tile cache. It is capable of chaining together a series of operations such as reading multiple images and overlaying them, applying a contrast stretch, and other operations. I think SWT could be used in a similar way (although it's very nice that JAI provides an in memory cache for the tiled results of its operations). JPEG2000 support is very important for our applications.
