Community
Participate
Working Groups
"Justin Kilimnik" <justink@au.ibm.com> wrote in message news:anqpm6 $vll$1@rogue.oti.com... > I have been doing a lot of work lately with the Eclipse workbench and > modifications to it, and a lot of work with SWT. > > One major problem I have noticed with SWT is the Image class. Doing a > image read (1600x1200 JPEG) from the file system takes: > > 500ms in C++/Intel JPEG Graphics Lib > ~1200ms in C# with .NET classes > ~2000ms in Java using BufferedImageReader > ~4000ms in SWT using the Image class and filename name constuctor > > There seems to be no way to convert the standard Java Images to SWT images > so I am stuck with the bad performance of swt.Image. Is there anyway to > speed this up? One of the features with the Intel Graphics lib is that you > can (in simple terms) read the image size before loading the image fully > and decide if you wish to only read every 2nd, 4th, 8th, 16th pixel from > the file (which is a great feature when you are generating Thumbnails from > large images. > > Is there someone out there on the SWT team that can help with any of this? > (I am prepared to get involved with this myself as well) > >
Justin, we need you benchmark code and an image so we can establish a benchmark and a baseline. Please include the C#, AWT and SWT samples. The Intel lib, while interesting, probably has legal issues that Eclipse as a whole needs to address, not SWT. Justin, you need to add yourself to the CC field of this PR.
It also seems like there needs to be some enhancement to the code so that you can directly access the FileFormat class (ie. JPEGFileFormat) instead of it being abstracted from the developer. Then we could add enhancements to the specified file format code and use it from our code.
Created attachment 2115 [details] JPEG Image used for benchmarks
Image: DSC00581-ed.jpg (2484x1729x24bit) SWT Version: swt-win32-2108.dll Timings: C++: 280ms C#: 410ms Java: 1352ms SWT: 3065ms =============================================================================== ================== C# Code: public Bitmap GetImage() { const string METHODNAME = "GetImage"; DateTime start = DateTime.Now; Bitmap picture = (Bitmap)Bitmap.FromFile(Text); DateTime end = DateTime.Now; TimeSpan res = (end - start); logger.text(CLASSNAME, METHODNAME, "C# Time to load image: " + res.Milliseconds + "ms"); return picture; } C# BenchMark * 8/10/2002 11:04:15 AM ImageFile GetImage C# Time to load image: 410ms =============================================================================== ================== Standard Java BufferedImage Code: public BufferedImage getJavaImage(String filename) { try { long start = System.currentTimeMillis(); BufferedImage img2 = ImageIO.read(new File(filename)); img2.getType(); long end = System.currentTimeMillis(); System.out.println( "Buffered Image Read took " + (end - start) + "ms"); return img2; } catch (Exception e) { e.printStackTrace(); return null; } } Standard Java Benchmark: Buffered Image Read took 1352ms =============================================================================== ================== SWT JPEG Image Code: public Image getImage(String filename) { long start, end; start = System.currentTimeMillis(); Image img = new Image(getDisplay(), filename); end = System.currentTimeMillis(); System.out.println("Image Read took " + (end - start) + "ms"); return img; } SWT JPEG Image benchmark: Image Read took 3065ms Image: DSC00581-ed.jpg (2484x1729x24bit) SWT Version: swt-win32-2108.dll =============================================================================== ================== C# Code: public Bitmap GetImage() { const string METHODNAME = "GetImage"; DateTime start = DateTime.Now; Bitmap picture = (Bitmap)Bitmap.FromFile(Text); DateTime end = DateTime.Now; TimeSpan res = (end - start); logger.text(CLASSNAME, METHODNAME, "C# Time to load image: " + res.Milliseconds + "ms"); return picture; } C# BenchMark * 8/10/2002 11:04:15 AM ImageFile GetImage C# Time to load image: 410ms =============================================================================== ================== Standard Java BufferedImage Code: public BufferedImage getJavaImage(String filename) { try { long start = System.currentTimeMillis(); BufferedImage img2 = ImageIO.read(new File(filename)); img2.getType(); long end = System.currentTimeMillis(); System.out.println( "Buffered Image Read took " + (end - start) + "ms"); return img2; } catch (Exception e) { e.printStackTrace(); return null; } } Standard Java Benchmark: Buffered Image Read took 1352ms =============================================================================== ================== SWT JPEG Image Code: public Image getImage(String filename) { long start, end; start = System.currentTimeMillis(); Image img = new Image(getDisplay(), filename); end = System.currentTimeMillis(); System.out.println("Image Read took " + (end - start) + "ms"); return img; } SWT JPEG Image benchmark: Image Read took 3065ms =============================================================================== ================== C++ Code (Intel Performance Graphics Lib 1.2) BOOL CJPGViewDoc::OnOpenDocument( LPCTSTR lpszPathName ) { if (!CDocument::OnOpenDocument(lpszPathName)) return FALSE; JPEG_CORE_PROPERTIES image; ZeroMemory( &image, sizeof( JPEG_CORE_PROPERTIES ) ); BYTE* imageData; BeginWaitCursor(); TRY if( ijlInit( &image ) != IJL_OK ) { TRACE( "Cannot initialize Intel JPEG library\n" ); AfxThrowUserException(); } image.JPGFile = const_cast<char*>(lpszPathName); if( ijlRead( &image, IJL_JFILE_READPARAMS ) != IJL_OK ) { TRACE( "Cannot read JPEG file header from %s file\n", image.JPGFile ); AfxThrowUserException(); } // Set the JPG color space ... this will always be // somewhat of an educated guess at best because JPEG // is "color blind" (i.e., nothing in the bit stream // tells you what color space the data was encoded from). // However, in this example we assume that we are // reading JFIF files which means that 3 channel images // are in the YCbCr color space and 1 channel images are // in the Y color space. switch(image.JPGChannels) { case 1: image.JPGColor = IJL_G; image.DIBChannels = 3; image.DIBColor = IJL_BGR; break; case 3: image.JPGColor = IJL_YCBCR; image.DIBChannels = 3; image.DIBColor = IJL_BGR; break; case 4: image.JPGColor = IJL_YCBCRA_FPX; image.DIBChannels = 4; image.DIBColor = IJL_RGBA_FPX; break; default: // This catches everything else, but no // color twist will be performed by the IJL. image.DIBColor = (IJL_COLOR)IJL_OTHER; image.JPGColor = (IJL_COLOR)IJL_OTHER; image.DIBChannels = image.JPGChannels; break; } RECT desktop; ::GetWindowRect ( ::GetDesktopWindow(), &desktop ); int load = IJL_JFILE_READWHOLEIMAGE; Configuration *cft = Configuration::getInstance(); factor = 1; if(cft->optimizeSize) { if(desktop.right * 8 <= image.JPGWidth || desktop.bottom * 8 <= image.JPGHeight) { load = IJL_JFILE_READONEEIGHTH; factor = 8; } else if(desktop.right * 4 <= image.JPGWidth || desktop.bottom * 4 <= image.JPGHeight) { load = IJL_JFILE_READONEQUARTER; factor = 4; } else if(desktop.right * 2 <= image.JPGWidth || desktop.bottom * 2 <= image.JPGHeight) { load = IJL_JFILE_READONEHALF; factor = 2; } } cft->factor = factor; image.DIBWidth = image.JPGWidth / factor; image.DIBHeight = image.JPGHeight / factor; image.DIBPadBytes = IJL_DIB_PAD_BYTES (image.DIBWidth,image.DIBChannels); int imageSize = (image.DIBWidth * image.DIBChannels + image.DIBPadBytes) * image.DIBHeight; imageData = new BYTE[ imageSize ]; if( imageData == NULL ) { TRACE( "Cannot allocate memory for image\n" ); AfxThrowUserException(); } image.DIBBytes = imageData; if( ijlRead( &image, (_IJLIOTYPE) load ) != IJL_OK ) { TRACE( "Cannot read image data from %s file\n", image.JPGFile ); delete[] imageData; AfxThrowUserException(); } if( ijlFree( &image ) != IJL_OK ) { TRACE( "Cannot free Intel(R) JPEG library" ); } if(image.DIBColor == IJL_RGBA_FPX) { RGBA_FPX_to_BGRA(imageData,image.DIBWidth,image.DIBHeight); } CATCH_ALL( e ) EndWaitCursor(); ijlFree( &image ); AfxMessageBox( "Error opening JPEG file" ); return FALSE; END_CATCH_ALL // initializing incapsulated image with correct values m_imageData = imageData; m_imageDims.cx = image.DIBWidth; m_imageDims.cy = image.DIBHeight; m_imageChannels = image.DIBChannels; EndWaitCursor(); SetModifiedFlag( FALSE ); // now we have // m_imageData containing image data, and // m_imageDims with image dimensions, and // m_imageChannels with image number of channels return TRUE; } // CJPGViewDoc::OnOpenDocument C++ Benchmark ** C++ Time to load image: 280ms
Please enter your request for direct access to the FileFormat class in a separate bug report. It needs to be treated as an enhancement and also it will just get lost here.
Thanks Justin for the results. I have some more questions. Which VM did you used? Did you run the java tests from within Eclipse? Did you pass some special flags to the VM? How much physical memory do you have? Can you run the SWT test with the VM flag -verbosegc? How many GC's are executed? This depends on the VM but may give some indication. Your answers may be useful to understand why you found that SWT decoded the image twice slower than AWT. On a different config, I found the two were comparable (this does not mean it should not be improved). Maybe it's the JRE, or because we use more memory etc. I run the java tests with the following config: - VM: Classic VM (build 1.4.0, J2RE 1.4.0 IBM Windows 32 build cn140-20020902 (JIT enabled: jitc)) - Pentium 4, 512Mo of RAM, W2K and WinXP AWT code (which requires jre1.4 due to ImageIO): 1622ms, 1592ms, 1682ms SWT code: 1612ms, 1483ms, 1502ms
I was running the original tests under Eclipse (bare minimum Workspace, with a couple of custom views) from the same section of code (one one run I would use the Java ImageIO and the next time I would use SWT. Java Version: java version "1.4.1-rc" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1-rc-b19) Java HotSpot(TM) Client VM (build 1.4.1-rc-b19, mixed mode) Computer: 1GHz, 384MB, Windows XP SP1. I wrote a quick main method to run the tests one after each other (had the initial heap size set to 64MB to get comparable results: These tests were run from inside Eclipse using the VM Args: - Djava.library.path=C:\eclipse\plugins\org.eclipse.swt.win32_2.1.0\os\win32\x86 - verbosegc -Xms64MB Run 1: SWT Image Read took 2884ms [GC 41909K->38275K(65088K), 0.0269836 secs] [GC 42368K->38275K(65088K), 0.0115554 secs] [GC 42367K->38275K(65088K), 0.0035734 secs] ImageIO Image Read took 1873ms Run 2: SWT Image Read took 2905ms [GC 41909K->38275K(65088K), 0.0232798 secs] [GC 42368K->38275K(65088K), 0.0105944 secs] [GC 42367K->38275K(65088K), 0.0036077 secs] ImageIO Image Read took 1882ms Run 3 (switch read order around): [GC 16674K->13012K(65088K), 0.0200766 secs] [GC 17105K->13009K(65088K), 0.0089263 secs] [GC 17101K->13009K(65088K), 0.0028881 secs] ImageIO Image Read took 1813ms SWT Image Read took 2924ms -- The main difference we probably have that would account for the different results is that I am running a Sun JVM and you are running an IBM one?? Should I get the same one you are running and try it?
Confirmed that the VM used accounts for the difference between AWT and SWT in the particular benchmark discussed above. Further results: I run the AWT test in a loop, 10 times (thus loading the same image 10 times). I then run the SWT test in a loop, 10 times. The average time is: IBM classic 1.4.1: AWT 1213ms, SWT 1183ms SUN hotspot client 1.4.0: AWT 1362ms, SWT 2044ms IBM classic 1.3.0: AWT n/a, SWT 1153ms This particular test only illustrates how variable a result can be based on a VM/single test case/set of images. Note that it's not meant to actually compare VMs. We are interested in gathering more data to identify and improve the most useful workflows. Former comparison with programs written in other languages is also interesting.
MediaTracker/Toolkit even faster: public static Image getToolkitImage(String file) { long start = System.currentTimeMillis(); Image image = Toolkit.getDefaultToolkit().createImage(file); Component comp = new Canvas(); MediaTracker mediatracker = new MediaTracker(comp); mediatracker.addImage(image, 0); try { mediatracker.waitForID(0); } catch(InterruptedException interruptedexception) { System.out.println("Couldn't load image: " + image); } long end = System.currentTimeMillis(); System.out.println("Toolkit took " + (end - start) + "ms"); return image; } } Gives: Toolkit took 1192ms There seems to be a bit of give in the Java libs with this issue. I understand that the IBM JVM is a lot better in handling the image loading. Is there someway we could make SWT image objects with AWT Images? This would also allow us to use the Java2D libs. As for speeding up the loading even more, could we move the JPEG encoding/decoding into the swt.dll? (ie make it platform specific). This would allow SWT to compete with C# at least (which is still twice as fast as Java) and maybe be closer to the limits set by the Intel JPEG graphics lib. I would like to see SWT be the performance choice for client apps in the future and this is one area that could be significantely improved apon.
Just a note that there are two options when it comes to poofing an Image up from a bag of image bytes in natives: you can have a decoder that converts from the raw bytes to an ImageData, which would simply be a replacement of the FileFormat class; or you can go whole hog and convert the raw bytes to an Image (bypass the transient use of ImageData). It would be interesting to see the timing breakdown of raw bytes -> ImageData and then ImageData->Image in SWT to see where we get the most bang for the buck. ie, convert the call Image img = new Image(getDisplay(), filename); into what's really going on behind the covers: ImageData imgData = new ImageData(fileName); Image img = new Image(getDisplay(),imgData); and time those pieces separately. Natives to convert raw bytes to ImageData can be completely portable across os's. Natives to convert ImageData to Image is window system specific. Note on the 'use an AWT image' bit; this would of course imply that you have an AWT available. On some platforms, you might not, or have a more interesting picture of an AWT implemented on top of SWT (for example, the J9 VM on QNX w/Personal Configuration (AWT)).
After refinement of the benchmarking code (a real plugin/control) I am writing for Eclipse, I still am finding the JPEG Image SWT code to be inefficent compared against AWT (and especially C#). I have got functionally equivalent bits of code displaying over 200 thumbnails (50x50 pixels) in both AWT and SWT. AWT takes ~9 seconds, SWT takes ~28 seconds. Also, the memory consumption for the SWT component seems to be significantly higher and could have something to do with the poorer performance. This is with the IBM JVM 1.4. With the SUN 1.4 JVM AWT is exteremly faster (see previous benchmarking above). Now, I am no expert on the JVM implementations so I could be way off on this one, but there is a DLL called JPEG.DLL which is part of the JRE (in both IBM and SUN). Is it possible that AWT is using a native compiled JPEG decoder/encoder? This would account for the speed difference. If so, can SWT hook into it? or maybe SWT should have it own JPEG.DLL (and also maybe MPEG, PNG, etc). If a native library is not available on a platform the Java code should be used, otherwise the native library should be. Also, can someone explain the difference in memory consumption? Does both the ImageData and the Image object contain the image (ie one in JVM bytes, the other one in native memory??) As I said before, it seems a shame that SWT is faster at the windowing level but slower in this area (especially since images are represented as native images).
Justin, Chrix is away this week at OOPSLA. Can you include your thumnail code as an attachment and (hopefully) as a stand alone example (ie. no Eclipse). That way, the comparison will be easier. I have heard talk on the new group that SWT image scaling is slow but every time I ask for a benchmark, I never get one. This could be what you are seeing in your thumbnail code. Thanks for taking an interest.
Sorry, cannot include the code since it is being included as part of a commercial product. All the code examples I have included already are examples of the code I am using. I am not talking about thumbnail generation here (although it really slow as well!). I am just talking about loading 200 50x50 images one after another. The awt code starts from around the same memory consumption as the Eclipse product code but doesn't seem to grow as much. Will wait till Chris gets back to discuss futher.
Ok, but reading over this PR, it seems the SWT times are different based on the VM (sometimes better tham the AWT ones) but they are not out by an order of magnitude. Your latest comparison show us about 3 times slower (9 seconds vs 28 seconds) but involves more than image loading. Is the AWT code also run under Eclipse? Chrix: please attach the exact benchmark code including the main, the number of runs etc. that you are using to get your numbers. Justin: I'm not sure what to tell you. Image does not hang onto ImageData after it is created. Eclipse uses and keeps tons of memory. The only way I can see to advance here is with benchmark code (both AWT and SWT) that does the same thing (thumbnails ... whatever). Otherwise, there's no place to start. The raw image loading times are not that far out and there's probably something we can do to speed it up without resorting to custom C code.
Created attachment 2909 [details] Simple benchmark
Created attachment 2910 [details] Simple benchmark UI main class
-- Details on the benchmark -- Attached the simple benchmark written by Charlie Gracie (OTI COOP). It can be run to have a rough comparison of SWT/AWT (AWT using the faster MediaTracker/Toolkit way). This utility decodes and benchmarks all graphic files found in a given directory. My results on a dual boot win98/linux box 1.4Mhz, 512Mo, IBM 1.3.1 on both Win98 and Linux - 5 runs loading/decoding the file DSC00581-ed.jpg (i.e. each test is repeated 5 times) Win98: SWT 13520ms / AWT 7360ms Linux GTK: SWT 15008ms / AWT 7944ms On this particular jpg file, the difference between SWT using java decoding and AWT using native decoding is roughly about 2.
I just tested the difference between loading a small JPG file (9k) 500 times using AWT and SWT. On a P4 1G, with Windows XP, I get these results: SWT= 5609 ms AWT= 67855 ms Over 10 times faster on SWT seems surprisingly good news (but strange) to me. Has this issue been taken care of since the last comment? Or is the SWT code doing any caching? Here is the source code: --------------------------------------- import java.awt.image.BufferedImage; import java.io.File; import javax.imageio.ImageIO; import org.eclipse.swt.graphics.Image; import org.eclipse.swt.widgets.Display; import org.eclipse.swt.widgets.Shell; public class TestImageLoad { public static BufferedImage getJavaImage(String filename) { try { BufferedImage img2 = ImageIO.read(new File(filename)); return img2; } catch (Exception e) { e.printStackTrace(); return null; } } public static Image getImage(String filename, Display display) { Image img = new Image(display, filename); return img; } public static void main(String[] args) { Shell shell = new Shell(); Display display = shell.getDisplay(); long now = System.currentTimeMillis(); for(int i = 0; i < 500; i++){ Image img = getImage(args[0], display); img.dispose(); } System.out.println("SWT=" + (System.currentTimeMillis() - now) + "ms"); now = System.currentTimeMillis(); for(int i = 0; i < 500; i++){ BufferedImage img = getJavaImage(args[0]); img.flush(); } System.out.println("AWT=" + (System.currentTimeMillis() - now) + "ms"); } }
Running with a bigger file (230K), loading 10 times, I get less attractive results in SWT: AWT = 5328 ms SWT = 13688 ms
What AWT implementation are you using? I'm just curious since Personal Profile does not include javax.imageio.ImageIO. Are you using J2SE?
I'm trying to save the GEF figure to a jpeg file. What happens is that the figure is fairly big. The one instance I used has the figure dimension 4500 x 6500. I read the documentation in the Image class constructor that there is a limit on the image size at 16MB. Now, in order to determine the image size, I would need to know the color depth. Is the color depth something I can pass in as a parameter? If not, how do I find out the value that the framework uses? Does the framework basically use the max depth the video card supports? Is there any way to get around this? My machine has 512MB physical memory and the image doesn't seem to be unmanageable. I have seen bug reports (53632) regarding the CreateCompatibleBitmap() as people are getting out of handle error. Sometimes, I get the same thing. Another time, I seem by pass that but get out of memory problem when saving to the file using the Image loader. So, it seems running out of memory when trying to outputing the image to the byteStream (i.e. JPEGFileFormat.unloadIntoByteStream). Would increasing the heapsize help? Here is the code segment: image = new Image(Display.getCurrent(), figureSize.width, figureSize.height); gc = new GC(image); SWTGraphics graphics = new SWTGraphics(gc); figure.paint(graphics); FileOutputStream fos = new FileOutputStream(file); ImageLoader loader = new ImageLoader(); loader.data = new ImageData[] { image.getImageData() }; loader.save(fos, SWT.IMAGE_JPEG); <-- out of memory here Is there a buffered image or any other implementation that I can use to resolve this? Or is it an issue to be addressed in the future release? Thank you.
Shelly, running out of memory is a Java VM thing. It seems that most Java VM's don't simply ask the operating system for more memory and instead, throw an exception and die. Try using "-Xmx256m" to increase available memory. The discussion about "no more handles" belongs in bug 53632.
Adding my name to the cc list as we are now tracking performance issues more closely. Please remove the performance keyword if this is not a performance bug.
*** Bug 14400 has been marked as a duplicate of this bug. ***
This PR has become quite muddied over time. Some comments to hopefully get us back on track: 1) The benchmarks should be run again, on current versions of SWT. If the peformance of the SWT implementation is slower (even by as little as 30%), regardless of what VM is being run, then we should either fix it or clearly state why we think this is acceptable. 2) Patrick's comment about possibly implementing a native for the bits -> ImageData conversion should be addressed. That is, we should at least do the benchmarking to figure out how much of a difference it would make. 3) Is this still a problem for GEF, and if so is it possible to implement the code that the GEF people need (i.e. save this GEF picture as a JPEG file, etc.) without creating the single large image? Could they do this by starting from our code, with our help?
*** Bug 92199 has been marked as a duplicate of this bug. ***
Here is the image from bug 92199: https://bugs.eclipse.org/bugs/attachment.cgi?id=20177
Created attachment 26063 [details] jpeg tester
Hi all, I'm very interested with such problem, because I developped a little game which requires using of many jpeg images (up to 4*80) I made a little tool to compare SWT and SWING performance under SUN JVM (I do not have IBM one) Here are the results : start SWT test... SWT: temps total de completion:26.063 sec SWT: temps moyen par image:52.126ms disposing start SWING test... SWING: temps total de completion:9.203 sec SWING: temps moyen par image:18.406ms 3 times slower for SWT !!! I read 500 different images from the same directory (average 100 K each) I attach the code (jpeg tester) Test is run with P4 2.8 with 1.5GB RAM and SUN 1.5.0_04 It is very simple but the 200% time difference could not be explained by JVM. (or SWT code for Image class should be redevelopped) Please fix that issue (or enable tuning of function like putting type of Image, or smg else...)
Try running the Swing test case first and SWT second.
Same thing happens... start SWING test... SWING: temps total de completion:8.578 sec SWING: temps moyen par image:17.156ms start SWT test... SWT: temps total de completion:25.984 sec SWT: temps moyen par image:51.968ms disposing Again SWING is largely the fastest of the two... forgot to write that SWT version is 3.2.0.1c But I test with 3.1.38i and the results are similar...
I estimated that an other test would be important : resizing image... In my application that I developped, i used a lot of image, but i need to resize them... I add a resize after the previous test : start SWING test... SWING: total completion time: 8.672 sec SWING: average time per image: 17.344 ms SWING: total time of resizing: 0.016 sec SWING: average resizing per image: 0.032 ms start SWT test... SWT: total completion time: 25.703 sec SWT: average time per image: 51.406 ms SWT: total time of resizing:1.641 sec SWT: average resizing per image:3.282 ms disposing The way of resize for SWING is AREA AVERAGING (16), the slowest mode of SWING but the most beautiful resized image (near perfect, text on image is fully readable) The SWT method is not very beautiful, text is not readable, so I do not understand the time difference !!! other SWING mode wrote 0sec of total completion (near instant) and SWT took less than 2 sec for a pretty bad result (sorry for the word "bad" but it is ... not good at all) Maybe the JPEG or ALL Image class in SWT have to be redevelopped. It is very astonishing that SWT is slower and have a poor result... I really didn't expect that before doing the test... the time difference is about 10000% for resiszing, I hope I did a mistake in doing that test, if someone could write code and confirm that it would be good for SWT community... I think.
Hi all, new tests are done, and I have to tell about mine error : Indeed SWING dos not resize image until it is used !! So during WE, I have written few algorithms and tests them (they are optimized but not at 100%) However SWING is always faster to read Images from disks, Resizing ... start SWING test... SWING: total completion time: 8.531 sec SWING: average time per image: 17.062 ms SWING SCALE FAST: total time of resizing: 6.782 sec SWING SCALE FAST: average resizing per image: 13.564 ms SWING SCALE SMOOTH: total time of resizing: 69.968 sec SWING SCALE SMOOTH: average resizing per image: 139.936 ms SWING SCALE REPLICATE: total time of resizing: 6.719 sec SWING SCALE REPLICATE: average resizing per image: 13.438 ms SWING AREA AVERAGING: total time of resizing: 71.719 sec SWING AREA AVERAGING: average resizing per image: 143.438 ms start SWT test... SWT: total completion time: 23.672 sec SWT: average time per image: 47.344 ms SWT: total time of resizing:1.078 sec SWT: average resizing per image:2.156 ms SWT FAST : total time of resizing:19.187 sec SWT FAST : average resizing per image:38.374 ms SWT NN: total time of resizing:16.984 sec SWT NN: average resizing per image:33.968 ms SWT HERMITE : total time of resizing:21.922 sec SWT HERMITE : average resizing per image:43.844 ms SWT TRIANGLE : total time of resizing:21.75 sec SWT TRIANGLE : average resizing per image:43.5 ms SWT BELL : total time of resizing:26.016 sec SWT BELL : average resizing per image:52.032 ms SWT CUBICBSPLINE : total time of resizing:31.484 sec SWT CUBICBSPLINE : average resizing per image:62.968 ms SWT CUBICCONVOLUTION : total time of resizing:39.157 sec SWT CUBICCONVOLUTION : average resizing per image:78.314 ms SWT MITCHELL : total time of resizing:21.937 sec SWT MITCHELL : average resizing per image:43.874 ms SWT COSINE : total time of resizing:21.656 sec SWT COSINE : average resizing per image:43.312 ms SWT QUADRATIC : total time of resizing:25.906 sec SWT QUADRATIC : average resizing per image:51.812 ms SWT QUADRATIC BSPLINE: total time of resizing:25.922 sec SWT QUADRATIC BSPLINE: average resizing per image:51.844 ms SWT LANCOZ3: total time of resizing:40.234 sec SWT LANCOZ3: average resizing per image:80.468 ms SWT LANCOZ8: total time of resizing:88.016 sec SWT LANCOZ8: average resizing per image:176.032 ms SWT CATMULL: total time of resizing:30.25 sec SWT CATMULL: average resizing per image:60.5 ms SWT FAST is bilinear interpolation but not very optimized (it is separated code from the others), NN is nearest neigbour. In all previous algorithm weights are calculated for each pixels in all cases (not FAST of course) so optimizations are possible. Those results show that SWING is not so good to resize with good quality, basic resize could compete with SCALE_FAST and SCALE_REPLICATE (I think) So, sorry for "SWING resize better", that was wrong... indeed...
forgot to tell that those numbers are for 500 resize of image from 65x90 to 200x400...
Finally broke out a profiler and began looking at this. To answer Pat Mueller's question in comment 10: ...97% of time spent loading the file into the ImageData ....3% of time spent creating the Image from the ImageData So I think I'll concentrate my efforts on the file loading at this time. Will keep y'all posted.
Released > 20050927. I spent 5 half-days on this, and improved our loading speed by an average of about 8% (benched loading 133 different jpegs). The actual improvement depends on the type of JPEG, because different types take different times to load. For example, a progressive JPEG takes longer to read and requires more memory to keep each stage around so that it can be refined as loading progresses. A sequential JPEG will take less time than progressive, but some have features that require extra processing. I benched loading the two JPEG files attached to this bug, because they seemed to be particularly slow: 1) DSC00581-ed.jpg from comment 3 (2484 x 1729 = ~4M pixels) 2) LIVER_MOUSE.jpg from comment 27 (1455 x 1604 = ~2M pixels) Both are 24 bit depth. Length x width x depth gives an idea of how much memory will be required for loading. LIVER_MOUSE.jpg is progressive, which takes more cpu & memory to decode than sequential. This file now loads 5% faster. DSC00581-ed.jpg is sequential but it has quite a few restart markers, which take longer to decode. This file now loads 12% faster. I believe there may be further reduction possible in the area of progressive JPEG - particularly in the memory footprint, not sure about speed. I am leaving this bug open to have a deeper look at that some day. At this time, however, I need to move on to other things. I think at this point that the best "bang for the buck" will come from using native loading where possible, which is covered in bug 53443. Slash, I couldn't exactly run your bench from comment 28 because I don't have your images (recall that different types of JPEG have different load processing requirements). In addition, in AWT/Swing the image is actually loaded in another thread, so I think your bench needs to be a bit more like the one in bug 53443 comment 3 in order to do an accurate comparison. I tried running a modified version of your bench on my assortment of JPEGs, but there were so many OutOfMemoryErrors in the AWT/Swing loading that I couldn't do a proper bench. (Interesting to note that for quite a few large JPEGS, AWT/Swing ran out of memory loading them, and we did not... ;)
Created attachment 27626 [details] image to test with I have 500 different image like this one
Hi, first thanks for working of this subject I altered my code like in bug you indicated and results are the same, I hope a made a bug in my code... I put an image in attachement to test with (called image to test with) I test with following PC SWT3.2.0.1c jvm -Xms512m -Xmx1024m p4 2.8 1.5go ram Windows XP pro (32bits) here are the results start SWING test... SWING: total completion time: 7.546 sec SWING: average time per image: 15.092 ms start SWT test... SWT: total completion time: 24.172 sec SWT: average time per image: 48.344 ms disposing I joined the code "code.java"
Created attachment 27636 [details] code to test images reading
The image in comment 37 now loads 9.6 % faster than before, in SWT. As I mentioned in comment 36, any further significant improvements may have to come from loading native as described in bug 53443.
I tried creating different directories that contained copies of 'Akatch.jpg' and ran the benchmark pointing to each directory. The results are really wierd. SWT takes about 70-90ms per file (except for the first time ~ 300ms). AWT takes about 40-100ms (except for the first time ~ 600ms) and seems to get faster the more times the file is loaded which doesn't make sense. Here are the numbers: 1 file: start SWING test... SWING: total completion time: 0.609 sec SWING: average time per image: 609.0 ms start SWT test... SWT: total completion time: 0.313 sec SWT: average time per image: 313.0 ms disposing 10 files: start SWING test... SWING: total completion time: 1.0 sec SWING: average time per image: 100.0 ms start SWT test... SWT: total completion time: 0.875 sec SWT: average time per image: 87.5 ms disposing 20 files: start SWING test... SWING: total completion time: 1.344 sec SWING: average time per image: 67.2 ms start SWT test... SWT: total completion time: 1.687 sec SWT: average time per image: 84.35 ms disposing 100 files: start SWING test... SWING: total completion time: 4.032 sec SWING: average time per image: 40.32 ms start SWT test... SWT: total completion time: 7.171 sec SWT: average time per image: 71.71 ms disposing
Very very strange indeed ... near 10% is a good optimisation.. thanks for your work on this problem... Maybe salvation will come with native loading !!!
I guess the results are not that strange... it is because the AWT code has a much larger overhead that first time, and the numbers are just get closer to the typical time for one image as more images are added. I rewrote the test to ignore the first time, and here's what I got (very consistently): start SWING test... SWING: total completion time: 3.328 sec SWING: average time per image: 33.28 ms start SWT test... SWT: total completion time: 6.389 sec SWT: average time per image: 63.89 ms disposing I suspect that the AWT overhead is simply more classes to load or something like that. Maybe they pre-allocate some buffers, etc. I could look at that type of thing when I get back to this, but it won't be for a while.
Interesting note for Pat Mueller... I took the same test (comment 39) and timed the ImageData load and Image creation separately using currentTimeMillis. The numbers show that the Image creation is actually taking more time than I thought: 18% of total vs the 3% that I saw when running under the profiler (see comment 35). Not sure what that's about, but I suspect the profiler was hiding some data from me in some other call tree or something. Anyhow, it just shows that we could look at image creation also the next time we work on image loading performance. start SWING test... SWING: total completion time: 3.219 sec SWING: average time per image: 32.19 ms start SWT test... SWT: total time to load ImageData: 5.392 sec SWT: average time to load ImageData per image: 53.92 ms SWT: total time to create Image from ImageData: 0.969 sec SWT: average time to create Image per image: 9.69 ms SWT: total completion time: 6.361 sec SWT: average time per image: 63.61 ms disposing
Argh. The results from comments 41 to 44 were with the old SWT code loaded. Here is one more run using the new image loading code: (which puts the Image creation time up to 11% of the total) start SWING test... SWING: total completion time: 3.25 sec SWING: average time per image: 32.5 ms start SWT test... SWT: total time to load ImageData: 5.172 sec SWT: average time to load ImageData per image: 51.72 ms SWT: total time to create Image from ImageData: 0.643 sec SWT: average time to create Image per image: 6.43 ms SWT: total completion time: 5.815 sec SWT: average time per image: 58.15 ms disposing
I'd like to add some anecdotal evidence to this bug. I hope to add some hard evidence in the form on benchmarks soon. I'm writing an image browser which displays thumbnails and fullscreen versions of images, which are JPEG images. Most of the images I have are taken with my digital camera so they're 1600x1200 (2 megapixel). I find it pretty slow! On my iBook running MacOS, 1.25Ghz G4, it takes approx 1-2 seconds per image to either load+scale to fullscreen or load+scale to thumbnail. This is running the Apple JVM 1.4.2. On my Linux machine which is an Athlon XP 1800+, it takes more like 5 seconds(!), running on GCJ/GIJ 4.0. This points to a strong VM dependency, obviously, since the raw power of the two machines should be similar. I have yet to try the option of using GCJ to 'precompile' the SWT jar, that should be interesting. (A further confusing factor is that the Linux machine is running SWT 3.1 released, whereas the iBook is running CVS HEAD from a week or two ago). Personally I suspect that a pure Java solution is never going to give really good performance on all VMs (although it may be fine on VMs with really good JIT). Since SWT is already a mixed java/native library, I would strongly suggest to move the image loaders into the platform dependent section of the library. Many platforms probably offer native JPEG loading, and those that don't you can use the C JPEG reference implementation. Having said that, I will run the code through a profiler and see if there are obvious bottlenecks in the JPEGFileFormat code. Moore's law is not going to help here: Digital Camera megapixel sizes are increasing faster than CPU speeds at the moment.
Thanks, Jules. I think we all agree that going native is the way to go. I'm going to try to schedule some time for it. Just for fun, would you be able to attach one of your images to this bug? Thanks!
Created attachment 29365 [details] 2 megapixel test image (approx 500K)
Results from a directory of 73 images like the one I attached (1600x1200, around 400K each): [MacOS carbon 1.25 Ghz G4 Apple Java VM 1.4.2] start SWING test... SWING: total completion time: 29.552 sec SWING: average time per image: 404.82191780821915 ms start SWT test... SWT: total completion time: 91.198 sec SWT: average time per image: 1249.2876712328766 ms disposing
You should never save a GEF screenshot to JPEG. PNG or GIF, please. I cannot take any more blurry lines in product documentation.... JPEG is for photos! Also, have we considered using ImageMagick (or a wrapper of it) for the image backend? It supports numerous file formats and we could limit the image library dependencies to just one. My $0.02.
20051201 We released some code yesterday that might offer a good work-around to this problem for many of you. The code is in HEAD now, and it will be in the M4 milestone release. We have rewritten ONE of the SWT Image constructors to use native loading: Image(Device, String) This will help if you are loading an image using a filename string. Even if you need ImageData, it is now faster to call image.getImageData(); after loading natively using Image image = new Image(Device, String); This currently works on Windows and GTK. We will make it work natively on the other platforms also if/when we can. Here are the cases that this does not cover: - If you need to load from a stream (i.e. Java's getResourceAsStream) then you still have to use Image(Device, InputStream) or ImageData(InputStream) and you will not get native image loading (at this time). - If you need to load a multi-image file, such as an animated GIF, a multi-page TIFF, or a multi-icon ICO, then you still have to use ImageLoader.load, which uses the stream code (i.e. does not load natively). (But you *can* use Image(Device, String) if you only need the first image in the file). - If you need to progressively display a progressive JPEG or interlaced GIF/PNG, then you need to use ImageLoader.load and add an ImageLoaderListener. (But you *can* use Image(Device, String) to read read progressive and interlaced images if you don't need to display them progressively). - ImageLoader.save is not native. The new Image(Device, String) native-loading constructor loads images quite a bit faster - we have seen 6x improvement for some JPEGs. Please try it out and see if it works for you. If there are any new problems noted, please open new bug reports.
> Even if you need ImageData, it is now faster to call image.getImageData(); > after loading natively using Image image = new Image(Device, String); Note that this may change over time. If you only need the ImageData you should probably still be using ImageData data = new ImageData(String);
Hi, I test the 3.2.17 updated from CVS... I see there was change since 3.2.16 regarding Image(Display, String) I retest using the code attached few weeks ago loadig 500 images 3.2.16: SWING:7.5sec SWT:21 sec 3.2.17: SWING:7.5sec SWT:28 sec loading 1 image, swt and swing 3.2.16 and 3.2.17 are all equals I hope I made a mistake...
Hi, I ran the 2 pieces of code attached to this bug and I have two cases: - JPegTester (added 2005-09-28) uses: imdata = new ImageData(BAIS); imgTab[i]=new Image(Display.getCurrent(),imdata); This contructor has not been changed so the timing will be the same as before. - Then there is JPEGTester (added 2005-08-12) which uses Image(Device,String) contructor. This constructor is the one using native loading so I observed 1/3 of the non-native implementation. Can you confirm for me that you are using a bechmark that uses Image(Device device, String pathname)? Also, this code will run on XP since it has GDI+ by default. If you are running Windows2000 then you won't have GDI+ but you can always download. In case you don't have GDI+, it will default to the old code and the timing will be the same.
I run under winXP, jvm 5.0 I compare using JPEGTester with 3.2.16b(same times than before) and 3.2.17 (which holds changes in implementation) start SWING test... SWING: total completion time: 7.922 sec SWING: average time per image: 15.844 ms start SWT test... SWT: total completion time: 27.765 sec SWT: average time per image: 55.53 ms disposing SWT 2: total completion time: 20.156 sec SWT 2: average time per image: 40.312 ms disposing SWT is the test with Image(Display.getCurrent(),String) SWT2 is the test with imdata = new ImageData(BAIS); imgTab[i]=new Image(Display.getCurrent(),imdata); I updated via CVS and build swt 3.217 without error, and there is a real difference between the 2 constructors ..., so they are really different... I read 500 images like Ak'atch.jpg (in attached pieces -> image to test with) Am I wrong ?
Hi, Please try again with the latest from head (it will be in Milestone 4). It should work now. Thanks for pointing it out. For 240 images: start SWT test... SWT: total time of completion: 3.922 sec SWT: avarage time per image: 16.410041841004183ms start SWING test... SWING: total time of completion: 3.953 sec SWING: avarage time per image: 16.539748953974897ms I only tested with images > 100K since SWING runs out of memory but if you want to test just the SWT implementation then you can test with up to 24M (that the biggest one i tested so far).
Hi !! I retested code that is in head (after 3.2.17) SWT is slightly faster than SWING for loading 500 jpeg (size 30 to 200k) now SWING : 7.7 sec SWT:7.5 sec Exactly what you mention in previous post !! Thanks a lot for that native loading implementation for images !! (SWT is now 3 times faster than before) :)
Add my email to CC list, because I do have performance issues.
For non-native loading of JPEG images, I would concentrate on the ever-popular "baseline" encoded images (SOF0) created by digital cameras. The most obvious thing that sticks out is the call to expandImageComponents in convertYCbCrToRGB. As far as I can tell, and despite some optimization work in this area, the RGB16 palette is not actually used. In the most typical case (4:2:0), expandImageComponents will double the amount of intermediate memory used. It should be more efficient to access the Cb and Cr image components in their original, smaller form. I would be happy to list other areas where the existing code could be improved in terms of speed and memory consumption for decoding baseline JPEGs. Is this the proper forum?
Yes, thank-you, Joe. This is the place to discuss improvements (and hopefully attach patches! <grin>). When I ran the profiler on this back in the fall, one of the things that stood out is for a progressive JPEG, the size of the JPEGFileFormat.dataUnits array can be really huge. In addition - and you are correct in this - a lot of time was spent in convertYCbCrToRGB. Ask questions here, throw code and suggestions here. Thanks!
Here are more notes on performance and memory use. Note that all three test images are baseline encoded (SOF0). It's also interesting that Akatch.jpg and DSC00581-ed.jpg have restart markers (RSTn), while DSCN0288.jpg doesn't. For decoding baseline JPEGs, I think memory consumption is the main problem of JPEGFileFormat. The memory is wasted in two places: 1. JPEGFrameHeader 2. expandImageComponents Details: JPEGFrameHeader allocates a full-size buffer for each of the 3 components (Y Cr Cb), but 16x16 is the largest buffer size that is actually needed by any component (the Y component). For large images encoded at 4:2:0, this amounts to 50% waste. (Decoding a 2MB image, for example, would require an additional 1MB in component buffers -- as opposed to only a few hundred extra bytes.) expandImageComponents adds another 50% waste, by expanding the Cr and Cb buffers from quarter-size to full-size. (So decoding a 2MB image ends up requiring 4MB at runtime.) In the baseline case, these buffers could be allocated to cover a minimum coding unit (MCU) and the decoder could be written to reuse these image component buffers for each MCU. This would amount to moving the convertYCbCrToRGB method into the decodeScan method. I list a few things that can be done to tune the performance below, but I don't think any of them are as important as trimming the memory use. 1. Unzigzag the quantization tables The dequantize code looks like: int zzIndex = ZigZag8x8[i]; dataUnit[zzIndex] = dataUnit[zzIndex] * qTable[i]; But this could be simplified to: dataUnit[i] *= qTable[i]; IF the quantization tables were de-zigzagged when the DQT segment was processed. 2. Optimize nextBit I would add a nextBits(nBits) methods that could extract multiple bits at a time. This would replace the receive method. Then I would implement nextBit() as: private int nextBit() { return (currentBitCount != 0) ? ((currentByte >> --currentBitCount) & 1) : nextBits(1); } 3. extendBy is probably inefficient: if (diff < ExtendTest[t]) { return diff + ExtendOffset[t]; } else { return diff; } I expect that the array dereferencing is not as fast as private static int extend(int val, int nBits) { // assert nBits > 0; int max = 1 << (nBits - 1); int diff = val - max; return (diff >= 0) ? val : diff - max + 1; } 4. For readability, I would factor-out the BufferedInputStream aspects of JPEGFileFormat into, well, BufferedInputStream.
Here are a few more performance-related suggestions. 1. Profiler identified decodeACCoefficients and nextBit as hot spots. I would merge decodeUsingTable into decodeACCoefficients to streamline the call overhead. I would also create a special nextBits(nBits, extend) to receive multiple bits at a time and return them sign-extended if requested. This would replace extendBy(receive(nBits), nBits). Assuming the input buffering has been moved into a dedicated input stream (in), these two methods could look like: private int nextBit() throws IOException { return (currentBitCount != 0) ? ((currentByte >> --currentBitCount) & 1) : nextBits(1, false); } private int nextBits(int nBits, boolean extend) throws IOException { // assert nBits > 0; int val = 0; for (int count = nBits; count != 0; ) { if (currentBitCount == 0) { currentByte = in.readUnsignedByte(); currentBitCount = 8; // 0xFF must be followed by 0x00 if (currentByte == 0xFF && in.readUnsignedByte() != 0) throwUnsupportedImage(); } int n = (count < currentBitCount) ? count : currentBitCount; count -= n; currentBitCount -= n; val = (val << n) | ((currentByte >> currentBitCount) & ((1 << n) - 1)); } // perform sign-extension if needed if (extend && (val < (1 << (nBits-1)))) val += (-1 << nBits) + 1; return val; } NOTE: DNL handling is omitted above. 2. Some performance can be gained by moving dequantization into inverseDCT. The zero-check in inverseDCT is supposed to succeed often, and when it succeeds the dequantization of that row was wasted effort. So it's more efficient to move the dequantization until after the first zero-check.
Joe, you are awesome! I am still swamped for another week, and then I am off for March break. So I have asked Beatriz to start looking at your suggestions on Monday. If you think of anything else on the weekend, please go ahead and add it here. Thanks very much for your valuable contribution! Carolyn
I tried the new code and I have 16x improvements with JPEGs. Tell me, was it suppose to improve BMP loading or not at all? Are you planning to modify the other constructors soon?
Hi, We have been looking into modifying the other constructors but it’s not as simple as the one that was already modified. A lot of other places would need to be changed in our code structure. So we don't really have a specific time frame for these modifications. Also, the performance of bmp should have improved when using the constructor Image(display,filename) assuming you have Gdip on your machine.
(In reply to comment #65) > Hi, > We have been looking into modifying the other constructors but it’s not as > simple as the one that was already modified. A lot of other places would need > to be changed in our code structure. So we don't really have a specific time > frame for these modifications. > Also, the performance of bmp should have improved when using the constructor > Image(display,filename) assuming you have Gdip on your machine. > Gdip? What is Gdip? I'm on a Fedora Core 4 embedded platform on which we ripped many stuff. Maybe that Gdip got ripped off, because I don't see any improvments.
Hi Jean, Yeah, if you are Fedora Core 4 then GDI+ won't help. Here is a link which explain what gdip is (just in case you are curious): http://msdn.microsoft.com/library/default.asp?url=/library/en-us/gdicpp/GDIPlus/GDIPlus.asp The reason you don't see any improvement in bmps is because jpegs were using (in the old code) JPEGFileFormat for loading and this was non-efficient. So basically, unless you are in Win you won't see a big improvement in bmps.
Even if you are on Win I doubt the performance improvement would be all that awesome for BMPs, there's not much to get wrong for loading them. On Linux I suspect the performance of the GdkPixbuf loader used with the speedy constructor and our Java code is about the same.
Fixed > 20060817. We have improved the JPEG loading performance in 30% to 70% depending on the of image. Please try the latest.
reopen
fixed
Note that these performance improvements include loading a JPEG Image or ImageData from stream. It is still true that the fastest way to load a JPEG image is by using Image(device, filename_string) because this method can use natives. However, Silenio has significantly improved the performance of the non-native code also. Thanks, Silenio!
backport to 3.2.1
(In reply to comment #57) > Hi !! > > I retested code that is in head (after 3.2.17) > > SWT is slightly faster than SWING for loading 500 jpeg (size 30 to 200k) > now SWING : 7.7 sec > SWT:7.5 sec > > Exactly what you mention in previous post !! > > Thanks a lot for that native loading implementation for images !! > (SWT is now 3 times faster than before) > > > :) > Hi Slash, I am facing the same problem you were facing. I have used SWT to load 100 images of 1 MB size. I use the program code.java you have used. With Swing it gave us the result in 41 to 42 seconds. But with SWT it took more than 3 mins before it ran out of Virtual Memory (1.75 GB allocated as VM). I am not able to understand this. I am using the lattest SWT provided in Eclipse 3.2 on Windows XP on Sun JDK 1.5_06. Please suggest. Regards, Purav Gandhi
Purav, You need to get eclipse 3.2.1 (not just 3.2) in order to get this fix. I believe this will be available later this week? In the meantime, you can get the latest 3.2.1 build, which is on this page: http://download.eclipse.org/eclipse/downloads/ Look for "3.2 Stream Maintenance Build". Carolyn
