Bug 246340 - [Performance improvements] Slow Chart export to PDF
Summary: [Performance improvements] Slow Chart export to PDF
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: BIRT (show other bugs)
Version: 2.3.0   Edit
Hardware: PC Windows XP
: P2 major with 1 vote (vote)
Target Milestone: 2.6.0   Edit
Assignee: Yi Wang CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2008-09-05 05:28 EDT by APTND CLA
Modified: 2010-04-07 22:36 EDT (History)
4 users (show)

See Also:


Attachments
Chart (65.03 KB, application/octet-stream)
2008-09-05 05:28 EDT, APTND CLA
no flags Details
No chart (25.49 KB, application/octet-stream)
2008-09-05 05:28 EDT, APTND CLA
no flags Details
java code to run reports (2.17 KB, text/x-java)
2008-09-05 05:31 EDT, APTND CLA
no flags Details
PNG output (58.49 KB, application/pdf)
2010-01-19 02:56 EST, Yi Wang CLA
no flags Details
jpg output with quality 0.75 (94.31 KB, application/pdf)
2010-01-19 02:57 EST, Yi Wang CLA
no flags Details
jpg output with quality 0.95 (122.54 KB, application/pdf)
2010-01-19 03:01 EST, Yi Wang CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description APTND CLA 2008-09-05 05:28:09 EDT
Created attachment 111788 [details]
Chart

Hi there,

I am using BIRT to produce PDF reports. My intention is to be able to programmatically run off, say a hundred, reports in one go. This will involve using the same report design and just changing the report parameters.

I have produced sample reports using the Classic Models Inc example database. One report just contains text. The other also contains a chart in SVG mode. When I ran fifty reports off using the API, I discovered that:

   type                av. time to produce a report / ms
   no chart (c4)       2227
   chart    (c3)       7080

I think that the time difference is too great. Taking over 7s to produce a report is too long - if you have to produce a hundred reports it will take ages. I have tried using other chart formats. PNG and BMP have the same performance as SVG. JPG is much faster but the quality of the chart is too bad - it is not acceptable for our purposes.

How can I speed up the generation of reports with charts? And in general, how to speed up the reports as 2s is also very slow. I was expecting a maximum time of 500ms per report.

I have attached both my sample reports: c3 and c4
as well as my source code to run the reports.
Comment 1 APTND CLA 2008-09-05 05:28:35 EDT
Created attachment 111789 [details]
No chart

Attached report with no charts
Comment 2 APTND CLA 2008-09-05 05:31:12 EDT
Created attachment 111790 [details]
java code to run reports
Comment 3 Wei Yan CLA 2008-09-05 05:43:55 EDT
Does each execution create a new JVM? If so, we can re-use the JVM to execute
multiple reports. To start the report engine costs pretty much time.

You can changes your code as:

start engine.
for (multiple reports)
{
   IRunAndRenderTask task = engine.createRunAndRenderTask()
   execute the task
   close the task
}
close the report engine.
Comment 4 APTND CLA 2008-09-05 06:34:00 EDT
My code looks the same as what you suggested, so what change did you mean? I am only creating the engine once, but am then creating many tasks in the FOR loop. My source code is attached so have a look.

In particular, I want to know why there is such a big time difference between exporting JPG charts and the others. When I did some profiling, I found that 80% of the time was being spent in com.lowagie.text.Image.getInstance(byte[]). More specifically in com.lowagie.text.pdf.codec.PngImage.getImage(byte[]) if I used SVG,PNG or BMP charts. Why are they all using this method and do you know where the time is going? With JPG charts it uses com.lowagie.text.Jpeg.<init>(byte[]) but this does not take up any time at all.

Any suggestions?
Comment 5 Anthony CLA 2008-09-25 04:44:08 EDT
A few seconds to produce a chart seems like a lot.  Are there any tips for speeding things up or is this to be expected at the moment?  i.e. is this a performance bug or expected behaviour?
Comment 6 Wenfeng Li CLA 2008-12-08 18:01:57 EST
schedule to investigate.
Comment 7 Wenfeng Li CLA 2009-03-10 20:12:41 EDT
PDF export profiling needed, check count and performance on com.lowagie.text.Image.getInstance(byte[]).
Comment 8 Wei Yan CLA 2009-05-18 02:57:42 EDT
suggest to defer to 2.5.1, Need more time to investigate the implementation of iText image
Comment 9 Wei Yan CLA 2009-10-11 23:32:47 EDT
we need compare the performance among the following cases:

1. no chart
2. jpg chart
3. png chart
4. svg chart.

By comparing those cases, we may find if the time is used to generate the chart or embed the image into the PDF.
Comment 10 Gang Liu CLA 2010-01-06 04:01:17 EST
Following is the test result, based on 2.5.2 nightlybuild v20100106-0630
--------------first round-----------
JPG chart: 817
PNG chart: 1104
SVG chart: 1100
BMP chart: 1115
No chart: 489
--------------second round-----------
BMP chart: 1093
PNG chart: 1098
JPG chart: 784
No chart: 520
SVG chart: 1064
Comment 11 Gang Liu CLA 2010-01-06 05:06:06 EST
I write a test case to calculate the cpu time used in iText, following is result: 
PNG---getInstance: 119.64  addImage : 47.2 total: 166.84
JPG---getInstance: 0.0  addImage : 0.0 total: 0.0
BMP---getInstance: 30.94  addImage : 37.16 total: 68.1
Comment 12 Wei Yan CLA 2010-01-14 01:36:31 EST
In my test the test results are (actually SVG is handled as PNG in CHART):

WITHOUT_CHART  CHART(JPG)  CHART(PNG)  CHART(BMP)
0.4s           0.77s        1.28s      1.12s

The CPU is DUO CPU E4500 (2.20GHZ). Of course I didn't take the platform start and shutdown into account.(Please note that the platform startup/shutdown time is considered in the total time in the attached test code). 

I think the performance is acceptable.

The difference between different chart format is caused by the image format supported by the Adobe Reader. In the PDF specification, we can find that the DCTDecode is supported natively. It means we can embedded the JPG image into the PDF directly without any extra encoding/decoding. While for PNG, we must first decode the image to get the raw image data (pixel) then encode them using FlatEncode (ZIP). It causes the PNG much slower than the JPG.

The test result suggest us that:
1. use JPG if we want better performance.
2. use PNG if we want better image quality.
3. use a lossless JPG if we need both performance and image quality.

Unfortunately, JAVA doesn't support lossless JPG in standard JVM. Re-assign to CHART team to see if there are any possible for the user to control the JPG quality.
Comment 13 Yulin Wang CLA 2010-01-14 01:41:57 EST
Need to investigation if OS build can support loss-less JPG format.
Comment 14 Yi Wang CLA 2010-01-19 02:55:10 EST
As explained in Comment 12, PNG chart images need extra time for converting during exporting to pdf, therefore jpg format is much more convenient for pdf output.  

Lossless JPG is not supported, it's also not based DCT and hence will not be helpful for pdf.

The most feasible solution is to change the jpg compression quality. The default quality is 0.75, which is not enough for chart, now we set it to 0.95, with which the output looks as good as png output. 

Following table shows the TRANSIENT test result on my machine, and the pdf file size.  We can see that the compression quality of jpg affects only on the file size not the computational performance.  

                    time used    file size
--------------------------------------------
NO Chart             1050ms          -
PNG Chart            1900ms        59KB
JPG quality 0.75     1500ms        95KB
JPG quality 0.95     1515ms       123KB
Comment 15 Yi Wang CLA 2010-01-19 02:56:47 EST
Created attachment 156480 [details]
PNG output
Comment 16 Yi Wang CLA 2010-01-19 02:57:50 EST
Created attachment 156481 [details]
jpg output with quality 0.75
Comment 17 Yi Wang CLA 2010-01-19 03:01:09 EST
Created attachment 156482 [details]
jpg output with quality 0.95

The output pdf files are attached for compare.
Comment 18 Xiang Li CLA 2010-04-07 22:36:22 EDT
Verified, please refer to Comment #14