[platform-core-dev] About the performance of jars and compression : some

["Philippe Ombredanne" <pombredanne@xxxxxxxx>]

Hi,
following a short email exchange with Jeff, I am posting that to the
list.
The topic is: class/resource loading from jars compressed or not, or
file system, and performance implications.
Some of you may have some valuable insights.

The rational I remember was:
- when loading many classes, dealing with jars is more efficient than
dealing with deployed .class, avoiding the overhead of many file systems
calls.
- uncompressed jars do not have to be decompressed

I made some research about performance of jarred vs. jarred compressed
vs. deployed class & resource loading and report.
An interesting reference can be found here :
http://safari.oreilly.com/JVXSL.asp?xmlid=0-596-00377-3/javapt2-CHP-3-SE
CT-12
Now in contradiction with that, the folks of knopflerfish/osgi advocate
for the opposite: a deployed .class format But they really are in the
embedded world. http://www.knopflerfish.org/download.html

Typically, JRE & JDK jars are shipped as uncompressed, like the
monstruous rt.jar. There must be a reason.

I am really not sure how this fares as it is always a compromise between
IO, memory and CPU.
A lot of pointer in that field are related to embedded devices and
tradeoffs between network bandwidth, CPU and memory/storage available on
small devices.

If done right this could have beneficial implication for Eclipse startup
and class-loading.
Comments are welcomed.....


jeff_mcaffer@xxxxxxxxxx wrote:
>Compressed vs. uncompressed is an interesting observation.  Can you
produce some numbers on this? 
> The direction for M6 is to make plugins (and fragments) ship as
self-contained Jars.  
Jeff,
Just as a note on that topic, I remember reading somewhere that class
loading from jars was faster than class loading from a class in the file
system, and that loading from uncompressed jars was also faster than
from compressed jars (no uncompress overhead).
Now if the above is true:
- The regular jars built with Eclipse plugins are compressed: this is
bad...
- The jarred plugins distro made by the pde export are compressed too:
this is
double bad...
Just my 2 cents, as those simple optimizations of the builds could make
overall class loading faster and Eclipse zippier without getting the
downloads bigger, as you want the downloads compressed.

BTW, there is a slight reduction in size when you compress an
uncompressed jar compared with having a jar whose individual entries
are compressed. Looks like the zip algorithms are able to make some
optimizations when working on larger uncompressed files.
So if you were to leave jars uncompressed, you may get:
- faster class loading i.e. faster overall eclipse
- slightly smaller compressed downloads (not a big deal)

But do not take my word for it!

-- 
Cheers
Philippe

philippe ombredanne | nexB - Open by Design (tm)
1 650 799 0949 | pombredanne at nexb.com 
http://www.nexb.com