| [aspectj-dev] RE: ajc scalability (structure model) |
Thanks,
Ron DiFrango
-----Original Message-----
From: Adrian Colyer [mailto:adrian_colyer@xxxxxxxxxx]
Sent: Tuesday, September 09, 2003 6:06 AM
To: aspectj-dev@xxxxxxxxxxx
Cc: DiFrango, Ron
Subject: ajc scalability (structure model)
I've been running some scalability tests against the ajc compiler and structure model - motivated by Ron DiFrango's performance problems with larger projects in AJDT and also by our recent problems with the structure model in the head. This is a relatively long note... but please keep reading to the end as it tells us a lot about the scalability of our structure model generation in 1.1.1.
First some background. All numbers are meaningful only on my machine - what's interesting here is the relative scalability, not the absolute numbers. The project source files are synthetic - each project consists of n classes, named C_1..n, each with 10 methods. The method bodies contain a single System.out.println() - not representative of the compilation workload on a real project, but enough for these purposes. Second a warning - these numbers are all based on the ajc code in CVS head on 8th September, they cannot be used to draw any conclusions about the 1.1,0 release currently in the field (see last table for one exception to this).
Use a fixed width font for viewing the numbers that follow...
Base comparison, ajc vs javac (all times in seconds)
Project Size javac(1) ajc
10 2 2
100 3 4
500 6 8
1000 12 15
2000 22 24
4000 67 55(2)
(1) Sun JDK 1.4.1
(2) This shows ajc starting to perform better than javac... on 'real' large projects (where the method bodies cause the compiler to do more work) we see that ajc is about 10% faster than javac.
Compilation time grows linearly with project size - good.
Base comparison, structure model generation
Project Size ajc ajc -emacssym
10 2 2
100 4 4
500 8 12
1000 15 24
2000 24 55
4000 55 108
Compilation time still growing linearly with project size - good.
This shows that the compile time is roughly doubling with the -emacssym option specified. One possible explanation here is that in this case the compilation time is bounded by i/o. With the -emacssym option specified we write twice as many files to disk (one .ajesym per class). This extra file writing is not done when generating structure models for IDEs, so don't get too worried by it here, I'm simply using -emacssym as a convenient way of forcing structure model generation on the command-line, and it gives us a baseline for what follows...
Cost of adding aspects to the project - straight compile
Project Size ajc ajc +1a(1) ajc +2a(2)
10 2 3 3
100 4 7 7
500 8 15 16
1000 15 26 27
2000 24 48 51
4000 55 98 104
(1) Added a BeforeAll aspect that adds advice before all method executions (10 methods in each class).
(2) Added an AfterAll aspect that adds advice after returning from all method executions
These are very pervasive aspects. A hypothesis from these figures is that the compile time is proportional to the number of joinpoints to be woven (not the number of aspects), and stays within 150% of the no weaving (no aspects) compilation time. On 'real' projects adding a variety of aspects (some very pervasive, some less so) we see between 30% and 50% increase in compile time, which backs this up.
Cost of adding aspects to the project - with structure model generation
Project Size ajc ajc +1a ajc +2a
-emacssym -emacssym -emacssym
10 2 3 3
100 4 7 8
500 12 21 23
1000 24 74 145
2000 55 142 286
4000 108 355 779
Oh dear. This is clearly no longer linear with respect to the number of files in the project. The numbers suggest that the dominant factor is the creation of a relationship in the structure model - see how the numbers roughly double from the +1a to the +2a column. A 1000 file project with the 2 aspects added creates 20,000 relationships, and takes about the same time as a 2000 file project with one aspect, which also creates 20,000 relationships.
I thought it would be interesting to run the same tests against 1.1.0 (the old structure model generation code) to see how that fared...
Cost of adding aspects to the project - with structure model generation, ajc v.1.1.0
Project Size ajc +1a ajc +2a
-emacssym -emacssym
1000 39 45
2000 89 112
4000 227 327
This is much better than 1.1.1, but still non-linear.
(I also measured compile times without the -emacssym option, which as expected are unchanged across 1.1.0 and 1.1.1).
These numbers were all taken from CVS head. There are at least two known problems in the head, relating to changes made last Thursday / Friday:
1) the cache that Andy & I added is unfortunately missing (though I thought your email said you'd left it in Mik? - missing a few changes in the commit??)
2) *serious*: the changes made to the path for source files broke all the ide support (structure node could never be found for any file, so AJDT et al. could not display anything).
Andy, George and I are investigating and will restore the cache and IDE function, then re-run the numbers. It seems mad that the cost of creating a relationship should be dwarfing the cost of compilation and weaving, which intuitively are much more expensive tasks! Andy has also exposed the option (which Mik added - thanks) to turn off structure model generation in AJDT. This brings AspectJ compilation times back to or better than Java compiles within Eclipse - but at the cost of losing all outline and relationship information.
-- Adrian
Adrian_Colyer@xxxxxxxxxx
************************************************************************** The information transmitted herewith is sensitive information intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.