[aspectj-dev] Spring cleaning in the tests department.... (committers pl

[Adrian Colyer <adrian_colyer@xxxxxxxxxx>]

It all started out simply enough. I wanted to write some unit tests for a
bug I'd been working on, and the test harness didn't quite support what I
wanted to do (or at least I couldn't figure out how to make it do what I
wanted it to do - one of the two anyway).

I wrote a helper class org.aspectj.tools.ajc.AjcTestCase which is now in
org.aspectj.ajdt.core/testsrc/ . This serves as a base class for any
programmatic JUnit test you want to write to drive the compiler. In the
same package under the testsrc folder are the helper classes it uses - Ajc
(to drive normal and incremental compiles - giving you full control over
when the next increment should occur), the CompilationResult class, and a
couple of others. I javadoc'd them as i think they're generally useful -
the doc is in tests/doc (the tests module).

Well, it was only a short step from there...
....to write another TestCase class that extends AjcTestCase, called
XMLBasedAjcTestCase. Given a suite file (of the ajcTests.xml format), it
allows you to run any individual test under JUnit.  This enabled me to do
some tidy up of the tests in the tests module. I divided the tests in
ajcTests.xml up into logical batches (using some archeology to help) - they
can now be found in src/org.aspectj.systemtest.xxx/xxx.xml, where xxx is
one of:

ajc121,ajc120,ajc11,ajc10x,pre10x,base,design,incremental,serialVerUID,xlint,options,aspectpath,inpath,java14,pureJava,
knownLimitations

These files are all included by a now much shorter (textually) ajcTests.xml
file so that you can run the harness as normal - but there's also now an
alternative to this mode of operation: In each of the
org.aspectj.systemtest.xxx packages is a JUnit test case class that runs
all the tests in that suite. These are bundled together into  AllTests (a
suite that can run under JDK 1.3), and AllTests14 (includes all the 1.3
tests plus the 1.4 specific ones). By launching one of these using JUnit
you can run all of the tests - with each individual ajc-test being it's own
JUnit test (for existing xml spec files, I wrote a class MakeTestCase in
the testing module that generates the JUnit test case classes, you should
never need to run this though because I've done them all).  This mode of
operation has a number of advantages:

* integrates better into Eclipse
* easy to see which tests have failed (normal JUnit view)
* easy to re-run or debug any failed test
* easy to run any individual test

all just using the normal JUnit facilities.

In run-all-junit-modules I checked in the Suite
"RunTheseBeforeYouCommitTests" that runs all the unit tests from all the
modules (including the ones from 'tests') - try it, you'll like it - much
easier to work with.

I wrote a much simpler and smaller set of classes to parse the ajc suite
files (still using the Digester) - these don't have the full checking and
sophisticated option handling of the harness ones, but they do contain all
of the function needed to run every single test case in the suite. The
exception is the incremental tests - I didn't add support for inc-compile.
Instead take a look at org.aspectj.systemtest.incremental.IncrementalTests
which demonstrates the really nice feature of being able to mingle
XML-based and programmatic testing in the same test. This makes the
incremental tests much easier to write and will finally enable me to write
some of the incremental tests I've been putting off since 1.2,

For new tests that you add as part of the 1.2.1 development, please add the
tests into the org.aspectj.systemtest.ajc121/ajc121-tests.xml file, and to
the org.aspectj.systemtest.ajc121.Ajc121Tests test case class. Since you
can easily run individual tests from JUnit, this should alleviate the need
to maintain your own xxxTests.xml files and cut-and-paste tests to and from
ajcTests.xml all the time.

On the way, I trimmed out a few tests in ajcTests.xml that were pure-java
tests (and that didn't have the pureJava keyword). These are all in the
suite PureJavaTests.xml, along with the ones that did have the keyword.
They are *not* included in the default suites, but with a little tidying up
(of the tests with the 'pureJava' keyword, that clearly haven't been run
for a while), these could be added to a release build test suite.

-- Adrian.
adrian_colyer@xxxxxxxxxx