Re: [aspectj-users] Announcing aUnit, a new unit testing tool for aspect

["Ron Bodkin" <rbodkin@xxxxxxxxxxxxxx>]

This is an interesting proposal that would definitely help. I see it as a kind of virtual mocking; today one can unit test abstract aspects by using an abstract pointcut for scope and writing a concrete test aspect that applies to a dummy object that generates the needed join points. I.e., the dummy object is a mock, that's testing the interface with the aspect (generating the needed join points).

However, I don't think the proposed approach would address all the problems in writing pointcuts: because the test description is close to the pointcut definition, it's easy to make consistent mistakes, e.g., writing "call(void Account.doFoo()) && within(org.xyz.abc)" when you want to match a call to void Account.doFoo(int)). 

I also wonder about the string for tested pointcuts. It seems to me that there are a whole range of possible values that couldn't be tested, e.g., what if I put as my test jp string "call(* *(..))"? How about "call(void Runnable.run())" or "within(org..*)"? Or "cflow(baz())" or "if(foo)"? How about if the pointcut being advised has runtime checks (cflow or if)? 

It seems to me that it would be better to have a compact form for describing a concrete join point that looks more like Java code. I wonder if it wouldn't be better to use a scripting language instead (like Bean Shell or Groovy), e.g., "package x.y.z; class Account { void doFoo() {} }"

How were you thinking about implementing playBack?  I could imagine using dynamic code generation (e.g., with CGLIB), and then weaving the resulting bytecode with the given aspect. The tricky part here would be generating code that matches the specified join point.

The other approach I can see would be to reflectively examine each piece of advice in the aspect (e.g., looking up the associated pointcut's description from class attributes) and then to write code that tests each against the given string. A benefit of this latter approach would be that you could potentially determine whether the test string always matches, never matches, sometimes matches, or matches in certain cases depending on the runtime state.

I also think the design for a tool like this should be integrated with a similar API style as virtual mocks for objects (e.g., VirtualMock.org and the virtual mock framework in aTrack's library). Virtual Mock frameworks typically provide means to specify simulated behavior, including simulated return results (which would be important for testing advice too), and allow instrumenting objects under test to determine what methods and arguments were called or executed (just as this code snippet suggests is needed in assertInvoked).

There's another little fly in the ointment here: because advice is not named, it's hard to test for a specific piece of advice being invoked. There is a trick available for naming advice: you can use an inner aspect with just one piece of advice to name the advice. When AspectJ supports Java 5.0, hopefully we will be able to annotate advice to give it a name. By the way, are you planning to allow annotations on advice and aspects in the first version of AspectJ with Java 5.0 support?

The test snippet you posted also reminded me of the previous thread "How to provide external pcd definition" in which you suggested a reflective API with an API like this:

Pointcut pcd = AspectJElementFactory.createPointcut(
  "call(void Account.doFoo()) && within(org.xyz.abc)");

Did you build any of this as part of integrating with Spring?

Ron

Ron Bodkin
Chief Technology Officer
New Aspects of Software
o: (415) 824-4690
m: (415) 509-2895


> ------------Original Message------------
> From: Adrian Colyer <adrian_colyer@xxxxxxxxxx>
> To: aspectj-users@xxxxxxxxxxx
> Date: Mon, Nov-8-2004 8:52 AM
> Subject: [aspectj-users] Announcing aUnit, a new unit testing tool for aspects
>
> How do you unit test aspects in isolation, in the same way that we 
> might 
> unit test a class?
> 
> Current unit-testing approaches for aspects are lacking in the 
> following 
> ways:
> 
> * you cannot easily unit test an individual aspect in isolation from 
> the 
> rest of the program
> * you cannot easily test whether the pointcut expression associated 
> with a 
> piece of advice matches the join points you expect
> * you cannot easily test whether the pointcut expression associated 
> with a 
> piece of advice matches unwanted join points
> * you cannot easily test the body of advice in isolation from the rest 
> of 
> the program
> 
> For example, given the following simple aspect:
> 
> public aspect X {
> 
>   pointcut anInterestingCall() : call(* Account+.do*(..)) && 
> within(org.xzy..*);
> 
>   before() : anInterestingCall() { ... }
> 
> }
> 
> We would like to write unit tests that verify:
> * the pointcut matches a call to Account.doFoo() from within 
> org.xyz.abc
> * the pointcut matches a call to SubAccount.doGoo(int x) from within 
> org.xyz.abc.def
> * the pointcut does not match a call to Account.doFoo() from within 
> org.qpr
> * after matching at a join point, the post-conditions of the advice 
> body 
> are established
> * and so on...
> 
> and we want to write these tests without necessarily having to create a 
> 
> package "org.qpr" and without having to weave and run external classes 
> (to 
> the one under test) in order to run the test cases.
> 
> Enter aUnit, a seamless extension to JUnit that makes it easy to write 
> unit tests for aspects. aUnit works by letting the test programmer 
> specify 
> a sequence of join points (either programmatically, or parsed from a 
> string format) that are then "played back" to the aspect. It is then 
> easy 
> to test after each join point or set of join points whether the aspect 
> has 
> responded as desired. Contextual information at the join points 
> required 
> by any advice (such as the objects bound to this or target, or the 
> values 
> of arguments) can be supplied by the test case programmer - as either 
> "real" objects or as mock objects, in accordance with normal unit 
> testing 
> conventions. 
> 
> Here's an example of a simple aUnit test case:
> 
> public void testCallMatching() {
>    String[] jps = new String[]({"call(void Account.doFoo()) 
> within(org.xyz.abc)"});
>    X x = X.aspectOf();
>    playBack(jps,x);
>    assertInvoked(x,"before","1");
>     // ...
> }
> 
> The exact style of the test cases has yet to be finalized, but this 
> should 
> give you the idea.
> 
> Where can I download a copy of aUnit?
> 
> Sadly, you can't. It doesn't exist yet. The idea came to me whilst I 
> was 
> out running in the woods this lunchtime. 
> 
> BUT, aUnit would make a great project for an MSc and/or for open-source 
> 
> development. Some of the implementation of aUnit would need to be tied 
> to 
> AspectJ implementation details, and we would work with the implementors 
> to 
> make sure that the necessary interfaces are in place and assistance is 
> given. When finished, we would like aUnit to be contributed to the 
> AspectJ 
> tree under the CPL, to be included as part of the AspectJ distribution. 
> 
> The project contains some non-trivial challenges (like how to 
> effectively 
> test around advice containing calls to proceed) that should make it 
> very 
> interesting to work on, and could form a core part of every AspectJ 
> developers toolkit.
> 
> It's something we'd love to work on but just don't have the time to 
> dedicate to it (hands full with Java 1.5 support). 
> 
> So - do you want to build aUnit?  I'd be very interested to hear from 
> developers or teams keen on taking on this challenge, or indeed from 
> anyone on the list who has ideas about what aUnit should be like.
> 
> -- Adrian
> Adrian_Colyer@xxxxxxxxxx
> _______________________________________________
> aspectj-users mailing list
> aspectj-users@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/aspectj-users
> 
>