[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [aspectj-users] Announcing aUnit, a new unit testing tool for aspects
|
I agree it's an interesting challenge and it would be a valuable contribution. Building out the reflective capabilities of AspectJ will be a valuable product of this effort. It might even act as a forcing function to expose the proceed closure as part of the join point context for adviceexecution.
I also realized on thinking more about this proposal and how it interacts with other virtual mock approaches, that you could completely define all mock testing in terms of pointcuts. I.e., if you can just define mock behavior (tracking results and returning values) based on pointcuts rather than method calls, you can do mock testing of aspects and on objects consistently.
aUnit will need to simulate join points that drive aspect behavior, track pointcut executions, and simulate the environment with which the aspect interacts (for ITD's, perxxx aspect state, proceeds and possibly helper calls). I think the latter two can use pointcuts as a consistent abstraction.(*)
I think a clean design for an aUnit could also grow into a fully general virtual mock solution that would work well even for projects that aren't writing aspects. It also occurs to me that a clean mechanism for specifying externally generated pointcuts would help a lot in testing OO code that uses callbacks (e.g., frameworks).
Ron
(*) Philosophical aside: you often see OO projects configuring things based on methods and/or packages, where they really want to talk about pointcuts. I am starting to believe that perwithin aspects are mostly trying to retrofit to a pre-AO design. E.g., log4j log configurations are really configuring a set of pointcuts at which logging should be enabled. Another reason to favor reflective pointcut evaluation.
Ron Bodkin
Chief Technology Officer
New Aspects of Software
o: (415) 824-4690
m: (415) 509-2895
> ------------Original Message------------
> From: Adrian Colyer <adrian_colyer@xxxxxxxxxx>
> To: aspectj-users@xxxxxxxxxxx
> Date: Tue, Nov-9-2004 3:26 AM
> Subject: Re: [aspectj-users] Announcing aUnit, a new unit testing tool for aspects
>
> Of course, I never said it was easy (unless of course all you have to
> do
> is write the announce) ;). That's what makes this worthy of a real
> project
> with some real fun issues to get into.
>
> > However, I don't think the proposed approach would address all the
> > problems in writing pointcuts...
>
> > I also wonder about the string for tested pointcuts...
>
> > It seems to me that it would be better to have a compact form for
> > describing a concrete join point that looks more like Java code....
>
> I think both are needed (as per my original mail). There's definitely
> some
> work to be done to determine the best way to succinctly capture join
> point
> information in a form suitable for test case execution, and that also
> maximizes the chances of finding and eliminating bugs. I was certainly
> thinking that you could only specify fully concrete join points (as per
>
> Nick's suggestion). It's not a great leap of the imagination to
> envisage a
> "join point recorder" either that captures join points from some real
> program execution and lets you play them back later.
>
> > How were you thinking about implementing playBack? ...
> >
> > The other approach I can see would be to reflectively examine each
> > piece of advice in the aspect...
> >
> > The test snippet you posted also reminded me of the previous thread
> > "How to provide external pcd definition" in which you suggested a
> > reflective API with an API like this: ...
> > Did you build any of this as part of integrating with Spring?
>
> Yes I did, and I plan to do more. The implementation as I see it needs
> to
> have at its core a way of reifying join points (from some kind of test
> description). Then it could use reflective apis to get hold of say an
> org.aspectj.lang.Aspect instance (with getAdvice(), getPointcuts() etc.
>
> methods), and by exploiting the ability to ask whether a pointcut
> matches
> a given JoinPoint (already in the tree), and some internal API to drive
>
> advice in a mock context off the back of that, it could do the
> appropriate
> thing. [Not all of these apis exist right now - that's the bit about
> working with the AspectJ team I was referring too].
>
> Some of the hard challenges include:
> * how to represent cflow etc. - if the test case is a stream of join
> points, then one way to do this is to have a cflow entering join point
> in
> the stream (or not).
> * how to handle proceed in around advice
> * context passing (mock or real)
> * aspects with perxxx instantiation models
> * making the test environment as close as possible to the code paths
> that
> execute when the aspect is deployed in the wild
> * and probably a bunch more waiting to be discovered!
>
> Cheers, Adrian.
>
> -- Adrian
> Adrian_Colyer@xxxxxxxxxx
>
>
>
> "Ron Bodkin" <rbodkin@xxxxxxxxxxxxxx>
> Sent by: aspectj-users-admin@xxxxxxxxxxx
> 08/11/2004 18:26
> Please respond to
> aspectj-users@xxxxxxxxxxx
>
>
> To
> aspectj-users@xxxxxxxxxxx
> cc
>
> Subject
> Re: [aspectj-users] Announcing aUnit, a new unit testing tool for
> aspects
>
>
>
>
>
>
> This is an interesting proposal that would definitely help. I see it as
> a
> kind of virtual mocking; today one can unit test abstract aspects by
> using
> an abstract pointcut for scope and writing a concrete test aspect that
> applies to a dummy object that generates the needed join points. I.e.,
> the
> dummy object is a mock, that's testing the interface with the aspect
> (generating the needed join points).
>
> However, I don't think the proposed approach would address all the
> problems in writing pointcuts: because the test description is close to
>
> the pointcut definition, it's easy to make consistent mistakes, e.g.,
> writing "call(void Account.doFoo()) && within(org.xyz.abc)" when you
> want
> to match a call to void Account.doFoo(int)).
>
> I also wonder about the string for tested pointcuts. It seems to me
> that
> there are a whole range of possible values that couldn't be tested,
> e.g.,
> what if I put as my test jp string "call(* *(..))"? How about
> "call(void
> Runnable.run())" or "within(org..*)"? Or "cflow(baz())" or "if(foo)"?
> How
> about if the pointcut being advised has runtime checks (cflow or if)?
>
> It seems to me that it would be better to have a compact form for
> describing a concrete join point that looks more like Java code. I
> wonder
> if it wouldn't be better to use a scripting language instead (like Bean
>
> Shell or Groovy), e.g., "package x.y.z; class Account { void doFoo() {}
> }"
>
> How were you thinking about implementing playBack? I could imagine
> using
> dynamic code generation (e.g., with CGLIB), and then weaving the
> resulting
> bytecode with the given aspect. The tricky part here would be
> generating
> code that matches the specified join point.
>
> The other approach I can see would be to reflectively examine each
> piece
> of advice in the aspect (e.g., looking up the associated pointcut's
> description from class attributes) and then to write code that tests
> each
> against the given string. A benefit of this latter approach would be
> that
> you could potentially determine whether the test string always matches,
>
> never matches, sometimes matches, or matches in certain cases depending
> on
> the runtime state.
>
> I also think the design for a tool like this should be integrated with
> a
> similar API style as virtual mocks for objects (e.g., VirtualMock.org
> and
> the virtual mock framework in aTrack's library). Virtual Mock
> frameworks
> typically provide means to specify simulated behavior, including
> simulated
> return results (which would be important for testing advice too), and
> allow instrumenting objects under test to determine what methods and
> arguments were called or executed (just as this code snippet suggests
> is
> needed in assertInvoked).
>
> There's another little fly in the ointment here: because advice is not
> named, it's hard to test for a specific piece of advice being invoked.
> There is a trick available for naming advice: you can use an inner
> aspect
> with just one piece of advice to name the advice. When AspectJ supports
>
> Java 5.0, hopefully we will be able to annotate advice to give it a
> name.
> By the way, are you planning to allow annotations on advice and aspects
> in
> the first version of AspectJ with Java 5.0 support?
>
> The test snippet you posted also reminded me of the previous thread
> "How
> to provide external pcd definition" in which you suggested a reflective
>
> API with an API like this:
>
> Pointcut pcd = AspectJElementFactory.createPointcut(
> "call(void Account.doFoo()) && within(org.xyz.abc)");
>
> Did you build any of this as part of integrating with Spring?
>
> Ron
>
> Ron Bodkin
> Chief Technology Officer
> New Aspects of Software
> o: (415) 824-4690
> m: (415) 509-2895
>
>
> > ------------Original Message------------
> > From: Adrian Colyer <adrian_colyer@xxxxxxxxxx>
> > To: aspectj-users@xxxxxxxxxxx
> > Date: Mon, Nov-8-2004 8:52 AM
> > Subject: [aspectj-users] Announcing aUnit, a new unit testing tool
> for
> aspects
> >
> > How do you unit test aspects in isolation, in the same way that we
> > might
> > unit test a class?
> >
> > Current unit-testing approaches for aspects are lacking in the
> > following
> > ways:
> >
> > * you cannot easily unit test an individual aspect in isolation from
> > the
> > rest of the program
> > * you cannot easily test whether the pointcut expression associated
> > with a
> > piece of advice matches the join points you expect
> > * you cannot easily test whether the pointcut expression associated
> > with a
> > piece of advice matches unwanted join points
> > * you cannot easily test the body of advice in isolation from the
> rest
> > of
> > the program
> >
> > For example, given the following simple aspect:
> >
> > public aspect X {
> >
> > pointcut anInterestingCall() : call(* Account+.do*(..)) &&
> > within(org.xzy..*);
> >
> > before() : anInterestingCall() { ... }
> >
> > }
> >
> > We would like to write unit tests that verify:
> > * the pointcut matches a call to Account.doFoo() from within
> > org.xyz.abc
> > * the pointcut matches a call to SubAccount.doGoo(int x) from within
> > org.xyz.abc.def
> > * the pointcut does not match a call to Account.doFoo() from within
> > org.qpr
> > * after matching at a join point, the post-conditions of the advice
> > body
> > are established
> > * and so on...
> >
> > and we want to write these tests without necessarily having to create
> a
> >
> > package "org.qpr" and without having to weave and run external
> classes
> > (to
> > the one under test) in order to run the test cases.
> >
> > Enter aUnit, a seamless extension to JUnit that makes it easy to
> write
> > unit tests for aspects. aUnit works by letting the test programmer
> > specify
> > a sequence of join points (either programmatically, or parsed from a
> > string format) that are then "played back" to the aspect. It is then
> > easy
> > to test after each join point or set of join points whether the
> aspect
> > has
> > responded as desired. Contextual information at the join points
> > required
> > by any advice (such as the objects bound to this or target, or the
> > values
> > of arguments) can be supplied by the test case programmer - as either
>
> > "real" objects or as mock objects, in accordance with normal unit
> > testing
> > conventions.
> >
> > Here's an example of a simple aUnit test case:
> >
> > public void testCallMatching() {
> > String[] jps = new String[]({"call(void Account.doFoo())
> > within(org.xyz.abc)"});
> > X x = X.aspectOf();
> > playBack(jps,x);
> > assertInvoked(x,"before","1");
> > // ...
> > }
> >
> > The exact style of the test cases has yet to be finalized, but this
> > should
> > give you the idea.
> >
> > Where can I download a copy of aUnit?
> >
> > Sadly, you can't. It doesn't exist yet. The idea came to me whilst I
> > was
> > out running in the woods this lunchtime.
> >
> > BUT, aUnit would make a great project for an MSc and/or for
> open-source
> >
> > development. Some of the implementation of aUnit would need to be
> tied
> > to
> > AspectJ implementation details, and we would work with the
> implementors
> > to
> > make sure that the necessary interfaces are in place and assistance
> is
> > given. When finished, we would like aUnit to be contributed to the
> > AspectJ
> > tree under the CPL, to be included as part of the AspectJ
> distribution.
> >
> > The project contains some non-trivial challenges (like how to
> > effectively
> > test around advice containing calls to proceed) that should make it
> > very
> > interesting to work on, and could form a core part of every AspectJ
> > developers toolkit.
> >
> > It's something we'd love to work on but just don't have the time to
> > dedicate to it (hands full with Java 1.5 support).
> >
> > So - do you want to build aUnit? I'd be very interested to hear from
>
> > developers or teams keen on taking on this challenge, or indeed from
> > anyone on the list who has ideas about what aUnit should be like.
> >
> > -- Adrian
> > Adrian_Colyer@xxxxxxxxxx
> > _______________________________________________
> > aspectj-users mailing list
> > aspectj-users@xxxxxxxxxxx
> > http://dev.eclipse.org/mailman/listinfo/aspectj-users
> >
> >
>
> _______________________________________________
> aspectj-users mailing list
> aspectj-users@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/aspectj-users
>
>
> _______________________________________________
> aspectj-users mailing list
> aspectj-users@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/aspectj-users
>
>
