[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[virgo-dev] Re: Test framework for MultiThreaded tests
- From: Olivier Girardot <ssaboum@xxxxxxxxx>
- Date: Sat, 14 Aug 2010 00:58:26 +0200
- Delivered-to: email@example.com
- Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=Ug/Ua0Yv9IqZFZ7Fzrx31Odv/dbxbqkbqPa1cSio0gmkwu5zzJSaAR+h18VnDU1vFa o5iXtPnL990SRARBFKmKvJ9/sNO30fz4/jw88cD0HJVBYuM0xjJ8yHEX7EhHyEi923hr 1mstU6sq6hism7zuirHxYlmwE5Fq/3ie4qJPw=
i did a little bit of testings (...yes, again) but this time with MultiThreadedTC and did a blog post on it and my opinion.
To sum it up, i think this test framework can greatly help several points :
* handlind errors
* handling thread synchronization
* improve tests "easiness"
i present a comparison between two ways of handling a test case ( a real one),
Thank you for your time and 'night everyone.
2010/8/11 Olivier Girardot <ssaboum@xxxxxxxxx>
i edited the wiki page Steve P. created about multi-threaded tests, i did a lot of testing and research to find a clean and easy way to handle the problem we found out during the creation of the test case https://bugs.eclipse.org/bugs/show_bug.cgi?id=321397
. Steve summed it up in a clear manner on the wiki :
- Child threads that make assertions have the results of those assertions undetected.
The (child) threads can terminate before or after the main thread (which is running the JUnit test), so even if the child assertions are correctly reported, there may be no-one to report them to. (Note that the overall result of a test needs to be gathered, too, so some sort of synchronisation is vital.)
- Mock objects (which make implicit assertions during use and on verify) shared between threads can have their assertions caught (as in the answer to the first problem) but verify calls need to be made after all the child threads have finished, or at least not too soon.
So what should we do ?
I thought about many ways we could handle that but the perfect solution isn't designed yet, so i submitted the following proposals :
- (old one) Brian Goetz mentionned in Java Concurrency in Practice that for the JSR 166 (creating the java.util.concurrent package) a test framework was created to handle assertions error in generated threads. I found back the class : JSR166TestCase and we should take a look;
- We can handle these problems using ThreadGroups exceptions handling, but exceptions will be ducked back into the "root" ThreadGroup. Killing potentially generated Threads;
- (new one) Using aspects to watch for AssertionsError, no matter what thread will launch it. This is the less intrusive way but its scope must be carefully studied. (Binary weaving can be painfull (especially on the 16 000 jdk classes), i think we should use Load Time Weaving as much as possible but it has to be tested).
I'd love some reviews on these and know what you think