Re: [linuxtools-dev] Fwd: [Hudson] Hudson build is still unstable: linux

[Roland Grunberg <rgrunber@xxxxxxxxxx>]

> In noticed that some JUnit tests of our build have been failing since
> yesterday. I focused on the LTTng/TMF/CTF test plug-ins. I noticed
> that the order of test methods executed within a JUnit test file
> changed for some reason. And it looks like that some test are
> dependent on other tests to be executed before. Then I was wondering
> why did the order change. I noticed that JUnit version 4.11 is used
> instead of 4.10 and in that release the way changed how the order of
> execution is determined (see [1]). There is a fix using an annotation
> provided by JUnit Test 4.11, however this would require every
> designer
> has to upgrade to 4.11. Since Eclipse Juno comes with 4.10 this is
> not
> really an option right now. So in the end we need to make all test
> cases independent from each other.
> 
> I'm not sure if the test failures for GCov are caused by this but it
> is worth considering it.
> 
> [1]
> http://randomallsorts.blogspot.ca/2012/12/junit-411-whats-new-test-execution-order.html

Thanks for looking into this! I was starting to suspect JUnit was doing
something weird with the test orderings.

Generally, I try to avoid having tests that must execute in some defined
order (I've seen these kinds of issues in the past), but I believe for
GCov, the tests only depend on some code from another test that can
easily be pushed into the setup phase.

Thanks Again,
-- 
Roland Grunberg