Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[linuxtools-dev] Announcement: SWTBot tests

I've committed a patch that ignores some of the failing tests.

I beg every contributor - PLEASE DO NOT ADD MORE SWTBOT tests.
I'm really tired of the instability of these tests. There are many problems with these tests like:
* They are fragile - a widget being added/removed in other components breaks them
* They are not real unit tests - failures are usually not in the component that is supposed to be under test but elsewhere.
* They do not help shaping good API as people don't use the API that are exposed but do "clicks"
* So much noise generated by swtbot tests in hudson that make the mailing list unusable
* No one cares about test failures anymore and ignores them
* ...............
I do not mind having SWTBot tests in the codebase but they should not be enabled in the default build and hudson. Every test that runs by default must work 100% when doing "mvn clean verify". 
>From now on, whenever I spot  a test case failing more than twice in gerrit when testing unrelated patch I'll add ignore to it as it is high time to get reliable builds. I'm really getting tired of these issues.

Tests that fail randomly are useless and even harmless. Period.


Alexander Kurtakov
Red Hat Eclipse team



Back to the top