Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[ecf-dev] Automated test execution for ECF build based on autotestsuite (pluginbuilder)

Hi Ted,

automated testing was one of yesterday's topics the ECF con call. Since I've integrated automated test execution into Versants build process recently, I'd like to share how it's done. You have probably an idea where it's best integrated into the ECF build.

I first looked into the Eclipse testing framework [1]. It's JUnit3 based and spawns of an Eclipse instance for test execution. It supports UI tests, headless tests as well as performance tests. Test execution is handled by Ant. It spawns an Eclipse instance and executes test suites. I find this way too complex and cumbersome. A developer simply wants to write a unit test. One doesn't want write/maintain xml that defines execution order. Especially since execution order shouldn't matter for unit tests.

Thus I continued to look for something that doesn't require additional xml next to the plain test cases and I found pluginbuilder autotestsuite (EPL) [2]. It consists of a bunch of OSGi bundles which come with an Equinox application. Basically one drops the unit tests + autotestsuite bundles into the Eclipse installation under test and starts the autotestsuite application (-application org.pluginbuilder.autotestsuite.application.AutoTestApplication). Autotestsuite then takes care of finding test cases by searching bundles and bundle classpaths based on regular expression. All test cases that are found are passed to a junit.framework.TestSuite and executed. Test results are collected by the junit xml formatter. Autotestsuite doesn't require lists of test cases nor hard coded junit.framework.TestSuites and IMO that's the beauty of it.

So how could the integration of autotestsuite in ECF look like?
1) Build test bundles. I'd suggest to create test features and build and package them independently. Like it is done for platform [3]. 2) Deploy Eclipse SDK + ECF + ECF tests + autotestsuite to some temp folder on the build box, either by unzipping everything into features/ and plugins/ (<3.4), using the links/ folder (that's how Versant does it) or by using the p2 director.
3) Launch the autotestsuite application.
4) Collect the JUnit xml output and post process it (html reports, email notification...)

Cheers
Markus

[1] http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.test/testframework.html
[2] http://www.pluginbuilder.org/documentation/run-tests/
[3] http://download.eclipse.org/eclipse/downloads/drops/R-3.4-200806172000/download.php?dropFile=eclipse-Automated-Tests-3.4.zip


Back to the top