Community
Participate
Working Groups
Efforts are under way to do automated testing of EPP packages with Jubula. From Felix Ziesel's email to epp-dev, the test process works this way: 1. A hudson job starts a download of the all targets for a package 2. For every target a job starts the AUTAgent (AUT=Application Under Test) 3. This job starts the AUT and run the tests 4. This job closes the AUTAgent again In order to execute tests we need: 1. A virtual machine or a server to run the tests for every target 2. Jubula installation on the targets 3. A centralized database for the test definitions and results When developing tests on your own you need: 1. A connection to the db (to reuse projects and add your project) 2. Configurations for starting the tests on eclipse via hudson 3. Configurations for starting the tests locally
Created attachment 237319 [details] automated Jubula UI tests for EPP The attachment has a POC for running Jubula tests with Tycho. It's implemented in such a way that the build runs without any pre-requisites other than Maven. When "mvn clean verify" is executed in the tests/org.eclipse.epp.tests/runner directory the following happens: * A separate Maven process that runs "mvn verify" in the ../agent directory is started that runs in parallel (see below). * The build waits for the autagent to startup * A local h2 database with test cases is created by importing db/EclipseIDE_1.0.xml * The Jubula testexec application is started to execute tests. This starts the EPP package, does all the Jubula UI testing magic and creates a report with test results. The agent build that runs in parallel does the following: * The Java EPP package matching the current platform is downloaded from a URL * The package is extracted * The package is instrumented with Jubula RCP support and started once with with "-clean" to complete the installation of the Jubula support bundles * The Jubula autagent is started It's a bit quirky that there are essentially two builds running in parallel but the way it's designed right now everything can run from a single Maven command and just works on whichever machine the build runs on. I have tested this setup successfully on Windows, Mac and Linux. To run this for EPP we would simply have to setup a Matrix build on Hudson with an Axis for the different slaves that we want to run tests on. The build would only need to be parameterized with the URL of the EPP build that we want to test. The database that is included in the zip only has a Hello world test right now that clicks two buttons in the UI but it should be fairly simple to automate some of the smoke tests such as creating a Java project etc. Just throwing this out here to see there is any interest in actually deploying this on a Hudson instance.
If you have Jubula running locally you can start the autagent from the test zip by running "mvn verify" in tests/org.eclipse.epp.tests/agent . If you then import the database from the db directory you can connect to the autagent and execute tests using the aut configuration for your platform. It looks something like this: pre.. # mvn verify [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building org.eclipse.epp.tests.agent 1.0.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-antrun-plugin:1.7:run (download-aut) @ org.eclipse.epp.tests.agent --- [INFO] Executing tasks main: [get] Getting: http://build.eclipse.org/technology/epp/epp_build/kepler/download/20130919-0822//20130919-0822_eclipse-java-kepler-SR1-RC4-macosx.cocoa.x86_64.tar.gz [get] To: /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target/20130919-0822_eclipse-java-kepler-SR1-RC4-macosx.cocoa.x86_64.tar.gz ...... [INFO] Executed tasks [INFO] [INFO] --- maven-antrun-plugin:1.7:run (untar-aut) @ org.eclipse.epp.tests.agent --- [INFO] Executing tasks main: [untar] Expanding: /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target/20130919-0822_eclipse-java-kepler-SR1-RC4-macosx.cocoa.x86_64.tar.gz into /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target [INFO] Executed tasks [INFO] [INFO] --- maven-antrun-plugin:1.7:run (unzip-aut) @ org.eclipse.epp.tests.agent --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-antrun-plugin:1.7:run (instrument-aut) @ org.eclipse.epp.tests.agent --- [INFO] Executing tasks main: [get] Getting: http://download.eclipse.org/jubula/release/kepler/plugins/org.eclipse.jubula.rc.rcp_2.1.0.201305291108.jar [get] To: /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target/eclipse/plugins/org.eclipse.jubula.rc.rcp_2.1.0.201305291108.jar ... [get] last modified = Mon Jun 10 08:20:09 CEST 2013 [propertyfile] Updating property file: /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target/eclipse/configuration/config.ini [INFO] Executed tasks [INFO] [INFO] --- tycho-eclipserun-plugin:0.18.1:eclipse-run (default-cli) @ org.eclipse.epp.tests.agent --- [INFO] Adding repository http://download.eclipse.org/jubula/release/kepler [INFO] Adding repository http://download.eclipse.org/jubula/release/kepler [INFO] Fetching p2.index from http://download.eclipse.org/releases/kepler/201309270900/ (0B at 0B/s) [INFO] Fetching p2.index from http://download.eclipse.org/releases/kepler/201309270900/ (365B at 0B/s) [INFO] Adding repository http://download.eclipse.org/releases/kepler/201309270900 [INFO] Fetching p2.index from http://download.eclipse.org/releases/kepler/201309270900/ (0B at 0B/s) [INFO] Fetching p2.index from http://download.eclipse.org/releases/kepler/201309270900/ (365B at 0B/s) [INFO] 1 operation remaining. [INFO] Adding repository http://download.eclipse.org/releases/kepler/201309270900 [INFO] Expected eclipse log file: /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target/work/data/.metadata/.log [INFO] Command line: /bin/sh -c cd /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent && /Library/Java/JavaVirtualMachines/jdk1.7.0_40.jdk/Contents/Home/jre/bin/java -jar /Users/steffen.pingel/.m2/repository/p2/osgi/bundle/org.eclipse.equinox.launcher/1.3.0.v20130327-1440/org.eclipse.equinox.launcher-1.3.0.v20130327-1440.jar -install /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target/work -configuration /Users/steffen.pingel/eclipse/org.eclipse.epp.packages/tests/org.eclipse.epp.tests/agent/target/work/configuration -application org.eclipse.jubula.app.autagent
