Community
Participate
Working Groups
If I want to configure Tycho to run a test suite I need to specify Testsuite and TestClass This took me relatively long to figure out, the TestSuite is actually the Bundle-Symbolic-Name and the TestClass points to my Test suite. https://wiki.eclipse.org/Tycho/Reference_Card#Setting_up_the_tests_to_run Maybe the error message could a bit more helpful? For example instead of: Both testSuite and testClass must be provided or both should be null. This one would have helped me: Both testSuite (pointing to the Bundle-Symbolic-Name) and testClass (pointing to the test class/suite) must be provided or both should be null. <build> <plugins> <plugin> <groupId>org.eclipse.tycho</groupId> <artifactId>tycho-surefire-plugin</artifactId> <version>${tycho.version}</version> <configuration> <testSuite>org.eclipse.e4.ui.tests.css.swt</testSuite> <testClass>org.eclipse.e4.ui.tests.css.swt.CssSwtTestSuite</testClass> </configuration> </plugin> </plugins> </build>
IMHO testSuite is a misnomer but too late to change the name, it's API http://eclipse.org/tycho/sitedocs/tycho-surefire/tycho-surefire-plugin/test-mojo.html#testSuite However I'm not sure why it's a must to specify both the suite test class and bundle-symbolicname of the the test suite class why would we not assume that the test suite class can be loaded by the current test bundle if it's not specified explicitly?
(In reply to Jan Sievers from comment #1) > IMHO testSuite is a misnomer but too late to change the name, it's API Maybe add a new parameter and deprecated the old one? > http://eclipse.org/tycho/sitedocs/tycho-surefire/tycho-surefire-plugin/test- > mojo.html#testSuite > > However I'm not sure why it's a must to specify both the suite test class > and bundle-symbolicname of the the test suite class > why would we not assume that the test suite class can be loaded by the > current test bundle if it's not specified explicitly? +1
The "testSuite" parameter is used to skip maven projects other than the one specified by "testSuite". If you are building e.g. 3 eclipse-test-plugins in one maven session, and want to execute only one test from the 2nd eclipse-test-plugin, the other 2 plugins must be skipped. (Executing the oterh 2 plugins would end up in "No Test found" error, since the specified testClass would not be found in the other 2 bundles) An alternativ might be to only specify the testClass, search for it in the current project and skip execution if it could not be found, but than it could not be distinguished if a testClass could not be found because it is not part of that bundle or it could not be found because the testClass value is wrong.
(In reply to Martin Schreiber from comment #3) > The "testSuite" parameter is used to skip maven projects other than the one > specified by "testSuite". If you are building e.g. 3 eclipse-test-plugins in > one maven session, and want to execute only one test from the 2nd > eclipse-test-plugin, the other 2 plugins must be skipped. > (Executing the oterh 2 plugins would end up in "No Test found" error, since > the specified testClass would not be found in the other 2 bundles) I think this is unrelated. Executing a specific test in a multi-module build with multiple test modules can be achieved with -Dtest=TestClass -DfailIfNoTests=false Looking at the implementation http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-surefire/tycho-surefire-plugin/src/main/java/org/eclipse/tycho/surefire/TestMojo.java#n600 I realize the parameter testSuite does not make any sense at all. The build will always fail if you don't repeat the Bundle-SymbolicName of the current test bundle as testSuite. All it does if check whether you specified the correct Bundle-SymbolicName of the current test module (but it's not used for anything). So unless I'm mistaken the parameter testSuite can be simply removed without any side-effect.
(In reply to Jan Sievers from comment #4) > (In reply to Martin Schreiber from comment #3) > > The "testSuite" parameter is used to skip maven projects other than the one > > specified by "testSuite". If you are building e.g. 3 eclipse-test-plugins in > > one maven session, and want to execute only one test from the 2nd > > eclipse-test-plugin, the other 2 plugins must be skipped. > > (Executing the oterh 2 plugins would end up in "No Test found" error, since > > the specified testClass would not be found in the other 2 bundles) > > I think this is unrelated. Executing a specific test in a multi-module build > with multiple test modules can be achieved with > > -Dtest=TestClass -DfailIfNoTests=false > Yes this can also be achieved with that, with one small difference. Running a) -Dtest=TestClass -DfailIfNoTests=false and b) -DtestSuite=myTestBundleId -DtestClass=foo.bar.TestClass a) will not fail if no TestClass could be found, b) will. > > Looking at the implementation > > http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-surefire/ > tycho-surefire-plugin/src/main/java/org/eclipse/tycho/surefire/TestMojo. > java#n600 > > > I realize the parameter testSuite does not make any sense at all. > The build will always fail if you don't repeat the Bundle-SymbolicName of > the current test bundle as testSuite. All it does if check whether you > specified the correct Bundle-SymbolicName of the current test module (but > it's not used for anything). > > So unless I'm mistaken the parameter testSuite can be simply removed without > any side-effect. When removing the testSuite parameter, would it make sense to keep the testClass parameter then? As you mentioned, executing a specific test can also be achieved by -Dtest=MyTest?
> When removing the testSuite parameter, would it make sense to keep the > testClass parameter then? As you mentioned, executing a specific test can > also be achieved by -Dtest=MyTest? We should be able to specify the Testclass in the pom.xml, I read this suggestion that it should only be possible via a parameter.
(In reply to Martin Schreiber from comment #5) > When removing the testSuite parameter, would it make sense to keep the > testClass parameter then? As you mentioned, executing a specific test can > also be achieved by -Dtest=MyTest? no, testClass should be kept for compatibility. testSuite has no effect anyway (apart from forcing the user to repeat the BSN of the current test bundle), so we can remove it.
(In reply to Lars Vogel from comment #6) > > When removing the testSuite parameter, would it make sense to keep the > > testClass parameter then? As you mentioned, executing a specific test can > > also be achieved by -Dtest=MyTest? > > We should be able to specify the Testclass in the pom.xml, I read this > suggestion that it should only be possible via a parameter. both "testClass" and "test" paramaters have property annotations, that means they can be specified both in pom.xml and/or on CLI, no change here. In general, mojo parameters can always be specified in pom.xml. You can only add the possibility to specify them on CLI.
Submitted a patch for review: https://git.eclipse.org/r/37488 (Removed the testSuite parameter)
The IT Test TYCHO45Test is failing because it tests a multi bundle build with different test plugins and had specified the testSuite and testClass parameter. As said, testSuite skipped all the test plugins that did not match the testSuite value. Now this test is failing because it tries to execute just the test specified in testClass and this could only be found in one test plugin, the others are failing because no test can be found. Is it ok to set failIfNoTests=false for that IT Test?
http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=4492fb81ea8a9a4e4238ba1425b487ef81386bc6 thanks for the patch!
