Description Lars Vogel 2015-12-09 04:38:12 EST

Currently the Eclipse platform project uses an after burner approach for running a significant part of their tests, i.e., we build first the Eclipse SDK and afterwards we run our tests. A small part of the platform tests have already been configured to run during the Gerrit build, but not all.

Unfortunately, activating the remaining tests turned out to be significant effort given the number of tests for platform (I have not counted them but the complete SDK test suite is > 80 000 test executions).  Lots of these tests run successfully, if started via Apache Ant or the Eclipse IDE, but not if started via Tycho. 

Detailed analysis is required to identify the dependencies of these failing tests and fixing the underlying problem. This detailed error analysis seems beyond the capacity of the current development team, even though the platform team set this as goal for Eclipse 4.6 M3, it could not be accomplished.

An FEEP item could help to get the focus on this work on this and provide patches for activating the tests. Acceptance criteria would be that the tests of a subproject are running successfully via Tycho. This would involve an analysis of at least the test projects in the following Git repositories:

eclipse.platform.ui
eclipse.platform.runtime
eclipse.platform.ui
eclipse.pde.ui
eclipse.platform.text
eclipse.platform.resources
eclipse.platform.compare

Also the tests for debug and ant should be reviewed in this effort. Not all test may be activated during the Tycho build, e.g.., the cvs tests require access to special hardware during their run, in this case they should be deactivated during the Tycho build.

This item was discussed with Dani Megert the Platform project lead. We think this item will add significant value for the committer workflow and therefore result in a higher quality and faster development cycles in the Eclipse platform.

See Bug 474328 for the aggregator bug for this in platform.