| Re: [cdt-dev] proposal for side-stepping known test failures |
|
There are failures in the following plugins when I run the tests locally on my Windows machine. I have the very latest master sources. codan/org.eclipse.cdt.codan.core.test core/org.eclipse.cdt.core.tests core/org.eclipse.cdt.ui.tests xlc/org.eclipse.cdt.core.lrparser.xlc.tests (*) build/org.eclipse.cdt.make.core.tests (*) I’ll be happy to communicate the details to anyone who’s interested in pursuing them. Note that the last two plugins are not in the Hudson report. Do you
know why they’re not being tested? Any test that intermittently fails should be optionally avoidable. John From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Sergey Prigogin https://hudson.eclipse.org/hudson/job/cdt-nightly/lastCompletedBuild/testReport/ shows only two failures, both of which are intermittent. Which
test failures are you referring to? -sergey On Thu, Aug 16, 2012 at 3:03 PM, Cortell John-RAT042 <RAT042@xxxxxxxxxxxxx> wrote: The CDT tests appear to have known failures--more than just a handful. This makes it very difficult to detect new failures caused by new changes.
I’d like to wrap known failures as follows: if (System.getProperty("cdt.skip.known.test.failures") == null) { // the failing test } This will allow me (and others) to get a clean test run before starting on a set of changes. I can then run the tests again after my changes and immediately find out if I’ve broken
anything. Right now, I’d have to manually/visually filter out dozens of known failures from the results, which is very tedious, time consuming and error prone. This is much better than simply commenting out broken tests and putting a “TODO”, as it actually
leaves the broken tests active in the codebase. Objections? John
|