Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jdt-core-dev] Testing ecj<->javac compatibility

Hi Stephan

Great work and thanks for doing this!

 The approach looks good to me. JDT Core team, please help Stephan to get the failure count down.

Dani



From:        Stephan Herrmann <stephan.herrmann@xxxxxxxxx>
To:        Eclipse JDT Core developers list <jdt-core-dev@xxxxxxxxxxx>
Date:        07.01.2018 12:35
Subject:        [jdt-core-dev] Testing ecj<->javac compatibility
Sent by:        jdt-core-dev-bounces@xxxxxxxxxxx




Hi Team,

Here's a New Year's Resolution of mine: "Always run all tests!" :)

Background:
When recently I worked on a bug about a difference between ecj & javac,
I had a patch that would cause a number of regressions in our suite,
but I was quite uncertain, whether the 'broken' tests actually expected
the 'correct' outcome. At that point I remembered, that all our compiler
tests can be run in a mode to compare against javac, but I'm afraid this
mode has seen a lot of bit rot over the years.

As a first counter measure and to make that test mode useful again,
I created a new Jenkins job [1], which currently runs once per week
(on a weekend night). I plan to add a similar job for Java 9, too.

The first run of this job (after configuration issues) was quite
devastating: close to 1000 failures.

While inching along fixing the most obvious cases, where ecj has
more diagnostics than javac (null annotations, optional warnings
configured as error etc.) I once again struggled with the overload
overkill in AbstractRegressionTest and 'invented' a style which I
hope to be better usable and maintainable, for details see [2] and
if you have any comments regarding the new style please drop them in
the bug (I know this style may look like a riddle on first glance :)).
For existing tests, this style facilitates adding a JavacTestOption
argument with minimal code changes (for minimal impact on blame history).
For new tests I'm planning to completely switch to this style
(unless s.o. raises any objections) (with somewhat more conventional
formatting :) ).

With technical issues out of the way, the *run.javac-1.8 job is
currently at 281 failures and I would greatly appreciate if we could
split this round of clean-up among the team. My idea is that in this
round we only address obvious "Excuses" and I expect that after this
we have a residuum of some tens of tests that perhaps signal relevant
differences between ecj & javac - to be addressed in a second round.

Finally, when/if we reach the state where these builds are error-free,
I suggest that everybody working on the compiler enter their email
in the job configuration (or ask me to do so) to be notified of any
failures.

let me know what you think,
Stephan


[1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__ci.eclipse.org_jdt_job_eclipse.jdt.core-2Drun.javac-2D1.8_&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=1UITCR5rxUZHSFczvfaNFK4ymEbEiccRX7VKchpqz0Y&m=4Kj8AcrlvGg1LN0vYgJCT03EmPFI8nnSRUufQfSE6ic&s=Q6RM6f4okcKi-qxABIxR2Nqd8rA31-iIyFxmu4M-vAQ&e=
[2]
https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.eclipse.org_bugs_show-5Fbug.cgi-3Fid-3D404648-23c45&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=1UITCR5rxUZHSFczvfaNFK4ymEbEiccRX7VKchpqz0Y&m=4Kj8AcrlvGg1LN0vYgJCT03EmPFI8nnSRUufQfSE6ic&s=Eyqvbb4SCh1B2OxOVqIT3L-HLxT6S5Q_cT98XN8zamk&e=
_______________________________________________
jdt-core-dev mailing list
jdt-core-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://urldefense.proofpoint.com/v2/url?u=https-3A__dev.eclipse.org_mailman_listinfo_jdt-2Dcore-2Ddev&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=1UITCR5rxUZHSFczvfaNFK4ymEbEiccRX7VKchpqz0Y&m=4Kj8AcrlvGg1LN0vYgJCT03EmPFI8nnSRUufQfSE6ic&s=tBgF42vOyQm9pSQZsR7QVQOJVxcvYG-0Qgnq82neB2c&e=





Back to the top