Description James Roper 2018-06-19 08:48:56 EDT

In the MicroProfile Reactive project (https://github.com/eclipse/microprofile-reactive), we're seeing IP validation only running sporadically - there have been times where a PR should have failed validation, but we merged it because the validation didn't run and we didn't notice. We also use the Jenkins GitHub PR validation plugin, so it's easy to fail to notice that the IP validation hasn't run since the status will be green from the Jenkins validation. Here are some PRs where IP validation didn't run:

https://github.com/eclipse/microprofile-reactive/pull/36
https://github.com/eclipse/microprofile-reactive/pull/28

Here are some PRs showing both that IP validation and Jenkins ran for comparison:

https://github.com/eclipse/microprofile-reactive/pull/29
https://github.com/eclipse/microprofile-reactive/pull/37

The above PRs are all closed now, to see which validations ran you need to click on the tick next to the commit - in the case that both Jenkins and IP validation ran, it will pop up a little box showing you both validations with links to see their results, in the case that only Jenkins ran, clicking the tick will take you straight to Jenkins.

Empirically, I've noticed it only running about 50% of the time.