I have now merged all test refactor PRs, and have changed the Jenkins pipeline for the develop branch.
The Jenkins pipeline now works as follows: a verify job runs all unit and compliance tests for its repo. If the build fails a mail is sent to this list. If the build succeeds, a new job is started to deploy new snapshots to Sonatype. When the new snapshots are deployed, a further job is kicked off to run unit tests against the downstream dependencies.
For example: we merge a PR in the rdf4j-repo. First, rdf4j-develop-verify is started (it is triggered by a poll for changes which runs every 5 minutes). This job runs all unit and compliance/integration tests against the head of the branch. When that succeeds, rdf4j-develop-deploy is started, which builds the artifacts and deploys them to Sonatype. When that successfully completes, two further jobs are started: rdf4j-storage-develop-verify-upstream and rdf4j-tools-develop-verify-upstream. Both of these run unit tests (not compliance tests!) to check that the changes in rdf4j modules are not causing errors in these dependent projects.
I have not configured all these jobs to send e-mails, only the rdf4j-**-verify jobs. This means that if deployment or upstream verification fails, we may not immediately see it. I want to see if this is workable for us before I configure more mails (it can very easily get too noisy with all the jenkins mails). So a polite request to occasionally check Jenkins and see if things are green.
Last but not least: I have disabled the Jenkins jobs on the rdf4j-testsuite develop branch. I haven't moved the benchmark tests yet because I want to take a closer look at their utility and their place in the build process, but eventually we will move those out as well and get rid of the testsuite repo altogether.
Oh and of course: all of this only applies to the develop branch. The master branch still runs on the old system. This may be a bit awkward when having to switch from working on develop to master (or vice versa) but these changes are at least partly backward incompatible (our testsuites are published artifacts after all, and we have third parties re-using them), so I had to restrict it to develop. Hopefully, we can make good time on 3.0 development so we can get develop and master back in sync soon :)
I'm sure there's further improvements we can do, but this should be a good first sweep.
Jeen
