Community
Participate
Working Groups
For CDT 10.5.0 I have ended up releasing from the cdt-master build job - it is the same commit as the cdt-10.5 job is building, but on the cdt-10.5 job there are lots of test failures. Compare these cdt-master builds: https://ci.eclipse.org/cdt/job/cdt-master/1118/ https://ci.eclipse.org/cdt/job/cdt-master/1119/ to these cdt-10.5 builds for the same commits: https://ci.eclipse.org/cdt/job/cdt-10.5/3/ https://ci.eclipse.org/cdt/job/cdt-10.5/4/ In the latter the code completion tests are failing with duplicate entries for all completions.
I tried to compare the console log between the different jobs. To compare 2 jobs, some parts of the console log needs to be filter to avoid listing differences on lines that are irrelevant. The filter I used is: sed -e ' s/^\[2021[^Z]*Z\] //; /^remote: Finding sources: /d; /^Receiving objects: /d; /^Resolving deltas: /d; /^vnc .*[0?] *[0:]\{8,8\}/d; /^.INFO. Total time:/d; /^.INFO. Finished at:/d; /^Total time: [0-9]* second/d; /^\[INFO\] *\[exec\] Done in .*s\.$/d; s/Time elapsed: [0-9.]* s//; /Downloaded.*B at [0-9.]* [kM]*B\/s/d; s/[0-9,]* \(\S* requesting \/shared\/common\/gdb\)/ \1/g; s/\(The project.s OSGi version is [0-9]*\.[0-9]*\.[0-9]*\)\.[0-9]*/\1/; s/\(Unable to satisfy dependency from \)[0-9]*\.Default/\1Default/g; s/\(.INFO.*SUCCESS \[\) *[0-9:.]* .*/\1/; s/cdt-\(10[-_]5|master\)[-_][0-9]*\(-[a-z0-9]\{5,5\}\)\{3,3\}/$CDTJOB/; s/\/home\/jenkins\/agent\/workspace\/cdt-\(10\.5\|master\)\//$WORKSPACE\//g; s/\/tmp\/production-.\{10,10\}/$TMP/; ' With this filer, I'm then doing the following compares: 3 vs 4: - Some of the test cases are executed in a different order, but the majority is in the same order 1118 vs 1119: - Some of the test cases are executed in a different order, but the majority is in the same order 4 vs 1119: - MAJOR change in the order the test cases are executed Maybe the order the test cases are executed matter, maybe it doesn't, but I'm fairly certain that it shouldn't. To go further down this path; Why isn't the order the test cases are executed stable? Is the Jenkins job name related to the ordering of the test cases in some mysterious way? Hope this gives you some inputs to what's going on.
