Community
Participate
Working Groups
Created attachment 260574 [details] "Summary of Unit Tests Results" in N20160324-2000 Please see the attached screenshot. If you click the highlighted link in it will take you to tests results with more than "0" failures. Here is the page where the screenshot was taken from: http://download.eclipse.org/eclipse/downloads/drops4/N20160324-2000/
Thanks. This may end up as a dup of bug 489534 but I will leave a separate for now, in case there are two issues.
This case is caused by the "centos" tests "replacing" the "shared Hudson linux" tests in our summary table. (CentOS taking longer, hence running "last"). The Hudson result of centos can be seen directly at https://hudson.eclipse.org/platform/view/Unit%20Tests/job/ep46N-unit-cen64/5/ The Hudson results of the shared instance can be seen at https://hudson.eclipse.org/shared/view/Eclipse%20and%20Equinox/job/ep46N-unit-lin64/215/testReport/ There really was 0 failures on the shared instance. One way to see that this replacement has taken place, is if you click on one of the errors (or, even a "0") in the summary table in question, in some of the top headers, on the right of the page, you will see a column that says "host" and for this problematic case it says "centos". This is actually a good problem to see! :) It means the CentOS results are being processed and summarized -- I just need to come up with with a way to fix the labels, and "identify" the input to the summary program. Perhaps I will have to add "host" to the identifiers. I will count bug 489534 as dup of this one since I have investigated and written findings here.
*** Bug 489534 has been marked as a duplicate of this bug. ***
Changing title to be more specific to the "root problem". The original title was but one symptom of the problem. Current Status: I've changed some the the "core" of how our jobs work. The "key" fix is to add Hudson's JOB_NAME to the "testedPlatform" variable, so we will end up with "configs" similar to mb4N-unit-lin64_linux.gtk.x86_64_8.0 instead of previous linux.gtk.x86_64_8.0 This will require some reformatting of some tables to everything fits in a reasonable way. = = = = = At the same time, indirectly related to the above, but more related to "the quiet holiday" I think it would be a good time to fix bug Bug 487044. = = = = = Unfortunately the CentOS machine keeps going offline a lot. But, I will be doing lot of N-builds over this quiet weekend to make as much progress as possible. = = = = = I am sure more issues will show up, the more that gets fixed. = = = = = This is also good preparation for the case(s) when we start testing again Java 9. Much of the same work would have to be done for it too.
after adding "hudson job name" to the "expected configuration" values, we get "double rows" in the test results summary. Such as, see http://download.eclipse.org/eclipse/downloads/drops4/N20160328-2100/testResults.php#UnitTest Or, a simplified representation: compare.tests - - - (0) compare.tests (0) (0) (0) - I have discovered the reason for that is that we must specify the "columns" in alphabetical order since the test results files are processed "in alphabetical order". So for now, will put the "ep46N-unit-cen64" in the first column, but opened bug 490624 to make this order more flexible in the future.
This is now basically fixed, by adding Hudson's JOB_NAME to the "testConfiguration" we can distinguish different "machines". The "expected text configs" now look are as follows: $expectedTestConfigs[]="ep46$BUILD_TYPE-unit-cen64_linux.gtk.x86_64_8.0"; $expectedTestConfigs[]="ep46$BUILD_TYPE-unit-lin64_linux.gtk.x86_64_8.0"; $expectedTestConfigs[]="ep46$BUILD_TYPE-unit-mac64_macosx.cocoa.x86_64_8.0"; $expectedTestConfigs[]="ep46$BUILD_TYPE-unit-win32_win32.win32.x86_8.0"; where build_type is I,M, or N. whereas previously, the only distinction we could make was based on arch, ws, os, and vm: $expectedTestConfigs[]="linux.gtk.x86_64_8.0"; $expectedTestConfigs[]="linux.gtk.x86_64_8.0"; $expectedTestConfigs[]="macosx.cocoa.x86_64_8.0"; $expectedTestConfigs[]="win32.win32.x86_8.0"; As I wrote this explanation, I realized one complication will be when we promote (and rename) the build, since then we change the value of $BUILD_TYPE but do not rename any test results files. So I opened bug 490725 for that work.
(In reply to David Williams from comment #6) > As I wrote this explanation, I realized one complication will be when we > promote (and rename) the build, since then we change the value of > $BUILD_TYPE but do not rename any test results files. So I opened bug 490725 > for that work. I just remembered another remaining "broken" piece -- though sort of broken anyway :) and that is the "missing files" list at the end of the summary. I ahve opened bug 490728 to track to work.
