Community
Participate
Working Groups
When a test log has no verdicts in it, the overall verdict on Overview page shows "inconclusive" which is confusing with the real "inconclusive" verdict. It can say "none" or just no value. Mark, please assign this one to me. Thx, -Bianca.
Reassign to Bianca.
Show "no verdict" on Overview page when no verdict event in log.
Rolled back the change to the default verdict in TPFExecutionResultImpl. Making the default value null causes NPE in other places. See bug 145631. Too late for 4.2, deferring to 4.3 when all references to the default verdict can be updated.
This amounts to an addition to the api if the content of the model is changing. If we are adding a "no verdict" to the model, and change default behaviour, we also need a no verdict event.
As discussed on Test Project meeting 07/24/2006, only UI change will be made for this defect. The model change will be tracked under a new enhancement request to be opened. The PII string for "no verdict" was added in a late 4.2 iteration. Needs to make sure if the PII change was actually included in translation before making this change...
Created attachment 47081 [details] Patch file The PII change for this was translated and available in TPTP 4.2 NL packs. So OK to deliver the change. Attaching patch file with the fix.
This is causing problems for consuming products in major release depending on 4.2.1, raising to major.
(In reply to comment #5) > As discussed on Test Project meeting 07/24/2006, only UI change will be made > for this defect. The model change will be tracked under a new enhancement > request to be opened. So do I understand then tha tin the 4.2.1 fix the model content will remain exactly as before with no changes to loaders etc., and that the UI will determine that it wants to show no verdict or inconclusive but not refelct that persistantly in the model?
(In reply to comment #8) > (In reply to comment #5) > > As discussed on Test Project meeting 07/24/2006, only UI change will be made > > for this defect. The model change will be tracked under a new enhancement > > request to be opened. > So do I understand then tha tin the 4.2.1 fix the model content will remain > exactly as before with no changes to loaders etc., and that the UI will > determine that it wants to show no verdict or inconclusive but not refelct that > persistantly in the model? > That is correct. The fix as in the attached Patch file only contains change in the UI class.
Enhancement request 153304 was submitted to track the needed model update.
Fix checked in 4.3.
This defect does not appear to be critical ("Prevents function from being used, no work around" - http://www.eclipse.org/tptp/home/documents/process/development/bugzilla.html). According to the severity definitions, it should be marked normal. Please adjust.
Since the Test Log viewer provides a visual viewer for the test execution model, we should not be interpreting the model data differently. We need the proposed fix but used an empty string instead of 'No Verdicts' string. Will this satisfy the requirement of the consuming product? Please include one or more test cases (e.g. test execution with no verdicts) for this fix.
(In reply to comment #13) > Since the Test Log viewer provides a visual viewer for the test execution > model, we should not be interpreting the model data differently. We need the > proposed fix but used an empty string instead of 'No Verdicts' string. Will > this satisfy the requirement of the consuming product? > > Please include one or more test cases (e.g. test execution with no verdicts) > for this fix. > The model data of the test log contains no verdict in this case. *Inconclusive* is one of the valid verdicts thus it is wrong to assign to the result in the first place. Foundamentally the model should be corrected to contain the right value. But before that's corrected by bug 153304, the least can be done is to correct the UI to show the right thing. This bug is about correct this piece of wrong information rather than a better way interpreting the model.
(In reply to comment #12) > This defect does not appear to be critical ("Prevents function from being used, > no work around" - > http://www.eclipse.org/tptp/home/documents/process/development/bugzilla.html). > According to the severity definitions, it should be marked normal. Please > adjust. > The test log is the only way TPTP uses to record test execution results. It is definitely not part of the functionality to record the wrong result. Also currently there is no way to change or correct the test log after it's generated. User will be stuck with the incorrect test log and lose the basic point of test logging. This problem started with the creation of the execution data model. It should have been corrected in releases earlier and shouldn't be left the wrong way for any more releases to come. Without correcting the model, it was agreed in 4.2 that the UI should be corrected at least to lower the risk. The model fix should happen in 4.3 with 153304.
It is consistant from multiple consuming products that the incorrect *inconclusive* need to be fixed earlier rather than let it slip for more releases. "no verdict" is more informational and accurate than empty string. The risk of correcting this only at the UI level is very low. Paul, is there any senario that "no verdict" is not appropriate for this case? If you need backup reasoning for this fix to pmc, please let me know. Thanks.
1) Explain why you believe this is a stop-ship defect. How does the defect manifest itself, and how will users of TPTP / consuming products be affected if the defect is not fixed? This is a critical defect because several consuming products have reported severe usability issues, and in some cases within those consuming products, the existing implementation reports incorrect/imprecise data about the status of a test log. 2) Is there a workaround? If so, why do you believe the workaround is insufficient? There is no workaround. 3) Is this a regression or API breakage? Explain. This is not a regression, however the defect was highlighted as severe by the consuming products several months ago. A prior solution was implemented, but was deemed improper and was rolled back, thus the need for the greater scrutiny this new solution received before finally garnering agreement. 4) Does this require new API? No 5) Who performed the code review? Joe Toomey, Paul Slauenwhite, others 6) Is there a test case attached to the bugzilla record? There will be. (Bianca, please add a test to cover the proper display of this information in the viewer -- ping me if you have questions about this.) 7) What is the risk associated with this fix? Low risk. Small size of code change and the code has been well tested and has been reviewed and re-reviewed by several committers. 8) Is this fix related to any standards that TPTP adheres to? If so, who has validated that the fix continues to adhere to the standard? The fix is a UI change only, and does not have a relationship to the U2TP.
Adding screenshots of how the existing TPTP reports display test logs with no verdicts. We reviewed these offline to determine if this fix introduced inconsistencies in our presentation of the data, and concluded that it did not.
Created attachment 48720 [details] BIRT Test Pass report on test log with no verdicts
Created attachment 48721 [details] BIRT Tabular report on test log with no verdicts
Created attachment 48722 [details] Old Test Pass report on test log with no verdicts
Added new test case for both windows and linux in CVS HEAD. Will deliver the same to 4.2.1 with the fix.
As approved for 4.2.1 by PMC on 8/25/06 and checked in the fix as well as new test case for it on the same day.
As approved for 4.2.1 by PMC on 8/25/06, the fix was checked in as well as new test case on the same day.
ACTION: Please verify/close this defect.
Reporter: Please verify and close in preparation for shutting down the TPTP 4.4 release. Thanks.
Closing by default since not verified/closed by the originator in the 6+ months since being resolved. Please reopen if the issue is still present in the latest TPTP release or the resolution is not correct.
Correcting component.
