[cosmos-dev] Hole in our system testing

[David Whiteman <David_Whiteman@xxxxxxxxxx>]

Currently the QA team is documented
as doing their testing based off the scenario described on the http://wiki.eclipse.org/COSMOS_Demo_i9
page.  This page roughly maps to what we've been calling our "demo".
 However, restricting QA testing to this scenario leaves a hole in
testing the externals of our components.  Two other areas of COSMOS
that mostly involve a UI are the SML tooling and the Data Manager toolkit.
 As it's well accepted that it's desirable to receive 3rd party testing
(i.e. testing by someone other than a component's developer), I think we
need to add those two areas to the list of QA responsibilities.  As
far as I know, neither of those areas has been looked at much by anyone
outside of the developers who built them, which as we saw in our i9 end-of-iteration
integration issues, can be a problem.

That said, since our packaging separates
our project into "demo", "SDK", "Resource Modeling",
etc., just tacking these scenarios onto the Demo page doesn't feel right.
 I think there are likely multiple actors in the audience for a demo:
one that wants to see the existing COSMOS components working, and one that
wants to extend COSMOS.  For the latter audience, I would suggest
we create an "Advanced Demo" page to cover the topics we have
targeted for the Developer Guide, such as the two I mentioned.  That
page would instruct the reader (i.e. QA) to download the SDK and the RM
packages and to walk through those use cases (and any others not currently
being tested independently).

Thoughts?

David