RE: [hyades-dev] Some Test Model & UI progress

[Michael.Norman@xxxxxxxxxxxxx]

Warning, Flaming expected
 
I think I'm OK with this, but I need to articulate what I think the 
requiremnt is here.  I'm sorry that for brevity I will ave to use 
terminology defined in my prototype document. I know the Pseudo-SUT 
concept doesn't appear the test model... yet.
 
We are dealing with two types of behaviour, behaviour that is defined in 
the Test Model against the interfaces of the Pseudo-SUT and Behaviour 
that is defined in the layer between the Pseudo-SUT and the SUT. (i.e. 
in the wrapper).  In my view of the world the former is part of the test 
case, the latter is on the other side of the black box interface that 
the test case speaks to. Behaviour in the test and behaviour in the 
wrapper are not exclusive.  Our Citrix product runs a load test over 
multiple copies of executing Visual Test Scripts.  We would anticipate 
porting this by defining the load test behaviour in the test model and 
retaining the script behaviour in the Visual Test script. The 
interaction is quite complex, the load test is not just invoking the 
Test Script it's sending messages into and out of it.  A simpler version 
of this might be to run Junit test cases in a load test to check for 
functional failure under load.
 
At the point in the model editor where you see a call to an interface to 
a Pseudo-SUT (e.g. a JUnit test case or a Visual test Script) we should 
have an extension point which effectively allows you to edit the 
wrapper. (i.e. pop across the black box interface to edit the 
externally-defined behaviour).  You would pop into Java, C, Rational 
Visual Test Script, or whatever. This applies even if we have behaviour 
in the test case.  
 
We also NEED a Universal test case editor which can handle any behaviour 
up to the Psuedo-SUT interface and which provides the extension point to 
the wrapper editor.  It doesn't have to be very good but it does have to 
be universal. Harm talked about being able to provide something by 
default from EMF.
 
Now the concept of behaviour-specific test editors is also important.  
In my view these are optional replacements rather than extensions of the 
Universal test case editor and offer the same sets of extension points 
to the wrapper editors (although there may be some Eclipse issues in 
doing it this way).
 
Finally, we need a point in the Test Model to carry a description of the 
interface to the Pseudo-SUT, not just the way we are exercising that 
interface in the test.  This allows the various test editors to offer 
only legal sets of interface calls up through the UI to the tester.  I 
know most people think they'll just be dealing with arbitrary pieces of 
Java as the interface to the SUT, but it aint necessarily so, in most 
real environments an IDL and type system will be extremely useful for 
bridging, and constraining the test creation logic in the editor.  I 
think there's a point in the test model which associates the SUT (i.e. 
in my terms the Pseudo-SUT) class to the test instance and we can extend 
away from this.
 
 
Mike
 
 
 

-----Original Message-----
From: jtoomey [mailto:jtoomey@xxxxxxxxxxxx]
Sent: Thursday, April 24, 2003 11:46 PM
To: hyades-dev
Subject: [hyades-dev] Some Test Model & UI progress



Hi all. 

Marcelo, Serge and I had a very productive discussion this afternoon, 
and I'd like to summarize some conclusions that were drawn:

1) We again debated the ownership of TestCase by TestSuite, and came to 
a very nice compromise.  While a TestCase is owned by a TestSuite, we 
will break the aggregation-by-value of the TestCase's behavior.  In 
fact, we will model neither the TestSuite's behavior nor the TestCase's 
behavior as owned by the TestCase or the TestSuite -- behaviors will be 
persisted in their own resources.  This solves many of the issues we 
have wrestled with in the TestModel group and also in the UI team.  When 
envisioning the use case where a behavior is not modeled using the test 
model, in a JUnit source file, for instance, it is clear that the 
behavior is persisted in a separate resource.  This works equally well 
if the behavior is modeled in the Test Model, as opening the TestCase 
behavior requires a new editor to be opened (this is an extension 
point), and in Eclipse this would imply that a separate resource is 
being opened.  Lastly, this will drive a clear separation between our 
U2TP Test Model implementation and whatever format the implemented 
behavior may take for a given TestCase type.  In short (or perhaps in 
long), we found many reasons to like this approach.

2) Based on the above, we identified and refined another extension point 
for Hyades -- a Behavior Editor Adapter (the name will certainly change 
to something better.)  When a user gestures to open a TestCase behavior, 
the Hyades TestTool will look up the Behavior Editor Adapter that is 
registered for this TestCase type (*).  It will hand that adapter the 
two attributes that will be stored for the behavior of that TestCase -- 
a URL (relative to the workspace for Hyades 1.0) and a behavior location 
(a string that specifies the behavior's location within the resource 
referenced by the URL -- can be empty).  The adapter is then responsible 
for opening the resource and navigating to the specified location.  The 
same flow will also apply for opening a TestSuite behavior.

Please post any concerns or comments that you have on these issues.  
We'll review them in the upcoming Test Model and UI meetings as well.

Thanks, 
--Joe 

(*) In truth, we will first check the preferences to see if the user has 
specified a preferred adapter.  It's not clear to me whether we will 
expose the Behavior Editor Adapter separately, or incorporate it into a 
larger adapter that encompasses other type specific extensibility points 
-- it's somewhat of a user experience question as well.  But the point 
is that we are not mandating that one and only adapter will exist for a 
given type.  It is expected that some adapters will be shared (such as 
the one that knows how to take a java source file and open it, scrolled 
to a specified method), and it is also allowed that a given type will be 
supported by more than one adapter (in which case we'll observe the 
preference if it is specified, and we can choose a strategy for dealing 
with the case in which a preference is not specified.)  We could also 
support an Open With command from the element in this case.


Joe Toomey 
Senior Staff Software Engineer 
Rational Software 
IBM Software Group 
tel: (781) 676-7668 
fax: (781) 676-7640