[news.eclipse.tools.hyades] Re: swt.Robot, was: automate GUI testing?

[mgn@xxxxxxxxxxxxx (Mike Norman)]

I'd like to enforce Harm's point about the top vs. bottom issue.  There is
(or should be) a balance in the Hyades material about these two approaches
reflecting the different world views of the respective vendors. 

The key here is that the test model and the SUT model are different.  The
test model describes the test; the SUT model describes the system under
test. So, for example, if you take a standard Windows GUI scripting tool
the "test model" is the script executed by the tool. The SUT model is very
light, and really only defines the set of Windows API calls which the
script can reference. 

One important area of hyades is that by mapping through a standard form of
test model, the scripts become interoperable so that any hyades-compliant
tool can manipulate them.  There is also another key interoperability
interface which is only briefly discussed in the documentation, a 
self-describing testability interface which is layered on top of the SUT. 
This is a standard way of presenting interface points of a SUT to a test
execution engine. It should lead to the type of interoperability that you
refer to.  This kind of architecture is currently rare, but we (Scapa
Technologies) do this and it's a part of the TTCN standard for embedded
testing, known as TRI.

I'm very interested in feedback in this area.

Do you think what I'm suggesting does actually solve the problem you are
concerned with?  Have I explained it?

Mike Norman,
Hyades project lead



Harm Sluiman wrote:

> Thanks for your thoughtful input Tom,

> To net it out, you are right in your observations and request. In fact it
> is a critera we are working from, that top down need not ever be
> considered by the user. A continuous \"bottom up\" level usage model is
> just as, if not more, important.

> When I mentioned the EMF model, this is not just for top down usage. We
> will use models to capture the results of execution as well as the
> configuration and definition of the test case. Behind every UI is a model.
> A very tiny model in some cases.

> There are in fact already some tools that can adapt to various GUI
> component models. Rational Robot for one, which can be used for SWT as
> well as AWT and Swing. We are considering very carefully where we should
> be doing things above SWT vs creating requirements on SWT itself. As
> Eclipse developers we all have a keen interest in being able to test our
> UI code well, and incrementally (with minimal maintainance I might add).

> As we get deeper into the design details and impl of this particular facet
> of Hyades (which will be very soon), I will be looking for your helpful
> insight.

> Tom Roche wrote:

> > Tom Roche wrote:
> >  >> Will Hyades

> > (http://www.eclipse.org/hyades/ for the folks on java-gui-testing)

> >  >> provide functionality for GUI test automation?

> > <snip>

> >  >> I\'d like to use something like Abbot

> >  >> http://abbot.sourceforge.net/

> >  >> but it relies on java.awt.Robot, and AFAIK there is no SWT analog.

> > sluiman@xxxxxxxxxx (Harm Sluiman) Sun, 22 Dec 2002 18:46:54 +0000
> >  > One of the most important things we will provide is an EMF model of
> >  > the test case structure, and a way to plug in (with good examples)of
> >  > how to generate runtime required artifacts.

> > Great, but allow me to suggest that your initial efforts not be too
> > entirely \"top-down.\"

> > It seems to me that Hyades wants most to enable a mode of development
> > in which test is fully integrated into design model, so that one\'s
> > test artifacts could be generated from The Model along with code,
> > documentation, etc. This is certainly a valid vision.

> > In any case, to realize this vision (completely), you\'re gonna need a
> > means to stimulate GUI components of SUTs (or SUTs through their
> > GUIs). java.awt.Robot is doing this now, in the Swing world. If you
> > were to provide an API-compatible \"swt.Robot,\" you would be able to
> > build a \"cross-Java\" GUI automation toolkit. You would also be able to
> > build upon a large amount of existing Swing-based work, including:
> > Abbot (above), JFCUnit

> > http://jfcunit.sourceforge.net/

> > Jemmy

> > http://jemmy.netbeans.org/

> > MarathonMan

> > http://marathonman.sourceforge.net/

> > XTest

> > http://xtest.netbeans.org/

> > to name just a few. An swt.Robot would also presumably make Eclipse
> > and SWT/JFace much more attractive to NetBeans and AWT/Swing users.

> > Furthermore, by providing GUI automation sooner, rather than later,
> > you would also be making Hyades relevant to XP-style developers. Note
> > that I\'m NOT interested in discussing the merits of XP vs more
> > model-oriented styles of development. I _am_ interested in test-first
> > development, as are practitioners of many (self-proclaimed) \"agile
> > methodologies.\" Their vision may be different from yours (and yours
> > may not be as I have described), but the entire Eclipse community\'s
> > interests are served by making SWT/JFace as testable as AWT/Swing.
> > As you note

> >  > I think getting Swing, AWT, SWT, HTML as GUI environments covered,
> >  > as well as non-visual environments like Java and Web Services will
> >  > be some of our top priorities. After all these are all environments
> >  > we all want to improve the quality of and automation of these
> >  > componentized environments is key to making that happen in a
> >  > productive fashion.

> > (And the entire Java community is served if the stimulation API is
> > consistent.)

> > So allow me to suggest that you make _everyone_ happy :-) by attacking
> > this problem from the bottom up, as well as the top down. Note also
> > that providing an \"swt.Robot\" is already a feature request in Bugzilla

> > http://bugs.eclipse.org/bugs/show_bug.cgi?id=15025

> > and the WebSphere Studio Feature Request Database

> > https://www7b.software.ibm.com/webapp/wsdd/studioServlet3

> > (scroll to #64).