[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [hyades-dev] Re: [JUnit extensions] goals for script/record/play


Hyades is notoriously difficult to understand the insides of. I think we 
spent the first year or so trying to find a common language and ended up 
with this hybrid vocabularly that doesn't correspond well to anything 

Anyway, quite obviously I have limited knowledge of JUNIT derivatives 
although some of our guys have worked with Abbot in particular and were 
very positive about it and in fact we support it as an external 
behaviour to our existing commerical testing tool (which in turn is an 
external behaviour as far as Hyades is concerned), and we have a bunch 
of additional APIs which filter back through a circuitous route into the 
Hyades Execution History. 

However this isn't what I am concerned with just now, what I am poking 
at is the idea that Hyades creates a new Hyades Sample tool, sharing 
code with the existing sample tools that Kent's team and others have 
been building, and following all the same architectural guidelines, and 
that this tool can import existing Abbot test assets, and becomes the 
migration path for Abbot users, with the standalone Abbot/Costello 
implementation disappearing. Oviously there are senitivities here 
because it might be perceived as an Open Source take-over bid "Hyades 
takes over Abbot", it's just that I don't think the fragemntation of 
open source initiatives to wrapper up JUnit in different ways for 
different purposes is helping anyone, and I think in Hyades we have all 
the bits to do this in a one-size-fits all manner and we have a mandate 
in the charter to drive standardization.

The advantage is that Hyades has a lot of resource (most of it busy it 
has to be said) and is expecting significant additional resource over 
the next few months, and the various process (hopefully) ensure 
production-quality code with documentation and maintenance. The project 
also has well-defined processes to consider what it is doing and how it 
applies resource, so this at the moment is not a proposal only an idea 
that I am floating, and I hope I'm not annoying anyone too much!

Mike Norman
Hyades Test Project lead

-----Original Message-----
From: tlroche [mailto:tlroche@xxxxxxxxxx]
Sent: 15 September 2004 21:54
To: hyades-dev
Subject: [hyades-dev] Re: [JUnit extensions] goals for

Tom Roche 14 September 2004 18:32
>>>> I'm wondering how folks feel about the following:

>>>> 0 The importance of record/play. My impression is that (ceteris
>>>>   paribus) we prefer JUnit extensions that support record/play
>>>>   to tools that do not (e.g. that only provide API with which
>>>>   users can write JUnit test methods). I would further assume
>>>>   that, c.p., we would prefer tools that provide *both* API and
>>>>   record/play to tools that provide [either] only API or only
>>>>   record/play. Does anyone disagree? If so, why?

Until someone disagrees, I'm assuming universal agreement on that ...

>>>> 1 The importance of human-writable formats. I consider
>>>>   script/record/play to be 3 sides of the same "aspect," since I
>>>>   assume that

>>>> * one's recorder will hafta record (write) to some format

>>>> * one's player will play (read) from that format

>>>> * that format should be read/writable by humans, so that folks
>>>>   can (minimally) edit their scripts for maintenance or
>>>>   (maximally) write their own scripts without needing to record
>>>>   everything

>>>>  Does anyone disagree? If so, why?

... and that. However ...

>>>> 2 The desirability of supporting script/record/play with
>>>>   procedural languages other than Java. My bias is toward
>>>>   persisting to either Java (since that's what the workbench
>>>>   currently best supports) or XML (since it's

>>>> * easily played-back by JUnit: e.g. Abbot provides a
>>>>   ScriptTestSuite and ScriptFixture that do this

>>>> * more cross-compatible: i.e. one can use XSLT to generate some
>>>>   procedural language from XML more easily than one can
>>>>   translate one procedural language into another

>>>>  ). I'm wondering, are there compelling reasons to persist to a
>>>>  non-Java procedural language?

... is more open, but see Hyades Trace Model below. (I'm assuming
that's EMF, no?) Even more open:

>>>> 3 The desirability of supporting one script/record/play format
>>>>   for all supported tools. Should we prefer to support tools
>>>>   that share some common persistence format? Should we provide
>>>>   some common "hub" format to which tools could plug in? Or
>>>>   should format not be a consideration, at least at this stage?

Michael.Norman 09/15/2004 06:52 AM 
>>> THis kind-of relates to an old issue of the replayability of the
>>> Hyades trace model - the old trace/test duality.

>>> There was a lot of discussion of this and I think the point we
>>> got to was that the current Hyades Trace Model would be extended
>>> to cover method data, C/C++ and protocols such as HTTP and would
>>> become the recording persistence format for the various sample
>>> tools.

>>> Then, the traces themselves wouldn't necessarily be used as
>>> replayable test models, there would be a process of trace-test
>>> conversion which mapped between them, and obviously


>>> allowed you to hang datapools and other things around them.

Just to clarify: I don't work in a Hyades group, and my "real job" is
not as a developer for test tooling. (Not that I wouldn't like to do
that, but until I can get a paycheck out of it, test tool development
is just an all-too-infrequent-engaged hobby.) In relation to Hyades,
think of me as a (hopeful) user, i.e. someone whose knowledge of "the
old trace/test duality" and the Hyades Trace Model (which I'll call
"HTrM" below) is miniscule.

Pointers to more information would be appreciated, esp regarding

* the Hyades Trace Model

* the Hyades Test Model

* how they differ

sluiman 15 September 2004 15:18
>> Yes a review of these in the f2f

* the "Hyades glossary" :-)

>> is very appropriate.

>> There are calls being run by Marius on the subject of adding
>> protocol etc. support to the trace model, starting with required
>> parameter capture. As that comes on line the http recorder will
>> have to migrate to that model. The base spec is due in the 3.1
>> time frame, and the impl is targeted in 3.2 based on our plan.

When you say "the http recorder," does that refer to one in Siefkes'
tooling, or to one included in one of the URL test tools under

>> UML2 is still a long way out with what adoption really means and
>> U2TP to be spec'd for 3.1. This work is not on track at this time.
>> The UML2 model project is also not yet confirmed with dates so
>> this topic is going to remain long range.

>> Dominique is driving the alignment of our test environments to
>> JUnit for the Java space with the first cut spec in 3.1 and the
>> impl currently targeted for 3.3.

>> So once the model is figured out for at least capture we can start
>> the work to record to the model and then the steps of creating
>> tests from recordings. Each "recorder" that is built to create the
>> skeleton of a test needs to be done in this context and any
>> requirements they drive will need assessment. It is this
>> multiplicity that makes this a somewhat tedious at times effort of
>> compromise ;-)

Michael.Norman Wed, 15 Sep 2004 16:43:15 +0100
> I think I understand Harm's response and I think it's what I thought
> the plan was from the Hyades project side. In other words the sample
> tools record into the trace model, the tests are generated from the
> traces, the editor and the codegen operate relatively generically on
> the tests.

> So this would give us a generic way of dealing with at least some of
> these types of tools e.g. Abbot & Costello by mapping their existing
> persistence formats

Just to clarify: Costello is the recorder, which "has" the format.
Abbot contains API (for writing JUnit test cases, and allowing a JUnit
test runner to run Costello scripts) as well as Costello.

> through to trace models describing interactions with the APIs and
> then running our generic editor and generic codegen on them. All we
> need is to get the Hyades trace stuff to act as a generic recorder
> and we've subsumed the tools. We can then release the thing inside
> RCP and call it Abbot & Costello ;-)

> However, there's another way of doing this which is to take the
> tools more or less as is with their own editors recorders and replay
> engines and integrate them as external behaviours which happen to
> fire some stuff back into the execution history.

OK, barring further correction, it sounds to me like

* We prefer JUnit extensions with recorders to those that don't. (I'd
  still like to know how folks feel about script format.)

* For now, if a JUnit extension has its own recorder, we use it as is.

* At some point, we should decide what is preferable between

· "recorder interfacing": i.e. providing an interface between an
  external script tool (EST) and the HTrM. One way to do this is to
  facilitate "[integration of ESTs] as external behaviours which
  happen to fire some stuff back into the execution history." Another
  way might be to read/write or import/export ("trace-test conversion")
  the scripts generated by an EST. In either case, the user would
  presumably be interacting separately with the EST (to record/play)
  and Hyades (for other magic).

· "recorder integration": i.e. "[getting] the Hyades trace stuff to
  act as a generic recorder [and subsuming] the [ESTs]."

· something else entirely. Am I missing one or more alternatives?

Is that correct?

> Which of these were you thinking of?

Recorder interfacing, specifically via read/write/import/export (see 
"hub format" above). This is because

* I didn't know about the HTrM (and don't know enough about it now :-)

* it seems like that would get us "off the ground" quicker, at least
  for tools that script in XML (which I prefer anyway, for the reasons

* it seems more extensible. IMHO there are always gonna be other
  record/play tools, just as there are JUnit extension domains that we
  have not tackled. (Not to mention xUnit domains.) If we create a
  clean HTrM interface, we can point other tools at it and say,
  "here's how you can Play Well With Hyades."

This is not to say that I oppose recorder integration, just that it
seems like more work. Am I missing something?

> Your postings to hyades-dev and abbot-users give entirely different
> impressions of this!

Can you cite the posts? Because IIRC I've not discussed integrating
Costello with Hyades outside this thread. What I _have_ previously
discussed is creating a plugin that would emit CBEs on Abbot-specific
events (e.g. {Component, Widget}NotFoundException, 
Multiple{Component, Widget}sFoundException), which seems to me like a
separate issue (i.e. logging).

Is that the same issue? Or were you referring to something else?

TIA, Tom Roche, IBM Rational Developer Model2 Tooling, abbot admin

hyades-dev mailing list