RE: [hyades-dev] integration vs interfacing, was: goals for script/recor

["Saliba, James" <James.Saliba@xxxxxx>]

From my point of view the larger portion our audience is not doing "green fields" development; people are extending and build on existing products/projects.  Therefore we must make Eclipse/TPTP attractive enough for people to want to switch.   I believe this would include items from three general areas:

1) Test/Project management (really another topic)
	How many tests do we still need to write?
      How many of the tests are passing/ fail?
	How many of the tests past before or failing now?
	How many defects are there?...
	Many teams don't have this level of management and sorely need it.  By providing this management support and reporting capabilities (canned or not) people have reason to come over.
	
2) Existing test cases   (interfacing)
	The testing teams already have a large investment in test cases.  We do not want them to rewrite them.  These test cases can be in the form of scripts (perl, python, etc.), open source tools (Grinder, HttpUnit etc.), and/or commercial tools (winrunner etc.)  
	Interfacing tools allows teams to keep their current investment.
 

3) The ease of writing new test cases with Eclipse/TPTP and the promise of further tool plug-in's from open and commercial sources (integration?)
	U2TP to auto generate test cases etc.
	Also, there are many testing tools that are junit extensions.  It seems that every time I talk to someone I find out about a new one. Should our Junit derivate support select a few tools and integrate them or do we provide a way for people to integrate any tool they want by giving a few examples?

Regards,
Jim Saliba
	




-----Original Message-----
From: hyades-dev-admin@xxxxxxxxxxx [mailto:hyades-dev-admin@xxxxxxxxxxx] On Behalf Of Michael.Norman@xxxxxxxxxxxxx
Sent: Friday, September 17, 2004 2:16 PM
To: hyades-dev@xxxxxxxxxxx
Subject: RE: [hyades-dev] integration vs interfacing, was: goals for script/record/play

Tom

You may or may not have noticed I was being deliberately provocative.  
Thanks for being such a good sport. I hope we can have a good con-call 
in a week or so's time. You may be amused that I argued strongly against 
allowing External Behaviours on Hyades-dev, lost the argument and now we 
use them in our product. I think a lot of the technical and 
non-technical issues are out on the open and we've brought some of those 
newer to Hyades a bit more up to speed. 

Our charter and the Eclipse project allow us to build both tools and 
frameworks.  Basically we can go 2 ways here, build a sample tool that 
maps scripts from the Junit derivatives into Hyades Behaviours, or 
integrate them all as external behaviours.  There's probably some 
intermediate position, and there may not be a one-size-fits-all soution.

Mike
-----Original Message-----
From: tlroche [mailto:tlroche@xxxxxxxxxx]
Sent: 16 September 2004 19:09
To: hyades-dev
Subject: [hyades-dev] integration vs interfacing, was: goals for
script/record/play


Note that, at some point, we quit talking about recorders and start
talking about entire projects:

Tom Roche Wed, 15 Sep 2004 16:54:25 -0400
>> * At some point, we should decide what is preferable between

>> · "recorder interfacing": i.e. providing an interface between an
>>   external script tool (EST) and the HTrM. One way to do this is to
>>   facilitate "[integration of ESTs] as external behaviours which
>>   happen to fire some stuff back into the execution history."
>>   Another way might be to read/write or import/export ("trace-test
>>   conversion") the scripts generated by an EST. In either case, the
>>   user would presumably be interacting separately with the EST (to
>>   record/play) and Hyades (for other magic).

>> · "recorder integration": i.e. "[getting] the Hyades trace stuff
>>   to act as a generic recorder [and subsuming] the [ESTs]."

>> · something else entirely. Am I missing one or more
>>   alternatives?

Michael.Norman Wed, 15 Sep 2004 16:43:15 +0100
>>> Which of these were you thinking of?

>> Recorder interfacing, specifically via read/write/import/export
>> (see "hub format" above). This is because

>> * I didn't know about the HTrM (and don't know enough about it now
>>   :-)

>> * it seems like that would get us "off the ground" quicker, at
>>   least for tools that script in XML (which I prefer anyway, for
>>   the reasons above).

>> * it seems more extensible. IMHO there are always gonna be other
>>   record/play tools, just as there are JUnit extension domains that
>>   we have not tackled. (Not to mention xUnit domains.) If we create
>>   a clean HTrM interface, we can point other tools at it and say,
>>   "here's how you can Play Well With Hyades."

>> This is not to say that I oppose recorder integration, just that it
>> seems like more work. Am I missing something?

>>> Your postings to hyades-dev and abbot-users give entirely different
>>> impressions of this!

>> Can you cite the posts? Because IIRC I've not discussed integrating
>> Costello with Hyades outside this thread. What I _have_ previously
>> discussed is creating a plugin that would emit CBEs on
>> Abbot-specific events (e.g. {Component, Widget}NotFoundException,
>> Multiple{Component, Widget}sFoundException), which seems to me like
>> a separate issue (i.e. logging).

>> Is that the same issue? Or were you referring to something else?

Michael.Norman Thu, 16 Sep 2004 14:04:49 +0100
> I am poking at is the idea that Hyades creates a new Hyades Sample
> tool, sharing code with the existing sample tools that Kent's team
> and others have been building, and following all the same
> architectural guidelines, 

hold that thought! more at end

> and that this tool can import existing Abbot test assets, and
> becomes the migration path for Abbot users, with the standalone
> Abbot/Costello implementation disappearing. [Obviously] there are
> [sensitivities] here because it might be perceived as an Open Source
> take-over bid "Hyades takes over Abbot",

and the other tools with which we wanna work (e.g. HTTPUnit, Cactus).
Frankly, you're gonna hafta sell this "migration path" to the
stakeholders: all these tools have 

* existing users (not all of whom are Eclipse users, either)

* developers

What's in it for them?

> it's just that I don't think the [fragmentation] of open source
> initiatives to wrapper up JUnit in different ways for different
> purposes is helping anyone, 

You're gonna hafta make that case. Remember, there are a LOT of folks
happily using these tools already. Not nearly enough (and not as many
as there could be with better tooling, but still a lot, and not as
happily as they might like (lots of traffic on lots of lists), but
still they're getting things done.

> and I think in Hyades we have all the bits to do this in a
> one-size-fits all manner and we have a mandate in the charter to
> drive standardization.

> The advantage is that Hyades has a lot of resource (most of it busy it
> has to be said) and is expecting significant additional resource over
> the next few months, and the various process (hopefully) ensure
> production-quality code with documentation and maintenance. The 
project
> also has well-defined processes to consider what it is doing and how 
it
> applies resource, so this at the moment is not a proposal only an idea
> that I am floating, and I hope I'm not annoying anyone too much!

No, and it's better to talk these ideas over--many eyes make better
proposals, too!

ISTMT you are very much in favor of integration (in its extreme form,
"consumption" :-) To the extent that I understand your position
correctly (which I may not), I think this is the wrong approach, in
fact, an "anti-Eclipse" one:

* IMHO the success of Eclipse has been not only due to the excellence
  of its implementations, but its ecologic ethos. IMHO Eclipse has
  provided a platform (== set of interfaces) to which anyone can plug
  in, and thus become a successful ecosystem (or community, if you
  prefer). It has not been about "disappearing."

* Another major advantage of inviting (i.e. interfacing) vs consuming
  is "collaboration capacity": it's a lot easier to have lots of
  people contributing when you have loose coupling than tight.
  Tightening the coupling (i.e. achieving integration) requires lots
  more project management--but you know that already :-)

That being said, ISTMT that you feel Hyades has the resource to create
a "reference implementation" (i.e. "Hyades Sample tool," HST below)
that would be greater than the sum of its parts (synergy being the
main motivation for integration). So I say, prove it :-)

If you've got the resources, why not do an integrated HST? The
developers of the selected integrands will _probably_ be willing to
advise you where their tools need work as long as they can reuse any
code in their existing projects (but, hey, that's why we're all CPL).
If the resulting HST is really a "better mousetrap," I suspect many
users will migrate from the existing projects--after all (AFAICS) most
users of JUnit extensions are developers who are automating the test
of their own code (i.e. they own the sources of their SUTs, they are
not conventional "testers"), and not primarily interested in
developing test tools per sè.

But you're gonna hafta *prove* that Hyades is a better home for these
{projects, communities, codebases} than the ones they have now, and
you're gonna hafta keep proving it (or else folks will migrate back to
"their old homes"). Maintenance is always the challenge: will you
always have the resource?

Frankly, I advise instead creating an extensible platform, and
demonstrating the ease of plugging into it, because you will never
have as much resource as the entire JUnit community. (Not to mention
xUnit--at some point you want Hyades to support other languages,
frameworks, and targets, n'c'est pas?)

In any case, why not start by publishing (or pointing to) the
"architectural guidelines [used by] the existing sample tools that
Kent's team and others have been building," so we can get a better
idea of what's involved in integration? Or, alternatively, if those
guidelines are better interpreted as platform interfaces?

FWIW, Tom Roche, IBM Rational Developer Model2 Tooling, abbot admin

_______________________________________________
hyades-dev mailing list
hyades-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/hyades-dev

_______________________________________________
hyades-dev mailing list
hyades-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/hyades-dev