Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [hyades-dev] black- and white-box, was: integration vs interfacing


Boy have we given you the right drugs Jim ;-)
You clearly have the message and intent of the project in your head.

Thanks for your time.
--------------------------------------------------------------------------
Harm Sluiman, STSM,  
phone:905-413-4032   fax: 4920  
cell: 416-432-9754
mailto:sluiman@xxxxxxxxxx
Admin : Gabrielle Chapman chapmaga@xxxxxxxxxx  Tie: 969-2323



"Saliba, James" <James.Saliba@xxxxxx>
Sent by: hyades-dev-admin@xxxxxxxxxxx

09/22/2004 10:07 AM

Please respond to
hyades-dev

To
<hyades-dev@xxxxxxxxxxx>
cc
Subject
RE: [hyades-dev] black- and white-box, was: integration vs interfacing





<snip>
>At some point current terminology, which assumes all testing is
>manual, and that developers don't test, will need to change--if one
>develops automated stress tests for application(s) to which one lacks
>access to sources, is one a "tester" or a "developer"? But for now,
>when I say "developer" I mean someone whose primary responsibility is
>to deliver sources (no matter how much testing of those sources s/he
>might do) and by "tester" I mean someone whose primary responsibility
>is to verify the quality of an application or its deployment (no
>matter how much coding s/he might do) and who lacks access to the
>target's sources.

>1 Presently testers are far better served by current tooling than are
>  developers. There is already a lotta black-box tooling out there.
>  Certainly much more time/energy has been invested in developing
>  black-box tooling (for dynamic testing) than has been invested in
>  white-box tooling. (Static analysis is a separate matter, and not
>  what I mean by "white-box" or "developer-time" testing.)


I can not disagree that there already is "a lotta" black-box tools out
there.  From my experience in running QA/testing teams this is also a
problem.   None of the tools work together.  The tests scripts are saved
in multiple formats and in multiple repositories make analysis and
project status difficult.   The power of TPTP is to bring these tools
together, save the test assets in one place hopefully in the same or
parallel change control system branch as the product code.  Many JUNIT
extensions support white-box testing while other addresses Black-box.
With Eclipse/TPTP & JUNIT they can, (and I believe they should) live
together.  Combine this with some enhancements to test management,
automated execution and general reporting, TPTP can house an amazing set
of test tools support the whole life cycle from product design to
production.  (Sorry, I'm off my soapbox now :)


Regards,
Jim Saliba

-----Original Message-----
From: hyades-dev-admin@xxxxxxxxxxx [mailto:hyades-dev-admin@xxxxxxxxxxx]
On Behalf Of Thomas L Roche
Sent: Tuesday, September 21, 2004 9:59 PM
To: hyades-dev@xxxxxxxxxxx
Subject: [hyades-dev] black- and white-box, was: integration vs
interfacing

"Saliba, James" Fri, 17 Sep 2004 17:13:20 -0400
>>> From my point of view the larger portion our audience is not doing
>>> "green fields" development; people are extending and build on
>>> existing products/projects. Therefore we must make Eclipse/TPTP
>>> attractive enough for people to want to switch. I believe this
>>> would include items from three general areas:

<big snip>

>>> 3) The ease of writing new test cases with Eclipse/TPTP

Tom Roche 19 September 2004 20:21
>> I would add one more

>> 4) The ease of maintaining test cases developed with Eclipse/TPTP

>> FWIW 3 and 4 are my primary concerns: specifically, reducing the
total
>> lifecycle cost of testing for *developers*, in order that they and
>> their management will see the (admittedly overhead) cost as
>> acceptable, and begin to consistently/continually do automated
>> testing, rather than pushing it off to "dedicated" or "3rd party"
>> acceptance-type/black-box testers. But that's a topic for another
>> thread ... suffice to say that, IMHO, while black-box tools have made
>> progress in lowering the cost of _creating_ tests, tests thus
>> created/run will never be as maintainable (or even, in many cases, as
>> easy to develop) as those developed using xUnit-based tools that

>> * allow developers to create tests as (or even before) they develop
>>   their sources

>> * leverage developers' {access to, knowledge of} their sources

>> * are more easily integrated into development tools and processes

>> <off soapbox/>

Michael.Norman Tue, 21 Sep 2004 20:36:08 +0100
> From the size of the soap box obviously this white-box vs. black-box
> thing exercises a lot of people, but personally I've always seen
> value in both approaches, and I think amongst Hyades launch goals
> was to be methodology-neutral.

Agreed. To express the above better, v0.1:

AFAICS most Hyades developers come from a black-box background, and
are mostly interested in doing more/better black-box tooling.
Black-box is not bad! but IMHO

0 The nature of defect-cost increase ensures that (ceteris paribus)
 there are more potential cost savings to be gained from increased/
 improved testing by developers (and architects, but that's another
 thread) than from increased/improved testing by traditional,
 acceptance-style testers.

At some point current terminology, which assumes all testing is
manual, and that developers don't test, will need to change--if one
develops automated stress tests for application(s) to which one lacks
access to sources, is one a "tester" or a "developer"? But for now,
when I say "developer" I mean someone whose primary responsibility is
to deliver sources (no matter how much testing of those sources s/he
might do) and by "tester" I mean someone whose primary responsibility
is to verify the quality of an application or its deployment (no
matter how much coding s/he might do) and who lacks access to the
target's sources.

1 Presently testers are far better served by current tooling than are
 developers. There is already a lotta black-box tooling out there.
 Certainly much more time/energy has been invested in developing
 black-box tooling (for dynamic testing) than has been invested in
 white-box tooling. (Static analysis is a separate matter, and not
 what I mean by "white-box" or "developer-time" testing.)

Therefore

2 Presently the marginal benefit likely derivable from investment in
 tooling intended to assist developers to do more/better automated
 testing greatly exceeds the marginal benefit likely derivable from
 investment in tooling intended to assist testers to do more/better
 automated testing.

How to do that?

3 "White-box" or "source-enabling" tools (like JUnit and its
 extensions) that

* allow developers to create tests as (or even before) they develop
 their sources

* leverage developers' {access to, knowledge of} their sources

* are more easily integrated into development tools and processes

 are more likely to realize the potential savings of developer-time
 testing. IMHO, while black-box tools have made progress in lowering
 the cost of _creating_ tests, tests thus created/run will never be
 as maintainable (or even, in many cases, as easy to develop) as
 those developed using xUnit-based tools possessing the properties
 listed.

This does not mean that JUnit extensions cannot be used to
acceptance-test or drive deployments, only that black-box tools are
(IMHO empirically) suboptimal for developer-time testing.

Conclusion:

4 I would like to see more Hyades effort devoted to assisting
 *developers* to do more/better automated testing using JUnit
 and its extensions.

_______________________________________________
hyades-dev mailing list
hyades-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/hyades-dev



_______________________________________________
hyades-dev mailing list
hyades-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/hyades-dev


Back to the top