[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [hyades-dev] Interest in contributing a test execution framework
|
Mike:
I understand and agree that there should be a cleavage point in what I
described below between the free, basic framework and value added layers
above. We have some thoughts on that, which I would be happy to speak
with you about at your convenience.
The purpose of my note was to see if there was any interest at all by
the group in making use of an existing framework, or if the problem has
already been solved. I'm glad to see that you all are still in the early
stages.
Having read the UML Testing Profile slides, I believe a basic subset of
the framework could easily serve as an executable representation of the
artifacts described, and provide a basis for additional tooling while
leaving room for commercial endeavors to be built on top of it.
Please feel free to contact me at your convenience at any time after
2:30 GMT at the number listed below, or let me know when to contact you.
I will be available every day this week except Friday.
- Mike
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mike Silverstein voice: (919) 858-8300 x29
SilverMark, Inc. fax: (919) 858-8308
5511 Capital Center Drive mobile: (919) 669-5906
Suite 510, Raleigh, NC 27606 toll free: (888) 588-0668
http://www.silvermark.com msilverstein@xxxxxxxxxxxxxx
> -----Original Message-----
> From: hyades-dev-admin@xxxxxxxxxxx
> [mailto:hyades-dev-admin@xxxxxxxxxxx] On Behalf Of
> Michael.Norman@xxxxxxxxxxxxx
> Sent: Monday, January 06, 2003 6:41 AM
> To: hyades-dev@xxxxxxxxxxx; msilverstein@xxxxxxxxxxxxxx
> Subject: RE: [hyades-dev] Interest in contributing a test
> execution framework
>
>
> Mike
>
> There is a guiding principle in this Hyadfes stuff. We need to leave
> space so we can all release commercial tooling built on top of the
> framework. This means that above all the project is about tools
> interoperability and not about providing a best-of-breed toolset into
> the public domain, and the decisions to put specific tools into the
> public domain are very sensitive. Some clarity over the
> natural boundary
> between paid-for and public domain tooling will emerge over
> the next six
> months or so.
>
> On this basis I think it is unlikely that your entire framework would
> make sense in the open source project, at least in the short term.
> However, it would be useful to identify a cleavage point in your
> architecture, the layers below which are supported by the
> public domain
> architecture, and the layers above which are your value-add. If
> modifications are required to allow the public domain architecture to
> support your tooling, these can form your contribution to the open
> source effort.
>
> What I will say is that although a a trace model will be
> released quite
> soon, the test model is not yet in a state where we can
> release it. The
> initial cut is based upon the OMG RFP revised initial response which,
> although not yet available from the OMG is to be found at
> http://www.fokus.gmd.de/u2tp/Related_Documents/related_documen
> ts.html#Latest%20Document
> Beyond this there will be certain modifications and extensions, which
> are due to be thrashed out amongst the initial participants
> at the end
> of the month. At that point something will be put into the
> public domain
> and you can start playing.
>
> I hope this makes sense.
>
> I'd be happy to talk to you about this stuff if you want. Please
> suggest some times for a call. I'm GMT, so avoid too late in the
> afternoon.
>
> Regards,
>
> Mike Norman,
> CEO Scapa Technologies
> Hyades project lead
>
>
>
>
>
> -----Original Message-----
> From: msilverstein [mailto:msilverstein@xxxxxxxxxxxxxx]
> Sent: Friday, January 03, 2003 7:32 PM
> To: hyades-dev
> Subject: [hyades-dev] Interest in contributing a test execution
> framework
>
>
>
> Hello:
>
> We'd like to gauge interest in a potential contribution to the Hyades
> project by SilverMark. We've been creating object testing tools since
> 1996 and have a framework that may be of use to the project. I'll
> briefly describe it without burdening you all with a lot of
> details at
> the outset.
>
> This is a test execution framework implemented in Java, based on
> concepts described in "Automated testing of Object
> Oriented-Components
> Using Intelligent Test Artifacts",
> http://servlet.java.sun.com/javaone/javaone2000/pdfs/TS-708
> and used as
> a basis for the SilverMark's Test Mentor - Java Edition (
> www.javatesting.com) commercial product. The essence of it is this:
>
> * The framework provides objects that each possess a type
> of testing
> behavior. These objects are called steps.
> * Tests are composed of instances of steps, configured to perform
> specific testing tasks.
> * The tests themselves may be considered "executable
> models". One of
> our areas of interest is to use the framework as basis for an
> executable
> form of the UML Testing Profile.
> * There are a variety of types of steps, and it is fairly easy to
> extend the framework to provide additional behaviors. Some
> examples of
> the behavior provided by specific step types are:
>
> * calling methods and constructors
> * setting/getting fields
> * parsing and iterating over test data
> * referencing test assets (asset reuse)
> * sequencing other test steps
> * running a piece of Java code (script)
> * recursively extracting object state
> * validation (assertion-like behavior)
> * calling out to a JUnit test
> * accessing distributed objects such as EJBs or through
> distribution
> schemes such as RMI or CORBA
>
> * Steps also share common properties, such as:
>
> * name and description, for documentation
> * expected exception (class of exception expected to be thrown by
> executing the step)
> * number of times to execute the step
> * performance criterion (maximum time step should take to
> execute)
> * precondition
> * postcondition
> * failure (step to execute if the owning step fails for
> any reason)
> * invariant (step to execute, which performs an assertion on some
> state that is assumed to be true under all circumstances)
> * maximum execution time
>
> * All of these common properties except for name are optional,
> which reduces the overhead of step configuration. Individual
> step types
> have their own configuration properties.
> * Steps know how to represent themselves as XML. The objects that
> represent the results of executing steps also know how to represent
> themselves as XML.
> * The framework includes mechanisms for reusing and passing
> parameters to test assets, as well as iterating over composite
> parameters
> * Test steps implement the JUnit junit.framework.Test
> interface, so
> they look and smell to JUnit like JUnit tests, and can even
> be run under
> JUnit
>
>
> Some benefits of using the framework are (in no particular order):
>
> * You already have a robust, flexible, well tested framework for
> composing and executing a variety of types of tests
> * At execution time test steps use reflection to interact with
> components under test, so complete tests can be implemented
> independent
> of the presence of the system under test
>
> * Steps may be configured incrementally, as more
> information about
> the system under test becomes known.
>
> * Test steps can conditionally override Java's private
> member access
> checks
> * Each test step measures its own execution time, so fine-grained
> performance metrics are gathered
> * Tests and test results are independent of presentation, so a
> variety of views can be created for them, specific to the target
> audience
> * The framework can be easily extended to support
> additional testing
> behaviors
> * The framework can be easily extended to gather additional
> fine-grained metrics during execution. For example, it would not be
> difficult to extend the steps' behavior to measure memory
> consumption or
> code coverage during individual method calls, given the presence of
> callable API for those metrics.
>
>
> I've gone on enough. Please let me know if any of this would be of
> interest to the group.
>
> - Mike
>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Mike Silverstein voice: (919) 858-8300 x29
> SilverMark, Inc. fax: (919) 858-8308
> 5511 Capital Center Drive mobile: (919) 669-5906
> Suite 510, Raleigh, NC 27606 toll free: (888) 588-0668
> http://www.silvermark.com msilverstein@xxxxxxxxxxxxxx
>
>
>
>
> _______________________________________________
> hyades-dev mailing list
> hyades-dev@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/hyades> -dev
>
