Re: [epp-dev] Testautomation for EPP

[Markus Knauer <mknauer@xxxxxxxxxxxxxxxxx>]

Hi all,

I am sure that this will help us to ensure a higher quality of all packages in the future, and when we started the discussion about this at EclipseCon, I was really happy to have a driving force behind this now. In the past there were many thoughts and requests to automate the testing of the packages, but unless there is someone with the time and with the experience nothing will happen in the end.

Last week we had a workshop where we discussed how we can proceed and what are potential priorities (in addition to that I learnt a lot about Jubula). From my perspective it reads like this:

* The first goal is to have some useful automated tests available in the Indigo time frame, i.e. within the next few weeks, and find a way to make the test results available. This does not interfere with our work on Indigo and I already saw some implemented tests running.

* As Felix mentioned above, they will contribute the existing tests via Bugzilla, but in the long term they should be able to enhance and to maintain these tests. Therefore I think the way to go is to give them committer status.

* In order to move the tests to the Eclipse Foundation infrastructure we need the support from the Foundation (and from the webmasters...) - that is something that I would like to start working on after the Indigo release in June.

* And then the tests itself... last week we went through the wiki page that David mentioned in his mail. The wiki page itself is a bit outdated but it is still a good starting point. On the other hand the package maintainers know their projects better than I do and we definitively need a way to add specific tests for each package. This brings up the question who will be implementing these tests. In the end I am dreaming a bit of a infrastructure where other people with the domain knowledge (package maintainers) can easily create and maintain additional tests.

(Whenever I am writing here about 'tests' I don't mean that s.o. should throw away any JUnit tests or any other project dependent tests. This here is about testing *packages* as a whole and all their specific problems in an automated way.)

Many thanks for the offer to work on this,

Markus

(and expect more mails from me in a minute...)

On 5 May 2011 16:00, Schaefer, Doug <Doug.Schaefer@xxxxxxxxxxxxx> wrote:

I’ll reserve judgment before taking a deeper look at Jubula. I’m planning on taking a look at the entire EPP build cycle and would like to see automated testing as part of that, just as we do with other projects. If Jubula is the right technology for that, then I’m all for it.

 

From: epp-dev-bounces@xxxxxxxxxxx [mailto:epp-dev-bounces@xxxxxxxxxxx] On Behalf Of David M Williams
Sent: Thursday, May 05, 2011 9:54 AM
To: Eclipse Packaging Project
Subject: Re: [epp-dev] Testautomation for EPP

 

+1

Sounds very interesting, very useful. I assume once people see it in EPP packages ... they'd be tempted to use it in many projects at Eclipse?

Is this a "replacement" for the Platform's "test framework", or (I'd guess) more of a wrapper, to it and other test runners?

> (which means you just tell us what your are doing now
> to test your EPP package). Smoke test will be
> * Installation tests (unzipping/untarring)
> * file permission tests
> * basic functions: welcome screen, online help, project creation/deletion
> * rough check if all features are installed
> * Installation of additional features and Software Update
> * whatever is needed by a particular EPP package (you tell us)

You probably already know this, but in case not, there are some "common" package testing recommendations that might serve as a rough starting point, at http://wiki.eclipse.org/EPP/Package_Testing

Thanks,

Felix Ziesel ---05/05/2011 04:43:08 AM---Hi EPP Team, I work as Test Consultant for BREDEX. We maintain the functional testing

From: Felix Ziesel <Felix.Ziesel@xxxxxxxxx>
To: epp-dev@xxxxxxxxxxx
Date: 05/05/2011 04:43 AM
Subject: [epp-dev] Testautomation for EPP
Sent by: epp-dev-bounces@xxxxxxxxxxx

---

Hi EPP Team,

I work as Test Consultant for BREDEX. We maintain the functional testing
tool Eclipse Jubula (http://www.eclipse.org/jubula). Jubula is part of
the Indigo release.

We'd like to support the EPP team by writing automated smoke tests with
Jubula. We talked to Markus about this topic and came to the conclusion
that this would be helpful to the package maintainers.

The idea is to provide basic testing for all packages and more specific
tests for packages we are familiar with or which provides us with some
sort of test specification (which means you just tell us what your are
doing now to test your EPP package). Smoke test will be
* Installation tests (unzipping/untarring)
* file permission tests
* basic functions: welcome screen, online help, project creation/deletion
* rough check if all features are installed
* Installation of additional features and Software Update
* whatever is needed by a particular EPP package (you tell us).

Of course you are free to specify your own tests with Jubula. We would
be glad to help you getting up to speed.

For the time being we would run the smoke tests on our internal test
infrastructure and publish the result on the Eclipse web site.

After Indigo we would try to move the tests to test instances provided
by the foundation on their servers.

We plan to work on this with 2 people, but not full time. We have a
budget for .5 persons :-) until end of 2012. If this work will be
successful I'm sure we will continue working on it.

We have some tests ready. If there are no -1 on our proposal we will
create a Bugzilla entry and contribute this initial tests.

Thanks and Regards,
Felix Ziesel

--
BREDEX GmbH
Mauernstr. 33
38100 Braunschweig

Tel.: +49-531-24330-0
Fax:  +49-531-24330-99
http: www.bredex.de

Geschäftsführer: Hans-J. Brede, Achim Lörke, Ulrich Obst
Amtsgericht Braunschweig HRB 2450

[attachment "smime.p7s" deleted by David M Williams/Raleigh/IBM] _______________________________________________
epp-dev mailing list
epp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/epp-dev

_______________________________________________
epp-dev mailing list
epp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/epp-dev