Re: [p2-dev] New p2 tests

[John Arthorne <John_Arthorne@xxxxxxxxxx>]

I have created a p2 testing home on
the wiki for details on p2 automated and manual tests:

http://wiki.eclipse.org/Equinox/p2/Testing

Feel free to add more details about
specific areas of the p2 tests that you think would be useful to other
test writers.

John

John Arthorne/Ottawa/IBM@IBMCA
Sent by: p2-dev-bounces@xxxxxxxxxxx

09/29/2008 09:42 PM

Please respond to
P2 developer discussions <p2-dev@xxxxxxxxxxx>

To	p2-dev@xxxxxxxxxxx
cc
Subject	[p2-dev] New p2 tests

Over the past several days we have completed integration of several new
p2 tests suites for the publisher, installer, UI, dropins reconciler, and
framework admin. Here are some things to know about these tests:

- The tests all now require Java 5 to run, due to the dependency on EasyMock.

 - Structurally all but the UI tests fall under the single AutomatedTests
suite in org.eclipse.equinox.p2.tests. The UI tests have their own separate
AutomatedTests entry point because they use the GUI test runner as opposed
to the headless test runner.  This means there are two suites to run
to exercise all the tests. Note that the UI tests also exercise core code
so it is valuable to run both suites even when changes are made in core
areas. 

 - With the exception of the End2EndTest, all of these tests are quite
fast to run, so we should all be in the habit of running these tests before
releasing any code changes. If you're on a slow connection you can either
comment out the End2EndTest, or kill the test run when the last test is
started (this test always runs last). 

- To run the reconciler tests, there is a one-time setup required because
it requires a copy of the platform runtime binary. You need to do the following:
  1) Download the Eclipse 3.4.0 platform runtime binary zip for your
os/ws/arch. 
  2) use a system property to specify the location of this zip in
the test launch configuration (-Dorg.eclipse.equinox.p2.reconciler.tests.platform.archive=<some_path>eclipse-platform-3.4-win32.zip).
The "all p2 tests" launch configuration in the p2.tests bundle
has this property set, but you'll likely need to tweak the path of the
zip for your machine. 

 - We now have a fairly good collection of *real* metadata repositories,
artifact repositories, and legacy update site repositories in org.eclipse.equinox.p2.tests/testData{artifactRepo,metadataRepo,updatesite}.
This makes it fairly easy to write tests involving various kinds of well-formed
and invalid repositories. Feel free to add further reasonably sized repositories
here for testing other code paths, failure cases, regression tests, etc.

One of our goals during our recent test week was to build up our testing
infrastructure so that the time investment required to write new tests
is decreased. Thus we now have quite wide test coverage across p2 components,
but the tests within each component are not particularly "deep".
 The hope is that when it comes down to crunch time and we have little
time for writing tests, having the infrastructure in place will allow us
to churn out regression tests and tests of new function relatively quickly.
Of course this does require that we all have some familiarity with all
of this new testing infrastructure. I would encourage everyone to spend
a bit of time getting familiar with the tests for any areas of p2 they
touch, so that when the time comes you can write new tests quickly. I have
found that once familiar with existing tests and their infrastructure (helper
classes, assert utility methods, test data, etc), you can bang out a regression
test for a bug fix in a matter of minutes. When you don't have this knowledge
it is easy to put off adding new tests when you become very busy because
of the larger time investment needed. 

John_______________________________________________
p2-dev mailing list
p2-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/p2-dev