[wtp-dev] API JUnit test coverage now support run time scanning

[Jeffrey Liu <jeffliu@xxxxxxxxxx>]

In the past, the API test coverage report
is generated based on static scanning. This means the scanner will read
the physical .class files of the JUnit testcases and determine what APIs
are being refereneced. This approach has limitation because it does not
allow the parser to trace through the run time method invocations (methods
called within a method). As a result, developers have to write extra code
to bring the coverage up.

After working with the TPTP folks, we
now have a way to do run time scanning. Here's what you need to do:

1. In org.eclipse.wtp.releng/testScript/test.xml,
instead of using the "runtests" target to run your JUnit tests,
use the "runapitests" target. You need to supply an extra parameter
called "package.includes". The value of "package.includes"
is used to filter out packages that you are not interested in. This is
a required parameter because run time scanning is time consuming, providing
filters helps the scan process to complete faster. Here's an example (for
the J2EE JUnit testcases):

Before:

<target name="jst-j2ee-core-tests"
description="Runs the org.eclipse.jst.j2ee.core.tests test.xml">
        <antcall
target="runtests">
         
      <param name="testPlugin" value="${org.eclipse.jst.j2ee.core.tests}"
/>
         
      <param name="report" value="org.eclipse.jst.j2ee.core.tests"
/>
        </antcall>
</target>

After:

<target name="jst-j2ee-core-tests"
description="Runs the org.eclipse.jst.j2ee.core.tests test.xml">
        <antcall
target="runapitests">
         
      <param name="testPlugin" value="${org.eclipse.jst.j2ee.core.tests}"
/>
         
      <param name="report" value="org.eclipse.jst.j2ee.core.tests"
/>
         
      <param
name="package.includes" value="org.eclipse.jst.j2ee."/>
        </antcall>
</target>

You can also specify multiple filters
like so (space separated):

<param name="package.includes"
value="org.eclipse.wst.server.ui org.eclipse.wst.server.core"/>

Don't go crazy on this because run time
scanning will increase the time to run the JUnit testcases by quite a bit.
Feel free to let me know if you have any questions. Now, there's no excuse
on why you don't have full JUnit testcoverage :)

Thanks,

Jeffrey Liu
IBM Rational Software - Performance Analyst
IBM Toronto Lab.
8200 Warden Ave. Markham, Ontario, L6G 1C7
Internal mail: D3/R8V/8200/MKM (D3-268)
T/L: 969 3531
Tel: (905) 413 3531
Fax: (905) 413 4920
jeffliu@xxxxxxxxxx