[jdt-core-dev] Problems while trying to calculate JDT core tests coverag

[Felipe Barros Pontes <felipepontes@xxxxxxxxxxxxxxxxx>]

Hi all.

   I'm trying to calculate JDT core tests coverage but I'm facing strange problems. I can run tests normally without instrument classes running the following commands:

$mvn clean -P build-individual-bundles package

$mvn -fae verify -V -B -Dmaven.test.skip=false -f pom.xml -P build-individual-bundles

   I tried to execute cobertura maven plugin but results pointed to 0%. It seems maven plugin clean instrumentation information before running tests. Using an adaptation of instructions from https://thomassundberg.wordpress.com/2012/02/18/test-coverage-in-a-multi-module-maven-project/ I can instrument code using ant and I can run almost all tests. To do that I executed the following:

$mvn clean -P build-individual-bundles package

$ant instrument

copy classes from org.eclipse.jdt.core/target/classes to org.eclipse.jdt.core.tests.compiler/target/classes (if I don't copy those classes I can't run any test because of dependency issues)

copy cobertura dependent classes (from .jar files of cobertura lib folder) to org.eclipse.jdt.core/target/classes (even adding cobertura as a maven dependency in pom.xml cobertura classes aren't found when running tests)

$mvn -fae verify -V -B -Dmaven.test.skip=false -f pom.xml -P build-individual-bundles

$ant report

   But I get 1138 (of 36621) tests with NoClassDefFoundError exceptions while executing org.eclipse.jdt.core.tests.compiler. E.g.:

testBug452788a - 1.8(org.eclipse.jdt.core.tests.compiler.regression.GenericsRegressionTest_1_8)  Time elapsed: 0.001 sec  <<< ERROR!

java.lang.NoClassDefFoundError: Could not initialize class org.eclipse.jdt.internal.compiler.lookup.ConstraintExpressionFormula

at org.eclipse.jdt.internal.compiler.lookup.InferenceContext18.inferInvocationApplicability(InferenceContext18.java:371)

at org.eclipse.jdt.internal.compiler.lookup.ParameterizedGenericMethodBinding.computeCompatibleMethod18(ParameterizedGenericMethodBinding.java:237)

at org.eclipse.jdt.internal.compiler.lookup.ParameterizedGenericMethodBinding.computeCompatibleMethod(ParameterizedGenericMethodBinding.java:84)

at org.eclipse.jdt.internal.compiler.lookup.Scope.computeCompatibleMethod(Scope.java:743)

at org.eclipse.jdt.internal.compiler.lookup.Scope.computeCompatibleMethod(Scope.java:700)

at org.eclipse.jdt.internal.compiler.lookup.Scope.findMethod0(Scope.java:1656)

at org.eclipse.jdt.internal.compiler.lookup.Scope.findMethod(Scope.java:1557)

at org.eclipse.jdt.internal.compiler.lookup.Scope.getImplicitMethod(Scope.java:2518)

at org.eclipse.jdt.internal.compiler.ast.MessageSend.findMethodBinding(MessageSend.java:914)

at org.eclipse.jdt.internal.compiler.ast.MessageSend.resolveType(MessageSend.java:729)

at org.eclipse.jdt.internal.compiler.ast.ReturnStatement.resolve(ReturnStatement.java:342)

at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.resolveStatements(AbstractMethodDeclaration.java:634)

at org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.resolveStatements(MethodDeclaration.java:306)

at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.resolve(AbstractMethodDeclaration.java:544)

at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1195)

at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1308)

at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.resolve(CompilationUnitDeclaration.java:593)

at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:867)

at org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest$4.process(AbstractRegressionTest.java:2599)

at org.eclipse.jdt.internal.compiler.Compiler.processCompiledUnits(Compiler.java:550)

at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:462)

at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:417)

at org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.runTest(AbstractRegressionTest.java:2611)

at org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.runTest(AbstractRegressionTest.java:2419)

at org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.runConformTest(AbstractRegressionTest.java:1237)

at org.eclipse.jdt.core.tests.compiler.regression.GenericsRegressionTest_1_8.testBug452788a(GenericsRegressionTest_1_8.java:4922)

   It's very strange because other tests that also depend on org.eclipse.jdt.core are executed normally. Is there something I'm doing wrong or missing?

   

build.xml

<project>

    <target name="instrument">

        <!-- Add all modules that should be included below -->

        <!-- <antcall target="instrumentAModule">

            <param name="module" value="MODULE_NAME_TO_REPLACE"/>

        </antcall> -->

    <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.annotation"/>

        </antcall>

    <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.annotation_v1"/>

        </antcall>

    <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.apt.core"/>

        </antcall>

    <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.apt.pluggable.core"/>

        </antcall>

    <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.apt.ui"/>

        </antcall>

    <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.compiler.apt"/>

        </antcall>

    <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.compiler.tool"/>

        </antcall>

        <antcall target="instrumentAModule">

            <param name="module" value="org.eclipse.jdt.core"/>

        </antcall>

    </target>

    <target name="report" depends="merge">

        <property name="src.dir" value="src/"/>

        <cobertura-report datafile="sum.ser"

                          format="html"

                          destdir="./target/report">

            <!-- Add all modules that should be included below -->

            <!-- fileset dir="./MODULE_NAME_TO_REPLACE/${src.dir}"/ -->

            <fileset dir="./org.eclipse.jdt.core/${src.dir}"/>

   <fileset dir="./org.eclipse.jdt.compiler.tool/${src.dir}"/>

            <fileset dir="./org.eclipse.jdt.annotation/${src.dir}"/>

            <fileset dir="./org.eclipse.jdt.annotation_v1/${src.dir}"/>

            <fileset dir="./org.eclipse.jdt.apt.core/${src.dir}"/>

            <fileset dir="./org.eclipse.jdt.apt.pluggable.core/${src.dir}"/>

            <fileset dir="./org.eclipse.jdt.apt.ui/${src.dir}"/>

            <fileset dir="./org.eclipse.jdt.compiler.apt/${src.dir}"/>

        </cobertura-report>

    </target>

    <target name="merge">

        <cobertura-merge datafile="sum.ser">

            <fileset dir=".">

                <include name="**/cobertura.ser"/>

            </fileset>

        </cobertura-merge>

    </target>

    <target name="instrumentAModule">

    <echo message="Instrumenting ${module}..."/>

        <property name="classes.dir" value="target/classes"/>

        <cobertura-instrument todir="./${module}/${classes.dir}">

            <fileset dir="./${module}/target/classes">

                <include name="**/*.class"/>

            </fileset>

        </cobertura-instrument>

    </target>

    <property environment="env"/>

    <property name="COBERTURA_HOME" value="~/cobertura-2.1.1"/>

    <property name="cobertura.dir" value="${COBERTURA_HOME}"/>

    <path id="cobertura.classpath">

        <fileset dir="${cobertura.dir}">

            <include name="cobertura-2.1.1.jar"/>

            <include name="lib/**/*.jar"/>

        </fileset>

    </path>

    <taskdef classpathref="cobertura.classpath" resource="tasks.properties"/>

</project>

Thanks in advance.

--

Felipe Pontes

PhD Candidate in Computer Science

Federal University of Campina Grande