Wednesday, August 15, 2012

Code coverage analysis in multi-module Maven projects

When doing refactorings and other maintenance work on mature projects, code coverage analysis is an invaluable tool to help ensuring that the changes have no unexpected side effects. Usually I do that using Cobertura because it's very easy to use in Maven projects (just type mvn cobertura:cobertura and open the report generated under target/site/cobertura). One of the shortcomings of Cobertura is its lack of support for multi-module Maven builds. That is really annoying because in many large projects, a significant amount of code coverage for a given Maven module is actually generated by tests run in other modules. E.g. there are many projects that have a dedicated Maven module for integration tests.

Recently I learnt about JaCoCo which is another code coverage analysis tool. Unfortunately it has the same shortcoming as Cobertura: out of the box, it doesn't support multi-module Maven builds. There is a bug report that contains a patch that solves this issue (at least to some degree; see below). The patch has not been applied yet; in order to use it you will have to build a patched version of JaCoCo manually. The patch can be applied cleanly to revision 1674 of JaCoCo trunk.

Here is the complete set of instructions to build a patched version of JaCoCo and deploy it to the local Maven repository:

svn co -r 1674 https://eclemma.svn.sourceforge.net/svnroot/eclemma/jacoco/trunk/jacoco
cd jacoco
curl http://sourceforge.net/apps/trac/eclemma/raw-attachment/ticket/186/jacoco-maven-aggregate.diff | patch -p0
cd org.jacoco.build
mvn clean install

After that you can use JaCoCo on a multi-module Maven project with the following commands:

mvn org.jacoco:jacoco-maven-plugin:0.5.8-SNAPSHOT:prepare-agent -Daggregate=true clean install
mvn org.jacoco:jacoco-maven-plugin:0.5.8-SNAPSHOT:report -Daggregate=true

There is however one important limitation: because of an issue in code added by the patch, it only works on multi-module projects where the root POM is also the parent POM. If this is not the case, then the plugin will fail with a NullPointerException.

5 comments:

  1. Hello,
    We have deeply nested maven multi-module project where root pom and parent poms are different. We are using sonar 3.2.1 for static analysis as well as code coverage tool. Jacoco is the default code coverage tool used in sonar and we are using that for generating code coverage report. We are using sonar as a post build action jenkins.

    The problem we are facing is that we cannot compute overall code coverage by unit tests across modules. When the unit test boundary is within a particular module it's showing correct code coverage. But while we are running unit test where the control flows from parent module to another module which is packed in a separate jar then it does not show any code coverage.

    Is it due to the issues that you have mentioned above?

    ReplyDelete
    Replies
    1. I'm not really familiar with the Sonar-JaCoCo integration and I don't know if there is a way to solve your problem with Sonar.

      What you can always do is to integrate JaCoCo into your Maven build process. With the help of maven-antrun-plugin you can then use the JaCoCo ant tasks to generate the coverage report. This gives you the required flexibility to get accurate results that include cross-module coverage.

      I recently did that for Apache Axiom. You can find the project's source code here:

      https://svn.apache.org/repos/asf/webservices/axiom/trunk/

      Delete
  2. Hi Andreas,

    I've setup a maven project to with ant tasks to generate the coverage report.
    And, sine i use on-the-fly instrumentation, i also made sure to instrument the jars before passing them to the ant tasks.

    However, the generated html report shows zero coverage for all classes.

    Any hints /suggestions?

    ReplyDelete
    Replies
    1. You should ask this kind of questions on a JaCoCo mailing list or forum, or on stackoverflow.com.

      Delete