Code coverage of OSGi applications

The question that disturbs me last night is about test quantity/quality. So, when I can say: « Ok, I’ve enough tests ». So, for sure, this question is open.

Freddy Mallet posted a comment on my last post about Sonar. Sonar (http://sonar.codehaus.org/) is an all-in-one code quality tracker. So, based on maven (and a set of plug-ins), it collects data about your code, tests … and assembles reports on one web site with useful metrics (efficiency, usability…).

I tested today and was really impressed. But, my code coverage metrics was desperately 0 ☹
After some web searches, I found the Cobertura maven plug-in (http://mojo.codehaus.org/cobertura-maven-plugin/) collecting code coverage during test execution thanks to the Cobertura framework (http://cobertura.sourceforge.net/). However, as my systems and applications are tested in an OSGi container with junit4osgi, I investigated how to collect the code coverage of tests executed with juni4osgi.

So, first it is possible after a very small hack of junit4osgi to manage cobertura packages.
Collecting the code coverage is a three-step process:

A] Creates a bundle with instrumented classes
First you need to select ONE project under test. You will collect metrics on this project. So you want several projects, you need to execute the process several times (one per project).
Once selected, edit the pom file to ignore the ‘net.sourceforge.cobertura.* packages such as in:

  org.osgi.framework;version=1.3,
  org.osgi.service.cm,
  org.osgi.service.log,
  !net.sourceforge.cobertura.*

Then, execute the following command to create the instrumented bundle:

mvn clean install cobertura:instrument package install

This command creates the bundle with the Cobertura instrumented code, and installs it in the local maven repository. Moreover, it creates a file containing code metadata in the target/cobertura folder (cobertura.ser).

B] Test execution
Once the bundle is created, you need to execute the test. The code coverage is computed during this execution. So to launch the execution, go in your integration-test project and execute the following command:

mvn clean integration-test
  -Dnet.sourceforge.cobertura.datafile
      =/path/to/your/tested/project/target/cobertura/cobertura.ser

The ‘net.sourceforge.cobertura.datafile’ property specified the location of the cobertura.ser file (created during the Cobertura code instrumentation).

Tests are executed normally… But when maven is stopping, two messages appear:

Cobertura: Loaded information on 81 classes.
Cobertura: Saved information on 81 classes.

This means that the metrics were correctly collected.

C] Generation of the report and analysis of the code coverage
Once collected, go back to your tested project. To generate the report, just launch the following command:

mvn cobertura:cobertura

The report is generated in the target/site/cobertura/ folder. Here is an example of generated report. You can also check your code coverage with:

mvn cobertura:check

This doesn’t create the report but just gives an idea of the code coverage. You can use the cobertura plug-in configuration to adapt the checking or the report to your requirements.

That’s it.

Reducing the pain of a release process

Who never crack down during a release process? Despite several tools ease the process; it’s generally a nightmare.

Why? Because of the number of task to execute during the process: the product has to be deeply tested before the release, then license/notice files have to be checked as well as license header, the release candidates artifacts have to be created, deployed, signed, don’t forget the release note, and to update the documentation…

So, I recently discover two maven plug-ins reducing (a little) the pain. They automate two checks: license headers and the code style consistency.

First the checkstyle plug-in (http://maven.apache.org/plugins/maven-checkstyle-plugin/) checks that your code is consistently formatted. It is, in my opinion, a critical aspect. So, diving into an unknown code consistently formatted is really easier. Using the plug-in is quite simple. Add the following plug-in configuration in the build section of your pom file:

<plugin>
<groupid>org.apache.maven.plugins</groupid>
    <artifactid>maven-checkstyle-plugin</artifactid>
    <configuration>           
        <enablerulessummary>false</enablerulessummary>
        <violationseverity>warning</violationseverity>
        <configlocation>http://felix.apache.org/ipojo/dev/checkstyle_ipojo.xml</configlocation>
    </configuration>
</plugin>

The ‘violationSeverity’ attribute sets the severity level from which the project build failed. So, if the code format breaks a rule with a level superior or equals to the set value, the build process failed. (Check levels can be set in the checkstyle file). The ‘configLocation’ attribute specifies the checkstyle file to use (containing the format rules).

So, once your project has the correct configuration, just launch the following command:

mvn checkstyle:check

to check if your project doesn’t break your format. If executed on a multi-module project, each module is checked (each module need to be configured).

So, thanks to this plug-in, checking the code format consistency is quite simple and can be automate during your project build process.

The second plug-in is the RAT plug-in (http://mojo.codehaus.org/rat-maven-plugin/). RAT (http://incubator.apache.org/rat/), is a tool to improve accuracy and efficiency when checking releases. It is heuristic in nature: making guesses about possible problems. It will produce false positives and cannot find every possible issue with a release. It reports require interpretation, but can also be automated…

In fact, RAT checks license header and tacks missing license. RAT provides a maven plug-in which can automate the missing license discovery during your build project. To use this plug-in, just add the following plug-in configuration in the build section of your pom file.

<plugin>
<groupid>org.codehaus.mojo</groupid>
<artifactid>rat-maven-plugin</artifactid>
<configuration>                
    <excludesubprojects>false</excludesubprojects>         
    <useeclipsedefaultexcludes>true</useeclipsedefaultexcludes>    
    <usemavendefaultexcludes>true</usemavendefaultexcludes>
    <excludes>
        <param>doc/**/*</param>
        <param>maven-eclipse.xml</param>
        <param>.checkstyle</param>
        <param>.externalToolBuilders/*</param>
        <param>LICENSE.*</param> 
        <param>.fbprefs</param>
    </excludes>
</configuration>
</plugin>

This configuration will check that non-excluded files have a correct license header. It excludes maven files (target folder), eclipse files (.project, .classpath). Those files don’t require a license, as they should not be on the source code repository. Then, others files can be excluded such as the doc folder (containing html files and release notes, used license files (LICENSE. *)... Extend the list (or modify it) according to your project.

To check that you have no file without a license header, just execute the following command:

mvn rat:check

The command failed if a corrupted file is detected. Executed on a multi-module project (each module need to be configured), it checks every module and failed as soon as a module has an invalid file.

So, by using these plug-ins, you can continuously check the consistency of your code format and of the license headers. This is not wonderful, but at least do one’s share of the work.