Code Coverage Test with EMMA Utility

Unit test is an essential stage in a software development lifecycle. After the code of features is completed, developer is required to design a number of test cases to guarantee the software performs desired functions.

There is no simple objective criteria how many test cases enough to assure the quality of a software. Being lack of deep knowledge of implementation, project managers reluctantly accept the test plan from developers. The question can not be answered by developer either. Since it is infeasible to exhaust every possible test scenario, the developer is eager to know the minimum number of cases that can touch the most part of the code.

The code coverage test is probably a feasible objective measure. A little open-source utility, EMMA, can generate a precise picture how much percentage the code is covered after the execution of a test suite. The data is broken down to packages, classes, and even methods. It helps the developer design new cases to increase the percentage rather than to repeat similar cases.

EMMA provides a lot of features, and I am very impressed with two of them: It reports rich coverage analysis without introducing significant overhead during either build or execution time. It is simple, lightweight, and easy to probe classes running in a J2EE container. Here is an example where the test suite is designed for foo.jar used by a JBoss application server.

First copy emma.jar into the lib directory of the application server.

Create an instrumented version of foo.jar. Put emma.jar into environment variable CLASSPATH. Create a directory output for the instrumented classes. Execute command java emma instr -d output -ip foo.jar. There is a file coverage.em created under the current working directory. Then package the instrumented classes under directory output into a new version of foo.jar, and replace the JAR file in the application server.

Start the application server. Execute the test suite as usual. Then stop the application server. There will be a raw result file coverage.ec generated under the directory where the application server is launched, in my case, C:\jboss-4.0.4.GA\bin\coverage.ec.

Generated the plain text and HTML reports with command java emma report -r txt,html -in coverage.em -in C:\jboss-4.0.4.GA\bin\coverage.ec. The plain text report is generated in the current working directory coverage.txt. The report in HTML format is generated as coverage\index.html.