Statement, Branch, and Path Coverage Testing in Javaby Joe Ponczak
Code coverage is a way to measure the level of testing you've performed on your software. Gathering coverage metrics is a straightforward process: Instrument your code and run your tests against the instrumented version. This produces data showing what code you did—or, more importantly, did not—execute. Coverage is the perfect complement to unit testing: unit tests tell you whether your code performed as expected, and code coverage tells you what remains to be tested.
Most developers understand this process and agree on its value proposition, and often target 100 percent coverage. Although 100 percent coverage is an admirable goal, 100 percent of the wrong type of coverage can still leave problems undiscovered. A typical software development effort measures coverage in terms of the number of either statements or branches to be tested. Even with 100 percent statement or branch coverage, critical bugs still may be present in the logic of your code, leaving both developers and managers with a false sense of security.
How can 100 percent coverage be insufficient? Because statement and branch coverage do not tell you whether the logic in your code was executed. Statement and branch coverage are great for uncovering glaring problems found in unexecuted blocks of code, but they often miss bugs related to both decision structures and decision interactions. Path coverage, on the other hand, is a more robust and comprehensive technique that helps reveal defects early.
Before you learn about path coverage, look at some of the problems with statement and branch coverage.
Statement coverage identifies which statements in a method or class have been executed. It is a simple metric to calculate, and a number of open source products exist that measure this level of coverage. Ultimately, the benefit of statement coverage is its ability to identify which blocks of code have not been executed. The problem with statement coverage, however, is that it does not identify bugs that arise from the control flow constructs in your source code, such as compound conditions or consecutive switch labels. This means that you easily can get 100 percent coverage and still have glaring, uncaught bugs.
The following example demonstrates this. Here, the
returnInput() method is made up of seven statements and has a simple requirement: its output should equal its input.
Figure 1. Code sample
Next, you can create one JUnit test case that satisfies the requirement and gets 100 percent statement coverage.
Figure 2. Statement coverage
There's an obvious bug in
returnInput(). If the first or second decision evaluates true and the other evaluates false, the return value will not equal the method's input. An astute software developer will notice this right away, but the statement coverage report shows 100 percent coverage. If a manager sees 100 percent coverage, he or she may get a false sense of security, decide that testing is complete, and release the buggy code into production.
Recognizing that statement coverage may not fit the bill, the developer decides to move on to a better testing technique: branch coverage.
A branch is the outcome of a decision, so branch coverage simply measures which decision outcomes have been tested. This sounds great because it takes a more in-depth view of the source code than simple statement coverage, but branch coverage can also leave you wanting more.
Determining the number of branches in a method is easy. Boolean decisions obviously have two outcomes, true and false, whereas switches have one outcome for each case—and don't forget the default case! The total number of decision outcomes in a method is therefore equal to the number of branches that need to be covered plus the entry branch in the method (after all, even methods with straight line code have one branch).
In the example above,
returnInput() has seven branches—three true, three false, and one invisible branch for the method entry. You can cover the six true and false branches with two test cases:
Figure 3. Branch coverage
Pages: 1, 2