The world had witnessed some catastrophic events merely because of the errors in the software. One such incident happened in 2008 where glitches in test coverage practices of baggage handling system software caused cancellation of 500+ flights.
Test Coverage is an important indicator of software quality and an essential part of software maintenance. It helps in evaluating the effectiveness of testing by providing data on different coverage items. This insight allows to improve testing by defining new test cases for the untested code, thereafter improving your code quality, which ultimately increases code coverage and reducing costs.
Why is Code Coverage required? – Purpose & Benefits
- Defect prevention at early stages of project life cycle
- It creates additional test cases to increase coverage
- Better ROI can be achieved by reduction in UAT defects and production defects
- It helps in finding areas of a program not exercised by a set of test cases
- Testing life will become smooth by managing the Risk based testing approach
- Helps in determining a quantitative measure of code coverage, which indirectly measures the quality of the application or product.
How to Choose a Code Coverage Tool?
This is the real challenge to choose which code coverage tools to use for application testing. An efficient tool can help measure the effectiveness of the testing efforts There are several options to create code coverage reports depending on the language(s) used for coding. Some of the popular tools with respect to coding languages are listed below:
| Java | Atlassian Clover, Cobertura, JaCoCo, NCover |
| Javascript | istanbul, Blanket.js |
| PHP | PHPUnit |
| Python | Coverage.py |
| Ruby | SimpleCov |
| C | GCov-LCov, Testwell CTC++ |
Some tools like istanbul(a JavaScript test coverage tool) will output the results straight into your terminal while others can generate a full HTML report that lets you explore which part of the code are lacking coverage.
Also, there are other coverage testing tools available for all the major languages and some of them are relevant not only for unit testing but also for the overall tests performed (Unit, integration, system, manual and more). Below is the list of some trending code coverage tools that would help you pick one while running testing procedure.
1. Default Visual Studio Code Coverage
Visual Studio Enterprise has Code Coverage collection as an inbuilt feature. It can be applied to both managed (CLI) and unmanaged (native) code. Code coverage is provided as an option while trying run test methods using Test Explorer. You can select all of your tests or subsets of them. It helps you visualize the coverage while you look at the code. The below table of results shows the percentage of the code that was run in each assembly, class, and method. In addition, the source editor shows which code has been tested.
Report can be obtained in blocks or in lines. It is also easy to manage the code coverage results as well as merge results from different runs.
2. Cobertura
Cobertura is one of the most used and widely known code coverage tools. It is popular for being a free Java tool that calculates the percentage of code accessed by various tests performed. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage.
Execution can be done via Ant, Command Line, Eclipse plugin, Maven plugin. It is easy to use and can measure coverage without having the source code. It provides reports in HTML or XML format and it has the capacity to test lines and branches of class and method.
3. Coverage.py
Coverage.py is a code coverage tool for Python. Main phases of using Coverage.py includes execution, analysis and reporting. It monitors the Python programs, notes which parts of the code have been executed, and then analyzes the source to identify code that could have been executed but was not.
Coverage.py is available with Tidelift to deliver one enterprise subscription. Other ways include execution via command line, APIs as well as Plug-ins for Django and Mako.
4. Gcov-Lcov
Gcov is a test coverage program used with GCC. It analyzes the programs to help create a more efficient, faster running code, to discover untested parts of the code.
It acts as a profiling tool, which helps in analyzing some basic performance statistics, such as:
- number of times each line of code executes
- what lines of code are actually executed
- how much computing time each section of code uses
Lcov is a graphical front-end for Gcov.It collects gcov data for multiple source files and creates HTML pages, which contains the source code annotated with coverage information. It also adds overview pages for easy navigation within the file structure. Lcov supports statement, function and branch coverage measurement.
5. Testwell CTC ++
Testwell CTC++ is a powerful instrumentation-based code coverage and dynamic analysis tool for C and C++ code. It can be used on C#, Java and Objective-C code with particular add-on components. It can also be used to analyse code basically at any embedded target machines.
CTC++ provides Line Coverage, Statement Coverage, Function Coverage, Decision Coverage, Multicondition Coverage, Modified Condition/Decision Coverage(MC/DC), Condition Coverage.
As a dynamic analysis tool, CTC++ shows the execution counters (how many times executed) in the code. It can also be used to measure function execution costs (normally time) and to enable function entry/exit tracing at test time.
This tool is user-friendly and fast. Report generation can be in straight text, HTML, XML, JSON (JavaScript Object Notation) and Excel input form. On some environments, e.g. Microsoft Visual Studio, CTC++ use is possible directly from the compiler IDE.
6. NCover
NCover is a code coverage tool for .NET programs and applications. NCover has been designed to integrate with the build process and has extensive command line support. The code coverage process is simple and can be automated. The NCover application suite is made up of three tools NCover.Explorer.exe, NCover.Console.exe and NCover.Reporting.exe.
NCover.Explorer.exe is a GUI program which allows to generate and view detailed code coverage information.
It supports statement coverage, branch coverage and cyclomatic complexity. It gives provision to select Low, Medium, High and Complete levels of code coverage, separately. It is very easy to use and fast, also available on open source and as well as on commercial license.
Conclusion:
For better product quality, depending on the specific needs of the project and the level of code coverage required, it is easy to select code coverage tool. All the available tools for the code coverage suffice the required capabilities of code coverage based on independent need of code language and structure.
Know more about our Quality Engineering Services.
See how we have helped our esteemed clients to ensure quality of their product – Read Case Studies.
About the Author: Hetal Chauhan
Hetal is associated with VOLANSYS Technologies as a Principal Engineer in QA division. She has around 7+ years of experience in performing Verification and Validation in a wide range of domains such as Embedded, IoT, Software and Aerospace.
