Code coverage of Manual Testing using Istanbul

Istanbul provides code coverage metrics for automated unit tests. It provides those metrics by instrumenting the JavaScript code. Official documentation on the website provides how to integrate Istanbul with various unit testing frameworks.

But, let’s say we do not have unit tests. And we do manual testing. How much of code coverage does our testing do? For this problem, there is very little official support. So, I thought I will come with a blog post on how to do this manually.

Sample App

Our Sample app is a Counter App. When the user clicks on a button, the counter increments. When we load the page and measure the code coverage, the code coverage should be less than 100%. On clicking the button and again measuring code coverage, the code coverage should be 100%. Let’s see if that is the case.

The sample app is available in a git repo with v1 tag.

Run the app using yarn start command. We should see a screen like the one below.

Istanbul integration

We instrument only our code (not node_modules) using Istanbul. Fortunately, there is a babel plugin for doing the integration. Also, there is a tutorial on how to do it from scratch. But this blog post assumes that we are integrating to an already existing app.

Install babel-plugin-istanbul:

yarn add babel-plugin-istanbul --dev

In .babelrc, add the plugin.

{
  "presets": ["env", "react"],
  "plugins": ["istanbul"]
}

Our bundle should now contain JavaScript code with instrumentation.

Getting coverage results

Run the app using yarn start command. Open the browser console. Type JSON.stringify(__coverage__). Copy the JSON and paste it on a file – .nyc_output/out.json.

Install the command line utility to report code coverage.

yarn add nyc --dev

Add a script in package.json named report.

"scripts": {
  "start": "webpack-dev-server",
  "build": "webpack",
  "report": "nyc report"
},

Run yarn report command.

Code coverage results show opening the app in the browser covers only 50% of the function and 75% of lines. It even reports the uncovered line #8 in index.js which corresponds to the click event.

Verifying the integration

The last step is verifying the integration. This is easily done by clicking on the button and measuring code coverage again.

It seems to work alright. Clicking the button gives 100% code coverage.

React Component to Display Coverage

To facilitate viewing of coverage data on the browser, there is a React component: istanbul-coverage-display. This component adds a button to the bottom left of the screen. On clicking the button, there is a popover displaying coverage information.

Clicking the Close button closes the popover. Import this component from the root component.

import { CoverageSummary } from 'istanbul-coverage-display';

Add it to anywhere in the component tree, but only for the test build or instrumentation build.

return (
    <div>
        {count}
        <button onClick={handleAdd}>+1</button>
        <CoverageSummary />
    </div>
)

By default, the button appears on the bottom left corner of the screen. Change this by specifying the position prop.

<Coverage position="topLeft" />

There is also a CoverageDetail component in the same package which displays all data in a tree like format.

There is a sample app hosted in Github pages.

But if you don’t like UI, there is a getCoverage function in the component. This function provides an array of objects corresponding to each source file. The data returned from getCoverage is of the form:

[{
  key: path,
  data: {
    branches: {
      total,
      covered,
      pct
    },
    functions: {
      total,
      covered,
      pct
    },
    lines: {
      total,
      covered,
      pct
    },
    statements: {
      total,
      covered,
      pct
    }
  }
}] 

To get this data, it makes use of istanbul-lib-coverage, official library from the istanbul team. Here is the source code of getCoverage function, if that is what you are after.

import libCoverage from 'istanbul-lib-coverage';

export default function getCoverage() {
    if (window.__coverage__ && libCoverage) {
        var map = libCoverage.createCoverageMap({});
        map.merge(window.__coverage__);
        return Object.keys(map.data).map(key => ({
                key,
                data: map.data[key].toSummary().data
            }));
    }
    return null;
}

It creates a coverage map from the istanbul-lib-coverage package. And uses toSummary function to get the coverage data for each file.

Summary

To find the code coverage for our manual testing effort, we do the following steps for our React app:

• Install babel-plugin-istanbul
• Add the plugin to .babelrc or webpack config.
• Use CoverageSummary or CoverageDetail from istanbul-coverage-display to view the coverage information within the app.