solidity-coverage

Code coverage for Solidity testing

• For more details about what this is, how it works and potential limitations, see the accompanying article.
• solidity-coverage is in development and its accuracy is unknown. If you find discrepancies between the coverage report and your suite's behavior, please open an issue.
• solidity-coverage is Solcover

Install

$ npm install --save-dev solidity-coverage

Run

Set a coverage network in truffle-config.js (see Network Configuration) and then run:

$ npx solidity-coverage

Usage notes:

• For pragma solidity >=0.4.22 <0.6.0 / Petersburg / Truffle v4 and v5
• Tests run more slowly while coverage is being generated.
• Your contracts will be double-compiled and a (long) delay between compilation and the beginning of test execution is possible if your contracts are large.
• Truffle should be globallly installed in your environment.. If you prefer running truffle as a local dependency, please see this section of the FAQ.
• If your suite uses native Solidity testing or accesses contracts via mocks stored in tests/ (a la Zeppelin), coverage will trigger test errors because it can't control the way truffle compiles that folder. Mocks should be relocated to the root contracts directory. More on why this is necessary at issue 146

Network Configuration

By default, this tool connects to a coverage-enabled fork of ganache-cli called testrpc-sc on port 8555.

• it's a dependency - there's nothing extra to download.
• the solidity-coverage command launches it automatically in the background. (See this section of the FAQ if you need to launch it separately yourself)

In truffle-config.js, add a coverage network following the example below.

Example

module.exports = {
  networks: {
    development: {
      host: "localhost",
      port: 8545,
      network_id: "*"
    },
    coverage: {
      host: "localhost",
      network_id: "*",
      port: 8555,         // <-- If you change this, also set the port option in .solcover.js.
      gas: 0xfffffffffff, // <-- Use this high gas value
      gasPrice: 0x01      // <-- Use this low gas price
    },
    ...etc...
  }
};

Options

Additional options can be specified in a .solcover.js config file located in the root directory of your project.

Example:

module.exports = {
    port: 6545,
    testrpcOptions: '-p 6545 -u 0x54fd80d6ae7584d8e9a19fe1df43f04e5282cc43',
    testCommand: 'mocha --timeout 5000',
    norpc: true,
    dir: './secretDirectory',
    copyPackages: ['openzeppelin-solidity'],
    skipFiles: ['Routers/EtherRouter.sol']
};

Option	Type	Default	Description
accounts	Number	35	Number of accounts to launch testrpc with.
port	Number	8555	Port to run testrpc on / have truffle connect to
norpc	Boolean	false	Prevent solidity-coverage from launching its own testrpc. Useful if you are managing a complex test suite with a shell script
testCommand	String	truffle test	Run an arbitrary test command. ex: mocha --timeout 5000. NB: Also set the port option to whatever your tests require (probably 8545).
testrpcOptions	String	--port 8555	options to append to a command line invocation of testrpc. NB: Using this overwrites the defaults so always specify a port in this string and in the port option
copyNodeModules	Boolean	false	⚠️ DEPRECATED use copyPackages instead ⚠️ Copies node_modules into the coverage environment. May significantly increase the time for coverage to complete if enabled. Useful if your contracts import solidity files from an npm installed package (and your node_modules is small).
copyPackages	Array	[]	Copies specific node_modules packages into the coverage environment. May significantly reduce the time for coverage to complete compared to copyNodeModules. Useful if your contracts import solidity files from an npm installed package.
skipFiles	Array	['Migrations.sol']	Array of contracts or folders (with paths expressed relative to the contracts directory) that should be skipped when doing instrumentation. Migrations.sol is skipped by default, and does not need to be added to this configuration option if it is used.
deepSkip	boolean	false	Use this if instrumentation hangs on large, skipped files (like Oraclize). It's faster.
dir	String	.	Solidity-coverage copies all the assets in your root directory (except node_modules) to a special folder where it instruments the contracts and executes the tests. dir allows you to define a relative path from the root directory to those assets. Useful if your contracts & tests are within their own folder as part of a larger project.
buildDirPath	String	/build/contracts	Build directory path for compiled smart contracts
istanbulReporter	Array	['html', 'lcov', 'text']	Coverage reporters for Istanbul. Optional reporter replaces the default reporters.

FAQ

Solutions to common problems people run into:

• Running out of gas
• Running out of memory (locally and in CI)
• Running out of time (in mocha)
• Running on windows
• Running testrpc-sc on its own
• Running truffle as a local dependency
• Integrating into CI
• Why are asserts and requires highlighted as branch points?

Example reports

• metacoin (Istanbul HTML)
• openzeppelin-solidity (Coveralls)

Contribution Guidelines

Contributions are welcome! If you're opening a PR that adds features please consider writing some unit tests for them. Bugs can be reported in the issues.

Set up the development environment with:

$ git clone https://github.com/sc-forks/solidity-coverage.git
$ yarn

Contributors

• @area
• @cgewecke
• @adriamb
• @cag
• @maurelian
• @rudolfix
• @phiferd
• @e11io
• @elenadimitrova
• @ukstv
• @vdrg
• @andresliva
• @DimitarSD
• @sohkai
• @bingen
• @pinkiebell
• @obernardovieira
• @angus-hamill
• @kandrianov