Python Code Coverage Module -- Per Erik Strandberg

I just had a look at a Python Code Coverage Module available. It is the first real code coverage analyzer I take a closer look at. It is written by Ned Batchelder [1]. You can see Ned speak about coverage testing at PyCon 2009 here: [2]

In itself coverage measurement can be used for many things - but to me it is a nice tool for validating that tests written really check the code. As Ned points out in his speech:

the tests test the product
coverage testing tests the tests

You might also want to check out trace that is in the python standard library (see [3] for Doug Hellmans take on it) or figleaf (see [4]).

Installation

It is available using pythons easy install:

$ coverage
coverage: command not found

$ easy_install --version
The program 'easy_install' is currently not installed.  You can install it by typing:
sudo apt-get install python-setuptools

$ sudo apt-get install python-setuptools
[sudo] password for perstr: 
[...]
Setting up python-setuptools (0.6.24-1ubuntu1) ...

$ easy_install --version
distribute 0.6.24dev-r0

$ sudo easy_install coverage
Searching for coverage
Reading http://pypi.python.org/simple/coverage/
[...]

$ coverage --version
Coverage.py, version 3.5.2.  http://nedbatchelder.com/code/coverage

First Look

I made two basic python files (see below) and to test them I call the coverage method twice. First with the run argument and my filename and the second time with the report argument.

$ coverage run my_math.py
37
2401

$ coverage report
Name      Stmts   Miss  Cover
-----------------------------
my_math      10      1    90%
util          5      1    80%
-----------------------------
TOTAL        15      2    87%

This is just the kind of Keep It Simple I love! What else can it do?

More features

Before we look at more features - let's have a look at my silly little example.

A quick look at my sample code

Ok, two files, util.py and my_math.py. First util.py that has the methods borkify and fnord:

from math import ceil, pi

def borkify(i):
    return int(ceil(i/pi) + 1)

def fnord(j):
    return j/2 - 3

Now my_math.py with the methods func1, func2 and func3:

from util import borkify

def func1(x):
    return x+3

def func2(y):
    return y**4

def func3(x, y):
    return x+y+3

if __name__ == '__main__':
    print func3(2, 32)
    print func2(borkify(18))

As you can see I added an if-branch for the case where I run the my_math as a stand-alone script. This branch executes methods func2, func3 and borkify. It does not execute func1 or fnord.

XML output

After running your scripts with the run argument you can pass the xml argument to get xml output - I'll show a sample here. The interesting line is marked with 0 hits! This is thus the line we never tested.

    <...>
    <class ... filename="util.py" ...>
        <methods/>
        <lines>
            <line hits="1" number="1"/>
            <line hits="1" number="3"/>
            <line hits="1" number="4"/>
            <line hits="1" number="6"/>
            <line hits="0" number="7"/>
        </lines>
    </class>
    <...>

See the full example here: [5]

HTML output

If you run coverage with the html flag (after the first run with the run flag) it generates a number of html and java script files. The summary, the index.html file, contains a summary and links to individual modules. Also notice the fancy clickable headers. See the result here: [6], it should look something like this:

Coverage report: 87%

Module   statements  missing  excluded  coverage
my_math    10          1         0       90%
util        5          1         0       80%
Total      15          2         0       87%

The module view will be color-coded depending on how you click it with red or green. As you can see the header of fnord is evaluated - but the method is never. See [7] and this screenshot:

Annotate

Using annotate will make copies of the source files and mark executed lines with a > and non-executed lines with a ! in a copy called utils.py,coverage:

> from math import ceil, pi
  
> def borkify(i):
>     return int(ceil(i/pi) + 1)
  
> def fnord(j):
!     return j/2 - 3

And what else?

As of version 3.5.2 this is a complete list of commands:

annotate: Annotate source files with execution information.
combine: Combine a number of data files.
erase: Erase previously collected coverage data.
help: Get help on using coverage.py.
html: Create an HTML report.
report: Report coverage stats on modules.
run: Run a Python program and measure code execution.
xml: Create an XML report of coverage results.

Branch measurement

By adding the --branch flag to the run command you also add measurements of code branches.

Excluding code

By adding the comment pragma: no cover the particular branch of code will not be measured.
You can also configure patterns to be excluded from the measurements by making a configuration file - see the official documentation for details: [8]

if 0: # pragma: no cover
    print "debug"

This page belongs to Kategori Programmering