Python Pattern Doctest
I realized today that I have a code coverage pattern that I use and wanted to describe.
It consists of three parts:
- The habit of always implementing for example a doctest or demo code in the if __name__ == "__main__" branch at the end of source code files.
- Measuring code coverage with Python Code Coverage Module
- Implementing a simple bash script (I guess this could be done in python as well - but people tend to like bash a my current assignment).
The if-name-equals-main-pattern - Part 1
It can be nice to not just treat python-files as "always run" scripts. With a if __name__ == "__main__" branch at the end we can make the script into a library that also functions as a script.
Have a look at this little library with two silly functions. But notice that it also contains some code to parse command-line argument and run:
"""
Minimal demo of the if-name-equals-main-pattern.
"""
from argparse import ArgumentParser
def my_add(a, b, c):
"""Triple addition"""
return a + b + c
def my_del(a, b, c):
"""Double deletion"""
return a - b - c
def parse_args(args=None):
"""Parse command line argumets."""
desc = "Do the magic with a, b and c"
parser = ArgumentParser(description=desc)
parser.add_argument('-a', type=int, default=42, help="Value for a.")
parser.add_argument('-b', type=int, default=1337, help="Value for b.")
parser.add_argument('-c', type=int, default=9000, help="Value for c.")
if args:
return parser.parse_args(args)
return parser.parse_args()
if __name__ == "__main__":
my_args = parse_args()
print "my_add %s" % my_add(my_args.a, my_args.b, my_args.c)
print "my_del %s" % my_del(my_args.a, my_args.b, my_args.c)
The thing here is that we can run it like any normal script:
$ python minidemo.py -a 1 -b 2 -c 4 my_add 7 my_del -5 $ python minidemo.py my_add 10379 my_del -10295
But we can also use it as a library and import it:
$ python Python 2.7.6 (default, Jun 22 2015, 17:58:13) [GCC 4.8.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from minidemo import my_del >>> my_del(1000, 100, 10) 890
The if-name-equals-main-pattern - Part 2
In short this pattern is just adding a check at the end of a code module to make it run in a stand alone mode. Typically you run doctests or implement the functional code here. A very simple example is to use something like this:
if __name__ == "__main__":
import doctest
res = doctest.testmod()
print("Tested %s cases, %s failed." % (res.attempted, res.failed))
Or as a more complete example with two functions and some doc tests:
"""
Minimal demo of the if-name-equals-main-pattern.
"""
def my_add(a, b, c):
"""
Triple addition
>>> my_add(1, 3, 5) == 9
True
"""
return a + b + c
def my_del(a, b, c):
"""
Double deletion
>>> my_del(10, 1, 2) == 7
True
"""
return a - b - c
if __name__ == "__main__":
import doctest
res = doctest.testmod()
print("Tested %s cases, %s failed." % (res.attempted, res.failed))
Running it in verbose mode $ python demo.py -v will show the expected values and received results:
Trying:
my_add(1, 3, 5) == 9
Expecting:
True
ok
Trying:
my_del(10, 1, 2) == 7
Expecting:
True
ok
1 items had no tests:
__main__
2 items passed all tests:
1 tests in __main__.my_add
1 tests in __main__.my_del
2 tests in 3 items.
2 passed and 0 failed.
Test passed.
Tested 2 cases, 0 failed.
Running it without the -v will just reveal the number of tested cases and the number of failures.
Tested 2 cases, 0 failed.
Code coverage analysis
In the simplest of worlds we can run code coverage analysis with
$ coverage erase # not needed in this example, but included for completeness $ coverage run demo.py Tested 2 cases, 0 failed. $ coverage report -m demo.py Name Stmts Miss Cover Missing ------------------------------------- demo 8 0 100%
A bash script to aggregate code coverage results
Running a module or library as a stand alone script is really good - and I like doing it with doc tests. If this is combined with a simple bash script that aggregates the code coverage results then you get something really powerful.
What I did was just a loop over the python files. But first we declare the files in an "array" (or whatever it is called in bash), a return value variable and clean up any old code coverage results.
files="file1.py file2.py ... fileN.py" ret=0 # clean up old code coverage results coverage erase
The loop iterates over the files and aggregates (or appends?) the code coverage results (with the -a flag). Also notice that the return value is taken care of. This will be used in as the exit value for the script.
for pyfile in $files
do
date +"%Y-%m-%d %H:%M:%S -- Running and checking code execution of $pyfile"
PYTHONPATH=. coverage run -a $pyfile
ret=$(($ret + $?))
echo ""
done
Finally we make a report (notice the -m for show missing lines) from the coverage analysis and exit:
echo "" echo "Code Coverage Report" coverage report -m $files echo "" echo "Sanity check will exit with status $ret" exit $ret
The report
There are three parts that are important in the output of this script.
- The final report of the code coverage will look something like the below. As one can clearly see each file has the coverage measured, and the untested lines are displayed. Having to look at this report each time I check in code really triggers me to look at the code that is not tested.
- By looking at the output from the individual files it is reasonably easy to see where there are problems.
- Finally: the return value can be used for automation - perhaps for continuous integration.
Code Coverage Report Name Stmts Miss Cover Missing ------------------------------------------ file1 37 0 100% file2 319 43 87% 59-60, 64, 176, [...] file3 120 9 93% 210-218 ... fileN 37 0 100% ------------------------------------------ TOTAL 1258 63 95% Sanity check will exit with status 0
See also Python Pattern Module
See also Python Code Coverage Module
Belongs in Kategori Test
Belongs in Kategori Programmering