When you are a self-taught programmer, it is very easy to miss what's the “proper” way of doing things. I had never thought of unit testing as a thing until I took a formal course in python. The course was all about doing things the way you would in a work environment.
The concept is that every line of code you write should be tested to avoid last minute surprises. It also helps in a way that you do not have to stop your working product to test the code but if you have testing set up, you can run the tests and if any errors happen, you will know which part of the code is broken (stating the obvious, obviously). Coverage is a tool that comes in handy in testing as it generates a report and gives you a percentage of your code that you have covered with testing. Coverage can be used in conjunction with unittest, pytest and even nosetest (but I am not familiar with that). We will do a basic and quick walk through of how to use coverage.
If you just want a quick basic guide, you can skip to the bottom of this article.
Source-code:
You can look at the github repo.
We have some python code that is pretty basic, there is a function that when given a name, will print out a hello statement with the name but if there's no input, it will print out a hello statement with stranger instead of the name. We also have some code that asks for user input, and then passes that input to the function for printing out the hello statement.
#tutorial.py
def say_hello(name=None):
if name != "":
print("Hello", name)
else:
print("Hello Stranger")
if __name__ == "__main__":
say_hello(input("What's your name? "))
Using unittest, we have another file, test_tutorial.py that tests the say_hello function but passing in “test” and checking if it prints out “Hello test” or not.
#test_tutorial.py
from tutorial import say_hello
from unittest import TestCase
from io import StringIO
from unittest.mock import patch
class PrintingTest(TestCase):
def test_say_hello(self):
name = 'test'
expected_output = 'Hello {}\n'.format(name)
with patch('sys.stdout', new=StringIO()) as fake_out:
say_hello(name)
self.assertEqual(fake_out.getvalue(), expected_output)
Now we will go through how can we use coverage to generate a report of how much code is covered with testing in out very simple code.
Installation
Like any other python library, Coverage can be installed with:
pip install coverage
If you are using anaconda distribution, you can use:
conda install coverage
You can verify your Coverage installation by checking the version:
coverage –version
Using Coverage
The idea is to use it along with your test runner. It is fairly simple through command line. We can go over some examples.
**Pytest: **If you are using pytest, you can add coverage -m before the command. So:
pytest arg1 arg2 arg3
It will become:
coverage run -m pytest arg1 arg2 arg3
**Unittest: **Personally, I am more used to unittest and using coverage with unittest is pretty simple. All you do is to replace python -m with coverage run -m. so:
python -m unittest test_code.py
It will become:
coverage run -m unittest test_code.py
Coverage will run the testing and collect data. In order for you to see it as a report, you type:
coverage report
Running all the steps on our code looks like the following:
>coverage run -m unittest test_tutorial.py
.
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Ran 1 test in 0.001s
OK
>coverage report
Name Stmts Miss Cover
— — — — — — — — — — — — — — — — — — — — — — — — —
test_tutorial.py 11 0 100%
tutorial.py 6 2 67%
— — — — — — — — — — — — — — — — — — — — — — — — —
TOTAL 17 2 88%
Why is the coverage 88%, you may ask. If we examine out tutorial.py once again, you can notice that the if statement has another branch, which is what it has to do in case of “else”. That also needs to be covered by our tests. Therefore we can write another method in the testing class to cover that branch.
#test_tutorial.py
class PrintingTest(TestCase):
.........
def test_say_hello_noname(self):
name = ''
expected_output = 'Hello Stranger\n'
with patch('sys.stdout', new=StringIO()) as fake_out:
say_hello(name)
self.assertEqual(fake_out.getvalue(), expected_output)
This second test passes an empty string and see if the output is “Hello Stranger”. Running coverage now generates the following report:
>coverage run -m unittest test_tutorial.py
..
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —
Ran 2 tests in 0.001s
OK
>coverage report
Name Stmts Miss Cover
— — — — — — — — — — — — — — — — — — — — — — — — — — —
test_tutorial.py 17 0 100%
tutorial.py 6 1 83%
— — — — — — — — — — — — — — — — — — — — — — — — — — —
TOTAL 23 1 96%
Definitely some improvement in coverage however, you can see that is it still not 100%. The reason is that the snippet of code in the end of the tutorial.py, that asks for user input and calls the function, that code is not covered by testing. But what if I do not want to cover it and also don't want Coverage to include that in test coverage report?
**Excluding code from coverage: **It is very simple to do that, just add “ # pragma: no cover”.
........
if __name__ == "__main__":
say_hello(input("What's your name? ")) # pragma: no cover
Lets run our testing and generate a coverage report once again.
...........
Name Stmts Miss Cover
— — — — — — — — — — — — — — — — — — —— — — — —
test_tutorial.py 17 0 100%
tutorial.py 5 0 100%
— — — — — — — — — — — — — — — — — — — — — — — —
TOTAL 22 0 100%
Now it doesn't end here! If you want a detailed report with a better interface that gives you line by line information, you type:
coverage html
This will generate a graphical interface in the form of a web page. This is usually saved in in a folder named “htmlcov” in the project folder and to view it, open the index.html file. I encourage you to do that, the report is very detailed displaying code that is covered and code that is not.
Quick Guide:
**Installation:**
$pip install coverage
If you are using anaconda distribution, you can use:
$conda install coverage
You can verify your Coverage installation by checking the version:
$coverage –version
**Using Coverage**
**Pytest**
$coverage run -m pytest arg1 arg2 arg3
**Unittest**
$coverage -m unittest test_code
**Generating report**
$coverage report
**
Generating HTML report**
$coverage html
**Excluding code from coverage
**Add a comment after the line "# pragma: no cover"
Articles that I found very useful and were my basic source of information for this tutorial are: Coverage.py - Coverage.py 5.1 documentation *Coverage.py is a tool for measuring code coverage of Python programs. It monitors your program, noting which parts of…*coverage.readthedocs.io
https://www.geeksforgeeks.org/python-testing-output-to-stdout/