Unit testing
In general, tests are an assessment of both the quality and the efficiency of your code.
Tests actually define the requirements of the code at various levels. From the basic method definition, to the full software validation.
For each of these levels, a type of test exists :
From this terminology, the unit tests are the basic elements, that should be run before any commit of the code. They are the one that will be the focus of this course.
Based on :
“The Art of Software Testing”, Glenford J. Myers, Corey Sandler, Tom Badgett, Wiley & Sons, 2011
Rightness validate the results against the requirements
Ex: a method supposed to select the largest number of a list should be checked by comparing the result with the identified maximum from a known list.
Inverse relationships
Ex1: a method calculating the square root of a number can be checked by squaring the result. Ex2: a method inserting a value in a file can be checked by searching this value after insertion.
This should be done with tools independent from the method to test (other library).
Cross-check:
Ex: an analytical result can be compared to a numerical calculations for values or conditions where it is possible.
Code logic:
Ex:
if ((x or y) and z):
decision_1
else:
decision_2
where x, yand z are called "conditions".
Combinations of conditions could be unexpected and lead to a wrong decision. Some decisions are never reached because a different combinations of conditions always lead to the same logical value.
Exceptions:
If the methods throws exceptions under a certain conditions, this should be checked with a test. See this section.
with pytest.raises(<NameOfException>):
# call to the method that should throw the exception
Boundary conditions:
Error conditions:
Force error conditions
Test-driven development (TDD) is a software development process, part of the Agile principles, that relies on the transcription of the software requirements into tests, before the code that passes the tests is written.
It is an iterative process that aims at starting with a basic test case andthen upgrading alternatively the test cases and the code until the requirements are met, depending on the expectations.
In [ ]:
%%file my_first_test.py
def identity(a):
return a
def test_a():
assert identity(1) == 1
The file has been saved in the current directory
In [ ]:
!ls *test.py
Launching pytest is as easy as move to the right directory and using the command line
py.test
It will start a recursive search from the current directory for Python files, look for methods containing "test" and run them.
In [ ]:
!py.test my_first_test.py
For a quick summary, use the quick option -q
In [ ]:
!py.test -q my_first_test.py
For more information on which test has been run, use the verbose option -v
In [ ]:
!py.test -v my_first_test.py
%%file path/to/filename.py magic command will write the remainder of the cell into the file.
In [ ]:
%%file my_second_test.py
def identity(a):
return a
def test_a():
assert identity(1) == 1
def test_b():
assert identity(2) == 1
def test_c():
assert identity(3) == 1 + 1 + 1
In [ ]:
!py.test -v my_second_test.py
We see pytest has collected and run 4 items, 1 from the first file, and 3 from the second.
As expected, one test has failed.
Therefore pytest shows the full traceback leading to the failure, and even gives the output value of the f method which can be useful for quick debugging.
The philosophy of Python is to try something first and then decide what to do in case of an error. This is the reason behind Python Exceptions. They inform on the issue that was detected and help the user debug or catch it and find another way to deal with the issue.
When testing a code, it is thus important to assess if these Exceptions are raised as they should be. However, since an exception raised but not caught in an environmment triggers an error, one cannot use the "assert" syntax but the context manager pytest.raises instead.
In [ ]:
%%file my_third_test.py
import pytest
def positive_identity(n):
if n < 0:
raise ValueError("Negative value detected")
return n
def test_positive_identity():
assert positive_identity(1) == 1
def test_exception_positive_identity():
with pytest.raises(ValueError):
positive_identity(-1)
In [ ]:
!py.test -v my_third_test.py
In [ ]:
import numpy.testing
import pandas.testing
In [ ]:
# List of NumPy assert methods
[func for func in dir(numpy.testing) if func.startswith('assert')]
In [ ]:
# List of Pandas assert methods
[func for func in dir(pandas.testing) if func.startswith('assert')]
The numpy assert methods will be used to write the tests in the next section.
In [ ]:
!rm -rf my*test.py
spectra_analysis moduleThis part explains how to organize your tests for testing a module. It uses the spectra_analysis module created and upgrade all along this course.
We will write tests for the two submodules:
spectra_analysis/preprocessing.pyspectra_analysis/regression.py
In [ ]:
# Cleaning current directory
!rm -rf spectra_analysis
In [ ]:
# Copy clean version from the previous course
!cp -vr solutions/04_modules/spectra_analysis .
Then we'll create the structure for the tests
In [ ]:
# Test directory
!mkdir spectra_analysis/tests
# __init__.py file so the directory is recognized as a submodule
!touch spectra_analysis/tests/__init__.py
# create a local copy of test data
!mkdir spectra_analysis/tests/data
!cp solutions/data/data.csv spectra_analysis/tests/data/.
In [ ]:
from spectra_analysis.preprocessing import read_spectra
For a closer look at the content of the method, use the "??" syntax
In [ ]:
#read_spectra??
We will create a file called test_preprocessing.py with two unit tests: one for testing the normal conditions, the other one for handling the exceptions.
In [ ]:
%%file spectra_analysis/tests/test_preprocessing.py
from spectra_analysis.preprocessing import read_spectra
def test_read_spectra():
# Write here the tests for the normal conditions
pass
def test_read_spectra_exceptions():
# Write here the tests to handle errors
pass
data.csv fileVerify that the tests pass by running pytest.
In [ ]:
!py.test -v spectra_analysis/tests/
We now focus on the second file regression.py.
In [ ]:
from spectra_analysis.regression import fit_params, transform
This time we will create a file called test_regression.py with two unit tests corresponding to the methods fit_params and transform.
In [ ]:
%%file spectra_analysis/tests/test_regression.py
from spectra_analysis.regression import fit_params
from spectra_analysis.regression import transform
def test_fit_params():
pass
def test_transform():
pass
X = np.array([[0, 0, 0],
[0, 0, 0],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[2, 2, 2]])
Again, verify that the tests pass.
In [ ]:
!py.test -v spectra_analysis/tests/
A test coverage is a report on the percentage of lines of a module that have been run during a test.
The higher the coverage, the greater the number of code lines that have been executed at least once during a test.
To use coverage with pytest, one must first install pytest coverage plugin, e.g. the pytest-cov library:
conda install pytest-cov
or
pip install pytest-cov
The coverage can then be run alongside the testing, by setting the path to the module to run the coverage against with an extra argument --cov=<path to the module>.
In [ ]:
!py.test -v --cov=spectra_analysis spectra_analysis/tests/
The coverage table lists for each file in the module tree
Stmts: the number of actual lines of code in the file,Miss: the number of code lines missed by the tests,Cover: the resulting coverage percentage.
In [ ]: