Regression Testing

We start to setup the regression testing infrastructure for our data processing pipeline by writting a test for the EyeSize class.

Consider reading Testing Your Code — The Hitchhiker's Guide to Python

EyeSize test

Identify inputs and outputs

Given a seed point, the EyeSize class provides an estimate method allowing to (1) segment the eye of the Saltritus and (2) compute the estimated eye radius.

Inputs	Saltritus image	seed point
Outputs	segmented eye	estimated eye radius

First draft

Let's have a look at a first implementation of function named eye_test that simply:

download from figshare an input image (id 1066744) and save it to disk as TralitusSaltrator.jpg
read the image with SimpleITK
set the estimator inputs
call estimate and get back the segmented eye and estimated radius



In [106]:

    
%cd
%cd dexy/test









    



/home/reproducible
/home/reproducible/dexy/test



In [107]:

    
%more eyesize_0_basic_test.py
%run eyesize_0_basic_test.py

eyesize_basic_test()

Exercise

Adapt the test to work with your figshare id.

How can we improve ?

1. Verbose on failure

Output information providing context, for example the current and expected value.

Note that logging information when functions and tests work as expected is not recommended, it will pollute the output and may prevent the developer from having his attention dran on real issue.



In [108]:

    
%more eyesize_1_noisy_test.py
%run eyesize_1_noisy_test.py

eyesize_noisy_test()

2. Clean plate

It is always good to start from a clean environment by removing files previously generated.

Instead of dumping the files along side the python scripts, let's organize files by having:

image downloaded into an inputs folder
segmented images written into an outputs folder



In [109]:

    
! rm TralitusSaltrator.jpg
! rm SegmentedEye.png



In [110]:

    
%more eyesize_2_cleanplate_test.py
%run eyesize_2_cleanplate_test.py

eyesize_cleanplate_test()

3. Associate names

Having a way to know the origin of your data or how they have been created can be very helpful. Especially when providing sscce

Doing this could be as simple as appending an id to an input name, and appending either a value or an experiment name to an output name.



In [112]:

    
%more eyesize_3_withorigin_test.py
%run eyesize_3_withorigin_test.py

eyesize_withorigin_test()



In [114]:

    
ls inputs









    



TralitusSaltrator_figshare_1066744.jpg



In [115]:

    
ls outputs









    



SegmentedEye_204_400.png

4. Automation

Considering that the number of tests will grow, calling each function one by one would be a tedious process. To help us, a test driver could be used.

Some major test drivers are:

5. Nosetests

Nosetests is a script that collects tests automatically.



In [117]:

    
!nosetests --nocapture -v









    



eyesize_0_basic_test.eyesize_basic_test ... ok
eyesize_1_noisy_test.eyesize_noisy_test ... ok
eyesize_2_cleanplate_test.eyesize_cleanplate_test ... ok
eyesize_3_withorigin_test.eyesize_withorigin_test ... ok

----------------------------------------------------------------------
Ran 4 tests in 7.174s

OK

6. Exercise

Add a test for the overlay_segmentation function available in eyesize.py