Testing

Purpose: when developping, make sure all the parts of your code are still working fine

Basic Testing and Debugging

assert: making sure horrible things don't happen in your code without you knowing about it.
ipdb: because print statements can only get you so far

assert causes an error when a condition fails, so it's a good way to make sure that if something breaks in your code you know about it right away, before the error propagates to somewhere else. Try out the example below:

def make_neuron_array(array_size):
    neuron_array = [0] * array_size
    assert len(neuron_array) == array_size
    return neuron_array

For info, check out this article.

ipdb.set_trace() can be used to open an iPython shell wherever in your code you have a problem. You can also step through the code using n. For example, after you've installed ipdb using pip try running this code and stepping through it:

import ipdb

print("start")
vals = [3, 1, 4]
ipdb.set_trace()
vals[0] += 5
print("done")

You can also use ipdb to open a debug shell whenever an exception occurs using the following code, taken from a StackOverflow question:

import sys
from IPython.core import ultratb
sys.excepthook = ultratb.FormattedTB(mode='Verbose',
     color_scheme='Linux', call_pdb=1)

Unit Testing

Forcing your code into little functions (that do one thing) so you can test in an automated manner.

Why?

• Forces you to write modular code
• Your tests double as documentation for examples on how the code should work
• Save time double checking little pieces of your code aren't breaking when you change something

Unit Testing in Python with PyTest

Python includes a unit testing module by default, but it has a lot boilerplate. Instead use PyTest!

Basic Testing with PyTest

The following examples use the file structure defined in Installation - setuptools.ipynb, repeated below:

|- README.md
|- killer
|   |-- __init__.py
|   |-- kill.py
|   | -- tests
|   |   |-- test_kill.py

In test_kill.py let's make the most basic test possible.

import pytest

import killer.kill as kk

def test_spray():
    kk.spray()

Note that all test functions have to start with the word test. Now run it with py.test test_kill.py and watch the test pass.

Fixtures

Basic Fixtures

In kill.py two more functions that take the same argument, but do different stuff with it.

def throw(thing):
    print("Throwing %s!" %(thing,))

def kick(thing):
    print("Kicking %s!" %(thing,))

To test these functions, we don't want to define the argument twice, so instead we use a fixture. Add the following code to test_kill.py.

@pytest.fixture
def thing():
   return "socks"

def test_kick(thing):
    kk.kick(thing)

def test_kick(thing):
    kk.throw(thing)

PyTest suppresses any print statements, so to convince yourself the argument is really being passed, comment out the fixture and notice that the tests fail.

Nengo Fixtures

Nengo has some useful fixtures built in for plotting the results of tests, setting random seeds, as well as logging the outputs and speed of test execution. To use them, add nengo.conftest import *. To see them in action, just run the tests in Nengo and look at the outputs.

Classes

You can also group tests in a class like so:

class TestBasic(object):

    def test_spray(self):
        kk.spray()

    def test_shoot(self):
        kk.shoot()

This also means that you can inherit classes for testing. For a good example of how and why to do this, check out this pull request by Jan where he tests copying Nengo objects with minimal repeating code.

Using conftest.py So You "Don't Repeat Yourself"

If you need fixtures or classes that should be shared accross various testfiles, you can put them in conftest.py. For example, if you needed the thing fixture for other testfiles, you could move it to conftest.py and it would be found automatically when you ran tests.

Things not covered in this talk, but possible with PyTest

• Logging how long tests take
• Using command-line options to run tests in specific manners
• Running multiple tests at once
• Testing for errors and warning