Testing your code is very important.

Getting used to writing testing code and running this code in parallel is now considered a good habit. Used wisely, this method helps you define more precisely your code’s intent and have a more decoupled architecture.

Some general rules of testing:

  • A testing unit should focus on one tiny bit of functionality and prove it correct.
  • Each test unit must be fully independent. Each test must be able to run alone, and also within the test suite, regardless of the order that they are called. The implication of this rule is that each test must be loaded with a fresh dataset and may have to do some cleanup afterwards. This is usually handled by setUp() and tearDown() methods.
  • Try hard to make tests that run fast. If one single test needs more than a few milliseconds to run, development will be slowed down or the tests will not be run as often as is desirable. In some cases, tests can’t be fast because they need a complex data structure to work on, and this data structure must be loaded every time the test runs. Keep these heavier tests in a separate test suite that is run by some scheduled task, and run all other tests as often as needed.
  • Learn your tools and learn how to run a single test or a test case. Then, when developing a function inside a module, run this function’s tests frequently, ideally automatically when you save the code.
  • Always run the full test suite before a coding session, and run it again after. This will give you more confidence that you did not break anything in the rest of the code.
  • It is a good idea to implement a hook that runs all tests before pushing code to a shared repository.
  • If you are in the middle of a development session and have to interrupt your work, it is a good idea to write a broken unit test about what you want to develop next. When coming back to work, you will have a pointer to where you were and get back on track faster.
  • The first step when you are debugging your code is to write a new test pinpointing the bug. While it is not always possible to do, those bug catching tests are among the most valuable pieces of code in your project.
  • Use long and descriptive names for testing functions. The style guide here is slightly different than that of running code, where short names are often preferred. The reason is testing functions are never called explicitly. square() or even sqr() is ok in running code, but in testing code you would have names such as test_square_of_number_2(), test_square_negative_number(). These function names are displayed when a test fails, and should be as descriptive as possible.
  • When something goes wrong or has to be changed, and if your code has a good set of tests, you or other maintainers will rely largely on the testing suite to fix the problem or modify a given behavior. Therefore the testing code will be read as much as or even more than the running code. A unit test whose purpose is unclear is not very helpful in this case.
  • Another use of the testing code is as an introduction to new developers. When someone will have to work on the code base, running and reading the related testing code is often the best thing that they can do to start. They will or should discover the hot spots, where most difficulties arise, and the corner cases. If they have to add some functionality, the first step should be to add a test to ensure that the new functionality is not already a working path that has not been plugged into the interface.

The Basics

Unittest

unittest is the batteries-included test module in the Python standard library. Its API will be familiar to anyone who has used any of the JUnit/nUnit/CppUnit series of tools.

Creating test cases is accomplished by subclassing unittest.TestCase.


In [1]:
from unittest import TestCase

def fun(x):
    return x + 1

class MyTest(TestCase):
    def setUp(self):
        pass
    
    def tearDown(self):
        pass
    
    def test_passing_int_value(self):
        self.assertEqual(fun(3), 4)

Doctest

The doctest module searches for pieces of text that look like interactive Python sessions in docstrings, and then executes those sessions to verify that they work exactly as shown.

Doctests have a different use case than proper unit tests: they are usually less detailed and don’t catch special cases or obscure regression bugs. They are useful as an expressive documentation of the main use cases of a module and its components. However, doctests should run automatically each time the full test suite runs.

A simple doctest in a function:


In [3]:
def square(x):
    """Return the square of x.
    
    this should return 4 as 2*2 = 4
    >>> square(2)
    4
    
    >>> square(-2)
    4
    
    This will return 1
    >>> square(-1)
    2
    """

    return x * x

import doctest
doctest.testmod()


**********************************************************************
File "__main__", line 12, in __main__.square
Failed example:
    square(-1)
Expected:
    2
Got:
    1
**********************************************************************
1 items had failures:
   1 of   3 in __main__.square
***Test Failed*** 1 failures.
Out[3]:
TestResults(failed=1, attempted=3)

In [ ]:


In [5]:
def listme(x):
    """Return the square of x.
    
    this should return 4 as 2*2 = 4
    >>> listme(2)
    4
    
    >>> listme(-2)
    4
    
    This will return 1
    >>> listme(-1)
    [-1, -1]
    """
    
    return x, x

import doctest
doctest.testmod()


**********************************************************************
File "__main__", line 5, in __main__.listme
Failed example:
    listme(2)
Expected:
    4
Got:
    (2, 2)
**********************************************************************
File "__main__", line 8, in __main__.listme
Failed example:
    listme(-2)
Expected:
    4
Got:
    (-2, -2)
**********************************************************************
File "__main__", line 12, in __main__.listme
Failed example:
    listme(-1)
Expected:
    [-1, -1]
Got:
    (-1, -1)
**********************************************************************
File "__main__", line 12, in __main__.square
Failed example:
    square(-1)
Expected:
    2
Got:
    1
**********************************************************************
2 items had failures:
   3 of   3 in __main__.listme
   1 of   3 in __main__.square
***Test Failed*** 4 failures.
Out[5]:
TestResults(failed=4, attempted=6)

When running this module from the command line as in python module.py, the doctests will run and complain if anything is not behaving as described in the docstrings.


In [ ]:


In [6]:
class Room(object):

    def __init__(self, name, description):
        self.name = name
        self.description = description
        self.paths = {}

    def go(self, direction):
        return self.paths.get(direction, None)

    def add_paths(self, paths):
        self.paths.update(paths)

Tools


Unit Testing Tools


In [ ]:

py.test

nose


In [3]:
from nose.tools import *


def test_room():
    gold = Room("GoldRoom",
                """This room has gold in it you can grab. There's a
                door to the north.""")
    assert_equal(gold.name, "GoldRoom")
    assert_equal(gold.paths, {})

def test_room_paths():
    center = Room("Center", "Test room in the center.")
    north = Room("North", "Test room in the north.")
    south = Room("South", "Test room in the south.")

    center.add_paths({'north': north, 'south': south})
    assert_equal(center.go('north'), north)
    assert_equal(center.go('south'), south)

def test_map():
    start = Room("Start", "You can go west and down a hole.")
    west = Room("Trees", "There are trees here, you can go east.")
    down = Room("Dungeon", "It's dark down here, you can go up.")

    start.add_paths({'west': west, 'down': down})
    west.add_paths({'east': start})
    down.add_paths({'up': start})

    assert_equal(start.go('west'), west)
    assert_equal(start.go('west').go('east'), start)
    assert_equal(start.go('down').go('up'), start)

In [3]:
import sys, os
import logging
import unittest

log = logging.getLogger()

from nose import core, loader

#logging.basicConfig(level=logging.DEBUG)

from types import ModuleType

In [4]:
from nose import SkipTest

def test_foo():
    assert True
    
def test_bar():
    assert False
    
def test_baz():
    raise SkipTest("Skipped")
    
import random
import time
def test_generate():
    for _ in range(random.randint(0, 10)):
        time.sleep(0.25)
        yield lambda x: None, _
    def fail(x):
        time.sleep(0.25)
        raise AssertionError("Failed")
    for _ in range(random.randint(0, 10)):
        yield fail, _
    def skip(x):
        time.sleep(0.25)
        raise SkipTest("Skipped")
    for _ in range(random.randint(0, 10)):
        yield skip, _

In [ ]: