Property-based testing

From the Hypothesis docs:

It works by letting you write tests that assert that something should be true for every case, not just the ones you happen to think of.

Think of a normal unit test as being something like the following:

Set up some data.

Perform some operations on the data.

Assert something about the result.

Hypothesis lets you write tests which instead look like this:

For all data matching some specification.

Perform some operations on the data.

Assert something about the result.

This is often called property based testing, and was popularised by the Haskell library Quickcheck.

Example

Here's an example with a standard test. Note how we've only tested a single value here.

def increment(x):
    return x + 1

def test_increment():
    x = 10  # We've only tested one value here.
    assert increment(x) - 1 == x

Let's modify this test with a "property-based" test. In this case, we may choose to test it with a bunch of integers sampled using Hypothesis' strategies module.

from hypothesis import given
from hypothesis import strategies as st

@given(st.integers())  # We are going to test a wide range of integers.
def test_increment(x):
    assert increment(x) - 1 == x

Exercise

Take a challenge here: where possible (and using the Hypothesis documentation to help you), create "property-based" versions of one or more other functions in your datafuncs.py module.

Encoding Assumptions

Your functions take in data and output other data. There may be assumptions implicit in your choice of data. Hypothesis can help you make these assumptions explicit as you iteratively build the test. Let's take the example where we're computing the roots of a quadratic equation.

To help you recall, a quadratic equation is of the form:

$$ ax^2 + bx + c $$

The roots of the equation are given by the formula:

$$ \frac{-a \pm \sqrt{b^2 - 4ac}}{2a}$$

The discriminant $ D $ is the portion in the square root:

$$ \sqrt{b^2 - 4ac} $$

If:

$ D > 0 $, the roots are real-valued.
$ D = 0 $, both roots are real-valued and identical.
$ D < 0 $, there are complex roots.

We can thus write an eq_roots(coefficients) function that returns the non-complex roots of a polynomial equation. Given the requirements of the function definition, what would we expect of the function that can be encoded in a test? These expressions can be encoded in the assume() function from hypothesis.

Exercise

To get a feel for how to use the assume() function from hypothesis, uncomment the line indicated in the code block below, complete the assume() conditional, and copy/paste it to test_datafuncs.py.

# In test_datafuncs.py
from hypothesis import assume

@given(st.integers(), st.integers(), st.integers())
def test_eq_roots(a, b, c):
    # assumption here can mirror the assertion in
    # the original function definition
    discriminant = b**2 - 4*a*c
    # Uncomment the following line.
    # assume(discriminant...)
    r1, r2 = eq_roots(coefficients)
    assert r1 >= r2

Now what's left is implementing the function. Implement the function, such that its signature is eq_roots(coefficients), and returns two values, root1, root2.

import math

def eq_roots(coefficients):
    """
    Returns the non-complex roots of a quadratic equation.
    """
    ...

    return root1, root2

Finally, run pytest. Look out to see if Hypothesis finds out a few edge cases you didn't already think about!

$ py.test



In [ ]: