PropertyCheck

PropertyCheck is a library for property based testing in Julia. In property based testing you check a proposition, $p$, against set of randomly generated data. If the proposition is true for all the data generated then the test passes. If the proposition is shown false for one of the examples then the example is shrunk and the smallest example breaking the proposition is returned.

The generator names in PropertyCheck are similar to those in Hypothesis. Generation of numbers (with signed, unsigned, floats), collections (with vectors, dicts, sets, tuples).

Examples



In [1]:

    
import PropertyCheck
const pc = PropertyCheck









    Out[1]:





PropertyCheck

Vector Append is Distributive

Suppose you wanted to make sure that your append method for vectors was distributive and that length of the vector returned after appending is equal to the sum of lengths of the original vectors. You could test your append method by simulating random vectors and verifying that the distributive and length properties holds for them.



In [2]:

    
# method you want test
function append(xs, ys)
    zs = copy(xs)
    append!(zs, ys)
    return zs
end

# distributive test
signeds = pc.signeds(0, 10)
vectors = pc.vectors(signeds, signeds)
vector_append_is_distributive(xs, ys, zs) = append(xs, append(ys, zs)) == append(append(xs, ys), zs)

pc.forAll(vector_append_is_distributive, (vectors, vectors, vectors))

# length preservation
vector_length_prop(xs, ys) = length(append(xs, ys)) == length(xs) + length(ys)
pc.forAll(vector_length_prop, (vectors, vectors))









    Out[2]:





true

Double Negation

Suppose you thought the double negation rule was true for floats ($x = -(-x)$). A quick test with PropertyCheck would show that it is not true.



In [3]:

    
# property you want to test
double_negation(x) = x == -(-x)

# property
pc.forAll(double_negation, (pc.floats(0.0, 50.0),), n=200)









    Out[3]:





true

However, if NaN values could not occur then the identity holds



In [4]:

    
pc.forAll(double_negation, 
          (pc.floats(0.0, 50.0, specialvalues=Float64[0.0, Inf, -Inf]),), 
          n=200)









    Out[4]:





true

Using with Factcheck

PropertyCheck works with FactCheck.



In [5]:

    
using FactCheck

@fact pc.forAll(double_negation, (pc.floats(0.0, 50.0),), n=200) --> true









    Out[5]:





Error :: (line:-1)
  Expression: pc.forAll(double_negation,(pc.floats(0.0,50.0),),n=200) --> true
  Found counterexample after 12 tests, 0 shrinks:
      
  counterexample: (NaN,)
  
  
   in forAll at /home/julia/.julia/v0.5/PropertyCheck/src/general.jl:81
   in anonymous at /home/julia/.julia/v0.5/FactCheck/src/FactCheck.jl:271
   in do_fact at /home/julia/.julia/v0.5/FactCheck/src/FactCheck.jl:333
   in include_string at loading.jl:266
   in execute_request_0x535c5df2 at /home/julia/.julia/v0.5/IJulia/src/execute_request.jl:177
   in eventloop at /home/julia/.julia/v0.5/IJulia/src/IJulia.jl:141
   in anonymous at task.jl:443

Using with `Base.Test` (v0.4)



In [6]:

    
using Base.Test

@test pc.forAll(double_negation, (pc.floats(0.0, 50.0),), n=200)









    



Error During Test
  Test threw an exception of type PropertyCheck.CounterexampleError{Tuple{Float64}}
  Expression: pc.





    



LoadError: There was an error during testing
while loading In[6], in expression starting on line 3

 in record at test.jl:274
 in do_test at test.jl:197





    



forAll(double_negation,(pc.floats(0.0,50.0),),n=200)
  Found counterexample after 77 tests, 0 shrinks:
      
  counterexample: (NaN,)
  
  
   in forAll at /home/julia/.julia/v0.5/PropertyCheck/src/general.jl:81
   in anonymous at test.jl:170
   in do_test at test.jl:186
   in include_string at loading.jl:266
   in execute_request_0x535c5df2 at /home/julia/.julia/v0.5/IJulia/src/execute_request.jl:177
   in eventloop at /home/julia/.julia/v0.5/IJulia/src/IJulia.jl:141
   in anonymous at task.jl:443

Using with `Base.Test` (v0.5)



In [7]:

    
@testset "Append Properties" begin
    @testset "Append is Distributive" begin
        @test pc.forAll(vector_append_is_distributive, (vectors, vectors, vectors))
    end
    @testset "Append and Addition Commute" begin
        @test pc.forAll(vector_length_prop, (vectors, vectors))
    end
end









    



Test Summary:     | Pass  Total
Append Properties |    2      2