Breaking simple things with battle_tested

While many will insist that since they kept their code "simple" enough so nothing has room to go wrong, I'd like to show you some interesting findings from poking different libraries with the fuzzer.


In [51]:
from battle_tested import fuzz, battle_tested

In [52]:
def add_to_hello(a):
    return 'hello '+a
add_to_hello('world')


Out[52]:
'hello world'

In [53]:
fuzz(add_to_hello, seconds=1, keep_testing=True)


testing: add_to_hello()
tests: 6            speed: 57/sec  avg: 57
tests: 307          speed: 851/sec  avg: 454
tests: 652          speed: 1064/sec  avg: 657
tests: 950          speed: 1090/sec  avg: 766
total tests: 1262
found 15 examples that break add_to_hello()

In [54]:
def add_to_hello_with_format(a):
    return 'hello {}'.format(a)
add_to_hello_with_format('world')


Out[54]:
'hello world'

In [55]:
fuzz(add_to_hello_with_format, seconds=1, keep_testing=True)


testing: add_to_hello_with_format()
tests: 1            speed: 9/sec  avg: 9
tests: 440          speed: 1206/sec  avg: 607
tests: 886          speed: 1420/sec  avg: 878
tests: 1390         speed: 1542/sec  avg: 1044
total tests: 1690
found 0 examples that break add_to_hello_with_format()

In [56]:
def count(a):
    return len(a)
count('hello')


Out[56]:
5

In [57]:
fuzz(count, seconds=1, keep_testing=True)


testing: count()tests: 1            speed: 9/sec  avg: 9

tests: 340          speed: 951/sec  avg: 480
tests: 754          speed: 1225/sec  avg: 728
tests: 1082         speed: 1247/sec  avg: 858
total tests: 1449
found 11 examples that break count()

In [ ]: