

In [35]:

    
# this is just to silence 
%xmode plain









    



Exception reporting mode: Plain

Using `battle_tested` to feel out new libraries.

battle_tested doesn't necessisarily need to be used as a fuzzer. I like to use its testing funcionality to literally "feel out" a library that is recommended to me so I know what works and what will cause issues.

Here is how I used battle_tested to "feel out" sqlitedict so when I'm using it, there aren't any surprises.

First, lets import SqliteDict and make a harness that will allow us to test what can be assigned and what will cause random explosions to happen.



In [36]:

    
from sqlitedict import SqliteDict

def harness(key, value):
    """ this tests what can be assigned in SqliteDict's keys and values """
    mydict = SqliteDict(":memory:")
    mydict[key] = value

Now, we import the tools we need from battle_tested and fuzz it.



In [37]:

    
from battle_tested import fuzz, success_map, crash_map

fuzz(harness, keep_testing=True) # keep testing allows us to collect "all" crashes









    



testing: harness()
tests: 9            speed: 88/sec  avg: 88
tests: 61           speed: 171/sec  avg: 130
tests: 115          speed: 188/sec  avg: 149
tests: 166          speed: 192/sec  avg: 160
tests: 218          speed: 195/sec  avg: 167
tests: 265          speed: 193/sec  avg: 171
tests: 311          speed: 191/sec  avg: 174
tests: 359          speed: 191/sec  avg: 176
total tests: 387
found 1 examples that break harness()

Now we can call success_map() and crash_map() to start to get a feel for what is accepted and what isn't.



In [38]:

    
crash_map()









    Out[38]:





({'arg_types': (dict, hypothesis.strategies.iterables.<locals>.PrettyIter),
  'args': ({'\n児\r': (0.248129851304151-5.746517236706694e+18j),
    '(\n*\x05c\U000a0c6dꍥ\x16': (),
    '\x90\x10\x02\U000606feG6\U0006c4d7': '\x9a',
    '\x92\x08óC\U0006a176aƕ': -2525536483606756998302721813900421},
   iter([])),
  'message': "Can't pickle local object 'iterables.<locals>.PrettyIter'",
  'type': AttributeError},)



In [40]:

    
success_map()









    Out[40]:





((decimal.Decimal, bytes),
 (decimal.Decimal, hypothesis.strategies.RandomSeeder),
 (decimal.Decimal, hypothesis.types.RandomWithSeed),
 (decimal.Decimal, str),
 (decimal.Decimal, bool),
 (decimal.Decimal, uuid.UUID),
 (decimal.Decimal, float),
 (decimal.Decimal, decimal.Decimal),
 (fractions.Fraction, list),
 (fractions.Fraction, complex),
 (fractions.Fraction, tuple),
 (fractions.Fraction, hypothesis.strategies.RandomSeeder),
 (fractions.Fraction, bytes),
 (fractions.Fraction, float),
 (NoneType, fractions.Fraction),
 (NoneType, list),
 (NoneType, hypothesis.types.RandomWithSeed),
 (NoneType, decimal.Decimal),
 (NoneType, hypothesis.strategies.RandomSeeder),
 (NoneType, str),
 (NoneType, int),
 (NoneType, NoneType),
 (NoneType, dict),
 (hypothesis.strategies.iterables.<locals>.PrettyIter, dict),
 (hypothesis.strategies.iterables.<locals>.PrettyIter, float),
 (hypothesis.strategies.iterables.<locals>.PrettyIter,
  hypothesis.types.RandomWithSeed),
 (hypothesis.strategies.iterables.<locals>.PrettyIter, list),
 (hypothesis.strategies.iterables.<locals>.PrettyIter, uuid.UUID),
 (hypothesis.strategies.iterables.<locals>.PrettyIter, bytes),
 (hypothesis.strategies.iterables.<locals>.PrettyIter, tuple),
 (hypothesis.strategies.iterables.<locals>.PrettyIter, complex),
 (hypothesis.strategies.RandomSeeder, hypothesis.types.RandomWithSeed),
 (hypothesis.strategies.RandomSeeder, str),
 (hypothesis.strategies.RandomSeeder, decimal.Decimal),
 (hypothesis.strategies.RandomSeeder, bytes),
 (hypothesis.strategies.RandomSeeder, float),
 (hypothesis.strategies.RandomSeeder, tuple),
 (hypothesis.strategies.RandomSeeder, fractions.Fraction),
 (hypothesis.strategies.RandomSeeder, dict),
 (hypothesis.strategies.RandomSeeder, uuid.UUID),
 (hypothesis.strategies.RandomSeeder, bool),
 (hypothesis.types.RandomWithSeed, NoneType),
 (hypothesis.types.RandomWithSeed, float),
 (hypothesis.types.RandomWithSeed, hypothesis.types.RandomWithSeed),
 (hypothesis.types.RandomWithSeed, int),
 (hypothesis.types.RandomWithSeed, tuple),
 (hypothesis.types.RandomWithSeed, fractions.Fraction),
 (hypothesis.types.RandomWithSeed, bool),
 (hypothesis.types.RandomWithSeed, dict),
 (hypothesis.types.RandomWithSeed, complex),
 (uuid.UUID, dict),
 (uuid.UUID, list),
 (uuid.UUID, complex),
 (uuid.UUID, bytes),
 (uuid.UUID, int),
 (uuid.UUID, decimal.Decimal),
 (uuid.UUID, uuid.UUID),
 (uuid.UUID, NoneType),
 (bool, bool),
 (bool, list),
 (bool, decimal.Decimal),
 (bool, bytes),
 (bool, hypothesis.strategies.RandomSeeder),
 (bool, uuid.UUID),
 (bool, hypothesis.types.RandomWithSeed),
 (bool, fractions.Fraction),
 (bool, str),
 (bool, float),
 (bool, NoneType),
 (bool, tuple),
 (bool, dict),
 (bool, complex),
 (bytes, NoneType),
 (bytes, uuid.UUID),
 (bytes, hypothesis.types.RandomWithSeed),
 (bytes, bool),
 (bytes, bytes),
 (bytes, str),
 (bytes, tuple),
 (bytes, dict),
 (bytes, list),
 (bytes, float),
 (bytes, hypothesis.strategies.RandomSeeder),
 (complex, float),
 (complex, fractions.Fraction),
 (complex, complex),
 (complex, int),
 (complex, hypothesis.strategies.RandomSeeder),
 (complex, str),
 (complex, hypothesis.types.RandomWithSeed),
 (dict, decimal.Decimal),
 (dict, hypothesis.types.RandomWithSeed),
 (dict, int),
 (dict, bool),
 (dict, dict),
 (dict, hypothesis.strategies.RandomSeeder),
 (dict, complex),
 (dict, tuple),
 (dict, float),
 (dict, bytes),
 (dict, str),
 (dict, NoneType),
 (float, hypothesis.types.RandomWithSeed),
 (float, int),
 (float, bool),
 (float, dict),
 (float, bytes),
 (float, str),
 (float, list),
 (float, NoneType),
 (int, hypothesis.strategies.RandomSeeder),
 (int, NoneType),
 (int, fractions.Fraction),
 (int, str),
 (int, bytes),
 (int, complex),
 (int, bool),
 (int, hypothesis.types.RandomWithSeed),
 (int, decimal.Decimal),
 (list, bool),
 (list, fractions.Fraction),
 (list, bytes),
 (list, str),
 (list, dict),
 (str, str),
 (str, NoneType),
 (str, hypothesis.types.RandomWithSeed),
 (str, int),
 (str, dict),
 (str, tuple),
 (str, list),
 (str, bytes),
 (str, uuid.UUID),
 (str, hypothesis.strategies.RandomSeeder),
 (str, decimal.Decimal),
 (str, complex),
 (tuple, bool),
 (tuple, dict),
 (tuple, fractions.Fraction),
 (tuple, uuid.UUID),
 (tuple, list),
 (tuple, float),
 (tuple, NoneType),
 (tuple, hypothesis.types.RandomWithSeed),
 (tuple, bytes),
 (tuple, str))



In [ ]:

Using battle_tested to feel out new libraries.

Using `battle_tested` to feel out new libraries.