Intro to Cryptol and High Assurance Crypto Engineering

LambdaConf 2015

Adam Foltzer, Research Engineer at Galois, Inc.

Goals

  • Understand the purpose of the Cryptol language and its role in high-assurance engineering
  • Create and modify cryptographic programs in Cryptol
  • Use the interactive Cryptol interpreter to develop, test, and prove properties about Cryptol programs
  • Learn how to participate in the Cryptol open source community
  • Anything else?

Outline

  • Introduce Cryptol (~15 min)
  • Hands-on lab for Chapter 2 (~10 min)
    • Get familiar with the Cryptol interpreter and basic expressions
  • Learn more Cryptol via examples and exercises (~20 min)
  • Hands-on lab for Chapter 3 (~15 min)
    • Classical cryptosystems: Caesar, Vigenère, Scytale
  • Break (~10min)
  • Introduce property-driven development (~15min)
  • Property-driven development exercises (~20min)
    • Hands-on lab for Chapter 5
    • Use :check, :sat, and :prove
  • ZUC cipher demo and closing discussion (~15min)

Formal Methods

  • How can software and systems be made robust (safe, secure, correct) in a cost-effective manner?
  • How can one obtain high assurance that a design has been faithfully implemented?
  • How can we ensure that other people’s systems are secure?
  • How can we compose a secure solution from black-box components?
  • Formal Methods are verification techniques that work by building a mathematical model of an artifact and proving properties about it
  • Formal methods are complementary to testing
    • Testing techniques generate weak evidence about the real artifact Worry: Have I tested enough?
    • Formal methods generate strong evidence about a model of the artifact Worry: Is the model faithful enough?

Our Take

  • Let the software itself be trustworthy
    • Software artifacts to speak for themselves
    • Reduce reliance on the process that created them
  • Use mathematical models to enable tractable analysis
    • Executable models and formal methods
    • A model is an abstraction that allows thought at a higher level
  • Follow open standards
    • Build individual components with high internal integrity
    • Maximize interoperability

Cryptol: Applying Formal Methods to Cryptography

  • Cryptography lacks clear reference implementations:

  • Cryptol is a domain-specific language for specifying cryptographic algorithms:
  • Size-polymorphic, and statically-typed with type inference
  • Lightweight Haskell-style module system
  • Interpreter with a read-eval-print loop (REPL)
  • Transparent integration with SAT and SMT solvers for proving properties expressed in Cryptol

Cryptol Specifications

  • File of mathematical definitions
    • Two kinds of definitions: values (x) and functions (F)
    • Definitions may be accompanied by a type declarations (a signature)
  • Definitions are computationally neutral
    • Cryptol tools provide the computational content (interpreters, compilers, code generators, verifiers)
  • Domain-specific data and control abstractions
    • Sequences (as in the definition of x)
    • Recurrence relations rather than loops
  • Algorithms parameterized on size
    • Size constraints are explicit in many specs
    • Number of iterations may depend on size
    • A sized type system captures and maintains size constraints

Basic Cryptol REPL Commands

N.B., these first few don't work in ICryptol notebooks

  • Load a module or a file
:m AES
:l AES.cry
  • Reload the current file
:r
  • Edit the current file
:e
  • Quit the interpreter
:q
  • Tab-complete identifiers
  • Browse current definitions

In [1]:
:browse


Type Synonyms
=============
    type Bool = Bit
    type Char = [8]
    type String n = [n][8]
    type Word n = [n]

Symbols
=======
    drop : {front, back, elem} (fin front) => [front +
                                               back]elem -> [back]elem
    groupBy : {each, parts, elem} (fin each) => [parts *
                                                 each]elem -> [parts][each]elem
    splitBy : {parts, each, elem} (fin each) => [each *
                                                 parts]elem -> [parts][each]elem
    tail : {a, b} [1 + a]b -> [a]b
    take : {front, back, elem} (fin front) => [front +
                                               back]elem -> [front]elem
    undefined : {a} a
    width : {bits, len, elem} (fin len, fin bits,
                               bits >= width len) => [len]elem -> [bits]

  • Get the type of an expression

In [2]:
:type 1+2
:type (+)
:type width


1 + 2 : {a} (fin a, a >= 2) => [a]
(+) : {a} (Arith a) => a -> a -> a
width : {bits, len, elem} (fin len, fin bits,
                           bits >= width len) => [len]elem -> [bits]
  • Set the base for output

In [3]:
:set base=8
10


Assuming a = 4
0o12
  • Show 8-bit sequences as ASCII

In [4]:
:set base=16
:set ascii=on
[0x48, 0x65, 0x6c, 0x6c, 0x6f] : [5][8]


"Hello"
  • Show all available commands

In [5]:
:help


  :t, :type          check the type of an expression
  :b, :browse        display the current environment
  :?, :help          display a brief description about a built-in operator
  :s, :set           set an environmental option (:set on its own displays current values)
  :check             use random testing to check that the argument always returns true (if no argument, check all properties)
  :exhaust           use exhaustive testing to prove that the argument always returns true (if no argument, check all properties)
  :prove             use an external solver to prove that the argument always returns true (if no argument, check all properties)
  :sat               use a solver to find a satisfying assignment for which the argument returns true (if no argument, find an assignment for all properties)
  :debug_specialize  do type specialization on a closed expression
  :q, :quit          exit the REPL
  :l, :load          load a module
  :r, :reload        reload the currently loaded module
  :e, :edit          edit the currently loaded module
  :!                 execute a command in the shell
  :cd                set the current working directory
  :m, :module        load a module
  • Show available user settings

In [6]:
:set


ascii = on
base = 16
debug = off
infLength = 5
iteSolver = off
mono-binds = on
prover = cvc4
satNum = 1
smtfile = -
tc-debug = 0
tc-solver = cvc4 --lang=smt2 --incremental --rewrite-divk
tests = 100
warnDefaulting = off
warnShadowing = off

Basic Cryptol Expressions

Bits


In [7]:
p = True
q = False


Homogeneous sequences


In [8]:
xs : [7]Bit
xs = [False, True, False, True, False, False, True]



In [9]:
xs @ 0


False

In [10]:
xs ! 0


True

In [11]:
[1 .. 5] # [3, 6, 8]


Assuming a = 4
[0x1, 0x2, 0x3, 0x4, 0x5, 0x3, 0x6, 0x8]

In [12]:
[0, 1, 2, 3] << 2


Assuming a = 2
[0x2, 0x3, 0x0, 0x0]

In [13]:
[0, 1, 2, 3] <<< 2


Assuming a = 2
[0x2, 0x3, 0x0, 0x1]

Nested sequences


In [14]:
ys : [2][4][4]Bit
ys = [[1, 2, 3, 4], [5, 6, 7, 8]]



In [15]:
ys @ 0


[0x1, 0x2, 0x3, 0x4]

In [16]:
ys @@ [0,1]


[[0x1, 0x2, 0x3, 0x4], [0x5, 0x6, 0x7, 0x8]]

In [17]:
width ys


Assuming a = 2
0x2

Sequence comprehensions


In [18]:
[ 2*x + 3 | x <- [1, 2, 3, 4] ] # [15]


Assuming a = 4
[0x5, 0x7, 0x9, 0xb, 0xf]

Words

  • Words are sequences of bits
  • Arithmetic is modulo word size

In [19]:
x, y, z : [8]
x = 123
y = 0xF4
z = 0b11110100



In [20]:
1 + 1


Assuming a = 1
0x0

In [21]:
(1 : [2]) + 1


0x2

Heterogeneous Tuples


In [22]:
t = (13, "hello", True)



In [23]:
t.1


"hello"

Records


In [24]:
type Point3D = { x : [16], y : [16], z : [16] }
p1 = { x = 22, y = 35, z = 18 } : Point3D



In [25]:
p1.x


0x0016

Boolean operators

  • Defined on bits as well as pointwise on structures

In [26]:
// Boolean operators defined on bits as well as pointwise on structures
b = True || False
w = 0xFFFF && 1
ws = [0x0000, 0xFFFF] ^ [0x1A1A, 0x0F0F]
p2 = ~p1



In [27]:
(b, w, ws, p2)


(True,
 0x0001,
 [0x1a1a, 0xf0f0],
 {x = 0xffe9, y = 0xffdc, z = 0xffed})

If-then-else


In [28]:
if 0xFF < 42 then 0xdead else 0xbeef


0xbeef

Local Bindings

  • where clauses bind values within definitions

In [29]:
isValid x = withinRange && isEven where
  withinRange = x > 5 && x < 10
  isEven = (x && 1) == 0


Functions

  • Functions are mathematical functions, not procedures that return values
  • Functions can have multiple arguments and return multiple results in a tuple

In [30]:
XYandXplusY : [8] -> [8] -> ([8], [8])
XYandXplusY x y = (xy, x + y) 
     where xy = x * y


  • Functions can be anonymous with lambda

In [31]:
(\(x, y) -> 2 + x*y) : ([8], [8]) -> [8]


<function>

Exercises

  • Chapter 2: Crash Course
    • Practice with basic language features
    • More exercises than we have time for; skim and refer to later

Sequence Comprehensions

(Exercises 1:7-43)

  • The most-used control structure in Cryptol

Cartesian comprehensions

Parallel comprehensions

Types

(Exercises 1:44-47)

  • All expressions have strong static types
  • Type inference adds flexibility
  • Monomorphic types:
    (2 >= 3) : Bit
    [0x02, 0x14, 0x05, 0x30] : [4][8]Bit      
    (3,5,True) : ([8],[32],Bool)
    F : ([16],[16]) -> [16]
  • Polymorphic types (a family of types):
    [2, 4, 5, 3] : {a}  [4][a]Bit
    tail : {a, b} [1 + a]b -> [a]b

Modules

  • A module Foo is defined in Foo.cry
module Foo where

import Bar

x = Bar.baz + 1
  • import statements go before declarations
  • Files with no module declaration are implicitly Main
  • Modules not currently supported in the ICryptol notebook

Recurrences

  • Shift circuits described in code
  • Stream definitions can be recursive and define infinite-length streams

In [32]:
nats = [0] # [ y + 1 | y <- nats ]


  • Not limited to one initial value and one recursive reference

In [33]:
as  = [0x3F, 0xE2, 0x65, 0xCA] # new
new = [ a ^ b ^ c | a <- as
                  | b <- drop`{1}as
                  | c <- drop`{3}as ]


Basic Functions

  • Some built-in primitives
  • Some functions defined in Cryptol.cry prelude

In [34]:
take`{2}[2 .. 10]


Assuming a = 4
[0x2, 0x3]

In [35]:
drop`{2}[2 .. 10]


Assuming a = 4
[0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa]

In [36]:
:type groupBy


groupBy : {each, parts, elem} (fin each) => [parts *
                                             each]elem -> [parts][each]elem

In [37]:
:set ascii=off
groupBy`{2}[1 .. 100]


Assuming a = 7
[[0x01, 0x02], [0x03, 0x04], [0x05, 0x06], [0x07, 0x08],
 [0x09, 0x0a], [0x0b, 0x0c], [0x0d, 0x0e], [0x0f, 0x10],
 [0x11, 0x12], [0x13, 0x14], [0x15, 0x16], [0x17, 0x18],
 [0x19, 0x1a], [0x1b, 0x1c], [0x1d, 0x1e], [0x1f, 0x20],
 [0x21, 0x22], [0x23, 0x24], [0x25, 0x26], [0x27, 0x28],
 [0x29, 0x2a], [0x2b, 0x2c], [0x2d, 0x2e], [0x2f, 0x30],
 [0x31, 0x32], [0x33, 0x34], [0x35, 0x36], [0x37, 0x38],
 [0x39, 0x3a], [0x3b, 0x3c], [0x3d, 0x3e], [0x3f, 0x40],
 [0x41, 0x42], [0x43, 0x44], [0x45, 0x46], [0x47, 0x48],
 [0x49, 0x4a], [0x4b, 0x4c], [0x4d, 0x4e], [0x4f, 0x50],
 [0x51, 0x52], [0x53, 0x54], [0x55, 0x56], [0x57, 0x58],
 [0x59, 0x5a], [0x5b, 0x5c], [0x5d, 0x5e], [0x5f, 0x60],
 [0x61, 0x62], [0x63, 0x64]]

In [38]:
(split [1 .. 100]) : [2]_


Assuming a = 7
[[0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b,
  0x0c, 0x0d, 0x0e, 0x0f, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16,
  0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f, 0x20, 0x21,
  0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c,
  0x2d, 0x2e, 0x2f, 0x30, 0x31, 0x32],
 [0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d,
  0x3e, 0x3f, 0x40, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48,
  0x49, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f, 0x50, 0x51, 0x52, 0x53,
  0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5a, 0x5b, 0x5c, 0x5d, 0x5e,
  0x5f, 0x60, 0x61, 0x62, 0x63, 0x64]]

Zero

  • Cryptol's zero is very flexible; it can have any type

In [39]:
zero : [2][2]Point3D


[[{x = 0x0000, y = 0x0000, z = 0x0000},
  {x = 0x0000, y = 0x0000, z = 0x0000}],
 [{x = 0x0000, y = 0x0000, z = 0x0000},
  {x = 0x0000, y = 0x0000, z = 0x0000}]]
  • Negating zero is handy for getting a value of all Trues

In [40]:
~zero : Point3D


{x = 0xffff, y = 0xffff, z = 0xffff}

ROT13

  • Substitution cipher
    • Each letter in the plaintext is replaced by a corresponding letter in the ciphertext
  • ROT13(ROT13(x)) == x
  • Hello World of cryptography (and Cryptol)

In [41]:
ROT13 : {n} [n][8] -> [n][8]
ROT13 msg = [ shift x | x <- msg ]
  where map     = ['A' .. 'Z'] <<< 13
        shift c = map @ (c - 'A')



In [42]:
map = ['A' .. 'Z'] <<< 13



In [43]:
:set ascii=on
map @@ [0,1,2,3,4,13]


Assuming a = 7
"NOPQRA"

In [44]:
shift c = map @ (c - 'A')



In [45]:
shift 'C'


Assuming a = 7
'P'

In [46]:
('C' - 'A') == 2


True

In [47]:
map @ 2


Assuming a = 7
'P'

In [48]:
[ shift x | x <- "HELLO" ]


Assuming a = 7
"URYYB"

Exercises

  • With ROT13 defined in ROT13.cry (in this repo)
Cryptol> :l ROT13.cry
Loading module Cryptol
Loading module Main
Main> :set ascii=on
Main> ROT13("HELLOWORLD")
“URYYBJBEYQ"
Main> ROT13(ROT13("HELLOWORLD"))
"HELLOWORLD"
  • Chapter 3: Classic Ciphers
    • Substitution ciphers: Caesar, Vigenère
    • Try to make it through Exercises 1-10

Property-Driven Development

Properties can express:

  • Correctness properties of a specification for validation
  • Equivalence of a high-level specification and an "implementation-specification"
  • Design principles that guide the development of a derived specification
  • The correctness of a compilation path
  • Equivalence of an implementation outside Cryptol and a specification

Design-Refinement Correctness

Properties in Cryptol

  • Cryptol values of type Bit: good for test vectors

In [49]:
property two_plus_two = 2 + 2 == 4
property ROT13_hello = ROT13("HELLO") == "URYYB"


  • Cryptol functions returning type Bit: good for broader statements

In [50]:
property plus_id_l x = 0 + x == x
property plus_assoc x y z = x + (y + z) == (x + y) + z


  • Arguments to properties can be any type

Randomized Testing

  • :check evaluates a property with random values (like QuickCheck)

In [51]:
:check \(x : [8]) -> x + 1 != x


Using random testing.
testing...
passed 100 tests.
Coverage: 39.06% (100 of 256 values)
  • :check either takes an expression, or checks all properties in the file
  • Intended as fast and easy checking as you program

Proving Properties

  • :check does not give proof
  • :prove has the same syntax, but proves properties for all values

In [52]:
:prove \(x : [8]) -> x + 1 != x


Q.E.D.
  • If a property is falsifiable, :check and :prove give counterexamples

In [53]:
:check \x -> x != 0x7


Using exhaustive testing.
FAILED for the following inputs:
0x7
  • Depending on the size of the input, :prove can be much more effective

In [54]:
haystack x = x != 0xdeadbeef



In [55]:
:check haystack


Using random testing.
testing...
passed 100 tests.
Coverage: 0.00% (100 of 2^^32 values)

In [56]:
:prove haystack


haystack 0xdeadbeef = False

Monomorphic Properties

  • Cryptol cannot currently reason about polymorphic functions

In [57]:
property plus_id_l x = 0 + x == x



In [58]:
:prove plus_id_l


Not a monomorphic type:
{a} (fin a) => [a] -> Bit
  • Instead, we provide a monomorphic type signature

In [59]:
:prove plus_id_l : [32] -> Bit


Q.E.D.
  • Increase assurance by making similar properties at multiple types

In [60]:
plus_id_l_1   : [1]   -> Bit
plus_id_l_8   : [8]   -> Bit
plus_id_l_32  : [32]  -> Bit
plus_id_l_128 : [128] -> Bit
property plus_id_l_1   x = plus_id_l x
property plus_id_l_8   x = plus_id_l x
property plus_id_l_32  x = plus_id_l x
property plus_id_l_128 x = plus_id_l x



In [61]:
:prove


:prove ROT13_hello
	
Q.E.D.
:prove plus_assoc
	
Not a monomorphic type:
{a} (Cmp a, Arith a) => a -> a -> a -> Bit
:prove plus_id_l
	
Not a monomorphic type:
{a} (fin a) => [a] -> Bit
:prove plus_id_l_1
	
Q.E.D.
:prove plus_id_l_128
	
Q.E.D.
:prove plus_id_l_32
	
Q.E.D.
:prove plus_id_l_8
	
Q.E.D.
:prove two_plus_two
	
Q.E.D.

Satisfiability

  • :prove: is a property true for all inputs?
  • :sat: is there any input that makes this property true?

Example

  • Cryptol does not come with matrix math routines, but we can implement them

In [62]:
mmult : {a, b, c, w} (fin a, fin b, fin w)
     =>  [a][b][w] -> [b][c][w] -> [a][c][w]
mmult xss yss = [ [ sum (col * row) | col <- transpose yss ] 
                  | row <- xss ]

sum : {a,n} (Arith a, fin n) => [n]a -> a
sum xs = sums!0
  where sums = [zero] # [ x + y | x <- xs | y <- sums ]

// 3x3 identity matrix
mi = [[1,0,0],
      [0,1,0],
      [0,0,1]]


  • Now we can use :sat to invert a matrix

In [63]:
ma : [3][3][72]
ma = [[4,2,3],
      [8,5,2],
      [5,8,9]]



In [64]:
:sat \x -> mmult ma x == mi


(\x -> mmult ma x == mi) [[0xbbe3d1070bbe3d1071,
                           0xf1e88385df1e88385e, 0x19d5b98a919d5b98a9],
                          [0x919d5b98a919d5b98a, 0xceadcc548ceadcc549, 0xda6c0964fda6c09650],
                          [0xa46756e62a46756e63, 0x33ab7315233ab73152,
                           0xf69b02593f69b02594]] = True

Exercises

  • Chapter 5: High-Assurance Programming
    • Example properties and intro to random testing, automated proving, and satisfiability checking
    • Try to look at Exercises 1-9, 12, 14, 15, 17-19

ZUC Demo

Open Source