Theano : The Basics

Theano is an optimizing compiler for symbolic math expressions.

( Credit for this workbook : Eben Olson :: https://github.com/ebenolson/pydata2015 )



In [ ]:

    
import theano
import theano.tensor as T

Symbolic variables

Rather than manipulate values directly, Theano has to be able to build up the answer from the inputs - so that it can then optimise how to get to the solution (using the CPU or GPU as efficiently as possible). Therefore, the basic building blocks are 'symbolic variables' (as opposed to variables with concrete values) :



In [ ]:

    
x = T.scalar()

x

Variables can be used in expressions, but (IMPORTANT!) the result is symbolic as well :



In [ ]:

    
y = 3*(x**2) + x

type(y)

Investigating expressions



In [ ]:

    
print(y)



In [ ]:

    
theano.pprint(y)



In [ ]:

    
theano.printing.debugprint(y)



In [ ]:

    
from IPython.display import SVG
SVG(theano.printing.pydotprint(y, return_image=True, format='svg'))

Evaluating expressions

Supply a dict mapping variables to values



In [ ]:

    
y.eval({x: 2})

Or compile a function



In [ ]:

    
f = theano.function([x], y)

f(2)

Compiled function has been transformed



In [ ]:

    
SVG(theano.printing.pydotprint(f, return_image=True, format='svg'))

Other tensor types



In [ ]:

    
X = T.vector()
X = T.matrix()
X = T.tensor3()
X = T.tensor4()

Numpy style indexing



In [ ]:

    
X = T.vector()



In [ ]:

    
X[1:-1:2]



In [ ]:

    
X[ [1,2,3] ]

Many functions/operations are available through theano.tensor or variable methods



In [ ]:

    
y = X.argmax()



In [ ]:

    
y = T.cosh(X)



In [ ]:

    
y = T.outer(X, X)

But don't try to use numpy functions on Theano variables. Results may vary!

Automatic differention

Here is why most people want to work the Theano way : Gradients are free!



In [ ]:

    
x = T.scalar()
y = T.log(x)



In [ ]:

    
gradient = T.grad(y, x)
gradient.eval({x: 2})

Shared Variables

Symbolic + Storage (key usage: computation using the GPU RAM)



In [ ]:

    
import numpy as np
x = theano.shared(np.zeros((2, 3), dtype=theano.config.floatX))



In [ ]:

    
x

We can get and set the variable's value



In [ ]:

    
values = x.get_value()
print(values.shape)
print(values)



In [ ]:

    
x.set_value(values)

Shared variables can be used in expressions as well



In [ ]:

    
(x + 2) ** 2

Their value is used as input when evaluating



In [ ]:

    
((x + 2) ** 2).eval()



In [ ]:

    
theano.function([], (x + 2) ** 2)()

Updates

Store results of function evalution
dict mapping shared variables to new values
Main reason for using : iterative methods can use these (and operate entirely within the GPU, since the updates flow 'locally)



In [ ]:

    
count = theano.shared(0)
new_count = count + 1
updates = {count: new_count}

g = theano.function([], count, updates=updates)



In [ ]:

    
g()



In [ ]:

    
g()



In [ ]:

    
g()

Debugging

The message below should give you an idea of how easy this is...



In [ ]:

    
x = T.matrix()
y = 3*(x**2) + x

h = theano.function([x], y)

h( 2 )

THE ABOVE SHOULD BE AN ERROR!



In [ ]:

    
h( np.array( [ [ 2. ] ] ) )

THE ABOVE SHOULD BE AN ERROR!

Other Gotchas

Matrix multiply is element-wise...



In [ ]:

    
h( np.array( [ [ 2. ] ], dtype='float32' ) )



In [ ]:

    
h( np.array( [ [ 2., 3. ],  [ 4., 5. ] ], dtype='float32' ) )



In [ ]:

    
h( np.array( [ [ 2., 3., 4. ],  [ 6., 7., 8. ] ], dtype='float32' ) )



In [ ]: