Theano

A language in a language

Dealing with weights matrices and gradients can be tricky and sometimes not trivial. Theano is a great framework for handling vectors, matrices and high dimensional tensor algebra. Most of this tutorial will refer to Theano however TensorFlow is another great framework capable of providing an incredible abstraction for complex algebra. More on TensorFlow in the next chapters.



In [1]:

    
import theano
import theano.tensor as T

Symbolic variables

Theano has it's own variables and functions, defined the following



In [2]:

    
x = T.scalar()



In [3]:

    
x









    Out[3]:





<TensorType(float64, scalar)>

Variables can be used in expressions



In [4]:

    
y = 3*(x**2) + 1

y is an expression now

Result is symbolic as well



In [5]:

    
type(y)
y.shape









    Out[5]:





Shape.0

printing

As we are about to see, normal printing isn't the best when it comes to theano



In [6]:

    
print(y)









    



Elemwise{add,no_inplace}.0



In [7]:

    
theano.pprint(y)









    Out[7]:





'((TensorConstant{3} * (<TensorType(float64, scalar)> ** TensorConstant{2})) + TensorConstant{1})'



In [8]:

    
theano.printing.debugprint(y)









    



Elemwise{add,no_inplace} [id A] ''   
 |Elemwise{mul,no_inplace} [id B] ''   
 | |TensorConstant{3} [id C]
 | |Elemwise{pow,no_inplace} [id D] ''   
 |   |<TensorType(float64, scalar)> [id E]
 |   |TensorConstant{2} [id F]
 |TensorConstant{1} [id G]

Evaluating expressions

Supply a dict mapping variables to values



In [9]:

    
y.eval({x: 2})









    Out[9]:





array(13.0)

Or compile a function



In [10]:

    
f = theano.function([x], y)



In [11]:

    
f(2)









    Out[11]:





array(13.0)

Other tensor types



In [12]:

    
X = T.vector()
X = T.matrix()
X = T.tensor3()
X = T.tensor4()

Automatic differention

Gradients are free!



In [13]:

    
x = T.scalar()
y = T.log(x)



In [14]:

    
gradient = T.grad(y, x)
print(gradient)
print(gradient.eval({x: 2}))
print((2 * gradient))









    



Elemwise{true_div}.0
0.5
Elemwise{mul,no_inplace}.0

Shared Variables

Symbolic + Storage



In [15]:

    
import numpy as np
x = theano.shared(np.zeros((2, 3), dtype=theano.config.floatX))



In [16]:

    
x









    Out[16]:





<TensorType(float64, matrix)>

We can get and set the variable's value



In [17]:

    
values = x.get_value()
print(values.shape)
print(values)









    



(2, 3)
[[ 0.  0.  0.]
 [ 0.  0.  0.]]



In [18]:

    
x.set_value(values)

Shared variables can be used in expressions as well



In [19]:

    
(x + 2) ** 2









    Out[19]:





Elemwise{pow,no_inplace}.0

Their value is used as input when evaluating



In [20]:

    
((x + 2) ** 2).eval()









    Out[20]:





array([[ 4.,  4.,  4.],
       [ 4.,  4.,  4.]])



In [21]:

    
theano.function([], (x + 2) ** 2)()









    Out[21]:





array([[ 4.,  4.,  4.],
       [ 4.,  4.,  4.]])

Updates

Store results of function evalution
dict mapping shared variables to new values



In [22]:

    
count = theano.shared(0)
new_count = count + 1
updates = {count: new_count}

f = theano.function([], count, updates=updates)



In [23]:

    
f()









    Out[23]:





array(0)



In [24]:

    
f()









    Out[24]:





array(1)



In [25]:

    
f()









    Out[25]:





array(2)