Start with TensorFlow



In [1]:

    
# import some python packages
%matplotlib inline
import matplotlib.pyplot as plt 
import seaborn

What is TensorFlow? TensorFlow is a deep learning library recently open-sourced by Google. It is a programming system in which you represent computations as graphs. Nodes in the graph are called ops (short for operations). An op takes zero or more Tensors, performs some computation, and produces zero or more Tensors. TensorFlow provides primitives for defining functions on tensors and automatically computing their derivatives.

Formally, tensors are mulinear maps from vectors spaces to the real numbers. A Tensor can be represented as a multi-dimensional array of numbers. For example, you can represent a mini-batch of images as a 4-D array of floating point numbers with dimensions [batch, height, width, channels].

Every operation of machine learning can be writen as a function on tensors

Why TensorFlow? TensorFlow is quite similar to Numpy, the most important difference is that numpy doesn't offer methods to create tensors as a functions and automatically compute derivatives (also, numpy does not provide GPU support)

TensorFlow Mechanics:

Prepare the Data
- Inputs and Placeholders
Build the Graph
- Inference
- Loss
- Training
Train The model
- The Session
- The Graph
- Train loop
Evaluate the model

Simple Numpy Recap



In [2]:

    
import numpy as np

a = np.zeros((2,2)); b = np.ones((2,2))
print np.sum(b,axis=1)









    



[ 2.  2.]



In [3]:

    
print a.shape









    



(2, 2)



In [4]:

    
print np.reshape(a,(1,4))









    



[[ 0.  0.  0.  0.]]

Let's do the same with TensorFlow



In [5]:

    
import tensorflow as tf



In [6]:

    
sess = tf.InteractiveSession()



In [7]:

    
a = tf.zeros((2,2)); b = tf.ones((2,2))



In [8]:

    
tf.reduce_sum(b, reduction_indices=1).eval()









    Out[8]:





array([ 2.,  2.], dtype=float32)



In [9]:

    
a.get_shape()









    Out[9]:





TensorShape([Dimension(2), Dimension(2)])



In [10]:

    
tf.reshape(a, (1, 4)).eval()









    Out[10]:





array([[ 0.,  0.,  0.,  0.]], dtype=float32)



In [11]:

    
a = np.zeros((2,2))
ta = tf.zeros((2,2))
print a
print ta









    



[[ 0.  0.]
 [ 0.  0.]]
Tensor("zeros_1:0", shape=(2, 2), dtype=float32)

TensorFlow computations define a computation graph that has no numerical value until it is evaluated!



In [7]:

    
print ta.eval()









    



[[ 0.  0.]
 [ 0.  0.]]

Placeholders

TensorFlow provides a placeholder operation that must be fed with data on execution. It will be used to load input data to the model.

x = tf.placeholder(tf.float32, shape=(2, 2))
y = tf.matmul(x, x)

with tf.Session() as sess:
  print(sess.run(y))  # ERROR: will fail because x was not fed.

  rand_array = np.random.rand(2, 2) # we should get data from some training data
  print(sess.run(y, feed_dict={x: rand_array}))  # Will succeed.



In [8]:

    
x = tf.placeholder(tf.float32, shape=(2, 2))
y = tf.matmul(x, x)

with tf.Session() as sess:
    rand_array = np.random.rand(2, 2) # we should get data from some training data
    print(sess.run(y, feed_dict={x: rand_array}))









    



[[ 0.77418947  1.26398075]
 [ 0.84835112  1.51908946]]






    



Exception AssertionError: AssertionError() in <bound method InteractiveSession.__del__ of <tensorflow.python.client.session.InteractiveSession object at 0x7f2ed661be90>> ignored

Session

A Session object encapsulates the environment in which Operation objects are executed, and Tensor objects are evaluated. A session may own resources, such as variables, queues, and readers. It is important to release these resources when they are no longer required.

Three basic ways to work:

1) Using the Session object:

a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
sess = tf.Session()
print sess.run(c)
sess.close()

2) Using the context manager:

a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
with tf.Session() as sess:
  print(c.eval())

3) Using Interactive Session:

sess = tf.InteractiveSession()
a = tf.constant(5.0)
b = tf.constant(6.0)
c = a * b
#We can just use 'c.eval()' without passing 'sess'
print(c.eval())
sess.close()

Variables

When you train a model, you use variables to hold and update parameters. Variables are in-memory buffers containing tensors. A tensorFlow variable does not exist until you initialize it, so they must be explicitly initialized and can be saved to disk during and after training. You can later restore saved values to exercise or analyse the model.



In [9]:

    
W1 = tf.ones((2,2))
W2 = tf.Variable(tf.zeros((2,2)),name = "weights")

with tf.Session() as sess:
    print(sess.run(W1))
    sess.run(tf.initialize_all_variables())
    print(sess.run(W2))









    



[[ 1.  1.]
 [ 1.  1.]]
[[ 0.  0.]
 [ 0.  0.]]



In [10]:

    
W = tf.Variable(tf.zeros((2,2)),name = "weights")
R = tf.Variable(tf.random_normal((2,2)),name = "random_weights")

with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())
    print(sess.run(W))
    print(sess.run(R))









    



[[ 0.  0.]
 [ 0.  0.]]
[[ 0.66482162 -0.209846  ]
 [-0.13357474 -2.57937217]]

Example: Counter



In [11]:

    
state = tf.Variable(0, name = "counter")
new_value = tf.add(state, tf.constant(1))
update = tf.assign(state,new_value)

with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())
    print(sess.run(state))
    for _ in range(3):
        sess.run(update)
        print(sess.run(state))