Eager Execution Tutorial: Basics

This notebook introduces the basics of using TensorFlow's eager execution capabilities. It covers concepts such as:

Importing required packages
Enabling eager execution
Creating and using TensorFlow Tensors and Variables
Using TensorFlow interactively
Using GPUs with eager execution enabled

This notebook does not cover modeling topics, such as gradients.

Step 1: Import Eager

The key imports for eager execution are the following:



In [0]:

    
# Import TensorFlow.
import tensorflow as tf

# Import TensorFlow eager execution support (subject to future changes).
import tensorflow.contrib.eager as tfe

Step 2: Enable eager execution

All future TensorFlow calls will execute the underlying TensorFlow ops immediately:



In [0]:

    
tfe.enable_eager_execution()

Step 3: Interactively Use TensorFlow!

Now you can call TensorFlow functions and get results, immediately! No more tf.Sessions!

TensorFlow will automatically wrap native Python types for you with operator overloading for TensorFlow Tensors.



In [0]:

    
print(tf.add(1, 2))
print(tf.add([1, 2], [3, 4]))
print(tf.square(5))
print(tf.reduce_sum([1, 2, 3]))
print(tf.encode_base64("hello world"))
print("")

x = tf.constant(2)
y = tf.constant(3)
print(x * y + 1)

# Most TensorFlow ops are directly usable with eager execution, giving
# results immediately.
print(tf.contrib.signal.hamming_window(x * y + 1))

Numpy arrays are supported, too:



In [0]:

    
import numpy as np

ones = np.ones([3, 3])

print("numpy 3x3 matrix of 1s:")
print(ones)
print("")

print("Multiplied by 42:")
print(tf.multiply(ones, 42))

Step 4: Define and Print TensorFlow Variables

To define TensorFlow variables, use the get_variable() function as follows:



In [0]:

    
x = tf.get_variable(name="x", shape=[], dtype=tf.float32, initializer=tf.zeros_initializer)

Printing TensorFlow Variables



In [0]:

    
# This does NOT print the Variable's actual value:
print("Printing a TensorFlow Variable:")
print(x)
print("")

# A TensorFlow variable represents a reference to a tensor.
# The `read_value()` method provides access to the current value of the
# variable. Tensorflow Variables are automatically initialized according to the
# semantics defined in tf.get_variable().
print("Printing a TensorFlow Variable's value using .read_value():")
print(x.read_value())
print("")

print("Printing a TensorFlow Variable's value using .read_value().numpy():")
print(x.read_value().numpy())

Changing a TensorFlow Variable's value

To change a TensorFlow Variable's value, use its .assign() or .assign_add() method:



In [0]:

    
x.assign(42)
print(x.read_value())

x.assign_add(3)
print(x.read_value())

Use a Variable just like any other Tensor



In [0]:

    
print(x + 3)

# This code will broadcast the value across the list of numbers:
print(x * [1, 2, 4])

Step 5: Debug Errors with Instant Feedback

TensorFlow's eager execution helps you identify and debug runtime issues through interactive exploration of code snippets.

Below, we'll define a length-4 vector, and attempt two tf.slice() operations, one being legal and the other being illegal, leading to a runtime error that is raised immediately.



In [0]:

    
vector = tf.constant([10.0, 20.0, 30.0, 40.0])



In [0]:

    
# Works, because the values of `begin` and `size` (the 2nd and 3rd input
# arguments) are within the bound of `vector`.
print(tf.slice(vector, [1], [3]))



In [0]:

    
# The following does NOT work, because the value of `size` (the 3rd
# argument) causes the indices to go out of the bounds of `vector`. The
# error is raised immediately.
try:
  print(tf.slice(vector, [1], [4]))
except tf.OpError as e:
  print("Caught error: %s" % e)

Step 6: Using the GPU

You can place Tensors on the GPU by calling a Tensor's .gpu() method.

The first operation executing on the GPU may be slow as TensorFlow initializes. Subsequent uses will be much faster.



In [0]:

    
# The example code from here on will work only if your notebook
# is running on a machine with a functional CUDA GPU. The following
# line checks that.
is_gpu_available = tfe.num_gpus() > 0

# Create some Tensors
SIZE = 1000
cpu_tensor = tf.random_normal([SIZE, SIZE])

if is_gpu_available:
  gpu_tensor = cpu_tensor.gpu()



In [0]:

    
# Time a CPU-based matrix multiplication

print("Time to conduct matmul on CPU:")
%time tf.matmul(cpu_tensor, cpu_tensor)



In [0]:

    
# Time GPU-based matrix multiplications.

if is_gpu_available:
  # First use of the GPU will be slow:
  print("Time to conduct first matmul on GPU:")
  %time tf.matmul(gpu_tensor, gpu_tensor)
  print()

  # Subsequent uses are much faster:
  print("Time to conduct second matmul on GPU:")
  %time tf.matmul(gpu_tensor, gpu_tensor)



In [0]:

    
# Second timing demo for GPUs, after it has been used once:

cpu_tensor = tf.random_normal([SIZE, SIZE])
print("Time to conduct CPU matmul:")
%time tf.matmul(cpu_tensor, cpu_tensor)
print()

if is_gpu_available:
  gpu_tensor = cpu_tensor.gpu()
  print("Time to conduct GPU matmul:")
  %time tf.matmul(gpu_tensor, gpu_tensor)