Express Deep Learning in Python - Part 3

Tensorboard and other cool stuff

We have seen so far how to build our networks and how to train them, but the hard part of Deep Learning radicates many times in finding the best architecture and hyperparameters for our task. Sometimes that is more an art than a science, because neural classifiers are opaque: their parameters don't have a clear interpretation. If we can't understand why the neural network is behaving in such a way, how can we improve the results?

We want to show you, as the last part of the tutorial, Tensorboard. This is a tool that will allow you to run experiments and visualize results in an easy and (hopefully) faster way.

Tensorboard has being designed to work with TensoFlow variables directly, but we can still use some basic functionalities from Keras. Let's see a quick example and what type of things it can do.



In [3]:

    
import keras
import numpy

from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.datasets import mnist

from keras.callbacks import TensorBoard

First we will load our dataset and compile our model just as before.



In [4]:

    
batch_size = 128
num_classes = 10
TRAIN_EXAMPLES = 60000
TEST_EXAMPLES = 10000

# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()

# reshape the dataset to convert the examples from 2D matrixes to 1D arrays.
x_train = x_train.reshape(60000, 28*28)
x_test = x_test.reshape(10000, 28*28)

# to make quick runs, select a smaller set of images.
train_mask = numpy.random.choice(x_train.shape[0], TRAIN_EXAMPLES, replace=False)
x_train = x_train[train_mask, :].astype('float32')
y_train = y_train[train_mask]
test_mask = numpy.random.choice(x_test.shape[0], TEST_EXAMPLES, replace=False)
x_test = x_test[test_mask, :].astype('float32')
y_test = y_test[test_mask]

# normalize the input
x_train /= 255
x_test /= 255

# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)



In [5]:

    
model = Sequential()
model.add(Dense(512, activation='relu', input_shape=(784,)))
model.add(Dense(10, activation='softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer=keras.optimizers.Adam(),
              metrics=['accuracy', 'mae'])

Now we will create a Callback to store the results in the required format so Tensorboard can graphic them later. This is the only modification we need in order to use Tensorboard. By default, the callback is going to store the loss and metric values, called scalars. You can also log the graph structure of your model and the gradients used during the optimization. However, take into account that the more things you log, the slower your classifier will be.

Every time we train the model we need to use a different folder so the results won't collapse with eachother, it's useful to create a counter to identify the runs.



In [6]:

    
EXPERIMENT_COUNTER = 1



In [10]:

    
tensorboard = TensorBoard(log_dir='./logs/experiment-{}'.format(EXPERIMENT_COUNTER), histogram_freq=0,
                          write_graph=False, write_images=False)
epochs = 10
history = model.fit(x_train, y_train,
                    batch_size=batch_size,
                    epochs=epochs,
                    verbose=0,
                    validation_data=(x_test, y_test),
                    callbacks=[tensorboard])
EXPERIMENT_COUNTER += 1

Once the model is trained, you need to open a console and run

$ tensorboard --logdir=path/to/log-directory

In this case, the path/to/log-directory is the path to the folder logs that you set when defininf the callback. If the command does not fail, go to http://localhost:6006/ in your browser and you should see the main screen of Tensorboard. If you run this code multiple times, new experiment folders will be created and more results will be available to compare.



In [ ]: