Simple Deep Neural Net in TFLearn

for MNIST Digit Classfication

based on https://github.com/tflearn/tflearn/blob/master/examples/images/dnn.py



In [1]:

    
from __future__ import division, print_function, absolute_import



In [2]:

    
import tflearn

Data Loading and Preprocessing



In [3]:

    
import tflearn.datasets.mnist as mnist



In [4]:

    
X, Y, testX, testY = mnist.load_data(one_hot=True)









    



Downloading MNIST...
Succesfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting mnist/train-images-idx3-ubyte.gz






    



/Users/jon/anaconda/lib/python3.5/gzip.py:274: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future
  return self._buffer.read(size)
/Users/jon/anaconda/lib/python3.5/site-packages/tflearn/datasets/mnist.py:52: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future
  data = data.reshape(num_images, rows, cols, 1)






    



Downloading MNIST...
Succesfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting mnist/train-labels-idx1-ubyte.gz
Downloading MNIST...
Succesfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting mnist/t10k-images-idx3-ubyte.gz
Downloading MNIST...
Succesfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting mnist/t10k-labels-idx1-ubyte.gz

Building Deep Neural Network



In [5]:

    
input_layer = tflearn.input_data(shape=[None, 784])



In [6]:

    
dense1 = tflearn.fully_connected(input_layer, 64, activation='tanh', regularizer='L2', weight_decay=0.001)



In [7]:

    
dropout1 = tflearn.dropout(dense1, 0.8)



In [8]:

    
dense2 = tflearn.fully_connected(dropout1, 64, activation='tanh', regularizer='L2', weight_decay=0.001)



In [9]:

    
dropout2 = tflearn.dropout(dense2, 0.8)



In [10]:

    
softmax = tflearn.fully_connected(dropout2, 10, activation='softmax')

Regression using SGD with learning rate decay and Top-3 accuracy



In [11]:

    
sgd = tflearn.SGD(learning_rate=0.1, lr_decay=0.96, decay_step=1000)



In [12]:

    
top_k = tflearn.metrics.Top_k(3)



In [13]:

    
net = tflearn.regression(softmax, optimizer=sgd, metric=top_k, loss='categorical_crossentropy')

Training



In [14]:

    
model = tflearn.DNN(net, tensorboard_verbose=0)



In [16]:

    
model.fit(X, Y, n_epoch=10, validation_set=(testX, testY), show_metric=True, run_id='dense_model')









    



Training Step: 9460  | total loss: 0.18185
| SGD | epoch: 010 | loss: 0.18185 - top3: 0.9903 | val_loss: 0.13278 - val_acc: 0.9944 -- iter: 55000/55000
Training Step: 9460  | total loss: 0.18185
| SGD | epoch: 010 | loss: 0.18185 - top3: 0.9903 | val_loss: 0.13278 - val_acc: 0.9944 -- iter: 55000/55000
--



In [ ]: