notebook.community

Edit and run

DLW Practical 1: MNIST

From linear to non-linear models with MNIST

Introduction

In this practical we will experiment further with linear and non-linear models using the MNIST dataset. MNIST consists of images of handwritten digits that we want to classify correctly.

Learning objectives:

  • Implement a linear classifier on the MNIST image data set in Tensorflow.
  • Modify the code to to make the classifier non-linear by introducing a hidden non-linear layer.

What is expected of you:

  • Step through the code and make sure you understand each step. What test set accuracy do you get?
  • Modify the code to make the classifier non-linear by adding a non-linear activation function layer in Tensorflow. What accuracy do you get now?

Some parts of the code were adapted from the DL Indaba practicals.





In [3]:



    


import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow.examples.tutorials.mnist import input_data


def display_mnist_images(gens, num_images):
    plt.rcParams['image.interpolation'] = 'nearest'
    plt.rcParams['image.cmap'] = 'gray'
    fig, axs = plt.subplots(1, num_images, figsize=(25, 3))
    for i in range(num_images):
        reshaped_img = (gens[i].reshape(28, 28) * 255).astype(np.uint8)
        axs.flat[i].imshow(reshaped_img)
    plt.show()


# download MNIST dataset #
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# visualize random MNIST images #
batch_xs, batch_ys = mnist.train.next_batch(10)
list_of_images = np.split(batch_xs, 10)
display_mnist_images(list_of_images, 10)

x_dim, train_examples, n_classes = mnist.train.images.shape[1], mnist.train.num_examples, mnist.train.labels.shape[1]

######################################
# define the model (build the graph) #
######################################

x = tf.placeholder(tf.float32, [None, x_dim])
W = tf.Variable(tf.random_normal([x_dim, n_classes]))
b = tf.Variable(tf.ones([n_classes]))
y = tf.placeholder(tf.float32, [None, n_classes])
y_ = tf.add(tf.matmul(x, W), b)
prob = tf.nn.softmax(y_)

########################
# define loss function #
########################

cross_entropy_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=y_, labels=y))

learning_rate = 0.01

train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cross_entropy_loss)

###########################
# define model evaluation #
###########################

actual_class, predicted_class = tf.argmax(y, 1), tf.argmax(prob, 1)
correct_prediction = tf.cast(tf.equal(predicted_class, actual_class), tf.float32)
classification_accuracy = tf.reduce_mean(correct_prediction)

#########################
# define training cycle #
#########################

num_epochs = 50
batch_size = 20

# initializing the variables before starting the session #
init = tf.global_variables_initializer()

# launch the graph in a session (use the session as a context manager) #
with tf.Session() as sess:
    # run session #
    sess.run(init)
    # start main training cycle #
    for epoch in range(num_epochs):
        avg_cost = 0.
        avg_acc = 0.
        total_batch = int(mnist.train.num_examples / batch_size)
        # loop over all batches #
        for i in range(total_batch):
            batch_x, batch_y = mnist.train.next_batch(batch_size)
            # run optimization op (backprop), cost op and accuracy op (to get training losses) #
            _, c, a = sess.run([train_step, cross_entropy_loss, classification_accuracy], feed_dict={x: batch_x, y: batch_y})
            # compute avg training loss and avg training accuracy #
            avg_cost += c / total_batch
            avg_acc += a / total_batch
        # display logs per epoch step #
        if epoch % 1 == 0:
            print("Epoch {}: cross-entropy-loss = {:.4f}, training-accuracy = {:.3f}%".format(epoch + 1, avg_cost, avg_acc * 100))
    print("Optimization Finished!")
    # calculate test set accuracy #
    test_accuracy = classification_accuracy.eval({x: mnist.test.images, y: mnist.test.labels})
    print("Accuracy on test set = {:.3f}%".format(test_accuracy * 100))



    


















    






Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz










    

















    






Epoch 1: cross-entropy-loss = 4.0596, training-accuracy = 45.491%
Epoch 2: cross-entropy-loss = 1.5277, training-accuracy = 70.385%
Epoch 3: cross-entropy-loss = 1.1457, training-accuracy = 76.660%
Epoch 4: cross-entropy-loss = 0.9737, training-accuracy = 79.775%
Epoch 5: cross-entropy-loss = 0.8710, training-accuracy = 81.507%
Epoch 6: cross-entropy-loss = 0.8011, training-accuracy = 82.822%
Epoch 7: cross-entropy-loss = 0.7506, training-accuracy = 83.647%
Epoch 8: cross-entropy-loss = 0.7104, training-accuracy = 84.425%
Epoch 9: cross-entropy-loss = 0.6780, training-accuracy = 84.980%
Epoch 10: cross-entropy-loss = 0.6518, training-accuracy = 85.360%
Epoch 11: cross-entropy-loss = 0.6290, training-accuracy = 85.760%
Epoch 12: cross-entropy-loss = 0.6095, training-accuracy = 86.127%
Epoch 13: cross-entropy-loss = 0.5925, training-accuracy = 86.422%
Epoch 14: cross-entropy-loss = 0.5767, training-accuracy = 86.705%
Epoch 15: cross-entropy-loss = 0.5631, training-accuracy = 86.938%
Epoch 16: cross-entropy-loss = 0.5510, training-accuracy = 87.115%
Epoch 17: cross-entropy-loss = 0.5394, training-accuracy = 87.331%
Epoch 18: cross-entropy-loss = 0.5292, training-accuracy = 87.453%
Epoch 19: cross-entropy-loss = 0.5197, training-accuracy = 87.633%
Epoch 20: cross-entropy-loss = 0.5107, training-accuracy = 87.753%
Epoch 21: cross-entropy-loss = 0.5026, training-accuracy = 87.884%
Epoch 22: cross-entropy-loss = 0.4948, training-accuracy = 88.002%
Epoch 23: cross-entropy-loss = 0.4877, training-accuracy = 88.107%
Epoch 24: cross-entropy-loss = 0.4810, training-accuracy = 88.285%
Epoch 25: cross-entropy-loss = 0.4744, training-accuracy = 88.424%
Epoch 26: cross-entropy-loss = 0.4685, training-accuracy = 88.551%
Epoch 27: cross-entropy-loss = 0.4629, training-accuracy = 88.678%
Epoch 28: cross-entropy-loss = 0.4573, training-accuracy = 88.740%
Epoch 29: cross-entropy-loss = 0.4527, training-accuracy = 88.840%
Epoch 30: cross-entropy-loss = 0.4475, training-accuracy = 88.967%
Epoch 31: cross-entropy-loss = 0.4430, training-accuracy = 89.009%
Epoch 32: cross-entropy-loss = 0.4388, training-accuracy = 89.151%
Epoch 33: cross-entropy-loss = 0.4346, training-accuracy = 89.175%
Epoch 34: cross-entropy-loss = 0.4304, training-accuracy = 89.289%
Epoch 35: cross-entropy-loss = 0.4265, training-accuracy = 89.376%
Epoch 36: cross-entropy-loss = 0.4230, training-accuracy = 89.411%
Epoch 37: cross-entropy-loss = 0.4192, training-accuracy = 89.464%
Epoch 38: cross-entropy-loss = 0.4159, training-accuracy = 89.582%
Epoch 39: cross-entropy-loss = 0.4127, training-accuracy = 89.607%
Epoch 40: cross-entropy-loss = 0.4094, training-accuracy = 89.713%
Epoch 41: cross-entropy-loss = 0.4066, training-accuracy = 89.702%
Epoch 42: cross-entropy-loss = 0.4032, training-accuracy = 89.769%
Epoch 43: cross-entropy-loss = 0.4007, training-accuracy = 89.811%
Epoch 44: cross-entropy-loss = 0.3982, training-accuracy = 89.882%
Epoch 45: cross-entropy-loss = 0.3952, training-accuracy = 89.909%
Epoch 46: cross-entropy-loss = 0.3926, training-accuracy = 89.991%
Epoch 47: cross-entropy-loss = 0.3902, training-accuracy = 90.056%
Epoch 48: cross-entropy-loss = 0.3882, training-accuracy = 90.082%
Epoch 49: cross-entropy-loss = 0.3856, training-accuracy = 90.131%
Epoch 50: cross-entropy-loss = 0.3834, training-accuracy = 90.182%
Optimization Finished!
Accuracy on test set = 89.990%

















In [ ]:



    






    











In [ ]:

Content source: nlauchande/panha

Similar notebooks:

notebook.community | gallery | about