Neural Networks with One Hidden Layer

Table of Contents

In this lab, you will use a single layer neural network to classify handwritten digits from the MNIST database.


Import the following libraries:


In [ ]:
!conda install -y torchvision
import torch 
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as dsets
import torch.nn.functional as F
import matplotlib.pylab as plt
import numpy as np

Helper Functions

Use the following helper functions for plotting the loss:


In [ ]:
def plot_accuracy_loss(training_results): 
   
    plt.subplot(2, 1, 1)

    plt.plot(training_results['training_loss'],'r')
    plt.ylabel('loss')
    plt.title('training loss iterations')
    
    plt.subplot(2,1,2)
    plt.plot( training_results['validation_accuracy'])

    plt.ylabel('accuracy')
    plt.xlabel('epochs ')   
    plt.show()

Use the following function for printing the model parameters:


In [ ]:
def pint_model_parameters(model):
    for i in range(len(list(model.parameters()))):
        if (i+1)%2==0:
            print("the number of bias parameters for layer",i)
        else:
            print("the number of parameters for layer",i+1)
    
        print( list(model.parameters())[i].size() )

Define the neural network module or class:


In [ ]:
def show_data(data_sample):

    plt.imshow(data_sample.numpy().reshape(28,28),cmap='gray')
    plt.show()

Neural Network Module and Training Function

Define the neural network module or class:


In [ ]:
class Net(nn.Module):
    def __init__(self,D_in,H,D_out):
        super(Net,self).__init__()
        self.linear1=nn.Linear(D_in,H)
        self.linear2=nn.Linear(H,D_out)

        
    def forward(self,x):
        x=torch.sigmoid(self.linear1(x))  
        x=self.linear2(x)
        return x

Define a function to train the model. In this case, the function returns a Python dictionary to store the training loss and accuracy on the validation data.


In [ ]:
def train(model,criterion, train_loader,validation_loader, optimizer, epochs=100):
    i=0
    useful_stuff={'training_loss':[],'validation_accuracy':[]}  
    
    #n_epochs
    for epoch in range(epochs):
        for i,(x, y) in enumerate(train_loader):

            #clear gradient 
            optimizer.zero_grad()
            #make a prediction logits 
            z=model(x.view(-1,28*28))
            # calculate loss 
            loss=criterion(z,y)
    
            # calculate gradients of parameters 
            loss.backward()
            # update parameters 
            optimizer.step()
            useful_stuff['training_loss'].append(loss.data.item())
        correct=0
        for x, y in validation_loader:
            #perform a prediction on the validation  data  
            yhat=model(x.view(-1,28*28))
            
            _,lable=torch.max(yhat,1)
            correct+=(lable==y).sum().item()
 
    
        accuracy=100*(correct/len(validation_dataset))
   
        useful_stuff['validation_accuracy'].append(accuracy)
    
    return useful_stuff

Prepare Data

Load the training dataset by setting the parameters train to True and convert it to a tensor by placing a transform object in the argument transform.


In [ ]:
train_dataset=dsets.MNIST(root='./data', train=True, download=True, transform=transforms.ToTensor())

Load the testing dataset by setting the parameters train to False and convert it to a tensor by placing a transform object in the argument transform:


In [ ]:
validation_dataset=dsets.MNIST(root='./data', train=False, download=True, transform=transforms.ToTensor())

Create the criterion function:


In [ ]:
criterion=nn.CrossEntropyLoss()

Create the training-data loader and the validation-data loader objects:


In [ ]:
train_loader=torch.utils.data.DataLoader(dataset=train_dataset,batch_size=2000,shuffle=True)
validation_loader=torch.utils.data.DataLoader(dataset=validation_dataset,batch_size=5000,shuffle=False)

Define the Neural Network, Criterion Function, Optimizer, and Train the Model

Create the criterion function:


In [ ]:
criterion=nn.CrossEntropyLoss()

Create the model with 100 hidden layers:


In [ ]:
input_dim=28*28
hidden_dim=100
output_dim=10

model=Net(input_dim,hidden_dim,output_dim)

Print the model parameters:


In [ ]:
pint_model_parameters(model)

Define the optimizer object with a learning rate of 0.01:


In [ ]:
learning_rate=0.01
optimizer=torch.optim.SGD(model.parameters(),lr=learning_rate)

Train the model by using 100 epochs (this process takes time):


In [ ]:
training_results=train(model,criterion, train_loader,validation_loader, optimizer, epochs=30)

Analyze Results

Plot the training total loss or cost for every iteration and plot the training accuracy for every epoch:


In [ ]:
plot_accuracy_loss(training_results)

Plot the first five misclassified samples:


In [ ]:
count=0
for x,y in validation_dataset:

    z=model(x.reshape(-1,28*28))
    _,yhat=torch.max(z,1)
    if yhat!=y:
        show_data(x)
        count+=1
    if count>=5:
        break

Practice Question

Use nn.Sequential to build exactly the same model as you just built. Use the function train to train the model and use the function plot_accuracy_loss to see the metrics. Also, try different epoch numbers.


In [ ]:

Double-click here for the solution.

About the Authors:

Joseph Santarcangelo has a PhD in Electrical Engineering. His research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition.

Other contributors: Michelle Carey ,Mavis Zhou


Copyright © 2018 cognitiveclass.ai. This notebook and its source code are released under the terms of the MIT License.