notebook.community

Edit and run

Image Classification

In this project, you'll classify images from the CIFAR-10 dataset. The dataset consists of airplanes, dogs, cats, and other objects. You'll preprocess the images, then train a convolutional neural network on all the samples. The images need to be normalized and the labels need to be one-hot encoded. You'll get to apply what you learned and build a convolutional, max pooling, dropout, and fully connected layers. At the end, you'll get to see your neural network's predictions on the sample images.

Get the Data

Run the following cell to download the CIFAR-10 dataset for python.





In [1]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
from urllib.request import urlretrieve
from os.path import isfile, isdir
from tqdm import tqdm
import problem_unittests as tests
import tarfile

cifar10_dataset_folder_path = 'cifar-10-batches-py'

# Use Floyd's cifar-10 dataset if present
floyd_cifar10_location = '/input/cifar-10/python.tar.gz'
if isfile(floyd_cifar10_location):
    tar_gz_path = floyd_cifar10_location
else:
    tar_gz_path = 'cifar-10-python.tar.gz'

class DLProgress(tqdm):
    last_block = 0

    def hook(self, block_num=1, block_size=1, total_size=None):
        self.total = total_size
        self.update((block_num - self.last_block) * block_size)
        self.last_block = block_num

if not isfile(tar_gz_path):
    with DLProgress(unit='B', unit_scale=True, miniters=1, desc='CIFAR-10 Dataset') as pbar:
        urlretrieve(
            'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz',
            tar_gz_path,
            pbar.hook)

if not isdir(cifar10_dataset_folder_path):
    with tarfile.open(tar_gz_path) as tar:
        tar.extractall()
        tar.close()


tests.test_folder_path(cifar10_dataset_folder_path)



    


















    






All files found!

Explore the Data

The dataset is broken into batches to prevent your machine from running out of memory. The CIFAR-10 dataset consists of 5 batches, named data_batch_1, data_batch_2, etc.. Each batch contains the labels and images that are one of the following:

  • airplane
  • automobile
  • bird
  • cat
  • deer
  • dog
  • frog
  • horse
  • ship
  • truck

Understanding a dataset is part of making predictions on the data. Play around with the code cell below by changing the batch_id and sample_id. The batch_id is the id for a batch (1-5). The sample_id is the id for a image and label pair in the batch.

Ask yourself "What are all possible labels?", "What is the range of values for the image data?", "Are the labels in order or random?". Answers to questions like these will help you preprocess the data and end up with better predictions.





In [2]:



    


%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import helper
import numpy as np

# Explore the dataset
batch_id = 4
sample_id = 2
helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)



    


















    






/usr/local/lib/python3.5/site-packages/matplotlib/font_manager.py:280: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  'Matplotlib is building the font cache using fc-list. '










    






Stats of batch 4:
Samples: 10000
Label Counts: {0: 1003, 1: 963, 2: 1041, 3: 976, 4: 1004, 5: 1021, 6: 1004, 7: 981, 8: 1024, 9: 983}
First 20 Labels: [0, 6, 0, 2, 7, 2, 1, 2, 4, 1, 5, 6, 6, 3, 1, 3, 5, 5, 8, 1]

Example of Image 2:
Image - Min Value: 2 Max Value: 204
Image - Shape: (32, 32, 3)
Label - Label Id: 0 Name: airplane

Implement Preprocess Functions

Normalize

In the cell below, implement the normalize function to take in image data, x, and return it as a normalized Numpy array. The values should be in the range of 0 to 1, inclusive. The return object should be the same shape as x.





In [3]:



    


def normalize(x):
    """
    Normalize a list of sample image data in the range of 0 to 1
    : x: List of image data.  The image shape is (32, 32, 3)
    : return: Numpy array of normalize data
    """
    # TODO: Implement Function
    # return None
    a = 0
    b = 255
    return (x-a)/(b-a)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_normalize(normalize)



    


















    






Tests Passed

One-hot encode

Just like the previous code cell, you'll be implementing a function for preprocessing. This time, you'll implement the one_hot_encode function. The input, x, are a list of labels. Implement the function to return the list of labels as One-Hot encoded Numpy array. The possible values for labels are 0 to 9. The one-hot encoding function should return the same encoding for each value between each call to one_hot_encode. Make sure to save the map of encodings outside the function.

Hint: Don't reinvent the wheel.





In [4]:



    


def one_hot_encode(x):
    """
    One hot encode a list of sample labels. Return a one-hot encoded vector for each label.
    : x: List of sample Labels
    : return: Numpy array of one-hot encoded labels
    """
    # TODO: Implement Function
    # return None
    y = np.zeros((len(x), 10))
    for i in range(len(x)):
        y[i,x[i]] = 1
    return y


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_one_hot_encode(one_hot_encode)



    


















    






Tests Passed

Randomize Data

As you saw from exploring the data above, the order of the samples are randomized. It doesn't hurt to randomize it again, but you don't need to for this dataset.

Preprocess all the data and save it

Running the code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation.





In [5]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)

Check Point

This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.





In [6]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle
import problem_unittests as tests
import helper

# Load the Preprocessed Validation data
valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))

Build the network

For the neural network, you'll build each layer into a function. Most of the code you've seen has been outside of functions. To test your code more thoroughly, we require that you put each layer in a function. This allows us to give you better feedback and test for simple mistakes using our unittests before you submit your project.

Note: If you're finding it hard to dedicate enough time for this course each week, we've provided a small shortcut to this part of the project. In the next couple of problems, you'll have the option to use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages to build each layer, except the layers you build in the "Convolutional and Max Pooling Layer" section. TF Layers is similar to Keras's and TFLearn's abstraction to layers, so it's easy to pickup.

However, if you would like to get the most out of this course, try to solve all the problems without using anything from the TF Layers packages. You can still use classes from other packages that happen to have the same name as ones you find in TF Layers! For example, instead of using the TF Layers version of the conv2d class, tf.layers.conv2d, you would want to use the TF Neural Network version of conv2d, tf.nn.conv2d.

Let's begin!

Input

The neural network needs to read the image data, one-hot encoded labels, and dropout keep probability. Implement the following functions

  • Implement neural_net_image_input
    • Return a TF Placeholder
    • Set the shape using image_shape with batch size set to None.
    • Name the TensorFlow placeholder "x" using the TensorFlow name parameter in the TF Placeholder.
  • Implement neural_net_label_input
    • Return a TF Placeholder
    • Set the shape using n_classes with batch size set to None.
    • Name the TensorFlow placeholder "y" using the TensorFlow name parameter in the TF Placeholder.
  • Implement neural_net_keep_prob_input
    • Return a TF Placeholder for dropout keep probability.
    • Name the TensorFlow placeholder "keep_prob" using the TensorFlow name parameter in the TF Placeholder.

These names will be used at the end of the project to load your saved model.

Note: None for shapes in TensorFlow allow for a dynamic size.





In [7]:



    


import tensorflow as tf

# Network Parameters
#n_input = 3072 # MNIST data input (img shape: 28*28)
#n_classes = 10 # MNIST total classes (0-9 digits)
dropout = 0.75 # Dropout, probability to keep units

def neural_net_image_input(image_shape):
    """
    Return a Tensor for a bach of image input
    : image_shape: Shape of the images
    : return: Tensor for image input.
    """
    # TODO: Implement Function
    # return None
    #print(image_shape)    
    # return tf.placeholder(tf.float32, shape=(32, 32, 3))
    return tf.placeholder(tf.float32, shape=[None, image_shape[0], image_shape[1], image_shape[2]], name='x')

def neural_net_label_input(n_classes):
    """
    Return a Tensor for a batch of label input
    : n_classes: Number of classes
    : return: Tensor for label input.
    """
    # TODO: Implement Function
    #print(n_classes)
    return tf.placeholder(tf.float32, [None, n_classes], name='y')


def neural_net_keep_prob_input():
    """
    Return a Tensor for keep probability
    : return: Tensor for keep probability.
    """
    # TODO: Implement Function
    return tf.placeholder(tf.float32, None, name='keep_prob')


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tf.reset_default_graph()
tests.test_nn_image_inputs(neural_net_image_input)
tests.test_nn_label_inputs(neural_net_label_input)
tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)



    


















    






Image Input Tests Passed.
Label Input Tests Passed.
Keep Prob Tests Passed.

Convolution and Max Pooling Layer

Convolution layers have a lot of success with images. For this code cell, you should implement the function conv2d_maxpool to apply convolution then max pooling:

  • Create the weight and bias using conv_ksize, conv_num_outputs and the shape of x_tensor.
  • Apply a convolution to x_tensor using weight and conv_strides.
    • We recommend you use same padding, but you're welcome to use any padding.
  • Add bias
  • Add a nonlinear activation to the convolution.
  • Apply Max Pooling using pool_ksize and pool_strides.
    • We recommend you use same padding, but you're welcome to use any padding.

Note: You can't use TensorFlow Layers or TensorFlow Layers (contrib) for this layer, but you can still use TensorFlow's Neural Network package. You may still use the shortcut option for all the other layers.





In [23]:



    


from math import sqrt
def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
    """
    Apply convolution then max pooling to x_tensor
    :param x_tensor: TensorFlow Tensor
    :param conv_num_outputs: Number of outputs for the convolutional layer
    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer
    :param conv_strides: Stride 2-D Tuple for convolution
    :param pool_ksize: kernal size 2-D Tuple for pool
    :param pool_strides: Stride 2-D Tuple for pool
    : return: A tensor that represents convolution and max pooling of x_tensor
    """
    # TODO: Implement Function
    #print(x_tensor)
    input_channel_depth = int(x_tensor.get_shape()[3])
    # The shape of the filter weight is (height, width, input_depth, output_depth)    
    filter_weights = sqrt(2.0/conv_num_outputs) * tf.Variable(tf.truncated_normal([*conv_ksize, input_channel_depth, conv_num_outputs], dtype=tf.float32))
    print(filter_weights)
    # The shape of the biases is equal the the number of outputs of the conv layer
    filter_biases = tf.Variable(tf.constant(0, shape=[conv_num_outputs], dtype=tf.float32))
        
    layer = tf.nn.conv2d(input=x_tensor, filter=filter_weights, strides=[1, *conv_strides, 1], padding='SAME')
    layer += filter_biases
    layer = tf.nn.max_pool(layer, [1, *pool_ksize, 1], strides=[1, *pool_strides, 1], padding='SAME')
    layer = tf.nn.relu(layer)
    return layer 
    


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_con_pool(conv2d_maxpool)



    


















    






Tensor("mul_2:0", shape=(2, 2, 5, 10), dtype=float32)
Tests Passed

Flatten Layer

Implement the flatten function to change the dimension of x_tensor from a 4-D tensor to a 2-D tensor. The output should be the shape (Batch Size, Flattened Image Size). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.





In [24]:



    


from numpy import prod
def flatten(x_tensor):
    """
    Flatten x_tensor to (Batch Size, Flattened Image Size)
    : x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.
    : return: A tensor of size (Batch Size, Flattened Image Size).
    """
    # TODO: Implement Function
    batch_size = x_tensor.get_shape()[0]
    #flattened_image_size = x_tensor.get_shape()[1]*x_tensor.get_shape()[2]*x_tensor.get_shape()[3]
    #print(flattened_image_size)
    #return tf.contrib.layers.flatten(x_tensor,[batch_size,flattened_image_size])
    #return tf.reshape(x_tensor, [batch_size,flattened_image_size])
    dimension = x_tensor.get_shape().as_list()    
    #print(prod(dimension[1:]))
    return tf.reshape(x_tensor,[-1,prod(dimension[1:])])
   

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_flatten(flatten)



    


















    






Tests Passed

Fully-Connected Layer

Implement the fully_conn function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.





In [25]:



    


def fully_conn(x_tensor, num_outputs):
    """
    Apply a fully connected layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    # TODO: Implement Function
    # return None
    #print(x_tensor)
    #print(num_outputs)
    shape = x_tensor.get_shape().as_list()[1:]
    shape.append(num_outputs)
    #print(shape)
    weight = tf.Variable(tf.truncated_normal(shape,0,0.1))
    bias = tf.Variable(tf.zeros(num_outputs))
    return tf.nn.relu(tf.add(tf.matmul(x_tensor,weight), bias))


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_fully_conn(fully_conn)



    


















    






Tests Passed

Output Layer

Implement the output function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.

Note: Activation, softmax, or cross entropy should not be applied to this.





In [26]:



    


def output(x_tensor, num_outputs):
    """
    Apply a output layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    # TODO: Implement Function
    #return None
    shape = x_tensor.get_shape().as_list()[1:]
    shape.append(num_outputs)
    #print(shape)
    weight = tf.Variable(tf.truncated_normal(shape,0,0.01))
    bias = tf.Variable(tf.zeros(num_outputs))
    return tf.add(tf.matmul(x_tensor,weight), bias)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_output(output)



    


















    






Tests Passed

Create Convolutional Model

Implement the function conv_net to create a convolutional neural network model. The function takes in a batch of images, x, and outputs logits. Use the layers you created above to create this model:

  • Apply 1, 2, or 3 Convolution and Max Pool layers
  • Apply a Flatten Layer
  • Apply 1, 2, or 3 Fully Connected Layers
  • Apply an Output Layer
  • Return the output
  • Apply TensorFlow's Dropout to one or more layers in the model using keep_prob. 




In [27]:



    


def conv_net(x, keep_prob):
    """
    Create a convolutional neural network model
    : x: Placeholder tensor that holds image data.
    : keep_prob: Placeholder tensor that hold dropout keep probability.
    : return: Tensor that represents logits
    """
    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
    #    Play around with different number of outputs, kernel size and stride
    # Function Definition from Above:
    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    #print(x)
    x_tensor = x
    conv_num_outputs = 10
    conv_ksize = (4, 4)
    conv_strides = (1, 1)
    pool_ksize = (8, 8)
    pool_strides = (1, 1)
    
    model = conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    
    # Apply some dropout
    model = tf.nn.dropout(model, keep_prob)
    

    # TODO: Apply a Flatten Layer
    # Function Definition from Above:
    model = flatten(model)
    

    # TODO: Apply 1, 2, or 3 Fully Connected Layers
    #    Play around with different number of outputs
    # Function Definition from Above:
    num_outputs = 10 
    model = fully_conn(model, num_outputs)
    
    
    # TODO: Apply an Output Layer
    #    Set this to the number of classes
    # Function Definition from Above:
    model = output(model, num_outputs)
    
    
    # TODO: return output
    return model


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""

##############################
## Build the Neural Network ##
##############################

# Remove previous weights, bias, inputs, etc..
tf.reset_default_graph()

# Inputs
x = neural_net_image_input((32, 32, 3))
y = neural_net_label_input(10)
keep_prob = neural_net_keep_prob_input()

# Model
logits = conv_net(x, keep_prob)

# Name logits Tensor, so that is can be loaded from disk after training
logits = tf.identity(logits, name='logits')

# Loss and Optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))
optimizer = tf.train.AdamOptimizer().minimize(cost)

# Accuracy
correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')

tests.test_conv_net(conv_net)



    


















    






Tensor("mul:0", shape=(4, 4, 3, 10), dtype=float32)
Tensor("mul_1:0", shape=(4, 4, 3, 10), dtype=float32)
Neural Network Built!

Train the Neural Network

Single Optimization

Implement the function train_neural_network to do a single optimization. The optimization should use optimizer to optimize in session with a feed_dict of the following:

  • x for image input
  • y for labels
  • keep_prob for keep probability for dropout

This function will be called for each batch, so tf.global_variables_initializer() has already been called.

Note: Nothing needs to be returned. This function is only optimizing the neural network.





In [28]:



    


def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):
    """
    Optimize the session on a batch of images and labels
    : session: Current TensorFlow session
    : optimizer: TensorFlow optimizer function
    : keep_probability: keep probability
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    """
    # TODO: Implement Function
    session.run(optimizer, feed_dict={x:feature_batch, y:label_batch, keep_prob:keep_probability})



"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_train_nn(train_neural_network)



    


















    






Tests Passed

Show Stats

Implement the function print_stats to print loss and validation accuracy. Use the global variables valid_features and valid_labels to calculate validation accuracy. Use a keep probability of 1.0 to calculate the loss and validation accuracy.





In [29]:



    


def print_stats(session, feature_batch, label_batch, cost, accuracy):
    """
    Print information about loss and validation accuracy
    : session: Current TensorFlow session
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    : cost: TensorFlow cost function
    : accuracy: TensorFlow accuracy function
    """
    # TODO: Implement Function
    #pass
    # Loss and Optimizer
    #cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))
    #optimizer = tf.train.AdamOptimizer().minimize(cost)

    # Accuracy
    #correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))
    #accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')
    
    #print(cost, accuracy)
    loss = session.run(cost, feed_dict={x:feature_batch, y:label_batch, keep_prob:1.0})
    valid_acc = sess.run(accuracy, feed_dict={
                x: valid_features,
                y: valid_labels,
                keep_prob: 1.})
    print('Loss: {:>10.4f} Validation Accuracy: {:.6f}'.format(
                loss,
                valid_acc))

Hyperparameters

Tune the following parameters:

  • Set epochs to the number of iterations until the network stops learning or start overfitting
  • Set batch_size to the highest number that your machine has memory for. Most people set them to common sizes of memory:
    • 64
    • 128
    • 256
    • ...
  • Set keep_probability to the probability of keeping a node using dropout




In [30]:



    


# TODO: Tune Parameters
epochs = 200
batch_size = 256
keep_probability = 0.60

Train on a Single CIFAR-10 Batch

Instead of training the neural network on all the CIFAR-10 batches of data, let's use a single batch. This should save time while you iterate on the model to get a better accuracy. Once the final validation accuracy is 50% or greater, run the model on all the data in the next section.





In [31]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
print('Checking the Training on a Single Batch...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())
    
    # Training cycle
    for epoch in range(epochs):
        batch_i = 1
        for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
        print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
        print_stats(sess, batch_features, batch_labels, cost, accuracy)



    


















    






Checking the Training on a Single Batch...
Epoch  1, CIFAR-10 Batch 1:  Loss:     2.1929 Validation Accuracy: 0.203800
Epoch  2, CIFAR-10 Batch 1:  Loss:     2.1779 Validation Accuracy: 0.246600
Epoch  3, CIFAR-10 Batch 1:  Loss:     2.1045 Validation Accuracy: 0.279000
Epoch  4, CIFAR-10 Batch 1:  Loss:     2.0540 Validation Accuracy: 0.310800
Epoch  5, CIFAR-10 Batch 1:  Loss:     2.0094 Validation Accuracy: 0.324200
Epoch  6, CIFAR-10 Batch 1:  Loss:     1.9694 Validation Accuracy: 0.329000
Epoch  7, CIFAR-10 Batch 1:  Loss:     1.9365 Validation Accuracy: 0.341400
Epoch  8, CIFAR-10 Batch 1:  Loss:     1.9057 Validation Accuracy: 0.347600
Epoch  9, CIFAR-10 Batch 1:  Loss:     1.8204 Validation Accuracy: 0.354200
Epoch 10, CIFAR-10 Batch 1:  Loss:     1.7507 Validation Accuracy: 0.366600
Epoch 11, CIFAR-10 Batch 1:  Loss:     1.6964 Validation Accuracy: 0.377200
Epoch 12, CIFAR-10 Batch 1:  Loss:     1.6585 Validation Accuracy: 0.386200
Epoch 13, CIFAR-10 Batch 1:  Loss:     1.6222 Validation Accuracy: 0.386600
Epoch 14, CIFAR-10 Batch 1:  Loss:     1.5841 Validation Accuracy: 0.391400
Epoch 15, CIFAR-10 Batch 1:  Loss:     1.5577 Validation Accuracy: 0.400800
Epoch 16, CIFAR-10 Batch 1:  Loss:     1.5340 Validation Accuracy: 0.404800
Epoch 17, CIFAR-10 Batch 1:  Loss:     1.4976 Validation Accuracy: 0.405000
Epoch 18, CIFAR-10 Batch 1:  Loss:     1.4824 Validation Accuracy: 0.415200
Epoch 19, CIFAR-10 Batch 1:  Loss:     1.4566 Validation Accuracy: 0.418000
Epoch 20, CIFAR-10 Batch 1:  Loss:     1.4318 Validation Accuracy: 0.421400
Epoch 21, CIFAR-10 Batch 1:  Loss:     1.4223 Validation Accuracy: 0.421800
Epoch 22, CIFAR-10 Batch 1:  Loss:     1.3979 Validation Accuracy: 0.425400
Epoch 23, CIFAR-10 Batch 1:  Loss:     1.3808 Validation Accuracy: 0.428200
Epoch 24, CIFAR-10 Batch 1:  Loss:     1.3575 Validation Accuracy: 0.424600
Epoch 25, CIFAR-10 Batch 1:  Loss:     1.3275 Validation Accuracy: 0.427800
Epoch 26, CIFAR-10 Batch 1:  Loss:     1.3134 Validation Accuracy: 0.430000
Epoch 27, CIFAR-10 Batch 1:  Loss:     1.3103 Validation Accuracy: 0.432000
Epoch 28, CIFAR-10 Batch 1:  Loss:     1.3021 Validation Accuracy: 0.438000
Epoch 29, CIFAR-10 Batch 1:  Loss:     1.2868 Validation Accuracy: 0.438000
Epoch 30, CIFAR-10 Batch 1:  Loss:     1.2779 Validation Accuracy: 0.440800
Epoch 31, CIFAR-10 Batch 1:  Loss:     1.2635 Validation Accuracy: 0.441400
Epoch 32, CIFAR-10 Batch 1:  Loss:     1.2557 Validation Accuracy: 0.444800
Epoch 33, CIFAR-10 Batch 1:  Loss:     1.2391 Validation Accuracy: 0.445400
Epoch 34, CIFAR-10 Batch 1:  Loss:     1.2236 Validation Accuracy: 0.449600
Epoch 35, CIFAR-10 Batch 1:  Loss:     1.2180 Validation Accuracy: 0.447800
Epoch 36, CIFAR-10 Batch 1:  Loss:     1.2112 Validation Accuracy: 0.448600
Epoch 37, CIFAR-10 Batch 1:  Loss:     1.1996 Validation Accuracy: 0.451200
Epoch 38, CIFAR-10 Batch 1:  Loss:     1.1849 Validation Accuracy: 0.455000
Epoch 39, CIFAR-10 Batch 1:  Loss:     1.1696 Validation Accuracy: 0.457200
Epoch 40, CIFAR-10 Batch 1:  Loss:     1.1627 Validation Accuracy: 0.458000
Epoch 41, CIFAR-10 Batch 1:  Loss:     1.1487 Validation Accuracy: 0.459800
Epoch 42, CIFAR-10 Batch 1:  Loss:     1.1410 Validation Accuracy: 0.462200
Epoch 43, CIFAR-10 Batch 1:  Loss:     1.1328 Validation Accuracy: 0.460600
Epoch 44, CIFAR-10 Batch 1:  Loss:     1.1323 Validation Accuracy: 0.465600
Epoch 45, CIFAR-10 Batch 1:  Loss:     1.1120 Validation Accuracy: 0.467600
Epoch 46, CIFAR-10 Batch 1:  Loss:     1.1099 Validation Accuracy: 0.468400
Epoch 47, CIFAR-10 Batch 1:  Loss:     1.1013 Validation Accuracy: 0.469200
Epoch 48, CIFAR-10 Batch 1:  Loss:     1.0906 Validation Accuracy: 0.468600
Epoch 49, CIFAR-10 Batch 1:  Loss:     1.0797 Validation Accuracy: 0.467000
Epoch 50, CIFAR-10 Batch 1:  Loss:     1.0733 Validation Accuracy: 0.471800
Epoch 51, CIFAR-10 Batch 1:  Loss:     1.0520 Validation Accuracy: 0.475000
Epoch 52, CIFAR-10 Batch 1:  Loss:     1.0437 Validation Accuracy: 0.473800
Epoch 53, CIFAR-10 Batch 1:  Loss:     1.0081 Validation Accuracy: 0.475000
Epoch 54, CIFAR-10 Batch 1:  Loss:     0.9899 Validation Accuracy: 0.472000
Epoch 55, CIFAR-10 Batch 1:  Loss:     0.9841 Validation Accuracy: 0.475800
Epoch 56, CIFAR-10 Batch 1:  Loss:     0.9562 Validation Accuracy: 0.481600
Epoch 57, CIFAR-10 Batch 1:  Loss:     0.9278 Validation Accuracy: 0.489600
Epoch 58, CIFAR-10 Batch 1:  Loss:     0.9067 Validation Accuracy: 0.488400
Epoch 59, CIFAR-10 Batch 1:  Loss:     0.8968 Validation Accuracy: 0.493800
Epoch 60, CIFAR-10 Batch 1:  Loss:     0.8845 Validation Accuracy: 0.493800
Epoch 61, CIFAR-10 Batch 1:  Loss:     0.8795 Validation Accuracy: 0.497800
Epoch 62, CIFAR-10 Batch 1:  Loss:     0.8779 Validation Accuracy: 0.494400
Epoch 63, CIFAR-10 Batch 1:  Loss:     0.8774 Validation Accuracy: 0.498200
Epoch 64, CIFAR-10 Batch 1:  Loss:     0.8598 Validation Accuracy: 0.497800
Epoch 65, CIFAR-10 Batch 1:  Loss:     0.8489 Validation Accuracy: 0.498800
Epoch 66, CIFAR-10 Batch 1:  Loss:     0.8434 Validation Accuracy: 0.499400
Epoch 67, CIFAR-10 Batch 1:  Loss:     0.8344 Validation Accuracy: 0.499400
Epoch 68, CIFAR-10 Batch 1:  Loss:     0.8217 Validation Accuracy: 0.500200
Epoch 69, CIFAR-10 Batch 1:  Loss:     0.8169 Validation Accuracy: 0.495200
Epoch 70, CIFAR-10 Batch 1:  Loss:     0.8182 Validation Accuracy: 0.501600
Epoch 71, CIFAR-10 Batch 1:  Loss:     0.8125 Validation Accuracy: 0.497000
Epoch 72, CIFAR-10 Batch 1:  Loss:     0.8146 Validation Accuracy: 0.500800
Epoch 73, CIFAR-10 Batch 1:  Loss:     0.7955 Validation Accuracy: 0.501400
Epoch 74, CIFAR-10 Batch 1:  Loss:     0.8071 Validation Accuracy: 0.498000
Epoch 75, CIFAR-10 Batch 1:  Loss:     0.7849 Validation Accuracy: 0.503000
Epoch 76, CIFAR-10 Batch 1:  Loss:     0.7832 Validation Accuracy: 0.505400
Epoch 77, CIFAR-10 Batch 1:  Loss:     0.7708 Validation Accuracy: 0.500400
Epoch 78, CIFAR-10 Batch 1:  Loss:     0.7670 Validation Accuracy: 0.503400
Epoch 79, CIFAR-10 Batch 1:  Loss:     0.7669 Validation Accuracy: 0.502000
Epoch 80, CIFAR-10 Batch 1:  Loss:     0.7617 Validation Accuracy: 0.500800
Epoch 81, CIFAR-10 Batch 1:  Loss:     0.7630 Validation Accuracy: 0.505000
Epoch 82, CIFAR-10 Batch 1:  Loss:     0.7592 Validation Accuracy: 0.504800
Epoch 83, CIFAR-10 Batch 1:  Loss:     0.7575 Validation Accuracy: 0.499800
Epoch 84, CIFAR-10 Batch 1:  Loss:     0.7504 Validation Accuracy: 0.503000
Epoch 85, CIFAR-10 Batch 1:  Loss:     0.7465 Validation Accuracy: 0.506400
Epoch 86, CIFAR-10 Batch 1:  Loss:     0.7391 Validation Accuracy: 0.504000
Epoch 87, CIFAR-10 Batch 1:  Loss:     0.7379 Validation Accuracy: 0.503400
Epoch 88, CIFAR-10 Batch 1:  Loss:     0.7341 Validation Accuracy: 0.503000
Epoch 89, CIFAR-10 Batch 1:  Loss:     0.7341 Validation Accuracy: 0.511800
Epoch 90, CIFAR-10 Batch 1:  Loss:     0.7364 Validation Accuracy: 0.503200
Epoch 91, CIFAR-10 Batch 1:  Loss:     0.7245 Validation Accuracy: 0.503200
Epoch 92, CIFAR-10 Batch 1:  Loss:     0.7186 Validation Accuracy: 0.502200
Epoch 93, CIFAR-10 Batch 1:  Loss:     0.7273 Validation Accuracy: 0.505600
Epoch 94, CIFAR-10 Batch 1:  Loss:     0.7107 Validation Accuracy: 0.505400
Epoch 95, CIFAR-10 Batch 1:  Loss:     0.7092 Validation Accuracy: 0.509200
Epoch 96, CIFAR-10 Batch 1:  Loss:     0.7109 Validation Accuracy: 0.505400
Epoch 97, CIFAR-10 Batch 1:  Loss:     0.7009 Validation Accuracy: 0.504600
Epoch 98, CIFAR-10 Batch 1:  Loss:     0.6916 Validation Accuracy: 0.503800
Epoch 99, CIFAR-10 Batch 1:  Loss:     0.6943 Validation Accuracy: 0.503800
Epoch 100, CIFAR-10 Batch 1:  Loss:     0.6953 Validation Accuracy: 0.506200
Epoch 101, CIFAR-10 Batch 1:  Loss:     0.6993 Validation Accuracy: 0.501000
Epoch 102, CIFAR-10 Batch 1:  Loss:     0.6888 Validation Accuracy: 0.504200
Epoch 103, CIFAR-10 Batch 1:  Loss:     0.6904 Validation Accuracy: 0.500200
Epoch 104, CIFAR-10 Batch 1:  Loss:     0.6868 Validation Accuracy: 0.504400
Epoch 105, CIFAR-10 Batch 1:  Loss:     0.6901 Validation Accuracy: 0.501600
Epoch 106, CIFAR-10 Batch 1:  Loss:     0.6958 Validation Accuracy: 0.505600
Epoch 107, CIFAR-10 Batch 1:  Loss:     0.6843 Validation Accuracy: 0.504400
Epoch 108, CIFAR-10 Batch 1:  Loss:     0.6821 Validation Accuracy: 0.510800
Epoch 109, CIFAR-10 Batch 1:  Loss:     0.6801 Validation Accuracy: 0.504400
Epoch 110, CIFAR-10 Batch 1:  Loss:     0.6763 Validation Accuracy: 0.512000
Epoch 111, CIFAR-10 Batch 1:  Loss:     0.6610 Validation Accuracy: 0.510400
Epoch 112, CIFAR-10 Batch 1:  Loss:     0.6691 Validation Accuracy: 0.508600
Epoch 113, CIFAR-10 Batch 1:  Loss:     0.6611 Validation Accuracy: 0.510000
Epoch 114, CIFAR-10 Batch 1:  Loss:     0.6509 Validation Accuracy: 0.510000
Epoch 115, CIFAR-10 Batch 1:  Loss:     0.6532 Validation Accuracy: 0.506600
Epoch 116, CIFAR-10 Batch 1:  Loss:     0.6496 Validation Accuracy: 0.509000
Epoch 117, CIFAR-10 Batch 1:  Loss:     0.6434 Validation Accuracy: 0.508000
Epoch 118, CIFAR-10 Batch 1:  Loss:     0.6362 Validation Accuracy: 0.509800
Epoch 119, CIFAR-10 Batch 1:  Loss:     0.6398 Validation Accuracy: 0.511000
Epoch 120, CIFAR-10 Batch 1:  Loss:     0.6441 Validation Accuracy: 0.513800
Epoch 121, CIFAR-10 Batch 1:  Loss:     0.6338 Validation Accuracy: 0.508600
Epoch 122, CIFAR-10 Batch 1:  Loss:     0.6411 Validation Accuracy: 0.506400
Epoch 123, CIFAR-10 Batch 1:  Loss:     0.6369 Validation Accuracy: 0.509800
Epoch 124, CIFAR-10 Batch 1:  Loss:     0.6383 Validation Accuracy: 0.508400
Epoch 125, CIFAR-10 Batch 1:  Loss:     0.6287 Validation Accuracy: 0.512000
Epoch 126, CIFAR-10 Batch 1:  Loss:     0.6336 Validation Accuracy: 0.511000
Epoch 127, CIFAR-10 Batch 1:  Loss:     0.6465 Validation Accuracy: 0.511400
Epoch 128, CIFAR-10 Batch 1:  Loss:     0.6251 Validation Accuracy: 0.512800
Epoch 129, CIFAR-10 Batch 1:  Loss:     0.6258 Validation Accuracy: 0.509000
Epoch 130, CIFAR-10 Batch 1:  Loss:     0.6240 Validation Accuracy: 0.510400
Epoch 131, CIFAR-10 Batch 1:  Loss:     0.6248 Validation Accuracy: 0.510400
Epoch 132, CIFAR-10 Batch 1:  Loss:     0.6199 Validation Accuracy: 0.507400
Epoch 133, CIFAR-10 Batch 1:  Loss:     0.6204 Validation Accuracy: 0.510400
Epoch 134, CIFAR-10 Batch 1:  Loss:     0.6126 Validation Accuracy: 0.512200
Epoch 135, CIFAR-10 Batch 1:  Loss:     0.6119 Validation Accuracy: 0.515000
Epoch 136, CIFAR-10 Batch 1:  Loss:     0.6264 Validation Accuracy: 0.508400
Epoch 137, CIFAR-10 Batch 1:  Loss:     0.6138 Validation Accuracy: 0.515400
Epoch 138, CIFAR-10 Batch 1:  Loss:     0.6028 Validation Accuracy: 0.512800
Epoch 139, CIFAR-10 Batch 1:  Loss:     0.6161 Validation Accuracy: 0.509800
Epoch 140, CIFAR-10 Batch 1:  Loss:     0.6079 Validation Accuracy: 0.511000
Epoch 141, CIFAR-10 Batch 1:  Loss:     0.6028 Validation Accuracy: 0.509800
Epoch 142, CIFAR-10 Batch 1:  Loss:     0.6101 Validation Accuracy: 0.512600
Epoch 143, CIFAR-10 Batch 1:  Loss:     0.6074 Validation Accuracy: 0.515800
Epoch 144, CIFAR-10 Batch 1:  Loss:     0.6106 Validation Accuracy: 0.512000
Epoch 145, CIFAR-10 Batch 1:  Loss:     0.6018 Validation Accuracy: 0.514000
Epoch 146, CIFAR-10 Batch 1:  Loss:     0.5845 Validation Accuracy: 0.514600
Epoch 147, CIFAR-10 Batch 1:  Loss:     0.5822 Validation Accuracy: 0.519000
Epoch 148, CIFAR-10 Batch 1:  Loss:     0.5943 Validation Accuracy: 0.517200
Epoch 149, CIFAR-10 Batch 1:  Loss:     0.5924 Validation Accuracy: 0.511800
Epoch 150, CIFAR-10 Batch 1:  Loss:     0.5799 Validation Accuracy: 0.514800
Epoch 151, CIFAR-10 Batch 1:  Loss:     0.5833 Validation Accuracy: 0.518800
Epoch 152, CIFAR-10 Batch 1:  Loss:     0.5909 Validation Accuracy: 0.517800
Epoch 153, CIFAR-10 Batch 1:  Loss:     0.5967 Validation Accuracy: 0.516600
Epoch 154, CIFAR-10 Batch 1:  Loss:     0.5773 Validation Accuracy: 0.520400
Epoch 155, CIFAR-10 Batch 1:  Loss:     0.5818 Validation Accuracy: 0.512200
Epoch 156, CIFAR-10 Batch 1:  Loss:     0.5708 Validation Accuracy: 0.512000
Epoch 157, CIFAR-10 Batch 1:  Loss:     0.5725 Validation Accuracy: 0.516200
Epoch 158, CIFAR-10 Batch 1:  Loss:     0.5643 Validation Accuracy: 0.516600
Epoch 159, CIFAR-10 Batch 1:  Loss:     0.5589 Validation Accuracy: 0.518600
Epoch 160, CIFAR-10 Batch 1:  Loss:     0.5591 Validation Accuracy: 0.518400
Epoch 161, CIFAR-10 Batch 1:  Loss:     0.5638 Validation Accuracy: 0.513000
Epoch 162, CIFAR-10 Batch 1:  Loss:     0.5657 Validation Accuracy: 0.511400
Epoch 163, CIFAR-10 Batch 1:  Loss:     0.5665 Validation Accuracy: 0.509400
Epoch 164, CIFAR-10 Batch 1:  Loss:     0.5596 Validation Accuracy: 0.513600
Epoch 165, CIFAR-10 Batch 1:  Loss:     0.5535 Validation Accuracy: 0.518000
Epoch 166, CIFAR-10 Batch 1:  Loss:     0.5471 Validation Accuracy: 0.517200
Epoch 167, CIFAR-10 Batch 1:  Loss:     0.5541 Validation Accuracy: 0.512200
Epoch 168, CIFAR-10 Batch 1:  Loss:     0.5535 Validation Accuracy: 0.522400
Epoch 169, CIFAR-10 Batch 1:  Loss:     0.5487 Validation Accuracy: 0.511200
Epoch 170, CIFAR-10 Batch 1:  Loss:     0.5562 Validation Accuracy: 0.516400
Epoch 171, CIFAR-10 Batch 1:  Loss:     0.5521 Validation Accuracy: 0.513400
Epoch 172, CIFAR-10 Batch 1:  Loss:     0.5524 Validation Accuracy: 0.519000
Epoch 173, CIFAR-10 Batch 1:  Loss:     0.5532 Validation Accuracy: 0.516400
Epoch 174, CIFAR-10 Batch 1:  Loss:     0.5421 Validation Accuracy: 0.515400
Epoch 175, CIFAR-10 Batch 1:  Loss:     0.5410 Validation Accuracy: 0.518800
Epoch 176, CIFAR-10 Batch 1:  Loss:     0.5519 Validation Accuracy: 0.520400
Epoch 177, CIFAR-10 Batch 1:  Loss:     0.5430 Validation Accuracy: 0.517600
Epoch 178, CIFAR-10 Batch 1:  Loss:     0.5484 Validation Accuracy: 0.520200
Epoch 179, CIFAR-10 Batch 1:  Loss:     0.5507 Validation Accuracy: 0.513000
Epoch 180, CIFAR-10 Batch 1:  Loss:     0.5445 Validation Accuracy: 0.522200
Epoch 181, CIFAR-10 Batch 1:  Loss:     0.5388 Validation Accuracy: 0.520800
Epoch 182, CIFAR-10 Batch 1:  Loss:     0.5360 Validation Accuracy: 0.516800
Epoch 183, CIFAR-10 Batch 1:  Loss:     0.5323 Validation Accuracy: 0.516000
Epoch 184, CIFAR-10 Batch 1:  Loss:     0.5409 Validation Accuracy: 0.522800
Epoch 185, CIFAR-10 Batch 1:  Loss:     0.5374 Validation Accuracy: 0.521000
Epoch 186, CIFAR-10 Batch 1:  Loss:     0.5237 Validation Accuracy: 0.520000
Epoch 187, CIFAR-10 Batch 1:  Loss:     0.5402 Validation Accuracy: 0.523200
Epoch 188, CIFAR-10 Batch 1:  Loss:     0.5311 Validation Accuracy: 0.519000
Epoch 189, CIFAR-10 Batch 1:  Loss:     0.5401 Validation Accuracy: 0.521600
Epoch 190, CIFAR-10 Batch 1:  Loss:     0.5357 Validation Accuracy: 0.523200
Epoch 191, CIFAR-10 Batch 1:  Loss:     0.5252 Validation Accuracy: 0.520800
Epoch 192, CIFAR-10 Batch 1:  Loss:     0.5259 Validation Accuracy: 0.518600
Epoch 193, CIFAR-10 Batch 1:  Loss:     0.5266 Validation Accuracy: 0.520200
Epoch 194, CIFAR-10 Batch 1:  Loss:     0.5304 Validation Accuracy: 0.521200
Epoch 195, CIFAR-10 Batch 1:  Loss:     0.5306 Validation Accuracy: 0.516800
Epoch 196, CIFAR-10 Batch 1:  Loss:     0.5255 Validation Accuracy: 0.518800
Epoch 197, CIFAR-10 Batch 1:  Loss:     0.5272 Validation Accuracy: 0.523200
Epoch 198, CIFAR-10 Batch 1:  Loss:     0.5247 Validation Accuracy: 0.523800
Epoch 199, CIFAR-10 Batch 1:  Loss:     0.5176 Validation Accuracy: 0.522000
Epoch 200, CIFAR-10 Batch 1:  Loss:     0.5352 Validation Accuracy: 0.518000

Fully Train the Model

Now that you got a good accuracy with a single CIFAR-10 batch, try it with all five batches.





In [32]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
save_model_path = './image_classification'

print('Training...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())
    
    # Training cycle
    for epoch in range(epochs):
        # Loop over all batches
        n_batches = 5
        for batch_i in range(1, n_batches + 1):
            for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
                train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
            print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
            print_stats(sess, batch_features, batch_labels, cost, accuracy)
            
    # Save Model
    saver = tf.train.Saver()
    save_path = saver.save(sess, save_model_path)



    


















    






Training...
Epoch  1, CIFAR-10 Batch 1:  Loss:     2.0902 Validation Accuracy: 0.205000
Epoch  1, CIFAR-10 Batch 2:  Loss:     1.8799 Validation Accuracy: 0.263000
Epoch  1, CIFAR-10 Batch 3:  Loss:     1.7308 Validation Accuracy: 0.306000
Epoch  1, CIFAR-10 Batch 4:  Loss:     1.7492 Validation Accuracy: 0.334400
Epoch  1, CIFAR-10 Batch 5:  Loss:     1.6701 Validation Accuracy: 0.354600
Epoch  2, CIFAR-10 Batch 1:  Loss:     1.8451 Validation Accuracy: 0.394200
Epoch  2, CIFAR-10 Batch 2:  Loss:     1.5906 Validation Accuracy: 0.395800
Epoch  2, CIFAR-10 Batch 3:  Loss:     1.4809 Validation Accuracy: 0.395800
Epoch  2, CIFAR-10 Batch 4:  Loss:     1.5291 Validation Accuracy: 0.429000
Epoch  2, CIFAR-10 Batch 5:  Loss:     1.4286 Validation Accuracy: 0.429400
Epoch  3, CIFAR-10 Batch 1:  Loss:     1.6615 Validation Accuracy: 0.434000
Epoch  3, CIFAR-10 Batch 2:  Loss:     1.4305 Validation Accuracy: 0.433600
Epoch  3, CIFAR-10 Batch 3:  Loss:     1.3673 Validation Accuracy: 0.427200
Epoch  3, CIFAR-10 Batch 4:  Loss:     1.4627 Validation Accuracy: 0.453200
Epoch  3, CIFAR-10 Batch 5:  Loss:     1.3161 Validation Accuracy: 0.450800
Epoch  4, CIFAR-10 Batch 1:  Loss:     1.5447 Validation Accuracy: 0.453200
Epoch  4, CIFAR-10 Batch 2:  Loss:     1.3347 Validation Accuracy: 0.457400
Epoch  4, CIFAR-10 Batch 3:  Loss:     1.2654 Validation Accuracy: 0.451400
Epoch  4, CIFAR-10 Batch 4:  Loss:     1.4066 Validation Accuracy: 0.465000
Epoch  4, CIFAR-10 Batch 5:  Loss:     1.2451 Validation Accuracy: 0.467000
Epoch  5, CIFAR-10 Batch 1:  Loss:     1.4777 Validation Accuracy: 0.468200
Epoch  5, CIFAR-10 Batch 2:  Loss:     1.2709 Validation Accuracy: 0.468800
Epoch  5, CIFAR-10 Batch 3:  Loss:     1.2101 Validation Accuracy: 0.464000
Epoch  5, CIFAR-10 Batch 4:  Loss:     1.3863 Validation Accuracy: 0.475400
Epoch  5, CIFAR-10 Batch 5:  Loss:     1.2184 Validation Accuracy: 0.478600
Epoch  6, CIFAR-10 Batch 1:  Loss:     1.4155 Validation Accuracy: 0.479800
Epoch  6, CIFAR-10 Batch 2:  Loss:     1.2344 Validation Accuracy: 0.475800
Epoch  6, CIFAR-10 Batch 3:  Loss:     1.1957 Validation Accuracy: 0.466600
Epoch  6, CIFAR-10 Batch 4:  Loss:     1.3430 Validation Accuracy: 0.487400
Epoch  6, CIFAR-10 Batch 5:  Loss:     1.1849 Validation Accuracy: 0.488000
Epoch  7, CIFAR-10 Batch 1:  Loss:     1.3751 Validation Accuracy: 0.490000
Epoch  7, CIFAR-10 Batch 2:  Loss:     1.2021 Validation Accuracy: 0.483200
Epoch  7, CIFAR-10 Batch 3:  Loss:     1.1857 Validation Accuracy: 0.473400
Epoch  7, CIFAR-10 Batch 4:  Loss:     1.3168 Validation Accuracy: 0.494000
Epoch  7, CIFAR-10 Batch 5:  Loss:     1.1689 Validation Accuracy: 0.493000
Epoch  8, CIFAR-10 Batch 1:  Loss:     1.3551 Validation Accuracy: 0.496800
Epoch  8, CIFAR-10 Batch 2:  Loss:     1.1760 Validation Accuracy: 0.489600
Epoch  8, CIFAR-10 Batch 3:  Loss:     1.1509 Validation Accuracy: 0.479600
Epoch  8, CIFAR-10 Batch 4:  Loss:     1.2894 Validation Accuracy: 0.496800
Epoch  8, CIFAR-10 Batch 5:  Loss:     1.1531 Validation Accuracy: 0.499400
Epoch  9, CIFAR-10 Batch 1:  Loss:     1.3183 Validation Accuracy: 0.501000
Epoch  9, CIFAR-10 Batch 2:  Loss:     1.1479 Validation Accuracy: 0.491400
Epoch  9, CIFAR-10 Batch 3:  Loss:     1.1499 Validation Accuracy: 0.486600
Epoch  9, CIFAR-10 Batch 4:  Loss:     1.2802 Validation Accuracy: 0.502000
Epoch  9, CIFAR-10 Batch 5:  Loss:     1.1462 Validation Accuracy: 0.500400
Epoch 10, CIFAR-10 Batch 1:  Loss:     1.3048 Validation Accuracy: 0.503200
Epoch 10, CIFAR-10 Batch 2:  Loss:     1.1281 Validation Accuracy: 0.493000
Epoch 10, CIFAR-10 Batch 3:  Loss:     1.1123 Validation Accuracy: 0.497400
Epoch 10, CIFAR-10 Batch 4:  Loss:     1.2625 Validation Accuracy: 0.507600
Epoch 10, CIFAR-10 Batch 5:  Loss:     1.1422 Validation Accuracy: 0.509800
Epoch 11, CIFAR-10 Batch 1:  Loss:     1.2896 Validation Accuracy: 0.507600
Epoch 11, CIFAR-10 Batch 2:  Loss:     1.1031 Validation Accuracy: 0.496600
Epoch 11, CIFAR-10 Batch 3:  Loss:     1.1047 Validation Accuracy: 0.500400
Epoch 11, CIFAR-10 Batch 4:  Loss:     1.2501 Validation Accuracy: 0.512400
Epoch 11, CIFAR-10 Batch 5:  Loss:     1.1252 Validation Accuracy: 0.512600
Epoch 12, CIFAR-10 Batch 1:  Loss:     1.2747 Validation Accuracy: 0.513000
Epoch 12, CIFAR-10 Batch 2:  Loss:     1.0932 Validation Accuracy: 0.499200
Epoch 12, CIFAR-10 Batch 3:  Loss:     1.0861 Validation Accuracy: 0.503000
Epoch 12, CIFAR-10 Batch 4:  Loss:     1.2387 Validation Accuracy: 0.516000
Epoch 12, CIFAR-10 Batch 5:  Loss:     1.1115 Validation Accuracy: 0.514600
Epoch 13, CIFAR-10 Batch 1:  Loss:     1.2604 Validation Accuracy: 0.513600
Epoch 13, CIFAR-10 Batch 2:  Loss:     1.0780 Validation Accuracy: 0.502000
Epoch 13, CIFAR-10 Batch 3:  Loss:     1.0730 Validation Accuracy: 0.507400
Epoch 13, CIFAR-10 Batch 4:  Loss:     1.2320 Validation Accuracy: 0.517200
Epoch 13, CIFAR-10 Batch 5:  Loss:     1.1051 Validation Accuracy: 0.521600
Epoch 14, CIFAR-10 Batch 1:  Loss:     1.2581 Validation Accuracy: 0.520600
Epoch 14, CIFAR-10 Batch 2:  Loss:     1.0543 Validation Accuracy: 0.505400
Epoch 14, CIFAR-10 Batch 3:  Loss:     1.0601 Validation Accuracy: 0.511400
Epoch 14, CIFAR-10 Batch 4:  Loss:     1.2074 Validation Accuracy: 0.518800
Epoch 14, CIFAR-10 Batch 5:  Loss:     1.0984 Validation Accuracy: 0.522600
Epoch 15, CIFAR-10 Batch 1:  Loss:     1.2488 Validation Accuracy: 0.521600
Epoch 15, CIFAR-10 Batch 2:  Loss:     1.0581 Validation Accuracy: 0.509000
Epoch 15, CIFAR-10 Batch 3:  Loss:     1.0482 Validation Accuracy: 0.510800
Epoch 15, CIFAR-10 Batch 4:  Loss:     1.2035 Validation Accuracy: 0.525200
Epoch 15, CIFAR-10 Batch 5:  Loss:     1.0895 Validation Accuracy: 0.525800
Epoch 16, CIFAR-10 Batch 1:  Loss:     1.2512 Validation Accuracy: 0.521600
Epoch 16, CIFAR-10 Batch 2:  Loss:     1.0330 Validation Accuracy: 0.509400
Epoch 16, CIFAR-10 Batch 3:  Loss:     1.0222 Validation Accuracy: 0.514400
Epoch 16, CIFAR-10 Batch 4:  Loss:     1.1870 Validation Accuracy: 0.522200
Epoch 16, CIFAR-10 Batch 5:  Loss:     1.0677 Validation Accuracy: 0.523600
Epoch 17, CIFAR-10 Batch 1:  Loss:     1.2400 Validation Accuracy: 0.526200
Epoch 17, CIFAR-10 Batch 2:  Loss:     1.0244 Validation Accuracy: 0.513000
Epoch 17, CIFAR-10 Batch 3:  Loss:     1.0330 Validation Accuracy: 0.512600
Epoch 17, CIFAR-10 Batch 4:  Loss:     1.1783 Validation Accuracy: 0.526200
Epoch 17, CIFAR-10 Batch 5:  Loss:     1.0757 Validation Accuracy: 0.533200
Epoch 18, CIFAR-10 Batch 1:  Loss:     1.2470 Validation Accuracy: 0.526200
Epoch 18, CIFAR-10 Batch 2:  Loss:     1.0159 Validation Accuracy: 0.516000
Epoch 18, CIFAR-10 Batch 3:  Loss:     1.0199 Validation Accuracy: 0.516400
Epoch 18, CIFAR-10 Batch 4:  Loss:     1.1731 Validation Accuracy: 0.525600
Epoch 18, CIFAR-10 Batch 5:  Loss:     1.0576 Validation Accuracy: 0.530800
Epoch 19, CIFAR-10 Batch 1:  Loss:     1.2309 Validation Accuracy: 0.526400
Epoch 19, CIFAR-10 Batch 2:  Loss:     0.9996 Validation Accuracy: 0.523400
Epoch 19, CIFAR-10 Batch 3:  Loss:     1.0094 Validation Accuracy: 0.519000
Epoch 19, CIFAR-10 Batch 4:  Loss:     1.1598 Validation Accuracy: 0.529600
Epoch 19, CIFAR-10 Batch 5:  Loss:     1.0484 Validation Accuracy: 0.536200
Epoch 20, CIFAR-10 Batch 1:  Loss:     1.2211 Validation Accuracy: 0.530000
Epoch 20, CIFAR-10 Batch 2:  Loss:     0.9845 Validation Accuracy: 0.519000
Epoch 20, CIFAR-10 Batch 3:  Loss:     0.9999 Validation Accuracy: 0.519600
Epoch 20, CIFAR-10 Batch 4:  Loss:     1.1557 Validation Accuracy: 0.534800
Epoch 20, CIFAR-10 Batch 5:  Loss:     1.0362 Validation Accuracy: 0.537200
Epoch 21, CIFAR-10 Batch 1:  Loss:     1.1947 Validation Accuracy: 0.530600
Epoch 21, CIFAR-10 Batch 2:  Loss:     0.9738 Validation Accuracy: 0.520600
Epoch 21, CIFAR-10 Batch 3:  Loss:     0.9995 Validation Accuracy: 0.527200
Epoch 21, CIFAR-10 Batch 4:  Loss:     1.1373 Validation Accuracy: 0.535800
Epoch 21, CIFAR-10 Batch 5:  Loss:     1.0258 Validation Accuracy: 0.537000
Epoch 22, CIFAR-10 Batch 1:  Loss:     1.2145 Validation Accuracy: 0.530200
Epoch 22, CIFAR-10 Batch 2:  Loss:     0.9588 Validation Accuracy: 0.518000
Epoch 22, CIFAR-10 Batch 3:  Loss:     0.9882 Validation Accuracy: 0.525800
Epoch 22, CIFAR-10 Batch 4:  Loss:     1.1372 Validation Accuracy: 0.541400
Epoch 22, CIFAR-10 Batch 5:  Loss:     1.0227 Validation Accuracy: 0.542600
Epoch 23, CIFAR-10 Batch 1:  Loss:     1.1900 Validation Accuracy: 0.535200
Epoch 23, CIFAR-10 Batch 2:  Loss:     0.9493 Validation Accuracy: 0.528400
Epoch 23, CIFAR-10 Batch 3:  Loss:     0.9792 Validation Accuracy: 0.523000
Epoch 23, CIFAR-10 Batch 4:  Loss:     1.1396 Validation Accuracy: 0.542200
Epoch 23, CIFAR-10 Batch 5:  Loss:     1.0100 Validation Accuracy: 0.539600
Epoch 24, CIFAR-10 Batch 1:  Loss:     1.1843 Validation Accuracy: 0.540600
Epoch 24, CIFAR-10 Batch 2:  Loss:     0.9458 Validation Accuracy: 0.526400
Epoch 24, CIFAR-10 Batch 3:  Loss:     0.9672 Validation Accuracy: 0.532600
Epoch 24, CIFAR-10 Batch 4:  Loss:     1.1211 Validation Accuracy: 0.543600
Epoch 24, CIFAR-10 Batch 5:  Loss:     1.0063 Validation Accuracy: 0.542200
Epoch 25, CIFAR-10 Batch 1:  Loss:     1.1866 Validation Accuracy: 0.539600
Epoch 25, CIFAR-10 Batch 2:  Loss:     0.9292 Validation Accuracy: 0.525400
Epoch 25, CIFAR-10 Batch 3:  Loss:     0.9533 Validation Accuracy: 0.532800
Epoch 25, CIFAR-10 Batch 4:  Loss:     1.1129 Validation Accuracy: 0.543600
Epoch 25, CIFAR-10 Batch 5:  Loss:     0.9957 Validation Accuracy: 0.542800
Epoch 26, CIFAR-10 Batch 1:  Loss:     1.1633 Validation Accuracy: 0.539600
Epoch 26, CIFAR-10 Batch 2:  Loss:     0.9192 Validation Accuracy: 0.531800
Epoch 26, CIFAR-10 Batch 3:  Loss:     0.9539 Validation Accuracy: 0.530800
Epoch 26, CIFAR-10 Batch 4:  Loss:     1.1188 Validation Accuracy: 0.543600
Epoch 26, CIFAR-10 Batch 5:  Loss:     0.9858 Validation Accuracy: 0.544800
Epoch 27, CIFAR-10 Batch 1:  Loss:     1.1589 Validation Accuracy: 0.545400
Epoch 27, CIFAR-10 Batch 2:  Loss:     0.9208 Validation Accuracy: 0.526800
Epoch 27, CIFAR-10 Batch 3:  Loss:     0.9599 Validation Accuracy: 0.538600
Epoch 27, CIFAR-10 Batch 4:  Loss:     1.1068 Validation Accuracy: 0.545000
Epoch 27, CIFAR-10 Batch 5:  Loss:     0.9836 Validation Accuracy: 0.544800
Epoch 28, CIFAR-10 Batch 1:  Loss:     1.1723 Validation Accuracy: 0.541200
Epoch 28, CIFAR-10 Batch 2:  Loss:     0.9198 Validation Accuracy: 0.534200
Epoch 28, CIFAR-10 Batch 3:  Loss:     0.9476 Validation Accuracy: 0.536800
Epoch 28, CIFAR-10 Batch 4:  Loss:     1.0872 Validation Accuracy: 0.546800
Epoch 28, CIFAR-10 Batch 5:  Loss:     0.9759 Validation Accuracy: 0.543200
Epoch 29, CIFAR-10 Batch 1:  Loss:     1.1635 Validation Accuracy: 0.542600
Epoch 29, CIFAR-10 Batch 2:  Loss:     0.9065 Validation Accuracy: 0.537600
Epoch 29, CIFAR-10 Batch 3:  Loss:     0.9398 Validation Accuracy: 0.540000
Epoch 29, CIFAR-10 Batch 4:  Loss:     1.0912 Validation Accuracy: 0.546600
Epoch 29, CIFAR-10 Batch 5:  Loss:     0.9868 Validation Accuracy: 0.553800
Epoch 30, CIFAR-10 Batch 1:  Loss:     1.1371 Validation Accuracy: 0.547800
Epoch 30, CIFAR-10 Batch 2:  Loss:     0.8991 Validation Accuracy: 0.534600
Epoch 30, CIFAR-10 Batch 3:  Loss:     0.9376 Validation Accuracy: 0.539200
Epoch 30, CIFAR-10 Batch 4:  Loss:     1.0814 Validation Accuracy: 0.549600
Epoch 30, CIFAR-10 Batch 5:  Loss:     0.9791 Validation Accuracy: 0.550600
Epoch 31, CIFAR-10 Batch 1:  Loss:     1.1388 Validation Accuracy: 0.544800
Epoch 31, CIFAR-10 Batch 2:  Loss:     0.8908 Validation Accuracy: 0.531600
Epoch 31, CIFAR-10 Batch 3:  Loss:     0.9404 Validation Accuracy: 0.539400
Epoch 31, CIFAR-10 Batch 4:  Loss:     1.0949 Validation Accuracy: 0.547600
Epoch 31, CIFAR-10 Batch 5:  Loss:     0.9706 Validation Accuracy: 0.552200
Epoch 32, CIFAR-10 Batch 1:  Loss:     1.1472 Validation Accuracy: 0.548800
Epoch 32, CIFAR-10 Batch 2:  Loss:     0.8823 Validation Accuracy: 0.536800
Epoch 32, CIFAR-10 Batch 3:  Loss:     0.9453 Validation Accuracy: 0.542400
Epoch 32, CIFAR-10 Batch 4:  Loss:     1.0899 Validation Accuracy: 0.550400
Epoch 32, CIFAR-10 Batch 5:  Loss:     0.9643 Validation Accuracy: 0.552600
Epoch 33, CIFAR-10 Batch 1:  Loss:     1.1284 Validation Accuracy: 0.549400
Epoch 33, CIFAR-10 Batch 2:  Loss:     0.8776 Validation Accuracy: 0.537600
Epoch 33, CIFAR-10 Batch 3:  Loss:     0.9148 Validation Accuracy: 0.547800
Epoch 33, CIFAR-10 Batch 4:  Loss:     1.0804 Validation Accuracy: 0.547400
Epoch 33, CIFAR-10 Batch 5:  Loss:     0.9658 Validation Accuracy: 0.551600
Epoch 34, CIFAR-10 Batch 1:  Loss:     1.1288 Validation Accuracy: 0.553600
Epoch 34, CIFAR-10 Batch 2:  Loss:     0.8821 Validation Accuracy: 0.537000
Epoch 34, CIFAR-10 Batch 3:  Loss:     0.9211 Validation Accuracy: 0.542800
Epoch 34, CIFAR-10 Batch 4:  Loss:     1.0638 Validation Accuracy: 0.550200
Epoch 34, CIFAR-10 Batch 5:  Loss:     0.9467 Validation Accuracy: 0.555800
Epoch 35, CIFAR-10 Batch 1:  Loss:     1.1245 Validation Accuracy: 0.552000
Epoch 35, CIFAR-10 Batch 2:  Loss:     0.8732 Validation Accuracy: 0.538400
Epoch 35, CIFAR-10 Batch 3:  Loss:     0.9069 Validation Accuracy: 0.542800
Epoch 35, CIFAR-10 Batch 4:  Loss:     1.0679 Validation Accuracy: 0.555200
Epoch 35, CIFAR-10 Batch 5:  Loss:     0.9477 Validation Accuracy: 0.553800
Epoch 36, CIFAR-10 Batch 1:  Loss:     1.1222 Validation Accuracy: 0.553800
Epoch 36, CIFAR-10 Batch 2:  Loss:     0.8720 Validation Accuracy: 0.540200
Epoch 36, CIFAR-10 Batch 3:  Loss:     0.9318 Validation Accuracy: 0.545000
Epoch 36, CIFAR-10 Batch 4:  Loss:     1.0746 Validation Accuracy: 0.551600
Epoch 36, CIFAR-10 Batch 5:  Loss:     0.9484 Validation Accuracy: 0.558200
Epoch 37, CIFAR-10 Batch 1:  Loss:     1.0987 Validation Accuracy: 0.553800
Epoch 37, CIFAR-10 Batch 2:  Loss:     0.8696 Validation Accuracy: 0.544600
Epoch 37, CIFAR-10 Batch 3:  Loss:     0.9172 Validation Accuracy: 0.546200
Epoch 37, CIFAR-10 Batch 4:  Loss:     1.0633 Validation Accuracy: 0.554000
Epoch 37, CIFAR-10 Batch 5:  Loss:     0.9490 Validation Accuracy: 0.555200
Epoch 38, CIFAR-10 Batch 1:  Loss:     1.1136 Validation Accuracy: 0.553400
Epoch 38, CIFAR-10 Batch 2:  Loss:     0.8556 Validation Accuracy: 0.540000
Epoch 38, CIFAR-10 Batch 3:  Loss:     0.9092 Validation Accuracy: 0.546600
Epoch 38, CIFAR-10 Batch 4:  Loss:     1.0641 Validation Accuracy: 0.557200
Epoch 38, CIFAR-10 Batch 5:  Loss:     0.9318 Validation Accuracy: 0.558600
Epoch 39, CIFAR-10 Batch 1:  Loss:     1.0952 Validation Accuracy: 0.554800
Epoch 39, CIFAR-10 Batch 2:  Loss:     0.8603 Validation Accuracy: 0.544800
Epoch 39, CIFAR-10 Batch 3:  Loss:     0.9027 Validation Accuracy: 0.546600
Epoch 39, CIFAR-10 Batch 4:  Loss:     1.0579 Validation Accuracy: 0.557400
Epoch 39, CIFAR-10 Batch 5:  Loss:     0.9383 Validation Accuracy: 0.555400
Epoch 40, CIFAR-10 Batch 1:  Loss:     1.0982 Validation Accuracy: 0.552800
Epoch 40, CIFAR-10 Batch 2:  Loss:     0.8558 Validation Accuracy: 0.548600
Epoch 40, CIFAR-10 Batch 3:  Loss:     0.8954 Validation Accuracy: 0.547400
Epoch 40, CIFAR-10 Batch 4:  Loss:     1.0621 Validation Accuracy: 0.557200
Epoch 40, CIFAR-10 Batch 5:  Loss:     0.9368 Validation Accuracy: 0.559000
Epoch 41, CIFAR-10 Batch 1:  Loss:     1.0930 Validation Accuracy: 0.555600
Epoch 41, CIFAR-10 Batch 2:  Loss:     0.8556 Validation Accuracy: 0.547800
Epoch 41, CIFAR-10 Batch 3:  Loss:     0.8918 Validation Accuracy: 0.549400
Epoch 41, CIFAR-10 Batch 4:  Loss:     1.0767 Validation Accuracy: 0.554600
Epoch 41, CIFAR-10 Batch 5:  Loss:     0.9335 Validation Accuracy: 0.558200
Epoch 42, CIFAR-10 Batch 1:  Loss:     1.0951 Validation Accuracy: 0.560200
Epoch 42, CIFAR-10 Batch 2:  Loss:     0.8536 Validation Accuracy: 0.549000
Epoch 42, CIFAR-10 Batch 3:  Loss:     0.8886 Validation Accuracy: 0.555400
Epoch 42, CIFAR-10 Batch 4:  Loss:     1.0611 Validation Accuracy: 0.557800
Epoch 42, CIFAR-10 Batch 5:  Loss:     0.9207 Validation Accuracy: 0.560600
Epoch 43, CIFAR-10 Batch 1:  Loss:     1.0742 Validation Accuracy: 0.554200
Epoch 43, CIFAR-10 Batch 2:  Loss:     0.8463 Validation Accuracy: 0.544200
Epoch 43, CIFAR-10 Batch 3:  Loss:     0.8850 Validation Accuracy: 0.552200
Epoch 43, CIFAR-10 Batch 4:  Loss:     1.0472 Validation Accuracy: 0.557600
Epoch 43, CIFAR-10 Batch 5:  Loss:     0.9173 Validation Accuracy: 0.564000
Epoch 44, CIFAR-10 Batch 1:  Loss:     1.0745 Validation Accuracy: 0.562000
Epoch 44, CIFAR-10 Batch 2:  Loss:     0.8431 Validation Accuracy: 0.552600
Epoch 44, CIFAR-10 Batch 3:  Loss:     0.8746 Validation Accuracy: 0.551400
Epoch 44, CIFAR-10 Batch 4:  Loss:     1.0581 Validation Accuracy: 0.557400
Epoch 44, CIFAR-10 Batch 5:  Loss:     0.9103 Validation Accuracy: 0.561800
Epoch 45, CIFAR-10 Batch 1:  Loss:     1.0811 Validation Accuracy: 0.559600
Epoch 45, CIFAR-10 Batch 2:  Loss:     0.8366 Validation Accuracy: 0.546400
Epoch 45, CIFAR-10 Batch 3:  Loss:     0.8831 Validation Accuracy: 0.553800
Epoch 45, CIFAR-10 Batch 4:  Loss:     1.0556 Validation Accuracy: 0.556000
Epoch 45, CIFAR-10 Batch 5:  Loss:     0.9170 Validation Accuracy: 0.564600
Epoch 46, CIFAR-10 Batch 1:  Loss:     1.0863 Validation Accuracy: 0.557000
Epoch 46, CIFAR-10 Batch 2:  Loss:     0.8388 Validation Accuracy: 0.547200
Epoch 46, CIFAR-10 Batch 3:  Loss:     0.8781 Validation Accuracy: 0.553000
Epoch 46, CIFAR-10 Batch 4:  Loss:     1.0603 Validation Accuracy: 0.558400
Epoch 46, CIFAR-10 Batch 5:  Loss:     0.9170 Validation Accuracy: 0.564200
Epoch 47, CIFAR-10 Batch 1:  Loss:     1.0638 Validation Accuracy: 0.560600
Epoch 47, CIFAR-10 Batch 2:  Loss:     0.8337 Validation Accuracy: 0.547400
Epoch 47, CIFAR-10 Batch 3:  Loss:     0.8718 Validation Accuracy: 0.556800
Epoch 47, CIFAR-10 Batch 4:  Loss:     1.0436 Validation Accuracy: 0.558800
Epoch 47, CIFAR-10 Batch 5:  Loss:     0.9057 Validation Accuracy: 0.563400
Epoch 48, CIFAR-10 Batch 1:  Loss:     1.0710 Validation Accuracy: 0.563200
Epoch 48, CIFAR-10 Batch 2:  Loss:     0.8317 Validation Accuracy: 0.549600
Epoch 48, CIFAR-10 Batch 3:  Loss:     0.8672 Validation Accuracy: 0.555800
Epoch 48, CIFAR-10 Batch 4:  Loss:     1.0541 Validation Accuracy: 0.560800
Epoch 48, CIFAR-10 Batch 5:  Loss:     0.9091 Validation Accuracy: 0.566200
Epoch 49, CIFAR-10 Batch 1:  Loss:     1.0666 Validation Accuracy: 0.562400
Epoch 49, CIFAR-10 Batch 2:  Loss:     0.8268 Validation Accuracy: 0.546000
Epoch 49, CIFAR-10 Batch 3:  Loss:     0.8691 Validation Accuracy: 0.555000
Epoch 49, CIFAR-10 Batch 4:  Loss:     1.0489 Validation Accuracy: 0.562800
Epoch 49, CIFAR-10 Batch 5:  Loss:     0.9072 Validation Accuracy: 0.566600
Epoch 50, CIFAR-10 Batch 1:  Loss:     1.0482 Validation Accuracy: 0.562200
Epoch 50, CIFAR-10 Batch 2:  Loss:     0.8328 Validation Accuracy: 0.549600
Epoch 50, CIFAR-10 Batch 3:  Loss:     0.8526 Validation Accuracy: 0.559600
Epoch 50, CIFAR-10 Batch 4:  Loss:     1.0533 Validation Accuracy: 0.564400
Epoch 50, CIFAR-10 Batch 5:  Loss:     0.9003 Validation Accuracy: 0.563000
Epoch 51, CIFAR-10 Batch 1:  Loss:     1.0482 Validation Accuracy: 0.562400
Epoch 51, CIFAR-10 Batch 2:  Loss:     0.8228 Validation Accuracy: 0.552400
Epoch 51, CIFAR-10 Batch 3:  Loss:     0.8550 Validation Accuracy: 0.558400
Epoch 51, CIFAR-10 Batch 4:  Loss:     1.0409 Validation Accuracy: 0.564000
Epoch 51, CIFAR-10 Batch 5:  Loss:     0.9008 Validation Accuracy: 0.564400
Epoch 52, CIFAR-10 Batch 1:  Loss:     1.0559 Validation Accuracy: 0.565400
Epoch 52, CIFAR-10 Batch 2:  Loss:     0.8201 Validation Accuracy: 0.553000
Epoch 52, CIFAR-10 Batch 3:  Loss:     0.8496 Validation Accuracy: 0.557400
Epoch 52, CIFAR-10 Batch 4:  Loss:     1.0420 Validation Accuracy: 0.563800
Epoch 52, CIFAR-10 Batch 5:  Loss:     0.8917 Validation Accuracy: 0.569600
Epoch 53, CIFAR-10 Batch 1:  Loss:     1.0500 Validation Accuracy: 0.566200
Epoch 53, CIFAR-10 Batch 2:  Loss:     0.8231 Validation Accuracy: 0.553200
Epoch 53, CIFAR-10 Batch 3:  Loss:     0.8402 Validation Accuracy: 0.562000
Epoch 53, CIFAR-10 Batch 4:  Loss:     1.0471 Validation Accuracy: 0.564600
Epoch 53, CIFAR-10 Batch 5:  Loss:     0.8950 Validation Accuracy: 0.570200
Epoch 54, CIFAR-10 Batch 1:  Loss:     1.0250 Validation Accuracy: 0.566400
Epoch 54, CIFAR-10 Batch 2:  Loss:     0.8247 Validation Accuracy: 0.553800
Epoch 54, CIFAR-10 Batch 3:  Loss:     0.8529 Validation Accuracy: 0.559200
Epoch 54, CIFAR-10 Batch 4:  Loss:     1.0230 Validation Accuracy: 0.571000
Epoch 54, CIFAR-10 Batch 5:  Loss:     0.8953 Validation Accuracy: 0.565000
Epoch 55, CIFAR-10 Batch 1:  Loss:     1.0173 Validation Accuracy: 0.562400
Epoch 55, CIFAR-10 Batch 2:  Loss:     0.8225 Validation Accuracy: 0.549000
Epoch 55, CIFAR-10 Batch 3:  Loss:     0.8307 Validation Accuracy: 0.559400
Epoch 55, CIFAR-10 Batch 4:  Loss:     1.0337 Validation Accuracy: 0.571200
Epoch 55, CIFAR-10 Batch 5:  Loss:     0.9061 Validation Accuracy: 0.564600
Epoch 56, CIFAR-10 Batch 1:  Loss:     1.0251 Validation Accuracy: 0.565800
Epoch 56, CIFAR-10 Batch 2:  Loss:     0.8177 Validation Accuracy: 0.547200
Epoch 56, CIFAR-10 Batch 3:  Loss:     0.8355 Validation Accuracy: 0.558000
Epoch 56, CIFAR-10 Batch 4:  Loss:     1.0444 Validation Accuracy: 0.566800
Epoch 56, CIFAR-10 Batch 5:  Loss:     0.8985 Validation Accuracy: 0.566400
Epoch 57, CIFAR-10 Batch 1:  Loss:     1.0217 Validation Accuracy: 0.564600
Epoch 57, CIFAR-10 Batch 2:  Loss:     0.8177 Validation Accuracy: 0.548000
Epoch 57, CIFAR-10 Batch 3:  Loss:     0.8369 Validation Accuracy: 0.562200
Epoch 57, CIFAR-10 Batch 4:  Loss:     1.0342 Validation Accuracy: 0.566200
Epoch 57, CIFAR-10 Batch 5:  Loss:     0.8905 Validation Accuracy: 0.566800
Epoch 58, CIFAR-10 Batch 1:  Loss:     1.0201 Validation Accuracy: 0.567400
Epoch 58, CIFAR-10 Batch 2:  Loss:     0.8093 Validation Accuracy: 0.553200
Epoch 58, CIFAR-10 Batch 3:  Loss:     0.8382 Validation Accuracy: 0.558000
Epoch 58, CIFAR-10 Batch 4:  Loss:     1.0373 Validation Accuracy: 0.570600
Epoch 58, CIFAR-10 Batch 5:  Loss:     0.8993 Validation Accuracy: 0.567800
Epoch 59, CIFAR-10 Batch 1:  Loss:     1.0339 Validation Accuracy: 0.564800
Epoch 59, CIFAR-10 Batch 2:  Loss:     0.8081 Validation Accuracy: 0.550800
Epoch 59, CIFAR-10 Batch 3:  Loss:     0.8289 Validation Accuracy: 0.560600
Epoch 59, CIFAR-10 Batch 4:  Loss:     1.0316 Validation Accuracy: 0.566600
Epoch 59, CIFAR-10 Batch 5:  Loss:     0.8953 Validation Accuracy: 0.570200
Epoch 60, CIFAR-10 Batch 1:  Loss:     1.0097 Validation Accuracy: 0.565800
Epoch 60, CIFAR-10 Batch 2:  Loss:     0.8122 Validation Accuracy: 0.544800
Epoch 60, CIFAR-10 Batch 3:  Loss:     0.8183 Validation Accuracy: 0.564000
Epoch 60, CIFAR-10 Batch 4:  Loss:     1.0294 Validation Accuracy: 0.570800
Epoch 60, CIFAR-10 Batch 5:  Loss:     0.8839 Validation Accuracy: 0.568000
Epoch 61, CIFAR-10 Batch 1:  Loss:     1.0193 Validation Accuracy: 0.566200
Epoch 61, CIFAR-10 Batch 2:  Loss:     0.8116 Validation Accuracy: 0.549200
Epoch 61, CIFAR-10 Batch 3:  Loss:     0.8092 Validation Accuracy: 0.562400
Epoch 61, CIFAR-10 Batch 4:  Loss:     1.0273 Validation Accuracy: 0.566600
Epoch 61, CIFAR-10 Batch 5:  Loss:     0.8842 Validation Accuracy: 0.566400
Epoch 62, CIFAR-10 Batch 1:  Loss:     1.0272 Validation Accuracy: 0.567400
Epoch 62, CIFAR-10 Batch 2:  Loss:     0.8089 Validation Accuracy: 0.548800
Epoch 62, CIFAR-10 Batch 3:  Loss:     0.8216 Validation Accuracy: 0.562200
Epoch 62, CIFAR-10 Batch 4:  Loss:     1.0101 Validation Accuracy: 0.571200
Epoch 62, CIFAR-10 Batch 5:  Loss:     0.8874 Validation Accuracy: 0.569600
Epoch 63, CIFAR-10 Batch 1:  Loss:     0.9949 Validation Accuracy: 0.570000
Epoch 63, CIFAR-10 Batch 2:  Loss:     0.8064 Validation Accuracy: 0.553000
Epoch 63, CIFAR-10 Batch 3:  Loss:     0.8198 Validation Accuracy: 0.563600
Epoch 63, CIFAR-10 Batch 4:  Loss:     1.0218 Validation Accuracy: 0.569400
Epoch 63, CIFAR-10 Batch 5:  Loss:     0.8860 Validation Accuracy: 0.570400
Epoch 64, CIFAR-10 Batch 1:  Loss:     1.0079 Validation Accuracy: 0.567600
Epoch 64, CIFAR-10 Batch 2:  Loss:     0.8072 Validation Accuracy: 0.553200
Epoch 64, CIFAR-10 Batch 3:  Loss:     0.8159 Validation Accuracy: 0.566000
Epoch 64, CIFAR-10 Batch 4:  Loss:     1.0142 Validation Accuracy: 0.573000
Epoch 64, CIFAR-10 Batch 5:  Loss:     0.8689 Validation Accuracy: 0.567600
Epoch 65, CIFAR-10 Batch 1:  Loss:     0.9938 Validation Accuracy: 0.567800
Epoch 65, CIFAR-10 Batch 2:  Loss:     0.8009 Validation Accuracy: 0.551800
Epoch 65, CIFAR-10 Batch 3:  Loss:     0.8114 Validation Accuracy: 0.564800
Epoch 65, CIFAR-10 Batch 4:  Loss:     1.0130 Validation Accuracy: 0.570000
Epoch 65, CIFAR-10 Batch 5:  Loss:     0.8752 Validation Accuracy: 0.573400
Epoch 66, CIFAR-10 Batch 1:  Loss:     1.0035 Validation Accuracy: 0.567600
Epoch 66, CIFAR-10 Batch 2:  Loss:     0.8008 Validation Accuracy: 0.556600
Epoch 66, CIFAR-10 Batch 3:  Loss:     0.8154 Validation Accuracy: 0.563600
Epoch 66, CIFAR-10 Batch 4:  Loss:     1.0103 Validation Accuracy: 0.570200
Epoch 66, CIFAR-10 Batch 5:  Loss:     0.8783 Validation Accuracy: 0.568000
Epoch 67, CIFAR-10 Batch 1:  Loss:     0.9979 Validation Accuracy: 0.569200
Epoch 67, CIFAR-10 Batch 2:  Loss:     0.8076 Validation Accuracy: 0.559000
Epoch 67, CIFAR-10 Batch 3:  Loss:     0.8165 Validation Accuracy: 0.561200
Epoch 67, CIFAR-10 Batch 4:  Loss:     1.0164 Validation Accuracy: 0.569200
Epoch 67, CIFAR-10 Batch 5:  Loss:     0.8801 Validation Accuracy: 0.571200
Epoch 68, CIFAR-10 Batch 1:  Loss:     1.0194 Validation Accuracy: 0.567800
Epoch 68, CIFAR-10 Batch 2:  Loss:     0.8030 Validation Accuracy: 0.554400
Epoch 68, CIFAR-10 Batch 3:  Loss:     0.8057 Validation Accuracy: 0.563200
Epoch 68, CIFAR-10 Batch 4:  Loss:     1.0152 Validation Accuracy: 0.571000
Epoch 68, CIFAR-10 Batch 5:  Loss:     0.8756 Validation Accuracy: 0.567400
Epoch 69, CIFAR-10 Batch 1:  Loss:     1.0069 Validation Accuracy: 0.566600
Epoch 69, CIFAR-10 Batch 2:  Loss:     0.7938 Validation Accuracy: 0.555200
Epoch 69, CIFAR-10 Batch 3:  Loss:     0.8063 Validation Accuracy: 0.565000
Epoch 69, CIFAR-10 Batch 4:  Loss:     1.0055 Validation Accuracy: 0.574800
Epoch 69, CIFAR-10 Batch 5:  Loss:     0.8763 Validation Accuracy: 0.568800
Epoch 70, CIFAR-10 Batch 1:  Loss:     0.9899 Validation Accuracy: 0.568800
Epoch 70, CIFAR-10 Batch 2:  Loss:     0.7957 Validation Accuracy: 0.552800
Epoch 70, CIFAR-10 Batch 3:  Loss:     0.8022 Validation Accuracy: 0.565200
Epoch 70, CIFAR-10 Batch 4:  Loss:     1.0067 Validation Accuracy: 0.573200
Epoch 70, CIFAR-10 Batch 5:  Loss:     0.8637 Validation Accuracy: 0.571000
Epoch 71, CIFAR-10 Batch 1:  Loss:     1.0020 Validation Accuracy: 0.568400
Epoch 71, CIFAR-10 Batch 2:  Loss:     0.7923 Validation Accuracy: 0.554400
Epoch 71, CIFAR-10 Batch 3:  Loss:     0.7988 Validation Accuracy: 0.565800
Epoch 71, CIFAR-10 Batch 4:  Loss:     1.0090 Validation Accuracy: 0.572200
Epoch 71, CIFAR-10 Batch 5:  Loss:     0.8681 Validation Accuracy: 0.567600
Epoch 72, CIFAR-10 Batch 1:  Loss:     0.9892 Validation Accuracy: 0.568600
Epoch 72, CIFAR-10 Batch 2:  Loss:     0.7971 Validation Accuracy: 0.558200
Epoch 72, CIFAR-10 Batch 3:  Loss:     0.7885 Validation Accuracy: 0.568200
Epoch 72, CIFAR-10 Batch 4:  Loss:     1.0137 Validation Accuracy: 0.571400
Epoch 72, CIFAR-10 Batch 5:  Loss:     0.8741 Validation Accuracy: 0.570000
Epoch 73, CIFAR-10 Batch 1:  Loss:     0.9881 Validation Accuracy: 0.569400
Epoch 73, CIFAR-10 Batch 2:  Loss:     0.7870 Validation Accuracy: 0.553600
Epoch 73, CIFAR-10 Batch 3:  Loss:     0.7874 Validation Accuracy: 0.567800
Epoch 73, CIFAR-10 Batch 4:  Loss:     1.0070 Validation Accuracy: 0.571200
Epoch 73, CIFAR-10 Batch 5:  Loss:     0.8680 Validation Accuracy: 0.571400
Epoch 74, CIFAR-10 Batch 1:  Loss:     0.9785 Validation Accuracy: 0.571600
Epoch 74, CIFAR-10 Batch 2:  Loss:     0.7984 Validation Accuracy: 0.562400
Epoch 74, CIFAR-10 Batch 3:  Loss:     0.7985 Validation Accuracy: 0.565600
Epoch 74, CIFAR-10 Batch 4:  Loss:     0.9949 Validation Accuracy: 0.575600
Epoch 74, CIFAR-10 Batch 5:  Loss:     0.8685 Validation Accuracy: 0.567800
Epoch 75, CIFAR-10 Batch 1:  Loss:     0.9778 Validation Accuracy: 0.571200
Epoch 75, CIFAR-10 Batch 2:  Loss:     0.7905 Validation Accuracy: 0.558400
Epoch 75, CIFAR-10 Batch 3:  Loss:     0.7957 Validation Accuracy: 0.569800
Epoch 75, CIFAR-10 Batch 4:  Loss:     0.9978 Validation Accuracy: 0.572800
Epoch 75, CIFAR-10 Batch 5:  Loss:     0.8601 Validation Accuracy: 0.572600
Epoch 76, CIFAR-10 Batch 1:  Loss:     0.9656 Validation Accuracy: 0.571200
Epoch 76, CIFAR-10 Batch 2:  Loss:     0.7911 Validation Accuracy: 0.556600
Epoch 76, CIFAR-10 Batch 3:  Loss:     0.7922 Validation Accuracy: 0.565200
Epoch 76, CIFAR-10 Batch 4:  Loss:     1.0044 Validation Accuracy: 0.572000
Epoch 76, CIFAR-10 Batch 5:  Loss:     0.8570 Validation Accuracy: 0.576600
Epoch 77, CIFAR-10 Batch 1:  Loss:     0.9760 Validation Accuracy: 0.568600
Epoch 77, CIFAR-10 Batch 2:  Loss:     0.7902 Validation Accuracy: 0.556000
Epoch 77, CIFAR-10 Batch 3:  Loss:     0.7983 Validation Accuracy: 0.565200
Epoch 77, CIFAR-10 Batch 4:  Loss:     1.0202 Validation Accuracy: 0.571200
Epoch 77, CIFAR-10 Batch 5:  Loss:     0.8688 Validation Accuracy: 0.572400
Epoch 78, CIFAR-10 Batch 1:  Loss:     0.9739 Validation Accuracy: 0.571000
Epoch 78, CIFAR-10 Batch 2:  Loss:     0.7891 Validation Accuracy: 0.556200
Epoch 78, CIFAR-10 Batch 3:  Loss:     0.7978 Validation Accuracy: 0.568000
Epoch 78, CIFAR-10 Batch 4:  Loss:     0.9942 Validation Accuracy: 0.571800
Epoch 78, CIFAR-10 Batch 5:  Loss:     0.8602 Validation Accuracy: 0.572200
Epoch 79, CIFAR-10 Batch 1:  Loss:     0.9647 Validation Accuracy: 0.571600
Epoch 79, CIFAR-10 Batch 2:  Loss:     0.7941 Validation Accuracy: 0.558400
Epoch 79, CIFAR-10 Batch 3:  Loss:     0.7844 Validation Accuracy: 0.569600
Epoch 79, CIFAR-10 Batch 4:  Loss:     1.0184 Validation Accuracy: 0.574200
Epoch 79, CIFAR-10 Batch 5:  Loss:     0.8574 Validation Accuracy: 0.569200
Epoch 80, CIFAR-10 Batch 1:  Loss:     0.9623 Validation Accuracy: 0.572200
Epoch 80, CIFAR-10 Batch 2:  Loss:     0.7789 Validation Accuracy: 0.552200
Epoch 80, CIFAR-10 Batch 3:  Loss:     0.7916 Validation Accuracy: 0.564200
Epoch 80, CIFAR-10 Batch 4:  Loss:     1.0008 Validation Accuracy: 0.573600
Epoch 80, CIFAR-10 Batch 5:  Loss:     0.8539 Validation Accuracy: 0.566400
Epoch 81, CIFAR-10 Batch 1:  Loss:     0.9609 Validation Accuracy: 0.571800
Epoch 81, CIFAR-10 Batch 2:  Loss:     0.7840 Validation Accuracy: 0.557400
Epoch 81, CIFAR-10 Batch 3:  Loss:     0.7939 Validation Accuracy: 0.567000
Epoch 81, CIFAR-10 Batch 4:  Loss:     0.9848 Validation Accuracy: 0.575200
Epoch 81, CIFAR-10 Batch 5:  Loss:     0.8473 Validation Accuracy: 0.574400
Epoch 82, CIFAR-10 Batch 1:  Loss:     0.9629 Validation Accuracy: 0.572600
Epoch 82, CIFAR-10 Batch 2:  Loss:     0.7837 Validation Accuracy: 0.561600
Epoch 82, CIFAR-10 Batch 3:  Loss:     0.7856 Validation Accuracy: 0.570200
Epoch 82, CIFAR-10 Batch 4:  Loss:     0.9895 Validation Accuracy: 0.571600
Epoch 82, CIFAR-10 Batch 5:  Loss:     0.8374 Validation Accuracy: 0.574000
Epoch 83, CIFAR-10 Batch 1:  Loss:     0.9509 Validation Accuracy: 0.572400
Epoch 83, CIFAR-10 Batch 2:  Loss:     0.7799 Validation Accuracy: 0.557800
Epoch 83, CIFAR-10 Batch 3:  Loss:     0.7854 Validation Accuracy: 0.571000
Epoch 83, CIFAR-10 Batch 4:  Loss:     1.0011 Validation Accuracy: 0.575000
Epoch 83, CIFAR-10 Batch 5:  Loss:     0.8469 Validation Accuracy: 0.573000
Epoch 84, CIFAR-10 Batch 1:  Loss:     0.9778 Validation Accuracy: 0.571000
Epoch 84, CIFAR-10 Batch 2:  Loss:     0.7848 Validation Accuracy: 0.559200
Epoch 84, CIFAR-10 Batch 3:  Loss:     0.7884 Validation Accuracy: 0.570200
Epoch 84, CIFAR-10 Batch 4:  Loss:     0.9946 Validation Accuracy: 0.574200
Epoch 84, CIFAR-10 Batch 5:  Loss:     0.8422 Validation Accuracy: 0.572200
Epoch 85, CIFAR-10 Batch 1:  Loss:     0.9517 Validation Accuracy: 0.571600
Epoch 85, CIFAR-10 Batch 2:  Loss:     0.7878 Validation Accuracy: 0.559200
Epoch 85, CIFAR-10 Batch 3:  Loss:     0.7856 Validation Accuracy: 0.570800
Epoch 85, CIFAR-10 Batch 4:  Loss:     0.9830 Validation Accuracy: 0.574800
Epoch 85, CIFAR-10 Batch 5:  Loss:     0.8459 Validation Accuracy: 0.574400
Epoch 86, CIFAR-10 Batch 1:  Loss:     0.9519 Validation Accuracy: 0.572800
Epoch 86, CIFAR-10 Batch 2:  Loss:     0.7820 Validation Accuracy: 0.559000
Epoch 86, CIFAR-10 Batch 3:  Loss:     0.7859 Validation Accuracy: 0.570200
Epoch 86, CIFAR-10 Batch 4:  Loss:     0.9892 Validation Accuracy: 0.575200
Epoch 86, CIFAR-10 Batch 5:  Loss:     0.8472 Validation Accuracy: 0.572400
Epoch 87, CIFAR-10 Batch 1:  Loss:     0.9399 Validation Accuracy: 0.571000
Epoch 87, CIFAR-10 Batch 2:  Loss:     0.7902 Validation Accuracy: 0.558000
Epoch 87, CIFAR-10 Batch 3:  Loss:     0.7974 Validation Accuracy: 0.566200
Epoch 87, CIFAR-10 Batch 4:  Loss:     0.9907 Validation Accuracy: 0.571000
Epoch 87, CIFAR-10 Batch 5:  Loss:     0.8361 Validation Accuracy: 0.572600
Epoch 88, CIFAR-10 Batch 1:  Loss:     0.9498 Validation Accuracy: 0.570800
Epoch 88, CIFAR-10 Batch 2:  Loss:     0.7839 Validation Accuracy: 0.557000
Epoch 88, CIFAR-10 Batch 3:  Loss:     0.7863 Validation Accuracy: 0.566800
Epoch 88, CIFAR-10 Batch 4:  Loss:     0.9824 Validation Accuracy: 0.573400
Epoch 88, CIFAR-10 Batch 5:  Loss:     0.8328 Validation Accuracy: 0.573800
Epoch 89, CIFAR-10 Batch 1:  Loss:     0.9454 Validation Accuracy: 0.572200
Epoch 89, CIFAR-10 Batch 2:  Loss:     0.7781 Validation Accuracy: 0.557600
Epoch 89, CIFAR-10 Batch 3:  Loss:     0.7814 Validation Accuracy: 0.568600
Epoch 89, CIFAR-10 Batch 4:  Loss:     0.9778 Validation Accuracy: 0.573000
Epoch 89, CIFAR-10 Batch 5:  Loss:     0.8381 Validation Accuracy: 0.576400
Epoch 90, CIFAR-10 Batch 1:  Loss:     0.9475 Validation Accuracy: 0.573600
Epoch 90, CIFAR-10 Batch 2:  Loss:     0.7738 Validation Accuracy: 0.561200
Epoch 90, CIFAR-10 Batch 3:  Loss:     0.7796 Validation Accuracy: 0.571200
Epoch 90, CIFAR-10 Batch 4:  Loss:     0.9893 Validation Accuracy: 0.572000
Epoch 90, CIFAR-10 Batch 5:  Loss:     0.8423 Validation Accuracy: 0.571800
Epoch 91, CIFAR-10 Batch 1:  Loss:     0.9478 Validation Accuracy: 0.571800
Epoch 91, CIFAR-10 Batch 2:  Loss:     0.7697 Validation Accuracy: 0.559800
Epoch 91, CIFAR-10 Batch 3:  Loss:     0.7762 Validation Accuracy: 0.568600
Epoch 91, CIFAR-10 Batch 4:  Loss:     0.9731 Validation Accuracy: 0.572800
Epoch 91, CIFAR-10 Batch 5:  Loss:     0.8406 Validation Accuracy: 0.570600
Epoch 92, CIFAR-10 Batch 1:  Loss:     0.9491 Validation Accuracy: 0.569600
Epoch 92, CIFAR-10 Batch 2:  Loss:     0.7712 Validation Accuracy: 0.555600
Epoch 92, CIFAR-10 Batch 3:  Loss:     0.7789 Validation Accuracy: 0.570200
Epoch 92, CIFAR-10 Batch 4:  Loss:     0.9818 Validation Accuracy: 0.572800
Epoch 92, CIFAR-10 Batch 5:  Loss:     0.8437 Validation Accuracy: 0.569800
Epoch 93, CIFAR-10 Batch 1:  Loss:     0.9533 Validation Accuracy: 0.573200
Epoch 93, CIFAR-10 Batch 2:  Loss:     0.7797 Validation Accuracy: 0.562000
Epoch 93, CIFAR-10 Batch 3:  Loss:     0.7822 Validation Accuracy: 0.572400
Epoch 93, CIFAR-10 Batch 4:  Loss:     0.9823 Validation Accuracy: 0.574000
Epoch 93, CIFAR-10 Batch 5:  Loss:     0.8294 Validation Accuracy: 0.572000
Epoch 94, CIFAR-10 Batch 1:  Loss:     0.9539 Validation Accuracy: 0.575200
Epoch 94, CIFAR-10 Batch 2:  Loss:     0.7674 Validation Accuracy: 0.559000
Epoch 94, CIFAR-10 Batch 3:  Loss:     0.7736 Validation Accuracy: 0.575200
Epoch 94, CIFAR-10 Batch 4:  Loss:     0.9733 Validation Accuracy: 0.575600
Epoch 94, CIFAR-10 Batch 5:  Loss:     0.8339 Validation Accuracy: 0.574000
Epoch 95, CIFAR-10 Batch 1:  Loss:     0.9377 Validation Accuracy: 0.573600
Epoch 95, CIFAR-10 Batch 2:  Loss:     0.7686 Validation Accuracy: 0.559400
Epoch 95, CIFAR-10 Batch 3:  Loss:     0.7819 Validation Accuracy: 0.571600
Epoch 95, CIFAR-10 Batch 4:  Loss:     0.9864 Validation Accuracy: 0.576600
Epoch 95, CIFAR-10 Batch 5:  Loss:     0.8314 Validation Accuracy: 0.574000
Epoch 96, CIFAR-10 Batch 1:  Loss:     0.9560 Validation Accuracy: 0.570400
Epoch 96, CIFAR-10 Batch 2:  Loss:     0.7709 Validation Accuracy: 0.555800
Epoch 96, CIFAR-10 Batch 3:  Loss:     0.7926 Validation Accuracy: 0.572000
Epoch 96, CIFAR-10 Batch 4:  Loss:     0.9793 Validation Accuracy: 0.572600
Epoch 96, CIFAR-10 Batch 5:  Loss:     0.8333 Validation Accuracy: 0.575800
Epoch 97, CIFAR-10 Batch 1:  Loss:     0.9366 Validation Accuracy: 0.574400
Epoch 97, CIFAR-10 Batch 2:  Loss:     0.7778 Validation Accuracy: 0.563600
Epoch 97, CIFAR-10 Batch 3:  Loss:     0.7845 Validation Accuracy: 0.572800
Epoch 97, CIFAR-10 Batch 4:  Loss:     0.9779 Validation Accuracy: 0.575800
Epoch 97, CIFAR-10 Batch 5:  Loss:     0.8268 Validation Accuracy: 0.579200
Epoch 98, CIFAR-10 Batch 1:  Loss:     0.9521 Validation Accuracy: 0.574000
Epoch 98, CIFAR-10 Batch 2:  Loss:     0.7805 Validation Accuracy: 0.559000
Epoch 98, CIFAR-10 Batch 3:  Loss:     0.7731 Validation Accuracy: 0.573600
Epoch 98, CIFAR-10 Batch 4:  Loss:     0.9695 Validation Accuracy: 0.579000
Epoch 98, CIFAR-10 Batch 5:  Loss:     0.8259 Validation Accuracy: 0.573800
Epoch 99, CIFAR-10 Batch 1:  Loss:     0.9357 Validation Accuracy: 0.572400
Epoch 99, CIFAR-10 Batch 2:  Loss:     0.7785 Validation Accuracy: 0.561800
Epoch 99, CIFAR-10 Batch 3:  Loss:     0.7797 Validation Accuracy: 0.572400
Epoch 99, CIFAR-10 Batch 4:  Loss:     0.9846 Validation Accuracy: 0.575200
Epoch 99, CIFAR-10 Batch 5:  Loss:     0.8270 Validation Accuracy: 0.574200
Epoch 100, CIFAR-10 Batch 1:  Loss:     0.9455 Validation Accuracy: 0.573200
Epoch 100, CIFAR-10 Batch 2:  Loss:     0.7735 Validation Accuracy: 0.560200
Epoch 100, CIFAR-10 Batch 3:  Loss:     0.7774 Validation Accuracy: 0.575800
Epoch 100, CIFAR-10 Batch 4:  Loss:     0.9756 Validation Accuracy: 0.574000
Epoch 100, CIFAR-10 Batch 5:  Loss:     0.8167 Validation Accuracy: 0.573200
Epoch 101, CIFAR-10 Batch 1:  Loss:     0.9450 Validation Accuracy: 0.572200
Epoch 101, CIFAR-10 Batch 2:  Loss:     0.7790 Validation Accuracy: 0.560600
Epoch 101, CIFAR-10 Batch 3:  Loss:     0.7779 Validation Accuracy: 0.576400
Epoch 101, CIFAR-10 Batch 4:  Loss:     0.9772 Validation Accuracy: 0.572000
Epoch 101, CIFAR-10 Batch 5:  Loss:     0.8205 Validation Accuracy: 0.573600
Epoch 102, CIFAR-10 Batch 1:  Loss:     0.9417 Validation Accuracy: 0.572600
Epoch 102, CIFAR-10 Batch 2:  Loss:     0.7759 Validation Accuracy: 0.560000
Epoch 102, CIFAR-10 Batch 3:  Loss:     0.7787 Validation Accuracy: 0.575000
Epoch 102, CIFAR-10 Batch 4:  Loss:     0.9842 Validation Accuracy: 0.574200
Epoch 102, CIFAR-10 Batch 5:  Loss:     0.8161 Validation Accuracy: 0.573800
Epoch 103, CIFAR-10 Batch 1:  Loss:     0.9580 Validation Accuracy: 0.575800
Epoch 103, CIFAR-10 Batch 2:  Loss:     0.7741 Validation Accuracy: 0.561000
Epoch 103, CIFAR-10 Batch 3:  Loss:     0.7786 Validation Accuracy: 0.572800
Epoch 103, CIFAR-10 Batch 4:  Loss:     0.9695 Validation Accuracy: 0.575200
Epoch 103, CIFAR-10 Batch 5:  Loss:     0.8169 Validation Accuracy: 0.576200
Epoch 104, CIFAR-10 Batch 1:  Loss:     0.9396 Validation Accuracy: 0.574400
Epoch 104, CIFAR-10 Batch 2:  Loss:     0.7705 Validation Accuracy: 0.558800
Epoch 104, CIFAR-10 Batch 3:  Loss:     0.7788 Validation Accuracy: 0.568600
Epoch 104, CIFAR-10 Batch 4:  Loss:     0.9808 Validation Accuracy: 0.575800
Epoch 104, CIFAR-10 Batch 5:  Loss:     0.8088 Validation Accuracy: 0.575000
Epoch 105, CIFAR-10 Batch 1:  Loss:     0.9441 Validation Accuracy: 0.574800
Epoch 105, CIFAR-10 Batch 2:  Loss:     0.7719 Validation Accuracy: 0.565600
Epoch 105, CIFAR-10 Batch 3:  Loss:     0.7653 Validation Accuracy: 0.574000
Epoch 105, CIFAR-10 Batch 4:  Loss:     0.9757 Validation Accuracy: 0.574600
Epoch 105, CIFAR-10 Batch 5:  Loss:     0.8115 Validation Accuracy: 0.572000
Epoch 106, CIFAR-10 Batch 1:  Loss:     0.9349 Validation Accuracy: 0.574400
Epoch 106, CIFAR-10 Batch 2:  Loss:     0.7685 Validation Accuracy: 0.560800
Epoch 106, CIFAR-10 Batch 3:  Loss:     0.7743 Validation Accuracy: 0.574800
Epoch 106, CIFAR-10 Batch 4:  Loss:     0.9707 Validation Accuracy: 0.577000
Epoch 106, CIFAR-10 Batch 5:  Loss:     0.8106 Validation Accuracy: 0.577000
Epoch 107, CIFAR-10 Batch 1:  Loss:     0.9405 Validation Accuracy: 0.576400
Epoch 107, CIFAR-10 Batch 2:  Loss:     0.7732 Validation Accuracy: 0.563000
Epoch 107, CIFAR-10 Batch 3:  Loss:     0.7773 Validation Accuracy: 0.573800
Epoch 107, CIFAR-10 Batch 4:  Loss:     0.9801 Validation Accuracy: 0.577400
Epoch 107, CIFAR-10 Batch 5:  Loss:     0.8115 Validation Accuracy: 0.580600
Epoch 108, CIFAR-10 Batch 1:  Loss:     0.9325 Validation Accuracy: 0.573600
Epoch 108, CIFAR-10 Batch 2:  Loss:     0.7708 Validation Accuracy: 0.562200
Epoch 108, CIFAR-10 Batch 3:  Loss:     0.7731 Validation Accuracy: 0.574600
Epoch 108, CIFAR-10 Batch 4:  Loss:     0.9856 Validation Accuracy: 0.577200
Epoch 108, CIFAR-10 Batch 5:  Loss:     0.8103 Validation Accuracy: 0.576400
Epoch 109, CIFAR-10 Batch 1:  Loss:     0.9337 Validation Accuracy: 0.576600
Epoch 109, CIFAR-10 Batch 2:  Loss:     0.7672 Validation Accuracy: 0.563800
Epoch 109, CIFAR-10 Batch 3:  Loss:     0.7716 Validation Accuracy: 0.577600
Epoch 109, CIFAR-10 Batch 4:  Loss:     0.9685 Validation Accuracy: 0.576200
Epoch 109, CIFAR-10 Batch 5:  Loss:     0.8089 Validation Accuracy: 0.573600
Epoch 110, CIFAR-10 Batch 1:  Loss:     0.9175 Validation Accuracy: 0.576800
Epoch 110, CIFAR-10 Batch 2:  Loss:     0.7699 Validation Accuracy: 0.562600
Epoch 110, CIFAR-10 Batch 3:  Loss:     0.7743 Validation Accuracy: 0.578400
Epoch 110, CIFAR-10 Batch 4:  Loss:     0.9710 Validation Accuracy: 0.575800
Epoch 110, CIFAR-10 Batch 5:  Loss:     0.8064 Validation Accuracy: 0.576600
Epoch 111, CIFAR-10 Batch 1:  Loss:     0.9302 Validation Accuracy: 0.576200
Epoch 111, CIFAR-10 Batch 2:  Loss:     0.7662 Validation Accuracy: 0.561400
Epoch 111, CIFAR-10 Batch 3:  Loss:     0.7742 Validation Accuracy: 0.576000
Epoch 111, CIFAR-10 Batch 4:  Loss:     0.9684 Validation Accuracy: 0.578200
Epoch 111, CIFAR-10 Batch 5:  Loss:     0.8076 Validation Accuracy: 0.575800
Epoch 112, CIFAR-10 Batch 1:  Loss:     0.9292 Validation Accuracy: 0.573000
Epoch 112, CIFAR-10 Batch 2:  Loss:     0.7676 Validation Accuracy: 0.557200
Epoch 112, CIFAR-10 Batch 3:  Loss:     0.7741 Validation Accuracy: 0.575000
Epoch 112, CIFAR-10 Batch 4:  Loss:     0.9606 Validation Accuracy: 0.578800
Epoch 112, CIFAR-10 Batch 5:  Loss:     0.8059 Validation Accuracy: 0.572000
Epoch 113, CIFAR-10 Batch 1:  Loss:     0.9282 Validation Accuracy: 0.575800
Epoch 113, CIFAR-10 Batch 2:  Loss:     0.7685 Validation Accuracy: 0.564600
Epoch 113, CIFAR-10 Batch 3:  Loss:     0.7709 Validation Accuracy: 0.571400
Epoch 113, CIFAR-10 Batch 4:  Loss:     0.9622 Validation Accuracy: 0.577400
Epoch 113, CIFAR-10 Batch 5:  Loss:     0.8080 Validation Accuracy: 0.572000
Epoch 114, CIFAR-10 Batch 1:  Loss:     0.9524 Validation Accuracy: 0.578800
Epoch 114, CIFAR-10 Batch 2:  Loss:     0.7709 Validation Accuracy: 0.562600
Epoch 114, CIFAR-10 Batch 3:  Loss:     0.7744 Validation Accuracy: 0.572000
Epoch 114, CIFAR-10 Batch 4:  Loss:     0.9667 Validation Accuracy: 0.576400
Epoch 114, CIFAR-10 Batch 5:  Loss:     0.7964 Validation Accuracy: 0.574800
Epoch 115, CIFAR-10 Batch 1:  Loss:     0.9100 Validation Accuracy: 0.576200
Epoch 115, CIFAR-10 Batch 2:  Loss:     0.7748 Validation Accuracy: 0.563600
Epoch 115, CIFAR-10 Batch 3:  Loss:     0.7663 Validation Accuracy: 0.574200
Epoch 115, CIFAR-10 Batch 4:  Loss:     0.9688 Validation Accuracy: 0.575200
Epoch 115, CIFAR-10 Batch 5:  Loss:     0.7996 Validation Accuracy: 0.577400
Epoch 116, CIFAR-10 Batch 1:  Loss:     0.9332 Validation Accuracy: 0.575200
Epoch 116, CIFAR-10 Batch 2:  Loss:     0.7746 Validation Accuracy: 0.564000
Epoch 116, CIFAR-10 Batch 3:  Loss:     0.7732 Validation Accuracy: 0.576400
Epoch 116, CIFAR-10 Batch 4:  Loss:     0.9756 Validation Accuracy: 0.575400
Epoch 116, CIFAR-10 Batch 5:  Loss:     0.8034 Validation Accuracy: 0.577400
Epoch 117, CIFAR-10 Batch 1:  Loss:     0.9155 Validation Accuracy: 0.574400
Epoch 117, CIFAR-10 Batch 2:  Loss:     0.7629 Validation Accuracy: 0.562600
Epoch 117, CIFAR-10 Batch 3:  Loss:     0.7700 Validation Accuracy: 0.578600
Epoch 117, CIFAR-10 Batch 4:  Loss:     0.9657 Validation Accuracy: 0.576600
Epoch 117, CIFAR-10 Batch 5:  Loss:     0.8042 Validation Accuracy: 0.577600
Epoch 118, CIFAR-10 Batch 1:  Loss:     0.9197 Validation Accuracy: 0.574000
Epoch 118, CIFAR-10 Batch 2:  Loss:     0.7706 Validation Accuracy: 0.565400
Epoch 118, CIFAR-10 Batch 3:  Loss:     0.7801 Validation Accuracy: 0.576800
Epoch 118, CIFAR-10 Batch 4:  Loss:     0.9606 Validation Accuracy: 0.575600
Epoch 118, CIFAR-10 Batch 5:  Loss:     0.7937 Validation Accuracy: 0.575000
Epoch 119, CIFAR-10 Batch 1:  Loss:     0.9073 Validation Accuracy: 0.575600
Epoch 119, CIFAR-10 Batch 2:  Loss:     0.7660 Validation Accuracy: 0.562000
Epoch 119, CIFAR-10 Batch 3:  Loss:     0.7797 Validation Accuracy: 0.580600
Epoch 119, CIFAR-10 Batch 4:  Loss:     0.9601 Validation Accuracy: 0.578800
Epoch 119, CIFAR-10 Batch 5:  Loss:     0.7903 Validation Accuracy: 0.573400
Epoch 120, CIFAR-10 Batch 1:  Loss:     0.9209 Validation Accuracy: 0.575600
Epoch 120, CIFAR-10 Batch 2:  Loss:     0.7690 Validation Accuracy: 0.563600
Epoch 120, CIFAR-10 Batch 3:  Loss:     0.7731 Validation Accuracy: 0.580400
Epoch 120, CIFAR-10 Batch 4:  Loss:     0.9631 Validation Accuracy: 0.578600
Epoch 120, CIFAR-10 Batch 5:  Loss:     0.7891 Validation Accuracy: 0.576800
Epoch 121, CIFAR-10 Batch 1:  Loss:     0.9138 Validation Accuracy: 0.572400
Epoch 121, CIFAR-10 Batch 2:  Loss:     0.7600 Validation Accuracy: 0.562600
Epoch 121, CIFAR-10 Batch 3:  Loss:     0.7619 Validation Accuracy: 0.577000
Epoch 121, CIFAR-10 Batch 4:  Loss:     0.9627 Validation Accuracy: 0.580400
Epoch 121, CIFAR-10 Batch 5:  Loss:     0.7945 Validation Accuracy: 0.572800
Epoch 122, CIFAR-10 Batch 1:  Loss:     0.9052 Validation Accuracy: 0.573800
Epoch 122, CIFAR-10 Batch 2:  Loss:     0.7671 Validation Accuracy: 0.567600
Epoch 122, CIFAR-10 Batch 3:  Loss:     0.7770 Validation Accuracy: 0.576800
Epoch 122, CIFAR-10 Batch 4:  Loss:     0.9664 Validation Accuracy: 0.577000
Epoch 122, CIFAR-10 Batch 5:  Loss:     0.7872 Validation Accuracy: 0.578000
Epoch 123, CIFAR-10 Batch 1:  Loss:     0.9079 Validation Accuracy: 0.576200
Epoch 123, CIFAR-10 Batch 2:  Loss:     0.7721 Validation Accuracy: 0.564000
Epoch 123, CIFAR-10 Batch 3:  Loss:     0.7797 Validation Accuracy: 0.575600
Epoch 123, CIFAR-10 Batch 4:  Loss:     0.9528 Validation Accuracy: 0.578800
Epoch 123, CIFAR-10 Batch 5:  Loss:     0.7793 Validation Accuracy: 0.577600
Epoch 124, CIFAR-10 Batch 1:  Loss:     0.9196 Validation Accuracy: 0.578000
Epoch 124, CIFAR-10 Batch 2:  Loss:     0.7611 Validation Accuracy: 0.566000
Epoch 124, CIFAR-10 Batch 3:  Loss:     0.7695 Validation Accuracy: 0.575000
Epoch 124, CIFAR-10 Batch 4:  Loss:     0.9745 Validation Accuracy: 0.577200
Epoch 124, CIFAR-10 Batch 5:  Loss:     0.7836 Validation Accuracy: 0.576400
Epoch 125, CIFAR-10 Batch 1:  Loss:     0.9199 Validation Accuracy: 0.573800
Epoch 125, CIFAR-10 Batch 2:  Loss:     0.7581 Validation Accuracy: 0.563000
Epoch 125, CIFAR-10 Batch 3:  Loss:     0.7799 Validation Accuracy: 0.572200
Epoch 125, CIFAR-10 Batch 4:  Loss:     0.9504 Validation Accuracy: 0.580400
Epoch 125, CIFAR-10 Batch 5:  Loss:     0.7888 Validation Accuracy: 0.576200
Epoch 126, CIFAR-10 Batch 1:  Loss:     0.9005 Validation Accuracy: 0.573600
Epoch 126, CIFAR-10 Batch 2:  Loss:     0.7608 Validation Accuracy: 0.567400
Epoch 126, CIFAR-10 Batch 3:  Loss:     0.7912 Validation Accuracy: 0.573400
Epoch 126, CIFAR-10 Batch 4:  Loss:     0.9493 Validation Accuracy: 0.581400
Epoch 126, CIFAR-10 Batch 5:  Loss:     0.7868 Validation Accuracy: 0.577400
Epoch 127, CIFAR-10 Batch 1:  Loss:     0.9016 Validation Accuracy: 0.575800
Epoch 127, CIFAR-10 Batch 2:  Loss:     0.7603 Validation Accuracy: 0.565000
Epoch 127, CIFAR-10 Batch 3:  Loss:     0.7720 Validation Accuracy: 0.574200
Epoch 127, CIFAR-10 Batch 4:  Loss:     0.9584 Validation Accuracy: 0.579200
Epoch 127, CIFAR-10 Batch 5:  Loss:     0.7879 Validation Accuracy: 0.575200
Epoch 128, CIFAR-10 Batch 1:  Loss:     0.9133 Validation Accuracy: 0.577000
Epoch 128, CIFAR-10 Batch 2:  Loss:     0.7570 Validation Accuracy: 0.566600
Epoch 128, CIFAR-10 Batch 3:  Loss:     0.7694 Validation Accuracy: 0.575000
Epoch 128, CIFAR-10 Batch 4:  Loss:     0.9544 Validation Accuracy: 0.577600
Epoch 128, CIFAR-10 Batch 5:  Loss:     0.7852 Validation Accuracy: 0.577000
Epoch 129, CIFAR-10 Batch 1:  Loss:     0.8920 Validation Accuracy: 0.576600
Epoch 129, CIFAR-10 Batch 2:  Loss:     0.7487 Validation Accuracy: 0.565000
Epoch 129, CIFAR-10 Batch 3:  Loss:     0.7648 Validation Accuracy: 0.575600
Epoch 129, CIFAR-10 Batch 4:  Loss:     0.9492 Validation Accuracy: 0.578000
Epoch 129, CIFAR-10 Batch 5:  Loss:     0.7770 Validation Accuracy: 0.575200
Epoch 130, CIFAR-10 Batch 1:  Loss:     0.9082 Validation Accuracy: 0.575600
Epoch 130, CIFAR-10 Batch 2:  Loss:     0.7439 Validation Accuracy: 0.565000
Epoch 130, CIFAR-10 Batch 3:  Loss:     0.7622 Validation Accuracy: 0.576800
Epoch 130, CIFAR-10 Batch 4:  Loss:     0.9482 Validation Accuracy: 0.578000
Epoch 130, CIFAR-10 Batch 5:  Loss:     0.7725 Validation Accuracy: 0.576000
Epoch 131, CIFAR-10 Batch 1:  Loss:     0.9045 Validation Accuracy: 0.575000
Epoch 131, CIFAR-10 Batch 2:  Loss:     0.7558 Validation Accuracy: 0.563800
Epoch 131, CIFAR-10 Batch 3:  Loss:     0.7603 Validation Accuracy: 0.577000
Epoch 131, CIFAR-10 Batch 4:  Loss:     0.9423 Validation Accuracy: 0.578000
Epoch 131, CIFAR-10 Batch 5:  Loss:     0.7908 Validation Accuracy: 0.575200
Epoch 132, CIFAR-10 Batch 1:  Loss:     0.8964 Validation Accuracy: 0.577200
Epoch 132, CIFAR-10 Batch 2:  Loss:     0.7426 Validation Accuracy: 0.569600
Epoch 132, CIFAR-10 Batch 3:  Loss:     0.7729 Validation Accuracy: 0.575000
Epoch 132, CIFAR-10 Batch 4:  Loss:     0.9474 Validation Accuracy: 0.580000
Epoch 132, CIFAR-10 Batch 5:  Loss:     0.7737 Validation Accuracy: 0.577400
Epoch 133, CIFAR-10 Batch 1:  Loss:     0.9046 Validation Accuracy: 0.580400
Epoch 133, CIFAR-10 Batch 2:  Loss:     0.7535 Validation Accuracy: 0.570000
Epoch 133, CIFAR-10 Batch 3:  Loss:     0.7712 Validation Accuracy: 0.576000
Epoch 133, CIFAR-10 Batch 4:  Loss:     0.9597 Validation Accuracy: 0.576800
Epoch 133, CIFAR-10 Batch 5:  Loss:     0.7671 Validation Accuracy: 0.575000
Epoch 134, CIFAR-10 Batch 1:  Loss:     0.9179 Validation Accuracy: 0.581800
Epoch 134, CIFAR-10 Batch 2:  Loss:     0.7493 Validation Accuracy: 0.566400
Epoch 134, CIFAR-10 Batch 3:  Loss:     0.7647 Validation Accuracy: 0.576800
Epoch 134, CIFAR-10 Batch 4:  Loss:     0.9359 Validation Accuracy: 0.577400
Epoch 134, CIFAR-10 Batch 5:  Loss:     0.7806 Validation Accuracy: 0.575000
Epoch 135, CIFAR-10 Batch 1:  Loss:     0.9028 Validation Accuracy: 0.578600
Epoch 135, CIFAR-10 Batch 2:  Loss:     0.7508 Validation Accuracy: 0.566800
Epoch 135, CIFAR-10 Batch 3:  Loss:     0.7657 Validation Accuracy: 0.578800
Epoch 135, CIFAR-10 Batch 4:  Loss:     0.9497 Validation Accuracy: 0.579000
Epoch 135, CIFAR-10 Batch 5:  Loss:     0.7749 Validation Accuracy: 0.576200
Epoch 136, CIFAR-10 Batch 1:  Loss:     0.9194 Validation Accuracy: 0.581200
Epoch 136, CIFAR-10 Batch 2:  Loss:     0.7481 Validation Accuracy: 0.568200
Epoch 136, CIFAR-10 Batch 3:  Loss:     0.7677 Validation Accuracy: 0.575800
Epoch 136, CIFAR-10 Batch 4:  Loss:     0.9554 Validation Accuracy: 0.577400
Epoch 136, CIFAR-10 Batch 5:  Loss:     0.7726 Validation Accuracy: 0.578000
Epoch 137, CIFAR-10 Batch 1:  Loss:     0.9114 Validation Accuracy: 0.578400
Epoch 137, CIFAR-10 Batch 2:  Loss:     0.7513 Validation Accuracy: 0.568600
Epoch 137, CIFAR-10 Batch 3:  Loss:     0.7659 Validation Accuracy: 0.574800
Epoch 137, CIFAR-10 Batch 4:  Loss:     0.9364 Validation Accuracy: 0.583400
Epoch 137, CIFAR-10 Batch 5:  Loss:     0.7796 Validation Accuracy: 0.575200
Epoch 138, CIFAR-10 Batch 1:  Loss:     0.8859 Validation Accuracy: 0.579000
Epoch 138, CIFAR-10 Batch 2:  Loss:     0.7441 Validation Accuracy: 0.570600
Epoch 138, CIFAR-10 Batch 3:  Loss:     0.7678 Validation Accuracy: 0.577200
Epoch 138, CIFAR-10 Batch 4:  Loss:     0.9374 Validation Accuracy: 0.575800
Epoch 138, CIFAR-10 Batch 5:  Loss:     0.7619 Validation Accuracy: 0.579200
Epoch 139, CIFAR-10 Batch 1:  Loss:     0.8900 Validation Accuracy: 0.579200
Epoch 139, CIFAR-10 Batch 2:  Loss:     0.7392 Validation Accuracy: 0.568600
Epoch 139, CIFAR-10 Batch 3:  Loss:     0.7707 Validation Accuracy: 0.578200
Epoch 139, CIFAR-10 Batch 4:  Loss:     0.9337 Validation Accuracy: 0.580200
Epoch 139, CIFAR-10 Batch 5:  Loss:     0.7634 Validation Accuracy: 0.578000
Epoch 140, CIFAR-10 Batch 1:  Loss:     0.8983 Validation Accuracy: 0.576000
Epoch 140, CIFAR-10 Batch 2:  Loss:     0.7393 Validation Accuracy: 0.567000
Epoch 140, CIFAR-10 Batch 3:  Loss:     0.7568 Validation Accuracy: 0.579000
Epoch 140, CIFAR-10 Batch 4:  Loss:     0.9367 Validation Accuracy: 0.576800
Epoch 140, CIFAR-10 Batch 5:  Loss:     0.7655 Validation Accuracy: 0.579600
Epoch 141, CIFAR-10 Batch 1:  Loss:     0.9123 Validation Accuracy: 0.579400
Epoch 141, CIFAR-10 Batch 2:  Loss:     0.7417 Validation Accuracy: 0.563400
Epoch 141, CIFAR-10 Batch 3:  Loss:     0.7638 Validation Accuracy: 0.577400
Epoch 141, CIFAR-10 Batch 4:  Loss:     0.9495 Validation Accuracy: 0.578000
Epoch 141, CIFAR-10 Batch 5:  Loss:     0.7728 Validation Accuracy: 0.576800
Epoch 142, CIFAR-10 Batch 1:  Loss:     0.8990 Validation Accuracy: 0.579600
Epoch 142, CIFAR-10 Batch 2:  Loss:     0.7358 Validation Accuracy: 0.568400
Epoch 142, CIFAR-10 Batch 3:  Loss:     0.7666 Validation Accuracy: 0.576200
Epoch 142, CIFAR-10 Batch 4:  Loss:     0.9451 Validation Accuracy: 0.578200
Epoch 142, CIFAR-10 Batch 5:  Loss:     0.7774 Validation Accuracy: 0.576200
Epoch 143, CIFAR-10 Batch 1:  Loss:     0.9012 Validation Accuracy: 0.578400
Epoch 143, CIFAR-10 Batch 2:  Loss:     0.7459 Validation Accuracy: 0.568000
Epoch 143, CIFAR-10 Batch 3:  Loss:     0.7658 Validation Accuracy: 0.579200
Epoch 143, CIFAR-10 Batch 4:  Loss:     0.9472 Validation Accuracy: 0.579400
Epoch 143, CIFAR-10 Batch 5:  Loss:     0.7732 Validation Accuracy: 0.576200
Epoch 144, CIFAR-10 Batch 1:  Loss:     0.8958 Validation Accuracy: 0.580800
Epoch 144, CIFAR-10 Batch 2:  Loss:     0.7417 Validation Accuracy: 0.564200
Epoch 144, CIFAR-10 Batch 3:  Loss:     0.7600 Validation Accuracy: 0.579400
Epoch 144, CIFAR-10 Batch 4:  Loss:     0.9410 Validation Accuracy: 0.578600
Epoch 144, CIFAR-10 Batch 5:  Loss:     0.7625 Validation Accuracy: 0.576600
Epoch 145, CIFAR-10 Batch 1:  Loss:     0.9013 Validation Accuracy: 0.579200
Epoch 145, CIFAR-10 Batch 2:  Loss:     0.7331 Validation Accuracy: 0.568200
Epoch 145, CIFAR-10 Batch 3:  Loss:     0.7664 Validation Accuracy: 0.577200
Epoch 145, CIFAR-10 Batch 4:  Loss:     0.9401 Validation Accuracy: 0.579200
Epoch 145, CIFAR-10 Batch 5:  Loss:     0.7648 Validation Accuracy: 0.575400
Epoch 146, CIFAR-10 Batch 1:  Loss:     0.9078 Validation Accuracy: 0.578600
Epoch 146, CIFAR-10 Batch 2:  Loss:     0.7464 Validation Accuracy: 0.570400
Epoch 146, CIFAR-10 Batch 3:  Loss:     0.7642 Validation Accuracy: 0.579200
Epoch 146, CIFAR-10 Batch 4:  Loss:     0.9454 Validation Accuracy: 0.580400
Epoch 146, CIFAR-10 Batch 5:  Loss:     0.7652 Validation Accuracy: 0.576400
Epoch 147, CIFAR-10 Batch 1:  Loss:     0.8950 Validation Accuracy: 0.578200
Epoch 147, CIFAR-10 Batch 2:  Loss:     0.7338 Validation Accuracy: 0.570800
Epoch 147, CIFAR-10 Batch 3:  Loss:     0.7657 Validation Accuracy: 0.579200
Epoch 147, CIFAR-10 Batch 4:  Loss:     0.9342 Validation Accuracy: 0.579800
Epoch 147, CIFAR-10 Batch 5:  Loss:     0.7548 Validation Accuracy: 0.578600
Epoch 148, CIFAR-10 Batch 1:  Loss:     0.8939 Validation Accuracy: 0.580200
Epoch 148, CIFAR-10 Batch 2:  Loss:     0.7408 Validation Accuracy: 0.570800
Epoch 148, CIFAR-10 Batch 3:  Loss:     0.7680 Validation Accuracy: 0.579400
Epoch 148, CIFAR-10 Batch 4:  Loss:     0.9301 Validation Accuracy: 0.579200
Epoch 148, CIFAR-10 Batch 5:  Loss:     0.7550 Validation Accuracy: 0.578200
Epoch 149, CIFAR-10 Batch 1:  Loss:     0.8880 Validation Accuracy: 0.579800
Epoch 149, CIFAR-10 Batch 2:  Loss:     0.7377 Validation Accuracy: 0.569200
Epoch 149, CIFAR-10 Batch 3:  Loss:     0.7622 Validation Accuracy: 0.577800
Epoch 149, CIFAR-10 Batch 4:  Loss:     0.9263 Validation Accuracy: 0.581600
Epoch 149, CIFAR-10 Batch 5:  Loss:     0.7552 Validation Accuracy: 0.576600
Epoch 150, CIFAR-10 Batch 1:  Loss:     0.8947 Validation Accuracy: 0.580200
Epoch 150, CIFAR-10 Batch 2:  Loss:     0.7287 Validation Accuracy: 0.568600
Epoch 150, CIFAR-10 Batch 3:  Loss:     0.7646 Validation Accuracy: 0.583200
Epoch 150, CIFAR-10 Batch 4:  Loss:     0.9235 Validation Accuracy: 0.583000
Epoch 150, CIFAR-10 Batch 5:  Loss:     0.7645 Validation Accuracy: 0.577400
Epoch 151, CIFAR-10 Batch 1:  Loss:     0.8816 Validation Accuracy: 0.580200
Epoch 151, CIFAR-10 Batch 2:  Loss:     0.7449 Validation Accuracy: 0.570400
Epoch 151, CIFAR-10 Batch 3:  Loss:     0.7676 Validation Accuracy: 0.580600
Epoch 151, CIFAR-10 Batch 4:  Loss:     0.9389 Validation Accuracy: 0.577200
Epoch 151, CIFAR-10 Batch 5:  Loss:     0.7614 Validation Accuracy: 0.574400
Epoch 152, CIFAR-10 Batch 1:  Loss:     0.8971 Validation Accuracy: 0.582800
Epoch 152, CIFAR-10 Batch 2:  Loss:     0.7352 Validation Accuracy: 0.572400
Epoch 152, CIFAR-10 Batch 3:  Loss:     0.7787 Validation Accuracy: 0.581200
Epoch 152, CIFAR-10 Batch 4:  Loss:     0.9317 Validation Accuracy: 0.579800
Epoch 152, CIFAR-10 Batch 5:  Loss:     0.7491 Validation Accuracy: 0.576200
Epoch 153, CIFAR-10 Batch 1:  Loss:     0.9058 Validation Accuracy: 0.580000
Epoch 153, CIFAR-10 Batch 2:  Loss:     0.7402 Validation Accuracy: 0.570400
Epoch 153, CIFAR-10 Batch 3:  Loss:     0.7600 Validation Accuracy: 0.581600
Epoch 153, CIFAR-10 Batch 4:  Loss:     0.9160 Validation Accuracy: 0.581000
Epoch 153, CIFAR-10 Batch 5:  Loss:     0.7534 Validation Accuracy: 0.578000
Epoch 154, CIFAR-10 Batch 1:  Loss:     0.8889 Validation Accuracy: 0.582000
Epoch 154, CIFAR-10 Batch 2:  Loss:     0.7387 Validation Accuracy: 0.573200
Epoch 154, CIFAR-10 Batch 3:  Loss:     0.7586 Validation Accuracy: 0.581800
Epoch 154, CIFAR-10 Batch 4:  Loss:     0.9223 Validation Accuracy: 0.579400
Epoch 154, CIFAR-10 Batch 5:  Loss:     0.7476 Validation Accuracy: 0.579000
Epoch 155, CIFAR-10 Batch 1:  Loss:     0.9003 Validation Accuracy: 0.582000
Epoch 155, CIFAR-10 Batch 2:  Loss:     0.7324 Validation Accuracy: 0.573000
Epoch 155, CIFAR-10 Batch 3:  Loss:     0.7638 Validation Accuracy: 0.579000
Epoch 155, CIFAR-10 Batch 4:  Loss:     0.9352 Validation Accuracy: 0.579400
Epoch 155, CIFAR-10 Batch 5:  Loss:     0.7488 Validation Accuracy: 0.575400
Epoch 156, CIFAR-10 Batch 1:  Loss:     0.8983 Validation Accuracy: 0.581800
Epoch 156, CIFAR-10 Batch 2:  Loss:     0.7368 Validation Accuracy: 0.571800
Epoch 156, CIFAR-10 Batch 3:  Loss:     0.7672 Validation Accuracy: 0.576400
Epoch 156, CIFAR-10 Batch 4:  Loss:     0.9315 Validation Accuracy: 0.579600
Epoch 156, CIFAR-10 Batch 5:  Loss:     0.7391 Validation Accuracy: 0.576600
Epoch 157, CIFAR-10 Batch 1:  Loss:     0.8817 Validation Accuracy: 0.580400
Epoch 157, CIFAR-10 Batch 2:  Loss:     0.7304 Validation Accuracy: 0.571400
Epoch 157, CIFAR-10 Batch 3:  Loss:     0.7748 Validation Accuracy: 0.577000
Epoch 157, CIFAR-10 Batch 4:  Loss:     0.9239 Validation Accuracy: 0.582200
Epoch 157, CIFAR-10 Batch 5:  Loss:     0.7516 Validation Accuracy: 0.578600
Epoch 158, CIFAR-10 Batch 1:  Loss:     0.8868 Validation Accuracy: 0.583800
Epoch 158, CIFAR-10 Batch 2:  Loss:     0.7351 Validation Accuracy: 0.573400
Epoch 158, CIFAR-10 Batch 3:  Loss:     0.7767 Validation Accuracy: 0.579400
Epoch 158, CIFAR-10 Batch 4:  Loss:     0.9170 Validation Accuracy: 0.579400
Epoch 158, CIFAR-10 Batch 5:  Loss:     0.7447 Validation Accuracy: 0.576600
Epoch 159, CIFAR-10 Batch 1:  Loss:     0.9080 Validation Accuracy: 0.582000
Epoch 159, CIFAR-10 Batch 2:  Loss:     0.7239 Validation Accuracy: 0.570200
Epoch 159, CIFAR-10 Batch 3:  Loss:     0.7756 Validation Accuracy: 0.575600
Epoch 159, CIFAR-10 Batch 4:  Loss:     0.9126 Validation Accuracy: 0.579400
Epoch 159, CIFAR-10 Batch 5:  Loss:     0.7449 Validation Accuracy: 0.576600
Epoch 160, CIFAR-10 Batch 1:  Loss:     0.8967 Validation Accuracy: 0.579000
Epoch 160, CIFAR-10 Batch 2:  Loss:     0.7336 Validation Accuracy: 0.572200
Epoch 160, CIFAR-10 Batch 3:  Loss:     0.7789 Validation Accuracy: 0.576200
Epoch 160, CIFAR-10 Batch 4:  Loss:     0.9323 Validation Accuracy: 0.582200
Epoch 160, CIFAR-10 Batch 5:  Loss:     0.7465 Validation Accuracy: 0.578200
Epoch 161, CIFAR-10 Batch 1:  Loss:     0.8901 Validation Accuracy: 0.581000
Epoch 161, CIFAR-10 Batch 2:  Loss:     0.7323 Validation Accuracy: 0.569400
Epoch 161, CIFAR-10 Batch 3:  Loss:     0.7764 Validation Accuracy: 0.578200
Epoch 161, CIFAR-10 Batch 4:  Loss:     0.9133 Validation Accuracy: 0.581400
Epoch 161, CIFAR-10 Batch 5:  Loss:     0.7512 Validation Accuracy: 0.578000
Epoch 162, CIFAR-10 Batch 1:  Loss:     0.8801 Validation Accuracy: 0.584200
Epoch 162, CIFAR-10 Batch 2:  Loss:     0.7344 Validation Accuracy: 0.570600
Epoch 162, CIFAR-10 Batch 3:  Loss:     0.7753 Validation Accuracy: 0.580600
Epoch 162, CIFAR-10 Batch 4:  Loss:     0.9139 Validation Accuracy: 0.582200
Epoch 162, CIFAR-10 Batch 5:  Loss:     0.7399 Validation Accuracy: 0.578400
Epoch 163, CIFAR-10 Batch 1:  Loss:     0.8726 Validation Accuracy: 0.583200
Epoch 163, CIFAR-10 Batch 2:  Loss:     0.7239 Validation Accuracy: 0.570600
Epoch 163, CIFAR-10 Batch 3:  Loss:     0.7698 Validation Accuracy: 0.579200
Epoch 163, CIFAR-10 Batch 4:  Loss:     0.9095 Validation Accuracy: 0.581800
Epoch 163, CIFAR-10 Batch 5:  Loss:     0.7358 Validation Accuracy: 0.579000
Epoch 164, CIFAR-10 Batch 1:  Loss:     0.8910 Validation Accuracy: 0.582200
Epoch 164, CIFAR-10 Batch 2:  Loss:     0.7338 Validation Accuracy: 0.575400
Epoch 164, CIFAR-10 Batch 3:  Loss:     0.7764 Validation Accuracy: 0.579200
Epoch 164, CIFAR-10 Batch 4:  Loss:     0.9158 Validation Accuracy: 0.580800
Epoch 164, CIFAR-10 Batch 5:  Loss:     0.7368 Validation Accuracy: 0.577400
Epoch 165, CIFAR-10 Batch 1:  Loss:     0.8788 Validation Accuracy: 0.584800
Epoch 165, CIFAR-10 Batch 2:  Loss:     0.7247 Validation Accuracy: 0.572400
Epoch 165, CIFAR-10 Batch 3:  Loss:     0.7738 Validation Accuracy: 0.578000
Epoch 165, CIFAR-10 Batch 4:  Loss:     0.9261 Validation Accuracy: 0.580400
Epoch 165, CIFAR-10 Batch 5:  Loss:     0.7417 Validation Accuracy: 0.576400
Epoch 166, CIFAR-10 Batch 1:  Loss:     0.8962 Validation Accuracy: 0.581800
Epoch 166, CIFAR-10 Batch 2:  Loss:     0.7276 Validation Accuracy: 0.572600
Epoch 166, CIFAR-10 Batch 3:  Loss:     0.7746 Validation Accuracy: 0.572000
Epoch 166, CIFAR-10 Batch 4:  Loss:     0.9251 Validation Accuracy: 0.580400
Epoch 166, CIFAR-10 Batch 5:  Loss:     0.7297 Validation Accuracy: 0.575600
Epoch 167, CIFAR-10 Batch 1:  Loss:     0.8840 Validation Accuracy: 0.585200
Epoch 167, CIFAR-10 Batch 2:  Loss:     0.7296 Validation Accuracy: 0.576200
Epoch 167, CIFAR-10 Batch 3:  Loss:     0.7775 Validation Accuracy: 0.578800
Epoch 167, CIFAR-10 Batch 4:  Loss:     0.9170 Validation Accuracy: 0.582800
Epoch 167, CIFAR-10 Batch 5:  Loss:     0.7382 Validation Accuracy: 0.577400
Epoch 168, CIFAR-10 Batch 1:  Loss:     0.8830 Validation Accuracy: 0.582400
Epoch 168, CIFAR-10 Batch 2:  Loss:     0.7210 Validation Accuracy: 0.576000
Epoch 168, CIFAR-10 Batch 3:  Loss:     0.7725 Validation Accuracy: 0.578200
Epoch 168, CIFAR-10 Batch 4:  Loss:     0.9289 Validation Accuracy: 0.583600
Epoch 168, CIFAR-10 Batch 5:  Loss:     0.7353 Validation Accuracy: 0.578600
Epoch 169, CIFAR-10 Batch 1:  Loss:     0.8926 Validation Accuracy: 0.582200
Epoch 169, CIFAR-10 Batch 2:  Loss:     0.7314 Validation Accuracy: 0.574600
Epoch 169, CIFAR-10 Batch 3:  Loss:     0.7740 Validation Accuracy: 0.577200
Epoch 169, CIFAR-10 Batch 4:  Loss:     0.9170 Validation Accuracy: 0.583600
Epoch 169, CIFAR-10 Batch 5:  Loss:     0.7354 Validation Accuracy: 0.581000
Epoch 170, CIFAR-10 Batch 1:  Loss:     0.8939 Validation Accuracy: 0.582000
Epoch 170, CIFAR-10 Batch 2:  Loss:     0.7361 Validation Accuracy: 0.572800
Epoch 170, CIFAR-10 Batch 3:  Loss:     0.7716 Validation Accuracy: 0.576600
Epoch 170, CIFAR-10 Batch 4:  Loss:     0.9177 Validation Accuracy: 0.582200
Epoch 170, CIFAR-10 Batch 5:  Loss:     0.7268 Validation Accuracy: 0.578200
Epoch 171, CIFAR-10 Batch 1:  Loss:     0.8857 Validation Accuracy: 0.582600
Epoch 171, CIFAR-10 Batch 2:  Loss:     0.7231 Validation Accuracy: 0.570400
Epoch 171, CIFAR-10 Batch 3:  Loss:     0.7731 Validation Accuracy: 0.580800
Epoch 171, CIFAR-10 Batch 4:  Loss:     0.9130 Validation Accuracy: 0.583600
Epoch 171, CIFAR-10 Batch 5:  Loss:     0.7225 Validation Accuracy: 0.579600
Epoch 172, CIFAR-10 Batch 1:  Loss:     0.8805 Validation Accuracy: 0.584200
Epoch 172, CIFAR-10 Batch 2:  Loss:     0.7333 Validation Accuracy: 0.575000
Epoch 172, CIFAR-10 Batch 3:  Loss:     0.7674 Validation Accuracy: 0.578000
Epoch 172, CIFAR-10 Batch 4:  Loss:     0.9097 Validation Accuracy: 0.584000
Epoch 172, CIFAR-10 Batch 5:  Loss:     0.7311 Validation Accuracy: 0.579200
Epoch 173, CIFAR-10 Batch 1:  Loss:     0.8749 Validation Accuracy: 0.582600
Epoch 173, CIFAR-10 Batch 2:  Loss:     0.7251 Validation Accuracy: 0.574000
Epoch 173, CIFAR-10 Batch 3:  Loss:     0.7793 Validation Accuracy: 0.576600
Epoch 173, CIFAR-10 Batch 4:  Loss:     0.9077 Validation Accuracy: 0.584000
Epoch 173, CIFAR-10 Batch 5:  Loss:     0.7354 Validation Accuracy: 0.579600
Epoch 174, CIFAR-10 Batch 1:  Loss:     0.8695 Validation Accuracy: 0.585200
Epoch 174, CIFAR-10 Batch 2:  Loss:     0.7241 Validation Accuracy: 0.575000
Epoch 174, CIFAR-10 Batch 3:  Loss:     0.7722 Validation Accuracy: 0.579600
Epoch 174, CIFAR-10 Batch 4:  Loss:     0.9113 Validation Accuracy: 0.585600
Epoch 174, CIFAR-10 Batch 5:  Loss:     0.7231 Validation Accuracy: 0.575600
Epoch 175, CIFAR-10 Batch 1:  Loss:     0.8740 Validation Accuracy: 0.581200
Epoch 175, CIFAR-10 Batch 2:  Loss:     0.7295 Validation Accuracy: 0.575800
Epoch 175, CIFAR-10 Batch 3:  Loss:     0.7764 Validation Accuracy: 0.580000
Epoch 175, CIFAR-10 Batch 4:  Loss:     0.8997 Validation Accuracy: 0.582400
Epoch 175, CIFAR-10 Batch 5:  Loss:     0.7297 Validation Accuracy: 0.577000
Epoch 176, CIFAR-10 Batch 1:  Loss:     0.8926 Validation Accuracy: 0.584400
Epoch 176, CIFAR-10 Batch 2:  Loss:     0.7271 Validation Accuracy: 0.577600
Epoch 176, CIFAR-10 Batch 3:  Loss:     0.7879 Validation Accuracy: 0.578000
Epoch 176, CIFAR-10 Batch 4:  Loss:     0.8967 Validation Accuracy: 0.580000
Epoch 176, CIFAR-10 Batch 5:  Loss:     0.7186 Validation Accuracy: 0.575400
Epoch 177, CIFAR-10 Batch 1:  Loss:     0.8885 Validation Accuracy: 0.583400
Epoch 177, CIFAR-10 Batch 2:  Loss:     0.7155 Validation Accuracy: 0.575200
Epoch 177, CIFAR-10 Batch 3:  Loss:     0.7712 Validation Accuracy: 0.576600
Epoch 177, CIFAR-10 Batch 4:  Loss:     0.9074 Validation Accuracy: 0.583600
Epoch 177, CIFAR-10 Batch 5:  Loss:     0.7225 Validation Accuracy: 0.579800
Epoch 178, CIFAR-10 Batch 1:  Loss:     0.8805 Validation Accuracy: 0.585200
Epoch 178, CIFAR-10 Batch 2:  Loss:     0.7190 Validation Accuracy: 0.572800
Epoch 178, CIFAR-10 Batch 3:  Loss:     0.7685 Validation Accuracy: 0.579800
Epoch 178, CIFAR-10 Batch 4:  Loss:     0.8923 Validation Accuracy: 0.582000
Epoch 178, CIFAR-10 Batch 5:  Loss:     0.7286 Validation Accuracy: 0.580000
Epoch 179, CIFAR-10 Batch 1:  Loss:     0.8800 Validation Accuracy: 0.583200
Epoch 179, CIFAR-10 Batch 2:  Loss:     0.7305 Validation Accuracy: 0.577400
Epoch 179, CIFAR-10 Batch 3:  Loss:     0.7643 Validation Accuracy: 0.577800
Epoch 179, CIFAR-10 Batch 4:  Loss:     0.8841 Validation Accuracy: 0.583200
Epoch 179, CIFAR-10 Batch 5:  Loss:     0.7252 Validation Accuracy: 0.577000
Epoch 180, CIFAR-10 Batch 1:  Loss:     0.8654 Validation Accuracy: 0.587600
Epoch 180, CIFAR-10 Batch 2:  Loss:     0.7143 Validation Accuracy: 0.573000
Epoch 180, CIFAR-10 Batch 3:  Loss:     0.7794 Validation Accuracy: 0.579200
Epoch 180, CIFAR-10 Batch 4:  Loss:     0.9014 Validation Accuracy: 0.586800
Epoch 180, CIFAR-10 Batch 5:  Loss:     0.7244 Validation Accuracy: 0.581600
Epoch 181, CIFAR-10 Batch 1:  Loss:     0.8938 Validation Accuracy: 0.583400
Epoch 181, CIFAR-10 Batch 2:  Loss:     0.7271 Validation Accuracy: 0.577800
Epoch 181, CIFAR-10 Batch 3:  Loss:     0.7734 Validation Accuracy: 0.583000
Epoch 181, CIFAR-10 Batch 4:  Loss:     0.9066 Validation Accuracy: 0.585200
Epoch 181, CIFAR-10 Batch 5:  Loss:     0.7221 Validation Accuracy: 0.575000
Epoch 182, CIFAR-10 Batch 1:  Loss:     0.8831 Validation Accuracy: 0.583000
Epoch 182, CIFAR-10 Batch 2:  Loss:     0.7079 Validation Accuracy: 0.575200
Epoch 182, CIFAR-10 Batch 3:  Loss:     0.7718 Validation Accuracy: 0.580800
Epoch 182, CIFAR-10 Batch 4:  Loss:     0.8955 Validation Accuracy: 0.583600
Epoch 182, CIFAR-10 Batch 5:  Loss:     0.7227 Validation Accuracy: 0.579200
Epoch 183, CIFAR-10 Batch 1:  Loss:     0.8850 Validation Accuracy: 0.584400
Epoch 183, CIFAR-10 Batch 2:  Loss:     0.7234 Validation Accuracy: 0.577200
Epoch 183, CIFAR-10 Batch 3:  Loss:     0.7731 Validation Accuracy: 0.580600
Epoch 183, CIFAR-10 Batch 4:  Loss:     0.8970 Validation Accuracy: 0.583400
Epoch 183, CIFAR-10 Batch 5:  Loss:     0.7223 Validation Accuracy: 0.579200
Epoch 184, CIFAR-10 Batch 1:  Loss:     0.8784 Validation Accuracy: 0.584800
Epoch 184, CIFAR-10 Batch 2:  Loss:     0.7169 Validation Accuracy: 0.578200
Epoch 184, CIFAR-10 Batch 3:  Loss:     0.7782 Validation Accuracy: 0.579800
Epoch 184, CIFAR-10 Batch 4:  Loss:     0.8916 Validation Accuracy: 0.587000
Epoch 184, CIFAR-10 Batch 5:  Loss:     0.7222 Validation Accuracy: 0.578400
Epoch 185, CIFAR-10 Batch 1:  Loss:     0.8760 Validation Accuracy: 0.588200
Epoch 185, CIFAR-10 Batch 2:  Loss:     0.7134 Validation Accuracy: 0.578400
Epoch 185, CIFAR-10 Batch 3:  Loss:     0.7757 Validation Accuracy: 0.579800
Epoch 185, CIFAR-10 Batch 4:  Loss:     0.8948 Validation Accuracy: 0.584800
Epoch 185, CIFAR-10 Batch 5:  Loss:     0.7211 Validation Accuracy: 0.582400
Epoch 186, CIFAR-10 Batch 1:  Loss:     0.8672 Validation Accuracy: 0.585600
Epoch 186, CIFAR-10 Batch 2:  Loss:     0.7223 Validation Accuracy: 0.577200
Epoch 186, CIFAR-10 Batch 3:  Loss:     0.7855 Validation Accuracy: 0.582400
Epoch 186, CIFAR-10 Batch 4:  Loss:     0.9073 Validation Accuracy: 0.587800
Epoch 186, CIFAR-10 Batch 5:  Loss:     0.7154 Validation Accuracy: 0.577600
Epoch 187, CIFAR-10 Batch 1:  Loss:     0.8732 Validation Accuracy: 0.585200
Epoch 187, CIFAR-10 Batch 2:  Loss:     0.7156 Validation Accuracy: 0.580600
Epoch 187, CIFAR-10 Batch 3:  Loss:     0.7843 Validation Accuracy: 0.582000
Epoch 187, CIFAR-10 Batch 4:  Loss:     0.8889 Validation Accuracy: 0.586000
Epoch 187, CIFAR-10 Batch 5:  Loss:     0.7194 Validation Accuracy: 0.578600
Epoch 188, CIFAR-10 Batch 1:  Loss:     0.8726 Validation Accuracy: 0.585800
Epoch 188, CIFAR-10 Batch 2:  Loss:     0.7300 Validation Accuracy: 0.578400
Epoch 188, CIFAR-10 Batch 3:  Loss:     0.7888 Validation Accuracy: 0.582400
Epoch 188, CIFAR-10 Batch 4:  Loss:     0.9059 Validation Accuracy: 0.586400
Epoch 188, CIFAR-10 Batch 5:  Loss:     0.7164 Validation Accuracy: 0.577800
Epoch 189, CIFAR-10 Batch 1:  Loss:     0.8680 Validation Accuracy: 0.586200
Epoch 189, CIFAR-10 Batch 2:  Loss:     0.7245 Validation Accuracy: 0.579600
Epoch 189, CIFAR-10 Batch 3:  Loss:     0.7868 Validation Accuracy: 0.579000
Epoch 189, CIFAR-10 Batch 4:  Loss:     0.8958 Validation Accuracy: 0.584200
Epoch 189, CIFAR-10 Batch 5:  Loss:     0.7186 Validation Accuracy: 0.581600
Epoch 190, CIFAR-10 Batch 1:  Loss:     0.8849 Validation Accuracy: 0.584800
Epoch 190, CIFAR-10 Batch 2:  Loss:     0.7153 Validation Accuracy: 0.577400
Epoch 190, CIFAR-10 Batch 3:  Loss:     0.7832 Validation Accuracy: 0.582200
Epoch 190, CIFAR-10 Batch 4:  Loss:     0.8974 Validation Accuracy: 0.584000
Epoch 190, CIFAR-10 Batch 5:  Loss:     0.7117 Validation Accuracy: 0.578000
Epoch 191, CIFAR-10 Batch 1:  Loss:     0.8722 Validation Accuracy: 0.584800
Epoch 191, CIFAR-10 Batch 2:  Loss:     0.7211 Validation Accuracy: 0.576400
Epoch 191, CIFAR-10 Batch 3:  Loss:     0.7783 Validation Accuracy: 0.577200
Epoch 191, CIFAR-10 Batch 4:  Loss:     0.8762 Validation Accuracy: 0.584600
Epoch 191, CIFAR-10 Batch 5:  Loss:     0.7107 Validation Accuracy: 0.577200
Epoch 192, CIFAR-10 Batch 1:  Loss:     0.8846 Validation Accuracy: 0.582800
Epoch 192, CIFAR-10 Batch 2:  Loss:     0.7210 Validation Accuracy: 0.578200
Epoch 192, CIFAR-10 Batch 3:  Loss:     0.7775 Validation Accuracy: 0.581000
Epoch 192, CIFAR-10 Batch 4:  Loss:     0.8894 Validation Accuracy: 0.582600
Epoch 192, CIFAR-10 Batch 5:  Loss:     0.7124 Validation Accuracy: 0.578600
Epoch 193, CIFAR-10 Batch 1:  Loss:     0.8779 Validation Accuracy: 0.584400
Epoch 193, CIFAR-10 Batch 2:  Loss:     0.7288 Validation Accuracy: 0.579000
Epoch 193, CIFAR-10 Batch 3:  Loss:     0.7735 Validation Accuracy: 0.584000
Epoch 193, CIFAR-10 Batch 4:  Loss:     0.8958 Validation Accuracy: 0.585800
Epoch 193, CIFAR-10 Batch 5:  Loss:     0.7117 Validation Accuracy: 0.580600
Epoch 194, CIFAR-10 Batch 1:  Loss:     0.8668 Validation Accuracy: 0.583400
Epoch 194, CIFAR-10 Batch 2:  Loss:     0.7146 Validation Accuracy: 0.577800
Epoch 194, CIFAR-10 Batch 3:  Loss:     0.7850 Validation Accuracy: 0.578800
Epoch 194, CIFAR-10 Batch 4:  Loss:     0.8910 Validation Accuracy: 0.586600
Epoch 194, CIFAR-10 Batch 5:  Loss:     0.7125 Validation Accuracy: 0.581200
Epoch 195, CIFAR-10 Batch 1:  Loss:     0.8740 Validation Accuracy: 0.582400
Epoch 195, CIFAR-10 Batch 2:  Loss:     0.7141 Validation Accuracy: 0.578800
Epoch 195, CIFAR-10 Batch 3:  Loss:     0.7658 Validation Accuracy: 0.580200
Epoch 195, CIFAR-10 Batch 4:  Loss:     0.8973 Validation Accuracy: 0.586200
Epoch 195, CIFAR-10 Batch 5:  Loss:     0.7130 Validation Accuracy: 0.579800
Epoch 196, CIFAR-10 Batch 1:  Loss:     0.8589 Validation Accuracy: 0.586600
Epoch 196, CIFAR-10 Batch 2:  Loss:     0.7125 Validation Accuracy: 0.582200
Epoch 196, CIFAR-10 Batch 3:  Loss:     0.7807 Validation Accuracy: 0.581200
Epoch 196, CIFAR-10 Batch 4:  Loss:     0.8881 Validation Accuracy: 0.586400
Epoch 196, CIFAR-10 Batch 5:  Loss:     0.7116 Validation Accuracy: 0.583200
Epoch 197, CIFAR-10 Batch 1:  Loss:     0.8735 Validation Accuracy: 0.587400
Epoch 197, CIFAR-10 Batch 2:  Loss:     0.7273 Validation Accuracy: 0.578200
Epoch 197, CIFAR-10 Batch 3:  Loss:     0.7925 Validation Accuracy: 0.578400
Epoch 197, CIFAR-10 Batch 4:  Loss:     0.8881 Validation Accuracy: 0.586000
Epoch 197, CIFAR-10 Batch 5:  Loss:     0.7152 Validation Accuracy: 0.578800
Epoch 198, CIFAR-10 Batch 1:  Loss:     0.8760 Validation Accuracy: 0.586200
Epoch 198, CIFAR-10 Batch 2:  Loss:     0.6997 Validation Accuracy: 0.579400
Epoch 198, CIFAR-10 Batch 3:  Loss:     0.7922 Validation Accuracy: 0.578800
Epoch 198, CIFAR-10 Batch 4:  Loss:     0.8867 Validation Accuracy: 0.584600
Epoch 198, CIFAR-10 Batch 5:  Loss:     0.7120 Validation Accuracy: 0.580400
Epoch 199, CIFAR-10 Batch 1:  Loss:     0.8592 Validation Accuracy: 0.584200
Epoch 199, CIFAR-10 Batch 2:  Loss:     0.7098 Validation Accuracy: 0.582600
Epoch 199, CIFAR-10 Batch 3:  Loss:     0.7808 Validation Accuracy: 0.580000
Epoch 199, CIFAR-10 Batch 4:  Loss:     0.8919 Validation Accuracy: 0.585200
Epoch 199, CIFAR-10 Batch 5:  Loss:     0.7043 Validation Accuracy: 0.579600
Epoch 200, CIFAR-10 Batch 1:  Loss:     0.8626 Validation Accuracy: 0.584400
Epoch 200, CIFAR-10 Batch 2:  Loss:     0.7182 Validation Accuracy: 0.580000
Epoch 200, CIFAR-10 Batch 3:  Loss:     0.7832 Validation Accuracy: 0.581600
Epoch 200, CIFAR-10 Batch 4:  Loss:     0.8916 Validation Accuracy: 0.587600
Epoch 200, CIFAR-10 Batch 5:  Loss:     0.7130 Validation Accuracy: 0.575600

Checkpoint

The model has been saved to disk.

Test Model

Test your model against the test dataset. This will be your final accuracy. You should have an accuracy greater than 50%. If you don't, keep tweaking the model architecture and parameters.





In [33]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import tensorflow as tf
import pickle
import helper
import random

# Set batch size if not already set
try:
    if batch_size:
        pass
except NameError:
    batch_size = 64

import os
print(os.getcwd())

save_model_path = './image_classification'
n_samples = 4
top_n_predictions = 3

def test_model():
    """
    Test the saved model against the test dataset
    """

    test_features, test_labels = pickle.load(open('preprocess_training.p', mode='rb'))
    loaded_graph = tf.Graph()

    with tf.Session(graph=loaded_graph) as sess:
        # Load model
        loader = tf.train.import_meta_graph(save_model_path + '.meta')
        loader.restore(sess, save_model_path)

        # Get Tensors from loaded model
        loaded_x = loaded_graph.get_tensor_by_name('x:0')
        loaded_y = loaded_graph.get_tensor_by_name('y:0')
        loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')
        loaded_logits = loaded_graph.get_tensor_by_name('logits:0')
        loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0')
        
        # Get accuracy in batches for memory limitations
        test_batch_acc_total = 0
        test_batch_count = 0
        
        for train_feature_batch, train_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size):
            test_batch_acc_total += sess.run(
                loaded_acc,
                feed_dict={loaded_x: train_feature_batch, loaded_y: train_label_batch, loaded_keep_prob: 1.0})
            test_batch_count += 1

        print('Testing Accuracy: {}\n'.format(test_batch_acc_total/test_batch_count))

        # Print Random Samples
        random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples)))
        random_test_predictions = sess.run(
            tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions),
            feed_dict={loaded_x: random_test_features, loaded_y: random_test_labels, loaded_keep_prob: 1.0})
        helper.display_image_predictions(random_test_features, random_test_labels, random_test_predictions)


test_model()



    


















    






/output
INFO:tensorflow:Restoring parameters from ./image_classification
Testing Accuracy: 0.58828125

Why 50-70% Accuracy?

You might be wondering why you can't get an accuracy any higher. First things first, 50% isn't bad for a simple CNN. Pure guessing would get you 10% accuracy. However, you might notice people are getting scores well above 70%. That's because we haven't taught you all there is to know about neural networks. We still need to cover a few more techniques.

Submitting This Project

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_image_classification.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.

Content source: rahulkgup/deep-learning-foundation

Similar notebooks:

notebook.community | gallery | about