Image Classification

In this project, you'll classify images from the CIFAR-10 dataset. The dataset consists of airplanes, dogs, cats, and other objects. You'll preprocess the images, then train a convolutional neural network on all the samples. The images need to be normalized and the labels need to be one-hot encoded. You'll get to apply what you learned and build a convolutional, max pooling, dropout, and fully connected layers. At the end, you'll get to see your neural network's predictions on the sample images.

Get the Data

Run the following cell to download the CIFAR-10 dataset for python.



In [1]:

    
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
from urllib.request import urlretrieve
from os.path import isfile, isdir
from tqdm import tqdm
import problem_unittests as tests
import tarfile

cifar10_dataset_folder_path = 'cifar-10-batches-py'

# Use Floyd's cifar-10 dataset if present
floyd_cifar10_location = '/cifar/cifar-10-python.tar.gz'
if isfile(floyd_cifar10_location):
    tar_gz_path = floyd_cifar10_location
else:
    tar_gz_path = 'cifar-10-python.tar.gz'

class DLProgress(tqdm):
    last_block = 0

    def hook(self, block_num=1, block_size=1, total_size=None):
        self.total = total_size
        self.update((block_num - self.last_block) * block_size)
        self.last_block = block_num

if not isfile(tar_gz_path):
    with DLProgress(unit='B', unit_scale=True, miniters=1, desc='CIFAR-10 Dataset') as pbar:
        urlretrieve(
            'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz',
            tar_gz_path,
            pbar.hook)

if not isdir(cifar10_dataset_folder_path):
    with tarfile.open(tar_gz_path) as tar:
        tar.extractall()
        tar.close()


tests.test_folder_path(cifar10_dataset_folder_path)









    



All files found!

Explore the Data

The dataset is broken into batches to prevent your machine from running out of memory. The CIFAR-10 dataset consists of 5 batches, named data_batch_1, data_batch_2, etc.. Each batch contains the labels and images that are one of the following:

airplane
automobile
bird
cat
deer
dog
frog
horse
ship
truck

Understanding a dataset is part of making predictions on the data. Play around with the code cell below by changing the batch_id and sample_id. The batch_id is the id for a batch (1-5). The sample_id is the id for a image and label pair in the batch.

Ask yourself "What are all possible labels?", "What is the range of values for the image data?", "Are the labels in order or random?". Answers to questions like these will help you preprocess the data and end up with better predictions.



In [2]:

    
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import helper
import numpy as np

# Explore the dataset
batch_id = 2
sample_id = 15
helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)









    



Stats of batch 2:
Samples: 10000
Label Counts: {0: 984, 1: 1007, 2: 1010, 3: 995, 4: 1010, 5: 988, 6: 1008, 7: 1026, 8: 987, 9: 985}
First 20 Labels: [1, 6, 6, 8, 8, 3, 4, 6, 0, 6, 0, 3, 6, 6, 5, 4, 8, 3, 2, 6]

Example of Image 15:
Image - Min Value: 1 Max Value: 225
Image - Shape: (32, 32, 3)
Label - Label Id: 4 Name: deer

Implement Preprocess Functions

Normalize

In the cell below, implement the normalize function to take in image data, x, and return it as a normalized Numpy array. The values should be in the range of 0 to 1, inclusive. The return object should be the same shape as x.



In [3]:

    
import numpy

def normalize(x):
    """
    Normalize a list of sample image data in the range of 0 to 1
    : x: List of image data.  The image shape is (32, 32, 3)
    : return: Numpy array of normalize data
    """
    # TODO: Implement Function
    x = numpy.array(x)
    x_normed = (x - x.min(0)) / x.ptp(0)
    return x_normed


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_normalize(normalize)









    



Tests Passed

One-hot encode

Just like the previous code cell, you'll be implementing a function for preprocessing. This time, you'll implement the one_hot_encode function. The input, x, are a list of labels. Implement the function to return the list of labels as One-Hot encoded Numpy array. The possible values for labels are 0 to 9. The one-hot encoding function should return the same encoding for each value between each call to one_hot_encode. Make sure to save the map of encodings outside the function.

Hint: Don't reinvent the wheel.



In [4]:

    
def one_hot_encode(x):
    """
    One hot encode a list of sample labels. Return a one-hot encoded vector for each label.
    : x: List of sample Labels
    : return: Numpy array of one-hot encoded labels
    """
    nb_classes = 10
    targets = numpy.array(x).reshape(-1)
    one_hot_targets = numpy.eye(nb_classes)[targets]
    return one_hot_targets


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_one_hot_encode(one_hot_encode)









    



Tests Passed

Randomize Data

As you saw from exploring the data above, the order of the samples are randomized. It doesn't hurt to randomize it again, but you don't need to for this dataset.

Preprocess all the data and save it

Running the code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation.



In [5]:

    
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)

Check Point

This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.



In [6]:

    
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle
import problem_unittests as tests
import helper

# Load the Preprocessed Validation data
valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))

Build the network

For the neural network, you'll build each layer into a function. Most of the code you've seen has been outside of functions. To test your code more thoroughly, we require that you put each layer in a function. This allows us to give you better feedback and test for simple mistakes using our unittests before you submit your project.

Note: If you're finding it hard to dedicate enough time for this course each week, we've provided a small shortcut to this part of the project. In the next couple of problems, you'll have the option to use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages to build each layer, except the layers you build in the "Convolutional and Max Pooling Layer" section. TF Layers is similar to Keras's and TFLearn's abstraction to layers, so it's easy to pickup.

However, if you would like to get the most out of this course, try to solve all the problems without using anything from the TF Layers packages. You can still use classes from other packages that happen to have the same name as ones you find in TF Layers! For example, instead of using the TF Layers version of the conv2d class, tf.layers.conv2d, you would want to use the TF Neural Network version of conv2d, tf.nn.conv2d.

Let's begin!

Input

The neural network needs to read the image data, one-hot encoded labels, and dropout keep probability. Implement the following functions

Implement neural_net_image_input
- Return a TF Placeholder
- Set the shape using image_shape with batch size set to None.
- Name the TensorFlow placeholder "x" using the TensorFlow name parameter in the TF Placeholder.
Implement neural_net_label_input
- Return a TF Placeholder
- Set the shape using n_classes with batch size set to None.
- Name the TensorFlow placeholder "y" using the TensorFlow name parameter in the TF Placeholder.
Implement neural_net_keep_prob_input
- Return a TF Placeholder for dropout keep probability.
- Name the TensorFlow placeholder "keep_prob" using the TensorFlow name parameter in the TF Placeholder.

These names will be used at the end of the project to load your saved model.

Note: None for shapes in TensorFlow allow for a dynamic size.



In [7]:

    
import tensorflow as tf

def neural_net_image_input(image_shape):
    """
    Return a Tensor for a batch of image input
    : image_shape: Shape of the images
    : return: Tensor for image input.
    """
    return tf.placeholder(tf.float32, shape=(None, image_shape[0], 
                                             image_shape[1], image_shape[2]), name="x")



def neural_net_label_input(n_classes):
    """
    Return a Tensor for a batch of label input
    : n_classes: Number of classes
    : return: Tensor for label input.
    """
    return tf.placeholder(tf.float32, shape=(None, n_classes), name="y")


def neural_net_keep_prob_input():
    """
    Return a Tensor for keep probability
    : return: Tensor for keep probability.
    """
    # TODO: Implement Function
    return tf.placeholder(tf.float32, name="keep_prob")


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tf.reset_default_graph()
tests.test_nn_image_inputs(neural_net_image_input)
tests.test_nn_label_inputs(neural_net_label_input)
tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)









    



Image Input Tests Passed.
Label Input Tests Passed.
Keep Prob Tests Passed.

Convolution and Max Pooling Layer

Convolution layers have a lot of success with images. For this code cell, you should implement the function conv2d_maxpool to apply convolution then max pooling:

Create the weight and bias using conv_ksize, conv_num_outputs and the shape of x_tensor.
Apply a convolution to x_tensor using weight and conv_strides.
- We recommend you use same padding, but you're welcome to use any padding.
Add bias
Add a nonlinear activation to the convolution.
Apply Max Pooling using pool_ksize and pool_strides.
- We recommend you use same padding, but you're welcome to use any padding.

Note: You can't use TensorFlow Layers or TensorFlow Layers (contrib) for this layer, but you can still use TensorFlow's Neural Network package. You may still use the shortcut option for all the other layers.



In [8]:

    
def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
    """
    Apply convolution then max pooling to x_tensor
    :param x_tensor: TensorFlow Tensor
    :param conv_num_outputs: Number of outputs for the convolutional layer
    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer
    :param conv_strides: Stride 2-D Tuple for convolution
    :param pool_ksize: kernal size 2-D Tuple for pool
    :param pool_strides: Stride 2-D Tuple for pool
    : return: A tensor that represents convolution and max pooling of x_tensor
    """
    batch_size, in_width, in_height, in_depth = x_tensor.get_shape().as_list()
    weights = tf.Variable(tf.truncated_normal([conv_ksize[0], conv_ksize[1], 
                                               in_depth, conv_num_outputs]))
    biases = tf.Variable(tf.zeros(conv_num_outputs))
    
    conv = tf.nn.conv2d(x_tensor, weights, strides=[1, conv_strides[0], conv_strides[1], 1],
                        padding='SAME')
    conv = tf.nn.bias_add(conv, biases)
    conv = tf.nn.relu(conv)
    
    filter_shape = [1, pool_ksize[0], pool_ksize[1], 1]
    strides = [1, pool_strides[0], pool_strides[1], 1]
    return tf.nn.max_pool(conv, filter_shape, strides, 'SAME')


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_con_pool(conv2d_maxpool)









    



Tests Passed

Flatten Layer

Implement the flatten function to change the dimension of x_tensor from a 4-D tensor to a 2-D tensor. The output should be the shape (Batch Size, Flattened Image Size). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.



In [9]:

    
def flatten(x_tensor):
    """
    Flatten x_tensor to (Batch Size, Flattened Image Size)
    : x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.
    : return: A tensor of size (Batch Size, Flattened Image Size).
    """
    # TODO: Implement Function
    return tf.contrib.layers.flatten(x_tensor)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_flatten(flatten)









    



Tests Passed

Fully-Connected Layer

Implement the fully_conn function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.



In [10]:

    
def fully_conn(x_tensor, num_outputs):
    """
    Apply a fully connected layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    # TODO: Implement Function
    return tf.contrib.layers.fully_connected(x_tensor, num_outputs)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_fully_conn(fully_conn)









    



Tests Passed

Output Layer

Implement the output function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.

Note: Activation, softmax, or cross entropy should not be applied to this.



In [11]:

    
def output(x_tensor, num_outputs):
    """
    Apply a output layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    # TODO: Implement Function
    return tf.contrib.layers.fully_connected(x_tensor, num_outputs)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_output(output)









    



Tests Passed

Create Convolutional Model

Implement the function conv_net to create a convolutional neural network model. The function takes in a batch of images, x, and outputs logits. Use the layers you created above to create this model:

Apply 1, 2, or 3 Convolution and Max Pool layers
Apply a Flatten Layer
Apply 1, 2, or 3 Fully Connected Layers
Apply an Output Layer
Return the output
Apply TensorFlow's Dropout to one or more layers in the model using keep_prob.



In [73]:

    
def conv_net(x, keep_prob):
    """
    Create a convolutional neural network model
    : x: Placeholder tensor that holds image data.
    : keep_prob: Placeholder tensor that hold dropout keep probability.
    : return: Tensor that represents logits
    """
    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
    #    Play around with different number of outputs, kernel size and stride
    # Function Definition from Above:
    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    conv_ksize = [2,2]
    conv_strides = [1,1]
    pool_ksize = [2,2]
    pool_strides = [1,1]

    conv_num_outputs = 16
    x_tensor = conv2d_maxpool(x, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    x_tensor = tf.nn.dropout(x_tensor, keep_prob)
    
    conv_ksize = [3,3]
    conv_strides = [2,2]
    pool_ksize = [2,2]
    pool_strides = [2,2]
    
    conv_num_outputs = 40
    x_tensor = conv2d_maxpool(x, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    x_tensor = tf.nn.dropout(x_tensor, keep_prob)
    
    conv_num_outputs = 10
    x_tensor = conv2d_maxpool(x, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    x_tensor = tf.nn.dropout(x_tensor, keep_prob)

    # TODO: Apply a Flatten Layer
    # Function Definition from Above:
    #   flatten(x_tensor)
    x_tensor = flatten(x_tensor)

    # TODO: Apply 1, 2, or 3 Fully Connected Layers
    #    Play around with different number of outputs
    num_outputs = 60
    x_tensor = fully_conn(x_tensor, num_outputs)
    num_outputs = 40
    x_tensor = fully_conn(x_tensor, num_outputs)
    num_outputs = 20
    x_tensor = fully_conn(x_tensor, num_outputs)

    num_classes = 10
    return output(x_tensor, num_classes)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""

##############################
## Build the Neural Network ##
##############################

# Remove previous weights, bias, inputs, etc..
tf.reset_default_graph()

# Inputs
x = neural_net_image_input((32, 32, 3))
y = neural_net_label_input(10)
keep_prob = neural_net_keep_prob_input()

# Model
logits = conv_net(x, keep_prob)

# Name logits Tensor, so that is can be loaded from disk after training
logits = tf.identity(logits, name='logits')

# Loss and Optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))
optimizer = tf.train.AdamOptimizer().minimize(cost)

# Accuracy
correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')

tests.test_conv_net(conv_net)









    



Neural Network Built!

Train the Neural Network

Single Optimization

Implement the function train_neural_network to do a single optimization. The optimization should use optimizer to optimize in session with a feed_dict of the following:

x for image input
y for labels
keep_prob for keep probability for dropout

This function will be called for each batch, so tf.global_variables_initializer() has already been called.

Note: Nothing needs to be returned. This function is only optimizing the neural network.



In [74]:

    
def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):
    """
    Optimize the session on a batch of images and labels
    : session: Current TensorFlow session
    : optimizer: TensorFlow optimizer function
    : keep_probability: keep probability
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    """
    session.run(optimizer, feed_dict={x: feature_batch, y: label_batch, keep_prob: keep_probability})


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_train_nn(train_neural_network)









    



Tests Passed

Show Stats

Implement the function print_stats to print loss and validation accuracy. Use the global variables valid_features and valid_labels to calculate validation accuracy. Use a keep probability of 1.0 to calculate the loss and validation accuracy.



In [75]:

    
def print_stats(session, feature_batch, label_batch, cost, accuracy):
    """
    Print information about loss and validation accuracy
    : session: Current TensorFlow session
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    : cost: TensorFlow cost function
    : accuracy: TensorFlow accuracy function
    """
    # TODO: Implement Function
    cost = session.run(cost, feed_dict={x: feature_batch, y: label_batch, keep_prob: keep_probability})
    acc = session.run(accuracy, feed_dict={x: feature_batch, y: label_batch, keep_prob: keep_probability})
    validation_accuracy = session.run(accuracy, feed_dict={x: valid_features, y: valid_labels, keep_prob: 1.0})

    print('cost: {}'.format(cost))
    print('accuracy: {}'.format(accuracy))    
    print('validation accuracy: {}'.format(validation_accuracy))

Hyperparameters

Tune the following parameters:

Set epochs to the number of iterations until the network stops learning or start overfitting
Set batch_size to the highest number that your machine has memory for. Most people set them to common sizes of memory:
- 64
- 128
- 256
- ...
Set keep_probability to the probability of keeping a node using dropout



In [78]:

    
# TODO: Tune Parameters
epochs = 70
batch_size = 256
keep_probability = 0.9

Train on a Single CIFAR-10 Batch

Instead of training the neural network on all the CIFAR-10 batches of data, let's use a single batch. This should save time while you iterate on the model to get a better accuracy. Once the final validation accuracy is 50% or greater, run the model on all the data in the next section.



In [79]:

    
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
print('Checking the Training on a Single Batch...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())
    
    # Training cycle
    for epoch in range(epochs):
        batch_i = 1
        for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
        print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
        print_stats(sess, batch_features, batch_labels, cost, accuracy)









    



Checking the Training on a Single Batch...
Epoch  1, CIFAR-10 Batch 1:  cost: 2.2504570484161377
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.16899999976158142
Epoch  2, CIFAR-10 Batch 1:  cost: 2.2488081455230713
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.2370000034570694
Epoch  3, CIFAR-10 Batch 1:  cost: 2.2423973083496094
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.26460000872612
Epoch  4, CIFAR-10 Batch 1:  cost: 2.0846874713897705
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.30239999294281006
Epoch  5, CIFAR-10 Batch 1:  cost: 2.1274802684783936
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.3197999894618988
Epoch  6, CIFAR-10 Batch 1:  cost: 1.9916942119598389
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.3492000102996826
Epoch  7, CIFAR-10 Batch 1:  cost: 1.8400757312774658
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.35899999737739563
Epoch  8, CIFAR-10 Batch 1:  cost: 1.7523689270019531
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.3758000135421753
Epoch  9, CIFAR-10 Batch 1:  cost: 1.6577694416046143
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.3864000141620636
Epoch 10, CIFAR-10 Batch 1:  cost: 1.6362148523330688
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.3937999904155731
Epoch 11, CIFAR-10 Batch 1:  cost: 1.5689085721969604
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4023999869823456
Epoch 12, CIFAR-10 Batch 1:  cost: 1.5279260873794556
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.40860000252723694
Epoch 13, CIFAR-10 Batch 1:  cost: 1.5162179470062256
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.40720000863075256
Epoch 14, CIFAR-10 Batch 1:  cost: 1.5261552333831787
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.412200003862381
Epoch 15, CIFAR-10 Batch 1:  cost: 1.4829967021942139
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4198000133037567
Epoch 16, CIFAR-10 Batch 1:  cost: 1.3880592584609985
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.42260000109672546
Epoch 17, CIFAR-10 Batch 1:  cost: 1.364946722984314
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.42579999566078186
Epoch 18, CIFAR-10 Batch 1:  cost: 1.3980528116226196
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4300000071525574
Epoch 19, CIFAR-10 Batch 1:  cost: 1.3050873279571533
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4392000138759613
Epoch 20, CIFAR-10 Batch 1:  cost: 1.312239408493042
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.44040000438690186
Epoch 21, CIFAR-10 Batch 1:  cost: 1.320919156074524
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4413999915122986
Epoch 22, CIFAR-10 Batch 1:  cost: 1.13723623752594
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4415999948978424
Epoch 23, CIFAR-10 Batch 1:  cost: 1.15621018409729
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4438000023365021
Epoch 24, CIFAR-10 Batch 1:  cost: 1.102591872215271
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.44999998807907104
Epoch 25, CIFAR-10 Batch 1:  cost: 1.076242446899414
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4519999921321869
Epoch 26, CIFAR-10 Batch 1:  cost: 1.0611063241958618
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.44699999690055847
Epoch 27, CIFAR-10 Batch 1:  cost: 0.9109357595443726
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4569999873638153
Epoch 28, CIFAR-10 Batch 1:  cost: 0.992428183555603
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45339998602867126
Epoch 29, CIFAR-10 Batch 1:  cost: 0.9878647923469543
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46140000224113464
Epoch 30, CIFAR-10 Batch 1:  cost: 0.874899685382843
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45899999141693115
Epoch 31, CIFAR-10 Batch 1:  cost: 0.8687017560005188
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4625999927520752
Epoch 32, CIFAR-10 Batch 1:  cost: 0.8059971928596497
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46459999680519104
Epoch 33, CIFAR-10 Batch 1:  cost: 0.8411073684692383
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.462799996137619
Epoch 34, CIFAR-10 Batch 1:  cost: 0.843868613243103
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46959999203681946
Epoch 35, CIFAR-10 Batch 1:  cost: 0.7779414057731628
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46939998865127563
Epoch 36, CIFAR-10 Batch 1:  cost: 0.7686623334884644
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.47200000286102295
Epoch 37, CIFAR-10 Batch 1:  cost: 0.725756049156189
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4713999927043915
Epoch 38, CIFAR-10 Batch 1:  cost: 0.7313383221626282
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4749999940395355
Epoch 39, CIFAR-10 Batch 1:  cost: 0.693063497543335
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4729999899864197
Epoch 40, CIFAR-10 Batch 1:  cost: 0.729953408241272
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4742000102996826
Epoch 41, CIFAR-10 Batch 1:  cost: 0.6619368195533752
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.47600001096725464
Epoch 42, CIFAR-10 Batch 1:  cost: 0.6581214070320129
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4749999940395355
Epoch 43, CIFAR-10 Batch 1:  cost: 0.6335317492485046
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4729999899864197
Epoch 44, CIFAR-10 Batch 1:  cost: 0.6196344494819641
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4634000062942505
Epoch 45, CIFAR-10 Batch 1:  cost: 0.6599828004837036
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46399998664855957
Epoch 46, CIFAR-10 Batch 1:  cost: 0.576208233833313
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45579999685287476
Epoch 47, CIFAR-10 Batch 1:  cost: 0.619949221611023
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4643999934196472
Epoch 48, CIFAR-10 Batch 1:  cost: 0.5611149668693542
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46880000829696655
Epoch 49, CIFAR-10 Batch 1:  cost: 0.5931788682937622
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4681999981403351
Epoch 50, CIFAR-10 Batch 1:  cost: 0.5547138452529907
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4697999954223633
Epoch 51, CIFAR-10 Batch 1:  cost: 0.5489969849586487
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4713999927043915
Epoch 52, CIFAR-10 Batch 1:  cost: 0.4963366389274597
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4715999960899353
Epoch 53, CIFAR-10 Batch 1:  cost: 0.48967820405960083
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46860000491142273
Epoch 54, CIFAR-10 Batch 1:  cost: 0.5152413249015808
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.47099998593330383
Epoch 55, CIFAR-10 Batch 1:  cost: 0.5062955617904663
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4749999940395355
Epoch 56, CIFAR-10 Batch 1:  cost: 0.5039693713188171
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4722000062465668
Epoch 57, CIFAR-10 Batch 1:  cost: 0.5154897570610046
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46720001101493835
Epoch 58, CIFAR-10 Batch 1:  cost: 0.5153714418411255
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46860000491142273
Epoch 59, CIFAR-10 Batch 1:  cost: 0.4735034108161926
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46779999136924744
Epoch 60, CIFAR-10 Batch 1:  cost: 0.48577937483787537
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46939998865127563
Epoch 61, CIFAR-10 Batch 1:  cost: 0.46766671538352966
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4611999988555908
Epoch 62, CIFAR-10 Batch 1:  cost: 0.45092231035232544
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.46860000491142273
Epoch 63, CIFAR-10 Batch 1:  cost: 0.45614495873451233
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4634000062942505
Epoch 64, CIFAR-10 Batch 1:  cost: 0.4363163113594055
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.462799996137619
Epoch 65, CIFAR-10 Batch 1:  cost: 0.4728681147098541
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45239999890327454
Epoch 66, CIFAR-10 Batch 1:  cost: 0.4680662751197815
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4537999927997589
Epoch 67, CIFAR-10 Batch 1:  cost: 0.4473404884338379
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4537999927997589
Epoch 68, CIFAR-10 Batch 1:  cost: 0.4745239317417145
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45579999685287476
Epoch 69, CIFAR-10 Batch 1:  cost: 0.4527933597564697
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4625999927520752
Epoch 70, CIFAR-10 Batch 1:  cost: 0.4115235209465027
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4586000144481659

Fully Train the Model

Now that you got a good accuracy with a single CIFAR-10 batch, try it with all five batches.



In [80]:

    
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
save_model_path = './image_classification'

print('Training...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())
    
    # Training cycle
    for epoch in range(epochs):
        # Loop over all batches
        n_batches = 5
        for batch_i in range(1, n_batches + 1):
            for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
                train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
            print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
            print_stats(sess, batch_features, batch_labels, cost, accuracy)
            
    # Save Model
    saver = tf.train.Saver()
    save_path = saver.save(sess, save_model_path)









    



Training...
Epoch  1, CIFAR-10 Batch 1:  cost: 2.1850666999816895
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.22020000219345093
Epoch  1, CIFAR-10 Batch 2:  cost: 2.034942150115967
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.28780001401901245
Epoch  1, CIFAR-10 Batch 3:  cost: 1.9634233713150024
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.3246000111103058
Epoch  1, CIFAR-10 Batch 4:  cost: 2.1030991077423096
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.33660000562667847
Epoch  1, CIFAR-10 Batch 5:  cost: 1.879726767539978
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.36800000071525574
Epoch  2, CIFAR-10 Batch 1:  cost: 1.8636233806610107
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.3880000114440918
Epoch  2, CIFAR-10 Batch 2:  cost: 1.8245601654052734
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.39820000529289246
Epoch  2, CIFAR-10 Batch 3:  cost: 1.6308691501617432
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4147999882698059
Epoch  2, CIFAR-10 Batch 4:  cost: 1.9271018505096436
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.42179998755455017
Epoch  2, CIFAR-10 Batch 5:  cost: 1.6163173913955688
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4311999976634979
Epoch  3, CIFAR-10 Batch 1:  cost: 1.778504729270935
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4334000051021576
Epoch  3, CIFAR-10 Batch 2:  cost: 1.6642658710479736
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4480000138282776
Epoch  3, CIFAR-10 Batch 3:  cost: 1.5630249977111816
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.44040000438690186
Epoch  3, CIFAR-10 Batch 4:  cost: 1.7543869018554688
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4505999982357025
Epoch  3, CIFAR-10 Batch 5:  cost: 1.5169460773468018
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45739999413490295
Epoch  4, CIFAR-10 Batch 1:  cost: 1.716301679611206
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45680001378059387
Epoch  4, CIFAR-10 Batch 2:  cost: 1.5448999404907227
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.462799996137619
Epoch  4, CIFAR-10 Batch 3:  cost: 1.3709664344787598
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.45899999141693115
Epoch  4, CIFAR-10 Batch 4:  cost: 1.5803598165512085
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4781999886035919
Epoch  4, CIFAR-10 Batch 5:  cost: 1.3760324716567993
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.48080000281333923
Epoch  5, CIFAR-10 Batch 1:  cost: 1.6169712543487549
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4749999940395355
Epoch  5, CIFAR-10 Batch 2:  cost: 1.3545976877212524
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4848000109195709
Epoch  5, CIFAR-10 Batch 3:  cost: 1.1790077686309814
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.47940000891685486
Epoch  5, CIFAR-10 Batch 4:  cost: 1.5139758586883545
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.49239999055862427
Epoch  5, CIFAR-10 Batch 5:  cost: 1.3266972303390503
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.498199999332428
Epoch  6, CIFAR-10 Batch 1:  cost: 1.53421151638031
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5
Epoch  6, CIFAR-10 Batch 2:  cost: 1.1953930854797363
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5005999803543091
Epoch  6, CIFAR-10 Batch 3:  cost: 1.0969202518463135
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.4943999946117401
Epoch  6, CIFAR-10 Batch 4:  cost: 1.4497590065002441
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.503000020980835
Epoch  6, CIFAR-10 Batch 5:  cost: 1.3055946826934814
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5080000162124634
Epoch  7, CIFAR-10 Batch 1:  cost: 1.4453130960464478
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5040000081062317
Epoch  7, CIFAR-10 Batch 2:  cost: 1.1429555416107178
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.508400022983551
Epoch  7, CIFAR-10 Batch 3:  cost: 1.0931556224822998
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.49939998984336853
Epoch  7, CIFAR-10 Batch 4:  cost: 1.3639720678329468
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5080000162124634
Epoch  7, CIFAR-10 Batch 5:  cost: 1.215024709701538
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5157999992370605
Epoch  8, CIFAR-10 Batch 1:  cost: 1.3442325592041016
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5139999985694885
Epoch  8, CIFAR-10 Batch 2:  cost: 1.112592339515686
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5113999843597412
Epoch  8, CIFAR-10 Batch 3:  cost: 1.0148134231567383
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5139999985694885
Epoch  8, CIFAR-10 Batch 4:  cost: 1.3091742992401123
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5167999863624573
Epoch  8, CIFAR-10 Batch 5:  cost: 1.1279957294464111
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5192000269889832
Epoch  9, CIFAR-10 Batch 1:  cost: 1.3164222240447998
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.517799973487854
Epoch  9, CIFAR-10 Batch 2:  cost: 1.0823938846588135
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5112000107765198
Epoch  9, CIFAR-10 Batch 3:  cost: 0.8843753933906555
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5216000080108643
Epoch  9, CIFAR-10 Batch 4:  cost: 1.2747938632965088
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5220000147819519
Epoch  9, CIFAR-10 Batch 5:  cost: 1.0794346332550049
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5264000296592712
Epoch 10, CIFAR-10 Batch 1:  cost: 1.23600172996521
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5221999883651733
Epoch 10, CIFAR-10 Batch 2:  cost: 1.01149582862854
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5224000215530396
Epoch 10, CIFAR-10 Batch 3:  cost: 0.8602017164230347
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.525600016117096
Epoch 10, CIFAR-10 Batch 4:  cost: 1.23415207862854
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5296000242233276
Epoch 10, CIFAR-10 Batch 5:  cost: 1.1638981103897095
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5317999720573425
Epoch 11, CIFAR-10 Batch 1:  cost: 1.093780279159546
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.526199996471405
Epoch 11, CIFAR-10 Batch 2:  cost: 1.0149816274642944
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5198000073432922
Epoch 11, CIFAR-10 Batch 3:  cost: 0.8874204754829407
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5281999707221985
Epoch 11, CIFAR-10 Batch 4:  cost: 1.254162311553955
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5325999855995178
Epoch 11, CIFAR-10 Batch 5:  cost: 1.0489122867584229
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.531000018119812
Epoch 12, CIFAR-10 Batch 1:  cost: 1.2372667789459229
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.525600016117096
Epoch 12, CIFAR-10 Batch 2:  cost: 0.9969391822814941
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5303999781608582
Epoch 12, CIFAR-10 Batch 3:  cost: 0.8717702031135559
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.52920001745224
Epoch 12, CIFAR-10 Batch 4:  cost: 1.1954036951065063
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5361999869346619
Epoch 12, CIFAR-10 Batch 5:  cost: 1.0266797542572021
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5289999842643738
Epoch 13, CIFAR-10 Batch 1:  cost: 1.1334902048110962
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.527400016784668
Epoch 13, CIFAR-10 Batch 2:  cost: 0.8616838455200195
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5231999754905701
Epoch 13, CIFAR-10 Batch 3:  cost: 0.7939457893371582
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5357999801635742
Epoch 13, CIFAR-10 Batch 4:  cost: 1.137255311012268
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5393999814987183
Epoch 13, CIFAR-10 Batch 5:  cost: 0.9337199330329895
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5335999727249146
Epoch 14, CIFAR-10 Batch 1:  cost: 1.0685465335845947
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5314000248908997
Epoch 14, CIFAR-10 Batch 2:  cost: 0.9233428835868835
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5320000052452087
Epoch 14, CIFAR-10 Batch 3:  cost: 0.6864466071128845
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5410000085830688
Epoch 14, CIFAR-10 Batch 4:  cost: 1.1071717739105225
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5396000146865845
Epoch 14, CIFAR-10 Batch 5:  cost: 1.0128259658813477
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5378000140190125
Epoch 15, CIFAR-10 Batch 1:  cost: 0.9867258071899414
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5371999740600586
Epoch 15, CIFAR-10 Batch 2:  cost: 0.905285656452179
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.52920001745224
Epoch 15, CIFAR-10 Batch 3:  cost: 0.7570382356643677
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5415999889373779
Epoch 15, CIFAR-10 Batch 4:  cost: 1.0631004571914673
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5393999814987183
Epoch 15, CIFAR-10 Batch 5:  cost: 0.9387670755386353
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5360000133514404
Epoch 16, CIFAR-10 Batch 1:  cost: 0.9976776242256165
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5371999740600586
Epoch 16, CIFAR-10 Batch 2:  cost: 0.8800676465034485
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5248000025749207
Epoch 16, CIFAR-10 Batch 3:  cost: 0.7507567405700684
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5428000092506409
Epoch 16, CIFAR-10 Batch 4:  cost: 0.9853419065475464
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5429999828338623
Epoch 16, CIFAR-10 Batch 5:  cost: 0.9791666865348816
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5379999876022339
Epoch 17, CIFAR-10 Batch 1:  cost: 1.0068048238754272
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5411999821662903
Epoch 17, CIFAR-10 Batch 2:  cost: 0.7830237150192261
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5317999720573425
Epoch 17, CIFAR-10 Batch 3:  cost: 0.753045916557312
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.545199990272522
Epoch 17, CIFAR-10 Batch 4:  cost: 1.035279631614685
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5465999841690063
Epoch 17, CIFAR-10 Batch 5:  cost: 0.8918310403823853
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5418000221252441
Epoch 18, CIFAR-10 Batch 1:  cost: 0.9473581314086914
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5415999889373779
Epoch 18, CIFAR-10 Batch 2:  cost: 0.7811251878738403
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5338000059127808
Epoch 18, CIFAR-10 Batch 3:  cost: 0.6816543936729431
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5454000234603882
Epoch 18, CIFAR-10 Batch 4:  cost: 1.0078508853912354
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5475999712944031
Epoch 18, CIFAR-10 Batch 5:  cost: 0.8507992029190063
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5429999828338623
Epoch 19, CIFAR-10 Batch 1:  cost: 0.8713579177856445
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5424000024795532
Epoch 19, CIFAR-10 Batch 2:  cost: 0.7563539743423462
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5339999794960022
Epoch 19, CIFAR-10 Batch 3:  cost: 0.6968569755554199
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5511999726295471
Epoch 19, CIFAR-10 Batch 4:  cost: 1.0133228302001953
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5486000180244446
Epoch 19, CIFAR-10 Batch 5:  cost: 0.8001751899719238
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5429999828338623
Epoch 20, CIFAR-10 Batch 1:  cost: 0.9475840330123901
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.545199990272522
Epoch 20, CIFAR-10 Batch 2:  cost: 0.7822636365890503
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5429999828338623
Epoch 20, CIFAR-10 Batch 3:  cost: 0.6329358816146851
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5511999726295471
Epoch 20, CIFAR-10 Batch 4:  cost: 0.9612535238265991
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5515999794006348
Epoch 20, CIFAR-10 Batch 5:  cost: 0.8532952070236206
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5450000166893005
Epoch 21, CIFAR-10 Batch 1:  cost: 0.9333626627922058
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5478000044822693
Epoch 21, CIFAR-10 Batch 2:  cost: 0.758365273475647
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5303999781608582
Epoch 21, CIFAR-10 Batch 3:  cost: 0.6460070610046387
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5504000186920166
Epoch 21, CIFAR-10 Batch 4:  cost: 0.9497119784355164
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.550599992275238
Epoch 21, CIFAR-10 Batch 5:  cost: 0.839985728263855
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5429999828338623
Epoch 22, CIFAR-10 Batch 1:  cost: 0.8304258584976196
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5473999977111816
Epoch 22, CIFAR-10 Batch 2:  cost: 0.7219851613044739
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5351999998092651
Epoch 22, CIFAR-10 Batch 3:  cost: 0.598834216594696
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5519999861717224
Epoch 22, CIFAR-10 Batch 4:  cost: 0.9969717860221863
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5511999726295471
Epoch 22, CIFAR-10 Batch 5:  cost: 0.7564474940299988
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5490000247955322
Epoch 23, CIFAR-10 Batch 1:  cost: 0.8954259753227234
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5514000058174133
Epoch 23, CIFAR-10 Batch 2:  cost: 0.733210027217865
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5388000011444092
Epoch 23, CIFAR-10 Batch 3:  cost: 0.6352530717849731
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5508000254631042
Epoch 23, CIFAR-10 Batch 4:  cost: 0.8916481137275696
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5551999807357788
Epoch 23, CIFAR-10 Batch 5:  cost: 0.8165737986564636
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5479999780654907
Epoch 24, CIFAR-10 Batch 1:  cost: 0.8131459951400757
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5511999726295471
Epoch 24, CIFAR-10 Batch 2:  cost: 0.6655158996582031
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5401999950408936
Epoch 24, CIFAR-10 Batch 3:  cost: 0.5895988345146179
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5533999800682068
Epoch 24, CIFAR-10 Batch 4:  cost: 0.8991701006889343
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5546000003814697
Epoch 24, CIFAR-10 Batch 5:  cost: 0.733284592628479
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5475999712944031
Epoch 25, CIFAR-10 Batch 1:  cost: 0.9181321859359741
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.550599992275238
Epoch 25, CIFAR-10 Batch 2:  cost: 0.6880676746368408
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5382000207901001
Epoch 25, CIFAR-10 Batch 3:  cost: 0.6418512463569641
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5511999726295471
Epoch 25, CIFAR-10 Batch 4:  cost: 0.9056242108345032
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5583999752998352
Epoch 25, CIFAR-10 Batch 5:  cost: 0.74828040599823
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5454000234603882
Epoch 26, CIFAR-10 Batch 1:  cost: 0.8178674578666687
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5519999861717224
Epoch 26, CIFAR-10 Batch 2:  cost: 0.6990607976913452
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5404000282287598
Epoch 26, CIFAR-10 Batch 3:  cost: 0.587999165058136
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5546000003814697
Epoch 26, CIFAR-10 Batch 4:  cost: 0.9223154187202454
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5559999942779541
Epoch 26, CIFAR-10 Batch 5:  cost: 0.7223896384239197
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5514000058174133
Epoch 27, CIFAR-10 Batch 1:  cost: 0.7825288772583008
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5569999814033508
Epoch 27, CIFAR-10 Batch 2:  cost: 0.7125088572502136
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5378000140190125
Epoch 27, CIFAR-10 Batch 3:  cost: 0.5881587266921997
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5583999752998352
Epoch 27, CIFAR-10 Batch 4:  cost: 0.8328893780708313
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5565999746322632
Epoch 27, CIFAR-10 Batch 5:  cost: 0.7262132167816162
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5497999787330627
Epoch 28, CIFAR-10 Batch 1:  cost: 0.790678858757019
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5550000071525574
Epoch 28, CIFAR-10 Batch 2:  cost: 0.7369346618652344
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5415999889373779
Epoch 28, CIFAR-10 Batch 3:  cost: 0.5406550168991089
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5550000071525574
Epoch 28, CIFAR-10 Batch 4:  cost: 0.8102155923843384
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5590000152587891
Epoch 28, CIFAR-10 Batch 5:  cost: 0.7351776361465454
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.550000011920929
Epoch 29, CIFAR-10 Batch 1:  cost: 0.8364912271499634
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5569999814033508
Epoch 29, CIFAR-10 Batch 2:  cost: 0.6785778403282166
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5375999808311462
Epoch 29, CIFAR-10 Batch 3:  cost: 0.6162036657333374
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5522000193595886
Epoch 29, CIFAR-10 Batch 4:  cost: 0.8339541554450989
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5568000078201294
Epoch 29, CIFAR-10 Batch 5:  cost: 0.6431655287742615
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5533999800682068
Epoch 30, CIFAR-10 Batch 1:  cost: 0.7866605520248413
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5608000159263611
Epoch 30, CIFAR-10 Batch 2:  cost: 0.5850771069526672
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5443999767303467
Epoch 30, CIFAR-10 Batch 3:  cost: 0.560531735420227
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5522000193595886
Epoch 30, CIFAR-10 Batch 4:  cost: 0.8296920657157898
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5551999807357788
Epoch 30, CIFAR-10 Batch 5:  cost: 0.616715133190155
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5522000193595886
Epoch 31, CIFAR-10 Batch 1:  cost: 0.7582348585128784
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5616000294685364
Epoch 31, CIFAR-10 Batch 2:  cost: 0.573839545249939
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.54339998960495
Epoch 31, CIFAR-10 Batch 3:  cost: 0.5393122434616089
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5511999726295471
Epoch 31, CIFAR-10 Batch 4:  cost: 0.8142692446708679
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5569999814033508
Epoch 31, CIFAR-10 Batch 5:  cost: 0.6095073223114014
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5546000003814697
Epoch 32, CIFAR-10 Batch 1:  cost: 0.7762285470962524
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5583999752998352
Epoch 32, CIFAR-10 Batch 2:  cost: 0.6182560920715332
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5443999767303467
Epoch 32, CIFAR-10 Batch 3:  cost: 0.5251539945602417
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5523999929428101
Epoch 32, CIFAR-10 Batch 4:  cost: 0.7811521291732788
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5555999875068665
Epoch 32, CIFAR-10 Batch 5:  cost: 0.6394309997558594
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5565999746322632
Epoch 33, CIFAR-10 Batch 1:  cost: 0.7181687355041504
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5631999969482422
Epoch 33, CIFAR-10 Batch 2:  cost: 0.594343900680542
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.546999990940094
Epoch 33, CIFAR-10 Batch 3:  cost: 0.4461020529270172
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5504000186920166
Epoch 33, CIFAR-10 Batch 4:  cost: 0.8134753108024597
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5576000213623047
Epoch 33, CIFAR-10 Batch 5:  cost: 0.659579873085022
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.550599992275238
Epoch 34, CIFAR-10 Batch 1:  cost: 0.7372661232948303
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5631999969482422
Epoch 34, CIFAR-10 Batch 2:  cost: 0.5255190134048462
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5496000051498413
Epoch 34, CIFAR-10 Batch 3:  cost: 0.4871288239955902
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5537999868392944
Epoch 34, CIFAR-10 Batch 4:  cost: 0.7642033100128174
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5586000084877014
Epoch 34, CIFAR-10 Batch 5:  cost: 0.6589193344116211
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5501999855041504
Epoch 35, CIFAR-10 Batch 1:  cost: 0.714661717414856
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5605999827384949
Epoch 35, CIFAR-10 Batch 2:  cost: 0.5437520742416382
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5482000112533569
Epoch 35, CIFAR-10 Batch 3:  cost: 0.5186263918876648
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.553600013256073
Epoch 35, CIFAR-10 Batch 4:  cost: 0.7830744385719299
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5586000084877014
Epoch 35, CIFAR-10 Batch 5:  cost: 0.6810117959976196
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.550599992275238
Epoch 36, CIFAR-10 Batch 1:  cost: 0.6860352754592896
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5648000240325928
Epoch 36, CIFAR-10 Batch 2:  cost: 0.6065075993537903
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5509999990463257
Epoch 36, CIFAR-10 Batch 3:  cost: 0.4816928803920746
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5555999875068665
Epoch 36, CIFAR-10 Batch 4:  cost: 0.8136123418807983
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5565999746322632
Epoch 36, CIFAR-10 Batch 5:  cost: 0.6426668763160706
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5504000186920166
Epoch 37, CIFAR-10 Batch 1:  cost: 0.6676598787307739
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5622000098228455
Epoch 37, CIFAR-10 Batch 2:  cost: 0.4914153516292572
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5490000247955322
Epoch 37, CIFAR-10 Batch 3:  cost: 0.4768773913383484
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5568000078201294
Epoch 37, CIFAR-10 Batch 4:  cost: 0.7340152859687805
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5600000023841858
Epoch 37, CIFAR-10 Batch 5:  cost: 0.6224759221076965
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.553600013256073
Epoch 38, CIFAR-10 Batch 1:  cost: 0.6417135000228882
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5590000152587891
Epoch 38, CIFAR-10 Batch 2:  cost: 0.4935029447078705
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5493999719619751
Epoch 38, CIFAR-10 Batch 3:  cost: 0.4413912892341614
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5555999875068665
Epoch 38, CIFAR-10 Batch 4:  cost: 0.7682058215141296
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5595999956130981
Epoch 38, CIFAR-10 Batch 5:  cost: 0.6066426038742065
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5527999997138977
Epoch 39, CIFAR-10 Batch 1:  cost: 0.6324161887168884
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5595999956130981
Epoch 39, CIFAR-10 Batch 2:  cost: 0.5477221608161926
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5465999841690063
Epoch 39, CIFAR-10 Batch 3:  cost: 0.4634169936180115
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5529999732971191
Epoch 39, CIFAR-10 Batch 4:  cost: 0.7375431060791016
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5608000159263611
Epoch 39, CIFAR-10 Batch 5:  cost: 0.6517878770828247
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5541999936103821
Epoch 40, CIFAR-10 Batch 1:  cost: 0.6403271555900574
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5594000220298767
Epoch 40, CIFAR-10 Batch 2:  cost: 0.4804544448852539
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5504000186920166
Epoch 40, CIFAR-10 Batch 3:  cost: 0.45610007643699646
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5532000064849854
Epoch 40, CIFAR-10 Batch 4:  cost: 0.7428295016288757
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5559999942779541
Epoch 40, CIFAR-10 Batch 5:  cost: 0.6057349443435669
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5573999881744385
Epoch 41, CIFAR-10 Batch 1:  cost: 0.6817575693130493
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5645999908447266
Epoch 41, CIFAR-10 Batch 2:  cost: 0.5182827711105347
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5418000221252441
Epoch 41, CIFAR-10 Batch 3:  cost: 0.43939846754074097
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5541999936103821
Epoch 41, CIFAR-10 Batch 4:  cost: 0.7107445001602173
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5627999901771545
Epoch 41, CIFAR-10 Batch 5:  cost: 0.5954044461250305
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5573999881744385
Epoch 42, CIFAR-10 Batch 1:  cost: 0.6705322265625
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5645999908447266
Epoch 42, CIFAR-10 Batch 2:  cost: 0.48352470993995667
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5537999868392944
Epoch 42, CIFAR-10 Batch 3:  cost: 0.5060991048812866
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5565999746322632
Epoch 42, CIFAR-10 Batch 4:  cost: 0.7293267250061035
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5612000226974487
Epoch 42, CIFAR-10 Batch 5:  cost: 0.5145225524902344
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5586000084877014
Epoch 43, CIFAR-10 Batch 1:  cost: 0.6257699131965637
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5654000043869019
Epoch 43, CIFAR-10 Batch 2:  cost: 0.5194635391235352
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5559999942779541
Epoch 43, CIFAR-10 Batch 3:  cost: 0.429627001285553
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5532000064849854
Epoch 43, CIFAR-10 Batch 4:  cost: 0.7725958228111267
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5633999705314636
Epoch 43, CIFAR-10 Batch 5:  cost: 0.5649921298027039
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5523999929428101
Epoch 44, CIFAR-10 Batch 1:  cost: 0.5957258939743042
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5636000037193298
Epoch 44, CIFAR-10 Batch 2:  cost: 0.43749362230300903
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.550599992275238
Epoch 44, CIFAR-10 Batch 3:  cost: 0.4089280068874359
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5573999881744385
Epoch 44, CIFAR-10 Batch 4:  cost: 0.696561336517334
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5612000226974487
Epoch 44, CIFAR-10 Batch 5:  cost: 0.5229592323303223
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5600000023841858
Epoch 45, CIFAR-10 Batch 1:  cost: 0.6024874448776245
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5619999766349792
Epoch 45, CIFAR-10 Batch 2:  cost: 0.48697933554649353
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.555400013923645
Epoch 45, CIFAR-10 Batch 3:  cost: 0.46366196870803833
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5555999875068665
Epoch 45, CIFAR-10 Batch 4:  cost: 0.7415486574172974
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5631999969482422
Epoch 45, CIFAR-10 Batch 5:  cost: 0.4882878363132477
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5619999766349792
Epoch 46, CIFAR-10 Batch 1:  cost: 0.5872781872749329
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5645999908447266
Epoch 46, CIFAR-10 Batch 2:  cost: 0.4727490544319153
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5583999752998352
Epoch 46, CIFAR-10 Batch 3:  cost: 0.39933183789253235
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5564000010490417
Epoch 46, CIFAR-10 Batch 4:  cost: 0.7286757230758667
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5595999956130981
Epoch 46, CIFAR-10 Batch 5:  cost: 0.5248907208442688
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5568000078201294
Epoch 47, CIFAR-10 Batch 1:  cost: 0.6201286315917969
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5612000226974487
Epoch 47, CIFAR-10 Batch 2:  cost: 0.44235411286354065
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.557200014591217
Epoch 47, CIFAR-10 Batch 3:  cost: 0.4575943350791931
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.557200014591217
Epoch 47, CIFAR-10 Batch 4:  cost: 0.6761326789855957
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5626000165939331
Epoch 47, CIFAR-10 Batch 5:  cost: 0.5176711082458496
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5587999820709229
Epoch 48, CIFAR-10 Batch 1:  cost: 0.5852130055427551
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5627999901771545
Epoch 48, CIFAR-10 Batch 2:  cost: 0.4737831652164459
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5598000288009644
Epoch 48, CIFAR-10 Batch 3:  cost: 0.3706980347633362
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5577999949455261
Epoch 48, CIFAR-10 Batch 4:  cost: 0.7360931634902954
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5612000226974487
Epoch 48, CIFAR-10 Batch 5:  cost: 0.5343433618545532
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5576000213623047
Epoch 49, CIFAR-10 Batch 1:  cost: 0.6021870374679565
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5673999786376953
Epoch 49, CIFAR-10 Batch 2:  cost: 0.5000531077384949
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5594000220298767
Epoch 49, CIFAR-10 Batch 3:  cost: 0.3933490216732025
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5541999936103821
Epoch 49, CIFAR-10 Batch 4:  cost: 0.6714150309562683
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5616000294685364
Epoch 49, CIFAR-10 Batch 5:  cost: 0.5245173573493958
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5551999807357788
Epoch 50, CIFAR-10 Batch 1:  cost: 0.5464602112770081
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5658000111579895
Epoch 50, CIFAR-10 Batch 2:  cost: 0.5566408634185791
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5580000281333923
Epoch 50, CIFAR-10 Batch 3:  cost: 0.39582714438438416
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5601999759674072
Epoch 50, CIFAR-10 Batch 4:  cost: 0.6972211599349976
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5623999834060669
Epoch 50, CIFAR-10 Batch 5:  cost: 0.5480585098266602
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5559999942779541
Epoch 51, CIFAR-10 Batch 1:  cost: 0.5349699854850769
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5645999908447266
Epoch 51, CIFAR-10 Batch 2:  cost: 0.4223613739013672
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5583999752998352
Epoch 51, CIFAR-10 Batch 3:  cost: 0.36475738883018494
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5541999936103821
Epoch 51, CIFAR-10 Batch 4:  cost: 0.6738499402999878
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5609999895095825
Epoch 51, CIFAR-10 Batch 5:  cost: 0.5370926856994629
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5583999752998352
Epoch 52, CIFAR-10 Batch 1:  cost: 0.5180691480636597
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5663999915122986
Epoch 52, CIFAR-10 Batch 2:  cost: 0.4393174648284912
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5609999895095825
Epoch 52, CIFAR-10 Batch 3:  cost: 0.39651599526405334
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5504000186920166
Epoch 52, CIFAR-10 Batch 4:  cost: 0.7335732579231262
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5594000220298767
Epoch 52, CIFAR-10 Batch 5:  cost: 0.48661455512046814
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5573999881744385
Epoch 53, CIFAR-10 Batch 1:  cost: 0.5260228514671326
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5631999969482422
Epoch 53, CIFAR-10 Batch 2:  cost: 0.40189725160598755
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5559999942779541
Epoch 53, CIFAR-10 Batch 3:  cost: 0.4388606548309326
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5544000267982483
Epoch 53, CIFAR-10 Batch 4:  cost: 0.6094866394996643
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5564000010490417
Epoch 53, CIFAR-10 Batch 5:  cost: 0.4912472665309906
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5558000206947327
Epoch 54, CIFAR-10 Batch 1:  cost: 0.5246866345405579
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5667999982833862
Epoch 54, CIFAR-10 Batch 2:  cost: 0.4265210032463074
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5587999820709229
Epoch 54, CIFAR-10 Batch 3:  cost: 0.39516758918762207
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5559999942779541
Epoch 54, CIFAR-10 Batch 4:  cost: 0.6389408111572266
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5654000043869019
Epoch 54, CIFAR-10 Batch 5:  cost: 0.500857949256897
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5608000159263611
Epoch 55, CIFAR-10 Batch 1:  cost: 0.5961089134216309
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5680000185966492
Epoch 55, CIFAR-10 Batch 2:  cost: 0.392406702041626
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5627999901771545
Epoch 55, CIFAR-10 Batch 3:  cost: 0.4309419095516205
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5533999800682068
Epoch 55, CIFAR-10 Batch 4:  cost: 0.695247232913971
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5583999752998352
Epoch 55, CIFAR-10 Batch 5:  cost: 0.5341053009033203
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5586000084877014
Epoch 56, CIFAR-10 Batch 1:  cost: 0.5174110531806946
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5649999976158142
Epoch 56, CIFAR-10 Batch 2:  cost: 0.4024113714694977
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5636000037193298
Epoch 56, CIFAR-10 Batch 3:  cost: 0.40414661169052124
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5529999732971191
Epoch 56, CIFAR-10 Batch 4:  cost: 0.7298717498779297
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5605999827384949
Epoch 56, CIFAR-10 Batch 5:  cost: 0.5023411512374878
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5604000091552734
Epoch 57, CIFAR-10 Batch 1:  cost: 0.5475084185600281
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5699999928474426
Epoch 57, CIFAR-10 Batch 2:  cost: 0.45268717408180237
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5591999888420105
Epoch 57, CIFAR-10 Batch 3:  cost: 0.35764080286026
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5576000213623047
Epoch 57, CIFAR-10 Batch 4:  cost: 0.6866479516029358
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5616000294685364
Epoch 57, CIFAR-10 Batch 5:  cost: 0.5214086174964905
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5601999759674072
Epoch 58, CIFAR-10 Batch 1:  cost: 0.5441111326217651
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5669999718666077
Epoch 58, CIFAR-10 Batch 2:  cost: 0.3936314582824707
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5637999773025513
Epoch 58, CIFAR-10 Batch 3:  cost: 0.3618367910385132
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5527999997138977
Epoch 58, CIFAR-10 Batch 4:  cost: 0.6581114530563354
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5613999962806702
Epoch 58, CIFAR-10 Batch 5:  cost: 0.5151031613349915
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5605999827384949
Epoch 59, CIFAR-10 Batch 1:  cost: 0.5424922704696655
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5659999847412109
Epoch 59, CIFAR-10 Batch 2:  cost: 0.3735049366950989
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.557200014591217
Epoch 59, CIFAR-10 Batch 3:  cost: 0.3761618137359619
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5509999990463257
Epoch 59, CIFAR-10 Batch 4:  cost: 0.6687957048416138
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5569999814033508
Epoch 59, CIFAR-10 Batch 5:  cost: 0.5282933115959167
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5497999787330627
Epoch 60, CIFAR-10 Batch 1:  cost: 0.5352619886398315
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5649999976158142
Epoch 60, CIFAR-10 Batch 2:  cost: 0.42362159490585327
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5587999820709229
Epoch 60, CIFAR-10 Batch 3:  cost: 0.43055295944213867
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5522000193595886
Epoch 60, CIFAR-10 Batch 4:  cost: 0.6277788877487183
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5612000226974487
Epoch 60, CIFAR-10 Batch 5:  cost: 0.5055710077285767
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5568000078201294
Epoch 61, CIFAR-10 Batch 1:  cost: 0.5482994914054871
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5618000030517578
Epoch 61, CIFAR-10 Batch 2:  cost: 0.4033995568752289
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5612000226974487
Epoch 61, CIFAR-10 Batch 3:  cost: 0.32802656292915344
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5546000003814697
Epoch 61, CIFAR-10 Batch 4:  cost: 0.6817282438278198
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5619999766349792
Epoch 61, CIFAR-10 Batch 5:  cost: 0.5373446941375732
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5595999956130981
Epoch 62, CIFAR-10 Batch 1:  cost: 0.4984724521636963
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5676000118255615
Epoch 62, CIFAR-10 Batch 2:  cost: 0.41591256856918335
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5655999779701233
Epoch 62, CIFAR-10 Batch 3:  cost: 0.32474279403686523
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5565999746322632
Epoch 62, CIFAR-10 Batch 4:  cost: 0.6650069355964661
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5600000023841858
Epoch 62, CIFAR-10 Batch 5:  cost: 0.496198832988739
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5604000091552734
Epoch 63, CIFAR-10 Batch 1:  cost: 0.5992010235786438
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5673999786376953
Epoch 63, CIFAR-10 Batch 2:  cost: 0.45530596375465393
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5619999766349792
Epoch 63, CIFAR-10 Batch 3:  cost: 0.3030206561088562
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5577999949455261
Epoch 63, CIFAR-10 Batch 4:  cost: 0.6614595651626587
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5619999766349792
Epoch 63, CIFAR-10 Batch 5:  cost: 0.482616662979126
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5533999800682068
Epoch 64, CIFAR-10 Batch 1:  cost: 0.528153657913208
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5676000118255615
Epoch 64, CIFAR-10 Batch 2:  cost: 0.3868577778339386
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5627999901771545
Epoch 64, CIFAR-10 Batch 3:  cost: 0.4052714705467224
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5565999746322632
Epoch 64, CIFAR-10 Batch 4:  cost: 0.703106701374054
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5604000091552734
Epoch 64, CIFAR-10 Batch 5:  cost: 0.5242943167686462
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5547999739646912
Epoch 65, CIFAR-10 Batch 1:  cost: 0.49169760942459106
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5618000030517578
Epoch 65, CIFAR-10 Batch 2:  cost: 0.3456120193004608
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5576000213623047
Epoch 65, CIFAR-10 Batch 3:  cost: 0.41074472665786743
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5569999814033508
Epoch 65, CIFAR-10 Batch 4:  cost: 0.7052297592163086
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5623999834060669
Epoch 65, CIFAR-10 Batch 5:  cost: 0.5081018209457397
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5600000023841858
Epoch 66, CIFAR-10 Batch 1:  cost: 0.5608131289482117
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5654000043869019
Epoch 66, CIFAR-10 Batch 2:  cost: 0.4301835000514984
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5587999820709229
Epoch 66, CIFAR-10 Batch 3:  cost: 0.3835882544517517
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5595999956130981
Epoch 66, CIFAR-10 Batch 4:  cost: 0.6928542256355286
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5633999705314636
Epoch 66, CIFAR-10 Batch 5:  cost: 0.4354894757270813
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5590000152587891
Epoch 67, CIFAR-10 Batch 1:  cost: 0.4904927611351013
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5673999786376953
Epoch 67, CIFAR-10 Batch 2:  cost: 0.3610917031764984
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5604000091552734
Epoch 67, CIFAR-10 Batch 3:  cost: 0.32230687141418457
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5582000017166138
Epoch 67, CIFAR-10 Batch 4:  cost: 0.6559661030769348
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5608000159263611
Epoch 67, CIFAR-10 Batch 5:  cost: 0.45970335602760315
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5573999881744385
Epoch 68, CIFAR-10 Batch 1:  cost: 0.4741639196872711
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5626000165939331
Epoch 68, CIFAR-10 Batch 2:  cost: 0.3709608018398285
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5644000172615051
Epoch 68, CIFAR-10 Batch 3:  cost: 0.3182133436203003
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5550000071525574
Epoch 68, CIFAR-10 Batch 4:  cost: 0.6627107858657837
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.557200014591217
Epoch 68, CIFAR-10 Batch 5:  cost: 0.49875515699386597
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5573999881744385
Epoch 69, CIFAR-10 Batch 1:  cost: 0.5265465974807739
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5622000098228455
Epoch 69, CIFAR-10 Batch 2:  cost: 0.4132497310638428
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5598000288009644
Epoch 69, CIFAR-10 Batch 3:  cost: 0.3418808579444885
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5533999800682068
Epoch 69, CIFAR-10 Batch 4:  cost: 0.6701520681381226
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5604000091552734
Epoch 69, CIFAR-10 Batch 5:  cost: 0.5683687329292297
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5533999800682068
Epoch 70, CIFAR-10 Batch 1:  cost: 0.4792167544364929
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5637999773025513
Epoch 70, CIFAR-10 Batch 2:  cost: 0.4381483197212219
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5547999739646912
Epoch 70, CIFAR-10 Batch 3:  cost: 0.35821667313575745
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5540000200271606
Epoch 70, CIFAR-10 Batch 4:  cost: 0.6262253522872925
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5604000091552734
Epoch 70, CIFAR-10 Batch 5:  cost: 0.462064653635025
accuracy: Tensor("accuracy:0", shape=(), dtype=float32)
validation accuracy: 0.5519999861717224

Checkpoint

The model has been saved to disk.

Test Model

Test your model against the test dataset. This will be your final accuracy. You should have an accuracy greater than 50%. If you don't, keep tweaking the model architecture and parameters.



In [81]:

    
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import tensorflow as tf
import pickle
import helper
import random

# Set batch size if not already set
try:
    if batch_size:
        pass
except NameError:
    batch_size = 64

save_model_path = './image_classification'
n_samples = 4
top_n_predictions = 3

def test_model():
    """
    Test the saved model against the test dataset
    """

    test_features, test_labels = pickle.load(open('preprocess_test.p', mode='rb'))
    loaded_graph = tf.Graph()

    with tf.Session(graph=loaded_graph) as sess:
        # Load model
        loader = tf.train.import_meta_graph(save_model_path + '.meta')
        loader.restore(sess, save_model_path)

        # Get Tensors from loaded model
        loaded_x = loaded_graph.get_tensor_by_name('x:0')
        loaded_y = loaded_graph.get_tensor_by_name('y:0')
        loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')
        loaded_logits = loaded_graph.get_tensor_by_name('logits:0')
        loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0')
        
        # Get accuracy in batches for memory limitations
        test_batch_acc_total = 0
        test_batch_count = 0
        
        for test_feature_batch, test_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size):
            test_batch_acc_total += sess.run(
                loaded_acc,
                feed_dict={loaded_x: test_feature_batch, loaded_y: test_label_batch, loaded_keep_prob: 1.0})
            test_batch_count += 1

        print('Testing Accuracy: {}\n'.format(test_batch_acc_total/test_batch_count))

        # Print Random Samples
        random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples)))
        random_test_predictions = sess.run(
            tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions),
            feed_dict={loaded_x: random_test_features, loaded_y: random_test_labels, loaded_keep_prob: 1.0})
        helper.display_image_predictions(random_test_features, random_test_labels, random_test_predictions)


test_model()









    



INFO:tensorflow:Restoring parameters from ./image_classification
Testing Accuracy: 0.5626953125

Why 50-80% Accuracy?

You might be wondering why you can't get an accuracy any higher. First things first, 50% isn't bad for a simple CNN. Pure guessing would get you 10% accuracy. However, you might notice people are getting scores well above 80%. That's because we haven't taught you all there is to know about neural networks. We still need to cover a few more techniques.

Submitting This Project

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_image_classification.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.