notebook.community

Edit and run

Image Classification

In this project, you'll classify images from the CIFAR-10 dataset. The dataset consists of airplanes, dogs, cats, and other objects. You'll preprocess the images, then train a convolutional neural network on all the samples. The images need to be normalized and the labels need to be one-hot encoded. You'll get to apply what you learned and build a convolutional, max pooling, dropout, and fully connected layers. At the end, you'll get to see your neural network's predictions on the sample images.

Get the Data

Run the following cell to download the CIFAR-10 dataset for python.





In [1]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
from urllib.request import urlretrieve
from os.path import isfile, isdir
from tqdm import tqdm
import problem_unittests as tests
import tarfile

cifar10_dataset_folder_path = 'cifar-10-batches-py'

# Use Floyd's cifar-10 dataset if present
floyd_cifar10_location = '/input/cifar-10/python.tar.gz'
if isfile(floyd_cifar10_location):
    tar_gz_path = floyd_cifar10_location
else:
    tar_gz_path = 'cifar-10-python.tar.gz'

class DLProgress(tqdm):
    last_block = 0

    def hook(self, block_num=1, block_size=1, total_size=None):
        self.total = total_size
        self.update((block_num - self.last_block) * block_size)
        self.last_block = block_num

if not isfile(tar_gz_path):
    with DLProgress(unit='B', unit_scale=True, miniters=1, desc='CIFAR-10 Dataset') as pbar:
        urlretrieve(
            'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz',
            tar_gz_path,
            pbar.hook)

if not isdir(cifar10_dataset_folder_path):
    with tarfile.open(tar_gz_path) as tar:
        tar.extractall()
        tar.close()


tests.test_folder_path(cifar10_dataset_folder_path)



    


















    






All files found!

Explore the Data

The dataset is broken into batches to prevent your machine from running out of memory. The CIFAR-10 dataset consists of 5 batches, named data_batch_1, data_batch_2, etc.. Each batch contains the labels and images that are one of the following:

  • airplane
  • automobile
  • bird
  • cat
  • deer
  • dog
  • frog
  • horse
  • ship
  • truck

Understanding a dataset is part of making predictions on the data. Play around with the code cell below by changing the batch_id and sample_id. The batch_id is the id for a batch (1-5). The sample_id is the id for a image and label pair in the batch.

Ask yourself "What are all possible labels?", "What is the range of values for the image data?", "Are the labels in order or random?". Answers to questions like these will help you preprocess the data and end up with better predictions.





In [2]:



    


%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import helper
import numpy as np

# Explore the dataset
batch_id = 1
sample_id = 5
helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)



    


















    






Stats of batch 1:
Samples: 10000
Label Counts: {0: 1005, 1: 974, 2: 1032, 3: 1016, 4: 999, 5: 937, 6: 1030, 7: 1001, 8: 1025, 9: 981}
First 20 Labels: [6, 9, 9, 4, 1, 1, 2, 7, 8, 3, 4, 7, 7, 2, 9, 9, 9, 3, 2, 6]

Example of Image 5:
Image - Min Value: 0 Max Value: 252
Image - Shape: (32, 32, 3)
Label - Label Id: 1 Name: automobile

Implement Preprocess Functions

Normalize

In the cell below, implement the normalize function to take in image data, x, and return it as a normalized Numpy array. The values should be in the range of 0 to 1, inclusive. The return object should be the same shape as x.





In [3]:



    


def normalize(x, range_min=0, range_max=255):
    """
    Normalize a list of sample image data in the range of 0 to 1
    : x: List of image data.  The image shape is (32, 32, 3)
    : return: Numpy array of normalize data
    """
    # Avoiding exactly zero and one, due to possible saturation issues with some activation functions
    # or risks of underflow
    a = 0
    b = 1.0
    range_min = 0
    range_max = 255
    return a + ( ( (x - range_min)*(b - a) )/( range_max - range_min ) )


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_normalize(normalize)



    


















    






Tests Passed

One-hot encode

Just like the previous code cell, you'll be implementing a function for preprocessing. This time, you'll implement the one_hot_encode function. The input, x, are a list of labels. Implement the function to return the list of labels as One-Hot encoded Numpy array. The possible values for labels are 0 to 9. The one-hot encoding function should return the same encoding for each value between each call to one_hot_encode. Make sure to save the map of encodings outside the function.

Hint: Don't reinvent the wheel.





In [4]:



    


def one_hot_encode(x, n_labels=10):
    """
    One hot encode a list of sample labels. Return a one-hot encoded vector for each label.
    : x: List of sample Labels
    : return: Numpy array of one-hot encoded labels
    """
    # ohe via identity matrix for labels times examples
    # should not change between uses unless labels change and there is
    # no need for outer scope mutation of variables
    return np.eye(n_labels)[x]


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_one_hot_encode(one_hot_encode)



    


















    






Tests Passed

Randomize Data

As you saw from exploring the data above, the order of the samples are randomized. It doesn't hurt to randomize it again, but you don't need to for this dataset.

Preprocess all the data and save it

Running the code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation.





In [5]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Preprocess Training, Validation, and Testing Data
helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)

Check Point

This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.





In [6]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle
import problem_unittests as tests
import helper

# Load the Preprocessed Validation data
valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))

Build the network

For the neural network, you'll build each layer into a function. Most of the code you've seen has been outside of functions. To test your code more thoroughly, we require that you put each layer in a function. This allows us to give you better feedback and test for simple mistakes using our unittests before you submit your project.

Note: If you're finding it hard to dedicate enough time for this course each week, we've provided a small shortcut to this part of the project. In the next couple of problems, you'll have the option to use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages to build each layer, except the layers you build in the "Convolutional and Max Pooling Layer" section. TF Layers is similar to Keras's and TFLearn's abstraction to layers, so it's easy to pickup.

However, if you would like to get the most out of this course, try to solve all the problems without using anything from the TF Layers packages. You can still use classes from other packages that happen to have the same name as ones you find in TF Layers! For example, instead of using the TF Layers version of the conv2d class, tf.layers.conv2d, you would want to use the TF Neural Network version of conv2d, tf.nn.conv2d.

Let's begin!

Input

The neural network needs to read the image data, one-hot encoded labels, and dropout keep probability. Implement the following functions

  • Implement neural_net_image_input
    • Return a TF Placeholder
    • Set the shape using image_shape with batch size set to None.
    • Name the TensorFlow placeholder "x" using the TensorFlow name parameter in the TF Placeholder.
  • Implement neural_net_label_input
    • Return a TF Placeholder
    • Set the shape using n_classes with batch size set to None.
    • Name the TensorFlow placeholder "y" using the TensorFlow name parameter in the TF Placeholder.
  • Implement neural_net_keep_prob_input
    • Return a TF Placeholder for dropout keep probability.
    • Name the TensorFlow placeholder "keep_prob" using the TensorFlow name parameter in the TF Placeholder.

These names will be used at the end of the project to load your saved model.

Note: None for shapes in TensorFlow allow for a dynamic size.





In [7]:



    


import tensorflow as tf

def neural_net_image_input(image_shape):
    """
    Return a Tensor for a batch of image input
    : image_shape: Shape of the images
    : return: Tensor for image input.
    """
    return tf.placeholder(tf.float32, shape=[None, *image_shape], name="x")


def neural_net_label_input(n_classes, channels=3):
    """
    Return a Tensor for a batch of label input
    : n_classes: Number of classes
    : return: Tensor for label input.
    """
    return tf.placeholder(tf.float32, shape=[None, n_classes,], name="y")


def neural_net_keep_prob_input():
    """
    Return a Tensor for keep probability
    : return: Tensor for keep probability.
    """
    return tf.placeholder(tf.float32, name="keep_prob")


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tf.reset_default_graph()
tests.test_nn_image_inputs(neural_net_image_input)
tests.test_nn_label_inputs(neural_net_label_input)
tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)



    


















    






Image Input Tests Passed.
Label Input Tests Passed.
Keep Prob Tests Passed.

Convolution and Max Pooling Layer

Convolution layers have a lot of success with images. For this code cell, you should implement the function conv2d_maxpool to apply convolution then max pooling:

  • Create the weight and bias using conv_ksize, conv_num_outputs and the shape of x_tensor.
  • Apply a convolution to x_tensor using weight and conv_strides.
    • We recommend you use same padding, but you're welcome to use any padding.
  • Add bias
  • Add a nonlinear activation to the convolution.
  • Apply Max Pooling using pool_ksize and pool_strides.
    • We recommend you use same padding, but you're welcome to use any padding.

Note: You can't use TensorFlow Layers or TensorFlow Layers (contrib) for this layer, but you can still use TensorFlow's Neural Network package. You may still use the shortcut option for all the other layers.





In [8]:



    


def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
    """
    Apply convolution then max pooling to x_tensor
    :param x_tensor: TensorFlow Tensor
    :param conv_num_outputs: Number of outputs for the convolutional layer
    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer
    :param conv_strides: Stride 2-D Tuple for convolution
    :param pool_ksize: kernal size 2-D Tuple for pool
    :param pool_strides: Stride 2-D Tuple for pool
    : return: A tensor that represents convolution and max pooling of x_tensor
    """
    W = tf.Variable(tf.random_normal(
        shape=[conv_ksize[0], conv_ksize[1], x_tensor.get_shape().as_list()[3], conv_num_outputs],
        mean=0.0,
        stddev=0.01,
        dtype=tf.float32))
    
    b = tf.Variable(tf.zeros([conv_num_outputs]))
    #print(conv_strides)
    conv = tf.nn.conv2d(x_tensor, W, strides=[1, *conv_strides, 1], padding="SAME")
    conv = tf.nn.bias_add(conv, b)
    conv = tf.nn.relu(conv)
    
    conv = tf.nn.max_pool(conv,
                          [1, *pool_ksize, 1],
                          [1, *pool_strides, 1],
                          padding="SAME")
    
    return conv

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_con_pool(conv2d_maxpool)



    


















    






Tests Passed

Flatten Layer

Implement the flatten function to change the dimension of x_tensor from a 4-D tensor to a 2-D tensor. The output should be the shape (Batch Size, Flattened Image Size). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.





In [9]:



    


def flatten(x_tensor):
    """
    Flatten x_tensor to (Batch Size, Flattened Image Size)
    : x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.
    : return: A tensor of size (Batch Size, Flattened Image Size).
    """
    # Highlevel is nice
    return tf.contrib.layers.flatten(x_tensor)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_flatten(flatten)



    


















    






Tests Passed

Fully-Connected Layer

Implement the fully_conn function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.





In [10]:



    


def fully_conn(x_tensor, num_outputs):
    """
    Apply a fully connected layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    return tf.contrib.layers.fully_connected(x_tensor,
                                            num_outputs,
                                            weights_initializer=tf.random_normal_initializer(mean=0.0, stddev=0.1),
                                            #biased in favor of activating, with biases > 0, since we use relu 
                                            biases_initializer=tf.random_normal_initializer(mean=0.1, stddev=0.01),
                                            activation_fn=tf.nn.relu)

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_fully_conn(fully_conn)



    


















    






Tests Passed

Output Layer

Implement the output function to apply a fully connected layer to x_tensor with the shape (Batch Size, num_outputs). Shortcut option: you can use classes from the TensorFlow Layers or TensorFlow Layers (contrib) packages for this layer. For more of a challenge, only use other TensorFlow packages.

Note: Activation, softmax, or cross entropy should not be applied to this.





In [11]:



    


def output(x_tensor, num_outputs):
    """
    Apply a output layer to x_tensor using weight and bias
    : x_tensor: A 2-D tensor where the first dimension is batch size.
    : num_outputs: The number of output that the new tensor should be.
    : return: A 2-D tensor where the second dimension is num_outputs.
    """
    return tf.contrib.layers.fully_connected(x_tensor,
                                            num_outputs,
                                            weights_initializer=tf.random_normal_initializer(mean=0.0, stddev=0.01),
                                            biases_initializer=tf.zeros_initializer(), activation_fn=None)             


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_output(output)



    


















    






Tests Passed

Create Convolutional Model

Implement the function conv_net to create a convolutional neural network model. The function takes in a batch of images, x, and outputs logits. Use the layers you created above to create this model:

  • Apply 1, 2, or 3 Convolution and Max Pool layers
  • Apply a Flatten Layer
  • Apply 1, 2, or 3 Fully Connected Layers
  • Apply an Output Layer
  • Return the output
  • Apply TensorFlow's Dropout to one or more layers in the model using keep_prob. 




In [24]:



    


def conv_net(x, keep_prob):
    """
    Create a convolutional neural network model
    : x: Placeholder tensor that holds image data.
    : keep_prob: Placeholder tensor that hold dropout keep probability.
    : return: Tensor that represents logits
    """
    #x_ = tf.cast(x, tf.float32)
    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
    #    Play around with different number of outputs, kernel size and stride
    # Function Definition from Above:
    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    
    conv1 = conv2d_maxpool(x, 32, (2,2), (2,2), (3,3), (2,2))
    conv2 = conv2d_maxpool(conv1, 64, (2,2), (2,2), (1,1), (1,1))
    conv3 = conv2d_maxpool(conv2, 128, (2,2), (2,2), (1,1), (1,1))
    
    # TODO: Apply a Flatten Layer
    # Function Definition from Above:
    #   flatten(x_tensor)
    f1 = flatten(conv3)

    # TODO: Apply 1, 2, or 3 Fully Connected Layers
    #    Play around with different number of outputs
    # Function Definition from Above:
    #   fully_conn(x_tensor, num_outputs)
    net = fully_conn(f1,400)
    drop1 = tf.nn.dropout(net, keep_prob)
    net2 = fully_conn(drop1,200)
    drop2 = tf.nn.dropout(net2, keep_prob)
    net3 = fully_conn(drop2,100)
    drop3 = tf.nn.dropout(net3, keep_prob)
    
    
    # TODO: Apply an Output Layer
    #    Set this to the number of classes
    # Function Definition from Above:
    #   output(x_tensor, num_outputs)
    return output(drop3,10)


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""

##############################
## Build the Neural Network ##
##############################

# Remove previous weights, bias, inputs, etc..
tf.reset_default_graph()

# Inputs
x = neural_net_image_input((32, 32, 3))
y = neural_net_label_input(10)
keep_prob = neural_net_keep_prob_input()

# Model
logits = conv_net(x, keep_prob)

# Name logits Tensor, so that is can be loaded from disk after training
logits = tf.identity(logits, name='logits')

# Loss and Optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))
optimizer = tf.train.AdamOptimizer().minimize(cost)

# Accuracy
correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')

tests.test_conv_net(conv_net)



    


















    






Neural Network Built!

Train the Neural Network

Single Optimization

Implement the function train_neural_network to do a single optimization. The optimization should use optimizer to optimize in session with a feed_dict of the following:

  • x for image input
  • y for labels
  • keep_prob for keep probability for dropout

This function will be called for each batch, so tf.global_variables_initializer() has already been called.

Note: Nothing needs to be returned. This function is only optimizing the neural network.





In [13]:



    


def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):
    """
    Optimize the session on a batch of images and labels
    : session: Current TensorFlow session|
    : optimizer: TensorFlow optimizer function
    : keep_probability: keep probability
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    """
    # Just the side-effect
    session.run(optimizer, feed_dict={x:feature_batch, y:label_batch, keep_prob:keep_probability})


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_train_nn(train_neural_network)



    


















    






Tests Passed

Show Stats

Implement the function print_stats to print loss and validation accuracy. Use the global variables valid_features and valid_labels to calculate validation accuracy. Use a keep probability of 1.0 to calculate the loss and validation accuracy.





In [14]:



    


def print_stats(session, feature_batch, label_batch, cost, accuracy):
    """
    Print information about loss and validation accuracy
    : session: Current TensorFlow session
    : feature_batch: Batch of Numpy image data
    : label_batch: Batch of Numpy label data
    : cost: TensorFlow cost function
    : accuracy: TensorFlow accuracy function
    """
    # TODO: Implement Function
    loss = session.run(cost, feed_dict={x: feature_batch, y:label_batch, keep_prob:1.0})
    valid_acc = session.run(accuracy, feed_dict={x:valid_features, y:valid_labels, keep_prob:1.0})
    print("Current loss: {0}, validation accuracy: {1}".format(loss, valid_acc))

Hyperparameters

Tune the following parameters:

  • Set epochs to the number of iterations until the network stops learning or start overfitting
  • Set batch_size to the highest number that your machine has memory for. Most people set them to common sizes of memory:
    • 64
    • 128
    • 256
    • ...
  • Set keep_probability to the probability of keeping a node using dropout




In [21]:



    


# TODO: Tune Parameters
epochs = 100
batch_size = 1024 # 1080 TI
keep_probability = 0.5

Train on a Single CIFAR-10 Batch

Instead of training the neural network on all the CIFAR-10 batches of data, let's use a single batch. This should save time while you iterate on the model to get a better accuracy. Once the final validation accuracy is 50% or greater, run the model on all the data in the next section.





In [16]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
print('Checking the Training on a Single Batch...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())
    
    # Training cycle
    for epoch in range(epochs):
        batch_i = 1
        for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
        print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
        print_stats(sess, batch_features, batch_labels, cost, accuracy)



    


















    






Checking the Training on a Single Batch...
Epoch  1, CIFAR-10 Batch 1:  Current loss: 2.248844623565674, validation accuracy: 0.20539997518062592
Epoch  2, CIFAR-10 Batch 1:  Current loss: 2.193892240524292, validation accuracy: 0.21699997782707214
Epoch  3, CIFAR-10 Batch 1:  Current loss: 2.053370475769043, validation accuracy: 0.2694000005722046
Epoch  4, CIFAR-10 Batch 1:  Current loss: 1.9182370901107788, validation accuracy: 0.29819998145103455
Epoch  5, CIFAR-10 Batch 1:  Current loss: 1.8380732536315918, validation accuracy: 0.32919999957084656
Epoch  6, CIFAR-10 Batch 1:  Current loss: 1.7471296787261963, validation accuracy: 0.34679996967315674
Epoch  7, CIFAR-10 Batch 1:  Current loss: 1.6630527973175049, validation accuracy: 0.35920000076293945
Epoch  8, CIFAR-10 Batch 1:  Current loss: 1.5910017490386963, validation accuracy: 0.35979998111724854
Epoch  9, CIFAR-10 Batch 1:  Current loss: 1.4300475120544434, validation accuracy: 0.3891999423503876
Epoch 10, CIFAR-10 Batch 1:  Current loss: 1.238672137260437, validation accuracy: 0.3993999660015106
Epoch 11, CIFAR-10 Batch 1:  Current loss: 1.160847783088684, validation accuracy: 0.3887999653816223
Epoch 12, CIFAR-10 Batch 1:  Current loss: 1.1160986423492432, validation accuracy: 0.40219998359680176
Epoch 13, CIFAR-10 Batch 1:  Current loss: 1.0851023197174072, validation accuracy: 0.4115999937057495
Epoch 14, CIFAR-10 Batch 1:  Current loss: 1.1535950899124146, validation accuracy: 0.37219998240470886
Epoch 15, CIFAR-10 Batch 1:  Current loss: 0.8306964039802551, validation accuracy: 0.42639994621276855
Epoch 16, CIFAR-10 Batch 1:  Current loss: 0.7638354301452637, validation accuracy: 0.42559996247291565
Epoch 17, CIFAR-10 Batch 1:  Current loss: 0.9095578193664551, validation accuracy: 0.39879995584487915
Epoch 18, CIFAR-10 Batch 1:  Current loss: 0.9466768503189087, validation accuracy: 0.3977999687194824
Epoch 19, CIFAR-10 Batch 1:  Current loss: 0.6658151745796204, validation accuracy: 0.4147999584674835
Epoch 20, CIFAR-10 Batch 1:  Current loss: 0.5380934476852417, validation accuracy: 0.42559996247291565
Epoch 21, CIFAR-10 Batch 1:  Current loss: 0.48428937792778015, validation accuracy: 0.42319995164871216
Epoch 22, CIFAR-10 Batch 1:  Current loss: 0.47009211778640747, validation accuracy: 0.43299993872642517
Epoch 23, CIFAR-10 Batch 1:  Current loss: 0.4796065092086792, validation accuracy: 0.4365999698638916
Epoch 24, CIFAR-10 Batch 1:  Current loss: 0.5535961985588074, validation accuracy: 0.4211999475955963
Epoch 25, CIFAR-10 Batch 1:  Current loss: 0.4773564338684082, validation accuracy: 0.4293999671936035
Epoch 26, CIFAR-10 Batch 1:  Current loss: 0.4587085247039795, validation accuracy: 0.3991999626159668
Epoch 27, CIFAR-10 Batch 1:  Current loss: 0.43005484342575073, validation accuracy: 0.41339996457099915
Epoch 28, CIFAR-10 Batch 1:  Current loss: 0.3547744154930115, validation accuracy: 0.43359994888305664
Epoch 29, CIFAR-10 Batch 1:  Current loss: 0.40824252367019653, validation accuracy: 0.42799994349479675
Epoch 30, CIFAR-10 Batch 1:  Current loss: 0.3893629312515259, validation accuracy: 0.4315999448299408
Epoch 31, CIFAR-10 Batch 1:  Current loss: 0.3072076439857483, validation accuracy: 0.438400000333786
Epoch 32, CIFAR-10 Batch 1:  Current loss: 0.2896547317504883, validation accuracy: 0.42659997940063477
Epoch 33, CIFAR-10 Batch 1:  Current loss: 0.284682959318161, validation accuracy: 0.41599997878074646
Epoch 34, CIFAR-10 Batch 1:  Current loss: 0.30325260758399963, validation accuracy: 0.4009999632835388
Epoch 35, CIFAR-10 Batch 1:  Current loss: 0.22628751397132874, validation accuracy: 0.4071999788284302
Epoch 36, CIFAR-10 Batch 1:  Current loss: 0.2069549262523651, validation accuracy: 0.41839995980262756
Epoch 37, CIFAR-10 Batch 1:  Current loss: 0.2152658998966217, validation accuracy: 0.4113999903202057
Epoch 38, CIFAR-10 Batch 1:  Current loss: 0.16393697261810303, validation accuracy: 0.42479997873306274
Epoch 39, CIFAR-10 Batch 1:  Current loss: 0.14927801489830017, validation accuracy: 0.41899996995925903
Epoch 40, CIFAR-10 Batch 1:  Current loss: 0.1186784878373146, validation accuracy: 0.4215999245643616
Epoch 41, CIFAR-10 Batch 1:  Current loss: 0.1091570109128952, validation accuracy: 0.407399982213974
Epoch 42, CIFAR-10 Batch 1:  Current loss: 0.11489968001842499, validation accuracy: 0.41239994764328003
Epoch 43, CIFAR-10 Batch 1:  Current loss: 0.10624594241380692, validation accuracy: 0.4269999861717224
Epoch 44, CIFAR-10 Batch 1:  Current loss: 0.1272270679473877, validation accuracy: 0.417199969291687
Epoch 45, CIFAR-10 Batch 1:  Current loss: 0.12597660720348358, validation accuracy: 0.4020000100135803
Epoch 46, CIFAR-10 Batch 1:  Current loss: 0.08959510922431946, validation accuracy: 0.41919997334480286
Epoch 47, CIFAR-10 Batch 1:  Current loss: 0.06634116172790527, validation accuracy: 0.433199942111969
Epoch 48, CIFAR-10 Batch 1:  Current loss: 0.10742978751659393, validation accuracy: 0.423799991607666
Epoch 49, CIFAR-10 Batch 1:  Current loss: 0.11164231598377228, validation accuracy: 0.41200000047683716
Epoch 50, CIFAR-10 Batch 1:  Current loss: 0.07921856641769409, validation accuracy: 0.4257999658584595

Fully Train the Model

Now that you got a good accuracy with a single CIFAR-10 batch, try it with all five batches.





In [22]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
save_model_path = './image_classification'

print('Training...')
with tf.Session() as sess:
    # Initializing the variables
    sess.run(tf.global_variables_initializer())
    
    # Training cycle
    for epoch in range(epochs):
        # Loop over all batches
        n_batches = 5
        for batch_i in range(1, n_batches + 1):
            for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):
                train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)
            print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')
            print_stats(sess, batch_features, batch_labels, cost, accuracy)
            
    # Save Model
    saver = tf.train.Saver()
    save_path = saver.save(sess, save_model_path)



    


















    






Training...
Epoch  1, CIFAR-10 Batch 1:  Current loss: 2.3033130168914795, validation accuracy: 0.11380000412464142
Epoch  1, CIFAR-10 Batch 2:  Current loss: 2.302288293838501, validation accuracy: 0.0997999906539917
Epoch  1, CIFAR-10 Batch 3:  Current loss: 2.296550750732422, validation accuracy: 0.11719999462366104
Epoch  1, CIFAR-10 Batch 4:  Current loss: 2.2673776149749756, validation accuracy: 0.17139999568462372
Epoch  1, CIFAR-10 Batch 5:  Current loss: 2.136650323867798, validation accuracy: 0.15359999239444733
Epoch  2, CIFAR-10 Batch 1:  Current loss: 2.174053430557251, validation accuracy: 0.17659997940063477
Epoch  2, CIFAR-10 Batch 2:  Current loss: 2.1049187183380127, validation accuracy: 0.20559997856616974
Epoch  2, CIFAR-10 Batch 3:  Current loss: 2.07749080657959, validation accuracy: 0.2279999852180481
Epoch  2, CIFAR-10 Batch 4:  Current loss: 2.044461250305176, validation accuracy: 0.23219998180866241
Epoch  2, CIFAR-10 Batch 5:  Current loss: 1.9970595836639404, validation accuracy: 0.24479998648166656
Epoch  3, CIFAR-10 Batch 1:  Current loss: 2.0666017532348633, validation accuracy: 0.25779998302459717
Epoch  3, CIFAR-10 Batch 2:  Current loss: 2.0238912105560303, validation accuracy: 0.26079997420310974
Epoch  3, CIFAR-10 Batch 3:  Current loss: 2.002115249633789, validation accuracy: 0.25540000200271606
Epoch  3, CIFAR-10 Batch 4:  Current loss: 1.9865777492523193, validation accuracy: 0.25759997963905334
Epoch  3, CIFAR-10 Batch 5:  Current loss: 1.9692015647888184, validation accuracy: 0.25599998235702515
Epoch  4, CIFAR-10 Batch 1:  Current loss: 2.0243892669677734, validation accuracy: 0.25759997963905334
Epoch  4, CIFAR-10 Batch 2:  Current loss: 1.9936773777008057, validation accuracy: 0.2694000005722046
Epoch  4, CIFAR-10 Batch 3:  Current loss: 1.9599580764770508, validation accuracy: 0.26440000534057617
Epoch  4, CIFAR-10 Batch 4:  Current loss: 1.9532328844070435, validation accuracy: 0.2637999653816223
Epoch  4, CIFAR-10 Batch 5:  Current loss: 1.9378063678741455, validation accuracy: 0.27079999446868896
Epoch  5, CIFAR-10 Batch 1:  Current loss: 1.9761768579483032, validation accuracy: 0.2671999931335449
Epoch  5, CIFAR-10 Batch 2:  Current loss: 1.941598653793335, validation accuracy: 0.2877999544143677
Epoch  5, CIFAR-10 Batch 3:  Current loss: 1.909096121788025, validation accuracy: 0.29019999504089355
Epoch  5, CIFAR-10 Batch 4:  Current loss: 1.8717933893203735, validation accuracy: 0.28599995374679565
Epoch  5, CIFAR-10 Batch 5:  Current loss: 1.8772006034851074, validation accuracy: 0.30139997601509094
Epoch  6, CIFAR-10 Batch 1:  Current loss: 1.8923395872116089, validation accuracy: 0.31120002269744873
Epoch  6, CIFAR-10 Batch 2:  Current loss: 1.8836779594421387, validation accuracy: 0.2993999719619751
Epoch  6, CIFAR-10 Batch 3:  Current loss: 1.8474403619766235, validation accuracy: 0.3091999888420105
Epoch  6, CIFAR-10 Batch 4:  Current loss: 1.7962137460708618, validation accuracy: 0.30799996852874756
Epoch  6, CIFAR-10 Batch 5:  Current loss: 1.8336968421936035, validation accuracy: 0.3172000050544739
Epoch  7, CIFAR-10 Batch 1:  Current loss: 1.806767225265503, validation accuracy: 0.34699997305870056
Epoch  7, CIFAR-10 Batch 2:  Current loss: 1.7758889198303223, validation accuracy: 0.3383999764919281
Epoch  7, CIFAR-10 Batch 3:  Current loss: 1.7339439392089844, validation accuracy: 0.3385999798774719
Epoch  7, CIFAR-10 Batch 4:  Current loss: 1.700564980506897, validation accuracy: 0.3497999608516693
Epoch  7, CIFAR-10 Batch 5:  Current loss: 1.7348965406417847, validation accuracy: 0.35179996490478516
Epoch  8, CIFAR-10 Batch 1:  Current loss: 1.7336143255233765, validation accuracy: 0.36580002307891846
Epoch  8, CIFAR-10 Batch 2:  Current loss: 1.7045890092849731, validation accuracy: 0.3539999723434448
Epoch  8, CIFAR-10 Batch 3:  Current loss: 1.6696527004241943, validation accuracy: 0.35999998450279236
Epoch  8, CIFAR-10 Batch 4:  Current loss: 1.6481900215148926, validation accuracy: 0.35739997029304504
Epoch  8, CIFAR-10 Batch 5:  Current loss: 1.694109320640564, validation accuracy: 0.37619996070861816
Epoch  9, CIFAR-10 Batch 1:  Current loss: 1.6758755445480347, validation accuracy: 0.382999986410141
Epoch  9, CIFAR-10 Batch 2:  Current loss: 1.6524684429168701, validation accuracy: 0.3821999728679657
Epoch  9, CIFAR-10 Batch 3:  Current loss: 1.6213955879211426, validation accuracy: 0.3791999816894531
Epoch  9, CIFAR-10 Batch 4:  Current loss: 1.6043428182601929, validation accuracy: 0.37619999051094055
Epoch  9, CIFAR-10 Batch 5:  Current loss: 1.6444075107574463, validation accuracy: 0.39739999175071716
Epoch 10, CIFAR-10 Batch 1:  Current loss: 1.644352674484253, validation accuracy: 0.39539995789527893
Epoch 10, CIFAR-10 Batch 2:  Current loss: 1.6123493909835815, validation accuracy: 0.3993999660015106
Epoch 10, CIFAR-10 Batch 3:  Current loss: 1.5603359937667847, validation accuracy: 0.4057999849319458
Epoch 10, CIFAR-10 Batch 4:  Current loss: 1.542702317237854, validation accuracy: 0.4023999869823456
Epoch 10, CIFAR-10 Batch 5:  Current loss: 1.585768222808838, validation accuracy: 0.4111999571323395
Epoch 11, CIFAR-10 Batch 1:  Current loss: 1.5886847972869873, validation accuracy: 0.4171999990940094
Epoch 11, CIFAR-10 Batch 2:  Current loss: 1.563481092453003, validation accuracy: 0.4214000105857849
Epoch 11, CIFAR-10 Batch 3:  Current loss: 1.5151405334472656, validation accuracy: 0.41519999504089355
Epoch 11, CIFAR-10 Batch 4:  Current loss: 1.500848412513733, validation accuracy: 0.41439998149871826
Epoch 11, CIFAR-10 Batch 5:  Current loss: 1.543463945388794, validation accuracy: 0.4244000017642975
Epoch 12, CIFAR-10 Batch 1:  Current loss: 1.5544182062149048, validation accuracy: 0.42800000309944153
Epoch 12, CIFAR-10 Batch 2:  Current loss: 1.5271373987197876, validation accuracy: 0.43759995698928833
Epoch 12, CIFAR-10 Batch 3:  Current loss: 1.4651789665222168, validation accuracy: 0.43140000104904175
Epoch 12, CIFAR-10 Batch 4:  Current loss: 1.472571611404419, validation accuracy: 0.43379995226860046
Epoch 12, CIFAR-10 Batch 5:  Current loss: 1.4972827434539795, validation accuracy: 0.44199997186660767
Epoch 13, CIFAR-10 Batch 1:  Current loss: 1.5251513719558716, validation accuracy: 0.4419999420642853
Epoch 13, CIFAR-10 Batch 2:  Current loss: 1.499840497970581, validation accuracy: 0.4423999786376953
Epoch 13, CIFAR-10 Batch 3:  Current loss: 1.438433289527893, validation accuracy: 0.44359999895095825
Epoch 13, CIFAR-10 Batch 4:  Current loss: 1.4496924877166748, validation accuracy: 0.44899997115135193
Epoch 13, CIFAR-10 Batch 5:  Current loss: 1.4706193208694458, validation accuracy: 0.4545999765396118
Epoch 14, CIFAR-10 Batch 1:  Current loss: 1.494853138923645, validation accuracy: 0.4519999623298645
Epoch 14, CIFAR-10 Batch 2:  Current loss: 1.473792552947998, validation accuracy: 0.45379993319511414
Epoch 14, CIFAR-10 Batch 3:  Current loss: 1.399970531463623, validation accuracy: 0.4599999785423279
Epoch 14, CIFAR-10 Batch 4:  Current loss: 1.4201065301895142, validation accuracy: 0.4545999765396118
Epoch 14, CIFAR-10 Batch 5:  Current loss: 1.4154447317123413, validation accuracy: 0.47039994597435
Epoch 15, CIFAR-10 Batch 1:  Current loss: 1.4562081098556519, validation accuracy: 0.4705999493598938
Epoch 15, CIFAR-10 Batch 2:  Current loss: 1.4246481657028198, validation accuracy: 0.4715999662876129
Epoch 15, CIFAR-10 Batch 3:  Current loss: 1.3763048648834229, validation accuracy: 0.47019997239112854
Epoch 15, CIFAR-10 Batch 4:  Current loss: 1.4038212299346924, validation accuracy: 0.46399998664855957
Epoch 15, CIFAR-10 Batch 5:  Current loss: 1.3955373764038086, validation accuracy: 0.4747999906539917
Epoch 16, CIFAR-10 Batch 1:  Current loss: 1.4334676265716553, validation accuracy: 0.47839999198913574
Epoch 16, CIFAR-10 Batch 2:  Current loss: 1.3886253833770752, validation accuracy: 0.4835999608039856
Epoch 16, CIFAR-10 Batch 3:  Current loss: 1.3484041690826416, validation accuracy: 0.47519993782043457
Epoch 16, CIFAR-10 Batch 4:  Current loss: 1.3653696775436401, validation accuracy: 0.4779999852180481
Epoch 16, CIFAR-10 Batch 5:  Current loss: 1.3654158115386963, validation accuracy: 0.48819997906684875
Epoch 17, CIFAR-10 Batch 1:  Current loss: 1.4470309019088745, validation accuracy: 0.4771999716758728
Epoch 17, CIFAR-10 Batch 2:  Current loss: 1.370904803276062, validation accuracy: 0.4901999533176422
Epoch 17, CIFAR-10 Batch 3:  Current loss: 1.3281278610229492, validation accuracy: 0.48159995675086975
Epoch 17, CIFAR-10 Batch 4:  Current loss: 1.3598885536193848, validation accuracy: 0.48079997301101685
Epoch 17, CIFAR-10 Batch 5:  Current loss: 1.3342607021331787, validation accuracy: 0.4873999357223511
Epoch 18, CIFAR-10 Batch 1:  Current loss: 1.4070751667022705, validation accuracy: 0.4893999695777893
Epoch 18, CIFAR-10 Batch 2:  Current loss: 1.3484323024749756, validation accuracy: 0.4999999701976776
Epoch 18, CIFAR-10 Batch 3:  Current loss: 1.3057059049606323, validation accuracy: 0.488599956035614
Epoch 18, CIFAR-10 Batch 4:  Current loss: 1.3354393243789673, validation accuracy: 0.4917999505996704
Epoch 18, CIFAR-10 Batch 5:  Current loss: 1.3146684169769287, validation accuracy: 0.4949999451637268
Epoch 19, CIFAR-10 Batch 1:  Current loss: 1.3831210136413574, validation accuracy: 0.4941999614238739
Epoch 19, CIFAR-10 Batch 2:  Current loss: 1.3203939199447632, validation accuracy: 0.501800000667572
Epoch 19, CIFAR-10 Batch 3:  Current loss: 1.2849769592285156, validation accuracy: 0.5003999471664429
Epoch 19, CIFAR-10 Batch 4:  Current loss: 1.3104519844055176, validation accuracy: 0.4963999390602112
Epoch 19, CIFAR-10 Batch 5:  Current loss: 1.2987958192825317, validation accuracy: 0.5031999349594116
Epoch 20, CIFAR-10 Batch 1:  Current loss: 1.3720602989196777, validation accuracy: 0.5003999471664429
Epoch 20, CIFAR-10 Batch 2:  Current loss: 1.3026702404022217, validation accuracy: 0.5108000040054321
Epoch 20, CIFAR-10 Batch 3:  Current loss: 1.2764108180999756, validation accuracy: 0.5037999749183655
Epoch 20, CIFAR-10 Batch 4:  Current loss: 1.3027362823486328, validation accuracy: 0.4999999403953552
Epoch 20, CIFAR-10 Batch 5:  Current loss: 1.2768350839614868, validation accuracy: 0.5137999057769775
Epoch 21, CIFAR-10 Batch 1:  Current loss: 1.3473451137542725, validation accuracy: 0.5089999437332153
Epoch 21, CIFAR-10 Batch 2:  Current loss: 1.281428575515747, validation accuracy: 0.5179999470710754
Epoch 21, CIFAR-10 Batch 3:  Current loss: 1.2455487251281738, validation accuracy: 0.511199951171875
Epoch 21, CIFAR-10 Batch 4:  Current loss: 1.2936944961547852, validation accuracy: 0.5101999044418335
Epoch 21, CIFAR-10 Batch 5:  Current loss: 1.280057668685913, validation accuracy: 0.5121999382972717
Epoch 22, CIFAR-10 Batch 1:  Current loss: 1.3237632513046265, validation accuracy: 0.5216000080108643
Epoch 22, CIFAR-10 Batch 2:  Current loss: 1.2662019729614258, validation accuracy: 0.5181999802589417
Epoch 22, CIFAR-10 Batch 3:  Current loss: 1.22526216506958, validation accuracy: 0.5185999870300293
Epoch 22, CIFAR-10 Batch 4:  Current loss: 1.2667317390441895, validation accuracy: 0.5221999287605286
Epoch 22, CIFAR-10 Batch 5:  Current loss: 1.2475265264511108, validation accuracy: 0.5227999687194824
Epoch 23, CIFAR-10 Batch 1:  Current loss: 1.3092594146728516, validation accuracy: 0.5233999490737915
Epoch 23, CIFAR-10 Batch 2:  Current loss: 1.2500752210617065, validation accuracy: 0.5283999443054199
Epoch 23, CIFAR-10 Batch 3:  Current loss: 1.2069116830825806, validation accuracy: 0.5227999091148376
Epoch 23, CIFAR-10 Batch 4:  Current loss: 1.2567604780197144, validation accuracy: 0.5267999172210693
Epoch 23, CIFAR-10 Batch 5:  Current loss: 1.236279010772705, validation accuracy: 0.5285999774932861
Epoch 24, CIFAR-10 Batch 1:  Current loss: 1.3153564929962158, validation accuracy: 0.524199903011322
Epoch 24, CIFAR-10 Batch 2:  Current loss: 1.2382774353027344, validation accuracy: 0.5321999192237854
Epoch 24, CIFAR-10 Batch 3:  Current loss: 1.207827091217041, validation accuracy: 0.5245999097824097
Epoch 24, CIFAR-10 Batch 4:  Current loss: 1.2404178380966187, validation accuracy: 0.5317999720573425
Epoch 24, CIFAR-10 Batch 5:  Current loss: 1.2110050916671753, validation accuracy: 0.5345999002456665
Epoch 25, CIFAR-10 Batch 1:  Current loss: 1.2746496200561523, validation accuracy: 0.535599946975708
Epoch 25, CIFAR-10 Batch 2:  Current loss: 1.2182354927062988, validation accuracy: 0.5347999334335327
Epoch 25, CIFAR-10 Batch 3:  Current loss: 1.1727912425994873, validation accuracy: 0.5329999327659607
Epoch 25, CIFAR-10 Batch 4:  Current loss: 1.2306857109069824, validation accuracy: 0.5377999544143677
Epoch 25, CIFAR-10 Batch 5:  Current loss: 1.2001768350601196, validation accuracy: 0.5429999828338623
Epoch 26, CIFAR-10 Batch 1:  Current loss: 1.255519986152649, validation accuracy: 0.5415999293327332
Epoch 26, CIFAR-10 Batch 2:  Current loss: 1.2332072257995605, validation accuracy: 0.5267999768257141
Epoch 26, CIFAR-10 Batch 3:  Current loss: 1.1751939058303833, validation accuracy: 0.5375999212265015
Epoch 26, CIFAR-10 Batch 4:  Current loss: 1.214090347290039, validation accuracy: 0.5393999218940735
Epoch 26, CIFAR-10 Batch 5:  Current loss: 1.1794085502624512, validation accuracy: 0.5445999503135681
Epoch 27, CIFAR-10 Batch 1:  Current loss: 1.2579810619354248, validation accuracy: 0.5435999035835266
Epoch 27, CIFAR-10 Batch 2:  Current loss: 1.2000651359558105, validation accuracy: 0.5461999177932739
Epoch 27, CIFAR-10 Batch 3:  Current loss: 1.1441724300384521, validation accuracy: 0.5446000099182129
Epoch 27, CIFAR-10 Batch 4:  Current loss: 1.2009212970733643, validation accuracy: 0.5483999252319336
Epoch 27, CIFAR-10 Batch 5:  Current loss: 1.1661474704742432, validation accuracy: 0.5473999381065369
Epoch 28, CIFAR-10 Batch 1:  Current loss: 1.2479504346847534, validation accuracy: 0.5477999448776245
Epoch 28, CIFAR-10 Batch 2:  Current loss: 1.1812927722930908, validation accuracy: 0.5523999333381653
Epoch 28, CIFAR-10 Batch 3:  Current loss: 1.1359316110610962, validation accuracy: 0.5517998933792114
Epoch 28, CIFAR-10 Batch 4:  Current loss: 1.181126594543457, validation accuracy: 0.554599940776825
Epoch 28, CIFAR-10 Batch 5:  Current loss: 1.1456291675567627, validation accuracy: 0.5527999401092529
Epoch 29, CIFAR-10 Batch 1:  Current loss: 1.2364027500152588, validation accuracy: 0.554599940776825
Epoch 29, CIFAR-10 Batch 2:  Current loss: 1.1732220649719238, validation accuracy: 0.5517998933792114
Epoch 29, CIFAR-10 Batch 3:  Current loss: 1.133467674255371, validation accuracy: 0.5483999252319336
Epoch 29, CIFAR-10 Batch 4:  Current loss: 1.1747194528579712, validation accuracy: 0.550399899482727
Epoch 29, CIFAR-10 Batch 5:  Current loss: 1.1462711095809937, validation accuracy: 0.5491999387741089
Epoch 30, CIFAR-10 Batch 1:  Current loss: 1.213852882385254, validation accuracy: 0.5525999069213867
Epoch 30, CIFAR-10 Batch 2:  Current loss: 1.158491849899292, validation accuracy: 0.5565999746322632
Epoch 30, CIFAR-10 Batch 3:  Current loss: 1.1156926155090332, validation accuracy: 0.5571999549865723
Epoch 30, CIFAR-10 Batch 4:  Current loss: 1.1662662029266357, validation accuracy: 0.5519999265670776
Epoch 30, CIFAR-10 Batch 5:  Current loss: 1.1319901943206787, validation accuracy: 0.5585998892784119
Epoch 31, CIFAR-10 Batch 1:  Current loss: 1.199095368385315, validation accuracy: 0.5649999380111694
Epoch 31, CIFAR-10 Batch 2:  Current loss: 1.1482092142105103, validation accuracy: 0.5613999366760254
Epoch 31, CIFAR-10 Batch 3:  Current loss: 1.1032757759094238, validation accuracy: 0.5623999834060669
Epoch 31, CIFAR-10 Batch 4:  Current loss: 1.1515402793884277, validation accuracy: 0.5619999170303345
Epoch 31, CIFAR-10 Batch 5:  Current loss: 1.1177241802215576, validation accuracy: 0.564799964427948
Epoch 32, CIFAR-10 Batch 1:  Current loss: 1.185390830039978, validation accuracy: 0.5605999231338501
Epoch 32, CIFAR-10 Batch 2:  Current loss: 1.1392009258270264, validation accuracy: 0.5643998980522156
Epoch 32, CIFAR-10 Batch 3:  Current loss: 1.0957480669021606, validation accuracy: 0.5577999353408813
Epoch 32, CIFAR-10 Batch 4:  Current loss: 1.158510446548462, validation accuracy: 0.5541999936103821
Epoch 32, CIFAR-10 Batch 5:  Current loss: 1.108923077583313, validation accuracy: 0.5641999244689941
Epoch 33, CIFAR-10 Batch 1:  Current loss: 1.2090040445327759, validation accuracy: 0.5567998886108398
Epoch 33, CIFAR-10 Batch 2:  Current loss: 1.1380857229232788, validation accuracy: 0.5609999299049377
Epoch 33, CIFAR-10 Batch 3:  Current loss: 1.0928553342819214, validation accuracy: 0.561799943447113
Epoch 33, CIFAR-10 Batch 4:  Current loss: 1.1365373134613037, validation accuracy: 0.5631999373435974
Epoch 33, CIFAR-10 Batch 5:  Current loss: 1.092085361480713, validation accuracy: 0.5713999271392822
Epoch 34, CIFAR-10 Batch 1:  Current loss: 1.1781851053237915, validation accuracy: 0.5597999095916748
Epoch 34, CIFAR-10 Batch 2:  Current loss: 1.1155260801315308, validation accuracy: 0.5703999400138855
Epoch 34, CIFAR-10 Batch 3:  Current loss: 1.0756553411483765, validation accuracy: 0.5663999319076538
Epoch 34, CIFAR-10 Batch 4:  Current loss: 1.1327528953552246, validation accuracy: 0.5633999109268188
Epoch 34, CIFAR-10 Batch 5:  Current loss: 1.0821495056152344, validation accuracy: 0.5735999345779419
Epoch 35, CIFAR-10 Batch 1:  Current loss: 1.1533920764923096, validation accuracy: 0.5761998891830444
Epoch 35, CIFAR-10 Batch 2:  Current loss: 1.109769582748413, validation accuracy: 0.5689999461174011
Epoch 35, CIFAR-10 Batch 3:  Current loss: 1.0623644590377808, validation accuracy: 0.5701999664306641
Epoch 35, CIFAR-10 Batch 4:  Current loss: 1.105736494064331, validation accuracy: 0.572399914264679
Epoch 35, CIFAR-10 Batch 5:  Current loss: 1.0727427005767822, validation accuracy: 0.5685999393463135
Epoch 36, CIFAR-10 Batch 1:  Current loss: 1.1456502676010132, validation accuracy: 0.5729999542236328
Epoch 36, CIFAR-10 Batch 2:  Current loss: 1.1057325601577759, validation accuracy: 0.5735999345779419
Epoch 36, CIFAR-10 Batch 3:  Current loss: 1.0541380643844604, validation accuracy: 0.5637999176979065
Epoch 36, CIFAR-10 Batch 4:  Current loss: 1.1119804382324219, validation accuracy: 0.5699999332427979
Epoch 36, CIFAR-10 Batch 5:  Current loss: 1.067123293876648, validation accuracy: 0.5753998756408691
Epoch 37, CIFAR-10 Batch 1:  Current loss: 1.1497722864151, validation accuracy: 0.5757999420166016
Epoch 37, CIFAR-10 Batch 2:  Current loss: 1.0944262742996216, validation accuracy: 0.5721999406814575
Epoch 37, CIFAR-10 Batch 3:  Current loss: 1.0458158254623413, validation accuracy: 0.5771999359130859
Epoch 37, CIFAR-10 Batch 4:  Current loss: 1.0792230367660522, validation accuracy: 0.5733999013900757
Epoch 37, CIFAR-10 Batch 5:  Current loss: 1.0487087965011597, validation accuracy: 0.5785998702049255
Epoch 38, CIFAR-10 Batch 1:  Current loss: 1.1214325428009033, validation accuracy: 0.5757998824119568
Epoch 38, CIFAR-10 Batch 2:  Current loss: 1.0814100503921509, validation accuracy: 0.5689998865127563
Epoch 38, CIFAR-10 Batch 3:  Current loss: 1.0381548404693604, validation accuracy: 0.574199914932251
Epoch 38, CIFAR-10 Batch 4:  Current loss: 1.0924148559570312, validation accuracy: 0.5719999074935913
Epoch 38, CIFAR-10 Batch 5:  Current loss: 1.0417213439941406, validation accuracy: 0.5783999562263489
Epoch 39, CIFAR-10 Batch 1:  Current loss: 1.1327917575836182, validation accuracy: 0.5767999291419983
Epoch 39, CIFAR-10 Batch 2:  Current loss: 1.0739701986312866, validation accuracy: 0.5815998911857605
Epoch 39, CIFAR-10 Batch 3:  Current loss: 1.0310364961624146, validation accuracy: 0.5795998573303223
Epoch 39, CIFAR-10 Batch 4:  Current loss: 1.0743944644927979, validation accuracy: 0.5751999020576477
Epoch 39, CIFAR-10 Batch 5:  Current loss: 1.0325847864151, validation accuracy: 0.5787999033927917
Epoch 40, CIFAR-10 Batch 1:  Current loss: 1.0968666076660156, validation accuracy: 0.5849999189376831
Epoch 40, CIFAR-10 Batch 2:  Current loss: 1.0713026523590088, validation accuracy: 0.577799916267395
Epoch 40, CIFAR-10 Batch 3:  Current loss: 1.0195324420928955, validation accuracy: 0.5805999040603638
Epoch 40, CIFAR-10 Batch 4:  Current loss: 1.0585062503814697, validation accuracy: 0.5773999094963074
Epoch 40, CIFAR-10 Batch 5:  Current loss: 1.0255659818649292, validation accuracy: 0.5803999304771423
Epoch 41, CIFAR-10 Batch 1:  Current loss: 1.1037236452102661, validation accuracy: 0.5785999298095703
Epoch 41, CIFAR-10 Batch 2:  Current loss: 1.0571413040161133, validation accuracy: 0.5809999108314514
Epoch 41, CIFAR-10 Batch 3:  Current loss: 1.0057547092437744, validation accuracy: 0.58079993724823
Epoch 41, CIFAR-10 Batch 4:  Current loss: 1.05045747756958, validation accuracy: 0.5869998931884766
Epoch 41, CIFAR-10 Batch 5:  Current loss: 1.0121383666992188, validation accuracy: 0.5829999446868896
Epoch 42, CIFAR-10 Batch 1:  Current loss: 1.0813716650009155, validation accuracy: 0.584399938583374
Epoch 42, CIFAR-10 Batch 2:  Current loss: 1.067267656326294, validation accuracy: 0.5753998756408691
Epoch 42, CIFAR-10 Batch 3:  Current loss: 1.0054987668991089, validation accuracy: 0.5831999778747559
Epoch 42, CIFAR-10 Batch 4:  Current loss: 1.0499014854431152, validation accuracy: 0.5803999304771423
Epoch 42, CIFAR-10 Batch 5:  Current loss: 1.002225637435913, validation accuracy: 0.5887999534606934
Epoch 43, CIFAR-10 Batch 1:  Current loss: 1.0664880275726318, validation accuracy: 0.5879999399185181
Epoch 43, CIFAR-10 Batch 2:  Current loss: 1.0392732620239258, validation accuracy: 0.5875998735427856
Epoch 43, CIFAR-10 Batch 3:  Current loss: 0.9955238699913025, validation accuracy: 0.5903998613357544
Epoch 43, CIFAR-10 Batch 4:  Current loss: 1.0445067882537842, validation accuracy: 0.582599937915802
Epoch 43, CIFAR-10 Batch 5:  Current loss: 0.9935741424560547, validation accuracy: 0.5885999202728271
Epoch 44, CIFAR-10 Batch 1:  Current loss: 1.057283878326416, validation accuracy: 0.5859999060630798
Epoch 44, CIFAR-10 Batch 2:  Current loss: 1.0275869369506836, validation accuracy: 0.5849999189376831
Epoch 44, CIFAR-10 Batch 3:  Current loss: 0.9963070154190063, validation accuracy: 0.5779998898506165
Epoch 44, CIFAR-10 Batch 4:  Current loss: 1.0283524990081787, validation accuracy: 0.5871999263763428
Epoch 44, CIFAR-10 Batch 5:  Current loss: 0.9967228174209595, validation accuracy: 0.5905998945236206
Epoch 45, CIFAR-10 Batch 1:  Current loss: 1.0484564304351807, validation accuracy: 0.5877999663352966
Epoch 45, CIFAR-10 Batch 2:  Current loss: 1.0298179388046265, validation accuracy: 0.592799961566925
Epoch 45, CIFAR-10 Batch 3:  Current loss: 0.9790416359901428, validation accuracy: 0.5921999216079712
Epoch 45, CIFAR-10 Batch 4:  Current loss: 1.0167112350463867, validation accuracy: 0.591999888420105
Epoch 45, CIFAR-10 Batch 5:  Current loss: 0.9798546433448792, validation accuracy: 0.5859999060630798
Epoch 46, CIFAR-10 Batch 1:  Current loss: 1.0476493835449219, validation accuracy: 0.5903998613357544
Epoch 46, CIFAR-10 Batch 2:  Current loss: 1.0145435333251953, validation accuracy: 0.5923998951911926
Epoch 46, CIFAR-10 Batch 3:  Current loss: 0.9772288799285889, validation accuracy: 0.578999936580658
Epoch 46, CIFAR-10 Batch 4:  Current loss: 0.9982055425643921, validation accuracy: 0.5931999087333679
Epoch 46, CIFAR-10 Batch 5:  Current loss: 0.9624266624450684, validation accuracy: 0.5921999216079712
Epoch 47, CIFAR-10 Batch 1:  Current loss: 1.0333575010299683, validation accuracy: 0.5893999338150024
Epoch 47, CIFAR-10 Batch 2:  Current loss: 1.007537603378296, validation accuracy: 0.5903999209403992
Epoch 47, CIFAR-10 Batch 3:  Current loss: 0.9854673743247986, validation accuracy: 0.5841999053955078
Epoch 47, CIFAR-10 Batch 4:  Current loss: 0.9904624819755554, validation accuracy: 0.5899999141693115
Epoch 47, CIFAR-10 Batch 5:  Current loss: 0.9592539668083191, validation accuracy: 0.5915999412536621
Epoch 48, CIFAR-10 Batch 1:  Current loss: 1.019315481185913, validation accuracy: 0.5953999161720276
Epoch 48, CIFAR-10 Batch 2:  Current loss: 1.0112082958221436, validation accuracy: 0.5809999704360962
Epoch 48, CIFAR-10 Batch 3:  Current loss: 0.94942307472229, validation accuracy: 0.5935999155044556
Epoch 48, CIFAR-10 Batch 4:  Current loss: 0.9863187074661255, validation accuracy: 0.5969999432563782
Epoch 48, CIFAR-10 Batch 5:  Current loss: 0.9493544101715088, validation accuracy: 0.5949998497962952
Epoch 49, CIFAR-10 Batch 1:  Current loss: 1.0246151685714722, validation accuracy: 0.595599889755249
Epoch 49, CIFAR-10 Batch 2:  Current loss: 0.9945570826530457, validation accuracy: 0.5981999039649963
Epoch 49, CIFAR-10 Batch 3:  Current loss: 0.9451367855072021, validation accuracy: 0.5905998945236206
Epoch 49, CIFAR-10 Batch 4:  Current loss: 0.9764630794525146, validation accuracy: 0.5947998762130737
Epoch 49, CIFAR-10 Batch 5:  Current loss: 0.9357438087463379, validation accuracy: 0.5977998971939087
Epoch 50, CIFAR-10 Batch 1:  Current loss: 1.0056740045547485, validation accuracy: 0.5995998978614807
Epoch 50, CIFAR-10 Batch 2:  Current loss: 0.9920178651809692, validation accuracy: 0.5945999026298523
Epoch 50, CIFAR-10 Batch 3:  Current loss: 0.9426530599594116, validation accuracy: 0.5923998951911926
Epoch 50, CIFAR-10 Batch 4:  Current loss: 0.9707605242729187, validation accuracy: 0.5931999683380127
Epoch 50, CIFAR-10 Batch 5:  Current loss: 0.9333773851394653, validation accuracy: 0.5899999141693115
Epoch 51, CIFAR-10 Batch 1:  Current loss: 1.013240933418274, validation accuracy: 0.5957999229431152
Epoch 51, CIFAR-10 Batch 2:  Current loss: 0.9827945232391357, validation accuracy: 0.5963999032974243
Epoch 51, CIFAR-10 Batch 3:  Current loss: 0.9575486779212952, validation accuracy: 0.5879999399185181
Epoch 51, CIFAR-10 Batch 4:  Current loss: 0.9613624811172485, validation accuracy: 0.5947998762130737
Epoch 51, CIFAR-10 Batch 5:  Current loss: 0.9253901243209839, validation accuracy: 0.5987998843193054
Epoch 52, CIFAR-10 Batch 1:  Current loss: 0.9950034618377686, validation accuracy: 0.6005999445915222
Epoch 52, CIFAR-10 Batch 2:  Current loss: 0.974238932132721, validation accuracy: 0.595599889755249
Epoch 52, CIFAR-10 Batch 3:  Current loss: 0.9324707388877869, validation accuracy: 0.5927999019622803
Epoch 52, CIFAR-10 Batch 4:  Current loss: 0.944351077079773, validation accuracy: 0.6009999513626099
Epoch 52, CIFAR-10 Batch 5:  Current loss: 0.9198946356773376, validation accuracy: 0.5989999175071716
Epoch 53, CIFAR-10 Batch 1:  Current loss: 0.9753502607345581, validation accuracy: 0.6031998991966248
Epoch 53, CIFAR-10 Batch 2:  Current loss: 0.9681172370910645, validation accuracy: 0.6009998917579651
Epoch 53, CIFAR-10 Batch 3:  Current loss: 0.9180055260658264, validation accuracy: 0.5961998701095581
Epoch 53, CIFAR-10 Batch 4:  Current loss: 0.9537676572799683, validation accuracy: 0.5977998971939087
Epoch 53, CIFAR-10 Batch 5:  Current loss: 0.9232391119003296, validation accuracy: 0.5959998965263367
Epoch 54, CIFAR-10 Batch 1:  Current loss: 0.979964017868042, validation accuracy: 0.6049998998641968
Epoch 54, CIFAR-10 Batch 2:  Current loss: 0.956017255783081, validation accuracy: 0.6007999181747437
Epoch 54, CIFAR-10 Batch 3:  Current loss: 0.9152208566665649, validation accuracy: 0.5955999493598938
Epoch 54, CIFAR-10 Batch 4:  Current loss: 0.9329077005386353, validation accuracy: 0.6001999378204346
Epoch 54, CIFAR-10 Batch 5:  Current loss: 0.9067485928535461, validation accuracy: 0.5963999032974243
Epoch 55, CIFAR-10 Batch 1:  Current loss: 0.9516561031341553, validation accuracy: 0.6061999797821045
Epoch 55, CIFAR-10 Batch 2:  Current loss: 0.9600745439529419, validation accuracy: 0.6061999201774597
Epoch 55, CIFAR-10 Batch 3:  Current loss: 0.9057983160018921, validation accuracy: 0.6037998795509338
Epoch 55, CIFAR-10 Batch 4:  Current loss: 0.9328957796096802, validation accuracy: 0.5977998971939087
Epoch 55, CIFAR-10 Batch 5:  Current loss: 0.8973745703697205, validation accuracy: 0.6029998660087585
Epoch 56, CIFAR-10 Batch 1:  Current loss: 0.9501095414161682, validation accuracy: 0.6105998754501343
Epoch 56, CIFAR-10 Batch 2:  Current loss: 0.9296905994415283, validation accuracy: 0.6011999249458313
Epoch 56, CIFAR-10 Batch 3:  Current loss: 0.901215136051178, validation accuracy: 0.6005998849868774
Epoch 56, CIFAR-10 Batch 4:  Current loss: 0.9208279848098755, validation accuracy: 0.603399932384491
Epoch 56, CIFAR-10 Batch 5:  Current loss: 0.883637547492981, validation accuracy: 0.6079999208450317
Epoch 57, CIFAR-10 Batch 1:  Current loss: 0.9538583755493164, validation accuracy: 0.6089999079704285
Epoch 57, CIFAR-10 Batch 2:  Current loss: 0.9257053136825562, validation accuracy: 0.6073998808860779
Epoch 57, CIFAR-10 Batch 3:  Current loss: 0.8857935070991516, validation accuracy: 0.608799934387207
Epoch 57, CIFAR-10 Batch 4:  Current loss: 0.915985107421875, validation accuracy: 0.6045999526977539
Epoch 57, CIFAR-10 Batch 5:  Current loss: 0.8880323171615601, validation accuracy: 0.6041998863220215
Epoch 58, CIFAR-10 Batch 1:  Current loss: 0.9547724723815918, validation accuracy: 0.6067999005317688
Epoch 58, CIFAR-10 Batch 2:  Current loss: 0.9247032403945923, validation accuracy: 0.603399932384491
Epoch 58, CIFAR-10 Batch 3:  Current loss: 0.8901357650756836, validation accuracy: 0.6013998985290527
Epoch 58, CIFAR-10 Batch 4:  Current loss: 0.9128149151802063, validation accuracy: 0.6019998788833618
Epoch 58, CIFAR-10 Batch 5:  Current loss: 0.921123206615448, validation accuracy: 0.5935998558998108
Epoch 59, CIFAR-10 Batch 1:  Current loss: 0.9426777362823486, validation accuracy: 0.6083999872207642
Epoch 59, CIFAR-10 Batch 2:  Current loss: 0.9602583050727844, validation accuracy: 0.5981999635696411
Epoch 59, CIFAR-10 Batch 3:  Current loss: 0.8895172476768494, validation accuracy: 0.6013999581336975
Epoch 59, CIFAR-10 Batch 4:  Current loss: 0.9074932336807251, validation accuracy: 0.6073999404907227
Epoch 59, CIFAR-10 Batch 5:  Current loss: 0.8844491839408875, validation accuracy: 0.6039999127388
Epoch 60, CIFAR-10 Batch 1:  Current loss: 0.9274271726608276, validation accuracy: 0.6107999086380005
Epoch 60, CIFAR-10 Batch 2:  Current loss: 0.9288491010665894, validation accuracy: 0.6005999445915222
Epoch 60, CIFAR-10 Batch 3:  Current loss: 0.8765219449996948, validation accuracy: 0.6035999059677124
Epoch 60, CIFAR-10 Batch 4:  Current loss: 0.8914965391159058, validation accuracy: 0.6079999208450317
Epoch 60, CIFAR-10 Batch 5:  Current loss: 0.8773447275161743, validation accuracy: 0.6065999269485474
Epoch 61, CIFAR-10 Batch 1:  Current loss: 0.9100280404090881, validation accuracy: 0.6029998660087585
Epoch 61, CIFAR-10 Batch 2:  Current loss: 0.9192982912063599, validation accuracy: 0.6045998334884644
Epoch 61, CIFAR-10 Batch 3:  Current loss: 0.8727716207504272, validation accuracy: 0.6083999276161194
Epoch 61, CIFAR-10 Batch 4:  Current loss: 0.8981800675392151, validation accuracy: 0.6063999533653259
Epoch 61, CIFAR-10 Batch 5:  Current loss: 0.8765140771865845, validation accuracy: 0.6071999073028564
Epoch 62, CIFAR-10 Batch 1:  Current loss: 0.8944552540779114, validation accuracy: 0.6109999418258667
Epoch 62, CIFAR-10 Batch 2:  Current loss: 0.905096709728241, validation accuracy: 0.6025999188423157
Epoch 62, CIFAR-10 Batch 3:  Current loss: 0.8773354887962341, validation accuracy: 0.6037998795509338
Epoch 62, CIFAR-10 Batch 4:  Current loss: 0.9144548773765564, validation accuracy: 0.6009999513626099
Epoch 62, CIFAR-10 Batch 5:  Current loss: 0.8861162066459656, validation accuracy: 0.6049998998641968
Epoch 63, CIFAR-10 Batch 1:  Current loss: 0.897059440612793, validation accuracy: 0.6109999418258667
Epoch 63, CIFAR-10 Batch 2:  Current loss: 0.8844340443611145, validation accuracy: 0.6103999018669128
Epoch 63, CIFAR-10 Batch 3:  Current loss: 0.8530965447425842, validation accuracy: 0.6125999093055725
Epoch 63, CIFAR-10 Batch 4:  Current loss: 0.8870092630386353, validation accuracy: 0.6067999005317688
Epoch 63, CIFAR-10 Batch 5:  Current loss: 0.8706952929496765, validation accuracy: 0.6037998199462891
Epoch 64, CIFAR-10 Batch 1:  Current loss: 0.8808228969573975, validation accuracy: 0.6097999215126038
Epoch 64, CIFAR-10 Batch 2:  Current loss: 0.8750692009925842, validation accuracy: 0.6097999215126038
Epoch 64, CIFAR-10 Batch 3:  Current loss: 0.8468093872070312, validation accuracy: 0.6159999370574951
Epoch 64, CIFAR-10 Batch 4:  Current loss: 0.8606327176094055, validation accuracy: 0.6153998970985413
Epoch 64, CIFAR-10 Batch 5:  Current loss: 0.8421880006790161, validation accuracy: 0.609799861907959
Epoch 65, CIFAR-10 Batch 1:  Current loss: 0.8736886978149414, validation accuracy: 0.6065998673439026
Epoch 65, CIFAR-10 Batch 2:  Current loss: 0.8816664218902588, validation accuracy: 0.601599931716919
Epoch 65, CIFAR-10 Batch 3:  Current loss: 0.8428242802619934, validation accuracy: 0.6123999357223511
Epoch 65, CIFAR-10 Batch 4:  Current loss: 0.8584738969802856, validation accuracy: 0.6153998970985413
Epoch 65, CIFAR-10 Batch 5:  Current loss: 0.9004998207092285, validation accuracy: 0.5963999032974243
Epoch 66, CIFAR-10 Batch 1:  Current loss: 0.9090099334716797, validation accuracy: 0.6073998808860779
Epoch 66, CIFAR-10 Batch 2:  Current loss: 0.8938666582107544, validation accuracy: 0.608799934387207
Epoch 66, CIFAR-10 Batch 3:  Current loss: 0.8476143479347229, validation accuracy: 0.6145999431610107
Epoch 66, CIFAR-10 Batch 4:  Current loss: 0.8643054366111755, validation accuracy: 0.6135998964309692
Epoch 66, CIFAR-10 Batch 5:  Current loss: 0.8497589826583862, validation accuracy: 0.6119999885559082
Epoch 67, CIFAR-10 Batch 1:  Current loss: 0.8672908544540405, validation accuracy: 0.6173999309539795
Epoch 67, CIFAR-10 Batch 2:  Current loss: 0.875007152557373, validation accuracy: 0.6173998713493347
Epoch 67, CIFAR-10 Batch 3:  Current loss: 0.8491383194923401, validation accuracy: 0.6113998889923096
Epoch 67, CIFAR-10 Batch 4:  Current loss: 0.846795916557312, validation accuracy: 0.6165999174118042
Epoch 67, CIFAR-10 Batch 5:  Current loss: 0.8219383955001831, validation accuracy: 0.6159998774528503
Epoch 68, CIFAR-10 Batch 1:  Current loss: 0.8587188720703125, validation accuracy: 0.6185999512672424
Epoch 68, CIFAR-10 Batch 2:  Current loss: 0.8530974984169006, validation accuracy: 0.6157999038696289
Epoch 68, CIFAR-10 Batch 3:  Current loss: 0.8254032135009766, validation accuracy: 0.618399977684021
Epoch 68, CIFAR-10 Batch 4:  Current loss: 0.8575564026832581, validation accuracy: 0.6091999411582947
Epoch 68, CIFAR-10 Batch 5:  Current loss: 0.8352237939834595, validation accuracy: 0.6115999221801758
Epoch 69, CIFAR-10 Batch 1:  Current loss: 0.8420690894126892, validation accuracy: 0.6167998909950256
Epoch 69, CIFAR-10 Batch 2:  Current loss: 0.8642302751541138, validation accuracy: 0.6105998754501343
Epoch 69, CIFAR-10 Batch 3:  Current loss: 0.8329021334648132, validation accuracy: 0.6159999370574951
Epoch 69, CIFAR-10 Batch 4:  Current loss: 0.8480412364006042, validation accuracy: 0.6113998889923096
Epoch 69, CIFAR-10 Batch 5:  Current loss: 0.8267591595649719, validation accuracy: 0.612799882888794
Epoch 70, CIFAR-10 Batch 1:  Current loss: 0.8537915349006653, validation accuracy: 0.6209999322891235
Epoch 70, CIFAR-10 Batch 2:  Current loss: 0.8435744047164917, validation accuracy: 0.6181998252868652
Epoch 70, CIFAR-10 Batch 3:  Current loss: 0.8190358281135559, validation accuracy: 0.6147999167442322
Epoch 70, CIFAR-10 Batch 4:  Current loss: 0.8316547870635986, validation accuracy: 0.6213998794555664
Epoch 70, CIFAR-10 Batch 5:  Current loss: 0.791297972202301, validation accuracy: 0.6159999370574951
Epoch 71, CIFAR-10 Batch 1:  Current loss: 0.8302674293518066, validation accuracy: 0.6165999174118042
Epoch 71, CIFAR-10 Batch 2:  Current loss: 0.845123291015625, validation accuracy: 0.6109998822212219
Epoch 71, CIFAR-10 Batch 3:  Current loss: 0.8106414079666138, validation accuracy: 0.6183998584747314
Epoch 71, CIFAR-10 Batch 4:  Current loss: 0.8209627866744995, validation accuracy: 0.6181999444961548
Epoch 71, CIFAR-10 Batch 5:  Current loss: 0.7951266765594482, validation accuracy: 0.6179999113082886
Epoch 72, CIFAR-10 Batch 1:  Current loss: 0.8209186792373657, validation accuracy: 0.6173998713493347
Epoch 72, CIFAR-10 Batch 2:  Current loss: 0.8314177989959717, validation accuracy: 0.6165999174118042
Epoch 72, CIFAR-10 Batch 3:  Current loss: 0.7934525012969971, validation accuracy: 0.6225998401641846
Epoch 72, CIFAR-10 Batch 4:  Current loss: 0.8285419940948486, validation accuracy: 0.6201999187469482
Epoch 72, CIFAR-10 Batch 5:  Current loss: 0.7913206219673157, validation accuracy: 0.6179999709129333
Epoch 73, CIFAR-10 Batch 1:  Current loss: 0.8315703272819519, validation accuracy: 0.6141999363899231
Epoch 73, CIFAR-10 Batch 2:  Current loss: 0.8345935344696045, validation accuracy: 0.6125999689102173
Epoch 73, CIFAR-10 Batch 3:  Current loss: 0.7861084938049316, validation accuracy: 0.6251999139785767
Epoch 73, CIFAR-10 Batch 4:  Current loss: 0.8031252026557922, validation accuracy: 0.6205999255180359
Epoch 73, CIFAR-10 Batch 5:  Current loss: 0.7803611755371094, validation accuracy: 0.6197999119758606
Epoch 74, CIFAR-10 Batch 1:  Current loss: 0.8122005462646484, validation accuracy: 0.6191998720169067
Epoch 74, CIFAR-10 Batch 2:  Current loss: 0.8226075172424316, validation accuracy: 0.6159998774528503
Epoch 74, CIFAR-10 Batch 3:  Current loss: 0.7922930121421814, validation accuracy: 0.6199999451637268
Epoch 74, CIFAR-10 Batch 4:  Current loss: 0.8169801235198975, validation accuracy: 0.6199999451637268
Epoch 74, CIFAR-10 Batch 5:  Current loss: 0.7772152423858643, validation accuracy: 0.6201999187469482
Epoch 75, CIFAR-10 Batch 1:  Current loss: 0.8397155404090881, validation accuracy: 0.6081998944282532
Epoch 75, CIFAR-10 Batch 2:  Current loss: 0.811980664730072, validation accuracy: 0.6221998929977417
Epoch 75, CIFAR-10 Batch 3:  Current loss: 0.798638641834259, validation accuracy: 0.6209999322891235
Epoch 75, CIFAR-10 Batch 4:  Current loss: 0.7916067838668823, validation accuracy: 0.6189998984336853
Epoch 75, CIFAR-10 Batch 5:  Current loss: 0.7652091979980469, validation accuracy: 0.6185998320579529
Epoch 76, CIFAR-10 Batch 1:  Current loss: 0.8062639236450195, validation accuracy: 0.6219999194145203
Epoch 76, CIFAR-10 Batch 2:  Current loss: 0.8047344088554382, validation accuracy: 0.6237999200820923
Epoch 76, CIFAR-10 Batch 3:  Current loss: 0.7835473418235779, validation accuracy: 0.6231998801231384
Epoch 76, CIFAR-10 Batch 4:  Current loss: 0.7902117967605591, validation accuracy: 0.621799886226654
Epoch 76, CIFAR-10 Batch 5:  Current loss: 0.7576729655265808, validation accuracy: 0.6237999200820923
Epoch 77, CIFAR-10 Batch 1:  Current loss: 0.796186089515686, validation accuracy: 0.6185998916625977
Epoch 77, CIFAR-10 Batch 2:  Current loss: 0.802761435508728, validation accuracy: 0.6277998685836792
Epoch 77, CIFAR-10 Batch 3:  Current loss: 0.7671138644218445, validation accuracy: 0.6303998827934265
Epoch 77, CIFAR-10 Batch 4:  Current loss: 0.7717053890228271, validation accuracy: 0.6219999194145203
Epoch 77, CIFAR-10 Batch 5:  Current loss: 0.7530354857444763, validation accuracy: 0.6165999174118042
Epoch 78, CIFAR-10 Batch 1:  Current loss: 0.7838191390037537, validation accuracy: 0.6249998807907104
Epoch 78, CIFAR-10 Batch 2:  Current loss: 0.8060716986656189, validation accuracy: 0.6251999139785767
Epoch 78, CIFAR-10 Batch 3:  Current loss: 0.7649095058441162, validation accuracy: 0.6253998875617981
Epoch 78, CIFAR-10 Batch 4:  Current loss: 0.7698356509208679, validation accuracy: 0.6257998943328857
Epoch 78, CIFAR-10 Batch 5:  Current loss: 0.7574146389961243, validation accuracy: 0.6213999390602112
Epoch 79, CIFAR-10 Batch 1:  Current loss: 0.7870264649391174, validation accuracy: 0.6253998875617981
Epoch 79, CIFAR-10 Batch 2:  Current loss: 0.790341854095459, validation accuracy: 0.6253998875617981
Epoch 79, CIFAR-10 Batch 3:  Current loss: 0.7591980695724487, validation accuracy: 0.6237998604774475
Epoch 79, CIFAR-10 Batch 4:  Current loss: 0.7646438479423523, validation accuracy: 0.6257998943328857
Epoch 79, CIFAR-10 Batch 5:  Current loss: 0.7383474111557007, validation accuracy: 0.6221998929977417
Epoch 80, CIFAR-10 Batch 1:  Current loss: 0.7896639704704285, validation accuracy: 0.625999927520752
Epoch 80, CIFAR-10 Batch 2:  Current loss: 0.7855045199394226, validation accuracy: 0.6209998726844788
Epoch 80, CIFAR-10 Batch 3:  Current loss: 0.7450329065322876, validation accuracy: 0.6285999417304993
Epoch 80, CIFAR-10 Batch 4:  Current loss: 0.7559282183647156, validation accuracy: 0.6223999261856079
Epoch 80, CIFAR-10 Batch 5:  Current loss: 0.7503805160522461, validation accuracy: 0.6183998584747314
Epoch 81, CIFAR-10 Batch 1:  Current loss: 0.7776069045066833, validation accuracy: 0.6275999546051025
Epoch 81, CIFAR-10 Batch 2:  Current loss: 0.7726061344146729, validation accuracy: 0.6225998997688293
Epoch 81, CIFAR-10 Batch 3:  Current loss: 0.7489537596702576, validation accuracy: 0.6273998618125916
Epoch 81, CIFAR-10 Batch 4:  Current loss: 0.7491588592529297, validation accuracy: 0.626599907875061
Epoch 81, CIFAR-10 Batch 5:  Current loss: 0.7198561429977417, validation accuracy: 0.6247998476028442
Epoch 82, CIFAR-10 Batch 1:  Current loss: 0.7583383917808533, validation accuracy: 0.6271998882293701
Epoch 82, CIFAR-10 Batch 2:  Current loss: 0.7651654481887817, validation accuracy: 0.622999906539917
Epoch 82, CIFAR-10 Batch 3:  Current loss: 0.7326721549034119, validation accuracy: 0.6297998428344727
Epoch 82, CIFAR-10 Batch 4:  Current loss: 0.7342830896377563, validation accuracy: 0.6271999478340149
Epoch 82, CIFAR-10 Batch 5:  Current loss: 0.7176027894020081, validation accuracy: 0.625999927520752
Epoch 83, CIFAR-10 Batch 1:  Current loss: 0.7774562239646912, validation accuracy: 0.6179999113082886
Epoch 83, CIFAR-10 Batch 2:  Current loss: 0.7782770991325378, validation accuracy: 0.624799907207489
Epoch 83, CIFAR-10 Batch 3:  Current loss: 0.7447217106819153, validation accuracy: 0.6255998611450195
Epoch 83, CIFAR-10 Batch 4:  Current loss: 0.7435073256492615, validation accuracy: 0.626599907875061
Epoch 83, CIFAR-10 Batch 5:  Current loss: 0.7349094748497009, validation accuracy: 0.6191999316215515
Epoch 84, CIFAR-10 Batch 1:  Current loss: 0.7610476016998291, validation accuracy: 0.6227999925613403
Epoch 84, CIFAR-10 Batch 2:  Current loss: 0.7524651885032654, validation accuracy: 0.6245999336242676
Epoch 84, CIFAR-10 Batch 3:  Current loss: 0.7287185788154602, validation accuracy: 0.6321998834609985
Epoch 84, CIFAR-10 Batch 4:  Current loss: 0.7328521609306335, validation accuracy: 0.6215999126434326
Epoch 84, CIFAR-10 Batch 5:  Current loss: 0.7439887523651123, validation accuracy: 0.6183999180793762
Epoch 85, CIFAR-10 Batch 1:  Current loss: 0.7690887451171875, validation accuracy: 0.6209999322891235
Epoch 85, CIFAR-10 Batch 2:  Current loss: 0.7706285119056702, validation accuracy: 0.6329998970031738
Epoch 85, CIFAR-10 Batch 3:  Current loss: 0.7509263753890991, validation accuracy: 0.6181999444961548
Epoch 85, CIFAR-10 Batch 4:  Current loss: 0.7251323461532593, validation accuracy: 0.627599835395813
Epoch 85, CIFAR-10 Batch 5:  Current loss: 0.7003554701805115, validation accuracy: 0.6275999546051025
Epoch 86, CIFAR-10 Batch 1:  Current loss: 0.7621873021125793, validation accuracy: 0.6231999397277832
Epoch 86, CIFAR-10 Batch 2:  Current loss: 0.7528713941574097, validation accuracy: 0.6277998685836792
Epoch 86, CIFAR-10 Batch 3:  Current loss: 0.7305139303207397, validation accuracy: 0.6269999146461487
Epoch 86, CIFAR-10 Batch 4:  Current loss: 0.7221006751060486, validation accuracy: 0.6279999017715454
Epoch 86, CIFAR-10 Batch 5:  Current loss: 0.692726194858551, validation accuracy: 0.6295998692512512
Epoch 87, CIFAR-10 Batch 1:  Current loss: 0.7554509043693542, validation accuracy: 0.6263998746871948
Epoch 87, CIFAR-10 Batch 2:  Current loss: 0.7367328405380249, validation accuracy: 0.6311999559402466
Epoch 87, CIFAR-10 Batch 3:  Current loss: 0.7330764532089233, validation accuracy: 0.6273998618125916
Epoch 87, CIFAR-10 Batch 4:  Current loss: 0.707938551902771, validation accuracy: 0.6343998908996582
Epoch 87, CIFAR-10 Batch 5:  Current loss: 0.6973574161529541, validation accuracy: 0.6241998672485352
Epoch 88, CIFAR-10 Batch 1:  Current loss: 0.745254397392273, validation accuracy: 0.6297999024391174
Epoch 88, CIFAR-10 Batch 2:  Current loss: 0.7437935471534729, validation accuracy: 0.6245999336242676
Epoch 88, CIFAR-10 Batch 3:  Current loss: 0.7047666311264038, validation accuracy: 0.626599907875061
Epoch 88, CIFAR-10 Batch 4:  Current loss: 0.7263152599334717, validation accuracy: 0.6235998868942261
Epoch 88, CIFAR-10 Batch 5:  Current loss: 0.7077502012252808, validation accuracy: 0.6237998604774475
Epoch 89, CIFAR-10 Batch 1:  Current loss: 0.7288833260536194, validation accuracy: 0.627799928188324
Epoch 89, CIFAR-10 Batch 2:  Current loss: 0.738667905330658, validation accuracy: 0.6309998631477356
Epoch 89, CIFAR-10 Batch 3:  Current loss: 0.7255632281303406, validation accuracy: 0.6267999410629272
Epoch 89, CIFAR-10 Batch 4:  Current loss: 0.7084396481513977, validation accuracy: 0.628600001335144
Epoch 89, CIFAR-10 Batch 5:  Current loss: 0.6962612271308899, validation accuracy: 0.6267998814582825
Epoch 90, CIFAR-10 Batch 1:  Current loss: 0.7389129400253296, validation accuracy: 0.6269998550415039
Epoch 90, CIFAR-10 Batch 2:  Current loss: 0.7285398840904236, validation accuracy: 0.6273999214172363
Epoch 90, CIFAR-10 Batch 3:  Current loss: 0.7072975635528564, validation accuracy: 0.6281998753547668
Epoch 90, CIFAR-10 Batch 4:  Current loss: 0.6899606585502625, validation accuracy: 0.6315999627113342
Epoch 90, CIFAR-10 Batch 5:  Current loss: 0.6714890003204346, validation accuracy: 0.6231999397277832
Epoch 91, CIFAR-10 Batch 1:  Current loss: 0.725045382976532, validation accuracy: 0.6237998604774475
Epoch 91, CIFAR-10 Batch 2:  Current loss: 0.7381336092948914, validation accuracy: 0.6255998611450195
Epoch 91, CIFAR-10 Batch 3:  Current loss: 0.706591784954071, validation accuracy: 0.6277998685836792
Epoch 91, CIFAR-10 Batch 4:  Current loss: 0.6995694041252136, validation accuracy: 0.6275998950004578
Epoch 91, CIFAR-10 Batch 5:  Current loss: 0.6791819334030151, validation accuracy: 0.6261999011039734
Epoch 92, CIFAR-10 Batch 1:  Current loss: 0.7240273356437683, validation accuracy: 0.6241998672485352
Epoch 92, CIFAR-10 Batch 2:  Current loss: 0.723568320274353, validation accuracy: 0.6375998854637146
Epoch 92, CIFAR-10 Batch 3:  Current loss: 0.7164618968963623, validation accuracy: 0.626599907875061
Epoch 92, CIFAR-10 Batch 4:  Current loss: 0.7006595730781555, validation accuracy: 0.6319998502731323
Epoch 92, CIFAR-10 Batch 5:  Current loss: 0.6670090556144714, validation accuracy: 0.6283999681472778
Epoch 93, CIFAR-10 Batch 1:  Current loss: 0.7371061444282532, validation accuracy: 0.6235998868942261
Epoch 93, CIFAR-10 Batch 2:  Current loss: 0.72050541639328, validation accuracy: 0.6313998699188232
Epoch 93, CIFAR-10 Batch 3:  Current loss: 0.7215253114700317, validation accuracy: 0.6191999316215515
Epoch 93, CIFAR-10 Batch 4:  Current loss: 0.6985093951225281, validation accuracy: 0.6297999024391174
Epoch 93, CIFAR-10 Batch 5:  Current loss: 0.6644858717918396, validation accuracy: 0.630599856376648
Epoch 94, CIFAR-10 Batch 1:  Current loss: 0.7213842272758484, validation accuracy: 0.624799907207489
Epoch 94, CIFAR-10 Batch 2:  Current loss: 0.7341374158859253, validation accuracy: 0.6291999816894531
Epoch 94, CIFAR-10 Batch 3:  Current loss: 0.6844889521598816, validation accuracy: 0.6319999098777771
Epoch 94, CIFAR-10 Batch 4:  Current loss: 0.6800118684768677, validation accuracy: 0.631199836730957
Epoch 94, CIFAR-10 Batch 5:  Current loss: 0.6893405914306641, validation accuracy: 0.6241999268531799
Epoch 95, CIFAR-10 Batch 1:  Current loss: 0.7386766672134399, validation accuracy: 0.6251999139785767
Epoch 95, CIFAR-10 Batch 2:  Current loss: 0.7175824046134949, validation accuracy: 0.6333999037742615
Epoch 95, CIFAR-10 Batch 3:  Current loss: 0.6897417902946472, validation accuracy: 0.631399929523468
Epoch 95, CIFAR-10 Batch 4:  Current loss: 0.6791183352470398, validation accuracy: 0.63239985704422
Epoch 95, CIFAR-10 Batch 5:  Current loss: 0.676848828792572, validation accuracy: 0.6283999085426331
Epoch 96, CIFAR-10 Batch 1:  Current loss: 0.7025590538978577, validation accuracy: 0.6345999240875244
Epoch 96, CIFAR-10 Batch 2:  Current loss: 0.7145280838012695, validation accuracy: 0.6347998976707458
Epoch 96, CIFAR-10 Batch 3:  Current loss: 0.684427797794342, validation accuracy: 0.631399929523468
Epoch 96, CIFAR-10 Batch 4:  Current loss: 0.6835457682609558, validation accuracy: 0.627799928188324
Epoch 96, CIFAR-10 Batch 5:  Current loss: 0.6634403467178345, validation accuracy: 0.6321998834609985
Epoch 97, CIFAR-10 Batch 1:  Current loss: 0.7049592137336731, validation accuracy: 0.6287999153137207
Epoch 97, CIFAR-10 Batch 2:  Current loss: 0.7061658501625061, validation accuracy: 0.6305999159812927
Epoch 97, CIFAR-10 Batch 3:  Current loss: 0.659064769744873, validation accuracy: 0.6341999173164368
Epoch 97, CIFAR-10 Batch 4:  Current loss: 0.6636720895767212, validation accuracy: 0.6313998699188232
Epoch 97, CIFAR-10 Batch 5:  Current loss: 0.6636860966682434, validation accuracy: 0.6337999105453491
Epoch 98, CIFAR-10 Batch 1:  Current loss: 0.7228919267654419, validation accuracy: 0.625999927520752
Epoch 98, CIFAR-10 Batch 2:  Current loss: 0.6952291131019592, validation accuracy: 0.6367999315261841
Epoch 98, CIFAR-10 Batch 3:  Current loss: 0.6687609553337097, validation accuracy: 0.6369999051094055
Epoch 98, CIFAR-10 Batch 4:  Current loss: 0.6679173707962036, validation accuracy: 0.6321998834609985
Epoch 98, CIFAR-10 Batch 5:  Current loss: 0.6511742472648621, validation accuracy: 0.6309999227523804
Epoch 99, CIFAR-10 Batch 1:  Current loss: 0.6936560869216919, validation accuracy: 0.6329998970031738
Epoch 99, CIFAR-10 Batch 2:  Current loss: 0.6875739097595215, validation accuracy: 0.6385998725891113
Epoch 99, CIFAR-10 Batch 3:  Current loss: 0.6458439826965332, validation accuracy: 0.6349998712539673
Epoch 99, CIFAR-10 Batch 4:  Current loss: 0.6465150713920593, validation accuracy: 0.6289999485015869
Epoch 99, CIFAR-10 Batch 5:  Current loss: 0.6229966878890991, validation accuracy: 0.6379998922348022
Epoch 100, CIFAR-10 Batch 1:  Current loss: 0.699246883392334, validation accuracy: 0.6287999153137207
Epoch 100, CIFAR-10 Batch 2:  Current loss: 0.6909055113792419, validation accuracy: 0.637199878692627
Epoch 100, CIFAR-10 Batch 3:  Current loss: 0.6443589925765991, validation accuracy: 0.6383999586105347
Epoch 100, CIFAR-10 Batch 4:  Current loss: 0.6626986861228943, validation accuracy: 0.6321998238563538
Epoch 100, CIFAR-10 Batch 5:  Current loss: 0.656640350818634, validation accuracy: 0.6281999349594116

Checkpoint

The model has been saved to disk.

Test Model

Test your model against the test dataset. This will be your final accuracy. You should have an accuracy greater than 50%. If you don't, keep tweaking the model architecture and parameters.





In [23]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import tensorflow as tf
import pickle
import helper
import random

# Set batch size if not already set
try:
    if batch_size:
        pass
except NameError:
    batch_size = 64

save_model_path = './image_classification'
n_samples = 4
top_n_predictions = 3

def test_model():
    """
    Test the saved model against the test dataset
    """

    test_features, test_labels = pickle.load(open('preprocess_test.p', mode='rb'))
    loaded_graph = tf.Graph()

    with tf.Session(graph=loaded_graph) as sess:
        # Load model
        loader = tf.train.import_meta_graph(save_model_path + '.meta')
        loader.restore(sess, save_model_path)

        # Get Tensors from loaded model
        loaded_x = loaded_graph.get_tensor_by_name('x:0')
        loaded_y = loaded_graph.get_tensor_by_name('y:0')
        loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')
        loaded_logits = loaded_graph.get_tensor_by_name('logits:0')
        loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0')
        
        # Get accuracy in batches for memory limitations
        test_batch_acc_total = 0
        test_batch_count = 0
        
        for test_feature_batch, test_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size):
            test_batch_acc_total += sess.run(
                loaded_acc,
                feed_dict={loaded_x: test_feature_batch, loaded_y: test_label_batch, loaded_keep_prob: 1.0})
            test_batch_count += 1

        print('Testing Accuracy: {}\n'.format(test_batch_acc_total/test_batch_count))

        # Print Random Samples
        random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples)))
        random_test_predictions = sess.run(
            tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions),
            feed_dict={loaded_x: random_test_features, loaded_y: random_test_labels, loaded_keep_prob: 1.0})
        helper.display_image_predictions(random_test_features, random_test_labels, random_test_predictions)


test_model()



    


















    






Testing Accuracy: 0.624711012840271

Why 50-80% Accuracy?

You might be wondering why you can't get an accuracy any higher. First things first, 50% isn't bad for a simple CNN. Pure guessing would get you 10% accuracy. However, you might notice people are getting scores well above 80%. That's because we haven't taught you all there is to know about neural networks. We still need to cover a few more techniques.

Submitting This Project

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_image_classification.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.

Content source: Riptawr/deep-learning

Similar notebooks:

notebook.community | gallery | about