notebook.community

Edit and run

Language Translation

In this project, you’re going to take a peek into the realm of neural network machine translation. You’ll be training a sequence to sequence model on a dataset of English and French sentences that can translate new sentences from English to French.

Get the Data

Since translating the whole language of English to French will take lots of time to train, we have provided you with a small portion of the English corpus.





In [1]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper
import problem_unittests as tests

source_path = 'data/small_vocab_en'
target_path = 'data/small_vocab_fr'
source_text = helper.load_data(source_path)
target_text = helper.load_data(target_path)

Explore the Data

Play around with view_sentence_range to view different parts of the data.





In [2]:



    


view_sentence_range = (0, 10)

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np

print('Dataset Stats')
print('Roughly the number of unique words: {}'.format(len({word: None for word in source_text.split()})))

sentences = source_text.split('\n')
word_counts = [len(sentence.split()) for sentence in sentences]
print('Number of sentences: {}'.format(len(sentences)))
print('Average number of words in a sentence: {}'.format(np.average(word_counts)))

print()
print('English sentences {} to {}:'.format(*view_sentence_range))
print('\n'.join(source_text.split('\n')[view_sentence_range[0]:view_sentence_range[1]]))
print()
print('French sentences {} to {}:'.format(*view_sentence_range))
print('\n'.join(target_text.split('\n')[view_sentence_range[0]:view_sentence_range[1]]))



    


















    






Dataset Stats
Roughly the number of unique words: 227
Number of sentences: 137861
Average number of words in a sentence: 13.225277634719028

English sentences 0 to 10:
new jersey is sometimes quiet during autumn , and it is snowy in april .
the united states is usually chilly during july , and it is usually freezing in november .
california is usually quiet during march , and it is usually hot in june .
the united states is sometimes mild during june , and it is cold in september .
your least liked fruit is the grape , but my least liked is the apple .
his favorite fruit is the orange , but my favorite is the grape .
paris is relaxing during december , but it is usually chilly in july .
new jersey is busy during spring , and it is never hot in march .
our least liked fruit is the lemon , but my least liked is the grape .
the united states is sometimes busy during january , and it is sometimes warm in november .

French sentences 0 to 10:
new jersey est parfois calme pendant l' automne , et il est neigeux en avril .
les états-unis est généralement froid en juillet , et il gèle habituellement en novembre .
california est généralement calme en mars , et il est généralement chaud en juin .
les états-unis est parfois légère en juin , et il fait froid en septembre .
votre moins aimé fruit est le raisin , mais mon moins aimé est la pomme .
son fruit préféré est l'orange , mais mon préféré est le raisin .
paris est relaxant en décembre , mais il est généralement froid en juillet .
new jersey est occupé au printemps , et il est jamais chaude en mars .
notre fruit est moins aimé le citron , mais mon moins aimé est le raisin .
les états-unis est parfois occupé en janvier , et il est parfois chaud en novembre .

Implement Preprocessing Function

Text to Word Ids

As you did with other RNNs, you must turn the text into a number so the computer can understand it. In the function text_to_ids(), you'll turn source_text and target_text from words to ids. However, you need to add the <EOS> word id at the end of target_text. This will help the neural network predict when the sentence should end.

You can get the <EOS> word id by doing:

target_vocab_to_int['<EOS>']

You can get other word ids using source_vocab_to_int and target_vocab_to_int.





In [25]:



    


def text_to_ids(source_text, target_text, source_vocab_to_int, target_vocab_to_int):
    """
    Convert source and target text to proper word ids
    :param source_text: String that contains all the source text.
    :param target_text: String that contains all the target text.
    :param source_vocab_to_int: Dictionary to go from the source words to an id
    :param target_vocab_to_int: Dictionary to go from the target words to an id
    :return: A tuple of lists (source_id_text, target_id_text)
    """
    # TODO: Implement Function
    
    source_id_text = []
    for sentence in source_text.split('\n'):
        source_id_list = []
        for word in sentence.split():
            source_id_list.append(source_vocab_to_int[word])
        source_id_text.append(source_id_list)
        
    target_id_text = []
    for sentence in target_text.split('\n'):
        target_id_list = []
        for word in sentence.split():
            target_id_list.append(target_vocab_to_int[word])
        target_id_list.append(target_vocab_to_int['<EOS>'])
        target_id_text.append(target_id_list)
    
    
    return source_id_text, target_id_text

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_text_to_ids(text_to_ids)



    


















    






Tests Passed

Preprocess all the data and save it

Running the code cell below will preprocess all the data and save it to file.





In [26]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
helper.preprocess_and_save_data(source_path, target_path, text_to_ids)

Check Point

This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk.





In [27]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np
import helper

(source_int_text, target_int_text), (source_vocab_to_int, target_vocab_to_int), _ = helper.load_preprocess()

Check the Version of TensorFlow and Access to GPU

This will check to make sure you have the correct version of TensorFlow and access to a GPU





In [28]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf
from tensorflow.python.layers.core import Dense

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.1'), 'Please use TensorFlow version 1.1 or newer'
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))



    


















    






TensorFlow Version: 1.1.0










    






/Users/arturops/anaconda3/envs/rnn/lib/python3.6/site-packages/ipykernel_launcher.py:15: UserWarning: No GPU found. Please use a GPU to train your neural network.
  from ipykernel import kernelapp as app

Build the Neural Network

You'll build the components necessary to build a Sequence-to-Sequence model by implementing the following functions below:

  • model_inputs
  • process_decoder_input
  • encoding_layer
  • decoding_layer_train
  • decoding_layer_infer
  • decoding_layer
  • seq2seq_model

Input

Implement the model_inputs() function to create TF Placeholders for the Neural Network. It should create the following placeholders:

  • Input text placeholder named "input" using the TF Placeholder name parameter with rank 2.
  • Targets placeholder with rank 2.
  • Learning rate placeholder with rank 0.
  • Keep probability placeholder named "keep_prob" using the TF Placeholder name parameter with rank 0.
  • Target sequence length placeholder named "target_sequence_length" with rank 1
  • Max target sequence length tensor named "max_target_len" getting its value from applying tf.reduce_max on the target_sequence_length placeholder. Rank 0.
  • Source sequence length placeholder named "source_sequence_length" with rank 1

Return the placeholders in the following the tuple (input, targets, learning rate, keep probability, target sequence length, max target sequence length, source sequence length)





In [29]:



    


def model_inputs():
    """
    Create TF Placeholders for input, targets, learning rate, and lengths of source and target sequences.
    :return: Tuple (input, targets, learning rate, keep probability, target sequence length,
    max target sequence length, source sequence length)
    """
    # TODO: Implement Function
    input_             = tf.placeholder(tf.int32,   shape=[None,None], name='input')
    targets            = tf.placeholder(tf.int32,   shape=[None,None], name='targets')
    learning_rate      = tf.placeholder(tf.float32, shape=None,        name='learning_rate')
    keep_prob          = tf.placeholder(tf.float32, shape=None,        name='keep_prob')
    target_seq_len     = tf.placeholder(tf.int32,   shape=[None],      name='target_sequence_length')
    max_target_seq_len = tf.reduce_max(input_tensor=target_seq_len,    name='max_target_len')
    source_seq_len     = tf.placeholder(tf.int32,   shape=[None],      name='source_sequence_length')
    return input_, targets, learning_rate, keep_prob, target_seq_len, max_target_seq_len, source_seq_len


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_inputs(model_inputs)



    


















    






Tests Passed

Process Decoder Input

Implement process_decoder_input by removing the last word id from each batch in target_data and concat the GO ID to the begining of each batch.





In [37]:



    


def process_decoder_input(target_data, target_vocab_to_int, batch_size):
    """
    Preprocess target data for encoding
    :param target_data: Target Placehoder
    :param target_vocab_to_int: Dictionary to go from the target words to an id
    :param batch_size: Batch Size
    :return: Preprocessed target data
    """
    # TODO: Implement Function
    # remove the last column on the batch (last sequence), strides=[1,1] means one colum
    target_data_removed_end = tf.strided_slice(target_data, begin=[0,0], end=[batch_size,-1], strides=[1,1])
    # append the <GO> int ID to the input data of the decoder
    #   a. Create a tensor with dimensions similar to the batch. Then, GO ID can be appended to all rows of the sequence
    GO_tensor_to_concat = tf.fill(dims=[batch_size,1], value=target_vocab_to_int['<GO>'])
    #   b. Append the GO tensor to the whole batch using axis=1 because appends in column shape
    preprocess_target_data = tf.concat([GO_tensor_to_concat,target_data_removed_end],axis=1)
            
    
    return preprocess_target_data

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_process_encoding_input(process_decoder_input)



    


















    






Tests Passed

Encoding

Implement encoding_layer() to create a Encoder RNN layer:





In [88]:



    


from imp import reload
reload(tests)

def createLSTMcell(rnn_size, keep_prob=None):
    """
    Create LSTM cell with give Dropout probability 1-keep_prob
    :param rnn_size: number of LSTM cells
    :keep_prob: Dropout keep probability
    """
    lstm = tf.contrib.rnn.LSTMCell(rnn_size,initializer=tf.random_uniform_initializer(-0.1, 0.1, seed=2))
    drop_lstm = tf.contrib.rnn.DropoutWrapper(lstm,input_keep_prob=keep_prob)#,output_keep_prob=keep_prob,state_keep_prob=keep_prob)
    
    return drop_lstm
    
def encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob, 
                   source_sequence_length, source_vocab_size, 
                   encoding_embedding_size):
    """
    Create encoding layer
    :param rnn_inputs: Inputs for the RNN
    :param rnn_size: RNN Size
    :param num_layers: Number of layers
    :param keep_prob: Dropout keep probability
    :param source_sequence_length: a list of the lengths of each sequence in the batch
    :param source_vocab_size: vocabulary size of source data
    :param encoding_embedding_size: embedding size of source data
    :return: tuple (RNN output, RNN state)
    """
    # TODO: Implement Function
    
    embed = tf.contrib.layers.embed_sequence(ids=rnn_inputs, vocab_size=source_vocab_size, embed_dim=encoding_embedding_size)
   
    multi_lstm = tf.contrib.rnn.MultiRNNCell([createLSTMcell(rnn_size,keep_prob) for _ in range(num_layers)])
    
    enc_output, enc_state = tf.nn.dynamic_rnn(multi_lstm, embed, sequence_length=source_sequence_length, dtype= tf.float32)
    
    return enc_output, enc_state

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_encoding_layer(encoding_layer)



    


















    






Tests Passed

Decoding - Training

Create a training decoding layer:





In [111]:



    


def decoding_layer_train(encoder_state, dec_cell, dec_embed_input, 
                         target_sequence_length, max_summary_length, 
                         output_layer, keep_prob):
    """
    Create a decoding layer for training
    :param encoder_state: Encoder State
    :param dec_cell: Decoder RNN Cell
    :param dec_embed_input: Decoder embedded input
    :param target_sequence_length: The lengths of each sequence in the target batch
    :param max_summary_length: The length of the longest sequence in the batch
    :param output_layer: Function to apply the output layer
    :param keep_prob: Dropout keep probability
    :return: BasicDecoderOutput containing training logits and sample_id
    """
    # TODO: Implement Function
    
    # :param Time_major: makes computation faster, returns Tensor in time major mode rather than Batch
    # This code fails if set to time_major set to True, do not set it
    #with tf.variable_scope("decoder"):
    training_helper = tf.contrib.seq2seq.TrainingHelper(dec_embed_input, target_sequence_length)
        
        
    basic_decoder = tf.contrib.seq2seq.BasicDecoder(dec_cell, training_helper,\
                                                        initial_state=encoder_state, output_layer=output_layer)
        
    BasicDecoderOutput,_ = tf.contrib.seq2seq.dynamic_decode(basic_decoder, impute_finished=True,\
                                                             maximum_iterations=max_summary_length)
        
    return BasicDecoderOutput



"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_decoding_layer_train(decoding_layer_train)



    


















    






Tests Passed

Decoding - Inference

Create inference decoder:





In [112]:



    


def decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id,
                         end_of_sequence_id, max_target_sequence_length,
                         vocab_size, output_layer, batch_size, keep_prob):
    """
    Create a decoding layer for inference
    :param encoder_state: Encoder state
    :param dec_cell: Decoder RNN Cell
    :param dec_embeddings: Decoder embeddings
    :param start_of_sequence_id: GO ID
    :param end_of_sequence_id: EOS Id
    :param max_target_sequence_length: Maximum length of target sequences
    :param vocab_size: Size of decoder/target vocabulary
    :param decoding_scope: TenorFlow Variable Scope for decoding
    :param output_layer: Function to apply the output layer
    :param batch_size: Batch size
    :param keep_prob: Dropout keep probability
    :return: BasicDecoderOutput containing inference logits and sample_id
    """
    # TODO: Implement Function
    #with tf.variable_scope("decoder", reuse=True):
    
    #column vector of ID's used in each sentence feed on beginning of first sequence of each batch
    start_id = tf.constant([start_of_sequence_id], dtype=tf.int32)
    start_id_tensor = tf.tile(start_id, [batch_size],name='start_tokens')
        
    # GreedyEmbeddingHelper - start_tokens = 1D Tensor of START IDs, end_token = scalar value of END ID
    dec_helper_greedy = tf.contrib.seq2seq.GreedyEmbeddingHelper(dec_embeddings, start_id_tensor, \
                                                                     end_of_sequence_id)
        
    dec_infer_basic = tf.contrib.seq2seq.BasicDecoder(dec_cell, dec_helper_greedy, encoder_state, output_layer=output_layer)
        
    dec_infer_output,_ = tf.contrib.seq2seq.dynamic_decode(dec_infer_basic, impute_finished=True,\
                                                           maximum_iterations=max_target_sequence_length)
        
    
    return dec_infer_output



"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_decoding_layer_infer(decoding_layer_infer)



    


















    






Tests Passed

Build the Decoding Layer

Implement decoding_layer() to create a Decoder RNN layer.

  • Embed the target sequences
  • Construct the decoder LSTM cell (just like you constructed the encoder cell above)
  • Create an output layer to map the outputs of the decoder to the elements of our vocabulary
  • Use the your decoding_layer_train(encoder_state, dec_cell, dec_embed_input, target_sequence_length, max_target_sequence_length, output_layer, keep_prob) function to get the training logits.
  • Use your decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id, end_of_sequence_id, max_target_sequence_length, vocab_size, output_layer, batch_size, keep_prob) function to get the inference logits.

Note: You'll need to use tf.variable_scope to share variables between training and inference.





In [121]:



    


def decoding_layer(dec_input, encoder_state,
                   target_sequence_length, max_target_sequence_length,
                   rnn_size,
                   num_layers, target_vocab_to_int, target_vocab_size,
                   batch_size, keep_prob, decoding_embedding_size):
    """
    Create decoding layer
    :param dec_input: Decoder input
    :param encoder_state: Encoder state
    :param target_sequence_length: The lengths of each sequence in the target batch
    :param max_target_sequence_length: Maximum length of target sequences
    :param rnn_size: RNN Size
    :param num_layers: Number of layers
    :param target_vocab_to_int: Dictionary to go from the target words to an id
    :param target_vocab_size: Size of target vocabulary
    :param batch_size: The size of the batch
    :param keep_prob: Dropout keep probability
    :return: Tuple of (Training BasicDecoderOutput, Inference BasicDecoderOutput)
    """
    # TODO: Implement Function
    
    # 1. EMBEDDING
    # inputs had been preprocessed in decoder_inputs, now we just need to do the embed look_up
    dec_embeddings = tf.Variable(tf.random_uniform([target_vocab_size, decoding_embedding_size]), name='decoder_embed')
    # dec_input - id's of the words to decode
    dec_embedding_input = tf.nn.embedding_lookup(dec_embeddings, dec_input,name='decoder_embed_look_up')
    
    # 2. BUILD DECODER
    decoder_stack_lstm = tf.contrib.rnn.MultiRNNCell([createLSTMcell(rnn_size, keep_prob) for _ in range(num_layers)])
    
    # 3. OUTPUT LAYER that maps output of decoder to elements of vocabulary
    
    output_layer = Dense(target_vocab_size, kernel_initializer=tf.truncated_normal_initializer(mean=0.0,stddev=0.1), \
                         name='output_layer')
    
    
    #--------------------------------------------------------------------------------------------------------------
    # The dec_embeddings MUST BE PASSED TO dec_inference, while the EMBEDDED_LOOKUP result is PASSED to dec train
    #--------------------------------------------------------------------------------------------------------------
    with tf.variable_scope("decoder"):
        train_logits = decoding_layer_train(encoder_state, decoder_stack_lstm, dec_embedding_input,\
                                            target_sequence_length, max_target_sequence_length, \
                                            output_layer, keep_prob)
        
    with tf.variable_scope("decoder", reuse=True):
        # dropout for inference ASSUMED = 1 since it is not training and all waits have been set in dec_train
        # so the Dropout should be at training time
        keep_prob_infer = tf.constant(1.0,dtype=tf.float32,name='dec_infer_Dropout')
        inference_logits = decoding_layer_infer(encoder_state, decoder_stack_lstm, dec_embeddings, \
                                                target_vocab_to_int['<GO>'], target_vocab_to_int['<EOS>'], \
                                                max_target_sequence_length, target_vocab_size, output_layer,\
                                                batch_size, keep_prob_infer)
    
    return train_logits, inference_logits



"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_decoding_layer(decoding_layer)



    


















    






Tests Passed

Build the Neural Network

Apply the functions you implemented above to:

  • Apply embedding to the input data for the encoder.
  • Encode the input using your encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob, source_sequence_length, source_vocab_size, encoding_embedding_size).
  • Process target data using your process_decoder_input(target_data, target_vocab_to_int, batch_size) function.
  • Apply embedding to the target data for the decoder.
  • Decode the encoded input using your decoding_layer(dec_input, enc_state, target_sequence_length, max_target_sentence_length, rnn_size, num_layers, target_vocab_to_int, target_vocab_size, batch_size, keep_prob, dec_embedding_size) function.




In [124]:



    


def seq2seq_model(input_data, target_data, keep_prob, batch_size,
                  source_sequence_length, target_sequence_length,
                  max_target_sentence_length,
                  source_vocab_size, target_vocab_size,
                  enc_embedding_size, dec_embedding_size,
                  rnn_size, num_layers, target_vocab_to_int):
    """
    Build the Sequence-to-Sequence part of the neural network
    :param input_data: Input placeholder
    :param target_data: Target placeholder
    :param keep_prob: Dropout keep probability placeholder
    :param batch_size: Batch Size
    :param source_sequence_length: Sequence Lengths of source sequences in the batch
    :param target_sequence_length: Sequence Lengths of target sequences in the batch
    :param source_vocab_size: Source vocabulary size
    :param target_vocab_size: Target vocabulary size
    :param max_target_sentence_length: Length of the longest target sentence
    :param enc_embedding_size: Decoder embedding size
    :param dec_embedding_size: Encoder embedding size
    :param rnn_size: RNN Size
    :param num_layers: Number of layers
    :param target_vocab_to_int: Dictionary to go from the target words to an id
    :return: Tuple of (Training BasicDecoderOutput, Inference BasicDecoderOutput)
    """
    # TODO: Implement Function
    
    # 1. Apply embedding to the input data for the encoder (embedded done inside encoding_layer())
    # 2. Create the encoding_layer
    BasicEncoderOuput, encoder_state = encoding_layer(input_data, rnn_size, num_layers, keep_prob,\
                                       source_sequence_length, source_vocab_size, enc_embedding_size)
    
    # 3. Preprocess the target data that will be feed to the decoder
    preprocess_dec_data = process_decoder_input(target_data, target_vocab_to_int, batch_size)
    
    # 4. Apply embedding to the input target data for the decoder (embedded done inside decoding_layer())
    # 5. Create the decoding_layer
    train_BasicDecoderOutput, inference_BasicDecoderOutput = decoding_layer(preprocess_dec_data, encoder_state,\
                                                                            target_sequence_length,\
                                                                            max_target_sentence_length, rnn_size,\
                                                                            num_layers, target_vocab_to_int,\
                                                                            target_vocab_size, batch_size,\
                                                                            keep_prob, dec_embedding_size)
    
    return train_BasicDecoderOutput, inference_BasicDecoderOutput


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_seq2seq_model(seq2seq_model)



    


















    






Tests Passed

Neural Network Training

Hyperparameters

Tune the following parameters:

  • Set epochs to the number of epochs.
  • Set batch_size to the batch size.
  • Set rnn_size to the size of the RNNs.
  • Set num_layers to the number of layers.
  • Set encoding_embedding_size to the size of the embedding for the encoder.
  • Set decoding_embedding_size to the size of the embedding for the decoder.
  • Set learning_rate to the learning rate.
  • Set keep_probability to the Dropout keep probability
  • Set display_step to state how many steps between each debug output statement




In [135]:



    


# Number of Epochs
epochs = 5
# Batch Size
batch_size = 64
# RNN Size
rnn_size = 128
# Number of Layers
num_layers = 1
# Embedding Size
encoding_embedding_size = 128
decoding_embedding_size = 128
# Learning Rate
learning_rate = 0.001
# Dropout Keep Probability
keep_probability = 0.5
display_step = 5

Build the Graph

Build the graph using the neural network you implemented.





In [136]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
save_path = 'checkpoints/dev'
(source_int_text, target_int_text), (source_vocab_to_int, target_vocab_to_int), _ = helper.load_preprocess()
max_target_sentence_length = max([len(sentence) for sentence in source_int_text])

train_graph = tf.Graph()
with train_graph.as_default():
    input_data, targets, lr, keep_prob, target_sequence_length, max_target_sequence_length, source_sequence_length = model_inputs()

    #sequence_length = tf.placeholder_with_default(max_target_sentence_length, None, name='sequence_length')
    input_shape = tf.shape(input_data)

    train_logits, inference_logits = seq2seq_model(tf.reverse(input_data, [-1]),
                                                   targets,
                                                   keep_prob,
                                                   batch_size,
                                                   source_sequence_length,
                                                   target_sequence_length,
                                                   max_target_sequence_length,
                                                   len(source_vocab_to_int),
                                                   len(target_vocab_to_int),
                                                   encoding_embedding_size,
                                                   decoding_embedding_size,
                                                   rnn_size,
                                                   num_layers,
                                                   target_vocab_to_int)


    training_logits = tf.identity(train_logits.rnn_output, name='logits')
    inference_logits = tf.identity(inference_logits.sample_id, name='predictions')

    masks = tf.sequence_mask(target_sequence_length, max_target_sequence_length, dtype=tf.float32, name='masks')

    with tf.name_scope("optimization"):
        # Loss function
        cost = tf.contrib.seq2seq.sequence_loss(
            training_logits,
            targets,
            masks)

        # Optimizer
        optimizer = tf.train.AdamOptimizer(lr)

        # Gradient Clipping
        gradients = optimizer.compute_gradients(cost)
        capped_gradients = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gradients if grad is not None]
        train_op = optimizer.apply_gradients(capped_gradients)

Batch and pad the source and target sequences





In [137]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
def pad_sentence_batch(sentence_batch, pad_int):
    """Pad sentences with <PAD> so that each sentence of a batch has the same length"""
    max_sentence = max([len(sentence) for sentence in sentence_batch])
    return [sentence + [pad_int] * (max_sentence - len(sentence)) for sentence in sentence_batch]


def get_batches(sources, targets, batch_size, source_pad_int, target_pad_int):
    """Batch targets, sources, and the lengths of their sentences together"""
    for batch_i in range(0, len(sources)//batch_size):
        start_i = batch_i * batch_size

        # Slice the right amount for the batch
        sources_batch = sources[start_i:start_i + batch_size]
        targets_batch = targets[start_i:start_i + batch_size]

        # Pad
        pad_sources_batch = np.array(pad_sentence_batch(sources_batch, source_pad_int))
        pad_targets_batch = np.array(pad_sentence_batch(targets_batch, target_pad_int))

        # Need the lengths for the _lengths parameters
        pad_targets_lengths = []
        for target in pad_targets_batch:
            pad_targets_lengths.append(len(target))

        pad_source_lengths = []
        for source in pad_sources_batch:
            pad_source_lengths.append(len(source))

        yield pad_sources_batch, pad_targets_batch, pad_source_lengths, pad_targets_lengths

Train

Train the neural network on the preprocessed data. If you have a hard time getting a good loss, check the forms to see if anyone is having the same problem.





In [138]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
def get_accuracy(target, logits):
    """
    Calculate accuracy
    """
    max_seq = max(target.shape[1], logits.shape[1])
    if max_seq - target.shape[1]:
        target = np.pad(
            target,
            [(0,0),(0,max_seq - target.shape[1])],
            'constant')
    if max_seq - logits.shape[1]:
        logits = np.pad(
            logits,
            [(0,0),(0,max_seq - logits.shape[1])],
            'constant')

    return np.mean(np.equal(target, logits))

# Split data to training and validation sets
train_source = source_int_text[batch_size:]
train_target = target_int_text[batch_size:]
valid_source = source_int_text[:batch_size]
valid_target = target_int_text[:batch_size]
(valid_sources_batch, valid_targets_batch, valid_sources_lengths, valid_targets_lengths ) = next(get_batches(valid_source,
                                                                                                             valid_target,
                                                                                                             batch_size,
                                                                                                             source_vocab_to_int['<PAD>'],
                                                                                                             target_vocab_to_int['<PAD>']))                                                                                                  
with tf.Session(graph=train_graph) as sess:
    sess.run(tf.global_variables_initializer())

    for epoch_i in range(epochs):
        for batch_i, (source_batch, target_batch, sources_lengths, targets_lengths) in enumerate(
                get_batches(train_source, train_target, batch_size,
                            source_vocab_to_int['<PAD>'],
                            target_vocab_to_int['<PAD>'])):

            _, loss = sess.run(
                [train_op, cost],
                {input_data: source_batch,
                 targets: target_batch,
                 lr: learning_rate,
                 target_sequence_length: targets_lengths,
                 source_sequence_length: sources_lengths,
                 keep_prob: keep_probability})


            if batch_i % display_step == 0 and batch_i > 0:


                batch_train_logits = sess.run(
                    inference_logits,
                    {input_data: source_batch,
                     source_sequence_length: sources_lengths,
                     target_sequence_length: targets_lengths,
                     keep_prob: 1.0})


                batch_valid_logits = sess.run(
                    inference_logits,
                    {input_data: valid_sources_batch,
                     source_sequence_length: valid_sources_lengths,
                     target_sequence_length: valid_targets_lengths,
                     keep_prob: 1.0})

                train_acc = get_accuracy(target_batch, batch_train_logits)

                valid_acc = get_accuracy(valid_targets_batch, batch_valid_logits)

                print('Epoch {:>3} Batch {:>4}/{} - Train Accuracy: {:>6.4f}, Validation Accuracy: {:>6.4f}, Loss: {:>6.4f}'
                      .format(epoch_i, batch_i, len(source_int_text) // batch_size, train_acc, valid_acc, loss))

    # Save Model
    saver = tf.train.Saver()
    saver.save(sess, save_path)
    print('Model Trained and Saved')



    


















    






Epoch   0 Batch    5/2154 - Train Accuracy: 0.2105, Validation Accuracy: 0.2841, Loss: 5.0353
Epoch   0 Batch   10/2154 - Train Accuracy: 0.2117, Validation Accuracy: 0.2841, Loss: 4.4711
Epoch   0 Batch   15/2154 - Train Accuracy: 0.2547, Validation Accuracy: 0.3146, Loss: 4.0545
Epoch   0 Batch   20/2154 - Train Accuracy: 0.2836, Validation Accuracy: 0.3210, Loss: 3.6104
Epoch   0 Batch   25/2154 - Train Accuracy: 0.2992, Validation Accuracy: 0.3494, Loss: 3.4997
Epoch   0 Batch   30/2154 - Train Accuracy: 0.2773, Validation Accuracy: 0.3409, Loss: 3.3205
Epoch   0 Batch   35/2154 - Train Accuracy: 0.3180, Validation Accuracy: 0.3516, Loss: 3.1711
Epoch   0 Batch   40/2154 - Train Accuracy: 0.2722, Validation Accuracy: 0.3530, Loss: 3.1970
Epoch   0 Batch   45/2154 - Train Accuracy: 0.2903, Validation Accuracy: 0.3672, Loss: 3.1638
Epoch   0 Batch   50/2154 - Train Accuracy: 0.2796, Validation Accuracy: 0.3750, Loss: 3.0745
Epoch   0 Batch   55/2154 - Train Accuracy: 0.3383, Validation Accuracy: 0.3963, Loss: 2.8545
Epoch   0 Batch   60/2154 - Train Accuracy: 0.3492, Validation Accuracy: 0.3942, Loss: 2.8309
Epoch   0 Batch   65/2154 - Train Accuracy: 0.4159, Validation Accuracy: 0.4013, Loss: 2.5934
Epoch   0 Batch   70/2154 - Train Accuracy: 0.3479, Validation Accuracy: 0.4276, Loss: 2.7881
Epoch   0 Batch   75/2154 - Train Accuracy: 0.3396, Validation Accuracy: 0.4354, Loss: 2.8207
Epoch   0 Batch   80/2154 - Train Accuracy: 0.3891, Validation Accuracy: 0.4560, Loss: 2.6078
Epoch   0 Batch   85/2154 - Train Accuracy: 0.4188, Validation Accuracy: 0.4567, Loss: 2.4369
Epoch   0 Batch   90/2154 - Train Accuracy: 0.3701, Validation Accuracy: 0.4666, Loss: 2.6130
Epoch   0 Batch   95/2154 - Train Accuracy: 0.4148, Validation Accuracy: 0.4453, Loss: 2.3749
Epoch   0 Batch  100/2154 - Train Accuracy: 0.3958, Validation Accuracy: 0.4531, Loss: 2.4051
Epoch   0 Batch  105/2154 - Train Accuracy: 0.4188, Validation Accuracy: 0.4609, Loss: 2.2901
Epoch   0 Batch  110/2154 - Train Accuracy: 0.3992, Validation Accuracy: 0.4673, Loss: 2.3685
Epoch   0 Batch  115/2154 - Train Accuracy: 0.5149, Validation Accuracy: 0.4808, Loss: 1.9568
Epoch   0 Batch  120/2154 - Train Accuracy: 0.4400, Validation Accuracy: 0.4602, Loss: 2.1953
Epoch   0 Batch  125/2154 - Train Accuracy: 0.4087, Validation Accuracy: 0.4638, Loss: 2.2325
Epoch   0 Batch  130/2154 - Train Accuracy: 0.4016, Validation Accuracy: 0.4538, Loss: 2.0337
Epoch   0 Batch  135/2154 - Train Accuracy: 0.3576, Validation Accuracy: 0.4176, Loss: 2.1536
Epoch   0 Batch  140/2154 - Train Accuracy: 0.4156, Validation Accuracy: 0.4240, Loss: 1.9329
Epoch   0 Batch  145/2154 - Train Accuracy: 0.3820, Validation Accuracy: 0.4240, Loss: 1.8956
Epoch   0 Batch  150/2154 - Train Accuracy: 0.4100, Validation Accuracy: 0.4126, Loss: 1.7569
Epoch   0 Batch  155/2154 - Train Accuracy: 0.3906, Validation Accuracy: 0.4148, Loss: 1.8102
Epoch   0 Batch  160/2154 - Train Accuracy: 0.3953, Validation Accuracy: 0.4169, Loss: 1.8185
Epoch   0 Batch  165/2154 - Train Accuracy: 0.4403, Validation Accuracy: 0.4098, Loss: 1.5341
Epoch   0 Batch  170/2154 - Train Accuracy: 0.4102, Validation Accuracy: 0.4148, Loss: 1.6185
Epoch   0 Batch  175/2154 - Train Accuracy: 0.4172, Validation Accuracy: 0.4261, Loss: 1.6274
Epoch   0 Batch  180/2154 - Train Accuracy: 0.3883, Validation Accuracy: 0.4183, Loss: 1.5665
Epoch   0 Batch  185/2154 - Train Accuracy: 0.3692, Validation Accuracy: 0.4062, Loss: 1.6697
Epoch   0 Batch  190/2154 - Train Accuracy: 0.4570, Validation Accuracy: 0.4276, Loss: 1.4959
Epoch   0 Batch  195/2154 - Train Accuracy: 0.4252, Validation Accuracy: 0.4169, Loss: 1.5634
Epoch   0 Batch  200/2154 - Train Accuracy: 0.4102, Validation Accuracy: 0.4276, Loss: 1.4449
Epoch   0 Batch  205/2154 - Train Accuracy: 0.4117, Validation Accuracy: 0.4197, Loss: 1.3844
Epoch   0 Batch  210/2154 - Train Accuracy: 0.3906, Validation Accuracy: 0.4794, Loss: 1.4472
Epoch   0 Batch  215/2154 - Train Accuracy: 0.4234, Validation Accuracy: 0.4531, Loss: 1.3162
Epoch   0 Batch  220/2154 - Train Accuracy: 0.4922, Validation Accuracy: 0.4872, Loss: 1.3026
Epoch   0 Batch  225/2154 - Train Accuracy: 0.4656, Validation Accuracy: 0.5085, Loss: 1.3835
Epoch   0 Batch  230/2154 - Train Accuracy: 0.4903, Validation Accuracy: 0.4709, Loss: 1.3075
Epoch   0 Batch  235/2154 - Train Accuracy: 0.4242, Validation Accuracy: 0.5128, Loss: 1.4031
Epoch   0 Batch  240/2154 - Train Accuracy: 0.4742, Validation Accuracy: 0.5114, Loss: 1.2636
Epoch   0 Batch  245/2154 - Train Accuracy: 0.5090, Validation Accuracy: 0.5028, Loss: 1.2397
Epoch   0 Batch  250/2154 - Train Accuracy: 0.5016, Validation Accuracy: 0.4986, Loss: 1.2369
Epoch   0 Batch  255/2154 - Train Accuracy: 0.4773, Validation Accuracy: 0.5078, Loss: 1.1822
Epoch   0 Batch  260/2154 - Train Accuracy: 0.5117, Validation Accuracy: 0.4986, Loss: 1.1997
Epoch   0 Batch  265/2154 - Train Accuracy: 0.4320, Validation Accuracy: 0.5028, Loss: 1.2316
Epoch   0 Batch  270/2154 - Train Accuracy: 0.5045, Validation Accuracy: 0.5114, Loss: 1.0914
Epoch   0 Batch  275/2154 - Train Accuracy: 0.5250, Validation Accuracy: 0.5142, Loss: 1.0510
Epoch   0 Batch  280/2154 - Train Accuracy: 0.4671, Validation Accuracy: 0.5107, Loss: 1.2162
Epoch   0 Batch  285/2154 - Train Accuracy: 0.4712, Validation Accuracy: 0.5071, Loss: 1.1510
Epoch   0 Batch  290/2154 - Train Accuracy: 0.4350, Validation Accuracy: 0.5121, Loss: 1.1491
Epoch   0 Batch  295/2154 - Train Accuracy: 0.4992, Validation Accuracy: 0.5249, Loss: 1.0873
Epoch   0 Batch  300/2154 - Train Accuracy: 0.4457, Validation Accuracy: 0.5178, Loss: 1.1700
Epoch   0 Batch  305/2154 - Train Accuracy: 0.5070, Validation Accuracy: 0.5014, Loss: 1.0953
Epoch   0 Batch  310/2154 - Train Accuracy: 0.4688, Validation Accuracy: 0.5021, Loss: 1.0692
Epoch   0 Batch  315/2154 - Train Accuracy: 0.4789, Validation Accuracy: 0.5085, Loss: 1.0283
Epoch   0 Batch  320/2154 - Train Accuracy: 0.5016, Validation Accuracy: 0.5170, Loss: 0.9776
Epoch   0 Batch  325/2154 - Train Accuracy: 0.4758, Validation Accuracy: 0.5128, Loss: 1.0599
Epoch   0 Batch  330/2154 - Train Accuracy: 0.4498, Validation Accuracy: 0.5085, Loss: 1.0376
Epoch   0 Batch  335/2154 - Train Accuracy: 0.5219, Validation Accuracy: 0.5057, Loss: 0.9763
Epoch   0 Batch  340/2154 - Train Accuracy: 0.4985, Validation Accuracy: 0.5099, Loss: 0.9649
Epoch   0 Batch  345/2154 - Train Accuracy: 0.4359, Validation Accuracy: 0.5007, Loss: 1.0368
Epoch   0 Batch  350/2154 - Train Accuracy: 0.4930, Validation Accuracy: 0.5099, Loss: 0.9881
Epoch   0 Batch  355/2154 - Train Accuracy: 0.4819, Validation Accuracy: 0.5121, Loss: 1.0869
Epoch   0 Batch  360/2154 - Train Accuracy: 0.5140, Validation Accuracy: 0.5014, Loss: 1.0053
Epoch   0 Batch  365/2154 - Train Accuracy: 0.4747, Validation Accuracy: 0.5135, Loss: 0.9392
Epoch   0 Batch  370/2154 - Train Accuracy: 0.5102, Validation Accuracy: 0.5128, Loss: 0.9530
Epoch   0 Batch  375/2154 - Train Accuracy: 0.5058, Validation Accuracy: 0.5170, Loss: 0.9492
Epoch   0 Batch  380/2154 - Train Accuracy: 0.4992, Validation Accuracy: 0.5163, Loss: 0.9393
Epoch   0 Batch  385/2154 - Train Accuracy: 0.5305, Validation Accuracy: 0.5192, Loss: 0.9219
Epoch   0 Batch  390/2154 - Train Accuracy: 0.5260, Validation Accuracy: 0.5256, Loss: 0.8604
Epoch   0 Batch  395/2154 - Train Accuracy: 0.5430, Validation Accuracy: 0.5277, Loss: 0.9021
Epoch   0 Batch  400/2154 - Train Accuracy: 0.4797, Validation Accuracy: 0.5142, Loss: 0.9478
Epoch   0 Batch  405/2154 - Train Accuracy: 0.5477, Validation Accuracy: 0.5199, Loss: 0.9170
Epoch   0 Batch  410/2154 - Train Accuracy: 0.5133, Validation Accuracy: 0.5298, Loss: 0.9077
Epoch   0 Batch  415/2154 - Train Accuracy: 0.4789, Validation Accuracy: 0.5298, Loss: 0.8913
Epoch   0 Batch  420/2154 - Train Accuracy: 0.5320, Validation Accuracy: 0.5291, Loss: 0.7950
Epoch   0 Batch  425/2154 - Train Accuracy: 0.5603, Validation Accuracy: 0.5312, Loss: 0.8447
Epoch   0 Batch  430/2154 - Train Accuracy: 0.5148, Validation Accuracy: 0.5227, Loss: 0.9134
Epoch   0 Batch  435/2154 - Train Accuracy: 0.5609, Validation Accuracy: 0.5263, Loss: 0.8652
Epoch   0 Batch  440/2154 - Train Accuracy: 0.5255, Validation Accuracy: 0.5284, Loss: 0.8842
Epoch   0 Batch  445/2154 - Train Accuracy: 0.4570, Validation Accuracy: 0.5291, Loss: 0.9428
Epoch   0 Batch  450/2154 - Train Accuracy: 0.5625, Validation Accuracy: 0.5312, Loss: 0.8350
Epoch   0 Batch  455/2154 - Train Accuracy: 0.4852, Validation Accuracy: 0.5298, Loss: 0.9028
Epoch   0 Batch  460/2154 - Train Accuracy: 0.5602, Validation Accuracy: 0.5256, Loss: 0.8460
Epoch   0 Batch  465/2154 - Train Accuracy: 0.5000, Validation Accuracy: 0.5263, Loss: 0.8759
Epoch   0 Batch  470/2154 - Train Accuracy: 0.4696, Validation Accuracy: 0.5291, Loss: 0.9512
Epoch   0 Batch  475/2154 - Train Accuracy: 0.4910, Validation Accuracy: 0.5391, Loss: 0.8317
Epoch   0 Batch  480/2154 - Train Accuracy: 0.5461, Validation Accuracy: 0.5241, Loss: 0.7718
Epoch   0 Batch  485/2154 - Train Accuracy: 0.5258, Validation Accuracy: 0.5263, Loss: 0.7635
Epoch   0 Batch  490/2154 - Train Accuracy: 0.5500, Validation Accuracy: 0.5284, Loss: 0.8262
Epoch   0 Batch  495/2154 - Train Accuracy: 0.5900, Validation Accuracy: 0.5391, Loss: 0.7947
Epoch   0 Batch  500/2154 - Train Accuracy: 0.5321, Validation Accuracy: 0.5440, Loss: 0.8269
Epoch   0 Batch  505/2154 - Train Accuracy: 0.5452, Validation Accuracy: 0.5185, Loss: 0.7919
Epoch   0 Batch  510/2154 - Train Accuracy: 0.5484, Validation Accuracy: 0.5277, Loss: 0.8280
Epoch   0 Batch  515/2154 - Train Accuracy: 0.5223, Validation Accuracy: 0.5305, Loss: 0.7635
Epoch   0 Batch  520/2154 - Train Accuracy: 0.5281, Validation Accuracy: 0.5305, Loss: 0.7709
Epoch   0 Batch  525/2154 - Train Accuracy: 0.5406, Validation Accuracy: 0.5263, Loss: 0.7706
Epoch   0 Batch  530/2154 - Train Accuracy: 0.5188, Validation Accuracy: 0.5462, Loss: 0.7715
Epoch   0 Batch  535/2154 - Train Accuracy: 0.5256, Validation Accuracy: 0.5469, Loss: 0.7580
Epoch   0 Batch  540/2154 - Train Accuracy: 0.5230, Validation Accuracy: 0.5447, Loss: 0.8063
Epoch   0 Batch  545/2154 - Train Accuracy: 0.5411, Validation Accuracy: 0.5490, Loss: 0.8628
Epoch   0 Batch  550/2154 - Train Accuracy: 0.5744, Validation Accuracy: 0.5497, Loss: 0.7571
Epoch   0 Batch  555/2154 - Train Accuracy: 0.5469, Validation Accuracy: 0.5504, Loss: 0.7710
Epoch   0 Batch  560/2154 - Train Accuracy: 0.5648, Validation Accuracy: 0.5582, Loss: 0.8229
Epoch   0 Batch  565/2154 - Train Accuracy: 0.5414, Validation Accuracy: 0.5604, Loss: 0.7583
Epoch   0 Batch  570/2154 - Train Accuracy: 0.5742, Validation Accuracy: 0.5533, Loss: 0.8306
Epoch   0 Batch  575/2154 - Train Accuracy: 0.5883, Validation Accuracy: 0.5554, Loss: 0.7272
Epoch   0 Batch  580/2154 - Train Accuracy: 0.5880, Validation Accuracy: 0.5668, Loss: 0.7598
Epoch   0 Batch  585/2154 - Train Accuracy: 0.6019, Validation Accuracy: 0.5611, Loss: 0.7293
Epoch   0 Batch  590/2154 - Train Accuracy: 0.6037, Validation Accuracy: 0.5632, Loss: 0.6944
Epoch   0 Batch  595/2154 - Train Accuracy: 0.5680, Validation Accuracy: 0.5732, Loss: 0.7680
Epoch   0 Batch  600/2154 - Train Accuracy: 0.5732, Validation Accuracy: 0.5710, Loss: 0.7180
Epoch   0 Batch  605/2154 - Train Accuracy: 0.6156, Validation Accuracy: 0.5632, Loss: 0.7615
Epoch   0 Batch  610/2154 - Train Accuracy: 0.5513, Validation Accuracy: 0.5518, Loss: 0.6907
Epoch   0 Batch  615/2154 - Train Accuracy: 0.5670, Validation Accuracy: 0.5661, Loss: 0.6922
Epoch   0 Batch  620/2154 - Train Accuracy: 0.5692, Validation Accuracy: 0.5661, Loss: 0.6851
Epoch   0 Batch  625/2154 - Train Accuracy: 0.5921, Validation Accuracy: 0.5696, Loss: 0.7826
Epoch   0 Batch  630/2154 - Train Accuracy: 0.6192, Validation Accuracy: 0.5668, Loss: 0.6957
Epoch   0 Batch  635/2154 - Train Accuracy: 0.6077, Validation Accuracy: 0.5746, Loss: 0.7672
Epoch   0 Batch  640/2154 - Train Accuracy: 0.5508, Validation Accuracy: 0.5639, Loss: 0.7324
Epoch   0 Batch  645/2154 - Train Accuracy: 0.5655, Validation Accuracy: 0.5739, Loss: 0.7092
Epoch   0 Batch  650/2154 - Train Accuracy: 0.5508, Validation Accuracy: 0.5866, Loss: 0.7415
Epoch   0 Batch  655/2154 - Train Accuracy: 0.5732, Validation Accuracy: 0.5739, Loss: 0.7566
Epoch   0 Batch  660/2154 - Train Accuracy: 0.5687, Validation Accuracy: 0.5661, Loss: 0.7395
Epoch   0 Batch  665/2154 - Train Accuracy: 0.5844, Validation Accuracy: 0.5689, Loss: 0.6634
Epoch   0 Batch  670/2154 - Train Accuracy: 0.5921, Validation Accuracy: 0.6065, Loss: 0.7059
Epoch   0 Batch  675/2154 - Train Accuracy: 0.5625, Validation Accuracy: 0.5881, Loss: 0.6864
Epoch   0 Batch  680/2154 - Train Accuracy: 0.5477, Validation Accuracy: 0.5788, Loss: 0.6913
Epoch   0 Batch  685/2154 - Train Accuracy: 0.5531, Validation Accuracy: 0.5767, Loss: 0.6773
Epoch   0 Batch  690/2154 - Train Accuracy: 0.5828, Validation Accuracy: 0.5597, Loss: 0.7193
Epoch   0 Batch  695/2154 - Train Accuracy: 0.5797, Validation Accuracy: 0.5732, Loss: 0.6854
Epoch   0 Batch  700/2154 - Train Accuracy: 0.5206, Validation Accuracy: 0.5703, Loss: 0.6867
Epoch   0 Batch  705/2154 - Train Accuracy: 0.6320, Validation Accuracy: 0.6108, Loss: 0.6744
Epoch   0 Batch  710/2154 - Train Accuracy: 0.5977, Validation Accuracy: 0.6122, Loss: 0.6693
Epoch   0 Batch  715/2154 - Train Accuracy: 0.6727, Validation Accuracy: 0.5866, Loss: 0.6523
Epoch   0 Batch  720/2154 - Train Accuracy: 0.6094, Validation Accuracy: 0.5930, Loss: 0.7104
Epoch   0 Batch  725/2154 - Train Accuracy: 0.6436, Validation Accuracy: 0.5675, Loss: 0.6229
Epoch   0 Batch  730/2154 - Train Accuracy: 0.5600, Validation Accuracy: 0.5938, Loss: 0.7636
Epoch   0 Batch  735/2154 - Train Accuracy: 0.6153, Validation Accuracy: 0.6087, Loss: 0.6050
Epoch   0 Batch  740/2154 - Train Accuracy: 0.5906, Validation Accuracy: 0.6115, Loss: 0.6549
Epoch   0 Batch  745/2154 - Train Accuracy: 0.5946, Validation Accuracy: 0.5703, Loss: 0.6627
Epoch   0 Batch  750/2154 - Train Accuracy: 0.5852, Validation Accuracy: 0.5923, Loss: 0.7049
Epoch   0 Batch  755/2154 - Train Accuracy: 0.5255, Validation Accuracy: 0.6080, Loss: 0.7243
Epoch   0 Batch  760/2154 - Train Accuracy: 0.6195, Validation Accuracy: 0.5895, Loss: 0.6732
Epoch   0 Batch  765/2154 - Train Accuracy: 0.6354, Validation Accuracy: 0.6222, Loss: 0.6724
Epoch   0 Batch  770/2154 - Train Accuracy: 0.6125, Validation Accuracy: 0.6278, Loss: 0.6102
Epoch   0 Batch  775/2154 - Train Accuracy: 0.6628, Validation Accuracy: 0.6207, Loss: 0.6589
Epoch   0 Batch  780/2154 - Train Accuracy: 0.6578, Validation Accuracy: 0.6037, Loss: 0.5936
Epoch   0 Batch  785/2154 - Train Accuracy: 0.6135, Validation Accuracy: 0.6115, Loss: 0.6847
Epoch   0 Batch  790/2154 - Train Accuracy: 0.6078, Validation Accuracy: 0.6108, Loss: 0.6367
Epoch   0 Batch  795/2154 - Train Accuracy: 0.6629, Validation Accuracy: 0.6186, Loss: 0.6064
Epoch   0 Batch  800/2154 - Train Accuracy: 0.5789, Validation Accuracy: 0.6136, Loss: 0.6358
Epoch   0 Batch  805/2154 - Train Accuracy: 0.6513, Validation Accuracy: 0.6214, Loss: 0.5993
Epoch   0 Batch  810/2154 - Train Accuracy: 0.6302, Validation Accuracy: 0.6200, Loss: 0.5936
Epoch   0 Batch  815/2154 - Train Accuracy: 0.6188, Validation Accuracy: 0.6129, Loss: 0.6528
Epoch   0 Batch  820/2154 - Train Accuracy: 0.5921, Validation Accuracy: 0.6278, Loss: 0.6437
Epoch   0 Batch  825/2154 - Train Accuracy: 0.6219, Validation Accuracy: 0.6193, Loss: 0.5971
Epoch   0 Batch  830/2154 - Train Accuracy: 0.5953, Validation Accuracy: 0.6200, Loss: 0.6273
Epoch   0 Batch  835/2154 - Train Accuracy: 0.6390, Validation Accuracy: 0.6136, Loss: 0.6354
Epoch   0 Batch  840/2154 - Train Accuracy: 0.6117, Validation Accuracy: 0.6307, Loss: 0.6103
Epoch   0 Batch  845/2154 - Train Accuracy: 0.6059, Validation Accuracy: 0.6378, Loss: 0.6794
Epoch   0 Batch  850/2154 - Train Accuracy: 0.5914, Validation Accuracy: 0.6108, Loss: 0.5851
Epoch   0 Batch  855/2154 - Train Accuracy: 0.5922, Validation Accuracy: 0.5930, Loss: 0.6448
Epoch   0 Batch  860/2154 - Train Accuracy: 0.6562, Validation Accuracy: 0.6229, Loss: 0.5956
Epoch   0 Batch  865/2154 - Train Accuracy: 0.6531, Validation Accuracy: 0.6115, Loss: 0.6023
Epoch   0 Batch  870/2154 - Train Accuracy: 0.6359, Validation Accuracy: 0.6172, Loss: 0.6045
Epoch   0 Batch  875/2154 - Train Accuracy: 0.5987, Validation Accuracy: 0.6186, Loss: 0.6213
Epoch   0 Batch  880/2154 - Train Accuracy: 0.6234, Validation Accuracy: 0.6250, Loss: 0.6171
Epoch   0 Batch  885/2154 - Train Accuracy: 0.6704, Validation Accuracy: 0.6420, Loss: 0.5785
Epoch   0 Batch  890/2154 - Train Accuracy: 0.6579, Validation Accuracy: 0.6413, Loss: 0.6096
Epoch   0 Batch  895/2154 - Train Accuracy: 0.5831, Validation Accuracy: 0.6179, Loss: 0.6275
Epoch   0 Batch  900/2154 - Train Accuracy: 0.6234, Validation Accuracy: 0.6044, Loss: 0.6402
Epoch   0 Batch  905/2154 - Train Accuracy: 0.6672, Validation Accuracy: 0.6264, Loss: 0.5646
Epoch   0 Batch  910/2154 - Train Accuracy: 0.6003, Validation Accuracy: 0.6243, Loss: 0.5988
Epoch   0 Batch  915/2154 - Train Accuracy: 0.6853, Validation Accuracy: 0.6222, Loss: 0.5340
Epoch   0 Batch  920/2154 - Train Accuracy: 0.6503, Validation Accuracy: 0.6328, Loss: 0.5466
Epoch   0 Batch  925/2154 - Train Accuracy: 0.6234, Validation Accuracy: 0.6278, Loss: 0.5497
Epoch   0 Batch  930/2154 - Train Accuracy: 0.6969, Validation Accuracy: 0.6520, Loss: 0.5719
Epoch   0 Batch  935/2154 - Train Accuracy: 0.7212, Validation Accuracy: 0.6357, Loss: 0.6238
Epoch   0 Batch  940/2154 - Train Accuracy: 0.6180, Validation Accuracy: 0.6357, Loss: 0.5800
Epoch   0 Batch  945/2154 - Train Accuracy: 0.6492, Validation Accuracy: 0.6229, Loss: 0.5552
Epoch   0 Batch  950/2154 - Train Accuracy: 0.6554, Validation Accuracy: 0.6570, Loss: 0.5941
Epoch   0 Batch  955/2154 - Train Accuracy: 0.6753, Validation Accuracy: 0.6442, Loss: 0.6171
Epoch   0 Batch  960/2154 - Train Accuracy: 0.6578, Validation Accuracy: 0.6286, Loss: 0.5799
Epoch   0 Batch  965/2154 - Train Accuracy: 0.6168, Validation Accuracy: 0.6406, Loss: 0.6449
Epoch   0 Batch  970/2154 - Train Accuracy: 0.6477, Validation Accuracy: 0.6357, Loss: 0.5652
Epoch   0 Batch  975/2154 - Train Accuracy: 0.6645, Validation Accuracy: 0.6420, Loss: 0.5738
Epoch   0 Batch  980/2154 - Train Accuracy: 0.6531, Validation Accuracy: 0.6626, Loss: 0.5483
Epoch   0 Batch  985/2154 - Train Accuracy: 0.6281, Validation Accuracy: 0.6399, Loss: 0.5732
Epoch   0 Batch  990/2154 - Train Accuracy: 0.6055, Validation Accuracy: 0.6577, Loss: 0.5301
Epoch   0 Batch  995/2154 - Train Accuracy: 0.6678, Validation Accuracy: 0.6428, Loss: 0.5500
Epoch   0 Batch 1000/2154 - Train Accuracy: 0.6384, Validation Accuracy: 0.6342, Loss: 0.5280
Epoch   0 Batch 1005/2154 - Train Accuracy: 0.6793, Validation Accuracy: 0.6584, Loss: 0.5388
Epoch   0 Batch 1010/2154 - Train Accuracy: 0.6336, Validation Accuracy: 0.6619, Loss: 0.5459
Epoch   0 Batch 1015/2154 - Train Accuracy: 0.6431, Validation Accuracy: 0.6754, Loss: 0.5560
Epoch   0 Batch 1020/2154 - Train Accuracy: 0.6898, Validation Accuracy: 0.6584, Loss: 0.5176
Epoch   0 Batch 1025/2154 - Train Accuracy: 0.6648, Validation Accuracy: 0.6776, Loss: 0.5188
Epoch   0 Batch 1030/2154 - Train Accuracy: 0.6480, Validation Accuracy: 0.6811, Loss: 0.5783
Epoch   0 Batch 1035/2154 - Train Accuracy: 0.7277, Validation Accuracy: 0.6712, Loss: 0.5049
Epoch   0 Batch 1040/2154 - Train Accuracy: 0.6992, Validation Accuracy: 0.6619, Loss: 0.4915
Epoch   0 Batch 1045/2154 - Train Accuracy: 0.6324, Validation Accuracy: 0.6605, Loss: 0.5830
Epoch   0 Batch 1050/2154 - Train Accuracy: 0.6148, Validation Accuracy: 0.6577, Loss: 0.5516
Epoch   0 Batch 1055/2154 - Train Accuracy: 0.6752, Validation Accuracy: 0.6534, Loss: 0.5248
Epoch   0 Batch 1060/2154 - Train Accuracy: 0.7039, Validation Accuracy: 0.6662, Loss: 0.5316
Epoch   0 Batch 1065/2154 - Train Accuracy: 0.6375, Validation Accuracy: 0.6974, Loss: 0.5436
Epoch   0 Batch 1070/2154 - Train Accuracy: 0.7031, Validation Accuracy: 0.6676, Loss: 0.5025
Epoch   0 Batch 1075/2154 - Train Accuracy: 0.7070, Validation Accuracy: 0.6832, Loss: 0.5020
Epoch   0 Batch 1080/2154 - Train Accuracy: 0.6391, Validation Accuracy: 0.6740, Loss: 0.5117
Epoch   0 Batch 1085/2154 - Train Accuracy: 0.6703, Validation Accuracy: 0.6754, Loss: 0.4976
Epoch   0 Batch 1090/2154 - Train Accuracy: 0.6686, Validation Accuracy: 0.6967, Loss: 0.5152
Epoch   0 Batch 1095/2154 - Train Accuracy: 0.7141, Validation Accuracy: 0.6918, Loss: 0.4861
Epoch   0 Batch 1100/2154 - Train Accuracy: 0.6461, Validation Accuracy: 0.6825, Loss: 0.5380
Epoch   0 Batch 1105/2154 - Train Accuracy: 0.6953, Validation Accuracy: 0.7017, Loss: 0.4759
Epoch   0 Batch 1110/2154 - Train Accuracy: 0.7016, Validation Accuracy: 0.6577, Loss: 0.5261
Epoch   0 Batch 1115/2154 - Train Accuracy: 0.6933, Validation Accuracy: 0.6854, Loss: 0.5191
Epoch   0 Batch 1120/2154 - Train Accuracy: 0.6742, Validation Accuracy: 0.6690, Loss: 0.4898
Epoch   0 Batch 1125/2154 - Train Accuracy: 0.7056, Validation Accuracy: 0.6776, Loss: 0.4954
Epoch   0 Batch 1130/2154 - Train Accuracy: 0.6859, Validation Accuracy: 0.6818, Loss: 0.5372
Epoch   0 Batch 1135/2154 - Train Accuracy: 0.7008, Validation Accuracy: 0.6705, Loss: 0.4793
Epoch   0 Batch 1140/2154 - Train Accuracy: 0.7133, Validation Accuracy: 0.6854, Loss: 0.4860
Epoch   0 Batch 1145/2154 - Train Accuracy: 0.6439, Validation Accuracy: 0.6868, Loss: 0.5076
Epoch   0 Batch 1150/2154 - Train Accuracy: 0.6924, Validation Accuracy: 0.6783, Loss: 0.5048
Epoch   0 Batch 1155/2154 - Train Accuracy: 0.6998, Validation Accuracy: 0.6655, Loss: 0.5076
Epoch   0 Batch 1160/2154 - Train Accuracy: 0.6734, Validation Accuracy: 0.6989, Loss: 0.4840
Epoch   0 Batch 1165/2154 - Train Accuracy: 0.6828, Validation Accuracy: 0.6932, Loss: 0.4975
Epoch   0 Batch 1170/2154 - Train Accuracy: 0.6585, Validation Accuracy: 0.7017, Loss: 0.4564
Epoch   0 Batch 1175/2154 - Train Accuracy: 0.7086, Validation Accuracy: 0.6974, Loss: 0.4869
Epoch   0 Batch 1180/2154 - Train Accuracy: 0.6661, Validation Accuracy: 0.7053, Loss: 0.4846
Epoch   0 Batch 1185/2154 - Train Accuracy: 0.7048, Validation Accuracy: 0.6776, Loss: 0.4903
Epoch   0 Batch 1190/2154 - Train Accuracy: 0.7133, Validation Accuracy: 0.6939, Loss: 0.4856
Epoch   0 Batch 1195/2154 - Train Accuracy: 0.6719, Validation Accuracy: 0.6918, Loss: 0.4613
Epoch   0 Batch 1200/2154 - Train Accuracy: 0.6776, Validation Accuracy: 0.7003, Loss: 0.5559
Epoch   0 Batch 1205/2154 - Train Accuracy: 0.6727, Validation Accuracy: 0.7173, Loss: 0.5086
Epoch   0 Batch 1210/2154 - Train Accuracy: 0.7164, Validation Accuracy: 0.7188, Loss: 0.4439
Epoch   0 Batch 1215/2154 - Train Accuracy: 0.6965, Validation Accuracy: 0.7251, Loss: 0.4819
Epoch   0 Batch 1220/2154 - Train Accuracy: 0.6898, Validation Accuracy: 0.7180, Loss: 0.4647
Epoch   0 Batch 1225/2154 - Train Accuracy: 0.6898, Validation Accuracy: 0.7315, Loss: 0.4445
Epoch   0 Batch 1230/2154 - Train Accuracy: 0.7351, Validation Accuracy: 0.7180, Loss: 0.4546
Epoch   0 Batch 1235/2154 - Train Accuracy: 0.7009, Validation Accuracy: 0.7280, Loss: 0.4544
Epoch   0 Batch 1240/2154 - Train Accuracy: 0.7171, Validation Accuracy: 0.7244, Loss: 0.4683
Epoch   0 Batch 1245/2154 - Train Accuracy: 0.7558, Validation Accuracy: 0.7074, Loss: 0.4682
Epoch   0 Batch 1250/2154 - Train Accuracy: 0.7430, Validation Accuracy: 0.7024, Loss: 0.4215
Epoch   0 Batch 1255/2154 - Train Accuracy: 0.6949, Validation Accuracy: 0.7003, Loss: 0.4681
Epoch   0 Batch 1260/2154 - Train Accuracy: 0.6982, Validation Accuracy: 0.7017, Loss: 0.5006
Epoch   0 Batch 1265/2154 - Train Accuracy: 0.7477, Validation Accuracy: 0.7273, Loss: 0.4351
Epoch   0 Batch 1270/2154 - Train Accuracy: 0.7046, Validation Accuracy: 0.7173, Loss: 0.4128
Epoch   0 Batch 1275/2154 - Train Accuracy: 0.7008, Validation Accuracy: 0.7060, Loss: 0.4850
Epoch   0 Batch 1280/2154 - Train Accuracy: 0.7056, Validation Accuracy: 0.6982, Loss: 0.4789
Epoch   0 Batch 1285/2154 - Train Accuracy: 0.6789, Validation Accuracy: 0.7109, Loss: 0.5007
Epoch   0 Batch 1290/2154 - Train Accuracy: 0.7773, Validation Accuracy: 0.7330, Loss: 0.4284
Epoch   0 Batch 1295/2154 - Train Accuracy: 0.7305, Validation Accuracy: 0.7038, Loss: 0.4196
Epoch   0 Batch 1300/2154 - Train Accuracy: 0.6820, Validation Accuracy: 0.7102, Loss: 0.4740
Epoch   0 Batch 1305/2154 - Train Accuracy: 0.7401, Validation Accuracy: 0.7287, Loss: 0.4587
Epoch   0 Batch 1310/2154 - Train Accuracy: 0.7500, Validation Accuracy: 0.7287, Loss: 0.4168
Epoch   0 Batch 1315/2154 - Train Accuracy: 0.7336, Validation Accuracy: 0.6903, Loss: 0.4440
Epoch   0 Batch 1320/2154 - Train Accuracy: 0.7612, Validation Accuracy: 0.7237, Loss: 0.3822
Epoch   0 Batch 1325/2154 - Train Accuracy: 0.7240, Validation Accuracy: 0.7138, Loss: 0.3946
Epoch   0 Batch 1330/2154 - Train Accuracy: 0.7945, Validation Accuracy: 0.7244, Loss: 0.3977
Epoch   0 Batch 1335/2154 - Train Accuracy: 0.7179, Validation Accuracy: 0.7294, Loss: 0.4682
Epoch   0 Batch 1340/2154 - Train Accuracy: 0.7477, Validation Accuracy: 0.7486, Loss: 0.4219
Epoch   0 Batch 1345/2154 - Train Accuracy: 0.7212, Validation Accuracy: 0.7301, Loss: 0.4572
Epoch   0 Batch 1350/2154 - Train Accuracy: 0.7320, Validation Accuracy: 0.7280, Loss: 0.4202
Epoch   0 Batch 1355/2154 - Train Accuracy: 0.7266, Validation Accuracy: 0.7457, Loss: 0.4497
Epoch   0 Batch 1360/2154 - Train Accuracy: 0.7253, Validation Accuracy: 0.7266, Loss: 0.4216
Epoch   0 Batch 1365/2154 - Train Accuracy: 0.7336, Validation Accuracy: 0.7372, Loss: 0.4089
Epoch   0 Batch 1370/2154 - Train Accuracy: 0.7297, Validation Accuracy: 0.7386, Loss: 0.4133
Epoch   0 Batch 1375/2154 - Train Accuracy: 0.7352, Validation Accuracy: 0.7308, Loss: 0.4484
Epoch   0 Batch 1380/2154 - Train Accuracy: 0.7711, Validation Accuracy: 0.7472, Loss: 0.4131
Epoch   0 Batch 1385/2154 - Train Accuracy: 0.7477, Validation Accuracy: 0.7415, Loss: 0.3912
Epoch   0 Batch 1390/2154 - Train Accuracy: 0.7552, Validation Accuracy: 0.7216, Loss: 0.3972
Epoch   0 Batch 1395/2154 - Train Accuracy: 0.6867, Validation Accuracy: 0.7223, Loss: 0.4310
Epoch   0 Batch 1400/2154 - Train Accuracy: 0.7492, Validation Accuracy: 0.6989, Loss: 0.4078
Epoch   0 Batch 1405/2154 - Train Accuracy: 0.7716, Validation Accuracy: 0.7358, Loss: 0.4220
Epoch   0 Batch 1410/2154 - Train Accuracy: 0.6992, Validation Accuracy: 0.7280, Loss: 0.4127
Epoch   0 Batch 1415/2154 - Train Accuracy: 0.7125, Validation Accuracy: 0.7337, Loss: 0.4089
Epoch   0 Batch 1420/2154 - Train Accuracy: 0.7451, Validation Accuracy: 0.7578, Loss: 0.4259
Epoch   0 Batch 1425/2154 - Train Accuracy: 0.7375, Validation Accuracy: 0.7280, Loss: 0.3784
Epoch   0 Batch 1430/2154 - Train Accuracy: 0.6682, Validation Accuracy: 0.7415, Loss: 0.4043
Epoch   0 Batch 1435/2154 - Train Accuracy: 0.7558, Validation Accuracy: 0.7543, Loss: 0.3802
Epoch   0 Batch 1440/2154 - Train Accuracy: 0.7562, Validation Accuracy: 0.7599, Loss: 0.4145
Epoch   0 Batch 1445/2154 - Train Accuracy: 0.7805, Validation Accuracy: 0.7635, Loss: 0.3568
Epoch   0 Batch 1450/2154 - Train Accuracy: 0.7459, Validation Accuracy: 0.7642, Loss: 0.4024
Epoch   0 Batch 1455/2154 - Train Accuracy: 0.7477, Validation Accuracy: 0.7500, Loss: 0.3827
Epoch   0 Batch 1460/2154 - Train Accuracy: 0.7422, Validation Accuracy: 0.7571, Loss: 0.4255
Epoch   0 Batch 1465/2154 - Train Accuracy: 0.7722, Validation Accuracy: 0.7386, Loss: 0.3848
Epoch   0 Batch 1470/2154 - Train Accuracy: 0.7566, Validation Accuracy: 0.7330, Loss: 0.4117
Epoch   0 Batch 1475/2154 - Train Accuracy: 0.7859, Validation Accuracy: 0.7585, Loss: 0.4371
Epoch   0 Batch 1480/2154 - Train Accuracy: 0.7602, Validation Accuracy: 0.7351, Loss: 0.3842
Epoch   0 Batch 1485/2154 - Train Accuracy: 0.7367, Validation Accuracy: 0.7393, Loss: 0.3946
Epoch   0 Batch 1490/2154 - Train Accuracy: 0.7594, Validation Accuracy: 0.7528, Loss: 0.3633
Epoch   0 Batch 1495/2154 - Train Accuracy: 0.7944, Validation Accuracy: 0.7663, Loss: 0.3547
Epoch   0 Batch 1500/2154 - Train Accuracy: 0.7632, Validation Accuracy: 0.7493, Loss: 0.4020
Epoch   0 Batch 1505/2154 - Train Accuracy: 0.6727, Validation Accuracy: 0.7479, Loss: 0.3774
Epoch   0 Batch 1510/2154 - Train Accuracy: 0.7525, Validation Accuracy: 0.7500, Loss: 0.4044
Epoch   0 Batch 1515/2154 - Train Accuracy: 0.8109, Validation Accuracy: 0.7599, Loss: 0.3542
Epoch   0 Batch 1520/2154 - Train Accuracy: 0.8076, Validation Accuracy: 0.7592, Loss: 0.3357
Epoch   0 Batch 1525/2154 - Train Accuracy: 0.7977, Validation Accuracy: 0.7521, Loss: 0.3669
Epoch   0 Batch 1530/2154 - Train Accuracy: 0.7821, Validation Accuracy: 0.7450, Loss: 0.3662
Epoch   0 Batch 1535/2154 - Train Accuracy: 0.8000, Validation Accuracy: 0.7216, Loss: 0.3701
Epoch   0 Batch 1540/2154 - Train Accuracy: 0.7977, Validation Accuracy: 0.7599, Loss: 0.3742
Epoch   0 Batch 1545/2154 - Train Accuracy: 0.7701, Validation Accuracy: 0.7734, Loss: 0.3388
Epoch   0 Batch 1550/2154 - Train Accuracy: 0.7773, Validation Accuracy: 0.7791, Loss: 0.3911
Epoch   0 Batch 1555/2154 - Train Accuracy: 0.7406, Validation Accuracy: 0.7734, Loss: 0.3691
Epoch   0 Batch 1560/2154 - Train Accuracy: 0.7281, Validation Accuracy: 0.7720, Loss: 0.3747
Epoch   0 Batch 1565/2154 - Train Accuracy: 0.7619, Validation Accuracy: 0.7578, Loss: 0.3735
Epoch   0 Batch 1570/2154 - Train Accuracy: 0.7312, Validation Accuracy: 0.7727, Loss: 0.3730
Epoch   0 Batch 1575/2154 - Train Accuracy: 0.7545, Validation Accuracy: 0.7720, Loss: 0.3130
Epoch   0 Batch 1580/2154 - Train Accuracy: 0.7344, Validation Accuracy: 0.7926, Loss: 0.3686
Epoch   0 Batch 1585/2154 - Train Accuracy: 0.7750, Validation Accuracy: 0.7869, Loss: 0.3653
Epoch   0 Batch 1590/2154 - Train Accuracy: 0.7688, Validation Accuracy: 0.7912, Loss: 0.3585
Epoch   0 Batch 1595/2154 - Train Accuracy: 0.7961, Validation Accuracy: 0.7649, Loss: 0.3564
Epoch   0 Batch 1600/2154 - Train Accuracy: 0.7508, Validation Accuracy: 0.7663, Loss: 0.3642
Epoch   0 Batch 1605/2154 - Train Accuracy: 0.7453, Validation Accuracy: 0.7571, Loss: 0.3734
Epoch   0 Batch 1610/2154 - Train Accuracy: 0.8281, Validation Accuracy: 0.7727, Loss: 0.3384
Epoch   0 Batch 1615/2154 - Train Accuracy: 0.8289, Validation Accuracy: 0.7855, Loss: 0.3086
Epoch   0 Batch 1620/2154 - Train Accuracy: 0.7508, Validation Accuracy: 0.7685, Loss: 0.3869
Epoch   0 Batch 1625/2154 - Train Accuracy: 0.7383, Validation Accuracy: 0.7912, Loss: 0.3505
Epoch   0 Batch 1630/2154 - Train Accuracy: 0.7730, Validation Accuracy: 0.7685, Loss: 0.3452
Epoch   0 Batch 1635/2154 - Train Accuracy: 0.7640, Validation Accuracy: 0.7699, Loss: 0.4029
Epoch   0 Batch 1640/2154 - Train Accuracy: 0.8320, Validation Accuracy: 0.7621, Loss: 0.3011
Epoch   0 Batch 1645/2154 - Train Accuracy: 0.7985, Validation Accuracy: 0.7876, Loss: 0.3604
Epoch   0 Batch 1650/2154 - Train Accuracy: 0.7679, Validation Accuracy: 0.7635, Loss: 0.3109
Epoch   0 Batch 1655/2154 - Train Accuracy: 0.7952, Validation Accuracy: 0.7905, Loss: 0.3423
Epoch   0 Batch 1660/2154 - Train Accuracy: 0.7796, Validation Accuracy: 0.7855, Loss: 0.3757
Epoch   0 Batch 1665/2154 - Train Accuracy: 0.7867, Validation Accuracy: 0.8033, Loss: 0.3207
Epoch   0 Batch 1670/2154 - Train Accuracy: 0.8164, Validation Accuracy: 0.7820, Loss: 0.3558
Epoch   0 Batch 1675/2154 - Train Accuracy: 0.7977, Validation Accuracy: 0.7727, Loss: 0.3668
Epoch   0 Batch 1680/2154 - Train Accuracy: 0.7844, Validation Accuracy: 0.8004, Loss: 0.3128
Epoch   0 Batch 1685/2154 - Train Accuracy: 0.7919, Validation Accuracy: 0.7898, Loss: 0.3418
Epoch   0 Batch 1690/2154 - Train Accuracy: 0.7760, Validation Accuracy: 0.7933, Loss: 0.2918
Epoch   0 Batch 1695/2154 - Train Accuracy: 0.7928, Validation Accuracy: 0.7969, Loss: 0.3352
Epoch   0 Batch 1700/2154 - Train Accuracy: 0.7719, Validation Accuracy: 0.7891, Loss: 0.3271
Epoch   0 Batch 1705/2154 - Train Accuracy: 0.7928, Validation Accuracy: 0.7947, Loss: 0.3527
Epoch   0 Batch 1710/2154 - Train Accuracy: 0.7664, Validation Accuracy: 0.7891, Loss: 0.3320
Epoch   0 Batch 1715/2154 - Train Accuracy: 0.8553, Validation Accuracy: 0.7777, Loss: 0.3291
Epoch   0 Batch 1720/2154 - Train Accuracy: 0.7919, Validation Accuracy: 0.7969, Loss: 0.3598
Epoch   0 Batch 1725/2154 - Train Accuracy: 0.8133, Validation Accuracy: 0.7891, Loss: 0.3391
Epoch   0 Batch 1730/2154 - Train Accuracy: 0.8250, Validation Accuracy: 0.7869, Loss: 0.3109
Epoch   0 Batch 1735/2154 - Train Accuracy: 0.7516, Validation Accuracy: 0.7884, Loss: 0.3491
Epoch   0 Batch 1740/2154 - Train Accuracy: 0.7796, Validation Accuracy: 0.7699, Loss: 0.3408
Epoch   0 Batch 1745/2154 - Train Accuracy: 0.8240, Validation Accuracy: 0.7891, Loss: 0.3000
Epoch   0 Batch 1750/2154 - Train Accuracy: 0.7937, Validation Accuracy: 0.7798, Loss: 0.3173
Epoch   0 Batch 1755/2154 - Train Accuracy: 0.7672, Validation Accuracy: 0.7912, Loss: 0.3095
Epoch   0 Batch 1760/2154 - Train Accuracy: 0.7977, Validation Accuracy: 0.7955, Loss: 0.3183
Epoch   0 Batch 1765/2154 - Train Accuracy: 0.8257, Validation Accuracy: 0.8040, Loss: 0.3526
Epoch   0 Batch 1770/2154 - Train Accuracy: 0.8008, Validation Accuracy: 0.7997, Loss: 0.3091
Epoch   0 Batch 1775/2154 - Train Accuracy: 0.8016, Validation Accuracy: 0.8097, Loss: 0.3130
Epoch   0 Batch 1780/2154 - Train Accuracy: 0.8359, Validation Accuracy: 0.8068, Loss: 0.3035
Epoch   0 Batch 1785/2154 - Train Accuracy: 0.8207, Validation Accuracy: 0.7898, Loss: 0.3069
Epoch   0 Batch 1790/2154 - Train Accuracy: 0.8073, Validation Accuracy: 0.8075, Loss: 0.3044
Epoch   0 Batch 1795/2154 - Train Accuracy: 0.8000, Validation Accuracy: 0.8054, Loss: 0.2945
Epoch   0 Batch 1800/2154 - Train Accuracy: 0.8187, Validation Accuracy: 0.8161, Loss: 0.2958
Epoch   0 Batch 1805/2154 - Train Accuracy: 0.7734, Validation Accuracy: 0.8068, Loss: 0.3289
Epoch   0 Batch 1810/2154 - Train Accuracy: 0.7523, Validation Accuracy: 0.7969, Loss: 0.3209
Epoch   0 Batch 1815/2154 - Train Accuracy: 0.8094, Validation Accuracy: 0.7919, Loss: 0.2977
Epoch   0 Batch 1820/2154 - Train Accuracy: 0.8148, Validation Accuracy: 0.8146, Loss: 0.2678
Epoch   0 Batch 1825/2154 - Train Accuracy: 0.7719, Validation Accuracy: 0.7642, Loss: 0.3054
Epoch   0 Batch 1830/2154 - Train Accuracy: 0.8503, Validation Accuracy: 0.7919, Loss: 0.2793
Epoch   0 Batch 1835/2154 - Train Accuracy: 0.8125, Validation Accuracy: 0.7692, Loss: 0.2803
Epoch   0 Batch 1840/2154 - Train Accuracy: 0.7960, Validation Accuracy: 0.7933, Loss: 0.3038
Epoch   0 Batch 1845/2154 - Train Accuracy: 0.7680, Validation Accuracy: 0.8054, Loss: 0.2965
Epoch   0 Batch 1850/2154 - Train Accuracy: 0.7812, Validation Accuracy: 0.7812, Loss: 0.3529
Epoch   0 Batch 1855/2154 - Train Accuracy: 0.8297, Validation Accuracy: 0.7898, Loss: 0.2962
Epoch   0 Batch 1860/2154 - Train Accuracy: 0.8232, Validation Accuracy: 0.8104, Loss: 0.3097
Epoch   0 Batch 1865/2154 - Train Accuracy: 0.8051, Validation Accuracy: 0.7940, Loss: 0.2850
Epoch   0 Batch 1870/2154 - Train Accuracy: 0.8141, Validation Accuracy: 0.7763, Loss: 0.2899
Epoch   0 Batch 1875/2154 - Train Accuracy: 0.8594, Validation Accuracy: 0.7855, Loss: 0.3107
Epoch   0 Batch 1880/2154 - Train Accuracy: 0.8148, Validation Accuracy: 0.7770, Loss: 0.2843
Epoch   0 Batch 1885/2154 - Train Accuracy: 0.7525, Validation Accuracy: 0.7940, Loss: 0.3189
Epoch   0 Batch 1890/2154 - Train Accuracy: 0.7999, Validation Accuracy: 0.7798, Loss: 0.2722
Epoch   0 Batch 1895/2154 - Train Accuracy: 0.7919, Validation Accuracy: 0.7933, Loss: 0.2775
Epoch   0 Batch 1900/2154 - Train Accuracy: 0.8758, Validation Accuracy: 0.7834, Loss: 0.2654
Epoch   0 Batch 1905/2154 - Train Accuracy: 0.8133, Validation Accuracy: 0.7820, Loss: 0.2559
Epoch   0 Batch 1910/2154 - Train Accuracy: 0.7599, Validation Accuracy: 0.7763, Loss: 0.2731
Epoch   0 Batch 1915/2154 - Train Accuracy: 0.8781, Validation Accuracy: 0.7997, Loss: 0.2465
Epoch   0 Batch 1920/2154 - Train Accuracy: 0.8470, Validation Accuracy: 0.7919, Loss: 0.2663
Epoch   0 Batch 1925/2154 - Train Accuracy: 0.8092, Validation Accuracy: 0.8011, Loss: 0.2750
Epoch   0 Batch 1930/2154 - Train Accuracy: 0.8180, Validation Accuracy: 0.7962, Loss: 0.2594
Epoch   0 Batch 1935/2154 - Train Accuracy: 0.7734, Validation Accuracy: 0.7805, Loss: 0.2894
Epoch   0 Batch 1940/2154 - Train Accuracy: 0.8378, Validation Accuracy: 0.7855, Loss: 0.2545
Epoch   0 Batch 1945/2154 - Train Accuracy: 0.8016, Validation Accuracy: 0.7713, Loss: 0.2546
Epoch   0 Batch 1950/2154 - Train Accuracy: 0.8023, Validation Accuracy: 0.7862, Loss: 0.2653
Epoch   0 Batch 1955/2154 - Train Accuracy: 0.8516, Validation Accuracy: 0.7933, Loss: 0.2592
Epoch   0 Batch 1960/2154 - Train Accuracy: 0.7697, Validation Accuracy: 0.8047, Loss: 0.3108
Epoch   0 Batch 1965/2154 - Train Accuracy: 0.8065, Validation Accuracy: 0.8239, Loss: 0.2694
Epoch   0 Batch 1970/2154 - Train Accuracy: 0.7648, Validation Accuracy: 0.8068, Loss: 0.2899
Epoch   0 Batch 1975/2154 - Train Accuracy: 0.7352, Validation Accuracy: 0.8033, Loss: 0.2806
Epoch   0 Batch 1980/2154 - Train Accuracy: 0.8461, Validation Accuracy: 0.8047, Loss: 0.2581
Epoch   0 Batch 1985/2154 - Train Accuracy: 0.8651, Validation Accuracy: 0.8004, Loss: 0.2709
Epoch   0 Batch 1990/2154 - Train Accuracy: 0.8602, Validation Accuracy: 0.8196, Loss: 0.2436
Epoch   0 Batch 1995/2154 - Train Accuracy: 0.8512, Validation Accuracy: 0.8139, Loss: 0.2794
Epoch   0 Batch 2000/2154 - Train Accuracy: 0.8523, Validation Accuracy: 0.8075, Loss: 0.2578
Epoch   0 Batch 2005/2154 - Train Accuracy: 0.8150, Validation Accuracy: 0.7869, Loss: 0.2692
Epoch   0 Batch 2010/2154 - Train Accuracy: 0.8092, Validation Accuracy: 0.7834, Loss: 0.3280
Epoch   0 Batch 2015/2154 - Train Accuracy: 0.8519, Validation Accuracy: 0.8026, Loss: 0.2364
Epoch   0 Batch 2020/2154 - Train Accuracy: 0.8865, Validation Accuracy: 0.8168, Loss: 0.2414
Epoch   0 Batch 2025/2154 - Train Accuracy: 0.8055, Validation Accuracy: 0.8189, Loss: 0.2494
Epoch   0 Batch 2030/2154 - Train Accuracy: 0.8023, Validation Accuracy: 0.8089, Loss: 0.2506
Epoch   0 Batch 2035/2154 - Train Accuracy: 0.8651, Validation Accuracy: 0.8011, Loss: 0.2538
Epoch   0 Batch 2040/2154 - Train Accuracy: 0.8273, Validation Accuracy: 0.7898, Loss: 0.2552
Epoch   0 Batch 2045/2154 - Train Accuracy: 0.8698, Validation Accuracy: 0.7926, Loss: 0.2042
Epoch   0 Batch 2050/2154 - Train Accuracy: 0.7919, Validation Accuracy: 0.8018, Loss: 0.3003
Epoch   0 Batch 2055/2154 - Train Accuracy: 0.7706, Validation Accuracy: 0.8040, Loss: 0.2851
Epoch   0 Batch 2060/2154 - Train Accuracy: 0.8331, Validation Accuracy: 0.8054, Loss: 0.2532
Epoch   0 Batch 2065/2154 - Train Accuracy: 0.8508, Validation Accuracy: 0.8040, Loss: 0.2483
Epoch   0 Batch 2070/2154 - Train Accuracy: 0.8102, Validation Accuracy: 0.8161, Loss: 0.2544
Epoch   0 Batch 2075/2154 - Train Accuracy: 0.8313, Validation Accuracy: 0.7969, Loss: 0.2477
Epoch   0 Batch 2080/2154 - Train Accuracy: 0.8460, Validation Accuracy: 0.8338, Loss: 0.2232
Epoch   0 Batch 2085/2154 - Train Accuracy: 0.8500, Validation Accuracy: 0.8317, Loss: 0.2418
Epoch   0 Batch 2090/2154 - Train Accuracy: 0.8187, Validation Accuracy: 0.8295, Loss: 0.2443
Epoch   0 Batch 2095/2154 - Train Accuracy: 0.8492, Validation Accuracy: 0.8281, Loss: 0.2337
Epoch   0 Batch 2100/2154 - Train Accuracy: 0.8727, Validation Accuracy: 0.8089, Loss: 0.2270
Epoch   0 Batch 2105/2154 - Train Accuracy: 0.8668, Validation Accuracy: 0.8239, Loss: 0.2409
Epoch   0 Batch 2110/2154 - Train Accuracy: 0.8453, Validation Accuracy: 0.7997, Loss: 0.2349
Epoch   0 Batch 2115/2154 - Train Accuracy: 0.8248, Validation Accuracy: 0.7926, Loss: 0.2564
Epoch   0 Batch 2120/2154 - Train Accuracy: 0.7944, Validation Accuracy: 0.7820, Loss: 0.2464
Epoch   0 Batch 2125/2154 - Train Accuracy: 0.8328, Validation Accuracy: 0.7869, Loss: 0.2521
Epoch   0 Batch 2130/2154 - Train Accuracy: 0.8890, Validation Accuracy: 0.7834, Loss: 0.2295
Epoch   0 Batch 2135/2154 - Train Accuracy: 0.8531, Validation Accuracy: 0.7969, Loss: 0.2436
Epoch   0 Batch 2140/2154 - Train Accuracy: 0.8445, Validation Accuracy: 0.8146, Loss: 0.2113
Epoch   0 Batch 2145/2154 - Train Accuracy: 0.8314, Validation Accuracy: 0.8246, Loss: 0.2280
Epoch   0 Batch 2150/2154 - Train Accuracy: 0.8512, Validation Accuracy: 0.8111, Loss: 0.2674
Epoch   1 Batch    5/2154 - Train Accuracy: 0.8314, Validation Accuracy: 0.8381, Loss: 0.2408
Epoch   1 Batch   10/2154 - Train Accuracy: 0.8352, Validation Accuracy: 0.8317, Loss: 0.2143
Epoch   1 Batch   15/2154 - Train Accuracy: 0.8742, Validation Accuracy: 0.8281, Loss: 0.2249
Epoch   1 Batch   20/2154 - Train Accuracy: 0.8258, Validation Accuracy: 0.8303, Loss: 0.2297
Epoch   1 Batch   25/2154 - Train Accuracy: 0.8391, Validation Accuracy: 0.8260, Loss: 0.2487
Epoch   1 Batch   30/2154 - Train Accuracy: 0.8187, Validation Accuracy: 0.8139, Loss: 0.1966
Epoch   1 Batch   35/2154 - Train Accuracy: 0.8758, Validation Accuracy: 0.8281, Loss: 0.2236
Epoch   1 Batch   40/2154 - Train Accuracy: 0.8026, Validation Accuracy: 0.7962, Loss: 0.2210
Epoch   1 Batch   45/2154 - Train Accuracy: 0.8405, Validation Accuracy: 0.7926, Loss: 0.2389
Epoch   1 Batch   50/2154 - Train Accuracy: 0.8454, Validation Accuracy: 0.8189, Loss: 0.2349
Epoch   1 Batch   55/2154 - Train Accuracy: 0.9000, Validation Accuracy: 0.8040, Loss: 0.1841
Epoch   1 Batch   60/2154 - Train Accuracy: 0.8688, Validation Accuracy: 0.8047, Loss: 0.2056
Epoch   1 Batch   65/2154 - Train Accuracy: 0.8653, Validation Accuracy: 0.8132, Loss: 0.2076
Epoch   1 Batch   70/2154 - Train Accuracy: 0.8322, Validation Accuracy: 0.8239, Loss: 0.2507
Epoch   1 Batch   75/2154 - Train Accuracy: 0.8289, Validation Accuracy: 0.8203, Loss: 0.2328
Epoch   1 Batch   80/2154 - Train Accuracy: 0.8336, Validation Accuracy: 0.8295, Loss: 0.2378
Epoch   1 Batch   85/2154 - Train Accuracy: 0.8359, Validation Accuracy: 0.7919, Loss: 0.2447
Epoch   1 Batch   90/2154 - Train Accuracy: 0.8289, Validation Accuracy: 0.8047, Loss: 0.2337
Epoch   1 Batch   95/2154 - Train Accuracy: 0.8930, Validation Accuracy: 0.8253, Loss: 0.1984
Epoch   1 Batch  100/2154 - Train Accuracy: 0.8609, Validation Accuracy: 0.8423, Loss: 0.2265
Epoch   1 Batch  105/2154 - Train Accuracy: 0.8594, Validation Accuracy: 0.8494, Loss: 0.2141
Epoch   1 Batch  110/2154 - Train Accuracy: 0.8844, Validation Accuracy: 0.8295, Loss: 0.2498
Epoch   1 Batch  115/2154 - Train Accuracy: 0.8246, Validation Accuracy: 0.8161, Loss: 0.2025
Epoch   1 Batch  120/2154 - Train Accuracy: 0.8610, Validation Accuracy: 0.8082, Loss: 0.2140
Epoch   1 Batch  125/2154 - Train Accuracy: 0.8405, Validation Accuracy: 0.8132, Loss: 0.2129
Epoch   1 Batch  130/2154 - Train Accuracy: 0.8648, Validation Accuracy: 0.8232, Loss: 0.1909
Epoch   1 Batch  135/2154 - Train Accuracy: 0.8333, Validation Accuracy: 0.8224, Loss: 0.2287
Epoch   1 Batch  140/2154 - Train Accuracy: 0.8469, Validation Accuracy: 0.8295, Loss: 0.2361
Epoch   1 Batch  145/2154 - Train Accuracy: 0.8617, Validation Accuracy: 0.8430, Loss: 0.2101
Epoch   1 Batch  150/2154 - Train Accuracy: 0.8899, Validation Accuracy: 0.8303, Loss: 0.1709
Epoch   1 Batch  155/2154 - Train Accuracy: 0.8281, Validation Accuracy: 0.8288, Loss: 0.2249
Epoch   1 Batch  160/2154 - Train Accuracy: 0.8742, Validation Accuracy: 0.8224, Loss: 0.2094
Epoch   1 Batch  165/2154 - Train Accuracy: 0.8977, Validation Accuracy: 0.8530, Loss: 0.1715
Epoch   1 Batch  170/2154 - Train Accuracy: 0.8695, Validation Accuracy: 0.8409, Loss: 0.1842
Epoch   1 Batch  175/2154 - Train Accuracy: 0.8297, Validation Accuracy: 0.8388, Loss: 0.2084
Epoch   1 Batch  180/2154 - Train Accuracy: 0.8320, Validation Accuracy: 0.8487, Loss: 0.1992
Epoch   1 Batch  185/2154 - Train Accuracy: 0.8709, Validation Accuracy: 0.8075, Loss: 0.1906
Epoch   1 Batch  190/2154 - Train Accuracy: 0.8906, Validation Accuracy: 0.8182, Loss: 0.1682
Epoch   1 Batch  195/2154 - Train Accuracy: 0.8396, Validation Accuracy: 0.8338, Loss: 0.2091
Epoch   1 Batch  200/2154 - Train Accuracy: 0.8883, Validation Accuracy: 0.8324, Loss: 0.1904
Epoch   1 Batch  205/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.8224, Loss: 0.1814
Epoch   1 Batch  210/2154 - Train Accuracy: 0.8783, Validation Accuracy: 0.8224, Loss: 0.1846
Epoch   1 Batch  215/2154 - Train Accuracy: 0.8534, Validation Accuracy: 0.8324, Loss: 0.1757
Epoch   1 Batch  220/2154 - Train Accuracy: 0.8875, Validation Accuracy: 0.8139, Loss: 0.1833
Epoch   1 Batch  225/2154 - Train Accuracy: 0.8453, Validation Accuracy: 0.7876, Loss: 0.2026
Epoch   1 Batch  230/2154 - Train Accuracy: 0.8936, Validation Accuracy: 0.8359, Loss: 0.1731
Epoch   1 Batch  235/2154 - Train Accuracy: 0.8273, Validation Accuracy: 0.8210, Loss: 0.2029
Epoch   1 Batch  240/2154 - Train Accuracy: 0.8812, Validation Accuracy: 0.7919, Loss: 0.1989
Epoch   1 Batch  245/2154 - Train Accuracy: 0.9194, Validation Accuracy: 0.8239, Loss: 0.1840
Epoch   1 Batch  250/2154 - Train Accuracy: 0.8664, Validation Accuracy: 0.8381, Loss: 0.2068
Epoch   1 Batch  255/2154 - Train Accuracy: 0.8750, Validation Accuracy: 0.8295, Loss: 0.1770
Epoch   1 Batch  260/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.8210, Loss: 0.1836
Epoch   1 Batch  265/2154 - Train Accuracy: 0.8891, Validation Accuracy: 0.8004, Loss: 0.2215
Epoch   1 Batch  270/2154 - Train Accuracy: 0.9070, Validation Accuracy: 0.8168, Loss: 0.1730
Epoch   1 Batch  275/2154 - Train Accuracy: 0.9180, Validation Accuracy: 0.8175, Loss: 0.1563
Epoch   1 Batch  280/2154 - Train Accuracy: 0.7796, Validation Accuracy: 0.7955, Loss: 0.2154
Epoch   1 Batch  285/2154 - Train Accuracy: 0.8446, Validation Accuracy: 0.8118, Loss: 0.2056
Epoch   1 Batch  290/2154 - Train Accuracy: 0.8421, Validation Accuracy: 0.8232, Loss: 0.1990
Epoch   1 Batch  295/2154 - Train Accuracy: 0.8500, Validation Accuracy: 0.8310, Loss: 0.1697
Epoch   1 Batch  300/2154 - Train Accuracy: 0.8676, Validation Accuracy: 0.8345, Loss: 0.1893
Epoch   1 Batch  305/2154 - Train Accuracy: 0.8445, Validation Accuracy: 0.8381, Loss: 0.2029
Epoch   1 Batch  310/2154 - Train Accuracy: 0.8725, Validation Accuracy: 0.8409, Loss: 0.1908
Epoch   1 Batch  315/2154 - Train Accuracy: 0.8352, Validation Accuracy: 0.8210, Loss: 0.1811
Epoch   1 Batch  320/2154 - Train Accuracy: 0.8930, Validation Accuracy: 0.8345, Loss: 0.1519
Epoch   1 Batch  325/2154 - Train Accuracy: 0.8203, Validation Accuracy: 0.8523, Loss: 0.1943
Epoch   1 Batch  330/2154 - Train Accuracy: 0.9145, Validation Accuracy: 0.8374, Loss: 0.1664
Epoch   1 Batch  335/2154 - Train Accuracy: 0.8758, Validation Accuracy: 0.8643, Loss: 0.1491
Epoch   1 Batch  340/2154 - Train Accuracy: 0.8795, Validation Accuracy: 0.8643, Loss: 0.1572
Epoch   1 Batch  345/2154 - Train Accuracy: 0.8906, Validation Accuracy: 0.8558, Loss: 0.1823
Epoch   1 Batch  350/2154 - Train Accuracy: 0.8586, Validation Accuracy: 0.8572, Loss: 0.1813
Epoch   1 Batch  355/2154 - Train Accuracy: 0.8701, Validation Accuracy: 0.8338, Loss: 0.1911
Epoch   1 Batch  360/2154 - Train Accuracy: 0.8766, Validation Accuracy: 0.8366, Loss: 0.1661
Epoch   1 Batch  365/2154 - Train Accuracy: 0.8504, Validation Accuracy: 0.8374, Loss: 0.1880
Epoch   1 Batch  370/2154 - Train Accuracy: 0.8711, Validation Accuracy: 0.8359, Loss: 0.1809
Epoch   1 Batch  375/2154 - Train Accuracy: 0.8479, Validation Accuracy: 0.8381, Loss: 0.1586
Epoch   1 Batch  380/2154 - Train Accuracy: 0.9273, Validation Accuracy: 0.8366, Loss: 0.1613
Epoch   1 Batch  385/2154 - Train Accuracy: 0.8992, Validation Accuracy: 0.8239, Loss: 0.1680
Epoch   1 Batch  390/2154 - Train Accuracy: 0.8824, Validation Accuracy: 0.8487, Loss: 0.1444
Epoch   1 Batch  395/2154 - Train Accuracy: 0.8984, Validation Accuracy: 0.8409, Loss: 0.1712
Epoch   1 Batch  400/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8430, Loss: 0.1648
Epoch   1 Batch  405/2154 - Train Accuracy: 0.8805, Validation Accuracy: 0.8352, Loss: 0.1662
Epoch   1 Batch  410/2154 - Train Accuracy: 0.8352, Validation Accuracy: 0.8232, Loss: 0.1806
Epoch   1 Batch  415/2154 - Train Accuracy: 0.8922, Validation Accuracy: 0.8466, Loss: 0.1483
Epoch   1 Batch  420/2154 - Train Accuracy: 0.8862, Validation Accuracy: 0.8317, Loss: 0.1315
Epoch   1 Batch  425/2154 - Train Accuracy: 0.9159, Validation Accuracy: 0.8239, Loss: 0.1534
Epoch   1 Batch  430/2154 - Train Accuracy: 0.8117, Validation Accuracy: 0.8352, Loss: 0.1778
Epoch   1 Batch  435/2154 - Train Accuracy: 0.9031, Validation Accuracy: 0.8317, Loss: 0.1609
Epoch   1 Batch  440/2154 - Train Accuracy: 0.8808, Validation Accuracy: 0.8182, Loss: 0.1611
Epoch   1 Batch  445/2154 - Train Accuracy: 0.8328, Validation Accuracy: 0.8374, Loss: 0.1763
Epoch   1 Batch  450/2154 - Train Accuracy: 0.9180, Validation Accuracy: 0.8310, Loss: 0.1572
Epoch   1 Batch  455/2154 - Train Accuracy: 0.8164, Validation Accuracy: 0.8303, Loss: 0.1918
Epoch   1 Batch  460/2154 - Train Accuracy: 0.8547, Validation Accuracy: 0.8125, Loss: 0.1519
Epoch   1 Batch  465/2154 - Train Accuracy: 0.8816, Validation Accuracy: 0.8416, Loss: 0.1518
Epoch   1 Batch  470/2154 - Train Accuracy: 0.8359, Validation Accuracy: 0.8295, Loss: 0.1936
Epoch   1 Batch  475/2154 - Train Accuracy: 0.8701, Validation Accuracy: 0.8409, Loss: 0.1489
Epoch   1 Batch  480/2154 - Train Accuracy: 0.9040, Validation Accuracy: 0.8516, Loss: 0.1271
Epoch   1 Batch  485/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8331, Loss: 0.1271
Epoch   1 Batch  490/2154 - Train Accuracy: 0.8734, Validation Accuracy: 0.8288, Loss: 0.1553
Epoch   1 Batch  495/2154 - Train Accuracy: 0.9092, Validation Accuracy: 0.8111, Loss: 0.1459
Epoch   1 Batch  500/2154 - Train Accuracy: 0.8882, Validation Accuracy: 0.8217, Loss: 0.1563
Epoch   1 Batch  505/2154 - Train Accuracy: 0.8355, Validation Accuracy: 0.8253, Loss: 0.1667
Epoch   1 Batch  510/2154 - Train Accuracy: 0.8305, Validation Accuracy: 0.8288, Loss: 0.1605
Epoch   1 Batch  515/2154 - Train Accuracy: 0.8772, Validation Accuracy: 0.8125, Loss: 0.1487
Epoch   1 Batch  520/2154 - Train Accuracy: 0.8766, Validation Accuracy: 0.8317, Loss: 0.1416
Epoch   1 Batch  525/2154 - Train Accuracy: 0.9273, Validation Accuracy: 0.8359, Loss: 0.1424
Epoch   1 Batch  530/2154 - Train Accuracy: 0.9195, Validation Accuracy: 0.8331, Loss: 0.1468
Epoch   1 Batch  535/2154 - Train Accuracy: 0.8750, Validation Accuracy: 0.8494, Loss: 0.1463
Epoch   1 Batch  540/2154 - Train Accuracy: 0.8692, Validation Accuracy: 0.8381, Loss: 0.1564
Epoch   1 Batch  545/2154 - Train Accuracy: 0.8684, Validation Accuracy: 0.8239, Loss: 0.1918
Epoch   1 Batch  550/2154 - Train Accuracy: 0.9107, Validation Accuracy: 0.8253, Loss: 0.1509
Epoch   1 Batch  555/2154 - Train Accuracy: 0.8789, Validation Accuracy: 0.8402, Loss: 0.1526
Epoch   1 Batch  560/2154 - Train Accuracy: 0.8781, Validation Accuracy: 0.8537, Loss: 0.1503
Epoch   1 Batch  565/2154 - Train Accuracy: 0.8680, Validation Accuracy: 0.8338, Loss: 0.1529
Epoch   1 Batch  570/2154 - Train Accuracy: 0.8969, Validation Accuracy: 0.8253, Loss: 0.1641
Epoch   1 Batch  575/2154 - Train Accuracy: 0.8477, Validation Accuracy: 0.8416, Loss: 0.1505
Epoch   1 Batch  580/2154 - Train Accuracy: 0.8553, Validation Accuracy: 0.8402, Loss: 0.1527
Epoch   1 Batch  585/2154 - Train Accuracy: 0.9085, Validation Accuracy: 0.8459, Loss: 0.1599
Epoch   1 Batch  590/2154 - Train Accuracy: 0.8800, Validation Accuracy: 0.8253, Loss: 0.1455
Epoch   1 Batch  595/2154 - Train Accuracy: 0.8672, Validation Accuracy: 0.8359, Loss: 0.1428
Epoch   1 Batch  600/2154 - Train Accuracy: 0.9285, Validation Accuracy: 0.8388, Loss: 0.1347
Epoch   1 Batch  605/2154 - Train Accuracy: 0.8758, Validation Accuracy: 0.8516, Loss: 0.1428
Epoch   1 Batch  610/2154 - Train Accuracy: 0.8735, Validation Accuracy: 0.8345, Loss: 0.1227
Epoch   1 Batch  615/2154 - Train Accuracy: 0.8862, Validation Accuracy: 0.8445, Loss: 0.1271
Epoch   1 Batch  620/2154 - Train Accuracy: 0.8549, Validation Accuracy: 0.8295, Loss: 0.1313
Epoch   1 Batch  625/2154 - Train Accuracy: 0.8503, Validation Accuracy: 0.8381, Loss: 0.1659
Epoch   1 Batch  630/2154 - Train Accuracy: 0.9285, Validation Accuracy: 0.8480, Loss: 0.1368
Epoch   1 Batch  635/2154 - Train Accuracy: 0.8635, Validation Accuracy: 0.8381, Loss: 0.1629
Epoch   1 Batch  640/2154 - Train Accuracy: 0.8531, Validation Accuracy: 0.8260, Loss: 0.1471
Epoch   1 Batch  645/2154 - Train Accuracy: 0.8638, Validation Accuracy: 0.8033, Loss: 0.1277
Epoch   1 Batch  650/2154 - Train Accuracy: 0.8742, Validation Accuracy: 0.8161, Loss: 0.1368
Epoch   1 Batch  655/2154 - Train Accuracy: 0.8972, Validation Accuracy: 0.8267, Loss: 0.1718
Epoch   1 Batch  660/2154 - Train Accuracy: 0.8805, Validation Accuracy: 0.8523, Loss: 0.1447
Epoch   1 Batch  665/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.8366, Loss: 0.1141
Epoch   1 Batch  670/2154 - Train Accuracy: 0.9178, Validation Accuracy: 0.8338, Loss: 0.1355
Epoch   1 Batch  675/2154 - Train Accuracy: 0.8805, Validation Accuracy: 0.8459, Loss: 0.1359
Epoch   1 Batch  680/2154 - Train Accuracy: 0.9054, Validation Accuracy: 0.8366, Loss: 0.1274
Epoch   1 Batch  685/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.8466, Loss: 0.1328
Epoch   1 Batch  690/2154 - Train Accuracy: 0.8555, Validation Accuracy: 0.8352, Loss: 0.1387
Epoch   1 Batch  695/2154 - Train Accuracy: 0.8641, Validation Accuracy: 0.8310, Loss: 0.1257
Epoch   1 Batch  700/2154 - Train Accuracy: 0.8380, Validation Accuracy: 0.8423, Loss: 0.1235
Epoch   1 Batch  705/2154 - Train Accuracy: 0.9047, Validation Accuracy: 0.8594, Loss: 0.1203
Epoch   1 Batch  710/2154 - Train Accuracy: 0.8852, Validation Accuracy: 0.8679, Loss: 0.1352
Epoch   1 Batch  715/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.8672, Loss: 0.1320
Epoch   1 Batch  720/2154 - Train Accuracy: 0.8882, Validation Accuracy: 0.8665, Loss: 0.1532
Epoch   1 Batch  725/2154 - Train Accuracy: 0.8996, Validation Accuracy: 0.8679, Loss: 0.1175
Epoch   1 Batch  730/2154 - Train Accuracy: 0.8438, Validation Accuracy: 0.8580, Loss: 0.1698
Epoch   1 Batch  735/2154 - Train Accuracy: 0.9249, Validation Accuracy: 0.8097, Loss: 0.1143
Epoch   1 Batch  740/2154 - Train Accuracy: 0.9125, Validation Accuracy: 0.8438, Loss: 0.1293
Epoch   1 Batch  745/2154 - Train Accuracy: 0.9104, Validation Accuracy: 0.8587, Loss: 0.1145
Epoch   1 Batch  750/2154 - Train Accuracy: 0.8922, Validation Accuracy: 0.8359, Loss: 0.1564
Epoch   1 Batch  755/2154 - Train Accuracy: 0.8775, Validation Accuracy: 0.8381, Loss: 0.1532
Epoch   1 Batch  760/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.8303, Loss: 0.1437
Epoch   1 Batch  765/2154 - Train Accuracy: 0.8765, Validation Accuracy: 0.8409, Loss: 0.1665
Epoch   1 Batch  770/2154 - Train Accuracy: 0.9187, Validation Accuracy: 0.8303, Loss: 0.1115
Epoch   1 Batch  775/2154 - Train Accuracy: 0.9194, Validation Accuracy: 0.8310, Loss: 0.1430
Epoch   1 Batch  780/2154 - Train Accuracy: 0.9133, Validation Accuracy: 0.8338, Loss: 0.0990
Epoch   1 Batch  785/2154 - Train Accuracy: 0.9186, Validation Accuracy: 0.8572, Loss: 0.1318
Epoch   1 Batch  790/2154 - Train Accuracy: 0.9078, Validation Accuracy: 0.8430, Loss: 0.1299
Epoch   1 Batch  795/2154 - Train Accuracy: 0.8906, Validation Accuracy: 0.8388, Loss: 0.1279
Epoch   1 Batch  800/2154 - Train Accuracy: 0.8783, Validation Accuracy: 0.8224, Loss: 0.1208
Epoch   1 Batch  805/2154 - Train Accuracy: 0.8857, Validation Accuracy: 0.8274, Loss: 0.1097
Epoch   1 Batch  810/2154 - Train Accuracy: 0.9293, Validation Accuracy: 0.8317, Loss: 0.1131
Epoch   1 Batch  815/2154 - Train Accuracy: 0.8883, Validation Accuracy: 0.8537, Loss: 0.1431
Epoch   1 Batch  820/2154 - Train Accuracy: 0.9038, Validation Accuracy: 0.8409, Loss: 0.1256
Epoch   1 Batch  825/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.8480, Loss: 0.1053
Epoch   1 Batch  830/2154 - Train Accuracy: 0.8461, Validation Accuracy: 0.8601, Loss: 0.1355
Epoch   1 Batch  835/2154 - Train Accuracy: 0.8438, Validation Accuracy: 0.8643, Loss: 0.1623
Epoch   1 Batch  840/2154 - Train Accuracy: 0.8758, Validation Accuracy: 0.8658, Loss: 0.1386
Epoch   1 Batch  845/2154 - Train Accuracy: 0.8637, Validation Accuracy: 0.8672, Loss: 0.1397
Epoch   1 Batch  850/2154 - Train Accuracy: 0.8555, Validation Accuracy: 0.8551, Loss: 0.1049
Epoch   1 Batch  855/2154 - Train Accuracy: 0.8602, Validation Accuracy: 0.8672, Loss: 0.1435
Epoch   1 Batch  860/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.8480, Loss: 0.1333
Epoch   1 Batch  865/2154 - Train Accuracy: 0.9023, Validation Accuracy: 0.8161, Loss: 0.1229
Epoch   1 Batch  870/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.8168, Loss: 0.1136
Epoch   1 Batch  875/2154 - Train Accuracy: 0.8972, Validation Accuracy: 0.8501, Loss: 0.1146
Epoch   1 Batch  880/2154 - Train Accuracy: 0.8773, Validation Accuracy: 0.8601, Loss: 0.1410
Epoch   1 Batch  885/2154 - Train Accuracy: 0.8631, Validation Accuracy: 0.8295, Loss: 0.1431
Epoch   1 Batch  890/2154 - Train Accuracy: 0.8865, Validation Accuracy: 0.8381, Loss: 0.1277
Epoch   1 Batch  895/2154 - Train Accuracy: 0.8618, Validation Accuracy: 0.8260, Loss: 0.1304
Epoch   1 Batch  900/2154 - Train Accuracy: 0.8956, Validation Accuracy: 0.8168, Loss: 0.1359
Epoch   1 Batch  905/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.8161, Loss: 0.1218
Epoch   1 Batch  910/2154 - Train Accuracy: 0.9095, Validation Accuracy: 0.8359, Loss: 0.1163
Epoch   1 Batch  915/2154 - Train Accuracy: 0.9167, Validation Accuracy: 0.8608, Loss: 0.0986
Epoch   1 Batch  920/2154 - Train Accuracy: 0.8824, Validation Accuracy: 0.8608, Loss: 0.0905
Epoch   1 Batch  925/2154 - Train Accuracy: 0.9055, Validation Accuracy: 0.8629, Loss: 0.1056
Epoch   1 Batch  930/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.8423, Loss: 0.1241
Epoch   1 Batch  935/2154 - Train Accuracy: 0.8766, Validation Accuracy: 0.8324, Loss: 0.1366
Epoch   1 Batch  940/2154 - Train Accuracy: 0.8562, Validation Accuracy: 0.8530, Loss: 0.1247
Epoch   1 Batch  945/2154 - Train Accuracy: 0.8867, Validation Accuracy: 0.8423, Loss: 0.1068
Epoch   1 Batch  950/2154 - Train Accuracy: 0.8766, Validation Accuracy: 0.8288, Loss: 0.1309
Epoch   1 Batch  955/2154 - Train Accuracy: 0.8837, Validation Accuracy: 0.8359, Loss: 0.1304
Epoch   1 Batch  960/2154 - Train Accuracy: 0.8938, Validation Accuracy: 0.8473, Loss: 0.1296
Epoch   1 Batch  965/2154 - Train Accuracy: 0.9038, Validation Accuracy: 0.8430, Loss: 0.1396
Epoch   1 Batch  970/2154 - Train Accuracy: 0.8930, Validation Accuracy: 0.8423, Loss: 0.1415
Epoch   1 Batch  975/2154 - Train Accuracy: 0.9153, Validation Accuracy: 0.8423, Loss: 0.1208
Epoch   1 Batch  980/2154 - Train Accuracy: 0.8922, Validation Accuracy: 0.8388, Loss: 0.1040
Epoch   1 Batch  985/2154 - Train Accuracy: 0.8719, Validation Accuracy: 0.8395, Loss: 0.1227
Epoch   1 Batch  990/2154 - Train Accuracy: 0.8781, Validation Accuracy: 0.8509, Loss: 0.0947
Epoch   1 Batch  995/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.8516, Loss: 0.1022
Epoch   1 Batch 1000/2154 - Train Accuracy: 0.8906, Validation Accuracy: 0.8601, Loss: 0.1080
Epoch   1 Batch 1005/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8651, Loss: 0.1002
Epoch   1 Batch 1010/2154 - Train Accuracy: 0.8844, Validation Accuracy: 0.8778, Loss: 0.1215
Epoch   1 Batch 1015/2154 - Train Accuracy: 0.8314, Validation Accuracy: 0.8530, Loss: 0.1271
Epoch   1 Batch 1020/2154 - Train Accuracy: 0.8766, Validation Accuracy: 0.8601, Loss: 0.1224
Epoch   1 Batch 1025/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.8594, Loss: 0.0995
Epoch   1 Batch 1030/2154 - Train Accuracy: 0.8882, Validation Accuracy: 0.8580, Loss: 0.1273
Epoch   1 Batch 1035/2154 - Train Accuracy: 0.8854, Validation Accuracy: 0.8679, Loss: 0.1029
Epoch   1 Batch 1040/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.8480, Loss: 0.0863
Epoch   1 Batch 1045/2154 - Train Accuracy: 0.8355, Validation Accuracy: 0.8430, Loss: 0.1411
Epoch   1 Batch 1050/2154 - Train Accuracy: 0.8797, Validation Accuracy: 0.8459, Loss: 0.1263
Epoch   1 Batch 1055/2154 - Train Accuracy: 0.9095, Validation Accuracy: 0.9098, Loss: 0.0974
Epoch   1 Batch 1060/2154 - Train Accuracy: 0.8508, Validation Accuracy: 0.8977, Loss: 0.1122
Epoch   1 Batch 1065/2154 - Train Accuracy: 0.8938, Validation Accuracy: 0.8878, Loss: 0.1182
Epoch   1 Batch 1070/2154 - Train Accuracy: 0.8958, Validation Accuracy: 0.8736, Loss: 0.1042
Epoch   1 Batch 1075/2154 - Train Accuracy: 0.9047, Validation Accuracy: 0.8977, Loss: 0.1007
Epoch   1 Batch 1080/2154 - Train Accuracy: 0.8539, Validation Accuracy: 0.8906, Loss: 0.1243
Epoch   1 Batch 1085/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.8622, Loss: 0.1016
Epoch   1 Batch 1090/2154 - Train Accuracy: 0.9095, Validation Accuracy: 0.8636, Loss: 0.1037
Epoch   1 Batch 1095/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8587, Loss: 0.1018
Epoch   1 Batch 1100/2154 - Train Accuracy: 0.8711, Validation Accuracy: 0.8665, Loss: 0.1181
Epoch   1 Batch 1105/2154 - Train Accuracy: 0.9070, Validation Accuracy: 0.8565, Loss: 0.0980
Epoch   1 Batch 1110/2154 - Train Accuracy: 0.9023, Validation Accuracy: 0.8707, Loss: 0.1191
Epoch   1 Batch 1115/2154 - Train Accuracy: 0.8947, Validation Accuracy: 0.8821, Loss: 0.1146
Epoch   1 Batch 1120/2154 - Train Accuracy: 0.8719, Validation Accuracy: 0.8622, Loss: 0.1030
Epoch   1 Batch 1125/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.8587, Loss: 0.0889
Epoch   1 Batch 1130/2154 - Train Accuracy: 0.8635, Validation Accuracy: 0.8523, Loss: 0.1194
Epoch   1 Batch 1135/2154 - Train Accuracy: 0.8914, Validation Accuracy: 0.8693, Loss: 0.0979
Epoch   1 Batch 1140/2154 - Train Accuracy: 0.8750, Validation Accuracy: 0.8487, Loss: 0.1119
Epoch   1 Batch 1145/2154 - Train Accuracy: 0.8799, Validation Accuracy: 0.8501, Loss: 0.1094
Epoch   1 Batch 1150/2154 - Train Accuracy: 0.8997, Validation Accuracy: 0.8807, Loss: 0.1114
Epoch   1 Batch 1155/2154 - Train Accuracy: 0.8873, Validation Accuracy: 0.8594, Loss: 0.1022
Epoch   1 Batch 1160/2154 - Train Accuracy: 0.8883, Validation Accuracy: 0.8558, Loss: 0.1023
Epoch   1 Batch 1165/2154 - Train Accuracy: 0.9109, Validation Accuracy: 0.8438, Loss: 0.1078
Epoch   1 Batch 1170/2154 - Train Accuracy: 0.9167, Validation Accuracy: 0.8395, Loss: 0.1032
Epoch   1 Batch 1175/2154 - Train Accuracy: 0.8977, Validation Accuracy: 0.8509, Loss: 0.1304
Epoch   1 Batch 1180/2154 - Train Accuracy: 0.8980, Validation Accuracy: 0.8636, Loss: 0.1028
Epoch   1 Batch 1185/2154 - Train Accuracy: 0.9013, Validation Accuracy: 0.8615, Loss: 0.1150
Epoch   1 Batch 1190/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8558, Loss: 0.1269
Epoch   1 Batch 1195/2154 - Train Accuracy: 0.8555, Validation Accuracy: 0.8672, Loss: 0.0965
Epoch   1 Batch 1200/2154 - Train Accuracy: 0.8816, Validation Accuracy: 0.8565, Loss: 0.1554
Epoch   1 Batch 1205/2154 - Train Accuracy: 0.8775, Validation Accuracy: 0.8466, Loss: 0.1246
Epoch   1 Batch 1210/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.8636, Loss: 0.0986
Epoch   1 Batch 1215/2154 - Train Accuracy: 0.9013, Validation Accuracy: 0.8807, Loss: 0.0968
Epoch   1 Batch 1220/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.8707, Loss: 0.0981
Epoch   1 Batch 1225/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.8359, Loss: 0.0785
Epoch   1 Batch 1230/2154 - Train Accuracy: 0.8802, Validation Accuracy: 0.8445, Loss: 0.1049
Epoch   1 Batch 1235/2154 - Train Accuracy: 0.8943, Validation Accuracy: 0.8565, Loss: 0.1009
Epoch   1 Batch 1240/2154 - Train Accuracy: 0.8964, Validation Accuracy: 0.8665, Loss: 0.0915
Epoch   1 Batch 1245/2154 - Train Accuracy: 0.8956, Validation Accuracy: 0.8423, Loss: 0.1241
Epoch   1 Batch 1250/2154 - Train Accuracy: 0.8961, Validation Accuracy: 0.8452, Loss: 0.0820
Epoch   1 Batch 1255/2154 - Train Accuracy: 0.8832, Validation Accuracy: 0.8558, Loss: 0.1006
Epoch   1 Batch 1260/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.8722, Loss: 0.1153
Epoch   1 Batch 1265/2154 - Train Accuracy: 0.8844, Validation Accuracy: 0.8736, Loss: 0.0871
Epoch   1 Batch 1270/2154 - Train Accuracy: 0.9018, Validation Accuracy: 0.8736, Loss: 0.0836
Epoch   1 Batch 1275/2154 - Train Accuracy: 0.8672, Validation Accuracy: 0.8594, Loss: 0.1170
Epoch   1 Batch 1280/2154 - Train Accuracy: 0.9046, Validation Accuracy: 0.8523, Loss: 0.1146
Epoch   1 Batch 1285/2154 - Train Accuracy: 0.8555, Validation Accuracy: 0.8750, Loss: 0.1097
Epoch   1 Batch 1290/2154 - Train Accuracy: 0.9336, Validation Accuracy: 0.8800, Loss: 0.0854
Epoch   1 Batch 1295/2154 - Train Accuracy: 0.9031, Validation Accuracy: 0.8601, Loss: 0.0989
Epoch   1 Batch 1300/2154 - Train Accuracy: 0.8797, Validation Accuracy: 0.8558, Loss: 0.1073
Epoch   1 Batch 1305/2154 - Train Accuracy: 0.9243, Validation Accuracy: 0.8771, Loss: 0.0868
Epoch   1 Batch 1310/2154 - Train Accuracy: 0.9313, Validation Accuracy: 0.8722, Loss: 0.0980
Epoch   1 Batch 1315/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.8736, Loss: 0.1002
Epoch   1 Batch 1320/2154 - Train Accuracy: 0.9405, Validation Accuracy: 0.8828, Loss: 0.0715
Epoch   1 Batch 1325/2154 - Train Accuracy: 0.9249, Validation Accuracy: 0.8714, Loss: 0.0723
Epoch   1 Batch 1330/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.8509, Loss: 0.0959
Epoch   1 Batch 1335/2154 - Train Accuracy: 0.8857, Validation Accuracy: 0.8722, Loss: 0.1276
Epoch   1 Batch 1340/2154 - Train Accuracy: 0.8875, Validation Accuracy: 0.8750, Loss: 0.0939
Epoch   1 Batch 1345/2154 - Train Accuracy: 0.9112, Validation Accuracy: 0.8807, Loss: 0.1096
Epoch   1 Batch 1350/2154 - Train Accuracy: 0.9172, Validation Accuracy: 0.8672, Loss: 0.0841
Epoch   1 Batch 1355/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.8771, Loss: 0.1099
Epoch   1 Batch 1360/2154 - Train Accuracy: 0.8964, Validation Accuracy: 0.8778, Loss: 0.0860
Epoch   1 Batch 1365/2154 - Train Accuracy: 0.8586, Validation Accuracy: 0.8736, Loss: 0.0928
Epoch   1 Batch 1370/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.8693, Loss: 0.0939
Epoch   1 Batch 1375/2154 - Train Accuracy: 0.9016, Validation Accuracy: 0.8885, Loss: 0.1167
Epoch   1 Batch 1380/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.8821, Loss: 0.1017
Epoch   1 Batch 1385/2154 - Train Accuracy: 0.9195, Validation Accuracy: 0.8722, Loss: 0.0856
Epoch   1 Batch 1390/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.8636, Loss: 0.0984
Epoch   1 Batch 1395/2154 - Train Accuracy: 0.9260, Validation Accuracy: 0.8722, Loss: 0.0952
Epoch   1 Batch 1400/2154 - Train Accuracy: 0.8832, Validation Accuracy: 0.8778, Loss: 0.0873
Epoch   1 Batch 1405/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.8722, Loss: 0.1153
Epoch   1 Batch 1410/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.8615, Loss: 0.1034
Epoch   1 Batch 1415/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.8736, Loss: 0.0823
Epoch   1 Batch 1420/2154 - Train Accuracy: 0.8684, Validation Accuracy: 0.8651, Loss: 0.1103
Epoch   1 Batch 1425/2154 - Train Accuracy: 0.9187, Validation Accuracy: 0.8643, Loss: 0.0696
Epoch   1 Batch 1430/2154 - Train Accuracy: 0.8780, Validation Accuracy: 0.8651, Loss: 0.1015
Epoch   1 Batch 1435/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.8594, Loss: 0.0886
Epoch   1 Batch 1440/2154 - Train Accuracy: 0.8836, Validation Accuracy: 0.8629, Loss: 0.1057
Epoch   1 Batch 1445/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.8871, Loss: 0.0866
Epoch   1 Batch 1450/2154 - Train Accuracy: 0.8988, Validation Accuracy: 0.8835, Loss: 0.0953
Epoch   1 Batch 1455/2154 - Train Accuracy: 0.8836, Validation Accuracy: 0.8842, Loss: 0.0955
Epoch   1 Batch 1460/2154 - Train Accuracy: 0.9016, Validation Accuracy: 0.8892, Loss: 0.1102
Epoch   1 Batch 1465/2154 - Train Accuracy: 0.9293, Validation Accuracy: 0.9112, Loss: 0.0773
Epoch   1 Batch 1470/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.8771, Loss: 0.0758
Epoch   1 Batch 1475/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.8892, Loss: 0.1177
Epoch   1 Batch 1480/2154 - Train Accuracy: 0.9336, Validation Accuracy: 0.8899, Loss: 0.0828
Epoch   1 Batch 1485/2154 - Train Accuracy: 0.8984, Validation Accuracy: 0.8743, Loss: 0.0867
Epoch   1 Batch 1490/2154 - Train Accuracy: 0.8797, Validation Accuracy: 0.8544, Loss: 0.0807
Epoch   1 Batch 1495/2154 - Train Accuracy: 0.8890, Validation Accuracy: 0.8580, Loss: 0.0719
Epoch   1 Batch 1500/2154 - Train Accuracy: 0.9038, Validation Accuracy: 0.8743, Loss: 0.1074
Epoch   1 Batch 1505/2154 - Train Accuracy: 0.8816, Validation Accuracy: 0.8743, Loss: 0.0879
Epoch   1 Batch 1510/2154 - Train Accuracy: 0.9021, Validation Accuracy: 0.8729, Loss: 0.1035
Epoch   1 Batch 1515/2154 - Train Accuracy: 0.9416, Validation Accuracy: 0.8672, Loss: 0.0744
Epoch   1 Batch 1520/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.8800, Loss: 0.0715
Epoch   1 Batch 1525/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.8991, Loss: 0.0808
Epoch   1 Batch 1530/2154 - Train Accuracy: 0.9054, Validation Accuracy: 0.8516, Loss: 0.0966
Epoch   1 Batch 1535/2154 - Train Accuracy: 0.9133, Validation Accuracy: 0.8729, Loss: 0.0768
Epoch   1 Batch 1540/2154 - Train Accuracy: 0.9273, Validation Accuracy: 0.8849, Loss: 0.0899
Epoch   1 Batch 1545/2154 - Train Accuracy: 0.9010, Validation Accuracy: 0.8750, Loss: 0.0748
Epoch   1 Batch 1550/2154 - Train Accuracy: 0.9125, Validation Accuracy: 0.8722, Loss: 0.0975
Epoch   1 Batch 1555/2154 - Train Accuracy: 0.8953, Validation Accuracy: 0.8636, Loss: 0.0893
Epoch   1 Batch 1560/2154 - Train Accuracy: 0.8930, Validation Accuracy: 0.8956, Loss: 0.0871
Epoch   1 Batch 1565/2154 - Train Accuracy: 0.8802, Validation Accuracy: 0.8920, Loss: 0.0866
Epoch   1 Batch 1570/2154 - Train Accuracy: 0.8672, Validation Accuracy: 0.8864, Loss: 0.0936
Epoch   1 Batch 1575/2154 - Train Accuracy: 0.9293, Validation Accuracy: 0.8849, Loss: 0.0683
Epoch   1 Batch 1580/2154 - Train Accuracy: 0.9030, Validation Accuracy: 0.8963, Loss: 0.0977
Epoch   1 Batch 1585/2154 - Train Accuracy: 0.8984, Validation Accuracy: 0.8857, Loss: 0.0931
Epoch   1 Batch 1590/2154 - Train Accuracy: 0.8984, Validation Accuracy: 0.8835, Loss: 0.0880
Epoch   1 Batch 1595/2154 - Train Accuracy: 0.9422, Validation Accuracy: 0.8736, Loss: 0.0763
Epoch   1 Batch 1600/2154 - Train Accuracy: 0.8516, Validation Accuracy: 0.8501, Loss: 0.0942
Epoch   1 Batch 1605/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.8487, Loss: 0.1052
Epoch   1 Batch 1610/2154 - Train Accuracy: 0.9656, Validation Accuracy: 0.8757, Loss: 0.0599
Epoch   1 Batch 1615/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.8864, Loss: 0.0601
Epoch   1 Batch 1620/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.8871, Loss: 0.1145
Epoch   1 Batch 1625/2154 - Train Accuracy: 0.8719, Validation Accuracy: 0.8857, Loss: 0.0917
Epoch   1 Batch 1630/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.8757, Loss: 0.0785
Epoch   1 Batch 1635/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.8743, Loss: 0.1090
Epoch   1 Batch 1640/2154 - Train Accuracy: 0.9172, Validation Accuracy: 0.8949, Loss: 0.0578
Epoch   1 Batch 1645/2154 - Train Accuracy: 0.8956, Validation Accuracy: 0.8800, Loss: 0.0766
Epoch   1 Batch 1650/2154 - Train Accuracy: 0.9092, Validation Accuracy: 0.8807, Loss: 0.0814
Epoch   1 Batch 1655/2154 - Train Accuracy: 0.9243, Validation Accuracy: 0.8857, Loss: 0.0863
Epoch   1 Batch 1660/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.8878, Loss: 0.1145
Epoch   1 Batch 1665/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8793, Loss: 0.0771
Epoch   1 Batch 1670/2154 - Train Accuracy: 0.9492, Validation Accuracy: 0.8906, Loss: 0.0850
Epoch   1 Batch 1675/2154 - Train Accuracy: 0.8766, Validation Accuracy: 0.9006, Loss: 0.1066
Epoch   1 Batch 1680/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.8928, Loss: 0.0661
Epoch   1 Batch 1685/2154 - Train Accuracy: 0.9079, Validation Accuracy: 0.8871, Loss: 0.0760
Epoch   1 Batch 1690/2154 - Train Accuracy: 0.9204, Validation Accuracy: 0.9098, Loss: 0.0771
Epoch   1 Batch 1695/2154 - Train Accuracy: 0.9350, Validation Accuracy: 0.8984, Loss: 0.0934
Epoch   1 Batch 1700/2154 - Train Accuracy: 0.9070, Validation Accuracy: 0.9134, Loss: 0.0811
Epoch   1 Batch 1705/2154 - Train Accuracy: 0.9071, Validation Accuracy: 0.9134, Loss: 0.0975
Epoch   1 Batch 1710/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.9176, Loss: 0.0920
Epoch   1 Batch 1715/2154 - Train Accuracy: 0.9342, Validation Accuracy: 0.8807, Loss: 0.0857
Epoch   1 Batch 1720/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.8899, Loss: 0.0894
Epoch   1 Batch 1725/2154 - Train Accuracy: 0.9120, Validation Accuracy: 0.8928, Loss: 0.0811
Epoch   1 Batch 1730/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.8778, Loss: 0.0757
Epoch   1 Batch 1735/2154 - Train Accuracy: 0.9046, Validation Accuracy: 0.8871, Loss: 0.1013
Epoch   1 Batch 1740/2154 - Train Accuracy: 0.9054, Validation Accuracy: 0.8778, Loss: 0.0834
Epoch   1 Batch 1745/2154 - Train Accuracy: 0.9235, Validation Accuracy: 0.8849, Loss: 0.0784
Epoch   1 Batch 1750/2154 - Train Accuracy: 0.9078, Validation Accuracy: 0.8835, Loss: 0.0795
Epoch   1 Batch 1755/2154 - Train Accuracy: 0.8609, Validation Accuracy: 0.8821, Loss: 0.0739
Epoch   1 Batch 1760/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.8821, Loss: 0.0788
Epoch   1 Batch 1765/2154 - Train Accuracy: 0.8956, Validation Accuracy: 0.8828, Loss: 0.0948
Epoch   1 Batch 1770/2154 - Train Accuracy: 0.9078, Validation Accuracy: 0.8977, Loss: 0.0815
Epoch   1 Batch 1775/2154 - Train Accuracy: 0.9008, Validation Accuracy: 0.9048, Loss: 0.0799
Epoch   1 Batch 1780/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.9105, Loss: 0.0758
Epoch   1 Batch 1785/2154 - Train Accuracy: 0.9581, Validation Accuracy: 0.8984, Loss: 0.0647
Epoch   1 Batch 1790/2154 - Train Accuracy: 0.9159, Validation Accuracy: 0.9020, Loss: 0.0880
Epoch   1 Batch 1795/2154 - Train Accuracy: 0.8930, Validation Accuracy: 0.9027, Loss: 0.0635
Epoch   1 Batch 1800/2154 - Train Accuracy: 0.9117, Validation Accuracy: 0.8913, Loss: 0.0742
Epoch   1 Batch 1805/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.8828, Loss: 0.0893
Epoch   1 Batch 1810/2154 - Train Accuracy: 0.8789, Validation Accuracy: 0.8679, Loss: 0.0802
Epoch   1 Batch 1815/2154 - Train Accuracy: 0.9484, Validation Accuracy: 0.8771, Loss: 0.0808
Epoch   1 Batch 1820/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.8928, Loss: 0.0663
Epoch   1 Batch 1825/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.8963, Loss: 0.0856
Epoch   1 Batch 1830/2154 - Train Accuracy: 0.9141, Validation Accuracy: 0.8906, Loss: 0.0982
Epoch   1 Batch 1835/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.8828, Loss: 0.0800
Epoch   1 Batch 1840/2154 - Train Accuracy: 0.9167, Validation Accuracy: 0.8693, Loss: 0.0676
Epoch   1 Batch 1845/2154 - Train Accuracy: 0.9031, Validation Accuracy: 0.8736, Loss: 0.0906
Epoch   1 Batch 1850/2154 - Train Accuracy: 0.9309, Validation Accuracy: 0.8807, Loss: 0.1064
Epoch   1 Batch 1855/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.8722, Loss: 0.0834
Epoch   1 Batch 1860/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.8608, Loss: 0.0835
Epoch   1 Batch 1865/2154 - Train Accuracy: 0.9424, Validation Accuracy: 0.8970, Loss: 0.0683
Epoch   1 Batch 1870/2154 - Train Accuracy: 0.9016, Validation Accuracy: 0.8970, Loss: 0.0710
Epoch   1 Batch 1875/2154 - Train Accuracy: 0.9104, Validation Accuracy: 0.8736, Loss: 0.0945
Epoch   1 Batch 1880/2154 - Train Accuracy: 0.9078, Validation Accuracy: 0.8793, Loss: 0.0839
Epoch   1 Batch 1885/2154 - Train Accuracy: 0.8972, Validation Accuracy: 0.8821, Loss: 0.0966
Epoch   1 Batch 1890/2154 - Train Accuracy: 0.9568, Validation Accuracy: 0.8821, Loss: 0.0700
Epoch   1 Batch 1895/2154 - Train Accuracy: 0.9235, Validation Accuracy: 0.8956, Loss: 0.0587
Epoch   1 Batch 1900/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.8942, Loss: 0.0680
Epoch   1 Batch 1905/2154 - Train Accuracy: 0.9344, Validation Accuracy: 0.8849, Loss: 0.0620
Epoch   1 Batch 1910/2154 - Train Accuracy: 0.9087, Validation Accuracy: 0.8608, Loss: 0.0657
Epoch   1 Batch 1915/2154 - Train Accuracy: 0.9594, Validation Accuracy: 0.8565, Loss: 0.0489
Epoch   1 Batch 1920/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.8665, Loss: 0.0617
Epoch   1 Batch 1925/2154 - Train Accuracy: 0.9276, Validation Accuracy: 0.8828, Loss: 0.0672
Epoch   1 Batch 1930/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.8643, Loss: 0.0667
Epoch   1 Batch 1935/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8629, Loss: 0.0858
Epoch   1 Batch 1940/2154 - Train Accuracy: 0.8981, Validation Accuracy: 0.8871, Loss: 0.0681
Epoch   1 Batch 1945/2154 - Train Accuracy: 0.9086, Validation Accuracy: 0.8892, Loss: 0.0654
Epoch   1 Batch 1950/2154 - Train Accuracy: 0.9313, Validation Accuracy: 0.8821, Loss: 0.0732
Epoch   1 Batch 1955/2154 - Train Accuracy: 0.9703, Validation Accuracy: 0.8707, Loss: 0.0571
Epoch   1 Batch 1960/2154 - Train Accuracy: 0.9112, Validation Accuracy: 0.8906, Loss: 0.0844
Epoch   1 Batch 1965/2154 - Train Accuracy: 0.8951, Validation Accuracy: 0.8807, Loss: 0.0894
Epoch   1 Batch 1970/2154 - Train Accuracy: 0.8945, Validation Accuracy: 0.8878, Loss: 0.0826
Epoch   1 Batch 1975/2154 - Train Accuracy: 0.8336, Validation Accuracy: 0.8928, Loss: 0.0743
Epoch   1 Batch 1980/2154 - Train Accuracy: 0.9117, Validation Accuracy: 0.9077, Loss: 0.0803
Epoch   1 Batch 1985/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9041, Loss: 0.0775
Epoch   1 Batch 1990/2154 - Train Accuracy: 0.9268, Validation Accuracy: 0.8977, Loss: 0.0631
Epoch   1 Batch 1995/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.9077, Loss: 0.0985
Epoch   1 Batch 2000/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.8999, Loss: 0.0854
Epoch   1 Batch 2005/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.8956, Loss: 0.0679
Epoch   1 Batch 2010/2154 - Train Accuracy: 0.9194, Validation Accuracy: 0.8913, Loss: 0.1185
Epoch   1 Batch 2015/2154 - Train Accuracy: 0.9412, Validation Accuracy: 0.8729, Loss: 0.0677
Epoch   1 Batch 2020/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.9048, Loss: 0.0603
Epoch   1 Batch 2025/2154 - Train Accuracy: 0.9039, Validation Accuracy: 0.9070, Loss: 0.0638
Epoch   1 Batch 2030/2154 - Train Accuracy: 0.8930, Validation Accuracy: 0.8736, Loss: 0.0720
Epoch   1 Batch 2035/2154 - Train Accuracy: 0.9013, Validation Accuracy: 0.8665, Loss: 0.0614
Epoch   1 Batch 2040/2154 - Train Accuracy: 0.8972, Validation Accuracy: 0.8722, Loss: 0.0720
Epoch   1 Batch 2045/2154 - Train Accuracy: 0.9427, Validation Accuracy: 0.8864, Loss: 0.0473
Epoch   1 Batch 2050/2154 - Train Accuracy: 0.8668, Validation Accuracy: 0.8849, Loss: 0.1097
Epoch   1 Batch 2055/2154 - Train Accuracy: 0.8166, Validation Accuracy: 0.8878, Loss: 0.0962
Epoch   1 Batch 2060/2154 - Train Accuracy: 0.8873, Validation Accuracy: 0.8999, Loss: 0.0693
Epoch   1 Batch 2065/2154 - Train Accuracy: 0.9406, Validation Accuracy: 0.8899, Loss: 0.0739
Epoch   1 Batch 2070/2154 - Train Accuracy: 0.8812, Validation Accuracy: 0.8807, Loss: 0.0804
Epoch   1 Batch 2075/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.8700, Loss: 0.0725
Epoch   1 Batch 2080/2154 - Train Accuracy: 0.8966, Validation Accuracy: 0.8608, Loss: 0.0658
Epoch   1 Batch 2085/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.8722, Loss: 0.0713
Epoch   1 Batch 2090/2154 - Train Accuracy: 0.9000, Validation Accuracy: 0.8686, Loss: 0.0813
Epoch   1 Batch 2095/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.8494, Loss: 0.0685
Epoch   1 Batch 2100/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.8864, Loss: 0.0612
Epoch   1 Batch 2105/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.8878, Loss: 0.0747
Epoch   1 Batch 2110/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.8892, Loss: 0.0691
Epoch   1 Batch 2115/2154 - Train Accuracy: 0.8734, Validation Accuracy: 0.8849, Loss: 0.0745
Epoch   1 Batch 2120/2154 - Train Accuracy: 0.8561, Validation Accuracy: 0.8956, Loss: 0.0742
Epoch   1 Batch 2125/2154 - Train Accuracy: 0.8930, Validation Accuracy: 0.8956, Loss: 0.0770
Epoch   1 Batch 2130/2154 - Train Accuracy: 0.9564, Validation Accuracy: 0.8729, Loss: 0.0666
Epoch   1 Batch 2135/2154 - Train Accuracy: 0.9125, Validation Accuracy: 0.8999, Loss: 0.0855
Epoch   1 Batch 2140/2154 - Train Accuracy: 0.9226, Validation Accuracy: 0.8800, Loss: 0.0498
Epoch   1 Batch 2145/2154 - Train Accuracy: 0.9342, Validation Accuracy: 0.8849, Loss: 0.0742
Epoch   1 Batch 2150/2154 - Train Accuracy: 0.8936, Validation Accuracy: 0.8722, Loss: 0.0934
Epoch   2 Batch    5/2154 - Train Accuracy: 0.9145, Validation Accuracy: 0.8807, Loss: 0.0676
Epoch   2 Batch   10/2154 - Train Accuracy: 0.9195, Validation Accuracy: 0.8764, Loss: 0.0670
Epoch   2 Batch   15/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.8885, Loss: 0.0710
Epoch   2 Batch   20/2154 - Train Accuracy: 0.8906, Validation Accuracy: 0.8615, Loss: 0.0618
Epoch   2 Batch   25/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.8501, Loss: 0.0739
Epoch   2 Batch   30/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.8857, Loss: 0.0611
Epoch   2 Batch   35/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.8999, Loss: 0.0588
Epoch   2 Batch   40/2154 - Train Accuracy: 0.9153, Validation Accuracy: 0.9176, Loss: 0.0734
Epoch   2 Batch   45/2154 - Train Accuracy: 0.9169, Validation Accuracy: 0.8786, Loss: 0.0687
Epoch   2 Batch   50/2154 - Train Accuracy: 0.9038, Validation Accuracy: 0.8714, Loss: 0.0797
Epoch   2 Batch   55/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9062, Loss: 0.0541
Epoch   2 Batch   60/2154 - Train Accuracy: 0.9336, Validation Accuracy: 0.9055, Loss: 0.0576
Epoch   2 Batch   65/2154 - Train Accuracy: 0.9420, Validation Accuracy: 0.8864, Loss: 0.0657
Epoch   2 Batch   70/2154 - Train Accuracy: 0.9046, Validation Accuracy: 0.8949, Loss: 0.0866
Epoch   2 Batch   75/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.8821, Loss: 0.0809
Epoch   2 Batch   80/2154 - Train Accuracy: 0.8945, Validation Accuracy: 0.8977, Loss: 0.0794
Epoch   2 Batch   85/2154 - Train Accuracy: 0.8984, Validation Accuracy: 0.8970, Loss: 0.0897
Epoch   2 Batch   90/2154 - Train Accuracy: 0.8988, Validation Accuracy: 0.8963, Loss: 0.0714
Epoch   2 Batch   95/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9077, Loss: 0.0605
Epoch   2 Batch  100/2154 - Train Accuracy: 0.8884, Validation Accuracy: 0.8864, Loss: 0.0889
Epoch   2 Batch  105/2154 - Train Accuracy: 0.9289, Validation Accuracy: 0.8757, Loss: 0.0738
Epoch   2 Batch  110/2154 - Train Accuracy: 0.8961, Validation Accuracy: 0.8906, Loss: 0.0987
Epoch   2 Batch  115/2154 - Train Accuracy: 0.8977, Validation Accuracy: 0.8857, Loss: 0.0751
Epoch   2 Batch  120/2154 - Train Accuracy: 0.9071, Validation Accuracy: 0.8963, Loss: 0.0619
Epoch   2 Batch  125/2154 - Train Accuracy: 0.9054, Validation Accuracy: 0.8935, Loss: 0.0615
Epoch   2 Batch  130/2154 - Train Accuracy: 0.9195, Validation Accuracy: 0.9041, Loss: 0.0585
Epoch   2 Batch  135/2154 - Train Accuracy: 0.9462, Validation Accuracy: 0.9148, Loss: 0.0730
Epoch   2 Batch  140/2154 - Train Accuracy: 0.9117, Validation Accuracy: 0.9062, Loss: 0.0841
Epoch   2 Batch  145/2154 - Train Accuracy: 0.9250, Validation Accuracy: 0.9062, Loss: 0.0736
Epoch   2 Batch  150/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.9041, Loss: 0.0500
Epoch   2 Batch  155/2154 - Train Accuracy: 0.9153, Validation Accuracy: 0.8906, Loss: 0.0718
Epoch   2 Batch  160/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.9013, Loss: 0.0659
Epoch   2 Batch  165/2154 - Train Accuracy: 0.9254, Validation Accuracy: 0.8984, Loss: 0.0619
Epoch   2 Batch  170/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.8899, Loss: 0.0518
Epoch   2 Batch  175/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9190, Loss: 0.0758
Epoch   2 Batch  180/2154 - Train Accuracy: 0.9172, Validation Accuracy: 0.8849, Loss: 0.0669
Epoch   2 Batch  185/2154 - Train Accuracy: 0.9268, Validation Accuracy: 0.8842, Loss: 0.0688
Epoch   2 Batch  190/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.8977, Loss: 0.0535
Epoch   2 Batch  195/2154 - Train Accuracy: 0.9013, Validation Accuracy: 0.9048, Loss: 0.0686
Epoch   2 Batch  200/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8835, Loss: 0.0505
Epoch   2 Batch  205/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9091, Loss: 0.0560
Epoch   2 Batch  210/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.8778, Loss: 0.0528
Epoch   2 Batch  215/2154 - Train Accuracy: 0.9025, Validation Accuracy: 0.8807, Loss: 0.0648
Epoch   2 Batch  220/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9034, Loss: 0.0571
Epoch   2 Batch  225/2154 - Train Accuracy: 0.9281, Validation Accuracy: 0.8786, Loss: 0.0625
Epoch   2 Batch  230/2154 - Train Accuracy: 0.9420, Validation Accuracy: 0.8871, Loss: 0.0536
Epoch   2 Batch  235/2154 - Train Accuracy: 0.8812, Validation Accuracy: 0.8842, Loss: 0.0681
Epoch   2 Batch  240/2154 - Train Accuracy: 0.9125, Validation Accuracy: 0.8828, Loss: 0.0709
Epoch   2 Batch  245/2154 - Train Accuracy: 0.9638, Validation Accuracy: 0.8757, Loss: 0.0568
Epoch   2 Batch  250/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.8807, Loss: 0.0652
Epoch   2 Batch  255/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.8913, Loss: 0.0581
Epoch   2 Batch  260/2154 - Train Accuracy: 0.9250, Validation Accuracy: 0.8729, Loss: 0.0706
Epoch   2 Batch  265/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8686, Loss: 0.0770
Epoch   2 Batch  270/2154 - Train Accuracy: 0.9509, Validation Accuracy: 0.8778, Loss: 0.0647
Epoch   2 Batch  275/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.8849, Loss: 0.0416
Epoch   2 Batch  280/2154 - Train Accuracy: 0.8766, Validation Accuracy: 0.8956, Loss: 0.0834
Epoch   2 Batch  285/2154 - Train Accuracy: 0.8988, Validation Accuracy: 0.8871, Loss: 0.0714
Epoch   2 Batch  290/2154 - Train Accuracy: 0.9252, Validation Accuracy: 0.8736, Loss: 0.0862
Epoch   2 Batch  295/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.8828, Loss: 0.0535
Epoch   2 Batch  300/2154 - Train Accuracy: 0.9202, Validation Accuracy: 0.8835, Loss: 0.0738
Epoch   2 Batch  305/2154 - Train Accuracy: 0.9070, Validation Accuracy: 0.8800, Loss: 0.0831
Epoch   2 Batch  310/2154 - Train Accuracy: 0.9145, Validation Accuracy: 0.8700, Loss: 0.0647
Epoch   2 Batch  315/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9020, Loss: 0.0505
Epoch   2 Batch  320/2154 - Train Accuracy: 0.9273, Validation Accuracy: 0.9091, Loss: 0.0417
Epoch   2 Batch  325/2154 - Train Accuracy: 0.9070, Validation Accuracy: 0.8885, Loss: 0.0723
Epoch   2 Batch  330/2154 - Train Accuracy: 0.9408, Validation Accuracy: 0.8899, Loss: 0.0510
Epoch   2 Batch  335/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8857, Loss: 0.0451
Epoch   2 Batch  340/2154 - Train Accuracy: 0.9360, Validation Accuracy: 0.8786, Loss: 0.0524
Epoch   2 Batch  345/2154 - Train Accuracy: 0.8988, Validation Accuracy: 0.9041, Loss: 0.0683
Epoch   2 Batch  350/2154 - Train Accuracy: 0.9180, Validation Accuracy: 0.8814, Loss: 0.0644
Epoch   2 Batch  355/2154 - Train Accuracy: 0.9013, Validation Accuracy: 0.8778, Loss: 0.0713
Epoch   2 Batch  360/2154 - Train Accuracy: 0.9515, Validation Accuracy: 0.8849, Loss: 0.0597
Epoch   2 Batch  365/2154 - Train Accuracy: 0.9182, Validation Accuracy: 0.8864, Loss: 0.0703
Epoch   2 Batch  370/2154 - Train Accuracy: 0.9117, Validation Accuracy: 0.8871, Loss: 0.0620
Epoch   2 Batch  375/2154 - Train Accuracy: 0.9260, Validation Accuracy: 0.8885, Loss: 0.0554
Epoch   2 Batch  380/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.8970, Loss: 0.0554
Epoch   2 Batch  385/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.9034, Loss: 0.0612
Epoch   2 Batch  390/2154 - Train Accuracy: 0.9598, Validation Accuracy: 0.9027, Loss: 0.0459
Epoch   2 Batch  395/2154 - Train Accuracy: 0.9336, Validation Accuracy: 0.9077, Loss: 0.0593
Epoch   2 Batch  400/2154 - Train Accuracy: 0.9305, Validation Accuracy: 0.9077, Loss: 0.0556
Epoch   2 Batch  405/2154 - Train Accuracy: 0.9430, Validation Accuracy: 0.9261, Loss: 0.0643
Epoch   2 Batch  410/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.9233, Loss: 0.0743
Epoch   2 Batch  415/2154 - Train Accuracy: 0.9187, Validation Accuracy: 0.9155, Loss: 0.0661
Epoch   2 Batch  420/2154 - Train Accuracy: 0.9494, Validation Accuracy: 0.8942, Loss: 0.0469
Epoch   2 Batch  425/2154 - Train Accuracy: 0.9494, Validation Accuracy: 0.8928, Loss: 0.0519
Epoch   2 Batch  430/2154 - Train Accuracy: 0.8742, Validation Accuracy: 0.8743, Loss: 0.0685
Epoch   2 Batch  435/2154 - Train Accuracy: 0.9477, Validation Accuracy: 0.8857, Loss: 0.0620
Epoch   2 Batch  440/2154 - Train Accuracy: 0.9202, Validation Accuracy: 0.8913, Loss: 0.0553
Epoch   2 Batch  445/2154 - Train Accuracy: 0.8859, Validation Accuracy: 0.8857, Loss: 0.0610
Epoch   2 Batch  450/2154 - Train Accuracy: 0.9492, Validation Accuracy: 0.8736, Loss: 0.0651
Epoch   2 Batch  455/2154 - Train Accuracy: 0.8781, Validation Accuracy: 0.8693, Loss: 0.0901
Epoch   2 Batch  460/2154 - Train Accuracy: 0.8773, Validation Accuracy: 0.8906, Loss: 0.0565
Epoch   2 Batch  465/2154 - Train Accuracy: 0.8923, Validation Accuracy: 0.8899, Loss: 0.0531
Epoch   2 Batch  470/2154 - Train Accuracy: 0.8785, Validation Accuracy: 0.8807, Loss: 0.0774
Epoch   2 Batch  475/2154 - Train Accuracy: 0.9400, Validation Accuracy: 0.8913, Loss: 0.0616
Epoch   2 Batch  480/2154 - Train Accuracy: 0.9472, Validation Accuracy: 0.8913, Loss: 0.0437
Epoch   2 Batch  485/2154 - Train Accuracy: 0.9727, Validation Accuracy: 0.8906, Loss: 0.0443
Epoch   2 Batch  490/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.8942, Loss: 0.0616
Epoch   2 Batch  495/2154 - Train Accuracy: 0.9464, Validation Accuracy: 0.8778, Loss: 0.0584
Epoch   2 Batch  500/2154 - Train Accuracy: 0.9762, Validation Accuracy: 0.8743, Loss: 0.0599
Epoch   2 Batch  505/2154 - Train Accuracy: 0.8627, Validation Accuracy: 0.8814, Loss: 0.0808
Epoch   2 Batch  510/2154 - Train Accuracy: 0.8703, Validation Accuracy: 0.8835, Loss: 0.0615
Epoch   2 Batch  515/2154 - Train Accuracy: 0.9167, Validation Accuracy: 0.8899, Loss: 0.0547
Epoch   2 Batch  520/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.9077, Loss: 0.0542
Epoch   2 Batch  525/2154 - Train Accuracy: 0.9727, Validation Accuracy: 0.9070, Loss: 0.0506
Epoch   2 Batch  530/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9077, Loss: 0.0540
Epoch   2 Batch  535/2154 - Train Accuracy: 0.9432, Validation Accuracy: 0.9084, Loss: 0.0665
Epoch   2 Batch  540/2154 - Train Accuracy: 0.9178, Validation Accuracy: 0.8984, Loss: 0.0639
Epoch   2 Batch  545/2154 - Train Accuracy: 0.9186, Validation Accuracy: 0.9041, Loss: 0.0803
Epoch   2 Batch  550/2154 - Train Accuracy: 0.9107, Validation Accuracy: 0.8849, Loss: 0.0690
Epoch   2 Batch  555/2154 - Train Accuracy: 0.9133, Validation Accuracy: 0.8892, Loss: 0.0712
Epoch   2 Batch  560/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.9055, Loss: 0.0705
Epoch   2 Batch  565/2154 - Train Accuracy: 0.9250, Validation Accuracy: 0.8920, Loss: 0.0586
Epoch   2 Batch  570/2154 - Train Accuracy: 0.9141, Validation Accuracy: 0.8857, Loss: 0.0701
Epoch   2 Batch  575/2154 - Train Accuracy: 0.9055, Validation Accuracy: 0.8793, Loss: 0.0553
Epoch   2 Batch  580/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8814, Loss: 0.0619
Epoch   2 Batch  585/2154 - Train Accuracy: 0.9435, Validation Accuracy: 0.8935, Loss: 0.0692
Epoch   2 Batch  590/2154 - Train Accuracy: 0.9382, Validation Accuracy: 0.8949, Loss: 0.0661
Epoch   2 Batch  595/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8679, Loss: 0.0640
Epoch   2 Batch  600/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.8906, Loss: 0.0451
Epoch   2 Batch  605/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.8906, Loss: 0.0592
Epoch   2 Batch  610/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9006, Loss: 0.0518
Epoch   2 Batch  615/2154 - Train Accuracy: 0.9167, Validation Accuracy: 0.8800, Loss: 0.0522
Epoch   2 Batch  620/2154 - Train Accuracy: 0.9278, Validation Accuracy: 0.8828, Loss: 0.0553
Epoch   2 Batch  625/2154 - Train Accuracy: 0.9235, Validation Accuracy: 0.8956, Loss: 0.0718
Epoch   2 Batch  630/2154 - Train Accuracy: 0.9498, Validation Accuracy: 0.8949, Loss: 0.0583
Epoch   2 Batch  635/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8800, Loss: 0.0758
Epoch   2 Batch  640/2154 - Train Accuracy: 0.9180, Validation Accuracy: 0.8807, Loss: 0.0549
Epoch   2 Batch  645/2154 - Train Accuracy: 0.9263, Validation Accuracy: 0.8714, Loss: 0.0550
Epoch   2 Batch  650/2154 - Train Accuracy: 0.8867, Validation Accuracy: 0.8714, Loss: 0.0568
Epoch   2 Batch  655/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.8700, Loss: 0.0674
Epoch   2 Batch  660/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8693, Loss: 0.0640
Epoch   2 Batch  665/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.8693, Loss: 0.0438
Epoch   2 Batch  670/2154 - Train Accuracy: 0.9605, Validation Accuracy: 0.8509, Loss: 0.0479
Epoch   2 Batch  675/2154 - Train Accuracy: 0.9117, Validation Accuracy: 0.8693, Loss: 0.0613
Epoch   2 Batch  680/2154 - Train Accuracy: 0.9276, Validation Accuracy: 0.8615, Loss: 0.0498
Epoch   2 Batch  685/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.8849, Loss: 0.0487
Epoch   2 Batch  690/2154 - Train Accuracy: 0.9070, Validation Accuracy: 0.9084, Loss: 0.0683
Epoch   2 Batch  695/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.8892, Loss: 0.0520
Epoch   2 Batch  700/2154 - Train Accuracy: 0.9038, Validation Accuracy: 0.8771, Loss: 0.0478
Epoch   2 Batch  705/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.9091, Loss: 0.0483
Epoch   2 Batch  710/2154 - Train Accuracy: 0.9133, Validation Accuracy: 0.9105, Loss: 0.0494
Epoch   2 Batch  715/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9162, Loss: 0.0601
Epoch   2 Batch  720/2154 - Train Accuracy: 0.9087, Validation Accuracy: 0.8729, Loss: 0.0653
Epoch   2 Batch  725/2154 - Train Accuracy: 0.9494, Validation Accuracy: 0.8970, Loss: 0.0554
Epoch   2 Batch  730/2154 - Train Accuracy: 0.9079, Validation Accuracy: 0.8977, Loss: 0.0898
Epoch   2 Batch  735/2154 - Train Accuracy: 0.9382, Validation Accuracy: 0.8786, Loss: 0.0459
Epoch   2 Batch  740/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.9006, Loss: 0.0585
Epoch   2 Batch  745/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.9112, Loss: 0.0483
Epoch   2 Batch  750/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.8999, Loss: 0.0642
Epoch   2 Batch  755/2154 - Train Accuracy: 0.9005, Validation Accuracy: 0.8999, Loss: 0.0656
Epoch   2 Batch  760/2154 - Train Accuracy: 0.9422, Validation Accuracy: 0.9098, Loss: 0.0706
Epoch   2 Batch  765/2154 - Train Accuracy: 0.9487, Validation Accuracy: 0.9126, Loss: 0.0850
Epoch   2 Batch  770/2154 - Train Accuracy: 0.9297, Validation Accuracy: 0.9119, Loss: 0.0457
Epoch   2 Batch  775/2154 - Train Accuracy: 0.9285, Validation Accuracy: 0.9077, Loss: 0.0646
Epoch   2 Batch  780/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9190, Loss: 0.0420
Epoch   2 Batch  785/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9197, Loss: 0.0491
Epoch   2 Batch  790/2154 - Train Accuracy: 0.9016, Validation Accuracy: 0.8999, Loss: 0.0449
Epoch   2 Batch  795/2154 - Train Accuracy: 0.9397, Validation Accuracy: 0.9261, Loss: 0.0557
Epoch   2 Batch  800/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.9070, Loss: 0.0510
Epoch   2 Batch  805/2154 - Train Accuracy: 0.9317, Validation Accuracy: 0.8857, Loss: 0.0466
Epoch   2 Batch  810/2154 - Train Accuracy: 0.9576, Validation Accuracy: 0.8835, Loss: 0.0466
Epoch   2 Batch  815/2154 - Train Accuracy: 0.8969, Validation Accuracy: 0.8835, Loss: 0.0618
Epoch   2 Batch  820/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.8970, Loss: 0.0633
Epoch   2 Batch  825/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.8878, Loss: 0.0482
Epoch   2 Batch  830/2154 - Train Accuracy: 0.8953, Validation Accuracy: 0.8935, Loss: 0.0690
Epoch   2 Batch  835/2154 - Train Accuracy: 0.8824, Validation Accuracy: 0.8757, Loss: 0.0873
Epoch   2 Batch  840/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.8878, Loss: 0.0646
Epoch   2 Batch  845/2154 - Train Accuracy: 0.8915, Validation Accuracy: 0.8864, Loss: 0.0671
Epoch   2 Batch  850/2154 - Train Accuracy: 0.9336, Validation Accuracy: 0.8835, Loss: 0.0407
Epoch   2 Batch  855/2154 - Train Accuracy: 0.8914, Validation Accuracy: 0.9055, Loss: 0.0714
Epoch   2 Batch  860/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.8821, Loss: 0.0600
Epoch   2 Batch  865/2154 - Train Accuracy: 0.9172, Validation Accuracy: 0.8700, Loss: 0.0575
Epoch   2 Batch  870/2154 - Train Accuracy: 0.9781, Validation Accuracy: 0.8544, Loss: 0.0449
Epoch   2 Batch  875/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.8729, Loss: 0.0500
Epoch   2 Batch  880/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.8835, Loss: 0.0757
Epoch   2 Batch  885/2154 - Train Accuracy: 0.9182, Validation Accuracy: 0.8743, Loss: 0.0792
Epoch   2 Batch  890/2154 - Train Accuracy: 0.9112, Validation Accuracy: 0.8658, Loss: 0.0650
Epoch   2 Batch  895/2154 - Train Accuracy: 0.9120, Validation Accuracy: 0.8757, Loss: 0.0602
Epoch   2 Batch  900/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.8807, Loss: 0.0610
Epoch   2 Batch  905/2154 - Train Accuracy: 0.9555, Validation Accuracy: 0.8864, Loss: 0.0488
Epoch   2 Batch  910/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.8778, Loss: 0.0552
Epoch   2 Batch  915/2154 - Train Accuracy: 0.9598, Validation Accuracy: 0.8999, Loss: 0.0436
Epoch   2 Batch  920/2154 - Train Accuracy: 0.9598, Validation Accuracy: 0.9141, Loss: 0.0360
Epoch   2 Batch  925/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.9055, Loss: 0.0476
Epoch   2 Batch  930/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9006, Loss: 0.0638
Epoch   2 Batch  935/2154 - Train Accuracy: 0.8956, Validation Accuracy: 0.8999, Loss: 0.0588
Epoch   2 Batch  940/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8928, Loss: 0.0642
Epoch   2 Batch  945/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9077, Loss: 0.0479
Epoch   2 Batch  950/2154 - Train Accuracy: 0.9079, Validation Accuracy: 0.9077, Loss: 0.0569
Epoch   2 Batch  955/2154 - Train Accuracy: 0.9401, Validation Accuracy: 0.8842, Loss: 0.0568
Epoch   2 Batch  960/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.8736, Loss: 0.0610
Epoch   2 Batch  965/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.8849, Loss: 0.0669
Epoch   2 Batch  970/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.9013, Loss: 0.0724
Epoch   2 Batch  975/2154 - Train Accuracy: 0.9054, Validation Accuracy: 0.8920, Loss: 0.0627
Epoch   2 Batch  980/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.8757, Loss: 0.0443
Epoch   2 Batch  985/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.8892, Loss: 0.0695
Epoch   2 Batch  990/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.8935, Loss: 0.0412
Epoch   2 Batch  995/2154 - Train Accuracy: 0.9252, Validation Accuracy: 0.8778, Loss: 0.0426
Epoch   2 Batch 1000/2154 - Train Accuracy: 0.9152, Validation Accuracy: 0.8892, Loss: 0.0531
Epoch   2 Batch 1005/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9055, Loss: 0.0482
Epoch   2 Batch 1010/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.8835, Loss: 0.0494
Epoch   2 Batch 1015/2154 - Train Accuracy: 0.8569, Validation Accuracy: 0.9055, Loss: 0.0620
Epoch   2 Batch 1020/2154 - Train Accuracy: 0.9141, Validation Accuracy: 0.9119, Loss: 0.0607
Epoch   2 Batch 1025/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.8651, Loss: 0.0435
Epoch   2 Batch 1030/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.8743, Loss: 0.0632
Epoch   2 Batch 1035/2154 - Train Accuracy: 0.9464, Validation Accuracy: 0.9048, Loss: 0.0471
Epoch   2 Batch 1040/2154 - Train Accuracy: 0.9664, Validation Accuracy: 0.9006, Loss: 0.0404
Epoch   2 Batch 1045/2154 - Train Accuracy: 0.8577, Validation Accuracy: 0.9155, Loss: 0.0689
Epoch   2 Batch 1050/2154 - Train Accuracy: 0.9109, Validation Accuracy: 0.8906, Loss: 0.0727
Epoch   2 Batch 1055/2154 - Train Accuracy: 0.9104, Validation Accuracy: 0.9006, Loss: 0.0491
Epoch   2 Batch 1060/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.9148, Loss: 0.0550
Epoch   2 Batch 1065/2154 - Train Accuracy: 0.9039, Validation Accuracy: 0.8871, Loss: 0.0659
Epoch   2 Batch 1070/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.8892, Loss: 0.0579
Epoch   2 Batch 1075/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.8928, Loss: 0.0453
Epoch   2 Batch 1080/2154 - Train Accuracy: 0.8914, Validation Accuracy: 0.9098, Loss: 0.0749
Epoch   2 Batch 1085/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9091, Loss: 0.0452
Epoch   2 Batch 1090/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.8949, Loss: 0.0371
Epoch   2 Batch 1095/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9119, Loss: 0.0404
Epoch   2 Batch 1100/2154 - Train Accuracy: 0.8789, Validation Accuracy: 0.9027, Loss: 0.0658
Epoch   2 Batch 1105/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.9070, Loss: 0.0446
Epoch   2 Batch 1110/2154 - Train Accuracy: 0.9289, Validation Accuracy: 0.9134, Loss: 0.0616
Epoch   2 Batch 1115/2154 - Train Accuracy: 0.9441, Validation Accuracy: 0.8942, Loss: 0.0528
Epoch   2 Batch 1120/2154 - Train Accuracy: 0.9414, Validation Accuracy: 0.8970, Loss: 0.0467
Epoch   2 Batch 1125/2154 - Train Accuracy: 0.9424, Validation Accuracy: 0.8970, Loss: 0.0465
Epoch   2 Batch 1130/2154 - Train Accuracy: 0.9095, Validation Accuracy: 0.9041, Loss: 0.0471
Epoch   2 Batch 1135/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9027, Loss: 0.0515
Epoch   2 Batch 1140/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9126, Loss: 0.0535
Epoch   2 Batch 1145/2154 - Train Accuracy: 0.9276, Validation Accuracy: 0.9112, Loss: 0.0576
Epoch   2 Batch 1150/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.9034, Loss: 0.0545
Epoch   2 Batch 1155/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9126, Loss: 0.0486
Epoch   2 Batch 1160/2154 - Train Accuracy: 0.9297, Validation Accuracy: 0.9439, Loss: 0.0439
Epoch   2 Batch 1165/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9283, Loss: 0.0476
Epoch   2 Batch 1170/2154 - Train Accuracy: 0.9382, Validation Accuracy: 0.9325, Loss: 0.0530
Epoch   2 Batch 1175/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9354, Loss: 0.0642
Epoch   2 Batch 1180/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.9169, Loss: 0.0544
Epoch   2 Batch 1185/2154 - Train Accuracy: 0.9482, Validation Accuracy: 0.9119, Loss: 0.0554
Epoch   2 Batch 1190/2154 - Train Accuracy: 0.9586, Validation Accuracy: 0.9041, Loss: 0.0649
Epoch   2 Batch 1195/2154 - Train Accuracy: 0.9109, Validation Accuracy: 0.8935, Loss: 0.0432
Epoch   2 Batch 1200/2154 - Train Accuracy: 0.9013, Validation Accuracy: 0.9027, Loss: 0.0932
Epoch   2 Batch 1205/2154 - Train Accuracy: 0.8972, Validation Accuracy: 0.9162, Loss: 0.0639
Epoch   2 Batch 1210/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.9027, Loss: 0.0570
Epoch   2 Batch 1215/2154 - Train Accuracy: 0.9252, Validation Accuracy: 0.9034, Loss: 0.0532
Epoch   2 Batch 1220/2154 - Train Accuracy: 0.9117, Validation Accuracy: 0.9148, Loss: 0.0484
Epoch   2 Batch 1225/2154 - Train Accuracy: 0.9805, Validation Accuracy: 0.9070, Loss: 0.0371
Epoch   2 Batch 1230/2154 - Train Accuracy: 0.9442, Validation Accuracy: 0.8835, Loss: 0.0475
Epoch   2 Batch 1235/2154 - Train Accuracy: 0.9107, Validation Accuracy: 0.9105, Loss: 0.0524
Epoch   2 Batch 1240/2154 - Train Accuracy: 0.8923, Validation Accuracy: 0.9105, Loss: 0.0467
Epoch   2 Batch 1245/2154 - Train Accuracy: 0.9326, Validation Accuracy: 0.9013, Loss: 0.0732
Epoch   2 Batch 1250/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.8857, Loss: 0.0439
Epoch   2 Batch 1255/2154 - Train Accuracy: 0.9169, Validation Accuracy: 0.8899, Loss: 0.0583
Epoch   2 Batch 1260/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.8885, Loss: 0.0635
Epoch   2 Batch 1265/2154 - Train Accuracy: 0.8977, Validation Accuracy: 0.8913, Loss: 0.0496
Epoch   2 Batch 1270/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.9240, Loss: 0.0348
Epoch   2 Batch 1275/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9254, Loss: 0.0581
Epoch   2 Batch 1280/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8935, Loss: 0.0612
Epoch   2 Batch 1285/2154 - Train Accuracy: 0.8820, Validation Accuracy: 0.8991, Loss: 0.0563
Epoch   2 Batch 1290/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9226, Loss: 0.0466
Epoch   2 Batch 1295/2154 - Train Accuracy: 0.9406, Validation Accuracy: 0.8942, Loss: 0.0474
Epoch   2 Batch 1300/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.9254, Loss: 0.0578
Epoch   2 Batch 1305/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9148, Loss: 0.0449
Epoch   2 Batch 1310/2154 - Train Accuracy: 0.9656, Validation Accuracy: 0.8970, Loss: 0.0547
Epoch   2 Batch 1315/2154 - Train Accuracy: 0.9548, Validation Accuracy: 0.9396, Loss: 0.0553
Epoch   2 Batch 1320/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9347, Loss: 0.0386
Epoch   2 Batch 1325/2154 - Train Accuracy: 0.9308, Validation Accuracy: 0.9411, Loss: 0.0382
Epoch   2 Batch 1330/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.9361, Loss: 0.0523
Epoch   2 Batch 1335/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9318, Loss: 0.0772
Epoch   2 Batch 1340/2154 - Train Accuracy: 0.9141, Validation Accuracy: 0.9212, Loss: 0.0569
Epoch   2 Batch 1345/2154 - Train Accuracy: 0.9490, Validation Accuracy: 0.9013, Loss: 0.0556
Epoch   2 Batch 1350/2154 - Train Accuracy: 0.9281, Validation Accuracy: 0.9112, Loss: 0.0509
Epoch   2 Batch 1355/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.9027, Loss: 0.0565
Epoch   2 Batch 1360/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.9389, Loss: 0.0401
Epoch   2 Batch 1365/2154 - Train Accuracy: 0.9000, Validation Accuracy: 0.9361, Loss: 0.0591
Epoch   2 Batch 1370/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.9190, Loss: 0.0462
Epoch   2 Batch 1375/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9240, Loss: 0.0559
Epoch   2 Batch 1380/2154 - Train Accuracy: 0.9781, Validation Accuracy: 0.9339, Loss: 0.0418
Epoch   2 Batch 1385/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9574, Loss: 0.0458
Epoch   2 Batch 1390/2154 - Train Accuracy: 0.9405, Validation Accuracy: 0.9396, Loss: 0.0502
Epoch   2 Batch 1395/2154 - Train Accuracy: 0.9400, Validation Accuracy: 0.9496, Loss: 0.0512
Epoch   2 Batch 1400/2154 - Train Accuracy: 0.9581, Validation Accuracy: 0.9311, Loss: 0.0400
Epoch   2 Batch 1405/2154 - Train Accuracy: 0.9159, Validation Accuracy: 0.9112, Loss: 0.0652
Epoch   2 Batch 1410/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9268, Loss: 0.0571
Epoch   2 Batch 1415/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.9062, Loss: 0.0504
Epoch   2 Batch 1420/2154 - Train Accuracy: 0.9186, Validation Accuracy: 0.9027, Loss: 0.0673
Epoch   2 Batch 1425/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9311, Loss: 0.0343
Epoch   2 Batch 1430/2154 - Train Accuracy: 0.9725, Validation Accuracy: 0.9183, Loss: 0.0487
Epoch   2 Batch 1435/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9290, Loss: 0.0421
Epoch   2 Batch 1440/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.9162, Loss: 0.0623
Epoch   2 Batch 1445/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.9162, Loss: 0.0451
Epoch   2 Batch 1450/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9339, Loss: 0.0541
Epoch   2 Batch 1455/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.9247, Loss: 0.0502
Epoch   2 Batch 1460/2154 - Train Accuracy: 0.9016, Validation Accuracy: 0.9247, Loss: 0.0681
Epoch   2 Batch 1465/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9318, Loss: 0.0399
Epoch   2 Batch 1470/2154 - Train Accuracy: 0.9482, Validation Accuracy: 0.9432, Loss: 0.0438
Epoch   2 Batch 1475/2154 - Train Accuracy: 0.9555, Validation Accuracy: 0.9503, Loss: 0.0742
Epoch   2 Batch 1480/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9503, Loss: 0.0484
Epoch   2 Batch 1485/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9141, Loss: 0.0423
Epoch   2 Batch 1490/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9006, Loss: 0.0499
Epoch   2 Batch 1495/2154 - Train Accuracy: 0.9285, Validation Accuracy: 0.9006, Loss: 0.0369
Epoch   2 Batch 1500/2154 - Train Accuracy: 0.9564, Validation Accuracy: 0.8885, Loss: 0.0485
Epoch   2 Batch 1505/2154 - Train Accuracy: 0.9112, Validation Accuracy: 0.8984, Loss: 0.0494
Epoch   2 Batch 1510/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9219, Loss: 0.0531
Epoch   2 Batch 1515/2154 - Train Accuracy: 0.9515, Validation Accuracy: 0.9126, Loss: 0.0332
Epoch   2 Batch 1520/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.8935, Loss: 0.0403
Epoch   2 Batch 1525/2154 - Train Accuracy: 0.9664, Validation Accuracy: 0.9105, Loss: 0.0457
Epoch   2 Batch 1530/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9134, Loss: 0.0540
Epoch   2 Batch 1535/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9197, Loss: 0.0412
Epoch   2 Batch 1540/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9304, Loss: 0.0570
Epoch   2 Batch 1545/2154 - Train Accuracy: 0.9345, Validation Accuracy: 0.9254, Loss: 0.0518
Epoch   2 Batch 1550/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9247, Loss: 0.0566
Epoch   2 Batch 1555/2154 - Train Accuracy: 0.9133, Validation Accuracy: 0.9467, Loss: 0.0561
Epoch   2 Batch 1560/2154 - Train Accuracy: 0.9125, Validation Accuracy: 0.9496, Loss: 0.0422
Epoch   2 Batch 1565/2154 - Train Accuracy: 0.9256, Validation Accuracy: 0.9531, Loss: 0.0502
Epoch   2 Batch 1570/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9311, Loss: 0.0431
Epoch   2 Batch 1575/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.9276, Loss: 0.0385
Epoch   2 Batch 1580/2154 - Train Accuracy: 0.9260, Validation Accuracy: 0.9439, Loss: 0.0534
Epoch   2 Batch 1585/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.9304, Loss: 0.0537
Epoch   2 Batch 1590/2154 - Train Accuracy: 0.9281, Validation Accuracy: 0.9432, Loss: 0.0396
Epoch   2 Batch 1595/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9425, Loss: 0.0397
Epoch   2 Batch 1600/2154 - Train Accuracy: 0.9094, Validation Accuracy: 0.9339, Loss: 0.0525
Epoch   2 Batch 1605/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9233, Loss: 0.0471
Epoch   2 Batch 1610/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9268, Loss: 0.0302
Epoch   2 Batch 1615/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9375, Loss: 0.0269
Epoch   2 Batch 1620/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9425, Loss: 0.0804
Epoch   2 Batch 1625/2154 - Train Accuracy: 0.8773, Validation Accuracy: 0.9347, Loss: 0.0485
Epoch   2 Batch 1630/2154 - Train Accuracy: 0.9326, Validation Accuracy: 0.9482, Loss: 0.0428
Epoch   2 Batch 1635/2154 - Train Accuracy: 0.9317, Validation Accuracy: 0.9474, Loss: 0.0553
Epoch   2 Batch 1640/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9361, Loss: 0.0400
Epoch   2 Batch 1645/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.9411, Loss: 0.0364
Epoch   2 Batch 1650/2154 - Train Accuracy: 0.9204, Validation Accuracy: 0.9403, Loss: 0.0496
Epoch   2 Batch 1655/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9233, Loss: 0.0463
Epoch   2 Batch 1660/2154 - Train Accuracy: 0.9770, Validation Accuracy: 0.9432, Loss: 0.0569
Epoch   2 Batch 1665/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9368, Loss: 0.0407
Epoch   2 Batch 1670/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9432, Loss: 0.0456
Epoch   2 Batch 1675/2154 - Train Accuracy: 0.9133, Validation Accuracy: 0.9439, Loss: 0.0695
Epoch   2 Batch 1680/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.9517, Loss: 0.0385
Epoch   2 Batch 1685/2154 - Train Accuracy: 0.9408, Validation Accuracy: 0.9517, Loss: 0.0420
Epoch   2 Batch 1690/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9418, Loss: 0.0421
Epoch   2 Batch 1695/2154 - Train Accuracy: 0.9581, Validation Accuracy: 0.9467, Loss: 0.0498
Epoch   2 Batch 1700/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.9453, Loss: 0.0449
Epoch   2 Batch 1705/2154 - Train Accuracy: 0.9202, Validation Accuracy: 0.9290, Loss: 0.0674
Epoch   2 Batch 1710/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.9375, Loss: 0.0557
Epoch   2 Batch 1715/2154 - Train Accuracy: 0.9293, Validation Accuracy: 0.9226, Loss: 0.0444
Epoch   2 Batch 1720/2154 - Train Accuracy: 0.9778, Validation Accuracy: 0.9396, Loss: 0.0525
Epoch   2 Batch 1725/2154 - Train Accuracy: 0.9153, Validation Accuracy: 0.9297, Loss: 0.0523
Epoch   2 Batch 1730/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.9226, Loss: 0.0439
Epoch   2 Batch 1735/2154 - Train Accuracy: 0.8980, Validation Accuracy: 0.9233, Loss: 0.0598
Epoch   2 Batch 1740/2154 - Train Accuracy: 0.9416, Validation Accuracy: 0.9190, Loss: 0.0495
Epoch   2 Batch 1745/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.9205, Loss: 0.0485
Epoch   2 Batch 1750/2154 - Train Accuracy: 0.9406, Validation Accuracy: 0.9119, Loss: 0.0494
Epoch   2 Batch 1755/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.9112, Loss: 0.0450
Epoch   2 Batch 1760/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9134, Loss: 0.0469
Epoch   2 Batch 1765/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9141, Loss: 0.0554
Epoch   2 Batch 1770/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.9197, Loss: 0.0505
Epoch   2 Batch 1775/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9276, Loss: 0.0457
Epoch   2 Batch 1780/2154 - Train Accuracy: 0.9484, Validation Accuracy: 0.9276, Loss: 0.0362
Epoch   2 Batch 1785/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9432, Loss: 0.0382
Epoch   2 Batch 1790/2154 - Train Accuracy: 0.9561, Validation Accuracy: 0.9332, Loss: 0.0441
Epoch   2 Batch 1795/2154 - Train Accuracy: 0.9414, Validation Accuracy: 0.9276, Loss: 0.0383
Epoch   2 Batch 1800/2154 - Train Accuracy: 0.9297, Validation Accuracy: 0.9233, Loss: 0.0536
Epoch   2 Batch 1805/2154 - Train Accuracy: 0.9187, Validation Accuracy: 0.9268, Loss: 0.0560
Epoch   2 Batch 1810/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.9325, Loss: 0.0478
Epoch   2 Batch 1815/2154 - Train Accuracy: 0.9602, Validation Accuracy: 0.9311, Loss: 0.0417
Epoch   2 Batch 1820/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9034, Loss: 0.0382
Epoch   2 Batch 1825/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9048, Loss: 0.0500
Epoch   2 Batch 1830/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9148, Loss: 0.0875
Epoch   2 Batch 1835/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9134, Loss: 0.0471
Epoch   2 Batch 1840/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9098, Loss: 0.0380
Epoch   2 Batch 1845/2154 - Train Accuracy: 0.9344, Validation Accuracy: 0.9062, Loss: 0.0548
Epoch   2 Batch 1850/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.9240, Loss: 0.0596
Epoch   2 Batch 1855/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9332, Loss: 0.0522
Epoch   2 Batch 1860/2154 - Train Accuracy: 0.9498, Validation Accuracy: 0.9283, Loss: 0.0469
Epoch   2 Batch 1865/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9268, Loss: 0.0406
Epoch   2 Batch 1870/2154 - Train Accuracy: 0.9289, Validation Accuracy: 0.9254, Loss: 0.0419
Epoch   2 Batch 1875/2154 - Train Accuracy: 0.9515, Validation Accuracy: 0.9354, Loss: 0.0677
Epoch   2 Batch 1880/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.9070, Loss: 0.0588
Epoch   2 Batch 1885/2154 - Train Accuracy: 0.9186, Validation Accuracy: 0.9254, Loss: 0.0574
Epoch   2 Batch 1890/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9155, Loss: 0.0334
Epoch   2 Batch 1895/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9112, Loss: 0.0444
Epoch   2 Batch 1900/2154 - Train Accuracy: 0.9672, Validation Accuracy: 0.9112, Loss: 0.0444
Epoch   2 Batch 1905/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9254, Loss: 0.0378
Epoch   2 Batch 1910/2154 - Train Accuracy: 0.9507, Validation Accuracy: 0.9112, Loss: 0.0352
Epoch   2 Batch 1915/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9197, Loss: 0.0300
Epoch   2 Batch 1920/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9197, Loss: 0.0407
Epoch   2 Batch 1925/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9361, Loss: 0.0447
Epoch   2 Batch 1930/2154 - Train Accuracy: 0.9938, Validation Accuracy: 0.9482, Loss: 0.0380
Epoch   2 Batch 1935/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.9339, Loss: 0.0531
Epoch   2 Batch 1940/2154 - Train Accuracy: 0.9338, Validation Accuracy: 0.9382, Loss: 0.0450
Epoch   2 Batch 1945/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.9212, Loss: 0.0410
Epoch   2 Batch 1950/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.9098, Loss: 0.0413
Epoch   2 Batch 1955/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9070, Loss: 0.0342
Epoch   2 Batch 1960/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9091, Loss: 0.0520
Epoch   2 Batch 1965/2154 - Train Accuracy: 0.9115, Validation Accuracy: 0.8935, Loss: 0.0496
Epoch   2 Batch 1970/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.8935, Loss: 0.0557
Epoch   2 Batch 1975/2154 - Train Accuracy: 0.9055, Validation Accuracy: 0.9006, Loss: 0.0446
Epoch   2 Batch 1980/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.9134, Loss: 0.0414
Epoch   2 Batch 1985/2154 - Train Accuracy: 0.9803, Validation Accuracy: 0.9261, Loss: 0.0453
Epoch   2 Batch 1990/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.9197, Loss: 0.0367
Epoch   2 Batch 1995/2154 - Train Accuracy: 0.9400, Validation Accuracy: 0.9162, Loss: 0.0532
Epoch   2 Batch 2000/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9027, Loss: 0.0568
Epoch   2 Batch 2005/2154 - Train Accuracy: 0.9490, Validation Accuracy: 0.8963, Loss: 0.0384
Epoch   2 Batch 2010/2154 - Train Accuracy: 0.9202, Validation Accuracy: 0.9098, Loss: 0.0780
Epoch   2 Batch 2015/2154 - Train Accuracy: 0.9278, Validation Accuracy: 0.9006, Loss: 0.0388
Epoch   2 Batch 2020/2154 - Train Accuracy: 0.9515, Validation Accuracy: 0.9098, Loss: 0.0338
Epoch   2 Batch 2025/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9084, Loss: 0.0354
Epoch   2 Batch 2030/2154 - Train Accuracy: 0.9594, Validation Accuracy: 0.9084, Loss: 0.0446
Epoch   2 Batch 2035/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.9112, Loss: 0.0310
Epoch   2 Batch 2040/2154 - Train Accuracy: 0.8964, Validation Accuracy: 0.9212, Loss: 0.0423
Epoch   2 Batch 2045/2154 - Train Accuracy: 0.9420, Validation Accuracy: 0.9048, Loss: 0.0331
Epoch   2 Batch 2050/2154 - Train Accuracy: 0.8857, Validation Accuracy: 0.9169, Loss: 0.0627
Epoch   2 Batch 2055/2154 - Train Accuracy: 0.8931, Validation Accuracy: 0.9489, Loss: 0.0469
Epoch   2 Batch 2060/2154 - Train Accuracy: 0.9400, Validation Accuracy: 0.9382, Loss: 0.0438
Epoch   2 Batch 2065/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9446, Loss: 0.0397
Epoch   2 Batch 2070/2154 - Train Accuracy: 0.9117, Validation Accuracy: 0.9226, Loss: 0.0401
Epoch   2 Batch 2075/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9226, Loss: 0.0437
Epoch   2 Batch 2080/2154 - Train Accuracy: 0.9167, Validation Accuracy: 0.9361, Loss: 0.0428
Epoch   2 Batch 2085/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9396, Loss: 0.0380
Epoch   2 Batch 2090/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9169, Loss: 0.0537
Epoch   2 Batch 2095/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9283, Loss: 0.0392
Epoch   2 Batch 2100/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9169, Loss: 0.0386
Epoch   2 Batch 2105/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.9403, Loss: 0.0522
Epoch   2 Batch 2110/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.9297, Loss: 0.0452
Epoch   2 Batch 2115/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9425, Loss: 0.0424
Epoch   2 Batch 2120/2154 - Train Accuracy: 0.9194, Validation Accuracy: 0.9205, Loss: 0.0387
Epoch   2 Batch 2125/2154 - Train Accuracy: 0.9109, Validation Accuracy: 0.9261, Loss: 0.0511
Epoch   2 Batch 2130/2154 - Train Accuracy: 0.9556, Validation Accuracy: 0.9148, Loss: 0.0416
Epoch   2 Batch 2135/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.9240, Loss: 0.0500
Epoch   2 Batch 2140/2154 - Train Accuracy: 0.9561, Validation Accuracy: 0.9148, Loss: 0.0256
Epoch   2 Batch 2145/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9105, Loss: 0.0466
Epoch   2 Batch 2150/2154 - Train Accuracy: 0.9598, Validation Accuracy: 0.9233, Loss: 0.0510
Epoch   3 Batch    5/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.9190, Loss: 0.0485
Epoch   3 Batch   10/2154 - Train Accuracy: 0.9187, Validation Accuracy: 0.9247, Loss: 0.0398
Epoch   3 Batch   15/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9062, Loss: 0.0363
Epoch   3 Batch   20/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.9070, Loss: 0.0434
Epoch   3 Batch   25/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.8821, Loss: 0.0484
Epoch   3 Batch   30/2154 - Train Accuracy: 0.9484, Validation Accuracy: 0.8977, Loss: 0.0371
Epoch   3 Batch   35/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9219, Loss: 0.0408
Epoch   3 Batch   40/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9361, Loss: 0.0411
Epoch   3 Batch   45/2154 - Train Accuracy: 0.9597, Validation Accuracy: 0.9517, Loss: 0.0485
Epoch   3 Batch   50/2154 - Train Accuracy: 0.9400, Validation Accuracy: 0.9368, Loss: 0.0634
Epoch   3 Batch   55/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9510, Loss: 0.0349
Epoch   3 Batch   60/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.9453, Loss: 0.0376
Epoch   3 Batch   65/2154 - Train Accuracy: 0.9769, Validation Accuracy: 0.9297, Loss: 0.0407
Epoch   3 Batch   70/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9503, Loss: 0.0416
Epoch   3 Batch   75/2154 - Train Accuracy: 0.9482, Validation Accuracy: 0.9503, Loss: 0.0492
Epoch   3 Batch   80/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9411, Loss: 0.0463
Epoch   3 Batch   85/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9389, Loss: 0.0624
Epoch   3 Batch   90/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.9482, Loss: 0.0375
Epoch   3 Batch   95/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9553, Loss: 0.0416
Epoch   3 Batch  100/2154 - Train Accuracy: 0.9092, Validation Accuracy: 0.9553, Loss: 0.0666
Epoch   3 Batch  105/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.9482, Loss: 0.0522
Epoch   3 Batch  110/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.9503, Loss: 0.0685
Epoch   3 Batch  115/2154 - Train Accuracy: 0.9020, Validation Accuracy: 0.9439, Loss: 0.0514
Epoch   3 Batch  120/2154 - Train Accuracy: 0.9260, Validation Accuracy: 0.9538, Loss: 0.0349
Epoch   3 Batch  125/2154 - Train Accuracy: 0.9087, Validation Accuracy: 0.9645, Loss: 0.0369
Epoch   3 Batch  130/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9432, Loss: 0.0359
Epoch   3 Batch  135/2154 - Train Accuracy: 0.9627, Validation Accuracy: 0.9318, Loss: 0.0451
Epoch   3 Batch  140/2154 - Train Accuracy: 0.9336, Validation Accuracy: 0.9233, Loss: 0.0589
Epoch   3 Batch  145/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9233, Loss: 0.0491
Epoch   3 Batch  150/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.9339, Loss: 0.0328
Epoch   3 Batch  155/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9347, Loss: 0.0487
Epoch   3 Batch  160/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9432, Loss: 0.0362
Epoch   3 Batch  165/2154 - Train Accuracy: 0.9183, Validation Accuracy: 0.9545, Loss: 0.0381
Epoch   3 Batch  170/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9332, Loss: 0.0341
Epoch   3 Batch  175/2154 - Train Accuracy: 0.9305, Validation Accuracy: 0.9453, Loss: 0.0455
Epoch   3 Batch  180/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9276, Loss: 0.0418
Epoch   3 Batch  185/2154 - Train Accuracy: 0.9350, Validation Accuracy: 0.9446, Loss: 0.0448
Epoch   3 Batch  190/2154 - Train Accuracy: 0.9336, Validation Accuracy: 0.9325, Loss: 0.0348
Epoch   3 Batch  195/2154 - Train Accuracy: 0.9235, Validation Accuracy: 0.9219, Loss: 0.0468
Epoch   3 Batch  200/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9297, Loss: 0.0297
Epoch   3 Batch  205/2154 - Train Accuracy: 0.9758, Validation Accuracy: 0.9290, Loss: 0.0337
Epoch   3 Batch  210/2154 - Train Accuracy: 0.9786, Validation Accuracy: 0.9297, Loss: 0.0352
Epoch   3 Batch  215/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.9126, Loss: 0.0452
Epoch   3 Batch  220/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9276, Loss: 0.0368
Epoch   3 Batch  225/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9041, Loss: 0.0352
Epoch   3 Batch  230/2154 - Train Accuracy: 0.9479, Validation Accuracy: 0.9091, Loss: 0.0360
Epoch   3 Batch  235/2154 - Train Accuracy: 0.9906, Validation Accuracy: 0.9190, Loss: 0.0358
Epoch   3 Batch  240/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9183, Loss: 0.0493
Epoch   3 Batch  245/2154 - Train Accuracy: 0.9679, Validation Accuracy: 0.9254, Loss: 0.0380
Epoch   3 Batch  250/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9148, Loss: 0.0400
Epoch   3 Batch  255/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.9084, Loss: 0.0409
Epoch   3 Batch  260/2154 - Train Accuracy: 0.9414, Validation Accuracy: 0.9169, Loss: 0.0435
Epoch   3 Batch  265/2154 - Train Accuracy: 0.9484, Validation Accuracy: 0.9268, Loss: 0.0464
Epoch   3 Batch  270/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9091, Loss: 0.0427
Epoch   3 Batch  275/2154 - Train Accuracy: 0.9797, Validation Accuracy: 0.9006, Loss: 0.0264
Epoch   3 Batch  280/2154 - Train Accuracy: 0.9153, Validation Accuracy: 0.8999, Loss: 0.0543
Epoch   3 Batch  285/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.9219, Loss: 0.0448
Epoch   3 Batch  290/2154 - Train Accuracy: 0.9178, Validation Accuracy: 0.9197, Loss: 0.0512
Epoch   3 Batch  295/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9325, Loss: 0.0317
Epoch   3 Batch  300/2154 - Train Accuracy: 0.9622, Validation Accuracy: 0.9339, Loss: 0.0428
Epoch   3 Batch  305/2154 - Train Accuracy: 0.9414, Validation Accuracy: 0.9418, Loss: 0.0543
Epoch   3 Batch  310/2154 - Train Accuracy: 0.9564, Validation Accuracy: 0.9134, Loss: 0.0390
Epoch   3 Batch  315/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9155, Loss: 0.0297
Epoch   3 Batch  320/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9155, Loss: 0.0289
Epoch   3 Batch  325/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.9084, Loss: 0.0554
Epoch   3 Batch  330/2154 - Train Accuracy: 0.9564, Validation Accuracy: 0.9155, Loss: 0.0353
Epoch   3 Batch  335/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9347, Loss: 0.0311
Epoch   3 Batch  340/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.9141, Loss: 0.0363
Epoch   3 Batch  345/2154 - Train Accuracy: 0.9178, Validation Accuracy: 0.9311, Loss: 0.0450
Epoch   3 Batch  350/2154 - Train Accuracy: 0.9414, Validation Accuracy: 0.9467, Loss: 0.0429
Epoch   3 Batch  355/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9510, Loss: 0.0478
Epoch   3 Batch  360/2154 - Train Accuracy: 0.9572, Validation Accuracy: 0.9304, Loss: 0.0357
Epoch   3 Batch  365/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9396, Loss: 0.0487
Epoch   3 Batch  370/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.9332, Loss: 0.0431
Epoch   3 Batch  375/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9325, Loss: 0.0345
Epoch   3 Batch  380/2154 - Train Accuracy: 0.9828, Validation Accuracy: 0.9070, Loss: 0.0322
Epoch   3 Batch  385/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.9162, Loss: 0.0413
Epoch   3 Batch  390/2154 - Train Accuracy: 0.9650, Validation Accuracy: 0.9382, Loss: 0.0283
Epoch   3 Batch  395/2154 - Train Accuracy: 0.9742, Validation Accuracy: 0.9425, Loss: 0.0405
Epoch   3 Batch  400/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9261, Loss: 0.0318
Epoch   3 Batch  405/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9034, Loss: 0.0445
Epoch   3 Batch  410/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9354, Loss: 0.0456
Epoch   3 Batch  415/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9354, Loss: 0.0428
Epoch   3 Batch  420/2154 - Train Accuracy: 0.9643, Validation Accuracy: 0.9574, Loss: 0.0308
Epoch   3 Batch  425/2154 - Train Accuracy: 0.9784, Validation Accuracy: 0.9332, Loss: 0.0257
Epoch   3 Batch  430/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.9339, Loss: 0.0431
Epoch   3 Batch  435/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9467, Loss: 0.0341
Epoch   3 Batch  440/2154 - Train Accuracy: 0.9507, Validation Accuracy: 0.9283, Loss: 0.0350
Epoch   3 Batch  445/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9268, Loss: 0.0374
Epoch   3 Batch  450/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9318, Loss: 0.0367
Epoch   3 Batch  455/2154 - Train Accuracy: 0.9055, Validation Accuracy: 0.9027, Loss: 0.0559
Epoch   3 Batch  460/2154 - Train Accuracy: 0.9008, Validation Accuracy: 0.9311, Loss: 0.0408
Epoch   3 Batch  465/2154 - Train Accuracy: 0.9285, Validation Accuracy: 0.9304, Loss: 0.0427
Epoch   3 Batch  470/2154 - Train Accuracy: 0.8976, Validation Accuracy: 0.9261, Loss: 0.0470
Epoch   3 Batch  475/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9162, Loss: 0.0402
Epoch   3 Batch  480/2154 - Train Accuracy: 0.9762, Validation Accuracy: 0.9077, Loss: 0.0310
Epoch   3 Batch  485/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9134, Loss: 0.0242
Epoch   3 Batch  490/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9205, Loss: 0.0403
Epoch   3 Batch  495/2154 - Train Accuracy: 0.9464, Validation Accuracy: 0.9155, Loss: 0.0443
Epoch   3 Batch  500/2154 - Train Accuracy: 0.9836, Validation Accuracy: 0.9276, Loss: 0.0425
Epoch   3 Batch  505/2154 - Train Accuracy: 0.9235, Validation Accuracy: 0.9205, Loss: 0.0559
Epoch   3 Batch  510/2154 - Train Accuracy: 0.9102, Validation Accuracy: 0.9339, Loss: 0.0440
Epoch   3 Batch  515/2154 - Train Accuracy: 0.9144, Validation Accuracy: 0.9339, Loss: 0.0402
Epoch   3 Batch  520/2154 - Train Accuracy: 0.9602, Validation Accuracy: 0.9474, Loss: 0.0280
Epoch   3 Batch  525/2154 - Train Accuracy: 0.9906, Validation Accuracy: 0.9439, Loss: 0.0260
Epoch   3 Batch  530/2154 - Train Accuracy: 0.9766, Validation Accuracy: 0.9446, Loss: 0.0341
Epoch   3 Batch  535/2154 - Train Accuracy: 0.9709, Validation Accuracy: 0.9318, Loss: 0.0397
Epoch   3 Batch  540/2154 - Train Accuracy: 0.9638, Validation Accuracy: 0.9318, Loss: 0.0392
Epoch   3 Batch  545/2154 - Train Accuracy: 0.9342, Validation Accuracy: 0.9375, Loss: 0.0488
Epoch   3 Batch  550/2154 - Train Accuracy: 0.9048, Validation Accuracy: 0.9503, Loss: 0.0481
Epoch   3 Batch  555/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.9453, Loss: 0.0432
Epoch   3 Batch  560/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9318, Loss: 0.0447
Epoch   3 Batch  565/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.9361, Loss: 0.0436
Epoch   3 Batch  570/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9126, Loss: 0.0493
Epoch   3 Batch  575/2154 - Train Accuracy: 0.9555, Validation Accuracy: 0.9091, Loss: 0.0473
Epoch   3 Batch  580/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9197, Loss: 0.0374
Epoch   3 Batch  585/2154 - Train Accuracy: 0.9621, Validation Accuracy: 0.9403, Loss: 0.0424
Epoch   3 Batch  590/2154 - Train Accuracy: 0.9624, Validation Accuracy: 0.9389, Loss: 0.0424
Epoch   3 Batch  595/2154 - Train Accuracy: 0.9594, Validation Accuracy: 0.9439, Loss: 0.0376
Epoch   3 Batch  600/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9446, Loss: 0.0310
Epoch   3 Batch  605/2154 - Train Accuracy: 0.9602, Validation Accuracy: 0.9446, Loss: 0.0442
Epoch   3 Batch  610/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9347, Loss: 0.0353
Epoch   3 Batch  615/2154 - Train Accuracy: 0.9427, Validation Accuracy: 0.9134, Loss: 0.0415
Epoch   3 Batch  620/2154 - Train Accuracy: 0.9561, Validation Accuracy: 0.9411, Loss: 0.0314
Epoch   3 Batch  625/2154 - Train Accuracy: 0.9400, Validation Accuracy: 0.9268, Loss: 0.0486
Epoch   3 Batch  630/2154 - Train Accuracy: 0.9515, Validation Accuracy: 0.9347, Loss: 0.0409
Epoch   3 Batch  635/2154 - Train Accuracy: 0.9655, Validation Accuracy: 0.9318, Loss: 0.0435
Epoch   3 Batch  640/2154 - Train Accuracy: 0.9586, Validation Accuracy: 0.9325, Loss: 0.0369
Epoch   3 Batch  645/2154 - Train Accuracy: 0.9315, Validation Accuracy: 0.9318, Loss: 0.0337
Epoch   3 Batch  650/2154 - Train Accuracy: 0.9195, Validation Accuracy: 0.9517, Loss: 0.0415
Epoch   3 Batch  655/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.9446, Loss: 0.0485
Epoch   3 Batch  660/2154 - Train Accuracy: 0.9250, Validation Accuracy: 0.9659, Loss: 0.0421
Epoch   3 Batch  665/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9709, Loss: 0.0278
Epoch   3 Batch  670/2154 - Train Accuracy: 0.9811, Validation Accuracy: 0.9709, Loss: 0.0282
Epoch   3 Batch  675/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9382, Loss: 0.0409
Epoch   3 Batch  680/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9070, Loss: 0.0284
Epoch   3 Batch  685/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9297, Loss: 0.0340
Epoch   3 Batch  690/2154 - Train Accuracy: 0.9250, Validation Accuracy: 0.9162, Loss: 0.0505
Epoch   3 Batch  695/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9361, Loss: 0.0275
Epoch   3 Batch  700/2154 - Train Accuracy: 0.9408, Validation Accuracy: 0.9297, Loss: 0.0347
Epoch   3 Batch  705/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9354, Loss: 0.0314
Epoch   3 Batch  710/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9652, Loss: 0.0374
Epoch   3 Batch  715/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9638, Loss: 0.0413
Epoch   3 Batch  720/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.9389, Loss: 0.0371
Epoch   3 Batch  725/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9311, Loss: 0.0415
Epoch   3 Batch  730/2154 - Train Accuracy: 0.9079, Validation Accuracy: 0.9368, Loss: 0.0594
Epoch   3 Batch  735/2154 - Train Accuracy: 0.9762, Validation Accuracy: 0.9318, Loss: 0.0208
Epoch   3 Batch  740/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9467, Loss: 0.0402
Epoch   3 Batch  745/2154 - Train Accuracy: 0.9556, Validation Accuracy: 0.9581, Loss: 0.0309
Epoch   3 Batch  750/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9666, Loss: 0.0387
Epoch   3 Batch  755/2154 - Train Accuracy: 0.9474, Validation Accuracy: 0.9560, Loss: 0.0438
Epoch   3 Batch  760/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9382, Loss: 0.0463
Epoch   3 Batch  765/2154 - Train Accuracy: 0.9702, Validation Accuracy: 0.9609, Loss: 0.0519
Epoch   3 Batch  770/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9510, Loss: 0.0311
Epoch   3 Batch  775/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9240, Loss: 0.0406
Epoch   3 Batch  780/2154 - Train Accuracy: 0.9281, Validation Accuracy: 0.9190, Loss: 0.0300
Epoch   3 Batch  785/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9389, Loss: 0.0330
Epoch   3 Batch  790/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9411, Loss: 0.0302
Epoch   3 Batch  795/2154 - Train Accuracy: 0.9635, Validation Accuracy: 0.9624, Loss: 0.0408
Epoch   3 Batch  800/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9247, Loss: 0.0435
Epoch   3 Batch  805/2154 - Train Accuracy: 0.9967, Validation Accuracy: 0.9375, Loss: 0.0261
Epoch   3 Batch  810/2154 - Train Accuracy: 0.9799, Validation Accuracy: 0.9403, Loss: 0.0276
Epoch   3 Batch  815/2154 - Train Accuracy: 0.9477, Validation Accuracy: 0.9290, Loss: 0.0467
Epoch   3 Batch  820/2154 - Train Accuracy: 0.9605, Validation Accuracy: 0.9531, Loss: 0.0541
Epoch   3 Batch  825/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9553, Loss: 0.0278
Epoch   3 Batch  830/2154 - Train Accuracy: 0.9008, Validation Accuracy: 0.9538, Loss: 0.0504
Epoch   3 Batch  835/2154 - Train Accuracy: 0.9186, Validation Accuracy: 0.9609, Loss: 0.0660
Epoch   3 Batch  840/2154 - Train Accuracy: 0.9727, Validation Accuracy: 0.9538, Loss: 0.0445
Epoch   3 Batch  845/2154 - Train Accuracy: 0.9401, Validation Accuracy: 0.9361, Loss: 0.0382
Epoch   3 Batch  850/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9361, Loss: 0.0286
Epoch   3 Batch  855/2154 - Train Accuracy: 0.9281, Validation Accuracy: 0.9375, Loss: 0.0479
Epoch   3 Batch  860/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9261, Loss: 0.0376
Epoch   3 Batch  865/2154 - Train Accuracy: 0.9203, Validation Accuracy: 0.9460, Loss: 0.0400
Epoch   3 Batch  870/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9311, Loss: 0.0276
Epoch   3 Batch  875/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9176, Loss: 0.0317
Epoch   3 Batch  880/2154 - Train Accuracy: 0.9195, Validation Accuracy: 0.9325, Loss: 0.0602
Epoch   3 Batch  885/2154 - Train Accuracy: 0.9256, Validation Accuracy: 0.9212, Loss: 0.0564
Epoch   3 Batch  890/2154 - Train Accuracy: 0.9498, Validation Accuracy: 0.9119, Loss: 0.0378
Epoch   3 Batch  895/2154 - Train Accuracy: 0.9161, Validation Accuracy: 0.9176, Loss: 0.0431
Epoch   3 Batch  900/2154 - Train Accuracy: 0.9490, Validation Accuracy: 0.9311, Loss: 0.0444
Epoch   3 Batch  905/2154 - Train Accuracy: 0.9594, Validation Accuracy: 0.9446, Loss: 0.0302
Epoch   3 Batch  910/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9453, Loss: 0.0307
Epoch   3 Batch  915/2154 - Train Accuracy: 0.9702, Validation Accuracy: 0.9581, Loss: 0.0272
Epoch   3 Batch  920/2154 - Train Accuracy: 0.9635, Validation Accuracy: 0.9538, Loss: 0.0217
Epoch   3 Batch  925/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9446, Loss: 0.0234
Epoch   3 Batch  930/2154 - Train Accuracy: 0.9281, Validation Accuracy: 0.9332, Loss: 0.0465
Epoch   3 Batch  935/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9425, Loss: 0.0439
Epoch   3 Batch  940/2154 - Train Accuracy: 0.9305, Validation Accuracy: 0.9510, Loss: 0.0354
Epoch   3 Batch  945/2154 - Train Accuracy: 0.9727, Validation Accuracy: 0.9375, Loss: 0.0296
Epoch   3 Batch  950/2154 - Train Accuracy: 0.9498, Validation Accuracy: 0.9375, Loss: 0.0360
Epoch   3 Batch  955/2154 - Train Accuracy: 0.9575, Validation Accuracy: 0.9538, Loss: 0.0390
Epoch   3 Batch  960/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9425, Loss: 0.0438
Epoch   3 Batch  965/2154 - Train Accuracy: 0.9655, Validation Accuracy: 0.9325, Loss: 0.0398
Epoch   3 Batch  970/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9240, Loss: 0.0459
Epoch   3 Batch  975/2154 - Train Accuracy: 0.9317, Validation Accuracy: 0.9148, Loss: 0.0388
Epoch   3 Batch  980/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9197, Loss: 0.0307
Epoch   3 Batch  985/2154 - Train Accuracy: 0.9320, Validation Accuracy: 0.9510, Loss: 0.0450
Epoch   3 Batch  990/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9510, Loss: 0.0278
Epoch   3 Batch  995/2154 - Train Accuracy: 0.9737, Validation Accuracy: 0.9538, Loss: 0.0273
Epoch   3 Batch 1000/2154 - Train Accuracy: 0.9501, Validation Accuracy: 0.9645, Loss: 0.0381
Epoch   3 Batch 1005/2154 - Train Accuracy: 0.9638, Validation Accuracy: 0.9645, Loss: 0.0334
Epoch   3 Batch 1010/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9588, Loss: 0.0295
Epoch   3 Batch 1015/2154 - Train Accuracy: 0.8898, Validation Accuracy: 0.9467, Loss: 0.0447
Epoch   3 Batch 1020/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.9339, Loss: 0.0538
Epoch   3 Batch 1025/2154 - Train Accuracy: 0.9766, Validation Accuracy: 0.9368, Loss: 0.0269
Epoch   3 Batch 1030/2154 - Train Accuracy: 0.9441, Validation Accuracy: 0.9403, Loss: 0.0360
Epoch   3 Batch 1035/2154 - Train Accuracy: 0.9903, Validation Accuracy: 0.9524, Loss: 0.0295
Epoch   3 Batch 1040/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9695, Loss: 0.0224
Epoch   3 Batch 1045/2154 - Train Accuracy: 0.9252, Validation Accuracy: 0.9624, Loss: 0.0438
Epoch   3 Batch 1050/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.9517, Loss: 0.0564
Epoch   3 Batch 1055/2154 - Train Accuracy: 0.9128, Validation Accuracy: 0.9659, Loss: 0.0337
Epoch   3 Batch 1060/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9567, Loss: 0.0340
Epoch   3 Batch 1065/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9581, Loss: 0.0482
Epoch   3 Batch 1070/2154 - Train Accuracy: 0.9226, Validation Accuracy: 0.9595, Loss: 0.0426
Epoch   3 Batch 1075/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9524, Loss: 0.0282
Epoch   3 Batch 1080/2154 - Train Accuracy: 0.9305, Validation Accuracy: 0.9616, Loss: 0.0530
Epoch   3 Batch 1085/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9524, Loss: 0.0248
Epoch   3 Batch 1090/2154 - Train Accuracy: 0.9482, Validation Accuracy: 0.9446, Loss: 0.0345
Epoch   3 Batch 1095/2154 - Train Accuracy: 0.9828, Validation Accuracy: 0.9411, Loss: 0.0268
Epoch   3 Batch 1100/2154 - Train Accuracy: 0.9273, Validation Accuracy: 0.9318, Loss: 0.0454
Epoch   3 Batch 1105/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9318, Loss: 0.0302
Epoch   3 Batch 1110/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9396, Loss: 0.0368
Epoch   3 Batch 1115/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9510, Loss: 0.0393
Epoch   3 Batch 1120/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9581, Loss: 0.0313
Epoch   3 Batch 1125/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9396, Loss: 0.0230
Epoch   3 Batch 1130/2154 - Train Accuracy: 0.9474, Validation Accuracy: 0.9553, Loss: 0.0350
Epoch   3 Batch 1135/2154 - Train Accuracy: 0.9141, Validation Accuracy: 0.9439, Loss: 0.0425
Epoch   3 Batch 1140/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.9325, Loss: 0.0353
Epoch   3 Batch 1145/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9318, Loss: 0.0450
Epoch   3 Batch 1150/2154 - Train Accuracy: 0.9326, Validation Accuracy: 0.9226, Loss: 0.0401
Epoch   3 Batch 1155/2154 - Train Accuracy: 0.9572, Validation Accuracy: 0.9261, Loss: 0.0281
Epoch   3 Batch 1160/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9503, Loss: 0.0324
Epoch   3 Batch 1165/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9403, Loss: 0.0364
Epoch   3 Batch 1170/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9389, Loss: 0.0280
Epoch   3 Batch 1175/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9453, Loss: 0.0410
Epoch   3 Batch 1180/2154 - Train Accuracy: 0.9622, Validation Accuracy: 0.9624, Loss: 0.0373
Epoch   3 Batch 1185/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.9631, Loss: 0.0419
Epoch   3 Batch 1190/2154 - Train Accuracy: 0.9742, Validation Accuracy: 0.9766, Loss: 0.0425
Epoch   3 Batch 1195/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.9581, Loss: 0.0307
Epoch   3 Batch 1200/2154 - Train Accuracy: 0.9317, Validation Accuracy: 0.9688, Loss: 0.0593
Epoch   3 Batch 1205/2154 - Train Accuracy: 0.9309, Validation Accuracy: 0.9595, Loss: 0.0397
Epoch   3 Batch 1210/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9616, Loss: 0.0348
Epoch   3 Batch 1215/2154 - Train Accuracy: 0.9827, Validation Accuracy: 0.9616, Loss: 0.0391
Epoch   3 Batch 1220/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9631, Loss: 0.0302
Epoch   3 Batch 1225/2154 - Train Accuracy: 0.9820, Validation Accuracy: 0.9489, Loss: 0.0243
Epoch   3 Batch 1230/2154 - Train Accuracy: 0.9598, Validation Accuracy: 0.9595, Loss: 0.0278
Epoch   3 Batch 1235/2154 - Train Accuracy: 0.9062, Validation Accuracy: 0.9595, Loss: 0.0318
Epoch   3 Batch 1240/2154 - Train Accuracy: 0.9178, Validation Accuracy: 0.9453, Loss: 0.0398
Epoch   3 Batch 1245/2154 - Train Accuracy: 0.9622, Validation Accuracy: 0.9545, Loss: 0.0496
Epoch   3 Batch 1250/2154 - Train Accuracy: 0.9703, Validation Accuracy: 0.9318, Loss: 0.0278
Epoch   3 Batch 1255/2154 - Train Accuracy: 0.9416, Validation Accuracy: 0.9496, Loss: 0.0373
Epoch   3 Batch 1260/2154 - Train Accuracy: 0.9753, Validation Accuracy: 0.9205, Loss: 0.0484
Epoch   3 Batch 1265/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.9624, Loss: 0.0321
Epoch   3 Batch 1270/2154 - Train Accuracy: 0.9546, Validation Accuracy: 0.9666, Loss: 0.0293
Epoch   3 Batch 1275/2154 - Train Accuracy: 0.9414, Validation Accuracy: 0.9638, Loss: 0.0425
Epoch   3 Batch 1280/2154 - Train Accuracy: 0.9400, Validation Accuracy: 0.9425, Loss: 0.0472
Epoch   3 Batch 1285/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9673, Loss: 0.0392
Epoch   3 Batch 1290/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9673, Loss: 0.0245
Epoch   3 Batch 1295/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9659, Loss: 0.0356
Epoch   3 Batch 1300/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9524, Loss: 0.0387
Epoch   3 Batch 1305/2154 - Train Accuracy: 0.9877, Validation Accuracy: 0.9297, Loss: 0.0338
Epoch   3 Batch 1310/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9489, Loss: 0.0379
Epoch   3 Batch 1315/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9531, Loss: 0.0410
Epoch   3 Batch 1320/2154 - Train Accuracy: 0.9442, Validation Accuracy: 0.9616, Loss: 0.0252
Epoch   3 Batch 1325/2154 - Train Accuracy: 0.9784, Validation Accuracy: 0.9688, Loss: 0.0222
Epoch   3 Batch 1330/2154 - Train Accuracy: 0.9594, Validation Accuracy: 0.9766, Loss: 0.0332
Epoch   3 Batch 1335/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9624, Loss: 0.0519
Epoch   3 Batch 1340/2154 - Train Accuracy: 0.9484, Validation Accuracy: 0.9261, Loss: 0.0324
Epoch   3 Batch 1345/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9261, Loss: 0.0398
Epoch   3 Batch 1350/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.9254, Loss: 0.0403
Epoch   3 Batch 1355/2154 - Train Accuracy: 0.9555, Validation Accuracy: 0.9261, Loss: 0.0348
Epoch   3 Batch 1360/2154 - Train Accuracy: 0.9441, Validation Accuracy: 0.9602, Loss: 0.0355
Epoch   3 Batch 1365/2154 - Train Accuracy: 0.9172, Validation Accuracy: 0.9517, Loss: 0.0384
Epoch   3 Batch 1370/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9354, Loss: 0.0329
Epoch   3 Batch 1375/2154 - Train Accuracy: 0.9703, Validation Accuracy: 0.9503, Loss: 0.0443
Epoch   3 Batch 1380/2154 - Train Accuracy: 0.9875, Validation Accuracy: 0.9616, Loss: 0.0311
Epoch   3 Batch 1385/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9467, Loss: 0.0353
Epoch   3 Batch 1390/2154 - Train Accuracy: 0.9353, Validation Accuracy: 0.9574, Loss: 0.0353
Epoch   3 Batch 1395/2154 - Train Accuracy: 0.9186, Validation Accuracy: 0.9744, Loss: 0.0398
Epoch   3 Batch 1400/2154 - Train Accuracy: 0.9671, Validation Accuracy: 0.9595, Loss: 0.0273
Epoch   3 Batch 1405/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.9638, Loss: 0.0519
Epoch   3 Batch 1410/2154 - Train Accuracy: 0.9844, Validation Accuracy: 0.9645, Loss: 0.0356
Epoch   3 Batch 1415/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9545, Loss: 0.0359
Epoch   3 Batch 1420/2154 - Train Accuracy: 0.9317, Validation Accuracy: 0.9411, Loss: 0.0464
Epoch   3 Batch 1425/2154 - Train Accuracy: 0.9828, Validation Accuracy: 0.9574, Loss: 0.0230
Epoch   3 Batch 1430/2154 - Train Accuracy: 0.9754, Validation Accuracy: 0.9581, Loss: 0.0308
Epoch   3 Batch 1435/2154 - Train Accuracy: 0.9877, Validation Accuracy: 0.9574, Loss: 0.0257
Epoch   3 Batch 1440/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9574, Loss: 0.0497
Epoch   3 Batch 1445/2154 - Train Accuracy: 0.9484, Validation Accuracy: 0.9411, Loss: 0.0285
Epoch   3 Batch 1450/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.9439, Loss: 0.0373
Epoch   3 Batch 1455/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9396, Loss: 0.0372
Epoch   3 Batch 1460/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9418, Loss: 0.0439
Epoch   3 Batch 1465/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9560, Loss: 0.0304
Epoch   3 Batch 1470/2154 - Train Accuracy: 0.9548, Validation Accuracy: 0.9553, Loss: 0.0322
Epoch   3 Batch 1475/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.9695, Loss: 0.0482
Epoch   3 Batch 1480/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9702, Loss: 0.0323
Epoch   3 Batch 1485/2154 - Train Accuracy: 0.9594, Validation Accuracy: 0.9723, Loss: 0.0309
Epoch   3 Batch 1490/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9389, Loss: 0.0379
Epoch   3 Batch 1495/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9169, Loss: 0.0257
Epoch   3 Batch 1500/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9375, Loss: 0.0350
Epoch   3 Batch 1505/2154 - Train Accuracy: 0.9490, Validation Accuracy: 0.9368, Loss: 0.0326
Epoch   3 Batch 1510/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9482, Loss: 0.0337
Epoch   3 Batch 1515/2154 - Train Accuracy: 0.9556, Validation Accuracy: 0.9517, Loss: 0.0275
Epoch   3 Batch 1520/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9418, Loss: 0.0265
Epoch   3 Batch 1525/2154 - Train Accuracy: 0.9812, Validation Accuracy: 0.9503, Loss: 0.0317
Epoch   3 Batch 1530/2154 - Train Accuracy: 0.9507, Validation Accuracy: 0.9396, Loss: 0.0375
Epoch   3 Batch 1535/2154 - Train Accuracy: 0.9844, Validation Accuracy: 0.9482, Loss: 0.0266
Epoch   3 Batch 1540/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9531, Loss: 0.0409
Epoch   3 Batch 1545/2154 - Train Accuracy: 0.9554, Validation Accuracy: 0.9673, Loss: 0.0358
Epoch   3 Batch 1550/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.9432, Loss: 0.0405
Epoch   3 Batch 1555/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9396, Loss: 0.0373
Epoch   3 Batch 1560/2154 - Train Accuracy: 0.9195, Validation Accuracy: 0.9489, Loss: 0.0345
Epoch   3 Batch 1565/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9609, Loss: 0.0354
Epoch   3 Batch 1570/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9645, Loss: 0.0334
Epoch   3 Batch 1575/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9616, Loss: 0.0288
Epoch   3 Batch 1580/2154 - Train Accuracy: 0.9572, Validation Accuracy: 0.9616, Loss: 0.0437
Epoch   3 Batch 1585/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.9730, Loss: 0.0373
Epoch   3 Batch 1590/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9659, Loss: 0.0269
Epoch   3 Batch 1595/2154 - Train Accuracy: 0.9656, Validation Accuracy: 0.9659, Loss: 0.0297
Epoch   3 Batch 1600/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9588, Loss: 0.0388
Epoch   3 Batch 1605/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9446, Loss: 0.0362
Epoch   3 Batch 1610/2154 - Train Accuracy: 0.9867, Validation Accuracy: 0.9773, Loss: 0.0170
Epoch   3 Batch 1615/2154 - Train Accuracy: 0.9758, Validation Accuracy: 0.9737, Loss: 0.0180
Epoch   3 Batch 1620/2154 - Train Accuracy: 0.9441, Validation Accuracy: 0.9666, Loss: 0.0606
Epoch   3 Batch 1625/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.9574, Loss: 0.0297
Epoch   3 Batch 1630/2154 - Train Accuracy: 0.9441, Validation Accuracy: 0.9460, Loss: 0.0310
Epoch   3 Batch 1635/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9567, Loss: 0.0368
Epoch   3 Batch 1640/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9688, Loss: 0.0314
Epoch   3 Batch 1645/2154 - Train Accuracy: 0.9860, Validation Accuracy: 0.9837, Loss: 0.0238
Epoch   3 Batch 1650/2154 - Train Accuracy: 0.9524, Validation Accuracy: 0.9631, Loss: 0.0381
Epoch   3 Batch 1655/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9780, Loss: 0.0335
Epoch   3 Batch 1660/2154 - Train Accuracy: 0.9712, Validation Accuracy: 0.9886, Loss: 0.0465
Epoch   3 Batch 1665/2154 - Train Accuracy: 0.9812, Validation Accuracy: 0.9673, Loss: 0.0271
Epoch   3 Batch 1670/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9773, Loss: 0.0334
Epoch   3 Batch 1675/2154 - Train Accuracy: 0.9430, Validation Accuracy: 0.9609, Loss: 0.0416
Epoch   3 Batch 1680/2154 - Train Accuracy: 0.9773, Validation Accuracy: 0.9609, Loss: 0.0256
Epoch   3 Batch 1685/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.9666, Loss: 0.0276
Epoch   3 Batch 1690/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9574, Loss: 0.0307
Epoch   3 Batch 1695/2154 - Train Accuracy: 0.9679, Validation Accuracy: 0.9652, Loss: 0.0402
Epoch   3 Batch 1700/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9659, Loss: 0.0343
Epoch   3 Batch 1705/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9631, Loss: 0.0562
Epoch   3 Batch 1710/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9602, Loss: 0.0423
Epoch   3 Batch 1715/2154 - Train Accuracy: 0.9235, Validation Accuracy: 0.9418, Loss: 0.0362
Epoch   3 Batch 1720/2154 - Train Accuracy: 0.9770, Validation Accuracy: 0.9560, Loss: 0.0381
Epoch   3 Batch 1725/2154 - Train Accuracy: 0.9079, Validation Accuracy: 0.9482, Loss: 0.0427
Epoch   3 Batch 1730/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9482, Loss: 0.0318
Epoch   3 Batch 1735/2154 - Train Accuracy: 0.9342, Validation Accuracy: 0.9482, Loss: 0.0455
Epoch   3 Batch 1740/2154 - Train Accuracy: 0.9408, Validation Accuracy: 0.9489, Loss: 0.0356
Epoch   3 Batch 1745/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9403, Loss: 0.0358
Epoch   3 Batch 1750/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9276, Loss: 0.0401
Epoch   3 Batch 1755/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9254, Loss: 0.0302
Epoch   3 Batch 1760/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9325, Loss: 0.0333
Epoch   3 Batch 1765/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9439, Loss: 0.0336
Epoch   3 Batch 1770/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9545, Loss: 0.0366
Epoch   3 Batch 1775/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9688, Loss: 0.0309
Epoch   3 Batch 1780/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9666, Loss: 0.0305
Epoch   3 Batch 1785/2154 - Train Accuracy: 0.9934, Validation Accuracy: 0.9538, Loss: 0.0249
Epoch   3 Batch 1790/2154 - Train Accuracy: 0.9784, Validation Accuracy: 0.9524, Loss: 0.0250
Epoch   3 Batch 1795/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9631, Loss: 0.0277
Epoch   3 Batch 1800/2154 - Train Accuracy: 0.9711, Validation Accuracy: 0.9766, Loss: 0.0349
Epoch   3 Batch 1805/2154 - Train Accuracy: 0.9023, Validation Accuracy: 0.9574, Loss: 0.0398
Epoch   3 Batch 1810/2154 - Train Accuracy: 0.8992, Validation Accuracy: 0.9517, Loss: 0.0371
Epoch   3 Batch 1815/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9531, Loss: 0.0320
Epoch   3 Batch 1820/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9723, Loss: 0.0263
Epoch   3 Batch 1825/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9361, Loss: 0.0343
Epoch   3 Batch 1830/2154 - Train Accuracy: 0.9512, Validation Accuracy: 0.9403, Loss: 0.0660
Epoch   3 Batch 1835/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9595, Loss: 0.0367
Epoch   3 Batch 1840/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9332, Loss: 0.0259
Epoch   3 Batch 1845/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9233, Loss: 0.0357
Epoch   3 Batch 1850/2154 - Train Accuracy: 0.9515, Validation Accuracy: 0.9609, Loss: 0.0428
Epoch   3 Batch 1855/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.9652, Loss: 0.0435
Epoch   3 Batch 1860/2154 - Train Accuracy: 0.9556, Validation Accuracy: 0.9560, Loss: 0.0310
Epoch   3 Batch 1865/2154 - Train Accuracy: 0.9548, Validation Accuracy: 0.9574, Loss: 0.0208
Epoch   3 Batch 1870/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9652, Loss: 0.0292
Epoch   3 Batch 1875/2154 - Train Accuracy: 0.9548, Validation Accuracy: 0.9574, Loss: 0.0500
Epoch   3 Batch 1880/2154 - Train Accuracy: 0.9430, Validation Accuracy: 0.9574, Loss: 0.0381
Epoch   3 Batch 1885/2154 - Train Accuracy: 0.9317, Validation Accuracy: 0.9545, Loss: 0.0403
Epoch   3 Batch 1890/2154 - Train Accuracy: 0.9628, Validation Accuracy: 0.9567, Loss: 0.0248
Epoch   3 Batch 1895/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9553, Loss: 0.0297
Epoch   3 Batch 1900/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9545, Loss: 0.0287
Epoch   3 Batch 1905/2154 - Train Accuracy: 0.9852, Validation Accuracy: 0.9680, Loss: 0.0255
Epoch   3 Batch 1910/2154 - Train Accuracy: 0.9844, Validation Accuracy: 0.9347, Loss: 0.0193
Epoch   3 Batch 1915/2154 - Train Accuracy: 0.9875, Validation Accuracy: 0.9467, Loss: 0.0217
Epoch   3 Batch 1920/2154 - Train Accuracy: 0.9581, Validation Accuracy: 0.9574, Loss: 0.0324
Epoch   3 Batch 1925/2154 - Train Accuracy: 0.9605, Validation Accuracy: 0.9702, Loss: 0.0357
Epoch   3 Batch 1930/2154 - Train Accuracy: 0.9930, Validation Accuracy: 0.9659, Loss: 0.0301
Epoch   3 Batch 1935/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9467, Loss: 0.0389
Epoch   3 Batch 1940/2154 - Train Accuracy: 0.9524, Validation Accuracy: 0.9808, Loss: 0.0312
Epoch   3 Batch 1945/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9375, Loss: 0.0259
Epoch   3 Batch 1950/2154 - Train Accuracy: 0.9672, Validation Accuracy: 0.9652, Loss: 0.0259
Epoch   3 Batch 1955/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9538, Loss: 0.0245
Epoch   3 Batch 1960/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9489, Loss: 0.0406
Epoch   3 Batch 1965/2154 - Train Accuracy: 0.9308, Validation Accuracy: 0.9375, Loss: 0.0353
Epoch   3 Batch 1970/2154 - Train Accuracy: 0.9406, Validation Accuracy: 0.9638, Loss: 0.0351
Epoch   3 Batch 1975/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.9645, Loss: 0.0301
Epoch   3 Batch 1980/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9545, Loss: 0.0345
Epoch   3 Batch 1985/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9538, Loss: 0.0423
Epoch   3 Batch 1990/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9496, Loss: 0.0271
Epoch   3 Batch 1995/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9489, Loss: 0.0468
Epoch   3 Batch 2000/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9190, Loss: 0.0462
Epoch   3 Batch 2005/2154 - Train Accuracy: 0.9581, Validation Accuracy: 0.9091, Loss: 0.0249
Epoch   3 Batch 2010/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9467, Loss: 0.0522
Epoch   3 Batch 2015/2154 - Train Accuracy: 0.9606, Validation Accuracy: 0.9403, Loss: 0.0347
Epoch   3 Batch 2020/2154 - Train Accuracy: 0.9663, Validation Accuracy: 0.9304, Loss: 0.0212
Epoch   3 Batch 2025/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9339, Loss: 0.0259
Epoch   3 Batch 2030/2154 - Train Accuracy: 0.9602, Validation Accuracy: 0.9332, Loss: 0.0307
Epoch   3 Batch 2035/2154 - Train Accuracy: 0.9786, Validation Accuracy: 0.9332, Loss: 0.0229
Epoch   3 Batch 2040/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9290, Loss: 0.0343
Epoch   3 Batch 2045/2154 - Train Accuracy: 0.9754, Validation Accuracy: 0.9432, Loss: 0.0270
Epoch   3 Batch 2050/2154 - Train Accuracy: 0.9301, Validation Accuracy: 0.9183, Loss: 0.0481
Epoch   3 Batch 2055/2154 - Train Accuracy: 0.8939, Validation Accuracy: 0.9183, Loss: 0.0350
Epoch   3 Batch 2060/2154 - Train Accuracy: 0.9679, Validation Accuracy: 0.9375, Loss: 0.0313
Epoch   3 Batch 2065/2154 - Train Accuracy: 0.9555, Validation Accuracy: 0.9467, Loss: 0.0265
Epoch   3 Batch 2070/2154 - Train Accuracy: 0.9258, Validation Accuracy: 0.9467, Loss: 0.0337
Epoch   3 Batch 2075/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9531, Loss: 0.0356
Epoch   3 Batch 2080/2154 - Train Accuracy: 0.9345, Validation Accuracy: 0.9624, Loss: 0.0339
Epoch   3 Batch 2085/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9567, Loss: 0.0317
Epoch   3 Batch 2090/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.9482, Loss: 0.0383
Epoch   3 Batch 2095/2154 - Train Accuracy: 0.9875, Validation Accuracy: 0.9595, Loss: 0.0247
Epoch   3 Batch 2100/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9759, Loss: 0.0253
Epoch   3 Batch 2105/2154 - Train Accuracy: 0.9663, Validation Accuracy: 0.9730, Loss: 0.0353
Epoch   3 Batch 2110/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.9503, Loss: 0.0311
Epoch   3 Batch 2115/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9503, Loss: 0.0295
Epoch   3 Batch 2120/2154 - Train Accuracy: 0.9309, Validation Accuracy: 0.9425, Loss: 0.0322
Epoch   3 Batch 2125/2154 - Train Accuracy: 0.9328, Validation Accuracy: 0.9389, Loss: 0.0342
Epoch   3 Batch 2130/2154 - Train Accuracy: 0.9712, Validation Accuracy: 0.9396, Loss: 0.0270
Epoch   3 Batch 2135/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.9517, Loss: 0.0401
Epoch   3 Batch 2140/2154 - Train Accuracy: 0.9710, Validation Accuracy: 0.9716, Loss: 0.0182
Epoch   3 Batch 2145/2154 - Train Accuracy: 0.9827, Validation Accuracy: 0.9631, Loss: 0.0324
Epoch   3 Batch 2150/2154 - Train Accuracy: 0.9628, Validation Accuracy: 0.9666, Loss: 0.0347
Epoch   4 Batch    5/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9588, Loss: 0.0326
Epoch   4 Batch   10/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9581, Loss: 0.0318
Epoch   4 Batch   15/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9553, Loss: 0.0277
Epoch   4 Batch   20/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.9602, Loss: 0.0325
Epoch   4 Batch   25/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9659, Loss: 0.0331
Epoch   4 Batch   30/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9751, Loss: 0.0225
Epoch   4 Batch   35/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9751, Loss: 0.0364
Epoch   4 Batch   40/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9652, Loss: 0.0225
Epoch   4 Batch   45/2154 - Train Accuracy: 0.9605, Validation Accuracy: 0.9759, Loss: 0.0385
Epoch   4 Batch   50/2154 - Train Accuracy: 0.9424, Validation Accuracy: 0.9567, Loss: 0.0496
Epoch   4 Batch   55/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9602, Loss: 0.0229
Epoch   4 Batch   60/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9709, Loss: 0.0268
Epoch   4 Batch   65/2154 - Train Accuracy: 0.9717, Validation Accuracy: 0.9716, Loss: 0.0304
Epoch   4 Batch   70/2154 - Train Accuracy: 0.9638, Validation Accuracy: 0.9709, Loss: 0.0352
Epoch   4 Batch   75/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9624, Loss: 0.0307
Epoch   4 Batch   80/2154 - Train Accuracy: 0.9430, Validation Accuracy: 0.9616, Loss: 0.0342
Epoch   4 Batch   85/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9588, Loss: 0.0487
Epoch   4 Batch   90/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.9460, Loss: 0.0301
Epoch   4 Batch   95/2154 - Train Accuracy: 0.9867, Validation Accuracy: 0.9467, Loss: 0.0274
Epoch   4 Batch  100/2154 - Train Accuracy: 0.9472, Validation Accuracy: 0.9375, Loss: 0.0567
Epoch   4 Batch  105/2154 - Train Accuracy: 0.9289, Validation Accuracy: 0.9375, Loss: 0.0444
Epoch   4 Batch  110/2154 - Train Accuracy: 0.9422, Validation Accuracy: 0.9510, Loss: 0.0629
Epoch   4 Batch  115/2154 - Train Accuracy: 0.9389, Validation Accuracy: 0.9595, Loss: 0.0327
Epoch   4 Batch  120/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9467, Loss: 0.0250
Epoch   4 Batch  125/2154 - Train Accuracy: 0.9424, Validation Accuracy: 0.9567, Loss: 0.0278
Epoch   4 Batch  130/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9567, Loss: 0.0254
Epoch   4 Batch  135/2154 - Train Accuracy: 0.9774, Validation Accuracy: 0.9560, Loss: 0.0269
Epoch   4 Batch  140/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9560, Loss: 0.0409
Epoch   4 Batch  145/2154 - Train Accuracy: 0.9508, Validation Accuracy: 0.9645, Loss: 0.0360
Epoch   4 Batch  150/2154 - Train Accuracy: 0.9732, Validation Accuracy: 0.9645, Loss: 0.0277
Epoch   4 Batch  155/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9602, Loss: 0.0315
Epoch   4 Batch  160/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9169, Loss: 0.0307
Epoch   4 Batch  165/2154 - Train Accuracy: 0.9304, Validation Accuracy: 0.9325, Loss: 0.0340
Epoch   4 Batch  170/2154 - Train Accuracy: 0.9664, Validation Accuracy: 0.9418, Loss: 0.0278
Epoch   4 Batch  175/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9553, Loss: 0.0334
Epoch   4 Batch  180/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9574, Loss: 0.0259
Epoch   4 Batch  185/2154 - Train Accuracy: 0.9548, Validation Accuracy: 0.9538, Loss: 0.0288
Epoch   4 Batch  190/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9638, Loss: 0.0305
Epoch   4 Batch  195/2154 - Train Accuracy: 0.9145, Validation Accuracy: 0.9773, Loss: 0.0340
Epoch   4 Batch  200/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9553, Loss: 0.0156
Epoch   4 Batch  205/2154 - Train Accuracy: 0.9867, Validation Accuracy: 0.9553, Loss: 0.0208
Epoch   4 Batch  210/2154 - Train Accuracy: 0.9836, Validation Accuracy: 0.9474, Loss: 0.0243
Epoch   4 Batch  215/2154 - Train Accuracy: 0.9241, Validation Accuracy: 0.9368, Loss: 0.0355
Epoch   4 Batch  220/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9624, Loss: 0.0245
Epoch   4 Batch  225/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9354, Loss: 0.0205
Epoch   4 Batch  230/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9375, Loss: 0.0305
Epoch   4 Batch  235/2154 - Train Accuracy: 0.9773, Validation Accuracy: 0.9425, Loss: 0.0235
Epoch   4 Batch  240/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9439, Loss: 0.0315
Epoch   4 Batch  245/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9467, Loss: 0.0351
Epoch   4 Batch  250/2154 - Train Accuracy: 0.9586, Validation Accuracy: 0.9496, Loss: 0.0327
Epoch   4 Batch  255/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9553, Loss: 0.0250
Epoch   4 Batch  260/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.9418, Loss: 0.0356
Epoch   4 Batch  265/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9347, Loss: 0.0307
Epoch   4 Batch  270/2154 - Train Accuracy: 0.9814, Validation Accuracy: 0.9489, Loss: 0.0261
Epoch   4 Batch  275/2154 - Train Accuracy: 0.9844, Validation Accuracy: 0.9467, Loss: 0.0227
Epoch   4 Batch  280/2154 - Train Accuracy: 0.9317, Validation Accuracy: 0.9602, Loss: 0.0392
Epoch   4 Batch  285/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9581, Loss: 0.0313
Epoch   4 Batch  290/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.9560, Loss: 0.0421
Epoch   4 Batch  295/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9645, Loss: 0.0212
Epoch   4 Batch  300/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.9666, Loss: 0.0301
Epoch   4 Batch  305/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9574, Loss: 0.0448
Epoch   4 Batch  310/2154 - Train Accuracy: 0.9712, Validation Accuracy: 0.9751, Loss: 0.0235
Epoch   4 Batch  315/2154 - Train Accuracy: 0.9797, Validation Accuracy: 0.9858, Loss: 0.0169
Epoch   4 Batch  320/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9773, Loss: 0.0183
Epoch   4 Batch  325/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.9595, Loss: 0.0399
Epoch   4 Batch  330/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9560, Loss: 0.0219
Epoch   4 Batch  335/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9503, Loss: 0.0257
Epoch   4 Batch  340/2154 - Train Accuracy: 0.9665, Validation Accuracy: 0.9524, Loss: 0.0248
Epoch   4 Batch  345/2154 - Train Accuracy: 0.9663, Validation Accuracy: 0.9496, Loss: 0.0325
Epoch   4 Batch  350/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.9247, Loss: 0.0369
Epoch   4 Batch  355/2154 - Train Accuracy: 0.9737, Validation Accuracy: 0.9489, Loss: 0.0411
Epoch   4 Batch  360/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9680, Loss: 0.0303
Epoch   4 Batch  365/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9680, Loss: 0.0439
Epoch   4 Batch  370/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9560, Loss: 0.0357
Epoch   4 Batch  375/2154 - Train Accuracy: 0.9704, Validation Accuracy: 0.9560, Loss: 0.0279
Epoch   4 Batch  380/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9432, Loss: 0.0246
Epoch   4 Batch  385/2154 - Train Accuracy: 0.9352, Validation Accuracy: 0.9737, Loss: 0.0293
Epoch   4 Batch  390/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9744, Loss: 0.0241
Epoch   4 Batch  395/2154 - Train Accuracy: 0.9836, Validation Accuracy: 0.9730, Loss: 0.0255
Epoch   4 Batch  400/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9730, Loss: 0.0222
Epoch   4 Batch  405/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9744, Loss: 0.0310
Epoch   4 Batch  410/2154 - Train Accuracy: 0.9383, Validation Accuracy: 0.9794, Loss: 0.0318
Epoch   4 Batch  415/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9787, Loss: 0.0364
Epoch   4 Batch  420/2154 - Train Accuracy: 0.9777, Validation Accuracy: 0.9787, Loss: 0.0213
Epoch   4 Batch  425/2154 - Train Accuracy: 0.9747, Validation Accuracy: 0.9780, Loss: 0.0220
Epoch   4 Batch  430/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9688, Loss: 0.0330
Epoch   4 Batch  435/2154 - Train Accuracy: 0.9766, Validation Accuracy: 0.9688, Loss: 0.0258
Epoch   4 Batch  440/2154 - Train Accuracy: 0.9819, Validation Accuracy: 0.9638, Loss: 0.0197
Epoch   4 Batch  445/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9638, Loss: 0.0285
Epoch   4 Batch  450/2154 - Train Accuracy: 0.9742, Validation Accuracy: 0.9638, Loss: 0.0261
Epoch   4 Batch  455/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9560, Loss: 0.0444
Epoch   4 Batch  460/2154 - Train Accuracy: 0.9422, Validation Accuracy: 0.9574, Loss: 0.0278
Epoch   4 Batch  465/2154 - Train Accuracy: 0.9441, Validation Accuracy: 0.9531, Loss: 0.0278
Epoch   4 Batch  470/2154 - Train Accuracy: 0.9323, Validation Accuracy: 0.9702, Loss: 0.0379
Epoch   4 Batch  475/2154 - Train Accuracy: 0.9548, Validation Accuracy: 0.9538, Loss: 0.0278
Epoch   4 Batch  480/2154 - Train Accuracy: 0.9881, Validation Accuracy: 0.9538, Loss: 0.0206
Epoch   4 Batch  485/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9311, Loss: 0.0224
Epoch   4 Batch  490/2154 - Train Accuracy: 0.9727, Validation Accuracy: 0.9361, Loss: 0.0308
Epoch   4 Batch  495/2154 - Train Accuracy: 0.9487, Validation Accuracy: 0.9403, Loss: 0.0325
Epoch   4 Batch  500/2154 - Train Accuracy: 0.9910, Validation Accuracy: 0.9389, Loss: 0.0355
Epoch   4 Batch  505/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.9432, Loss: 0.0515
Epoch   4 Batch  510/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9517, Loss: 0.0356
Epoch   4 Batch  515/2154 - Train Accuracy: 0.9204, Validation Accuracy: 0.9602, Loss: 0.0269
Epoch   4 Batch  520/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9602, Loss: 0.0210
Epoch   4 Batch  525/2154 - Train Accuracy: 0.9953, Validation Accuracy: 0.9638, Loss: 0.0203
Epoch   4 Batch  530/2154 - Train Accuracy: 0.9789, Validation Accuracy: 0.9688, Loss: 0.0245
Epoch   4 Batch  535/2154 - Train Accuracy: 0.9709, Validation Accuracy: 0.9652, Loss: 0.0324
Epoch   4 Batch  540/2154 - Train Accuracy: 0.9737, Validation Accuracy: 0.9474, Loss: 0.0253
Epoch   4 Batch  545/2154 - Train Accuracy: 0.9474, Validation Accuracy: 0.9474, Loss: 0.0393
Epoch   4 Batch  550/2154 - Train Accuracy: 0.9196, Validation Accuracy: 0.9574, Loss: 0.0384
Epoch   4 Batch  555/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9567, Loss: 0.0275
Epoch   4 Batch  560/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9574, Loss: 0.0259
Epoch   4 Batch  565/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9631, Loss: 0.0359
Epoch   4 Batch  570/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9631, Loss: 0.0314
Epoch   4 Batch  575/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9588, Loss: 0.0378
Epoch   4 Batch  580/2154 - Train Accuracy: 0.9581, Validation Accuracy: 0.9616, Loss: 0.0264
Epoch   4 Batch  585/2154 - Train Accuracy: 0.9725, Validation Accuracy: 0.9616, Loss: 0.0315
Epoch   4 Batch  590/2154 - Train Accuracy: 0.9567, Validation Accuracy: 0.9631, Loss: 0.0300
Epoch   4 Batch  595/2154 - Train Accuracy: 0.9766, Validation Accuracy: 0.9553, Loss: 0.0252
Epoch   4 Batch  600/2154 - Train Accuracy: 0.9572, Validation Accuracy: 0.9588, Loss: 0.0214
Epoch   4 Batch  605/2154 - Train Accuracy: 0.9766, Validation Accuracy: 0.9432, Loss: 0.0358
Epoch   4 Batch  610/2154 - Train Accuracy: 0.9509, Validation Accuracy: 0.9531, Loss: 0.0235
Epoch   4 Batch  615/2154 - Train Accuracy: 0.9427, Validation Accuracy: 0.9411, Loss: 0.0278
Epoch   4 Batch  620/2154 - Train Accuracy: 0.9464, Validation Accuracy: 0.9411, Loss: 0.0255
Epoch   4 Batch  625/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9425, Loss: 0.0330
Epoch   4 Batch  630/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9531, Loss: 0.0282
Epoch   4 Batch  635/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9425, Loss: 0.0352
Epoch   4 Batch  640/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9425, Loss: 0.0342
Epoch   4 Batch  645/2154 - Train Accuracy: 0.9561, Validation Accuracy: 0.9411, Loss: 0.0290
Epoch   4 Batch  650/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.9560, Loss: 0.0335
Epoch   4 Batch  655/2154 - Train Accuracy: 0.9556, Validation Accuracy: 0.9432, Loss: 0.0395
Epoch   4 Batch  660/2154 - Train Accuracy: 0.9344, Validation Accuracy: 0.9446, Loss: 0.0354
Epoch   4 Batch  665/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9574, Loss: 0.0223
Epoch   4 Batch  670/2154 - Train Accuracy: 0.9836, Validation Accuracy: 0.9510, Loss: 0.0238
Epoch   4 Batch  675/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9375, Loss: 0.0326
Epoch   4 Batch  680/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9411, Loss: 0.0253
Epoch   4 Batch  685/2154 - Train Accuracy: 0.9703, Validation Accuracy: 0.9517, Loss: 0.0233
Epoch   4 Batch  690/2154 - Train Accuracy: 0.9273, Validation Accuracy: 0.9524, Loss: 0.0446
Epoch   4 Batch  695/2154 - Train Accuracy: 0.9938, Validation Accuracy: 0.9801, Loss: 0.0243
Epoch   4 Batch  700/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9609, Loss: 0.0278
Epoch   4 Batch  705/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9616, Loss: 0.0256
Epoch   4 Batch  710/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9744, Loss: 0.0241
Epoch   4 Batch  715/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9751, Loss: 0.0287
Epoch   4 Batch  720/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9759, Loss: 0.0246
Epoch   4 Batch  725/2154 - Train Accuracy: 0.9643, Validation Accuracy: 0.9616, Loss: 0.0260
Epoch   4 Batch  730/2154 - Train Accuracy: 0.9408, Validation Accuracy: 0.9616, Loss: 0.0475
Epoch   4 Batch  735/2154 - Train Accuracy: 0.9866, Validation Accuracy: 0.9645, Loss: 0.0171
Epoch   4 Batch  740/2154 - Train Accuracy: 0.9414, Validation Accuracy: 0.9801, Loss: 0.0354
Epoch   4 Batch  745/2154 - Train Accuracy: 0.9901, Validation Accuracy: 0.9822, Loss: 0.0263
Epoch   4 Batch  750/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9759, Loss: 0.0285
Epoch   4 Batch  755/2154 - Train Accuracy: 0.9424, Validation Accuracy: 0.9709, Loss: 0.0334
Epoch   4 Batch  760/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9645, Loss: 0.0329
Epoch   4 Batch  765/2154 - Train Accuracy: 0.9769, Validation Accuracy: 0.9567, Loss: 0.0418
Epoch   4 Batch  770/2154 - Train Accuracy: 0.9820, Validation Accuracy: 0.9403, Loss: 0.0209
Epoch   4 Batch  775/2154 - Train Accuracy: 0.9729, Validation Accuracy: 0.9503, Loss: 0.0249
Epoch   4 Batch  780/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9510, Loss: 0.0185
Epoch   4 Batch  785/2154 - Train Accuracy: 0.9852, Validation Accuracy: 0.9645, Loss: 0.0199
Epoch   4 Batch  790/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.9538, Loss: 0.0199
Epoch   4 Batch  795/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9545, Loss: 0.0277
Epoch   4 Batch  800/2154 - Train Accuracy: 0.9712, Validation Accuracy: 0.9496, Loss: 0.0283
Epoch   4 Batch  805/2154 - Train Accuracy: 1.0000, Validation Accuracy: 0.9503, Loss: 0.0139
Epoch   4 Batch  810/2154 - Train Accuracy: 0.9814, Validation Accuracy: 0.9482, Loss: 0.0204
Epoch   4 Batch  815/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9751, Loss: 0.0307
Epoch   4 Batch  820/2154 - Train Accuracy: 0.9729, Validation Accuracy: 0.9538, Loss: 0.0393
Epoch   4 Batch  825/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9645, Loss: 0.0237
Epoch   4 Batch  830/2154 - Train Accuracy: 0.9164, Validation Accuracy: 0.9609, Loss: 0.0378
Epoch   4 Batch  835/2154 - Train Accuracy: 0.9350, Validation Accuracy: 0.9716, Loss: 0.0576
Epoch   4 Batch  840/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9645, Loss: 0.0324
Epoch   4 Batch  845/2154 - Train Accuracy: 0.9462, Validation Accuracy: 0.9560, Loss: 0.0296
Epoch   4 Batch  850/2154 - Train Accuracy: 0.9445, Validation Accuracy: 0.9560, Loss: 0.0247
Epoch   4 Batch  855/2154 - Train Accuracy: 0.9672, Validation Accuracy: 0.9517, Loss: 0.0368
Epoch   4 Batch  860/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9503, Loss: 0.0236
Epoch   4 Batch  865/2154 - Train Accuracy: 0.9313, Validation Accuracy: 0.9517, Loss: 0.0303
Epoch   4 Batch  870/2154 - Train Accuracy: 0.9867, Validation Accuracy: 0.9425, Loss: 0.0225
Epoch   4 Batch  875/2154 - Train Accuracy: 0.9704, Validation Accuracy: 0.9524, Loss: 0.0261
Epoch   4 Batch  880/2154 - Train Accuracy: 0.9656, Validation Accuracy: 0.9474, Loss: 0.0444
Epoch   4 Batch  885/2154 - Train Accuracy: 0.9315, Validation Accuracy: 0.9524, Loss: 0.0461
Epoch   4 Batch  890/2154 - Train Accuracy: 0.9671, Validation Accuracy: 0.9474, Loss: 0.0345
Epoch   4 Batch  895/2154 - Train Accuracy: 0.9227, Validation Accuracy: 0.9517, Loss: 0.0303
Epoch   4 Batch  900/2154 - Train Accuracy: 0.9745, Validation Accuracy: 0.9517, Loss: 0.0328
Epoch   4 Batch  905/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9567, Loss: 0.0236
Epoch   4 Batch  910/2154 - Train Accuracy: 0.9622, Validation Accuracy: 0.9780, Loss: 0.0293
Epoch   4 Batch  915/2154 - Train Accuracy: 0.9591, Validation Accuracy: 0.9780, Loss: 0.0204
Epoch   4 Batch  920/2154 - Train Accuracy: 0.9628, Validation Accuracy: 0.9680, Loss: 0.0191
Epoch   4 Batch  925/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9602, Loss: 0.0281
Epoch   4 Batch  930/2154 - Train Accuracy: 0.9500, Validation Accuracy: 0.9517, Loss: 0.0338
Epoch   4 Batch  935/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9531, Loss: 0.0362
Epoch   4 Batch  940/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9709, Loss: 0.0237
Epoch   4 Batch  945/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9609, Loss: 0.0223
Epoch   4 Batch  950/2154 - Train Accuracy: 0.9803, Validation Accuracy: 0.9609, Loss: 0.0299
Epoch   4 Batch  955/2154 - Train Accuracy: 0.9592, Validation Accuracy: 0.9524, Loss: 0.0281
Epoch   4 Batch  960/2154 - Train Accuracy: 0.9406, Validation Accuracy: 0.9538, Loss: 0.0355
Epoch   4 Batch  965/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9489, Loss: 0.0327
Epoch   4 Batch  970/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9517, Loss: 0.0334
Epoch   4 Batch  975/2154 - Train Accuracy: 0.9819, Validation Accuracy: 0.9510, Loss: 0.0242
Epoch   4 Batch  980/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9510, Loss: 0.0273
Epoch   4 Batch  985/2154 - Train Accuracy: 0.9305, Validation Accuracy: 0.9624, Loss: 0.0431
Epoch   4 Batch  990/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.9645, Loss: 0.0258
Epoch   4 Batch  995/2154 - Train Accuracy: 0.9712, Validation Accuracy: 0.9553, Loss: 0.0238
Epoch   4 Batch 1000/2154 - Train Accuracy: 0.9762, Validation Accuracy: 0.9609, Loss: 0.0266
Epoch   4 Batch 1005/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9666, Loss: 0.0226
Epoch   4 Batch 1010/2154 - Train Accuracy: 0.9398, Validation Accuracy: 0.9723, Loss: 0.0258
Epoch   4 Batch 1015/2154 - Train Accuracy: 0.9202, Validation Accuracy: 0.9609, Loss: 0.0390
Epoch   4 Batch 1020/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9624, Loss: 0.0425
Epoch   4 Batch 1025/2154 - Train Accuracy: 0.9812, Validation Accuracy: 0.9496, Loss: 0.0216
Epoch   4 Batch 1030/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9496, Loss: 0.0281
Epoch   4 Batch 1035/2154 - Train Accuracy: 0.9888, Validation Accuracy: 0.9510, Loss: 0.0220
Epoch   4 Batch 1040/2154 - Train Accuracy: 0.9758, Validation Accuracy: 0.9695, Loss: 0.0197
Epoch   4 Batch 1045/2154 - Train Accuracy: 0.9252, Validation Accuracy: 0.9666, Loss: 0.0346
Epoch   4 Batch 1050/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9553, Loss: 0.0506
Epoch   4 Batch 1055/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9794, Loss: 0.0330
Epoch   4 Batch 1060/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9879, Loss: 0.0229
Epoch   4 Batch 1065/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9822, Loss: 0.0347
Epoch   4 Batch 1070/2154 - Train Accuracy: 0.9412, Validation Accuracy: 0.9780, Loss: 0.0253
Epoch   4 Batch 1075/2154 - Train Accuracy: 0.9758, Validation Accuracy: 0.9680, Loss: 0.0236
Epoch   4 Batch 1080/2154 - Train Accuracy: 0.9219, Validation Accuracy: 0.9680, Loss: 0.0443
Epoch   4 Batch 1085/2154 - Train Accuracy: 0.9836, Validation Accuracy: 0.9659, Loss: 0.0183
Epoch   4 Batch 1090/2154 - Train Accuracy: 0.9622, Validation Accuracy: 0.9588, Loss: 0.0223
Epoch   4 Batch 1095/2154 - Train Accuracy: 0.9969, Validation Accuracy: 0.9588, Loss: 0.0258
Epoch   4 Batch 1100/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9560, Loss: 0.0373
Epoch   4 Batch 1105/2154 - Train Accuracy: 0.9664, Validation Accuracy: 0.9361, Loss: 0.0290
Epoch   4 Batch 1110/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9695, Loss: 0.0318
Epoch   4 Batch 1115/2154 - Train Accuracy: 0.9474, Validation Accuracy: 0.9688, Loss: 0.0259
Epoch   4 Batch 1120/2154 - Train Accuracy: 0.9516, Validation Accuracy: 0.9688, Loss: 0.0247
Epoch   4 Batch 1125/2154 - Train Accuracy: 0.9416, Validation Accuracy: 0.9688, Loss: 0.0190
Epoch   4 Batch 1130/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9560, Loss: 0.0286
Epoch   4 Batch 1135/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9574, Loss: 0.0267
Epoch   4 Batch 1140/2154 - Train Accuracy: 0.9555, Validation Accuracy: 0.9652, Loss: 0.0271
Epoch   4 Batch 1145/2154 - Train Accuracy: 0.9679, Validation Accuracy: 0.9581, Loss: 0.0331
Epoch   4 Batch 1150/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.9688, Loss: 0.0355
Epoch   4 Batch 1155/2154 - Train Accuracy: 0.9589, Validation Accuracy: 0.9638, Loss: 0.0169
Epoch   4 Batch 1160/2154 - Train Accuracy: 0.9703, Validation Accuracy: 0.9652, Loss: 0.0254
Epoch   4 Batch 1165/2154 - Train Accuracy: 0.9609, Validation Accuracy: 0.9503, Loss: 0.0296
Epoch   4 Batch 1170/2154 - Train Accuracy: 0.9769, Validation Accuracy: 0.9645, Loss: 0.0205
Epoch   4 Batch 1175/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9638, Loss: 0.0346
Epoch   4 Batch 1180/2154 - Train Accuracy: 0.9737, Validation Accuracy: 0.9645, Loss: 0.0286
Epoch   4 Batch 1185/2154 - Train Accuracy: 0.9712, Validation Accuracy: 0.9716, Loss: 0.0363
Epoch   4 Batch 1190/2154 - Train Accuracy: 0.9852, Validation Accuracy: 0.9631, Loss: 0.0307
Epoch   4 Batch 1195/2154 - Train Accuracy: 0.9156, Validation Accuracy: 0.9538, Loss: 0.0248
Epoch   4 Batch 1200/2154 - Train Accuracy: 0.9309, Validation Accuracy: 0.9538, Loss: 0.0481
Epoch   4 Batch 1205/2154 - Train Accuracy: 0.9441, Validation Accuracy: 0.9730, Loss: 0.0373
Epoch   4 Batch 1210/2154 - Train Accuracy: 0.9586, Validation Accuracy: 0.9751, Loss: 0.0333
Epoch   4 Batch 1215/2154 - Train Accuracy: 0.9868, Validation Accuracy: 0.9624, Loss: 0.0351
Epoch   4 Batch 1220/2154 - Train Accuracy: 0.9297, Validation Accuracy: 0.9467, Loss: 0.0233
Epoch   4 Batch 1225/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9517, Loss: 0.0172
Epoch   4 Batch 1230/2154 - Train Accuracy: 0.9665, Validation Accuracy: 0.9524, Loss: 0.0222
Epoch   4 Batch 1235/2154 - Train Accuracy: 0.9397, Validation Accuracy: 0.9553, Loss: 0.0262
Epoch   4 Batch 1240/2154 - Train Accuracy: 0.9416, Validation Accuracy: 0.9688, Loss: 0.0276
Epoch   4 Batch 1245/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.9631, Loss: 0.0511
Epoch   4 Batch 1250/2154 - Train Accuracy: 0.9742, Validation Accuracy: 0.9538, Loss: 0.0200
Epoch   4 Batch 1255/2154 - Train Accuracy: 0.9416, Validation Accuracy: 0.9595, Loss: 0.0318
Epoch   4 Batch 1260/2154 - Train Accuracy: 0.9877, Validation Accuracy: 0.9325, Loss: 0.0351
Epoch   4 Batch 1265/2154 - Train Accuracy: 0.9586, Validation Accuracy: 0.9425, Loss: 0.0258
Epoch   4 Batch 1270/2154 - Train Accuracy: 0.9888, Validation Accuracy: 0.9581, Loss: 0.0179
Epoch   4 Batch 1275/2154 - Train Accuracy: 0.9469, Validation Accuracy: 0.9709, Loss: 0.0305
Epoch   4 Batch 1280/2154 - Train Accuracy: 0.9334, Validation Accuracy: 0.9609, Loss: 0.0437
Epoch   4 Batch 1285/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9616, Loss: 0.0318
Epoch   4 Batch 1290/2154 - Train Accuracy: 0.9477, Validation Accuracy: 0.9531, Loss: 0.0195
Epoch   4 Batch 1295/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9574, Loss: 0.0264
Epoch   4 Batch 1300/2154 - Train Accuracy: 0.9586, Validation Accuracy: 0.9666, Loss: 0.0303
Epoch   4 Batch 1305/2154 - Train Accuracy: 0.9844, Validation Accuracy: 0.9702, Loss: 0.0218
Epoch   4 Batch 1310/2154 - Train Accuracy: 0.9828, Validation Accuracy: 0.9652, Loss: 0.0352
Epoch   4 Batch 1315/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9531, Loss: 0.0306
Epoch   4 Batch 1320/2154 - Train Accuracy: 0.9665, Validation Accuracy: 0.9631, Loss: 0.0185
Epoch   4 Batch 1325/2154 - Train Accuracy: 0.9606, Validation Accuracy: 0.9737, Loss: 0.0187
Epoch   4 Batch 1330/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9737, Loss: 0.0280
Epoch   4 Batch 1335/2154 - Train Accuracy: 0.9597, Validation Accuracy: 0.9624, Loss: 0.0392
Epoch   4 Batch 1340/2154 - Train Accuracy: 0.9664, Validation Accuracy: 0.9631, Loss: 0.0309
Epoch   4 Batch 1345/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9311, Loss: 0.0290
Epoch   4 Batch 1350/2154 - Train Accuracy: 0.9453, Validation Accuracy: 0.9467, Loss: 0.0274
Epoch   4 Batch 1355/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9467, Loss: 0.0259
Epoch   4 Batch 1360/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9567, Loss: 0.0215
Epoch   4 Batch 1365/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9560, Loss: 0.0323
Epoch   4 Batch 1370/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9688, Loss: 0.0312
Epoch   4 Batch 1375/2154 - Train Accuracy: 0.9797, Validation Accuracy: 0.9780, Loss: 0.0280
Epoch   4 Batch 1380/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9780, Loss: 0.0204
Epoch   4 Batch 1385/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9688, Loss: 0.0249
Epoch   4 Batch 1390/2154 - Train Accuracy: 0.9479, Validation Accuracy: 0.9666, Loss: 0.0295
Epoch   4 Batch 1395/2154 - Train Accuracy: 0.9408, Validation Accuracy: 0.9616, Loss: 0.0329
Epoch   4 Batch 1400/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9545, Loss: 0.0202
Epoch   4 Batch 1405/2154 - Train Accuracy: 0.9561, Validation Accuracy: 0.9780, Loss: 0.0457
Epoch   4 Batch 1410/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.9631, Loss: 0.0298
Epoch   4 Batch 1415/2154 - Train Accuracy: 0.9594, Validation Accuracy: 0.9489, Loss: 0.0313
Epoch   4 Batch 1420/2154 - Train Accuracy: 0.9515, Validation Accuracy: 0.9830, Loss: 0.0361
Epoch   4 Batch 1425/2154 - Train Accuracy: 0.9906, Validation Accuracy: 0.9730, Loss: 0.0234
Epoch   4 Batch 1430/2154 - Train Accuracy: 0.9717, Validation Accuracy: 0.9560, Loss: 0.0280
Epoch   4 Batch 1435/2154 - Train Accuracy: 0.9803, Validation Accuracy: 0.9652, Loss: 0.0175
Epoch   4 Batch 1440/2154 - Train Accuracy: 0.9773, Validation Accuracy: 0.9638, Loss: 0.0322
Epoch   4 Batch 1445/2154 - Train Accuracy: 0.9672, Validation Accuracy: 0.9751, Loss: 0.0251
Epoch   4 Batch 1450/2154 - Train Accuracy: 0.9087, Validation Accuracy: 0.9652, Loss: 0.0341
Epoch   4 Batch 1455/2154 - Train Accuracy: 0.9656, Validation Accuracy: 0.9602, Loss: 0.0284
Epoch   4 Batch 1460/2154 - Train Accuracy: 0.9555, Validation Accuracy: 0.9545, Loss: 0.0386
Epoch   4 Batch 1465/2154 - Train Accuracy: 0.9712, Validation Accuracy: 0.9709, Loss: 0.0243
Epoch   4 Batch 1470/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9709, Loss: 0.0277
Epoch   4 Batch 1475/2154 - Train Accuracy: 0.9812, Validation Accuracy: 0.9545, Loss: 0.0375
Epoch   4 Batch 1480/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9759, Loss: 0.0253
Epoch   4 Batch 1485/2154 - Train Accuracy: 0.9633, Validation Accuracy: 0.9808, Loss: 0.0198
Epoch   4 Batch 1490/2154 - Train Accuracy: 0.9812, Validation Accuracy: 0.9730, Loss: 0.0292
Epoch   4 Batch 1495/2154 - Train Accuracy: 0.9613, Validation Accuracy: 0.9517, Loss: 0.0183
Epoch   4 Batch 1500/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9624, Loss: 0.0250
Epoch   4 Batch 1505/2154 - Train Accuracy: 0.9457, Validation Accuracy: 0.9751, Loss: 0.0249
Epoch   4 Batch 1510/2154 - Train Accuracy: 0.9753, Validation Accuracy: 0.9751, Loss: 0.0343
Epoch   4 Batch 1515/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9695, Loss: 0.0180
Epoch   4 Batch 1520/2154 - Train Accuracy: 0.9474, Validation Accuracy: 0.9695, Loss: 0.0272
Epoch   4 Batch 1525/2154 - Train Accuracy: 0.9672, Validation Accuracy: 0.9702, Loss: 0.0240
Epoch   4 Batch 1530/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9688, Loss: 0.0335
Epoch   4 Batch 1535/2154 - Train Accuracy: 0.9945, Validation Accuracy: 0.9482, Loss: 0.0219
Epoch   4 Batch 1540/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9595, Loss: 0.0358
Epoch   4 Batch 1545/2154 - Train Accuracy: 0.9435, Validation Accuracy: 0.9688, Loss: 0.0330
Epoch   4 Batch 1550/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9688, Loss: 0.0288
Epoch   4 Batch 1555/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9688, Loss: 0.0277
Epoch   4 Batch 1560/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9893, Loss: 0.0277
Epoch   4 Batch 1565/2154 - Train Accuracy: 0.9531, Validation Accuracy: 0.9787, Loss: 0.0302
Epoch   4 Batch 1570/2154 - Train Accuracy: 0.9578, Validation Accuracy: 0.9787, Loss: 0.0201
Epoch   4 Batch 1575/2154 - Train Accuracy: 0.9844, Validation Accuracy: 0.9645, Loss: 0.0194
Epoch   4 Batch 1580/2154 - Train Accuracy: 0.9572, Validation Accuracy: 0.9624, Loss: 0.0293
Epoch   4 Batch 1585/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9581, Loss: 0.0286
Epoch   4 Batch 1590/2154 - Train Accuracy: 0.9359, Validation Accuracy: 0.9680, Loss: 0.0229
Epoch   4 Batch 1595/2154 - Train Accuracy: 0.9766, Validation Accuracy: 0.9695, Loss: 0.0246
Epoch   4 Batch 1600/2154 - Train Accuracy: 0.9656, Validation Accuracy: 0.9588, Loss: 0.0341
Epoch   4 Batch 1605/2154 - Train Accuracy: 0.9625, Validation Accuracy: 0.9794, Loss: 0.0252
Epoch   4 Batch 1610/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9794, Loss: 0.0132
Epoch   4 Batch 1615/2154 - Train Accuracy: 0.9867, Validation Accuracy: 0.9794, Loss: 0.0118
Epoch   4 Batch 1620/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9695, Loss: 0.0512
Epoch   4 Batch 1625/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9680, Loss: 0.0225
Epoch   4 Batch 1630/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9588, Loss: 0.0231
Epoch   4 Batch 1635/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9588, Loss: 0.0297
Epoch   4 Batch 1640/2154 - Train Accuracy: 0.9734, Validation Accuracy: 0.9808, Loss: 0.0208
Epoch   4 Batch 1645/2154 - Train Accuracy: 0.9819, Validation Accuracy: 0.9695, Loss: 0.0173
Epoch   4 Batch 1650/2154 - Train Accuracy: 0.9449, Validation Accuracy: 0.9794, Loss: 0.0320
Epoch   4 Batch 1655/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9886, Loss: 0.0218
Epoch   4 Batch 1660/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9844, Loss: 0.0339
Epoch   4 Batch 1665/2154 - Train Accuracy: 0.9750, Validation Accuracy: 0.9595, Loss: 0.0209
Epoch   4 Batch 1670/2154 - Train Accuracy: 0.9969, Validation Accuracy: 0.9595, Loss: 0.0268
Epoch   4 Batch 1675/2154 - Train Accuracy: 0.9688, Validation Accuracy: 0.9673, Loss: 0.0286
Epoch   4 Batch 1680/2154 - Train Accuracy: 0.9820, Validation Accuracy: 0.9666, Loss: 0.0248
Epoch   4 Batch 1685/2154 - Train Accuracy: 0.9696, Validation Accuracy: 0.9645, Loss: 0.0183
Epoch   4 Batch 1690/2154 - Train Accuracy: 0.9673, Validation Accuracy: 0.9688, Loss: 0.0245
Epoch   4 Batch 1695/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9780, Loss: 0.0247
Epoch   4 Batch 1700/2154 - Train Accuracy: 0.9477, Validation Accuracy: 0.9673, Loss: 0.0221
Epoch   4 Batch 1705/2154 - Train Accuracy: 0.9211, Validation Accuracy: 0.9780, Loss: 0.0450
Epoch   4 Batch 1710/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9886, Loss: 0.0326
Epoch   4 Batch 1715/2154 - Train Accuracy: 0.9252, Validation Accuracy: 0.9794, Loss: 0.0300
Epoch   4 Batch 1720/2154 - Train Accuracy: 0.9803, Validation Accuracy: 0.9659, Loss: 0.0250
Epoch   4 Batch 1725/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9538, Loss: 0.0317
Epoch   4 Batch 1730/2154 - Train Accuracy: 0.9242, Validation Accuracy: 0.9538, Loss: 0.0227
Epoch   4 Batch 1735/2154 - Train Accuracy: 0.9268, Validation Accuracy: 0.9652, Loss: 0.0367
Epoch   4 Batch 1740/2154 - Train Accuracy: 0.9465, Validation Accuracy: 0.9616, Loss: 0.0293
Epoch   4 Batch 1745/2154 - Train Accuracy: 0.9753, Validation Accuracy: 0.9709, Loss: 0.0334
Epoch   4 Batch 1750/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.9538, Loss: 0.0298
Epoch   4 Batch 1755/2154 - Train Accuracy: 0.9266, Validation Accuracy: 0.9567, Loss: 0.0304
Epoch   4 Batch 1760/2154 - Train Accuracy: 0.9655, Validation Accuracy: 0.9588, Loss: 0.0253
Epoch   4 Batch 1765/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9588, Loss: 0.0275
Epoch   4 Batch 1770/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9638, Loss: 0.0315
Epoch   4 Batch 1775/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9744, Loss: 0.0216
Epoch   4 Batch 1780/2154 - Train Accuracy: 0.9547, Validation Accuracy: 0.9801, Loss: 0.0235
Epoch   4 Batch 1785/2154 - Train Accuracy: 0.9934, Validation Accuracy: 0.9702, Loss: 0.0164
Epoch   4 Batch 1790/2154 - Train Accuracy: 0.9874, Validation Accuracy: 0.9673, Loss: 0.0153
Epoch   4 Batch 1795/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9702, Loss: 0.0177
Epoch   4 Batch 1800/2154 - Train Accuracy: 0.9711, Validation Accuracy: 0.9624, Loss: 0.0306
Epoch   4 Batch 1805/2154 - Train Accuracy: 0.9234, Validation Accuracy: 0.9730, Loss: 0.0320
Epoch   4 Batch 1810/2154 - Train Accuracy: 0.9391, Validation Accuracy: 0.9730, Loss: 0.0318
Epoch   4 Batch 1815/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.9631, Loss: 0.0283
Epoch   4 Batch 1820/2154 - Train Accuracy: 0.9805, Validation Accuracy: 0.9822, Loss: 0.0211
Epoch   4 Batch 1825/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9723, Loss: 0.0237
Epoch   4 Batch 1830/2154 - Train Accuracy: 0.9512, Validation Accuracy: 0.9723, Loss: 0.0515
Epoch   4 Batch 1835/2154 - Train Accuracy: 0.9742, Validation Accuracy: 0.9695, Loss: 0.0351
Epoch   4 Batch 1840/2154 - Train Accuracy: 0.9540, Validation Accuracy: 0.9609, Loss: 0.0179
Epoch   4 Batch 1845/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9666, Loss: 0.0271
Epoch   4 Batch 1850/2154 - Train Accuracy: 0.9581, Validation Accuracy: 0.9673, Loss: 0.0278
Epoch   4 Batch 1855/2154 - Train Accuracy: 0.9563, Validation Accuracy: 0.9709, Loss: 0.0352
Epoch   4 Batch 1860/2154 - Train Accuracy: 0.9720, Validation Accuracy: 0.9574, Loss: 0.0229
Epoch   4 Batch 1865/2154 - Train Accuracy: 0.9786, Validation Accuracy: 0.9567, Loss: 0.0204
Epoch   4 Batch 1870/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9567, Loss: 0.0192
Epoch   4 Batch 1875/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9517, Loss: 0.0407
Epoch   4 Batch 1880/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9524, Loss: 0.0331
Epoch   4 Batch 1885/2154 - Train Accuracy: 0.9260, Validation Accuracy: 0.9418, Loss: 0.0317
Epoch   4 Batch 1890/2154 - Train Accuracy: 0.9807, Validation Accuracy: 0.9645, Loss: 0.0183
Epoch   4 Batch 1895/2154 - Train Accuracy: 0.9490, Validation Accuracy: 0.9531, Loss: 0.0229
Epoch   4 Batch 1900/2154 - Train Accuracy: 0.9641, Validation Accuracy: 0.9503, Loss: 0.0256
Epoch   4 Batch 1905/2154 - Train Accuracy: 0.9852, Validation Accuracy: 0.9503, Loss: 0.0161
Epoch   4 Batch 1910/2154 - Train Accuracy: 0.9967, Validation Accuracy: 0.9553, Loss: 0.0177
Epoch   4 Batch 1915/2154 - Train Accuracy: 0.9938, Validation Accuracy: 0.9560, Loss: 0.0151
Epoch   4 Batch 1920/2154 - Train Accuracy: 0.9646, Validation Accuracy: 0.9602, Loss: 0.0252
Epoch   4 Batch 1925/2154 - Train Accuracy: 0.9827, Validation Accuracy: 0.9645, Loss: 0.0324
Epoch   4 Batch 1930/2154 - Train Accuracy: 0.9945, Validation Accuracy: 0.9680, Loss: 0.0307
Epoch   4 Batch 1935/2154 - Train Accuracy: 0.9586, Validation Accuracy: 0.9631, Loss: 0.0329
Epoch   4 Batch 1940/2154 - Train Accuracy: 0.9375, Validation Accuracy: 0.9631, Loss: 0.0307
Epoch   4 Batch 1945/2154 - Train Accuracy: 0.9719, Validation Accuracy: 0.9631, Loss: 0.0204
Epoch   4 Batch 1950/2154 - Train Accuracy: 0.9727, Validation Accuracy: 0.9553, Loss: 0.0196
Epoch   4 Batch 1955/2154 - Train Accuracy: 0.9703, Validation Accuracy: 0.9602, Loss: 0.0186
Epoch   4 Batch 1960/2154 - Train Accuracy: 0.9729, Validation Accuracy: 0.9688, Loss: 0.0310
Epoch   4 Batch 1965/2154 - Train Accuracy: 0.9479, Validation Accuracy: 0.9624, Loss: 0.0265
Epoch   4 Batch 1970/2154 - Train Accuracy: 0.9539, Validation Accuracy: 0.9496, Loss: 0.0315
Epoch   4 Batch 1975/2154 - Train Accuracy: 0.9297, Validation Accuracy: 0.9545, Loss: 0.0256
Epoch   4 Batch 1980/2154 - Train Accuracy: 0.9461, Validation Accuracy: 0.9716, Loss: 0.0295
Epoch   4 Batch 1985/2154 - Train Accuracy: 0.9671, Validation Accuracy: 0.9673, Loss: 0.0336
Epoch   4 Batch 1990/2154 - Train Accuracy: 0.9622, Validation Accuracy: 0.9616, Loss: 0.0170
Epoch   4 Batch 1995/2154 - Train Accuracy: 0.9597, Validation Accuracy: 0.9716, Loss: 0.0395
Epoch   4 Batch 2000/2154 - Train Accuracy: 0.9523, Validation Accuracy: 0.9730, Loss: 0.0412
Epoch   4 Batch 2005/2154 - Train Accuracy: 0.9605, Validation Accuracy: 0.9609, Loss: 0.0180
Epoch   4 Batch 2010/2154 - Train Accuracy: 0.9433, Validation Accuracy: 0.9553, Loss: 0.0432
Epoch   4 Batch 2015/2154 - Train Accuracy: 0.9650, Validation Accuracy: 0.9545, Loss: 0.0257
Epoch   4 Batch 2020/2154 - Train Accuracy: 0.9811, Validation Accuracy: 0.9439, Loss: 0.0164
Epoch   4 Batch 2025/2154 - Train Accuracy: 0.9711, Validation Accuracy: 0.9446, Loss: 0.0251
Epoch   4 Batch 2030/2154 - Train Accuracy: 0.9828, Validation Accuracy: 0.9659, Loss: 0.0225
Epoch   4 Batch 2035/2154 - Train Accuracy: 0.9679, Validation Accuracy: 0.9553, Loss: 0.0180
Epoch   4 Batch 2040/2154 - Train Accuracy: 0.9416, Validation Accuracy: 0.9545, Loss: 0.0258
Epoch   4 Batch 2045/2154 - Train Accuracy: 0.9844, Validation Accuracy: 0.9602, Loss: 0.0152
Epoch   4 Batch 2050/2154 - Train Accuracy: 0.9293, Validation Accuracy: 0.9616, Loss: 0.0411
Epoch   4 Batch 2055/2154 - Train Accuracy: 0.9202, Validation Accuracy: 0.9602, Loss: 0.0341
Epoch   4 Batch 2060/2154 - Train Accuracy: 0.9655, Validation Accuracy: 0.9702, Loss: 0.0229
Epoch   4 Batch 2065/2154 - Train Accuracy: 0.9680, Validation Accuracy: 0.9830, Loss: 0.0256
Epoch   4 Batch 2070/2154 - Train Accuracy: 0.9367, Validation Accuracy: 0.9730, Loss: 0.0287
Epoch   4 Batch 2075/2154 - Train Accuracy: 0.9742, Validation Accuracy: 0.9609, Loss: 0.0359
Epoch   4 Batch 2080/2154 - Train Accuracy: 0.9732, Validation Accuracy: 0.9709, Loss: 0.0235
Epoch   4 Batch 2085/2154 - Train Accuracy: 0.9648, Validation Accuracy: 0.9553, Loss: 0.0207
Epoch   4 Batch 2090/2154 - Train Accuracy: 0.9570, Validation Accuracy: 0.9595, Loss: 0.0300
Epoch   4 Batch 2095/2154 - Train Accuracy: 0.9938, Validation Accuracy: 0.9524, Loss: 0.0223
Epoch   4 Batch 2100/2154 - Train Accuracy: 0.9695, Validation Accuracy: 0.9517, Loss: 0.0195
Epoch   4 Batch 2105/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9702, Loss: 0.0248
Epoch   4 Batch 2110/2154 - Train Accuracy: 0.9430, Validation Accuracy: 0.9567, Loss: 0.0227
Epoch   4 Batch 2115/2154 - Train Accuracy: 0.9630, Validation Accuracy: 0.9567, Loss: 0.0266
Epoch   4 Batch 2120/2154 - Train Accuracy: 0.9498, Validation Accuracy: 0.9467, Loss: 0.0274
Epoch   4 Batch 2125/2154 - Train Accuracy: 0.9437, Validation Accuracy: 0.9467, Loss: 0.0298
Epoch   4 Batch 2130/2154 - Train Accuracy: 0.9819, Validation Accuracy: 0.9609, Loss: 0.0208
Epoch   4 Batch 2135/2154 - Train Accuracy: 0.9617, Validation Accuracy: 0.9609, Loss: 0.0282
Epoch   4 Batch 2140/2154 - Train Accuracy: 0.9807, Validation Accuracy: 0.9688, Loss: 0.0159
Epoch   4 Batch 2145/2154 - Train Accuracy: 0.9827, Validation Accuracy: 0.9453, Loss: 0.0276
Epoch   4 Batch 2150/2154 - Train Accuracy: 0.9859, Validation Accuracy: 0.9730, Loss: 0.0273
Model Trained and Saved

Save Parameters

Save the batch_size and save_path parameters for inference.





In [139]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# Save parameters for checkpoint
helper.save_params(save_path)

Checkpoint





In [140]:



    


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import tensorflow as tf
import numpy as np
import helper
import problem_unittests as tests

_, (source_vocab_to_int, target_vocab_to_int), (source_int_to_vocab, target_int_to_vocab) = helper.load_preprocess()
load_path = helper.load_params()

Sentence to Sequence

To feed a sentence into the model for translation, you first need to preprocess it. Implement the function sentence_to_seq() to preprocess new sentences.

  • Convert the sentence to lowercase
  • Convert words into ids using vocab_to_int
    • Convert words not in the vocabulary, to the <UNK> word id.




In [141]:



    


def sentence_to_seq(sentence, vocab_to_int):
    """
    Convert a sentence to a sequence of ids
    :param sentence: String
    :param vocab_to_int: Dictionary to go from the words to an id
    :return: List of word ids
    """
    # TODO: Implement Function
    
    sentence_lowercase = sentence.lower() 
    words2int = []
    for word in sentence.split():
        if word in vocab_to_int:
            words2int.append(vocab_to_int[word])
        else:
            words2int.append(vocab_to_int['<UNK>'])
    
    return words2int


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_sentence_to_seq(sentence_to_seq)



    


















    






Tests Passed

Translate

This will translate translate_sentence from English to French.





In [146]:



    


translate_sentence = 'he saw a old yellow truck .'


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
translate_sentence = sentence_to_seq(translate_sentence, source_vocab_to_int)

loaded_graph = tf.Graph()
with tf.Session(graph=loaded_graph) as sess:
    # Load saved model
    loader = tf.train.import_meta_graph(load_path + '.meta')
    loader.restore(sess, load_path)

    input_data = loaded_graph.get_tensor_by_name('input:0')
    logits = loaded_graph.get_tensor_by_name('predictions:0')
    target_sequence_length = loaded_graph.get_tensor_by_name('target_sequence_length:0')
    source_sequence_length = loaded_graph.get_tensor_by_name('source_sequence_length:0')
    keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')

    translate_logits = sess.run(logits, {input_data: [translate_sentence]*batch_size,
                                         target_sequence_length: [len(translate_sentence)*2]*batch_size,
                                         source_sequence_length: [len(translate_sentence)]*batch_size,
                                         keep_prob: 1.0})[0]

print('Input')
print('  Word Ids:      {}'.format([i for i in translate_sentence]))
print('  English Words: {}'.format([source_int_to_vocab[i] for i in translate_sentence]))

print('\nPrediction')
print('  Word Ids:      {}'.format([i for i in translate_logits]))
print('  French Words: {}'.format(" ".join([target_int_to_vocab[i] for i in translate_logits])))



    


















    






INFO:tensorflow:Restoring parameters from checkpoints/dev
Input
  Word Ids:      [147, 31, 115, 154, 82, 8, 193]
  English Words: ['he', 'saw', 'a', 'old', 'yellow', 'truck', '.']

Prediction
  Word Ids:      [205, 316, 142, 296, 164, 140, 247, 344, 1]
  French Words: il a vu un vieux camion jaune . <EOS>

Imperfect Translation

You might notice that some sentences translate better than others. Since the dataset you're using only has a vocabulary of 227 English words of the thousands that you use, you're only going to see good results using these words. For this project, you don't need a perfect translation. However, if you want to create a better translation model, you'll need better data.

You can train on the WMT10 French-English corpus. This dataset has more vocabulary and richer in topics discussed. However, this will take you days to train, so make sure you've a GPU and the neural network is performing well on dataset we provided. Just make sure you play with the WMT10 corpus after you've submitted this project.

Submitting This Project

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_language_translation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.

Content source: arturops/deep-learning

Similar notebooks:

notebook.community | gallery | about