Anna KaRNNa

In this notebook, I'll build a character-wise RNN trained on Anna Karenina, one of my all-time favorite books. It'll be able to generate new text based on the text from the book.

This network is based off of Andrej Karpathy's post on RNNs and implementation in Torch. Also, some information here at r2rt and from Sherjil Ozair on GitHub. Below is the general architecture of the character-wise RNN.



In [1]:

    
import time
from collections import namedtuple

import numpy as np
import tensorflow as tf

First we'll load the text file and convert it into integers for our network to use.



In [2]:

    
with open('anna.txt', 'r') as f:
    text=f.read()
vocab = set(text)
vocab_to_int = {c: i for i, c in enumerate(vocab)}
int_to_vocab = dict(enumerate(vocab))
chars = np.array([vocab_to_int[c] for c in text], dtype=np.int32)



In [3]:

    
text[:100]









    Out[3]:





'Chapter 1\n\n\nHappy families are all alike; every unhappy family is unhappy in its own\nway.\n\nEverythin'



In [4]:

    
chars[:100]









    Out[4]:





array([21, 50, 74, 44, 12, 68, 48, 45, 79,  6,  6,  6, 51, 74, 44, 44, 32,
       45, 43, 74, 24, 10, 22, 10, 68, 80, 45, 74, 48, 68, 45, 74, 22, 22,
       45, 74, 22, 10, 37, 68, 69, 45, 68, 18, 68, 48, 32, 45, 76, 72, 50,
       74, 44, 44, 32, 45, 43, 74, 24, 10, 22, 32, 45, 10, 80, 45, 76, 72,
       50, 74, 44, 44, 32, 45, 10, 72, 45, 10, 12, 80, 45,  0, 34, 72,  6,
       34, 74, 32, 35,  6,  6,  9, 18, 68, 48, 32, 12, 50, 10, 72], dtype=int32)

Now I need to split up the data into batches, and into training and validation sets. I should be making a test set here, but I'm not going to worry about that. My test will be if the network can generate new text.

Here I'll make both input and target arrays. The targets are the same as the inputs, except shifted one character over. I'll also drop the last bit of data so that I'll only have completely full batches.

The idea here is to make a 2D matrix where the number of rows is equal to the number of batches. Each row will be one long concatenated string from the character data. We'll split this data into a training set and validation set using the split_frac keyword. This will keep 90% of the batches in the training set, the other 10% in the validation set.



In [5]:

    
def split_data(chars, batch_size, num_steps, split_frac=0.9):
    """ 
    Split character data into training and validation sets, inputs and targets for each set.
    
    Arguments
    ---------
    chars: character array
    batch_size: Size of examples in each of batch
    num_steps: Number of sequence steps to keep in the input and pass to the network
    split_frac: Fraction of batches to keep in the training set
    
    
    Returns train_x, train_y, val_x, val_y
    """
    
    slice_size = batch_size * num_steps
    n_batches = int(len(chars) / slice_size)
    
    # Drop the last few characters to make only full batches
    x = chars[: n_batches*slice_size]
    y = chars[1: n_batches*slice_size + 1]
    
    # Split the data into batch_size slices, then stack them into a 2D matrix 
    x = np.stack(np.split(x, batch_size))
    y = np.stack(np.split(y, batch_size))
    
    # Now x and y are arrays with dimensions batch_size x n_batches*num_steps
    
    # Split into training and validation sets, keep the virst split_frac batches for training
    split_idx = int(n_batches*split_frac)
    train_x, train_y= x[:, :split_idx*num_steps], y[:, :split_idx*num_steps]
    val_x, val_y = x[:, split_idx*num_steps:], y[:, split_idx*num_steps:]
    
    return train_x, train_y, val_x, val_y



In [6]:

    
train_x, train_y, val_x, val_y = split_data(chars, 10, 200)



In [7]:

    
train_x.shape









    Out[7]:





(10, 178400)



In [8]:

    
train_x[:,:10]









    Out[8]:





array([[21, 50, 74, 44, 12, 68, 48, 45, 79,  6],
       [16, 72,  4, 45, 50, 68, 45, 24,  0, 18],
       [45, 73, 74, 12, 73, 50, 10, 72, 58, 45],
       [ 0, 12, 50, 68, 48, 45, 34,  0, 76, 22],
       [45, 12, 50, 68, 45, 22, 74, 72,  4, 64],
       [45, 81, 50, 48,  0, 76, 58, 50, 45, 22],
       [12, 45, 12,  0,  6,  4,  0, 35,  6,  6],
       [ 0, 45, 50, 68, 48, 80, 68, 22, 43, 57],
       [50, 74, 12, 45, 10, 80, 45, 12, 50, 68],
       [68, 48, 80, 68, 22, 43, 45, 74, 72,  4]], dtype=int32)

I'll write another function to grab batches out of the arrays made by split data. Here each batch will be a sliding window on these arrays with size batch_size X num_steps. For example, if we want our network to train on a sequence of 100 characters, num_steps = 100. For the next batch, we'll shift this window the next sequence of num_steps characters. In this way we can feed batches to the network and the cell states will continue through on each batch.



In [9]:

    
def get_batch(arrs, num_steps):
    batch_size, slice_size = arrs[0].shape
    
    n_batches = int(slice_size/num_steps)
    for b in range(n_batches):
        yield [x[:, b*num_steps: (b+1)*num_steps] for x in arrs]



In [10]:

    
def build_rnn(num_classes, batch_size=50, num_steps=50, lstm_size=128, num_layers=2,
              learning_rate=0.001, grad_clip=5, sampling=False):
        
    if sampling == True:
        batch_size, num_steps = 1, 1

    tf.reset_default_graph()
    
    # Declare placeholders we'll feed into the graph
    with tf.name_scope('inputs'):
        inputs = tf.placeholder(tf.int32, [batch_size, num_steps], name='inputs')
        x_one_hot = tf.one_hot(inputs, num_classes, name='x_one_hot')
    
    with tf.name_scope('targets'):
        targets = tf.placeholder(tf.int32, [batch_size, num_steps], name='targets')
        y_one_hot = tf.one_hot(targets, num_classes, name='y_one_hot')
        y_reshaped = tf.reshape(y_one_hot, [-1, num_classes])
    
    keep_prob = tf.placeholder(tf.float32, name='keep_prob')
    
    # Build the RNN layers
    with tf.name_scope("RNN_cells"):
        lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size)
        drop = tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)
        cell = tf.contrib.rnn.MultiRNNCell([drop] * num_layers)
    
    with tf.name_scope("RNN_init_state"):
        initial_state = cell.zero_state(batch_size, tf.float32)

    # Run the data through the RNN layers
    with tf.name_scope("RNN_forward"):
        outputs, state = tf.nn.dynamic_rnn(cell, x_one_hot, initial_state=initial_state)
    
    final_state = state
    
    # Reshape output so it's a bunch of rows, one row for each cell output
    with tf.name_scope('sequence_reshape'):
        seq_output = tf.concat(outputs, axis=1,name='seq_output')
        output = tf.reshape(seq_output, [-1, lstm_size], name='graph_output')
    
    # Now connect the RNN outputs to a softmax layer and calculate the cost
    with tf.name_scope('logits'):
        softmax_w = tf.Variable(tf.truncated_normal((lstm_size, num_classes), stddev=0.1),
                               name='softmax_w')
        softmax_b = tf.Variable(tf.zeros(num_classes), name='softmax_b')
        logits = tf.matmul(output, softmax_w) + softmax_b
        tf.summary.histogram('softmax_w', softmax_w)
        tf.summary.histogram('softmax_b', softmax_b)

    with tf.name_scope('predictions'):
        preds = tf.nn.softmax(logits, name='predictions')
        tf.summary.histogram('predictions', preds)
    
    with tf.name_scope('cost'):
        loss = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y_reshaped, name='loss')
        cost = tf.reduce_mean(loss, name='cost')
        tf.summary.scalar('cost', cost)

    # Optimizer for training, using gradient clipping to control exploding gradients
    with tf.name_scope('train'):
        tvars = tf.trainable_variables()
        grads, _ = tf.clip_by_global_norm(tf.gradients(cost, tvars), grad_clip)
        train_op = tf.train.AdamOptimizer(learning_rate)
        optimizer = train_op.apply_gradients(zip(grads, tvars))
    
    merged = tf.summary.merge_all()
    
    # Export the nodes 
    export_nodes = ['inputs', 'targets', 'initial_state', 'final_state',
                    'keep_prob', 'cost', 'preds', 'optimizer', 'merged']
    Graph = namedtuple('Graph', export_nodes)
    local_dict = locals()
    graph = Graph(*[local_dict[each] for each in export_nodes])
    
    return graph

Hyperparameters

Here I'm defining the hyperparameters for the network. The two you probably haven't seen before are lstm_size and num_layers. These set the number of hidden units in the LSTM layers and the number of LSTM layers, respectively. Of course, making these bigger will improve the network's performance but you'll have to watch out for overfitting. If your validation loss is much larger than the training loss, you're probably overfitting. Decrease the size of the network or decrease the dropout keep probability.



In [11]:

    
batch_size = 100
num_steps = 100
lstm_size = 512
num_layers = 2
learning_rate = 0.001

Training

Time for training which is is pretty straightforward. Here I pass in some data, and get an LSTM state back. Then I pass that state back in to the network so the next batch can continue the state from the previous batch. And every so often (set by save_every_n) I calculate the validation loss and save a checkpoint.



In [12]:

    
!mkdir -p checkpoints/anna



In [13]:

    
epochs = 10
save_every_n = 100
train_x, train_y, val_x, val_y = split_data(chars, batch_size, num_steps)

model = build_rnn(len(vocab), 
                  batch_size=batch_size,
                  num_steps=num_steps,
                  learning_rate=learning_rate,
                  lstm_size=lstm_size,
                  num_layers=num_layers)

saver = tf.train.Saver(max_to_keep=100)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    train_writer = tf.summary.FileWriter('./logs/2/train', sess.graph)
    test_writer = tf.summary.FileWriter('./logs/2/test')
    
    # Use the line below to load a checkpoint and resume training
    #saver.restore(sess, 'checkpoints/anna20.ckpt')
    
    n_batches = int(train_x.shape[1]/num_steps)
    iterations = n_batches * epochs
    for e in range(epochs):
        
        # Train network
        new_state = sess.run(model.initial_state)
        loss = 0
        for b, (x, y) in enumerate(get_batch([train_x, train_y], num_steps), 1):
            iteration = e*n_batches + b
            start = time.time()
            feed = {model.inputs: x,
                    model.targets: y,
                    model.keep_prob: 0.5,
                    model.initial_state: new_state}
            summary, batch_loss, new_state, _ = sess.run([model.merged, model.cost, 
                                                          model.final_state, model.optimizer], 
                                                          feed_dict=feed)
            loss += batch_loss
            end = time.time()
            print('Epoch {}/{} '.format(e+1, epochs),
                  'Iteration {}/{}'.format(iteration, iterations),
                  'Training loss: {:.4f}'.format(loss/b),
                  '{:.4f} sec/batch'.format((end-start)))
            
            train_writer.add_summary(summary, iteration)
        
            if (iteration%save_every_n == 0) or (iteration == iterations):
                # Check performance, notice dropout has been set to 1
                val_loss = []
                new_state = sess.run(model.initial_state)
                for x, y in get_batch([val_x, val_y], num_steps):
                    feed = {model.inputs: x,
                            model.targets: y,
                            model.keep_prob: 1.,
                            model.initial_state: new_state}
                    summary, batch_loss, new_state = sess.run([model.merged, model.cost, 
                                                               model.final_state], feed_dict=feed)
                    val_loss.append(batch_loss)
                    
                test_writer.add_summary(summary, iteration)

                print('Validation loss:', np.mean(val_loss),
                      'Saving checkpoint!')
                #saver.save(sess, "checkpoints/anna/i{}_l{}_{:.3f}.ckpt".format(iteration, lstm_size, np.mean(val_loss)))









    



Epoch 1/10  Iteration 1/1780 Training loss: 4.4217 14.2128 sec/batch
Epoch 1/10  Iteration 2/1780 Training loss: 4.3788 12.9779 sec/batch
Epoch 1/10  Iteration 3/1780 Training loss: 4.2085 18.6766 sec/batch
Epoch 1/10  Iteration 4/1780 Training loss: 4.5575 19.3183 sec/batch
Epoch 1/10  Iteration 5/1780 Training loss: 4.4877 15.3934 sec/batch
Epoch 1/10  Iteration 6/1780 Training loss: 4.3779 16.0402 sec/batch
Epoch 1/10  Iteration 7/1780 Training loss: 4.2948 22.0292 sec/batch
Epoch 1/10  Iteration 8/1780 Training loss: 4.2143 14.5269 sec/batch
Epoch 1/10  Iteration 9/1780 Training loss: 4.1382 15.4512 sec/batch
Epoch 1/10  Iteration 10/1780 Training loss: 4.0700 17.2698 sec/batch
Epoch 1/10  Iteration 11/1780 Training loss: 4.0093 17.2070 sec/batch
Epoch 1/10  Iteration 12/1780 Training loss: 3.9580 16.2353 sec/batch
Epoch 1/10  Iteration 13/1780 Training loss: 3.9125 15.4778 sec/batch
Epoch 1/10  Iteration 14/1780 Training loss: 3.8739 16.2295 sec/batch
Epoch 1/10  Iteration 15/1780 Training loss: 3.8387 17.0356 sec/batch
Epoch 1/10  Iteration 16/1780 Training loss: 3.8065 13.8377 sec/batch
Epoch 1/10  Iteration 17/1780 Training loss: 3.7770 11.9660 sec/batch
Epoch 1/10  Iteration 18/1780 Training loss: 3.7521 14.9782 sec/batch
Epoch 1/10  Iteration 19/1780 Training loss: 3.7283 11.9410 sec/batch
Epoch 1/10  Iteration 20/1780 Training loss: 3.7046 13.8231 sec/batch
Epoch 1/10  Iteration 21/1780 Training loss: 3.6841 13.1241 sec/batch
Epoch 1/10  Iteration 22/1780 Training loss: 3.6648 12.5774 sec/batch
Epoch 1/10  Iteration 23/1780 Training loss: 3.6471 13.7308 sec/batch
Epoch 1/10  Iteration 24/1780 Training loss: 3.6304 14.5333 sec/batch
Epoch 1/10  Iteration 25/1780 Training loss: 3.6150 14.2089 sec/batch
Epoch 1/10  Iteration 26/1780 Training loss: 3.6009 16.0848 sec/batch
Epoch 1/10  Iteration 27/1780 Training loss: 3.5878 12.9938 sec/batch
Epoch 1/10  Iteration 28/1780 Training loss: 3.5746 13.9524 sec/batch
Epoch 1/10  Iteration 29/1780 Training loss: 3.5624 14.0048 sec/batch
Epoch 1/10  Iteration 30/1780 Training loss: 3.5511 12.2805 sec/batch
Epoch 1/10  Iteration 31/1780 Training loss: 3.5410 16.0285 sec/batch
Epoch 1/10  Iteration 32/1780 Training loss: 3.5304 16.8094 sec/batch
Epoch 1/10  Iteration 33/1780 Training loss: 3.5202 16.3403 sec/batch
Epoch 1/10  Iteration 34/1780 Training loss: 3.5112 16.9136 sec/batch
Epoch 1/10  Iteration 35/1780 Training loss: 3.5020 15.5161 sec/batch
Epoch 1/10  Iteration 36/1780 Training loss: 3.4937 10.7103 sec/batch
Epoch 1/10  Iteration 37/1780 Training loss: 3.4853 8.4184 sec/batch
Epoch 1/10  Iteration 38/1780 Training loss: 3.4774 8.0702 sec/batch
Epoch 1/10  Iteration 39/1780 Training loss: 3.4696 7.0793 sec/batch
Epoch 1/10  Iteration 40/1780 Training loss: 3.4623 6.9482 sec/batch
Epoch 1/10  Iteration 41/1780 Training loss: 3.4552 6.8993 sec/batch
Epoch 1/10  Iteration 42/1780 Training loss: 3.4485 6.8721 sec/batch
Epoch 1/10  Iteration 43/1780 Training loss: 3.4419 6.9086 sec/batch
Epoch 1/10  Iteration 44/1780 Training loss: 3.4355 6.8589 sec/batch
Epoch 1/10  Iteration 45/1780 Training loss: 3.4291 6.9006 sec/batch
Epoch 1/10  Iteration 46/1780 Training loss: 3.4235 6.8751 sec/batch
Epoch 1/10  Iteration 47/1780 Training loss: 3.4181 6.9235 sec/batch
Epoch 1/10  Iteration 48/1780 Training loss: 3.4131 6.9282 sec/batch
Epoch 1/10  Iteration 49/1780 Training loss: 3.4082 6.8448 sec/batch
Epoch 1/10  Iteration 50/1780 Training loss: 3.4035 7.0918 sec/batch
Epoch 1/10  Iteration 51/1780 Training loss: 3.3988 6.9369 sec/batch
Epoch 1/10  Iteration 52/1780 Training loss: 3.3941 6.7699 sec/batch
Epoch 1/10  Iteration 53/1780 Training loss: 3.3898 6.8732 sec/batch
Epoch 1/10  Iteration 54/1780 Training loss: 3.3852 6.9597 sec/batch
Epoch 1/10  Iteration 55/1780 Training loss: 3.3810 7.0531 sec/batch
Epoch 1/10  Iteration 56/1780 Training loss: 3.3768 7.0394 sec/batch
Epoch 1/10  Iteration 57/1780 Training loss: 3.3726 6.9713 sec/batch
Epoch 1/10  Iteration 58/1780 Training loss: 3.3688 6.9764 sec/batch
Epoch 1/10  Iteration 59/1780 Training loss: 3.3647 9.1216 sec/batch
Epoch 1/10  Iteration 60/1780 Training loss: 3.3611 10.8296 sec/batch
Epoch 1/10  Iteration 61/1780 Training loss: 3.3576 10.2718 sec/batch
Epoch 1/10  Iteration 62/1780 Training loss: 3.3545 8.1424 sec/batch
Epoch 1/10  Iteration 63/1780 Training loss: 3.3515 6.7415 sec/batch
Epoch 1/10  Iteration 64/1780 Training loss: 3.3480 6.7506 sec/batch
Epoch 1/10  Iteration 65/1780 Training loss: 3.3446 6.7043 sec/batch
Epoch 1/10  Iteration 66/1780 Training loss: 3.3417 6.8183 sec/batch
Epoch 1/10  Iteration 67/1780 Training loss: 3.3387 6.7087 sec/batch
Epoch 1/10  Iteration 68/1780 Training loss: 3.3351 6.7961 sec/batch
Epoch 1/10  Iteration 69/1780 Training loss: 3.3321 6.7909 sec/batch
Epoch 1/10  Iteration 70/1780 Training loss: 3.3294 6.7278 sec/batch
Epoch 1/10  Iteration 71/1780 Training loss: 3.3265 7.0493 sec/batch
Epoch 1/10  Iteration 72/1780 Training loss: 3.3240 7.1285 sec/batch
Epoch 1/10  Iteration 73/1780 Training loss: 3.3212 6.7564 sec/batch
Epoch 1/10  Iteration 74/1780 Training loss: 3.3187 6.7843 sec/batch
Epoch 1/10  Iteration 75/1780 Training loss: 3.3162 6.6906 sec/batch
Epoch 1/10  Iteration 76/1780 Training loss: 3.3139 6.7512 sec/batch
Epoch 1/10  Iteration 77/1780 Training loss: 3.3115 6.6982 sec/batch
Epoch 1/10  Iteration 78/1780 Training loss: 3.3092 6.7299 sec/batch
Epoch 1/10  Iteration 79/1780 Training loss: 3.3068 6.8291 sec/batch
Epoch 1/10  Iteration 80/1780 Training loss: 3.3042 6.7649 sec/batch
Epoch 1/10  Iteration 81/1780 Training loss: 3.3018 6.8288 sec/batch
Epoch 1/10  Iteration 82/1780 Training loss: 3.2996 6.8598 sec/batch
Epoch 1/10  Iteration 83/1780 Training loss: 3.2974 6.9290 sec/batch
Epoch 1/10  Iteration 84/1780 Training loss: 3.2953 6.9460 sec/batch
Epoch 1/10  Iteration 85/1780 Training loss: 3.2929 6.9106 sec/batch
Epoch 1/10  Iteration 86/1780 Training loss: 3.2907 6.8634 sec/batch
Epoch 1/10  Iteration 87/1780 Training loss: 3.2885 6.7399 sec/batch
Epoch 1/10  Iteration 88/1780 Training loss: 3.2863 7.1735 sec/batch
Epoch 1/10  Iteration 89/1780 Training loss: 3.2842 6.9254 sec/batch
Epoch 1/10  Iteration 90/1780 Training loss: 3.2823 6.8733 sec/batch
Epoch 1/10  Iteration 91/1780 Training loss: 3.2803 6.8242 sec/batch
Epoch 1/10  Iteration 92/1780 Training loss: 3.2782 10.0844 sec/batch
Epoch 1/10  Iteration 93/1780 Training loss: 3.2762 10.3454 sec/batch
Epoch 1/10  Iteration 94/1780 Training loss: 3.2741 8.7794 sec/batch
Epoch 1/10  Iteration 95/1780 Training loss: 3.2720 7.1881 sec/batch
Epoch 1/10  Iteration 96/1780 Training loss: 3.2699 6.8932 sec/batch
Epoch 1/10  Iteration 97/1780 Training loss: 3.2679 6.8507 sec/batch
Epoch 1/10  Iteration 98/1780 Training loss: 3.2659 7.0328 sec/batch
Epoch 1/10  Iteration 99/1780 Training loss: 3.2641 6.8963 sec/batch
Epoch 1/10  Iteration 100/1780 Training loss: 3.2623 6.8478 sec/batch
Validation loss: 3.0373 Saving checkpoint!
Epoch 1/10  Iteration 101/1780 Training loss: 3.2607 6.7864 sec/batch
Epoch 1/10  Iteration 102/1780 Training loss: 3.2589 6.7199 sec/batch
Epoch 1/10  Iteration 103/1780 Training loss: 3.2571 6.7609 sec/batch
Epoch 1/10  Iteration 104/1780 Training loss: 3.2552 6.7341 sec/batch
Epoch 1/10  Iteration 105/1780 Training loss: 3.2534 6.6587 sec/batch
Epoch 1/10  Iteration 106/1780 Training loss: 3.2516 6.6308 sec/batch
Epoch 1/10  Iteration 107/1780 Training loss: 3.2495 6.7101 sec/batch
Epoch 1/10  Iteration 108/1780 Training loss: 3.2475 7.0824 sec/batch
Epoch 1/10  Iteration 109/1780 Training loss: 3.2457 6.8152 sec/batch
Epoch 1/10  Iteration 110/1780 Training loss: 3.2435 6.7059 sec/batch
Epoch 1/10  Iteration 111/1780 Training loss: 3.2416 6.7337 sec/batch
Epoch 1/10  Iteration 112/1780 Training loss: 3.2396 6.9073 sec/batch
Epoch 1/10  Iteration 113/1780 Training loss: 3.2374 6.7669 sec/batch
Epoch 1/10  Iteration 114/1780 Training loss: 3.2353 6.7778 sec/batch
Epoch 1/10  Iteration 115/1780 Training loss: 3.2330 811.8452 sec/batch
Epoch 1/10  Iteration 116/1780 Training loss: 3.2307 10.7499 sec/batch
Epoch 1/10  Iteration 117/1780 Training loss: 3.2284 10.4014 sec/batch
Epoch 1/10  Iteration 118/1780 Training loss: 3.2264 7.1952 sec/batch
Epoch 1/10  Iteration 119/1780 Training loss: 3.2243 7.3503 sec/batch
Epoch 1/10  Iteration 120/1780 Training loss: 3.2220 8.8955 sec/batch
Epoch 1/10  Iteration 121/1780 Training loss: 3.2199 6.9386 sec/batch
Epoch 1/10  Iteration 122/1780 Training loss: 3.2175 7.2040 sec/batch
Epoch 1/10  Iteration 123/1780 Training loss: 3.2152 6.9822 sec/batch
Epoch 1/10  Iteration 124/1780 Training loss: 3.2131 7.5660 sec/batch
Epoch 1/10  Iteration 125/1780 Training loss: 3.2108 7.8601 sec/batch
Epoch 1/10  Iteration 126/1780 Training loss: 3.2082 7.3857 sec/batch
Epoch 1/10  Iteration 127/1780 Training loss: 3.2061 7.4056 sec/batch
Epoch 1/10  Iteration 128/1780 Training loss: 3.2038 7.2270 sec/batch
Epoch 1/10  Iteration 129/1780 Training loss: 3.2014 7.0963 sec/batch
Epoch 1/10  Iteration 130/1780 Training loss: 3.1990 7.5135 sec/batch
Epoch 1/10  Iteration 131/1780 Training loss: 3.1966 7.6186 sec/batch
Epoch 1/10  Iteration 132/1780 Training loss: 3.1940 7.1486 sec/batch
Epoch 1/10  Iteration 133/1780 Training loss: 3.1916 6.9880 sec/batch
Epoch 1/10  Iteration 134/1780 Training loss: 3.1890 7.0428 sec/batch
Epoch 1/10  Iteration 135/1780 Training loss: 3.1862 7.5574 sec/batch
Epoch 1/10  Iteration 136/1780 Training loss: 3.1835 7.2993 sec/batch
Epoch 1/10  Iteration 137/1780 Training loss: 3.1808 7.2241 sec/batch
Epoch 1/10  Iteration 138/1780 Training loss: 3.1780 7.3488 sec/batch
Epoch 1/10  Iteration 139/1780 Training loss: 3.1755 7.3173 sec/batch
Epoch 1/10  Iteration 140/1780 Training loss: 3.1728 7.1618 sec/batch
Epoch 1/10  Iteration 141/1780 Training loss: 3.1701 7.1954 sec/batch
Epoch 1/10  Iteration 142/1780 Training loss: 3.1673 7.4972 sec/batch
Epoch 1/10  Iteration 143/1780 Training loss: 3.1645 7.7868 sec/batch
Epoch 1/10  Iteration 144/1780 Training loss: 3.1616 7.3377 sec/batch
Epoch 1/10  Iteration 145/1780 Training loss: 3.1588 7.1937 sec/batch
Epoch 1/10  Iteration 146/1780 Training loss: 3.1561 7.3709 sec/batch
Epoch 1/10  Iteration 147/1780 Training loss: 3.1533 8.6500 sec/batch
Epoch 1/10  Iteration 148/1780 Training loss: 3.1506 8.2810 sec/batch
Epoch 1/10  Iteration 149/1780 Training loss: 3.1476 7.4693 sec/batch
Epoch 1/10  Iteration 150/1780 Training loss: 3.1446 7.7248 sec/batch
Epoch 1/10  Iteration 151/1780 Training loss: 3.1419 7.5603 sec/batch
Epoch 1/10  Iteration 152/1780 Training loss: 3.1393 8.2602 sec/batch
Epoch 1/10  Iteration 153/1780 Training loss: 3.1364 7.8796 sec/batch
Epoch 1/10  Iteration 154/1780 Training loss: 3.1335 7.3492 sec/batch
Epoch 1/10  Iteration 155/1780 Training loss: 3.1305 6.9982 sec/batch
Epoch 1/10  Iteration 156/1780 Training loss: 3.1275 6.9733 sec/batch
Epoch 1/10  Iteration 157/1780 Training loss: 3.1244 7.1509 sec/batch
Epoch 1/10  Iteration 158/1780 Training loss: 3.1213 7.0571 sec/batch
Epoch 1/10  Iteration 159/1780 Training loss: 3.1181 7.1417 sec/batch
Epoch 1/10  Iteration 160/1780 Training loss: 3.1151 9.0249 sec/batch
Epoch 1/10  Iteration 161/1780 Training loss: 3.1121 12.9170 sec/batch
Epoch 1/10  Iteration 162/1780 Training loss: 3.1089 10.4890 sec/batch
Epoch 1/10  Iteration 163/1780 Training loss: 3.1056 9.9073 sec/batch
Epoch 1/10  Iteration 164/1780 Training loss: 3.1025 9.3950 sec/batch
Epoch 1/10  Iteration 165/1780 Training loss: 3.0995 9.4413 sec/batch
Epoch 1/10  Iteration 166/1780 Training loss: 3.0964 9.4902 sec/batch
Epoch 1/10  Iteration 167/1780 Training loss: 3.0933 9.2683 sec/batch
Epoch 1/10  Iteration 168/1780 Training loss: 3.0902 9.7007 sec/batch
Epoch 1/10  Iteration 169/1780 Training loss: 3.0871 9.3585 sec/batch
Epoch 1/10  Iteration 170/1780 Training loss: 3.0838 9.4635 sec/batch
Epoch 1/10  Iteration 171/1780 Training loss: 3.0808 9.2699 sec/batch
Epoch 1/10  Iteration 172/1780 Training loss: 3.0778 9.3761 sec/batch
Epoch 1/10  Iteration 173/1780 Training loss: 3.0750 9.3253 sec/batch
Epoch 1/10  Iteration 174/1780 Training loss: 3.0721 9.4542 sec/batch
Epoch 1/10  Iteration 175/1780 Training loss: 3.0694 9.3162 sec/batch
Epoch 1/10  Iteration 176/1780 Training loss: 3.0665 9.8546 sec/batch
Epoch 1/10  Iteration 177/1780 Training loss: 3.0634 10.7106 sec/batch
Epoch 1/10  Iteration 178/1780 Training loss: 3.0603 9.9179 sec/batch
Epoch 2/10  Iteration 179/1780 Training loss: 2.5807 10.1821 sec/batch
Epoch 2/10  Iteration 180/1780 Training loss: 2.5342 11.5611 sec/batch
Epoch 2/10  Iteration 181/1780 Training loss: 2.5233 11.4725 sec/batch
Epoch 2/10  Iteration 182/1780 Training loss: 2.5178 9.8376 sec/batch
Epoch 2/10  Iteration 183/1780 Training loss: 2.5146 10.3051 sec/batch
Epoch 2/10  Iteration 184/1780 Training loss: 2.5116 10.3519 sec/batch
Epoch 2/10  Iteration 185/1780 Training loss: 2.5082 15.0716 sec/batch
Epoch 2/10  Iteration 186/1780 Training loss: 2.5079 17.6590 sec/batch
Epoch 2/10  Iteration 187/1780 Training loss: 2.5076 18.4038 sec/batch
Epoch 2/10  Iteration 188/1780 Training loss: 2.5051 12.7675 sec/batch
Epoch 2/10  Iteration 189/1780 Training loss: 2.5017 12.1525 sec/batch
Epoch 2/10  Iteration 190/1780 Training loss: 2.5004 9.8212 sec/batch
Epoch 2/10  Iteration 191/1780 Training loss: 2.4981 10.7444 sec/batch
Epoch 2/10  Iteration 192/1780 Training loss: 2.4987 10.0353 sec/batch
Epoch 2/10  Iteration 193/1780 Training loss: 2.4970 9.4025 sec/batch
Epoch 2/10  Iteration 194/1780 Training loss: 2.4957 10.0830 sec/batch
Epoch 2/10  Iteration 195/1780 Training loss: 2.4938 10.3553 sec/batch
Epoch 2/10  Iteration 196/1780 Training loss: 2.4934 10.8893 sec/batch
Epoch 2/10  Iteration 197/1780 Training loss: 2.4925 11.2471 sec/batch
Epoch 2/10  Iteration 198/1780 Training loss: 2.4896 10.1686 sec/batch
Epoch 2/10  Iteration 199/1780 Training loss: 2.4874 11.5518 sec/batch
Epoch 2/10  Iteration 200/1780 Training loss: 2.4870 9.7768 sec/batch
Validation loss: 2.35732 Saving checkpoint!
Epoch 2/10  Iteration 201/1780 Training loss: 2.4861 12.0283 sec/batch
Epoch 2/10  Iteration 202/1780 Training loss: 2.4839 11.0132 sec/batch
Epoch 2/10  Iteration 203/1780 Training loss: 2.4816 10.8663 sec/batch
Epoch 2/10  Iteration 204/1780 Training loss: 2.4797 9.6225 sec/batch
Epoch 2/10  Iteration 205/1780 Training loss: 2.4775 9.4330 sec/batch
Epoch 2/10  Iteration 206/1780 Training loss: 2.4756 9.4319 sec/batch
Epoch 2/10  Iteration 207/1780 Training loss: 2.4743 9.4775 sec/batch
Epoch 2/10  Iteration 208/1780 Training loss: 2.4726 9.4515 sec/batch
Epoch 2/10  Iteration 209/1780 Training loss: 2.4713 10.0620 sec/batch
Epoch 2/10  Iteration 210/1780 Training loss: 2.4690 9.3115 sec/batch
Epoch 2/10  Iteration 211/1780 Training loss: 2.4670 9.8112 sec/batch
Epoch 2/10  Iteration 212/1780 Training loss: 2.4653 9.4611 sec/batch
Epoch 2/10  Iteration 213/1780 Training loss: 2.4634 8.3765 sec/batch
Epoch 2/10  Iteration 214/1780 Training loss: 2.4621 6.8705 sec/batch
Epoch 2/10  Iteration 215/1780 Training loss: 2.4604 6.9473 sec/batch
Epoch 2/10  Iteration 216/1780 Training loss: 2.4579 7.0015 sec/batch
Epoch 2/10  Iteration 217/1780 Training loss: 2.4559 6.9784 sec/batch
Epoch 2/10  Iteration 218/1780 Training loss: 2.4541 7.0482 sec/batch
Epoch 2/10  Iteration 219/1780 Training loss: 2.4520 6.8791 sec/batch
Epoch 2/10  Iteration 220/1780 Training loss: 2.4502 7.0491 sec/batch
Epoch 2/10  Iteration 221/1780 Training loss: 2.4482 6.9326 sec/batch
Epoch 2/10  Iteration 222/1780 Training loss: 2.4464 7.0973 sec/batch
Epoch 2/10  Iteration 223/1780 Training loss: 2.4447 7.2336 sec/batch
Epoch 2/10  Iteration 224/1780 Training loss: 2.4422 6.9319 sec/batch
Epoch 2/10  Iteration 225/1780 Training loss: 2.4409 6.9696 sec/batch
Epoch 2/10  Iteration 226/1780 Training loss: 2.4392 6.9784 sec/batch
Epoch 2/10  Iteration 227/1780 Training loss: 2.4378 7.5013 sec/batch
Epoch 2/10  Iteration 228/1780 Training loss: 2.4367 7.2066 sec/batch
Epoch 2/10  Iteration 229/1780 Training loss: 2.4349 7.3732 sec/batch
Epoch 2/10  Iteration 230/1780 Training loss: 2.4336 7.0334 sec/batch
Epoch 2/10  Iteration 231/1780 Training loss: 2.4320 8.1201 sec/batch
Epoch 2/10  Iteration 232/1780 Training loss: 2.4304 7.3672 sec/batch
Epoch 2/10  Iteration 233/1780 Training loss: 2.4287 7.4558 sec/batch
Epoch 2/10  Iteration 234/1780 Training loss: 2.4277 7.3887 sec/batch
Epoch 2/10  Iteration 235/1780 Training loss: 2.4263 7.2683 sec/batch
Epoch 2/10  Iteration 236/1780 Training loss: 2.4247 7.3762 sec/batch
Epoch 2/10  Iteration 237/1780 Training loss: 2.4232 7.2152 sec/batch
Epoch 2/10  Iteration 238/1780 Training loss: 2.4223 7.1796 sec/batch
Epoch 2/10  Iteration 239/1780 Training loss: 2.4207 7.2652 sec/batch
Epoch 2/10  Iteration 240/1780 Training loss: 2.4196 7.4880 sec/batch
Epoch 2/10  Iteration 241/1780 Training loss: 2.4186 7.3873 sec/batch
Epoch 2/10  Iteration 242/1780 Training loss: 2.4174 7.2274 sec/batch
Epoch 2/10  Iteration 243/1780 Training loss: 2.4159 6.9943 sec/batch
Epoch 2/10  Iteration 244/1780 Training loss: 2.4148 7.0073 sec/batch
Epoch 2/10  Iteration 245/1780 Training loss: 2.4135 7.0294 sec/batch
Epoch 2/10  Iteration 246/1780 Training loss: 2.4118 6.9447 sec/batch
Epoch 2/10  Iteration 247/1780 Training loss: 2.4101 7.0778 sec/batch
Epoch 2/10  Iteration 248/1780 Training loss: 2.4091 6.9408 sec/batch
Epoch 2/10  Iteration 249/1780 Training loss: 2.4081 7.0807 sec/batch
Epoch 2/10  Iteration 250/1780 Training loss: 2.4070 6.9965 sec/batch
Epoch 2/10  Iteration 251/1780 Training loss: 2.4057 6.9683 sec/batch
Epoch 2/10  Iteration 252/1780 Training loss: 2.4044 6.9402 sec/batch
Epoch 2/10  Iteration 253/1780 Training loss: 2.4031 7.0920 sec/batch
Epoch 2/10  Iteration 254/1780 Training loss: 2.4025 6.8619 sec/batch
Epoch 2/10  Iteration 255/1780 Training loss: 2.4012 7.0744 sec/batch
Epoch 2/10  Iteration 256/1780 Training loss: 2.4001 6.9485 sec/batch
Epoch 2/10  Iteration 257/1780 Training loss: 2.3986 7.1601 sec/batch
Epoch 2/10  Iteration 258/1780 Training loss: 2.3972 7.6058 sec/batch
Epoch 2/10  Iteration 259/1780 Training loss: 2.3958 6.9469 sec/batch
Epoch 2/10  Iteration 260/1780 Training loss: 2.3948 7.0896 sec/batch
Epoch 2/10  Iteration 261/1780 Training loss: 2.3932 7.0686 sec/batch
Epoch 2/10  Iteration 262/1780 Training loss: 2.3918 7.0560 sec/batch
Epoch 2/10  Iteration 263/1780 Training loss: 2.3900 7.1006 sec/batch
Epoch 2/10  Iteration 264/1780 Training loss: 2.3886 7.0105 sec/batch
Epoch 2/10  Iteration 265/1780 Training loss: 2.3874 7.1217 sec/batch
Epoch 2/10  Iteration 266/1780 Training loss: 2.3860 7.0419 sec/batch
Epoch 2/10  Iteration 267/1780 Training loss: 2.3845 7.2272 sec/batch
Epoch 2/10  Iteration 268/1780 Training loss: 2.3833 7.0445 sec/batch
Epoch 2/10  Iteration 269/1780 Training loss: 2.3819 7.1089 sec/batch
Epoch 2/10  Iteration 270/1780 Training loss: 2.3806 7.2579 sec/batch
Epoch 2/10  Iteration 271/1780 Training loss: 2.3792 7.0299 sec/batch
Epoch 2/10  Iteration 272/1780 Training loss: 2.3777 7.0263 sec/batch
Epoch 2/10  Iteration 273/1780 Training loss: 2.3763 7.2125 sec/batch
Epoch 2/10  Iteration 274/1780 Training loss: 2.3749 7.0090 sec/batch
Epoch 2/10  Iteration 275/1780 Training loss: 2.3736 6.9163 sec/batch
Epoch 2/10  Iteration 276/1780 Training loss: 2.3723 7.0818 sec/batch
Epoch 2/10  Iteration 277/1780 Training loss: 2.3709 6.9145 sec/batch
Epoch 2/10  Iteration 278/1780 Training loss: 2.3695 6.8998 sec/batch
Epoch 2/10  Iteration 279/1780 Training loss: 2.3684 7.0842 sec/batch
Epoch 2/10  Iteration 280/1780 Training loss: 2.3673 7.7837 sec/batch
Epoch 2/10  Iteration 281/1780 Training loss: 2.3659 6.8799 sec/batch
Epoch 2/10  Iteration 282/1780 Training loss: 2.3646 6.9583 sec/batch
Epoch 2/10  Iteration 283/1780 Training loss: 2.3633 6.9977 sec/batch
Epoch 2/10  Iteration 284/1780 Training loss: 2.3621 6.8967 sec/batch
Epoch 2/10  Iteration 285/1780 Training loss: 2.3608 7.1116 sec/batch
Epoch 2/10  Iteration 286/1780 Training loss: 2.3599 7.0441 sec/batch
Epoch 2/10  Iteration 287/1780 Training loss: 2.3589 7.2746 sec/batch
Epoch 2/10  Iteration 288/1780 Training loss: 2.3575 7.1023 sec/batch
Epoch 2/10  Iteration 289/1780 Training loss: 2.3565 6.9183 sec/batch
Epoch 2/10  Iteration 290/1780 Training loss: 2.3556 7.0137 sec/batch
Epoch 2/10  Iteration 291/1780 Training loss: 2.3542 7.3178 sec/batch
Epoch 2/10  Iteration 292/1780 Training loss: 2.3530 6.9944 sec/batch
Epoch 2/10  Iteration 293/1780 Training loss: 2.3517 6.9765 sec/batch
Epoch 2/10  Iteration 294/1780 Training loss: 2.3502 7.0266 sec/batch
Epoch 2/10  Iteration 295/1780 Training loss: 2.3491 6.9799 sec/batch
Epoch 2/10  Iteration 296/1780 Training loss: 2.3479 6.9617 sec/batch
Epoch 2/10  Iteration 297/1780 Training loss: 2.3469 6.9515 sec/batch
Epoch 2/10  Iteration 298/1780 Training loss: 2.3458 7.1615 sec/batch
Epoch 2/10  Iteration 299/1780 Training loss: 2.3448 6.9306 sec/batch
Epoch 2/10  Iteration 300/1780 Training loss: 2.3435 6.9891 sec/batch
Validation loss: 2.11823 Saving checkpoint!
Epoch 2/10  Iteration 301/1780 Training loss: 2.3424 7.5650 sec/batch
Epoch 2/10  Iteration 302/1780 Training loss: 2.3415 7.0455 sec/batch
Epoch 2/10  Iteration 303/1780 Training loss: 2.3404 6.8211 sec/batch
Epoch 2/10  Iteration 304/1780 Training loss: 2.3391 6.9556 sec/batch
Epoch 2/10  Iteration 305/1780 Training loss: 2.3381 6.9539 sec/batch
Epoch 2/10  Iteration 306/1780 Training loss: 2.3371 6.9746 sec/batch
Epoch 2/10  Iteration 307/1780 Training loss: 2.3361 6.8702 sec/batch
Epoch 2/10  Iteration 308/1780 Training loss: 2.3349 7.7249 sec/batch
Epoch 2/10  Iteration 309/1780 Training loss: 2.3338 6.9452 sec/batch
Epoch 2/10  Iteration 310/1780 Training loss: 2.3325 7.1964 sec/batch
Epoch 2/10  Iteration 311/1780 Training loss: 2.3314 6.9758 sec/batch
Epoch 2/10  Iteration 312/1780 Training loss: 2.3305 7.1486 sec/batch
Epoch 2/10  Iteration 313/1780 Training loss: 2.3293 6.8093 sec/batch
Epoch 2/10  Iteration 314/1780 Training loss: 2.3283 7.1180 sec/batch
Epoch 2/10  Iteration 315/1780 Training loss: 2.3273 7.4818 sec/batch
Epoch 2/10  Iteration 316/1780 Training loss: 2.3263 6.8312 sec/batch
Epoch 2/10  Iteration 317/1780 Training loss: 2.3255 6.9392 sec/batch
Epoch 2/10  Iteration 318/1780 Training loss: 2.3244 7.1153 sec/batch
Epoch 2/10  Iteration 319/1780 Training loss: 2.3235 7.0980 sec/batch
Epoch 2/10  Iteration 320/1780 Training loss: 2.3224 6.9704 sec/batch
Epoch 2/10  Iteration 321/1780 Training loss: 2.3213 6.9037 sec/batch
Epoch 2/10  Iteration 322/1780 Training loss: 2.3202 6.7931 sec/batch
Epoch 2/10  Iteration 323/1780 Training loss: 2.3191 6.9681 sec/batch
Epoch 2/10  Iteration 324/1780 Training loss: 2.3182 6.8803 sec/batch
Epoch 2/10  Iteration 325/1780 Training loss: 2.3172 6.9790 sec/batch
Epoch 2/10  Iteration 326/1780 Training loss: 2.3164 6.9092 sec/batch
Epoch 2/10  Iteration 327/1780 Training loss: 2.3153 6.9695 sec/batch
Epoch 2/10  Iteration 328/1780 Training loss: 2.3142 6.9130 sec/batch
Epoch 2/10  Iteration 329/1780 Training loss: 2.3131 6.8698 sec/batch
Epoch 2/10  Iteration 330/1780 Training loss: 2.3123 7.6945 sec/batch
Epoch 2/10  Iteration 331/1780 Training loss: 2.3114 7.0367 sec/batch
Epoch 2/10  Iteration 332/1780 Training loss: 2.3104 7.0807 sec/batch
Epoch 2/10  Iteration 333/1780 Training loss: 2.3093 6.8311 sec/batch
Epoch 2/10  Iteration 334/1780 Training loss: 2.3083 6.9270 sec/batch
Epoch 2/10  Iteration 335/1780 Training loss: 2.3073 6.8361 sec/batch
Epoch 2/10  Iteration 336/1780 Training loss: 2.3062 6.9792 sec/batch
Epoch 2/10  Iteration 337/1780 Training loss: 2.3050 6.8403 sec/batch
Epoch 2/10  Iteration 338/1780 Training loss: 2.3042 6.8910 sec/batch
Epoch 2/10  Iteration 339/1780 Training loss: 2.3033 6.8983 sec/batch
Epoch 2/10  Iteration 340/1780 Training loss: 2.3022 6.9851 sec/batch
Epoch 2/10  Iteration 341/1780 Training loss: 2.3012 6.8478 sec/batch
Epoch 2/10  Iteration 342/1780 Training loss: 2.3002 6.7910 sec/batch
Epoch 2/10  Iteration 343/1780 Training loss: 2.2993 6.8628 sec/batch
Epoch 2/10  Iteration 344/1780 Training loss: 2.2983 6.9502 sec/batch
Epoch 2/10  Iteration 345/1780 Training loss: 2.2974 6.8171 sec/batch
Epoch 2/10  Iteration 346/1780 Training loss: 2.2966 6.8937 sec/batch
Epoch 2/10  Iteration 347/1780 Training loss: 2.2956 7.3841 sec/batch
Epoch 2/10  Iteration 348/1780 Training loss: 2.2945 7.0907 sec/batch
Epoch 2/10  Iteration 349/1780 Training loss: 2.2934 7.0332 sec/batch
Epoch 2/10  Iteration 350/1780 Training loss: 2.2924 7.1953 sec/batch
Epoch 2/10  Iteration 351/1780 Training loss: 2.2916 7.0024 sec/batch
Epoch 2/10  Iteration 352/1780 Training loss: 2.2907 7.2399 sec/batch
Epoch 2/10  Iteration 353/1780 Training loss: 2.2898 7.1793 sec/batch
Epoch 2/10  Iteration 354/1780 Training loss: 2.2888 6.8808 sec/batch
Epoch 2/10  Iteration 355/1780 Training loss: 2.2878 6.7756 sec/batch
Epoch 2/10  Iteration 356/1780 Training loss: 2.2868 6.9950 sec/batch
Epoch 3/10  Iteration 357/1780 Training loss: 2.1902 6.9989 sec/batch
Epoch 3/10  Iteration 358/1780 Training loss: 2.1386 7.1549 sec/batch
Epoch 3/10  Iteration 359/1780 Training loss: 2.1206 6.8268 sec/batch
Epoch 3/10  Iteration 360/1780 Training loss: 2.1143 6.9045 sec/batch
Epoch 3/10  Iteration 361/1780 Training loss: 2.1120 7.0022 sec/batch
Epoch 3/10  Iteration 362/1780 Training loss: 2.1063 7.4660 sec/batch
Epoch 3/10  Iteration 363/1780 Training loss: 2.1075 6.9231 sec/batch
Epoch 3/10  Iteration 364/1780 Training loss: 2.1068 6.9305 sec/batch
Epoch 3/10  Iteration 365/1780 Training loss: 2.1097 7.0724 sec/batch
Epoch 3/10  Iteration 366/1780 Training loss: 2.1085 7.0685 sec/batch
Epoch 3/10  Iteration 367/1780 Training loss: 2.1053 7.0664 sec/batch
Epoch 3/10  Iteration 368/1780 Training loss: 2.1031 7.1410 sec/batch
Epoch 3/10  Iteration 369/1780 Training loss: 2.1025 7.1098 sec/batch
Epoch 3/10  Iteration 370/1780 Training loss: 2.1046 6.9442 sec/batch
Epoch 3/10  Iteration 371/1780 Training loss: 2.1029 7.4033 sec/batch
Epoch 3/10  Iteration 372/1780 Training loss: 2.1014 7.0219 sec/batch
Epoch 3/10  Iteration 373/1780 Training loss: 2.1002 6.9702 sec/batch
Epoch 3/10  Iteration 374/1780 Training loss: 2.1017 7.0284 sec/batch
Epoch 3/10  Iteration 375/1780 Training loss: 2.1010 6.8720 sec/batch
Epoch 3/10  Iteration 376/1780 Training loss: 2.1002 6.9002 sec/batch
Epoch 3/10  Iteration 377/1780 Training loss: 2.0984 6.7972 sec/batch
Epoch 3/10  Iteration 378/1780 Training loss: 2.0995 6.8543 sec/batch
Epoch 3/10  Iteration 379/1780 Training loss: 2.0980 6.9625 sec/batch
Epoch 3/10  Iteration 380/1780 Training loss: 2.0965 7.0222 sec/batch
Epoch 3/10  Iteration 381/1780 Training loss: 2.0959 6.7683 sec/batch
Epoch 3/10  Iteration 382/1780 Training loss: 2.0942 6.7866 sec/batch
Epoch 3/10  Iteration 383/1780 Training loss: 2.0928 6.9850 sec/batch
Epoch 3/10  Iteration 384/1780 Training loss: 2.0922 7.0731 sec/batch
Epoch 3/10  Iteration 385/1780 Training loss: 2.0923 7.0090 sec/batch
Epoch 3/10  Iteration 386/1780 Training loss: 2.0917 7.1019 sec/batch
Epoch 3/10  Iteration 387/1780 Training loss: 2.0912 7.0778 sec/batch
Epoch 3/10  Iteration 388/1780 Training loss: 2.0898 9.1169 sec/batch
Epoch 3/10  Iteration 389/1780 Training loss: 2.0888 6.8848 sec/batch
Epoch 3/10  Iteration 390/1780 Training loss: 2.0888 6.8766 sec/batch
Epoch 3/10  Iteration 391/1780 Training loss: 2.0877 6.9632 sec/batch
Epoch 3/10  Iteration 392/1780 Training loss: 2.0867 6.8826 sec/batch
Epoch 3/10  Iteration 393/1780 Training loss: 2.0859 6.8317 sec/batch
Epoch 3/10  Iteration 394/1780 Training loss: 2.0838 6.7787 sec/batch
Epoch 3/10  Iteration 395/1780 Training loss: 2.0820 6.8636 sec/batch
Epoch 3/10  Iteration 396/1780 Training loss: 2.0805 6.8867 sec/batch
Epoch 3/10  Iteration 397/1780 Training loss: 2.0792 6.8231 sec/batch
Epoch 3/10  Iteration 398/1780 Training loss: 2.0783 6.8380 sec/batch
Epoch 3/10  Iteration 399/1780 Training loss: 2.0770 6.7613 sec/batch
Epoch 3/10  Iteration 400/1780 Training loss: 2.0757 6.8516 sec/batch
Validation loss: 1.94085 Saving checkpoint!
Epoch 3/10  Iteration 401/1780 Training loss: 2.0757 6.7949 sec/batch
Epoch 3/10  Iteration 402/1780 Training loss: 2.0736 6.9053 sec/batch
Epoch 3/10  Iteration 403/1780 Training loss: 2.0729 6.8188 sec/batch
Epoch 3/10  Iteration 404/1780 Training loss: 2.0715 6.7814 sec/batch
Epoch 3/10  Iteration 405/1780 Training loss: 2.0706 6.8840 sec/batch
Epoch 3/10  Iteration 406/1780 Training loss: 2.0706 7.1220 sec/batch
Epoch 3/10  Iteration 407/1780 Training loss: 2.0692 6.8077 sec/batch
Epoch 3/10  Iteration 408/1780 Training loss: 2.0691 6.9386 sec/batch
Epoch 3/10  Iteration 409/1780 Training loss: 2.0681 6.9262 sec/batch
Epoch 3/10  Iteration 410/1780 Training loss: 2.0672 7.1723 sec/batch
Epoch 3/10  Iteration 411/1780 Training loss: 2.0664 6.8566 sec/batch
Epoch 3/10  Iteration 412/1780 Training loss: 2.0661 10.5486 sec/batch
Epoch 3/10  Iteration 413/1780 Training loss: 2.0656 9.2334 sec/batch
Epoch 3/10  Iteration 414/1780 Training loss: 2.0650 10.7702 sec/batch
Epoch 3/10  Iteration 415/1780 Training loss: 2.0641 9.2303 sec/batch
Epoch 3/10  Iteration 416/1780 Training loss: 2.0638 8.7202 sec/batch
Epoch 3/10  Iteration 417/1780 Training loss: 2.0631 8.8498 sec/batch
Epoch 3/10  Iteration 418/1780 Training loss: 2.0633 8.7464 sec/batch
Epoch 3/10  Iteration 419/1780 Training loss: 2.0631 8.5261 sec/batch
Epoch 3/10  Iteration 420/1780 Training loss: 2.0626 9.1371 sec/batch
Epoch 3/10  Iteration 421/1780 Training loss: 2.0618 10.3922 sec/batch
Epoch 3/10  Iteration 422/1780 Training loss: 2.0614 9.8122 sec/batch
Epoch 3/10  Iteration 423/1780 Training loss: 2.0610 9.4165 sec/batch
Epoch 3/10  Iteration 424/1780 Training loss: 2.0598 8.8679 sec/batch
Epoch 3/10  Iteration 425/1780 Training loss: 2.0589 8.8670 sec/batch
Epoch 3/10  Iteration 426/1780 Training loss: 2.0583 8.5538 sec/batch
Epoch 3/10  Iteration 427/1780 Training loss: 2.0581 8.6322 sec/batch
Epoch 3/10  Iteration 428/1780 Training loss: 2.0575 9.2960 sec/batch
Epoch 3/10  Iteration 429/1780 Training loss: 2.0571 9.0986 sec/batch
Epoch 3/10  Iteration 430/1780 Training loss: 2.0561 9.4075 sec/batch
Epoch 3/10  Iteration 431/1780 Training loss: 2.0553 3604.9575 sec/batch
Epoch 3/10  Iteration 432/1780 Training loss: 2.0549 8.3439 sec/batch
Epoch 3/10  Iteration 433/1780 Training loss: 2.0541 11.6221 sec/batch
Epoch 3/10  Iteration 434/1780 Training loss: 2.0536 12.6213 sec/batch
Epoch 3/10  Iteration 435/1780 Training loss: 2.0526 12.6683 sec/batch
Epoch 3/10  Iteration 436/1780 Training loss: 2.0518 12.3524 sec/batch
Epoch 3/10  Iteration 437/1780 Training loss: 2.0506 13.3493 sec/batch
Epoch 3/10  Iteration 438/1780 Training loss: 2.0502 11.9949 sec/batch
Epoch 3/10  Iteration 439/1780 Training loss: 2.0491 12.2854 sec/batch
Epoch 3/10  Iteration 440/1780 Training loss: 2.0482 11.8733 sec/batch
Epoch 3/10  Iteration 441/1780 Training loss: 2.0471 12.0315 sec/batch
Epoch 3/10  Iteration 442/1780 Training loss: 2.0463 11.8910 sec/batch
Epoch 3/10  Iteration 443/1780 Training loss: 2.0455 12.3048 sec/batch
Epoch 3/10  Iteration 444/1780 Training loss: 2.0445 12.1885 sec/batch
Epoch 3/10  Iteration 445/1780 Training loss: 2.0435 12.0530 sec/batch
Epoch 3/10  Iteration 446/1780 Training loss: 2.0429 12.8856 sec/batch
Epoch 3/10  Iteration 447/1780 Training loss: 2.0420 12.3844 sec/batch
Epoch 3/10  Iteration 448/1780 Training loss: 2.0414 11.9875 sec/batch
Epoch 3/10  Iteration 449/1780 Training loss: 2.0402 12.2801 sec/batch
Epoch 3/10  Iteration 450/1780 Training loss: 2.0392 12.6565 sec/batch
Epoch 3/10  Iteration 451/1780 Training loss: 2.0383 12.3362 sec/batch
Epoch 3/10  Iteration 452/1780 Training loss: 2.0374 12.6443 sec/batch
Epoch 3/10  Iteration 453/1780 Training loss: 2.0367 12.1997 sec/batch
Epoch 3/10  Iteration 454/1780 Training loss: 2.0358 12.7513 sec/batch
Epoch 3/10  Iteration 455/1780 Training loss: 2.0347 12.0234 sec/batch
Epoch 3/10  Iteration 456/1780 Training loss: 2.0336 12.3029 sec/batch
Epoch 3/10  Iteration 457/1780 Training loss: 2.0330 12.0869 sec/batch
Epoch 3/10  Iteration 458/1780 Training loss: 2.0323 12.3627 sec/batch
Epoch 3/10  Iteration 459/1780 Training loss: 2.0315 11.9973 sec/batch
Epoch 3/10  Iteration 460/1780 Training loss: 2.0307 12.1853 sec/batch
Epoch 3/10  Iteration 461/1780 Training loss: 2.0299 11.7070 sec/batch
Epoch 3/10  Iteration 462/1780 Training loss: 2.0292 12.7208 sec/batch
Epoch 3/10  Iteration 463/1780 Training loss: 2.0286 12.3874 sec/batch
Epoch 3/10  Iteration 464/1780 Training loss: 2.0280 12.1240 sec/batch
Epoch 3/10  Iteration 465/1780 Training loss: 2.0275 12.7120 sec/batch
Epoch 3/10  Iteration 466/1780 Training loss: 2.0268 12.3943 sec/batch
Epoch 3/10  Iteration 467/1780 Training loss: 2.0261 12.2766 sec/batch
Epoch 3/10  Iteration 468/1780 Training loss: 2.0254 12.9195 sec/batch
Epoch 3/10  Iteration 469/1780 Training loss: 2.0247 12.5929 sec/batch
Epoch 3/10  Iteration 470/1780 Training loss: 2.0239 12.6291 sec/batch
Epoch 3/10  Iteration 471/1780 Training loss: 2.0230 12.3376 sec/batch
Epoch 3/10  Iteration 472/1780 Training loss: 2.0220 12.8298 sec/batch
Epoch 3/10  Iteration 473/1780 Training loss: 2.0213 11.9882 sec/batch
Epoch 3/10  Iteration 474/1780 Training loss: 2.0206 12.3945 sec/batch
Epoch 3/10  Iteration 475/1780 Training loss: 2.0199 11.9685 sec/batch
Epoch 3/10  Iteration 476/1780 Training loss: 2.0193 12.1424 sec/batch
Epoch 3/10  Iteration 477/1780 Training loss: 2.0187 11.9160 sec/batch
Epoch 3/10  Iteration 478/1780 Training loss: 2.0178 11.5446 sec/batch
Epoch 3/10  Iteration 479/1780 Training loss: 2.0170 11.1357 sec/batch
Epoch 3/10  Iteration 480/1780 Training loss: 2.0165 11.2234 sec/batch
Epoch 3/10  Iteration 481/1780 Training loss: 2.0159 11.5205 sec/batch
Epoch 3/10  Iteration 482/1780 Training loss: 2.0149 10.8157 sec/batch
Epoch 3/10  Iteration 483/1780 Training loss: 2.0144 10.7139 sec/batch
Epoch 3/10  Iteration 484/1780 Training loss: 2.0139 10.8881 sec/batch
Epoch 3/10  Iteration 485/1780 Training loss: 2.0132 11.0982 sec/batch
Epoch 3/10  Iteration 486/1780 Training loss: 2.0126 10.7577 sec/batch
Epoch 3/10  Iteration 487/1780 Training loss: 2.0119 10.8168 sec/batch
Epoch 3/10  Iteration 488/1780 Training loss: 2.0110 10.9978 sec/batch
Epoch 3/10  Iteration 489/1780 Training loss: 2.0105 10.5637 sec/batch
Epoch 3/10  Iteration 490/1780 Training loss: 2.0100 10.8809 sec/batch
Epoch 3/10  Iteration 491/1780 Training loss: 2.0094 11.1973 sec/batch
Epoch 3/10  Iteration 492/1780 Training loss: 2.0089 10.6933 sec/batch
Epoch 3/10  Iteration 493/1780 Training loss: 2.0083 11.2172 sec/batch
Epoch 3/10  Iteration 494/1780 Training loss: 2.0078 10.8311 sec/batch
Epoch 3/10  Iteration 495/1780 Training loss: 2.0074 10.9341 sec/batch
Epoch 3/10  Iteration 496/1780 Training loss: 2.0067 10.9607 sec/batch
Epoch 3/10  Iteration 497/1780 Training loss: 2.0064 11.1657 sec/batch
Epoch 3/10  Iteration 498/1780 Training loss: 2.0057 10.6676 sec/batch
Epoch 3/10  Iteration 499/1780 Training loss: 2.0050 10.9325 sec/batch
Epoch 3/10  Iteration 500/1780 Training loss: 2.0044 11.1584 sec/batch
Validation loss: 1.81833 Saving checkpoint!
Epoch 3/10  Iteration 501/1780 Training loss: 2.0040 16.0211 sec/batch
Epoch 3/10  Iteration 502/1780 Training loss: 2.0035 13.3402 sec/batch
Epoch 3/10  Iteration 503/1780 Training loss: 2.0029 13.6242 sec/batch
Epoch 3/10  Iteration 504/1780 Training loss: 2.0026 15.1152 sec/batch
Epoch 3/10  Iteration 505/1780 Training loss: 2.0020 19.2052 sec/batch
Epoch 3/10  Iteration 506/1780 Training loss: 2.0013 27.3514 sec/batch
Epoch 3/10  Iteration 507/1780 Training loss: 2.0007 24.6905 sec/batch
Epoch 3/10  Iteration 508/1780 Training loss: 2.0003 23.3035 sec/batch
Epoch 3/10  Iteration 509/1780 Training loss: 1.9997 16.0641 sec/batch
Epoch 3/10  Iteration 510/1780 Training loss: 1.9992 14.6918 sec/batch
Epoch 3/10  Iteration 511/1780 Training loss: 1.9986 14.6235 sec/batch
Epoch 3/10  Iteration 512/1780 Training loss: 1.9980 20.4454 sec/batch
Epoch 3/10  Iteration 513/1780 Training loss: 1.9974 27.2581 sec/batch
Epoch 3/10  Iteration 514/1780 Training loss: 1.9967 24.4856 sec/batch
Epoch 3/10  Iteration 515/1780 Training loss: 1.9960 24.2903 sec/batch
Epoch 3/10  Iteration 516/1780 Training loss: 1.9956 18.9933 sec/batch
Epoch 3/10  Iteration 517/1780 Training loss: 1.9952 15.0696 sec/batch
Epoch 3/10  Iteration 518/1780 Training loss: 1.9946 16.0661 sec/batch
Epoch 3/10  Iteration 519/1780 Training loss: 1.9941 17.3323 sec/batch
Epoch 3/10  Iteration 520/1780 Training loss: 1.9935 21.8341 sec/batch
Epoch 3/10  Iteration 521/1780 Training loss: 1.9929 24.3947 sec/batch
Epoch 3/10  Iteration 522/1780 Training loss: 1.9923 24.7233 sec/batch
Epoch 3/10  Iteration 523/1780 Training loss: 1.9918 19.7125 sec/batch
Epoch 3/10  Iteration 524/1780 Training loss: 1.9915 17.5116 sec/batch
Epoch 3/10  Iteration 525/1780 Training loss: 1.9909 16.7108 sec/batch
Epoch 3/10  Iteration 526/1780 Training loss: 1.9903 18.6587 sec/batch
Epoch 3/10  Iteration 527/1780 Training loss: 1.9897 22.9868 sec/batch
Epoch 3/10  Iteration 528/1780 Training loss: 1.9890 25.9164 sec/batch
Epoch 3/10  Iteration 529/1780 Training loss: 1.9885 24.4077 sec/batch
Epoch 3/10  Iteration 530/1780 Training loss: 1.9880 19.0289 sec/batch
Epoch 3/10  Iteration 531/1780 Training loss: 1.9875 17.8454 sec/batch
Epoch 3/10  Iteration 532/1780 Training loss: 1.9869 18.3322 sec/batch
Epoch 3/10  Iteration 533/1780 Training loss: 1.9862 21.5359 sec/batch
Epoch 3/10  Iteration 534/1780 Training loss: 1.9857 24.8877 sec/batch
Epoch 4/10  Iteration 535/1780 Training loss: 1.9635 24.6636 sec/batch
Epoch 4/10  Iteration 536/1780 Training loss: 1.9144 21.1631 sec/batch
Epoch 4/10  Iteration 537/1780 Training loss: 1.8988 18.8282 sec/batch
Epoch 4/10  Iteration 538/1780 Training loss: 1.8921 18.8620 sec/batch
Epoch 4/10  Iteration 539/1780 Training loss: 1.8883 20.2528 sec/batch
Epoch 4/10  Iteration 540/1780 Training loss: 1.8804 24.2054 sec/batch
Epoch 4/10  Iteration 541/1780 Training loss: 1.8803 24.5167 sec/batch
Epoch 4/10  Iteration 542/1780 Training loss: 1.8793 21.9633 sec/batch
Epoch 4/10  Iteration 543/1780 Training loss: 1.8813 19.5686 sec/batch
Epoch 4/10  Iteration 544/1780 Training loss: 1.8798 24.2078 sec/batch
Epoch 4/10  Iteration 545/1780 Training loss: 1.8761 26.9452 sec/batch
Epoch 4/10  Iteration 546/1780 Training loss: 1.8744 31.7684 sec/batch
Epoch 4/10  Iteration 547/1780 Training loss: 1.8740 30.4109 sec/batch
Epoch 4/10  Iteration 548/1780 Training loss: 1.8761 24.0584 sec/batch
Epoch 4/10  Iteration 549/1780 Training loss: 1.8753 22.4180 sec/batch
Epoch 4/10  Iteration 550/1780 Training loss: 1.8738 21.9094 sec/batch
Epoch 4/10  Iteration 551/1780 Training loss: 1.8733 25.7856 sec/batch
Epoch 4/10  Iteration 552/1780 Training loss: 1.8754 23.7094 sec/batch
Epoch 4/10  Iteration 553/1780 Training loss: 1.8751 22.5321 sec/batch
Epoch 4/10  Iteration 554/1780 Training loss: 1.8756 21.4744 sec/batch
Epoch 4/10  Iteration 555/1780 Training loss: 1.8742 21.7946 sec/batch
Epoch 4/10  Iteration 556/1780 Training loss: 1.8753 24.2971 sec/batch
Epoch 4/10  Iteration 557/1780 Training loss: 1.8743 25.0342 sec/batch
Epoch 4/10  Iteration 558/1780 Training loss: 1.8737 23.7416 sec/batch
Epoch 4/10  Iteration 559/1780 Training loss: 1.8727 24.7903 sec/batch
Epoch 4/10  Iteration 560/1780 Training loss: 1.8707 22.7485 sec/batch
Epoch 4/10  Iteration 561/1780 Training loss: 1.8689 24.1735 sec/batch
Epoch 4/10  Iteration 562/1780 Training loss: 1.8687 26.0242 sec/batch
Epoch 4/10  Iteration 563/1780 Training loss: 1.8695 23.7397 sec/batch
Epoch 4/10  Iteration 564/1780 Training loss: 1.8693 23.4912 sec/batch
Epoch 4/10  Iteration 565/1780 Training loss: 1.8687 22.5797 sec/batch
Epoch 4/10  Iteration 566/1780 Training loss: 1.8670 24.2282 sec/batch
Epoch 4/10  Iteration 567/1780 Training loss: 1.8664 24.8514 sec/batch
Epoch 4/10  Iteration 568/1780 Training loss: 1.8668 22.9378 sec/batch
Epoch 4/10  Iteration 569/1780 Training loss: 1.8663 24.2701 sec/batch
Epoch 4/10  Iteration 570/1780 Training loss: 1.8657 24.0564 sec/batch
Epoch 4/10  Iteration 571/1780 Training loss: 1.8649 23.5234 sec/batch
Epoch 4/10  Iteration 572/1780 Training loss: 1.8635 24.7833 sec/batch
Epoch 4/10  Iteration 573/1780 Training loss: 1.8618 23.3680 sec/batch
Epoch 4/10  Iteration 574/1780 Training loss: 1.8606 22.7327 sec/batch
Epoch 4/10  Iteration 575/1780 Training loss: 1.8597 24.9990 sec/batch
Epoch 4/10  Iteration 576/1780 Training loss: 1.8594 23.7783 sec/batch
Epoch 4/10  Iteration 577/1780 Training loss: 1.8584 24.2855 sec/batch
Epoch 4/10  Iteration 578/1780 Training loss: 1.8572 24.2373 sec/batch
Epoch 4/10  Iteration 579/1780 Training loss: 1.8568 22.4670 sec/batch
Epoch 4/10  Iteration 580/1780 Training loss: 1.8550 24.2606 sec/batch
Epoch 4/10  Iteration 581/1780 Training loss: 1.8542 22.2424 sec/batch
Epoch 4/10  Iteration 582/1780 Training loss: 1.8532 24.0529 sec/batch
Epoch 4/10  Iteration 583/1780 Training loss: 1.8527 24.6248 sec/batch
Epoch 4/10  Iteration 584/1780 Training loss: 1.8528 26.4233 sec/batch
Epoch 4/10  Iteration 585/1780 Training loss: 1.8517 27.4043 sec/batch
Epoch 4/10  Iteration 586/1780 Training loss: 1.8523 26.5664 sec/batch
Epoch 4/10  Iteration 587/1780 Training loss: 1.8517 24.2547 sec/batch
Epoch 4/10  Iteration 588/1780 Training loss: 1.8511 21.9004 sec/batch
Epoch 4/10  Iteration 589/1780 Training loss: 1.8502 24.3566 sec/batch
Epoch 4/10  Iteration 590/1780 Training loss: 1.8501 22.3133 sec/batch
Epoch 4/10  Iteration 591/1780 Training loss: 1.8500 24.8072 sec/batch
Epoch 4/10  Iteration 592/1780 Training loss: 1.8492 22.2221 sec/batch
Epoch 4/10  Iteration 593/1780 Training loss: 1.8484 23.5325 sec/batch
Epoch 4/10  Iteration 594/1780 Training loss: 1.8486 22.7533 sec/batch
Epoch 4/10  Iteration 595/1780 Training loss: 1.8480 27.0753 sec/batch
Epoch 4/10  Iteration 596/1780 Training loss: 1.8484 26.6192 sec/batch
Epoch 4/10  Iteration 597/1780 Training loss: 1.8484 23.0584 sec/batch
Epoch 4/10  Iteration 598/1780 Training loss: 1.8483 22.0932 sec/batch
Epoch 4/10  Iteration 599/1780 Training loss: 1.8479 23.3785 sec/batch
Epoch 4/10  Iteration 600/1780 Training loss: 1.8480 22.3273 sec/batch
Validation loss: 1.71183 Saving checkpoint!
Epoch 4/10  Iteration 601/1780 Training loss: 1.8484 29.4970 sec/batch
Epoch 4/10  Iteration 602/1780 Training loss: 1.8478 24.4736 sec/batch
Epoch 4/10  Iteration 603/1780 Training loss: 1.8472 22.2805 sec/batch
Epoch 4/10  Iteration 604/1780 Training loss: 1.8467 34.0755 sec/batch
Epoch 4/10  Iteration 605/1780 Training loss: 1.8468 34.7766 sec/batch
Epoch 4/10  Iteration 606/1780 Training loss: 1.8464 35.5899 sec/batch
Epoch 4/10  Iteration 607/1780 Training loss: 1.8463 31.5960 sec/batch
Epoch 4/10  Iteration 608/1780 Training loss: 1.8457 30.5205 sec/batch
Epoch 4/10  Iteration 609/1780 Training loss: 1.8453 30.1313 sec/batch
Epoch 4/10  Iteration 610/1780 Training loss: 1.8452 28.7374 sec/batch
Epoch 4/10  Iteration 611/1780 Training loss: 1.8447 24.0716 sec/batch
Epoch 4/10  Iteration 612/1780 Training loss: 1.8444 22.0976 sec/batch
Epoch 4/10  Iteration 613/1780 Training loss: 1.8435 23.4752 sec/batch
Epoch 4/10  Iteration 614/1780 Training loss: 1.8429 22.0504 sec/batch
Epoch 4/10  Iteration 615/1780 Training loss: 1.8420 22.6395 sec/batch
Epoch 4/10  Iteration 616/1780 Training loss: 1.8417 22.0354 sec/batch
Epoch 4/10  Iteration 617/1780 Training loss: 1.8408 23.3201 sec/batch
Epoch 4/10  Iteration 618/1780 Training loss: 1.8404 22.8644 sec/batch
Epoch 4/10  Iteration 619/1780 Training loss: 1.8396 21.9459 sec/batch
Epoch 4/10  Iteration 620/1780 Training loss: 1.8389 24.6722 sec/batch
Epoch 4/10  Iteration 621/1780 Training loss: 1.8383 24.6634 sec/batch
Epoch 4/10  Iteration 622/1780 Training loss: 1.8376 22.7441 sec/batch
Epoch 4/10  Iteration 623/1780 Training loss: 1.8367 22.4176 sec/batch
Epoch 4/10  Iteration 624/1780 Training loss: 1.8363 22.8743 sec/batch
Epoch 4/10  Iteration 625/1780 Training loss: 1.8355 29.1046 sec/batch
Epoch 4/10  Iteration 626/1780 Training loss: 1.8350 28.5266 sec/batch
Epoch 4/10  Iteration 627/1780 Training loss: 1.8341 22.6285 sec/batch
Epoch 4/10  Iteration 628/1780 Training loss: 1.8334 23.5053 sec/batch
Epoch 4/10  Iteration 629/1780 Training loss: 1.8326 23.1774 sec/batch
Epoch 4/10  Iteration 630/1780 Training loss: 1.8322 23.5583 sec/batch
Epoch 4/10  Iteration 631/1780 Training loss: 1.8317 22.5613 sec/batch
Epoch 4/10  Iteration 632/1780 Training loss: 1.8309 20.6249 sec/batch
Epoch 4/10  Iteration 633/1780 Training loss: 1.8302 23.4492 sec/batch
Epoch 4/10  Iteration 634/1780 Training loss: 1.8293 22.8402 sec/batch
Epoch 4/10  Iteration 635/1780 Training loss: 1.8289 25.1102 sec/batch
Epoch 4/10  Iteration 636/1780 Training loss: 1.8284 23.0686 sec/batch
Epoch 4/10  Iteration 637/1780 Training loss: 1.8277 22.0537 sec/batch
Epoch 4/10  Iteration 638/1780 Training loss: 1.8272 21.8511 sec/batch
Epoch 4/10  Iteration 639/1780 Training loss: 1.8266 20.8359 sec/batch
Epoch 4/10  Iteration 640/1780 Training loss: 1.8261 22.6692 sec/batch
Epoch 4/10  Iteration 641/1780 Training loss: 1.8257 25.0073 sec/batch
Epoch 4/10  Iteration 642/1780 Training loss: 1.8253 22.7214 sec/batch
Epoch 4/10  Iteration 643/1780 Training loss: 1.8249 21.9382 sec/batch
Epoch 4/10  Iteration 644/1780 Training loss: 1.8245 20.3288 sec/batch
Epoch 4/10  Iteration 645/1780 Training loss: 1.8241 21.8213 sec/batch
Epoch 4/10  Iteration 646/1780 Training loss: 1.8236 25.0651 sec/batch
Epoch 4/10  Iteration 647/1780 Training loss: 1.8231 27.1133 sec/batch
Epoch 4/10  Iteration 648/1780 Training loss: 1.8226 21.4947 sec/batch
Epoch 4/10  Iteration 649/1780 Training loss: 1.8220 3546.0459 sec/batch
Epoch 4/10  Iteration 650/1780 Training loss: 1.8213 8.4220 sec/batch
Epoch 4/10  Iteration 651/1780 Training loss: 1.8208 11.5055 sec/batch
Epoch 4/10  Iteration 652/1780 Training loss: 1.8203 12.5319 sec/batch
Epoch 4/10  Iteration 653/1780 Training loss: 1.8199 12.0577 sec/batch
Epoch 4/10  Iteration 654/1780 Training loss: 1.8194 11.9819 sec/batch
Epoch 4/10  Iteration 655/1780 Training loss: 1.8191 11.8451 sec/batch
Epoch 4/10  Iteration 656/1780 Training loss: 1.8184 12.3749 sec/batch
Epoch 4/10  Iteration 657/1780 Training loss: 1.8177 11.8282 sec/batch
Epoch 4/10  Iteration 658/1780 Training loss: 1.8174 12.0194 sec/batch
Epoch 4/10  Iteration 659/1780 Training loss: 1.8169 11.9927 sec/batch
Epoch 4/10  Iteration 660/1780 Training loss: 1.8161 11.7923 sec/batch
Epoch 4/10  Iteration 661/1780 Training loss: 1.8159 15.0952 sec/batch
Epoch 4/10  Iteration 662/1780 Training loss: 1.8155 12.1861 sec/batch
Epoch 4/10  Iteration 663/1780 Training loss: 1.8151 11.9211 sec/batch
Epoch 4/10  Iteration 664/1780 Training loss: 1.8146 12.6358 sec/batch
Epoch 4/10  Iteration 665/1780 Training loss: 1.8140 12.1066 sec/batch
Epoch 4/10  Iteration 666/1780 Training loss: 1.8133 12.2033 sec/batch
Epoch 4/10  Iteration 667/1780 Training loss: 1.8130 12.1964 sec/batch
Epoch 4/10  Iteration 668/1780 Training loss: 1.8127 12.3023 sec/batch
Epoch 4/10  Iteration 669/1780 Training loss: 1.8123 12.4557 sec/batch
Epoch 4/10  Iteration 670/1780 Training loss: 1.8120 12.1175 sec/batch
Epoch 4/10  Iteration 671/1780 Training loss: 1.8116 12.3255 sec/batch
Epoch 4/10  Iteration 672/1780 Training loss: 1.8114 11.9742 sec/batch
Epoch 4/10  Iteration 673/1780 Training loss: 1.8111 12.0562 sec/batch
Epoch 4/10  Iteration 674/1780 Training loss: 1.8107 12.1549 sec/batch
Epoch 4/10  Iteration 675/1780 Training loss: 1.8106 11.8092 sec/batch
Epoch 4/10  Iteration 676/1780 Training loss: 1.8101 12.3783 sec/batch
Epoch 4/10  Iteration 677/1780 Training loss: 1.8097 12.2190 sec/batch
Epoch 4/10  Iteration 678/1780 Training loss: 1.8094 12.0136 sec/batch
Epoch 4/10  Iteration 679/1780 Training loss: 1.8089 12.4060 sec/batch
Epoch 4/10  Iteration 680/1780 Training loss: 1.8086 12.3562 sec/batch
Epoch 4/10  Iteration 681/1780 Training loss: 1.8083 12.3030 sec/batch
Epoch 4/10  Iteration 682/1780 Training loss: 1.8081 13.3168 sec/batch
Epoch 4/10  Iteration 683/1780 Training loss: 1.8077 12.6342 sec/batch
Epoch 4/10  Iteration 684/1780 Training loss: 1.8072 12.0727 sec/batch
Epoch 4/10  Iteration 685/1780 Training loss: 1.8067 12.9314 sec/batch
Epoch 4/10  Iteration 686/1780 Training loss: 1.8064 12.1202 sec/batch
Epoch 4/10  Iteration 687/1780 Training loss: 1.8060 12.0200 sec/batch
Epoch 4/10  Iteration 688/1780 Training loss: 1.8057 12.3997 sec/batch
Epoch 4/10  Iteration 689/1780 Training loss: 1.8053 12.4431 sec/batch
Epoch 4/10  Iteration 690/1780 Training loss: 1.8049 12.6162 sec/batch
Epoch 4/10  Iteration 691/1780 Training loss: 1.8046 12.6201 sec/batch
Epoch 4/10  Iteration 692/1780 Training loss: 1.8042 11.9137 sec/batch
Epoch 4/10  Iteration 693/1780 Training loss: 1.8036 12.5613 sec/batch
Epoch 4/10  Iteration 694/1780 Training loss: 1.8034 12.1833 sec/batch
Epoch 4/10  Iteration 695/1780 Training loss: 1.8032 11.5883 sec/batch
Epoch 4/10  Iteration 696/1780 Training loss: 1.8028 12.5444 sec/batch
Epoch 4/10  Iteration 697/1780 Training loss: 1.8026 11.7602 sec/batch
Epoch 4/10  Iteration 698/1780 Training loss: 1.8023 12.4248 sec/batch
Epoch 4/10  Iteration 699/1780 Training loss: 1.8019 11.9087 sec/batch
Epoch 4/10  Iteration 700/1780 Training loss: 1.8015 12.0042 sec/batch
Validation loss: 1.62455 Saving checkpoint!
Epoch 4/10  Iteration 701/1780 Training loss: 1.8016 13.7549 sec/batch
Epoch 4/10  Iteration 702/1780 Training loss: 1.8016 11.4107 sec/batch
Epoch 4/10  Iteration 703/1780 Training loss: 1.8012 11.6764 sec/batch
Epoch 4/10  Iteration 704/1780 Training loss: 1.8008 11.4227 sec/batch
Epoch 4/10  Iteration 705/1780 Training loss: 1.8004 11.0647 sec/batch
Epoch 4/10  Iteration 706/1780 Training loss: 1.7999 11.2215 sec/batch
Epoch 4/10  Iteration 707/1780 Training loss: 1.7997 10.7108 sec/batch
Epoch 4/10  Iteration 708/1780 Training loss: 1.7994 10.7691 sec/batch
Epoch 4/10  Iteration 709/1780 Training loss: 1.7992 10.7415 sec/batch
Epoch 4/10  Iteration 710/1780 Training loss: 1.7988 10.5147 sec/batch
Epoch 4/10  Iteration 711/1780 Training loss: 1.7983 10.7046 sec/batch
Epoch 4/10  Iteration 712/1780 Training loss: 1.7980 10.9458 sec/batch
Epoch 5/10  Iteration 713/1780 Training loss: 1.8304 10.7623 sec/batch
Epoch 5/10  Iteration 714/1780 Training loss: 1.7805 10.7120 sec/batch
Epoch 5/10  Iteration 715/1780 Training loss: 1.7649 10.9362 sec/batch
Epoch 5/10  Iteration 716/1780 Training loss: 1.7556 10.5251 sec/batch
Epoch 5/10  Iteration 717/1780 Training loss: 1.7502 10.7362 sec/batch
Epoch 5/10  Iteration 718/1780 Training loss: 1.7392 10.8890 sec/batch
Epoch 5/10  Iteration 719/1780 Training loss: 1.7393 10.6159 sec/batch
Epoch 5/10  Iteration 720/1780 Training loss: 1.7350 10.9808 sec/batch
Epoch 5/10  Iteration 721/1780 Training loss: 1.7368 10.7219 sec/batch
Epoch 5/10  Iteration 722/1780 Training loss: 1.7362 10.6936 sec/batch
Epoch 5/10  Iteration 723/1780 Training loss: 1.7321 10.8085 sec/batch
Epoch 5/10  Iteration 724/1780 Training loss: 1.7293 10.8859 sec/batch
Epoch 5/10  Iteration 725/1780 Training loss: 1.7291 10.5144 sec/batch
Epoch 5/10  Iteration 726/1780 Training loss: 1.7310 11.0345 sec/batch
Epoch 5/10  Iteration 727/1780 Training loss: 1.7294 10.8072 sec/batch
Epoch 5/10  Iteration 728/1780 Training loss: 1.7284 10.7312 sec/batch
Epoch 5/10  Iteration 729/1780 Training loss: 1.7281 11.0405 sec/batch
Epoch 5/10  Iteration 730/1780 Training loss: 1.7293 10.8826 sec/batch
Epoch 5/10  Iteration 731/1780 Training loss: 1.7289 10.6818 sec/batch
Epoch 5/10  Iteration 732/1780 Training loss: 1.7289 10.7947 sec/batch
Epoch 5/10  Iteration 733/1780 Training loss: 1.7276 10.9193 sec/batch
Epoch 5/10  Iteration 734/1780 Training loss: 1.7285 10.6309 sec/batch
Epoch 5/10  Iteration 735/1780 Training loss: 1.7269 11.5939 sec/batch
Epoch 5/10  Iteration 736/1780 Training loss: 1.7265 16.6852 sec/batch
Epoch 5/10  Iteration 737/1780 Training loss: 1.7264 24.1174 sec/batch
Epoch 5/10  Iteration 738/1780 Training loss: 1.7246 24.3230 sec/batch
Epoch 5/10  Iteration 739/1780 Training loss: 1.7229 19.1177 sec/batch
Epoch 5/10  Iteration 740/1780 Training loss: 1.7229 13.2794 sec/batch
Epoch 5/10  Iteration 741/1780 Training loss: 1.7235 12.0681 sec/batch
Epoch 5/10  Iteration 742/1780 Training loss: 1.7234 12.8000 sec/batch
Epoch 5/10  Iteration 743/1780 Training loss: 1.7230 3603.9318 sec/batch
Epoch 5/10  Iteration 744/1780 Training loss: 1.7217 7.8985 sec/batch
Epoch 5/10  Iteration 745/1780 Training loss: 1.7217 11.0100 sec/batch
Epoch 5/10  Iteration 746/1780 Training loss: 1.7220 12.1220 sec/batch
Epoch 5/10  Iteration 747/1780 Training loss: 1.7214 11.3451 sec/batch
Epoch 5/10  Iteration 748/1780 Training loss: 1.7208 11.9841 sec/batch
Epoch 5/10  Iteration 749/1780 Training loss: 1.7197 12.1852 sec/batch
Epoch 5/10  Iteration 750/1780 Training loss: 1.7182 11.9295 sec/batch
Epoch 5/10  Iteration 751/1780 Training loss: 1.7164 11.7558 sec/batch
Epoch 5/10  Iteration 752/1780 Training loss: 1.7153 13.3400 sec/batch
Epoch 5/10  Iteration 753/1780 Training loss: 1.7144 11.8335 sec/batch
Epoch 5/10  Iteration 754/1780 Training loss: 1.7144 11.4036 sec/batch
Epoch 5/10  Iteration 755/1780 Training loss: 1.7136 11.9355 sec/batch
Epoch 5/10  Iteration 756/1780 Training loss: 1.7124 11.7671 sec/batch
Epoch 5/10  Iteration 757/1780 Training loss: 1.7124 12.0191 sec/batch
Epoch 5/10  Iteration 758/1780 Training loss: 1.7113 11.6611 sec/batch
Epoch 5/10  Iteration 759/1780 Training loss: 1.7107 11.8699 sec/batch
Epoch 5/10  Iteration 760/1780 Training loss: 1.7099 11.6894 sec/batch
Epoch 5/10  Iteration 761/1780 Training loss: 1.7093 11.5544 sec/batch
Epoch 5/10  Iteration 762/1780 Training loss: 1.7097 11.6042 sec/batch
Epoch 5/10  Iteration 763/1780 Training loss: 1.7088 11.6564 sec/batch
Epoch 5/10  Iteration 764/1780 Training loss: 1.7095 11.7724 sec/batch
Epoch 5/10  Iteration 765/1780 Training loss: 1.7091 11.8277 sec/batch
Epoch 5/10  Iteration 766/1780 Training loss: 1.7087 11.8448 sec/batch
Epoch 5/10  Iteration 767/1780 Training loss: 1.7080 11.5933 sec/batch
Epoch 5/10  Iteration 768/1780 Training loss: 1.7080 11.6312 sec/batch
Epoch 5/10  Iteration 769/1780 Training loss: 1.7080 11.8905 sec/batch
Epoch 5/10  Iteration 770/1780 Training loss: 1.7075 11.9921 sec/batch
Epoch 5/10  Iteration 771/1780 Training loss: 1.7068 12.8327 sec/batch
Epoch 5/10  Iteration 772/1780 Training loss: 1.7069 11.7750 sec/batch
Epoch 5/10  Iteration 773/1780 Training loss: 1.7065 11.4515 sec/batch
Epoch 5/10  Iteration 774/1780 Training loss: 1.7071 12.1688 sec/batch
Epoch 5/10  Iteration 775/1780 Training loss: 1.7073 11.6041 sec/batch
Epoch 5/10  Iteration 776/1780 Training loss: 1.7072 11.5308 sec/batch
Epoch 5/10  Iteration 777/1780 Training loss: 1.7068 12.0617 sec/batch
Epoch 5/10  Iteration 778/1780 Training loss: 1.7067 11.7516 sec/batch
Epoch 5/10  Iteration 779/1780 Training loss: 1.7066 11.8731 sec/batch
Epoch 5/10  Iteration 780/1780 Training loss: 1.7060 13.7302 sec/batch
Epoch 5/10  Iteration 781/1780 Training loss: 1.7056 11.5377 sec/batch
Epoch 5/10  Iteration 782/1780 Training loss: 1.7052 11.8193 sec/batch
Epoch 5/10  Iteration 783/1780 Training loss: 1.7056 11.6119 sec/batch
Epoch 5/10  Iteration 784/1780 Training loss: 1.7054 12.1739 sec/batch
Epoch 5/10  Iteration 785/1780 Training loss: 1.7056 11.8479 sec/batch
Epoch 5/10  Iteration 786/1780 Training loss: 1.7050 11.6355 sec/batch
Epoch 5/10  Iteration 787/1780 Training loss: 1.7047 12.1075 sec/batch
Epoch 5/10  Iteration 788/1780 Training loss: 1.7047 11.6304 sec/batch
Epoch 5/10  Iteration 789/1780 Training loss: 1.7043 11.8211 sec/batch
Epoch 5/10  Iteration 790/1780 Training loss: 1.7041 11.9996 sec/batch
Epoch 5/10  Iteration 791/1780 Training loss: 1.7033 11.3758 sec/batch
Epoch 5/10  Iteration 792/1780 Training loss: 1.7028 11.9084 sec/batch
Epoch 5/10  Iteration 793/1780 Training loss: 1.7021 11.9538 sec/batch
Epoch 5/10  Iteration 794/1780 Training loss: 1.7017 11.6419 sec/batch
Epoch 5/10  Iteration 795/1780 Training loss: 1.7010 12.5458 sec/batch
Epoch 5/10  Iteration 796/1780 Training loss: 1.7007 11.8944 sec/batch
Epoch 5/10  Iteration 797/1780 Training loss: 1.7001 11.5915 sec/batch
Epoch 5/10  Iteration 798/1780 Training loss: 1.6995 12.1054 sec/batch
Epoch 5/10  Iteration 799/1780 Training loss: 1.6990 11.8520 sec/batch
Epoch 5/10  Iteration 800/1780 Training loss: 1.6985 11.7755 sec/batch
Validation loss: 1.56039 Saving checkpoint!
Epoch 5/10  Iteration 801/1780 Training loss: 1.6986 11.2504 sec/batch
Epoch 5/10  Iteration 802/1780 Training loss: 1.6985 10.8727 sec/batch
Epoch 5/10  Iteration 803/1780 Training loss: 1.6979 11.0837 sec/batch
Epoch 5/10  Iteration 804/1780 Training loss: 1.6974 11.0422 sec/batch
Epoch 5/10  Iteration 805/1780 Training loss: 1.6967 12.3638 sec/batch
Epoch 5/10  Iteration 806/1780 Training loss: 1.6963 11.5037 sec/batch
Epoch 5/10  Iteration 807/1780 Training loss: 1.6957 10.9650 sec/batch
Epoch 5/10  Iteration 808/1780 Training loss: 1.6953 10.8471 sec/batch
Epoch 5/10  Iteration 809/1780 Training loss: 1.6950 11.0233 sec/batch
Epoch 5/10  Iteration 810/1780 Training loss: 1.6943 11.0962 sec/batch
Epoch 5/10  Iteration 811/1780 Training loss: 1.6937 10.7031 sec/batch
Epoch 5/10  Iteration 812/1780 Training loss: 1.6929 10.8424 sec/batch
Epoch 5/10  Iteration 813/1780 Training loss: 1.6926 10.9499 sec/batch
Epoch 5/10  Iteration 814/1780 Training loss: 1.6921 10.7662 sec/batch
Epoch 5/10  Iteration 815/1780 Training loss: 1.6917 10.8959 sec/batch
Epoch 5/10  Iteration 816/1780 Training loss: 1.6912 10.9870 sec/batch
Epoch 5/10  Iteration 817/1780 Training loss: 1.6908 10.6980 sec/batch
Epoch 5/10  Iteration 818/1780 Training loss: 1.6905 10.8886 sec/batch
Epoch 5/10  Iteration 819/1780 Training loss: 1.6902 11.0755 sec/batch
Epoch 5/10  Iteration 820/1780 Training loss: 1.6899 10.8709 sec/batch
Epoch 5/10  Iteration 821/1780 Training loss: 1.6897 10.5629 sec/batch
Epoch 5/10  Iteration 822/1780 Training loss: 1.6894 10.8738 sec/batch
Epoch 5/10  Iteration 823/1780 Training loss: 1.6890 11.0326 sec/batch
Epoch 5/10  Iteration 824/1780 Training loss: 1.6886 10.6423 sec/batch
Epoch 5/10  Iteration 825/1780 Training loss: 1.6882 12.4909 sec/batch
Epoch 5/10  Iteration 826/1780 Training loss: 1.6878 10.4424 sec/batch
Epoch 5/10  Iteration 827/1780 Training loss: 1.6873 10.4053 sec/batch
Epoch 5/10  Iteration 828/1780 Training loss: 1.6867 10.8254 sec/batch
Epoch 5/10  Iteration 829/1780 Training loss: 1.6864 13.4818 sec/batch
Epoch 5/10  Iteration 830/1780 Training loss: 1.6860 3549.3605 sec/batch
Epoch 5/10  Iteration 831/1780 Training loss: 1.6857 8.0623 sec/batch
Epoch 5/10  Iteration 832/1780 Training loss: 1.6853 10.7216 sec/batch
Epoch 5/10  Iteration 833/1780 Training loss: 1.6851 12.6685 sec/batch
Epoch 5/10  Iteration 834/1780 Training loss: 1.6845 11.7165 sec/batch
Epoch 5/10  Iteration 835/1780 Training loss: 1.6839 11.4050 sec/batch
Epoch 5/10  Iteration 836/1780 Training loss: 1.6837 12.3988 sec/batch
Epoch 5/10  Iteration 837/1780 Training loss: 1.6833 11.8475 sec/batch
Epoch 5/10  Iteration 838/1780 Training loss: 1.6827 11.9454 sec/batch
Epoch 5/10  Iteration 839/1780 Training loss: 1.6824 13.9259 sec/batch
Epoch 5/10  Iteration 840/1780 Training loss: 1.6822 16.9345 sec/batch
Epoch 5/10  Iteration 841/1780 Training loss: 1.6819 19.0434 sec/batch
Epoch 5/10  Iteration 842/1780 Training loss: 1.6815 20.5340 sec/batch
Epoch 5/10  Iteration 843/1780 Training loss: 1.6809 15.2776 sec/batch
Epoch 5/10  Iteration 844/1780 Training loss: 1.6804 17.0230 sec/batch
Epoch 5/10  Iteration 845/1780 Training loss: 1.6802 14.2877 sec/batch
Epoch 5/10  Iteration 846/1780 Training loss: 1.6800 14.9633 sec/batch
Epoch 5/10  Iteration 847/1780 Training loss: 1.6797 13.0549 sec/batch
Epoch 5/10  Iteration 848/1780 Training loss: 1.6794 12.5774 sec/batch
Epoch 5/10  Iteration 849/1780 Training loss: 1.6794 12.5643 sec/batch
Epoch 5/10  Iteration 850/1780 Training loss: 1.6792 12.3430 sec/batch
Epoch 5/10  Iteration 851/1780 Training loss: 1.6791 12.2925 sec/batch
Epoch 5/10  Iteration 852/1780 Training loss: 1.6788 12.3786 sec/batch
Epoch 5/10  Iteration 853/1780 Training loss: 1.6789 11.9307 sec/batch
Epoch 5/10  Iteration 854/1780 Training loss: 1.6786 12.2215 sec/batch
Epoch 5/10  Iteration 855/1780 Training loss: 1.6784 12.1867 sec/batch
Epoch 5/10  Iteration 856/1780 Training loss: 1.6782 11.8010 sec/batch
Epoch 5/10  Iteration 857/1780 Training loss: 1.6780 12.7318 sec/batch
Epoch 5/10  Iteration 858/1780 Training loss: 1.6779 11.8499 sec/batch
Epoch 5/10  Iteration 859/1780 Training loss: 1.6777 12.6804 sec/batch
Epoch 5/10  Iteration 860/1780 Training loss: 1.6778 12.2300 sec/batch
Epoch 5/10  Iteration 861/1780 Training loss: 1.6776 11.3909 sec/batch
Epoch 5/10  Iteration 862/1780 Training loss: 1.6772 11.5557 sec/batch
Epoch 5/10  Iteration 863/1780 Training loss: 1.6768 11.5334 sec/batch
Epoch 5/10  Iteration 864/1780 Training loss: 1.6766 11.7100 sec/batch
Epoch 5/10  Iteration 865/1780 Training loss: 1.6765 11.7718 sec/batch
Epoch 5/10  Iteration 866/1780 Training loss: 1.6763 11.7316 sec/batch
Epoch 5/10  Iteration 867/1780 Training loss: 1.6761 11.9192 sec/batch
Epoch 5/10  Iteration 868/1780 Training loss: 1.6758 11.6676 sec/batch
Epoch 5/10  Iteration 869/1780 Training loss: 1.6757 11.8121 sec/batch
Epoch 5/10  Iteration 870/1780 Training loss: 1.6755 11.7703 sec/batch
Epoch 5/10  Iteration 871/1780 Training loss: 1.6749 11.9186 sec/batch
Epoch 5/10  Iteration 872/1780 Training loss: 1.6748 11.7925 sec/batch
Epoch 5/10  Iteration 873/1780 Training loss: 1.6747 11.9674 sec/batch
Epoch 5/10  Iteration 874/1780 Training loss: 1.6745 11.5654 sec/batch
Epoch 5/10  Iteration 875/1780 Training loss: 1.6742 12.0844 sec/batch
Epoch 5/10  Iteration 876/1780 Training loss: 1.6740 11.7260 sec/batch
Epoch 5/10  Iteration 877/1780 Training loss: 1.6737 11.8436 sec/batch
Epoch 5/10  Iteration 878/1780 Training loss: 1.6734 11.3630 sec/batch
Epoch 5/10  Iteration 879/1780 Training loss: 1.6733 12.0492 sec/batch
Epoch 5/10  Iteration 880/1780 Training loss: 1.6734 11.3843 sec/batch
Epoch 5/10  Iteration 881/1780 Training loss: 1.6731 11.8796 sec/batch
Epoch 5/10  Iteration 882/1780 Training loss: 1.6729 11.4190 sec/batch
Epoch 5/10  Iteration 883/1780 Training loss: 1.6726 12.0271 sec/batch
Epoch 5/10  Iteration 884/1780 Training loss: 1.6722 11.2983 sec/batch
Epoch 5/10  Iteration 885/1780 Training loss: 1.6721 12.1332 sec/batch
Epoch 5/10  Iteration 886/1780 Training loss: 1.6719 11.6208 sec/batch
Epoch 5/10  Iteration 887/1780 Training loss: 1.6717 12.0389 sec/batch
Epoch 5/10  Iteration 888/1780 Training loss: 1.6715 11.5161 sec/batch
Epoch 5/10  Iteration 889/1780 Training loss: 1.6711 12.1100 sec/batch
Epoch 5/10  Iteration 890/1780 Training loss: 1.6710 11.5090 sec/batch
Epoch 6/10  Iteration 891/1780 Training loss: 1.7203 11.8907 sec/batch
Epoch 6/10  Iteration 892/1780 Training loss: 1.6712 11.4724 sec/batch
Epoch 6/10  Iteration 893/1780 Training loss: 1.6553 11.4195 sec/batch
Epoch 6/10  Iteration 894/1780 Training loss: 1.6507 11.1478 sec/batch
Epoch 6/10  Iteration 895/1780 Training loss: 1.6448 10.9504 sec/batch
Epoch 6/10  Iteration 896/1780 Training loss: 1.6340 11.2430 sec/batch
Epoch 6/10  Iteration 897/1780 Training loss: 1.6334 10.6956 sec/batch
Epoch 6/10  Iteration 898/1780 Training loss: 1.6312 10.6072 sec/batch
Epoch 6/10  Iteration 899/1780 Training loss: 1.6320 10.8900 sec/batch
Epoch 6/10  Iteration 900/1780 Training loss: 1.6306 10.5274 sec/batch
Validation loss: 1.50366 Saving checkpoint!
Epoch 6/10  Iteration 901/1780 Training loss: 1.6324 10.4924 sec/batch
Epoch 6/10  Iteration 902/1780 Training loss: 1.6300 10.4500 sec/batch
Epoch 6/10  Iteration 903/1780 Training loss: 1.6295 10.8712 sec/batch
Epoch 6/10  Iteration 904/1780 Training loss: 1.6309 10.4504 sec/batch
Epoch 6/10  Iteration 905/1780 Training loss: 1.6291 10.4917 sec/batch
Epoch 6/10  Iteration 906/1780 Training loss: 1.6269 10.8416 sec/batch
Epoch 6/10  Iteration 907/1780 Training loss: 1.6266 10.6064 sec/batch
Epoch 6/10  Iteration 908/1780 Training loss: 1.6275 10.4523 sec/batch
Epoch 6/10  Iteration 909/1780 Training loss: 1.6275 10.6156 sec/batch
Epoch 6/10  Iteration 910/1780 Training loss: 1.6280 10.7135 sec/batch
Epoch 6/10  Iteration 911/1780 Training loss: 1.6272 10.4784 sec/batch
Epoch 6/10  Iteration 912/1780 Training loss: 1.6281 10.8300 sec/batch
Epoch 6/10  Iteration 913/1780 Training loss: 1.6268 10.6185 sec/batch
Epoch 6/10  Iteration 914/1780 Training loss: 1.6264 10.4014 sec/batch
Epoch 6/10  Iteration 915/1780 Training loss: 1.6260 10.7119 sec/batch
Epoch 6/10  Iteration 916/1780 Training loss: 1.6246 10.6404 sec/batch
Epoch 6/10  Iteration 917/1780 Training loss: 1.6231 10.4713 sec/batch
Epoch 6/10  Iteration 918/1780 Training loss: 1.6235 10.7576 sec/batch
Epoch 6/10  Iteration 919/1780 Training loss: 1.6238 10.7721 sec/batch
Epoch 6/10  Iteration 920/1780 Training loss: 1.6239 12.1575 sec/batch
Epoch 6/10  Iteration 921/1780 Training loss: 1.6235 15.9032 sec/batch
Epoch 6/10  Iteration 922/1780 Training loss: 1.6224 23.1327 sec/batch
Epoch 6/10  Iteration 923/1780 Training loss: 1.6222 24.6008 sec/batch
Epoch 6/10  Iteration 924/1780 Training loss: 1.6223 3604.9767 sec/batch
Epoch 6/10  Iteration 925/1780 Training loss: 1.6219 7.5043 sec/batch
Epoch 6/10  Iteration 926/1780 Training loss: 1.6214 10.0433 sec/batch
Epoch 6/10  Iteration 927/1780 Training loss: 1.6202 10.5861 sec/batch
Epoch 6/10  Iteration 928/1780 Training loss: 1.6188 10.6252 sec/batch
Epoch 6/10  Iteration 929/1780 Training loss: 1.6169 10.7511 sec/batch
Epoch 6/10  Iteration 930/1780 Training loss: 1.6159 11.6172 sec/batch
Epoch 6/10  Iteration 931/1780 Training loss: 1.6152 10.6295 sec/batch
Epoch 6/10  Iteration 932/1780 Training loss: 1.6154 11.2664 sec/batch
Epoch 6/10  Iteration 933/1780 Training loss: 1.6146 11.1795 sec/batch
Epoch 6/10  Iteration 934/1780 Training loss: 1.6134 11.7277 sec/batch
Epoch 6/10  Iteration 935/1780 Training loss: 1.6134 10.6048 sec/batch
Epoch 6/10  Iteration 936/1780 Training loss: 1.6122 10.9357 sec/batch
Epoch 6/10  Iteration 937/1780 Training loss: 1.6116 11.1372 sec/batch
Epoch 6/10  Iteration 938/1780 Training loss: 1.6109 11.1298 sec/batch
Epoch 6/10  Iteration 939/1780 Training loss: 1.6105 10.6950 sec/batch
Epoch 6/10  Iteration 940/1780 Training loss: 1.6108 11.1310 sec/batch
Epoch 6/10  Iteration 941/1780 Training loss: 1.6100 11.2047 sec/batch
Epoch 6/10  Iteration 942/1780 Training loss: 1.6106 10.9882 sec/batch
Epoch 6/10  Iteration 943/1780 Training loss: 1.6102 11.0874 sec/batch
Epoch 6/10  Iteration 944/1780 Training loss: 1.6099 10.4068 sec/batch
Epoch 6/10  Iteration 945/1780 Training loss: 1.6093 10.6821 sec/batch
Epoch 6/10  Iteration 946/1780 Training loss: 1.6092 10.8164 sec/batch
Epoch 6/10  Iteration 947/1780 Training loss: 1.6095 10.9041 sec/batch
Epoch 6/10  Iteration 948/1780 Training loss: 1.6090 10.9992 sec/batch
Epoch 6/10  Iteration 949/1780 Training loss: 1.6085 11.0280 sec/batch
Epoch 6/10  Iteration 950/1780 Training loss: 1.6087 10.9235 sec/batch
Epoch 6/10  Iteration 951/1780 Training loss: 1.6084 10.7852 sec/batch
Epoch 6/10  Iteration 952/1780 Training loss: 1.6091 10.4077 sec/batch
Epoch 6/10  Iteration 953/1780 Training loss: 1.6092 10.7579 sec/batch
Epoch 6/10  Iteration 954/1780 Training loss: 1.6092 10.8515 sec/batch
Epoch 6/10  Iteration 955/1780 Training loss: 1.6089 10.6449 sec/batch
Epoch 6/10  Iteration 956/1780 Training loss: 1.6088 11.0287 sec/batch
Epoch 6/10  Iteration 957/1780 Training loss: 1.6088 10.9038 sec/batch
Epoch 6/10  Iteration 958/1780 Training loss: 1.6082 10.9222 sec/batch
Epoch 6/10  Iteration 959/1780 Training loss: 1.6080 10.9108 sec/batch
Epoch 6/10  Iteration 960/1780 Training loss: 1.6077 10.6497 sec/batch
Epoch 6/10  Iteration 961/1780 Training loss: 1.6081 10.6672 sec/batch
Epoch 6/10  Iteration 962/1780 Training loss: 1.6080 11.0506 sec/batch
Epoch 6/10  Iteration 963/1780 Training loss: 1.6083 10.7042 sec/batch
Epoch 6/10  Iteration 964/1780 Training loss: 1.6079 10.9341 sec/batch
Epoch 6/10  Iteration 965/1780 Training loss: 1.6076 10.8923 sec/batch
Epoch 6/10  Iteration 966/1780 Training loss: 1.6075 10.6008 sec/batch
Epoch 6/10  Iteration 967/1780 Training loss: 1.6071 10.7243 sec/batch
Epoch 6/10  Iteration 968/1780 Training loss: 1.6069 10.9488 sec/batch
Epoch 6/10  Iteration 969/1780 Training loss: 1.6062 10.8411 sec/batch
Epoch 6/10  Iteration 970/1780 Training loss: 1.6059 10.7407 sec/batch
Epoch 6/10  Iteration 971/1780 Training loss: 1.6052 12.5340 sec/batch
Epoch 6/10  Iteration 972/1780 Training loss: 1.6050 13.1665 sec/batch
Epoch 6/10  Iteration 973/1780 Training loss: 1.6042 12.7823 sec/batch
Epoch 6/10  Iteration 974/1780 Training loss: 1.6040 12.6080 sec/batch
Epoch 6/10  Iteration 975/1780 Training loss: 1.6036 10.8584 sec/batch
Epoch 6/10  Iteration 976/1780 Training loss: 1.6031 10.9617 sec/batch
Epoch 6/10  Iteration 977/1780 Training loss: 1.6027 11.0557 sec/batch
Epoch 6/10  Iteration 978/1780 Training loss: 1.6023 11.1831 sec/batch
Epoch 6/10  Iteration 979/1780 Training loss: 1.6017 10.7109 sec/batch
Epoch 6/10  Iteration 980/1780 Training loss: 1.6017 10.9222 sec/batch
Epoch 6/10  Iteration 981/1780 Training loss: 1.6011 11.2099 sec/batch
Epoch 6/10  Iteration 982/1780 Training loss: 1.6008 11.0933 sec/batch
Epoch 6/10  Iteration 983/1780 Training loss: 1.6003 11.2683 sec/batch
Epoch 6/10  Iteration 984/1780 Training loss: 1.5999 11.1765 sec/batch
Epoch 6/10  Iteration 985/1780 Training loss: 1.5995 11.3375 sec/batch
Epoch 6/10  Iteration 986/1780 Training loss: 1.5994 11.2670 sec/batch
Epoch 6/10  Iteration 987/1780 Training loss: 1.5991 10.9294 sec/batch
Epoch 6/10  Iteration 988/1780 Training loss: 1.5985 11.1437 sec/batch
Epoch 6/10  Iteration 989/1780 Training loss: 1.5980 10.9093 sec/batch
Epoch 6/10  Iteration 990/1780 Training loss: 1.5972 11.3553 sec/batch
Epoch 6/10  Iteration 991/1780 Training loss: 1.5971 10.5515 sec/batch
Epoch 6/10  Iteration 992/1780 Training loss: 1.5967 10.4174 sec/batch
Epoch 6/10  Iteration 993/1780 Training loss: 1.5963 10.2493 sec/batch
Epoch 6/10  Iteration 994/1780 Training loss: 1.5960 10.3642 sec/batch
Epoch 6/10  Iteration 995/1780 Training loss: 1.5957 10.3438 sec/batch
Epoch 6/10  Iteration 996/1780 Training loss: 1.5953 1751.2504 sec/batch
Epoch 6/10  Iteration 997/1780 Training loss: 1.5951 17.1582 sec/batch
Epoch 6/10  Iteration 998/1780 Training loss: 1.5949 8.9056 sec/batch
Epoch 6/10  Iteration 999/1780 Training loss: 1.5947 11.7009 sec/batch
Epoch 6/10  Iteration 1000/1780 Training loss: 1.5945 7.8676 sec/batch
Validation loss: 1.45939 Saving checkpoint!
Epoch 6/10  Iteration 1001/1780 Training loss: 1.5949 7.5726 sec/batch
Epoch 6/10  Iteration 1002/1780 Training loss: 1.5946 7.3073 sec/batch
Epoch 6/10  Iteration 1003/1780 Training loss: 1.5942 7.4692 sec/batch
Epoch 6/10  Iteration 1004/1780 Training loss: 1.5938 7.6687 sec/batch
Epoch 6/10  Iteration 1005/1780 Training loss: 1.5934 7.2520 sec/batch
Epoch 6/10  Iteration 1006/1780 Training loss: 1.5928 7.3179 sec/batch
Epoch 6/10  Iteration 1007/1780 Training loss: 1.5926 7.4682 sec/batch
Epoch 6/10  Iteration 1008/1780 Training loss: 1.5924 7.2330 sec/batch
Epoch 6/10  Iteration 1009/1780 Training loss: 1.5921 7.5494 sec/batch
Epoch 6/10  Iteration 1010/1780 Training loss: 1.5917 7.4135 sec/batch
Epoch 6/10  Iteration 1011/1780 Training loss: 1.5915 7.3745 sec/batch
Epoch 6/10  Iteration 1012/1780 Training loss: 1.5909 7.5078 sec/batch
Epoch 6/10  Iteration 1013/1780 Training loss: 1.5904 7.3872 sec/batch
Epoch 6/10  Iteration 1014/1780 Training loss: 1.5902 7.3995 sec/batch
Epoch 6/10  Iteration 1015/1780 Training loss: 1.5900 7.7093 sec/batch
Epoch 6/10  Iteration 1016/1780 Training loss: 1.5893 7.3311 sec/batch
Epoch 6/10  Iteration 1017/1780 Training loss: 1.5893 7.6276 sec/batch
Epoch 6/10  Iteration 1018/1780 Training loss: 1.5892 7.3737 sec/batch
Epoch 6/10  Iteration 1019/1780 Training loss: 1.5889 7.3366 sec/batch
Epoch 6/10  Iteration 1020/1780 Training loss: 1.5885 7.4886 sec/batch
Epoch 6/10  Iteration 1021/1780 Training loss: 1.5880 7.4054 sec/batch
Epoch 6/10  Iteration 1022/1780 Training loss: 1.5875 7.3921 sec/batch
Epoch 6/10  Iteration 1023/1780 Training loss: 1.5875 7.4747 sec/batch
Epoch 6/10  Iteration 1024/1780 Training loss: 1.5873 7.4253 sec/batch
Epoch 6/10  Iteration 1025/1780 Training loss: 1.5871 7.3669 sec/batch
Epoch 6/10  Iteration 1026/1780 Training loss: 1.5869 7.3412 sec/batch
Epoch 6/10  Iteration 1027/1780 Training loss: 1.5868 7.3163 sec/batch
Epoch 6/10  Iteration 1028/1780 Training loss: 1.5867 7.4935 sec/batch
Epoch 6/10  Iteration 1029/1780 Training loss: 1.5866 7.3866 sec/batch
Epoch 6/10  Iteration 1030/1780 Training loss: 1.5863 12.1084 sec/batch
Epoch 6/10  Iteration 1031/1780 Training loss: 1.5865 12.0606 sec/batch
Epoch 6/10  Iteration 1032/1780 Training loss: 1.5863 11.5537 sec/batch
Epoch 6/10  Iteration 1033/1780 Training loss: 1.5861 11.0219 sec/batch
Epoch 6/10  Iteration 1034/1780 Training loss: 1.5860 10.3215 sec/batch
Epoch 6/10  Iteration 1035/1780 Training loss: 1.5858 9.6670 sec/batch
Epoch 6/10  Iteration 1036/1780 Training loss: 1.5857 9.5645 sec/batch
Epoch 6/10  Iteration 1037/1780 Training loss: 1.5855 9.6164 sec/batch
Epoch 6/10  Iteration 1038/1780 Training loss: 1.5856 9.4632 sec/batch
Epoch 6/10  Iteration 1039/1780 Training loss: 1.5855 9.7215 sec/batch
Epoch 6/10  Iteration 1040/1780 Training loss: 1.5852 9.6600 sec/batch
Epoch 6/10  Iteration 1041/1780 Training loss: 1.5847 10.0798 sec/batch
Epoch 6/10  Iteration 1042/1780 Training loss: 1.5845 10.1544 sec/batch
Epoch 6/10  Iteration 1043/1780 Training loss: 1.5845 10.1245 sec/batch
Epoch 6/10  Iteration 1044/1780 Training loss: 1.5842 9.6997 sec/batch
Epoch 6/10  Iteration 1045/1780 Training loss: 1.5840 9.8053 sec/batch
Epoch 6/10  Iteration 1046/1780 Training loss: 1.5838 9.8515 sec/batch
Epoch 6/10  Iteration 1047/1780 Training loss: 1.5837 9.6193 sec/batch
Epoch 6/10  Iteration 1048/1780 Training loss: 1.5834 9.6259 sec/batch
Epoch 6/10  Iteration 1049/1780 Training loss: 1.5830 9.5401 sec/batch
Epoch 6/10  Iteration 1050/1780 Training loss: 1.5830 10.4069 sec/batch
Epoch 6/10  Iteration 1051/1780 Training loss: 1.5830 10.9631 sec/batch
Epoch 6/10  Iteration 1052/1780 Training loss: 1.5827 10.4082 sec/batch
Epoch 6/10  Iteration 1053/1780 Training loss: 1.5826 9.9553 sec/batch
Epoch 6/10  Iteration 1054/1780 Training loss: 1.5825 9.7852 sec/batch
Epoch 6/10  Iteration 1055/1780 Training loss: 1.5822 10.0814 sec/batch
Epoch 6/10  Iteration 1056/1780 Training loss: 1.5820 10.5060 sec/batch
Epoch 6/10  Iteration 1057/1780 Training loss: 1.5820 9.7209 sec/batch
Epoch 6/10  Iteration 1058/1780 Training loss: 1.5823 11.0091 sec/batch
Epoch 6/10  Iteration 1059/1780 Training loss: 1.5821 10.6933 sec/batch
Epoch 6/10  Iteration 1060/1780 Training loss: 1.5819 10.8528 sec/batch
Epoch 6/10  Iteration 1061/1780 Training loss: 1.5816 11.0116 sec/batch
Epoch 6/10  Iteration 1062/1780 Training loss: 1.5813 10.0224 sec/batch
Epoch 6/10  Iteration 1063/1780 Training loss: 1.5812 9.9222 sec/batch
Epoch 6/10  Iteration 1064/1780 Training loss: 1.5810 9.5997 sec/batch
Epoch 6/10  Iteration 1065/1780 Training loss: 1.5809 9.7222 sec/batch
Epoch 6/10  Iteration 1066/1780 Training loss: 1.5807 11.0664 sec/batch
Epoch 6/10  Iteration 1067/1780 Training loss: 1.5804 9.6661 sec/batch
Epoch 6/10  Iteration 1068/1780 Training loss: 1.5803 10.2563 sec/batch
Epoch 7/10  Iteration 1069/1780 Training loss: 1.6386 10.2832 sec/batch
Epoch 7/10  Iteration 1070/1780 Training loss: 1.5998 9.4528 sec/batch
Epoch 7/10  Iteration 1071/1780 Training loss: 1.5845 10.0775 sec/batch
Epoch 7/10  Iteration 1072/1780 Training loss: 1.5769 10.5666 sec/batch
Epoch 7/10  Iteration 1073/1780 Training loss: 1.5691 10.7858 sec/batch
Epoch 7/10  Iteration 1074/1780 Training loss: 1.5565 10.0221 sec/batch
Epoch 7/10  Iteration 1075/1780 Training loss: 1.5558 9.8507 sec/batch
Epoch 7/10  Iteration 1076/1780 Training loss: 1.5521 9.8593 sec/batch
Epoch 7/10  Iteration 1077/1780 Training loss: 1.5517 9.9729 sec/batch
Epoch 7/10  Iteration 1078/1780 Training loss: 1.5501 9.9459 sec/batch
Epoch 7/10  Iteration 1079/1780 Training loss: 1.5478 10.1087 sec/batch
Epoch 7/10  Iteration 1080/1780 Training loss: 1.5469 10.0258 sec/batch
Epoch 7/10  Iteration 1081/1780 Training loss: 1.5485 10.1539 sec/batch
Epoch 7/10  Iteration 1082/1780 Training loss: 1.5518 9.9679 sec/batch
Epoch 7/10  Iteration 1083/1780 Training loss: 1.5518 10.3286 sec/batch
Epoch 7/10  Iteration 1084/1780 Training loss: 1.5506 9.8249 sec/batch
Epoch 7/10  Iteration 1085/1780 Training loss: 1.5514 10.2417 sec/batch
Epoch 7/10  Iteration 1086/1780 Training loss: 1.5529 10.4530 sec/batch
Epoch 7/10  Iteration 1087/1780 Training loss: 1.5535 10.0363 sec/batch
Epoch 7/10  Iteration 1088/1780 Training loss: 1.5547 9.8162 sec/batch
Epoch 7/10  Iteration 1089/1780 Training loss: 1.5544 10.0687 sec/batch
Epoch 7/10  Iteration 1090/1780 Training loss: 1.5557 8.1457 sec/batch
Epoch 7/10  Iteration 1091/1780 Training loss: 1.5549 7.1716 sec/batch
Epoch 7/10  Iteration 1092/1780 Training loss: 1.5548 7.1209 sec/batch
Epoch 7/10  Iteration 1093/1780 Training loss: 1.5548 7.7122 sec/batch
Epoch 7/10  Iteration 1094/1780 Training loss: 1.5531 7.3558 sec/batch
Epoch 7/10  Iteration 1095/1780 Training loss: 1.5516 7.2398 sec/batch
Epoch 7/10  Iteration 1096/1780 Training loss: 1.5522 7.2123 sec/batch
Epoch 7/10  Iteration 1097/1780 Training loss: 1.5526 7.2793 sec/batch
Epoch 7/10  Iteration 1098/1780 Training loss: 1.5530 7.2991 sec/batch
Epoch 7/10  Iteration 1099/1780 Training loss: 1.5525 7.1694 sec/batch
Epoch 7/10  Iteration 1100/1780 Training loss: 1.5516 825.6933 sec/batch
Validation loss: 1.42551 Saving checkpoint!
Epoch 7/10  Iteration 1101/1780 Training loss: 1.5538 19.1653 sec/batch
Epoch 7/10  Iteration 1102/1780 Training loss: 1.5544 14.6627 sec/batch
Epoch 7/10  Iteration 1103/1780 Training loss: 1.5539 12.3441 sec/batch
Epoch 7/10  Iteration 1104/1780 Training loss: 1.5534 16.9064 sec/batch
Epoch 7/10  Iteration 1105/1780 Training loss: 1.5526 11.0817 sec/batch
Epoch 7/10  Iteration 1106/1780 Training loss: 1.5513 10.4945 sec/batch
Epoch 7/10  Iteration 1107/1780 Training loss: 1.5499 10.4128 sec/batch
Epoch 7/10  Iteration 1108/1780 Training loss: 1.5486 10.1793 sec/batch
Epoch 7/10  Iteration 1109/1780 Training loss: 1.5477 9.8301 sec/batch
Epoch 7/10  Iteration 1110/1780 Training loss: 1.5480 9.4132 sec/batch
Epoch 7/10  Iteration 1111/1780 Training loss: 1.5472 9.7466 sec/batch
Epoch 7/10  Iteration 1112/1780 Training loss: 1.5463 9.7908 sec/batch
Epoch 7/10  Iteration 1113/1780 Training loss: 1.5463 10.5338 sec/batch
Epoch 7/10  Iteration 1114/1780 Training loss: 1.5452 9.4649 sec/batch
Epoch 7/10  Iteration 1115/1780 Training loss: 1.5449 9.4156 sec/batch
Epoch 7/10  Iteration 1116/1780 Training loss: 1.5442 10.7474 sec/batch
Epoch 7/10  Iteration 1117/1780 Training loss: 1.5439 9.8759 sec/batch
Epoch 7/10  Iteration 1118/1780 Training loss: 1.5441 9.6866 sec/batch
Epoch 7/10  Iteration 1119/1780 Training loss: 1.5434 10.5576 sec/batch
Epoch 7/10  Iteration 1120/1780 Training loss: 1.5439 9.7548 sec/batch
Epoch 7/10  Iteration 1121/1780 Training loss: 1.5435 11.1633 sec/batch
Epoch 7/10  Iteration 1122/1780 Training loss: 1.5434 8.5266 sec/batch
Epoch 7/10  Iteration 1123/1780 Training loss: 1.5429 8.5100 sec/batch
Epoch 7/10  Iteration 1124/1780 Training loss: 1.5427 8.2843 sec/batch
Epoch 7/10  Iteration 1125/1780 Training loss: 1.5429 8.5950 sec/batch
Epoch 7/10  Iteration 1126/1780 Training loss: 1.5424 8.2998 sec/batch
Epoch 7/10  Iteration 1127/1780 Training loss: 1.5417 8.2685 sec/batch
Epoch 7/10  Iteration 1128/1780 Training loss: 1.5422 8.5789 sec/batch
Epoch 7/10  Iteration 1129/1780 Training loss: 1.5419 8.2603 sec/batch
Epoch 7/10  Iteration 1130/1780 Training loss: 1.5426 8.3385 sec/batch
Epoch 7/10  Iteration 1131/1780 Training loss: 1.5429 8.6528 sec/batch
Epoch 7/10  Iteration 1132/1780 Training loss: 1.5429 8.3092 sec/batch
Epoch 7/10  Iteration 1133/1780 Training loss: 1.5426 8.3437 sec/batch
Epoch 7/10  Iteration 1134/1780 Training loss: 1.5425 8.2693 sec/batch
Epoch 7/10  Iteration 1135/1780 Training loss: 1.5424 8.8699 sec/batch
Epoch 7/10  Iteration 1136/1780 Training loss: 1.5418 8.3694 sec/batch
Epoch 7/10  Iteration 1137/1780 Training loss: 1.5417 8.1551 sec/batch
Epoch 7/10  Iteration 1138/1780 Training loss: 1.5413 8.3609 sec/batch
Epoch 7/10  Iteration 1139/1780 Training loss: 1.5416 8.2014 sec/batch
Epoch 7/10  Iteration 1140/1780 Training loss: 1.5416 8.2433 sec/batch
Epoch 7/10  Iteration 1141/1780 Training loss: 1.5418 8.3856 sec/batch
Epoch 7/10  Iteration 1142/1780 Training loss: 1.5412 8.1853 sec/batch
Epoch 7/10  Iteration 1143/1780 Training loss: 1.5408 8.3631 sec/batch
Epoch 7/10  Iteration 1144/1780 Training loss: 1.5408 8.4164 sec/batch
Epoch 7/10  Iteration 1145/1780 Training loss: 1.5404 8.3788 sec/batch
Epoch 7/10  Iteration 1146/1780 Training loss: 1.5401 8.2536 sec/batch
Epoch 7/10  Iteration 1147/1780 Training loss: 1.5395 8.9991 sec/batch
Epoch 7/10  Iteration 1148/1780 Training loss: 1.5392 9.2282 sec/batch
Epoch 7/10  Iteration 1149/1780 Training loss: 1.5387 8.2441 sec/batch
Epoch 7/10  Iteration 1150/1780 Training loss: 1.5385 9.2812 sec/batch
Epoch 7/10  Iteration 1151/1780 Training loss: 1.5378 9.0968 sec/batch
Epoch 7/10  Iteration 1152/1780 Training loss: 1.5376 8.5507 sec/batch
Epoch 7/10  Iteration 1153/1780 Training loss: 1.5372 8.5706 sec/batch
Epoch 7/10  Iteration 1154/1780 Training loss: 1.5368 9.1839 sec/batch
Epoch 7/10  Iteration 1155/1780 Training loss: 1.5363 8.3082 sec/batch
Epoch 7/10  Iteration 1156/1780 Training loss: 1.5359 8.3843 sec/batch
Epoch 7/10  Iteration 1157/1780 Training loss: 1.5353 13.1444 sec/batch
Epoch 7/10  Iteration 1158/1780 Training loss: 1.5352 12.7418 sec/batch
Epoch 7/10  Iteration 1159/1780 Training loss: 1.5348 12.6897 sec/batch
Epoch 7/10  Iteration 1160/1780 Training loss: 1.5346 18.8016 sec/batch
Epoch 7/10  Iteration 1161/1780 Training loss: 1.5340 12.1895 sec/batch
Epoch 7/10  Iteration 1162/1780 Training loss: 1.5335 11.8602 sec/batch
Epoch 7/10  Iteration 1163/1780 Training loss: 1.5330 12.0961 sec/batch
Epoch 7/10  Iteration 1164/1780 Training loss: 1.5329 13.2543 sec/batch
Epoch 7/10  Iteration 1165/1780 Training loss: 1.5327 11.0925 sec/batch
Epoch 7/10  Iteration 1166/1780 Training loss: 1.5322 10.3726 sec/batch
Epoch 7/10  Iteration 1167/1780 Training loss: 1.5317 9.4406 sec/batch
Epoch 7/10  Iteration 1168/1780 Training loss: 1.5310 8.7504 sec/batch
Epoch 7/10  Iteration 1169/1780 Training loss: 1.5308 11.9387 sec/batch
Epoch 7/10  Iteration 1170/1780 Training loss: 1.5305 14.8974 sec/batch
Epoch 7/10  Iteration 1171/1780 Training loss: 1.5302 12.3002 sec/batch
Epoch 7/10  Iteration 1172/1780 Training loss: 1.5299 11.5762 sec/batch
Epoch 7/10  Iteration 1173/1780 Training loss: 1.5296 11.9898 sec/batch
Epoch 7/10  Iteration 1174/1780 Training loss: 1.5293 13.2038 sec/batch
Epoch 7/10  Iteration 1175/1780 Training loss: 1.5291 12.9043 sec/batch
Epoch 7/10  Iteration 1176/1780 Training loss: 1.5289 10.5551 sec/batch
Epoch 7/10  Iteration 1177/1780 Training loss: 1.5286 8.8826 sec/batch
Epoch 7/10  Iteration 1178/1780 Training loss: 1.5286 8.9132 sec/batch
Epoch 7/10  Iteration 1179/1780 Training loss: 1.5283 8.6730 sec/batch
Epoch 7/10  Iteration 1180/1780 Training loss: 1.5281 8.6832 sec/batch
Epoch 7/10  Iteration 1181/1780 Training loss: 1.5277 8.6382 sec/batch
Epoch 7/10  Iteration 1182/1780 Training loss: 1.5274 8.7175 sec/batch
Epoch 7/10  Iteration 1183/1780 Training loss: 1.5271 8.5199 sec/batch
Epoch 7/10  Iteration 1184/1780 Training loss: 1.5266 8.7795 sec/batch
Epoch 7/10  Iteration 1185/1780 Training loss: 1.5264 8.9189 sec/batch
Epoch 7/10  Iteration 1186/1780 Training loss: 1.5262 16.9803 sec/batch
Epoch 7/10  Iteration 1187/1780 Training loss: 1.5259 9.4420 sec/batch
Epoch 7/10  Iteration 1188/1780 Training loss: 1.5256 10.0512 sec/batch
Epoch 7/10  Iteration 1189/1780 Training loss: 1.5254 9.2592 sec/batch
Epoch 7/10  Iteration 1190/1780 Training loss: 1.5249 9.2659 sec/batch
Epoch 7/10  Iteration 1191/1780 Training loss: 1.5244 9.7535 sec/batch
Epoch 7/10  Iteration 1192/1780 Training loss: 1.5243 9.0349 sec/batch
Epoch 7/10  Iteration 1193/1780 Training loss: 1.5240 9.6638 sec/batch
Epoch 7/10  Iteration 1194/1780 Training loss: 1.5234 9.5994 sec/batch
Epoch 7/10  Iteration 1195/1780 Training loss: 1.5233 17.4348 sec/batch
Epoch 7/10  Iteration 1196/1780 Training loss: 1.5232 12.6481 sec/batch
Epoch 7/10  Iteration 1197/1780 Training loss: 1.5229 15.3301 sec/batch
Epoch 7/10  Iteration 1198/1780 Training loss: 1.5225 16.5513 sec/batch
Epoch 7/10  Iteration 1199/1780 Training loss: 1.5220 14.0465 sec/batch
Epoch 7/10  Iteration 1200/1780 Training loss: 1.5215 10.4085 sec/batch
Validation loss: 1.38599 Saving checkpoint!
Epoch 7/10  Iteration 1201/1780 Training loss: 1.5221 11.6197 sec/batch
Epoch 7/10  Iteration 1202/1780 Training loss: 1.5219 9.7896 sec/batch
Epoch 7/10  Iteration 1203/1780 Training loss: 1.5218 9.2786 sec/batch
Epoch 7/10  Iteration 1204/1780 Training loss: 1.5217 10.6876 sec/batch
Epoch 7/10  Iteration 1205/1780 Training loss: 1.5216 9.1978 sec/batch
Epoch 7/10  Iteration 1206/1780 Training loss: 1.5216 9.0522 sec/batch
Epoch 7/10  Iteration 1207/1780 Training loss: 1.5214 10.6384 sec/batch
Epoch 7/10  Iteration 1208/1780 Training loss: 1.5213 12.0941 sec/batch
Epoch 7/10  Iteration 1209/1780 Training loss: 1.5215 12.2146 sec/batch
Epoch 7/10  Iteration 1210/1780 Training loss: 1.5213 9.1682 sec/batch
Epoch 7/10  Iteration 1211/1780 Training loss: 1.5210 10.0441 sec/batch
Epoch 7/10  Iteration 1212/1780 Training loss: 1.5210 16.3634 sec/batch
Epoch 7/10  Iteration 1213/1780 Training loss: 1.5208 17.5612 sec/batch
Epoch 7/10  Iteration 1214/1780 Training loss: 1.5208 11.2393 sec/batch
Epoch 7/10  Iteration 1215/1780 Training loss: 1.5207 9.5028 sec/batch
Epoch 7/10  Iteration 1216/1780 Training loss: 1.5208 9.1001 sec/batch
Epoch 7/10  Iteration 1217/1780 Training loss: 1.5207 9.1479 sec/batch
Epoch 7/10  Iteration 1218/1780 Training loss: 1.5204 9.1322 sec/batch
Epoch 7/10  Iteration 1219/1780 Training loss: 1.5200 9.9356 sec/batch
Epoch 7/10  Iteration 1220/1780 Training loss: 1.5198 9.3394 sec/batch
Epoch 7/10  Iteration 1221/1780 Training loss: 1.5197 9.2105 sec/batch
Epoch 7/10  Iteration 1222/1780 Training loss: 1.5196 10.0676 sec/batch
Epoch 7/10  Iteration 1223/1780 Training loss: 1.5195 8.7160 sec/batch
Epoch 7/10  Iteration 1224/1780 Training loss: 1.5193 8.8361 sec/batch
Epoch 7/10  Iteration 1225/1780 Training loss: 1.5192 8.8049 sec/batch
Epoch 7/10  Iteration 1226/1780 Training loss: 1.5190 8.8016 sec/batch
Epoch 7/10  Iteration 1227/1780 Training loss: 1.5185 9.2605 sec/batch
Epoch 7/10  Iteration 1228/1780 Training loss: 1.5184 7.5718 sec/batch
Epoch 7/10  Iteration 1229/1780 Training loss: 1.5185 9.3716 sec/batch
Epoch 7/10  Iteration 1230/1780 Training loss: 1.5184 6.7023 sec/batch
Epoch 7/10  Iteration 1231/1780 Training loss: 1.5182 7.0022 sec/batch
Epoch 7/10  Iteration 1232/1780 Training loss: 1.5181 6.7568 sec/batch
Epoch 7/10  Iteration 1233/1780 Training loss: 1.5179 6.6925 sec/batch
Epoch 7/10  Iteration 1234/1780 Training loss: 1.5177 6.7393 sec/batch
Epoch 7/10  Iteration 1235/1780 Training loss: 1.5178 6.9430 sec/batch
Epoch 7/10  Iteration 1236/1780 Training loss: 1.5182 6.7726 sec/batch
Epoch 7/10  Iteration 1237/1780 Training loss: 1.5180 6.7763 sec/batch
Epoch 7/10  Iteration 1238/1780 Training loss: 1.5178 6.7923 sec/batch
Epoch 7/10  Iteration 1239/1780 Training loss: 1.5176 6.7702 sec/batch
Epoch 7/10  Iteration 1240/1780 Training loss: 1.5173 6.7867 sec/batch
Epoch 7/10  Iteration 1241/1780 Training loss: 1.5173 6.9037 sec/batch
Epoch 7/10  Iteration 1242/1780 Training loss: 1.5171 6.8144 sec/batch
Epoch 7/10  Iteration 1243/1780 Training loss: 1.5171 6.8193 sec/batch
Epoch 7/10  Iteration 1244/1780 Training loss: 1.5169 6.8080 sec/batch
Epoch 7/10  Iteration 1245/1780 Training loss: 1.5166 6.8885 sec/batch
Epoch 7/10  Iteration 1246/1780 Training loss: 1.5165 6.7912 sec/batch
Epoch 8/10  Iteration 1247/1780 Training loss: 1.5764 6.7367 sec/batch
Epoch 8/10  Iteration 1248/1780 Training loss: 1.5358 6.7951 sec/batch
Epoch 8/10  Iteration 1249/1780 Training loss: 1.5181 6.7337 sec/batch
Epoch 8/10  Iteration 1250/1780 Training loss: 1.5133 6.8106 sec/batch
Epoch 8/10  Iteration 1251/1780 Training loss: 1.5054 6.7847 sec/batch
Epoch 8/10  Iteration 1252/1780 Training loss: 1.4956 6.9346 sec/batch
Epoch 8/10  Iteration 1253/1780 Training loss: 1.4954 6.7943 sec/batch
Epoch 8/10  Iteration 1254/1780 Training loss: 1.4928 6.8102 sec/batch
Epoch 8/10  Iteration 1255/1780 Training loss: 1.4928 6.7121 sec/batch
Epoch 8/10  Iteration 1256/1780 Training loss: 1.4920 6.8017 sec/batch
Epoch 8/10  Iteration 1257/1780 Training loss: 1.4880 6.8322 sec/batch
Epoch 8/10  Iteration 1258/1780 Training loss: 1.4861 6.7058 sec/batch
Epoch 8/10  Iteration 1259/1780 Training loss: 1.4864 6.7775 sec/batch
Epoch 8/10  Iteration 1260/1780 Training loss: 1.4879 6.7596 sec/batch
Epoch 8/10  Iteration 1261/1780 Training loss: 1.4872 6.7907 sec/batch
Epoch 8/10  Iteration 1262/1780 Training loss: 1.4851 6.7603 sec/batch
Epoch 8/10  Iteration 1263/1780 Training loss: 1.4855 6.9226 sec/batch
Epoch 8/10  Iteration 1264/1780 Training loss: 1.4867 6.7597 sec/batch
Epoch 8/10  Iteration 1265/1780 Training loss: 1.4862 6.7805 sec/batch
Epoch 8/10  Iteration 1266/1780 Training loss: 1.4877 6.7539 sec/batch
Epoch 8/10  Iteration 1267/1780 Training loss: 1.4866 6.8086 sec/batch
Epoch 8/10  Iteration 1268/1780 Training loss: 1.4872 6.7738 sec/batch
Epoch 8/10  Iteration 1269/1780 Training loss: 1.4862 6.8614 sec/batch
Epoch 8/10  Iteration 1270/1780 Training loss: 1.4857 6.8752 sec/batch
Epoch 8/10  Iteration 1271/1780 Training loss: 1.4855 6.7123 sec/batch
Epoch 8/10  Iteration 1272/1780 Training loss: 1.4836 6.7987 sec/batch
Epoch 8/10  Iteration 1273/1780 Training loss: 1.4822 8.1336 sec/batch
Epoch 8/10  Iteration 1274/1780 Training loss: 1.4826 12.9507 sec/batch
Epoch 8/10  Iteration 1275/1780 Training loss: 1.4832 9.8170 sec/batch
Epoch 8/10  Iteration 1276/1780 Training loss: 1.4832 10.0057 sec/batch
Epoch 8/10  Iteration 1277/1780 Training loss: 1.4827 10.0538 sec/batch
Epoch 8/10  Iteration 1278/1780 Training loss: 1.4818 10.0582 sec/batch
Epoch 8/10  Iteration 1279/1780 Training loss: 1.4821 9.1174 sec/batch
Epoch 8/10  Iteration 1280/1780 Training loss: 1.4821 9.3117 sec/batch
Epoch 8/10  Iteration 1281/1780 Training loss: 1.4817 9.3826 sec/batch
Epoch 8/10  Iteration 1282/1780 Training loss: 1.4814 9.8056 sec/batch
Epoch 8/10  Iteration 1283/1780 Training loss: 1.4806 9.9768 sec/batch
Epoch 8/10  Iteration 1284/1780 Training loss: 1.4795 9.1761 sec/batch
Epoch 8/10  Iteration 1285/1780 Training loss: 1.4780 9.3857 sec/batch
Epoch 8/10  Iteration 1286/1780 Training loss: 1.4773 9.6684 sec/batch
Epoch 8/10  Iteration 1287/1780 Training loss: 1.4768 9.4043 sec/batch
Epoch 8/10  Iteration 1288/1780 Training loss: 1.4772 9.1586 sec/batch
Epoch 8/10  Iteration 1289/1780 Training loss: 1.4767 9.2867 sec/batch
Epoch 8/10  Iteration 1290/1780 Training loss: 1.4757 9.2270 sec/batch
Epoch 8/10  Iteration 1291/1780 Training loss: 1.4755 9.4038 sec/batch
Epoch 8/10  Iteration 1292/1780 Training loss: 1.4747 9.1886 sec/batch
Epoch 8/10  Iteration 1293/1780 Training loss: 1.4741 9.2006 sec/batch
Epoch 8/10  Iteration 1294/1780 Training loss: 1.4737 9.3998 sec/batch
Epoch 8/10  Iteration 1295/1780 Training loss: 1.4736 9.6906 sec/batch
Epoch 8/10  Iteration 1296/1780 Training loss: 1.4739 9.0529 sec/batch
Epoch 8/10  Iteration 1297/1780 Training loss: 1.4733 9.3366 sec/batch
Epoch 8/10  Iteration 1298/1780 Training loss: 1.4739 9.3207 sec/batch
Epoch 8/10  Iteration 1299/1780 Training loss: 1.4736 9.3145 sec/batch
Epoch 8/10  Iteration 1300/1780 Training loss: 1.4737 9.4561 sec/batch
Validation loss: 1.35873 Saving checkpoint!
Epoch 8/10  Iteration 1301/1780 Training loss: 1.4751 10.3507 sec/batch
Epoch 8/10  Iteration 1302/1780 Training loss: 1.4751 9.3336 sec/batch
Epoch 8/10  Iteration 1303/1780 Training loss: 1.4753 9.1867 sec/batch
Epoch 8/10  Iteration 1304/1780 Training loss: 1.4749 9.2849 sec/batch
Epoch 8/10  Iteration 1305/1780 Training loss: 1.4743 9.4553 sec/batch
Epoch 8/10  Iteration 1306/1780 Training loss: 1.4748 9.1569 sec/batch
Epoch 8/10  Iteration 1307/1780 Training loss: 1.4745 9.3705 sec/batch
Epoch 8/10  Iteration 1308/1780 Training loss: 1.4754 9.1711 sec/batch
Epoch 8/10  Iteration 1309/1780 Training loss: 1.4755 9.4088 sec/batch
Epoch 8/10  Iteration 1310/1780 Training loss: 1.4755 9.2873 sec/batch
Epoch 8/10  Iteration 1311/1780 Training loss: 1.4753 9.5498 sec/batch
Epoch 8/10  Iteration 1312/1780 Training loss: 1.4752 9.0310 sec/batch
Epoch 8/10  Iteration 1313/1780 Training loss: 1.4753 9.2106 sec/batch
Epoch 8/10  Iteration 1314/1780 Training loss: 1.4748 9.3216 sec/batch
Epoch 8/10  Iteration 1315/1780 Training loss: 1.4747 9.4749 sec/batch
Epoch 8/10  Iteration 1316/1780 Training loss: 1.4746 9.7023 sec/batch
Epoch 8/10  Iteration 1317/1780 Training loss: 1.4750 9.7489 sec/batch
Epoch 8/10  Iteration 1318/1780 Training loss: 1.4751 9.9400 sec/batch
Epoch 8/10  Iteration 1319/1780 Training loss: 1.4754 9.7438 sec/batch
Epoch 8/10  Iteration 1320/1780 Training loss: 1.4750 9.8272 sec/batch
Epoch 8/10  Iteration 1321/1780 Training loss: 1.4747 9.9090 sec/batch
Epoch 8/10  Iteration 1322/1780 Training loss: 1.4746 9.8105 sec/batch
Epoch 8/10  Iteration 1323/1780 Training loss: 1.4744 10.1241 sec/batch
Epoch 8/10  Iteration 1324/1780 Training loss: 1.4742 9.9431 sec/batch
Epoch 8/10  Iteration 1325/1780 Training loss: 1.4736 9.9879 sec/batch
Epoch 8/10  Iteration 1326/1780 Training loss: 1.4735 9.5240 sec/batch
Epoch 8/10  Iteration 1327/1780 Training loss: 1.4728 9.9259 sec/batch
Epoch 8/10  Iteration 1328/1780 Training loss: 1.4726 10.7800 sec/batch
Epoch 8/10  Iteration 1329/1780 Training loss: 1.4720 10.9227 sec/batch
Epoch 8/10  Iteration 1330/1780 Training loss: 1.4718 8.9838 sec/batch
Epoch 8/10  Iteration 1331/1780 Training loss: 1.4714 6.5682 sec/batch
Epoch 8/10  Iteration 1332/1780 Training loss: 1.4711 6.5149 sec/batch
Epoch 8/10  Iteration 1333/1780 Training loss: 1.4708 6.4648 sec/batch
Epoch 8/10  Iteration 1334/1780 Training loss: 1.4705 6.7369 sec/batch
Epoch 8/10  Iteration 1335/1780 Training loss: 1.4699 6.7496 sec/batch
Epoch 8/10  Iteration 1336/1780 Training loss: 1.4700 6.5065 sec/batch
Epoch 8/10  Iteration 1337/1780 Training loss: 1.4696 6.5699 sec/batch
Epoch 8/10  Iteration 1338/1780 Training loss: 1.4694 6.5213 sec/batch
Epoch 8/10  Iteration 1339/1780 Training loss: 1.4688 3604.5376 sec/batch
Epoch 8/10  Iteration 1340/1780 Training loss: 1.4685 7.9133 sec/batch
Epoch 8/10  Iteration 1341/1780 Training loss: 1.4681 9.1033 sec/batch
Epoch 8/10  Iteration 1342/1780 Training loss: 1.4679 11.4747 sec/batch
Epoch 8/10  Iteration 1343/1780 Training loss: 1.4677 11.0029 sec/batch
Epoch 8/10  Iteration 1344/1780 Training loss: 1.4672 10.6322 sec/batch
Epoch 8/10  Iteration 1345/1780 Training loss: 1.4666 11.3095 sec/batch
Epoch 8/10  Iteration 1346/1780 Training loss: 1.4660 11.2176 sec/batch
Epoch 8/10  Iteration 1347/1780 Training loss: 1.4658 11.4041 sec/batch
Epoch 8/10  Iteration 1348/1780 Training loss: 1.4655 11.0339 sec/batch
Epoch 8/10  Iteration 1349/1780 Training loss: 1.4651 11.2942 sec/batch
Epoch 8/10  Iteration 1350/1780 Training loss: 1.4649 12.3455 sec/batch
Epoch 8/10  Iteration 1351/1780 Training loss: 1.4645 11.1849 sec/batch
Epoch 8/10  Iteration 1352/1780 Training loss: 1.4643 11.6560 sec/batch
Epoch 8/10  Iteration 1353/1780 Training loss: 1.4640 10.8922 sec/batch
Epoch 8/10  Iteration 1354/1780 Training loss: 1.4637 11.4656 sec/batch
Epoch 8/10  Iteration 1355/1780 Training loss: 1.4634 10.8549 sec/batch
Epoch 8/10  Iteration 1356/1780 Training loss: 1.4634 11.3539 sec/batch
Epoch 8/10  Iteration 1357/1780 Training loss: 1.4631 11.0971 sec/batch
Epoch 8/10  Iteration 1358/1780 Training loss: 1.4629 11.1306 sec/batch
Epoch 8/10  Iteration 1359/1780 Training loss: 1.4626 11.0736 sec/batch
Epoch 8/10  Iteration 1360/1780 Training loss: 1.4622 11.1842 sec/batch
Epoch 8/10  Iteration 1361/1780 Training loss: 1.4618 12.0770 sec/batch
Epoch 8/10  Iteration 1362/1780 Training loss: 1.4613 11.0553 sec/batch
Epoch 8/10  Iteration 1363/1780 Training loss: 1.4611 11.3398 sec/batch
Epoch 8/10  Iteration 1364/1780 Training loss: 1.4610 11.0189 sec/batch
Epoch 8/10  Iteration 1365/1780 Training loss: 1.4607 11.4094 sec/batch
Epoch 8/10  Iteration 1366/1780 Training loss: 1.4605 11.0227 sec/batch
Epoch 8/10  Iteration 1367/1780 Training loss: 1.4602 11.4766 sec/batch
Epoch 8/10  Iteration 1368/1780 Training loss: 1.4596 10.9783 sec/batch
Epoch 8/10  Iteration 1369/1780 Training loss: 1.4591 11.4680 sec/batch
Epoch 8/10  Iteration 1370/1780 Training loss: 1.4590 11.2374 sec/batch
Epoch 8/10  Iteration 1371/1780 Training loss: 1.4587 11.6170 sec/batch
Epoch 8/10  Iteration 1372/1780 Training loss: 1.4581 11.0926 sec/batch
Epoch 8/10  Iteration 1373/1780 Training loss: 1.4580 11.5251 sec/batch
Epoch 8/10  Iteration 1374/1780 Training loss: 1.4579 11.2041 sec/batch
Epoch 8/10  Iteration 1375/1780 Training loss: 1.4575 13.3888 sec/batch
Epoch 8/10  Iteration 1376/1780 Training loss: 1.4571 10.9982 sec/batch
Epoch 8/10  Iteration 1377/1780 Training loss: 1.4566 11.3347 sec/batch
Epoch 8/10  Iteration 1378/1780 Training loss: 1.4562 11.3330 sec/batch
Epoch 8/10  Iteration 1379/1780 Training loss: 1.4561 11.0310 sec/batch
Epoch 8/10  Iteration 1380/1780 Training loss: 1.4560 11.4779 sec/batch
Epoch 8/10  Iteration 1381/1780 Training loss: 1.4558 11.0804 sec/batch
Epoch 8/10  Iteration 1382/1780 Training loss: 1.4558 11.4636 sec/batch
Epoch 8/10  Iteration 1383/1780 Training loss: 1.4558 10.9504 sec/batch
Epoch 8/10  Iteration 1384/1780 Training loss: 1.4557 11.4899 sec/batch
Epoch 8/10  Iteration 1385/1780 Training loss: 1.4556 11.3134 sec/batch
Epoch 8/10  Iteration 1386/1780 Training loss: 1.4555 12.0695 sec/batch
Epoch 8/10  Iteration 1387/1780 Training loss: 1.4557 12.3522 sec/batch
Epoch 8/10  Iteration 1388/1780 Training loss: 1.4556 13.1758 sec/batch
Epoch 8/10  Iteration 1389/1780 Training loss: 1.4553 12.2839 sec/batch
Epoch 8/10  Iteration 1390/1780 Training loss: 1.4554 12.8818 sec/batch
Epoch 8/10  Iteration 1391/1780 Training loss: 1.4551 10.7847 sec/batch
Epoch 8/10  Iteration 1392/1780 Training loss: 1.4551 10.8798 sec/batch
Epoch 8/10  Iteration 1393/1780 Training loss: 1.4550 10.5654 sec/batch
Epoch 8/10  Iteration 1394/1780 Training loss: 1.4551 10.4813 sec/batch
Epoch 8/10  Iteration 1395/1780 Training loss: 1.4550 10.7455 sec/batch
Epoch 8/10  Iteration 1396/1780 Training loss: 1.4547 10.4931 sec/batch
Epoch 8/10  Iteration 1397/1780 Training loss: 1.4543 10.5804 sec/batch
Epoch 8/10  Iteration 1398/1780 Training loss: 1.4541 10.2214 sec/batch
Epoch 8/10  Iteration 1399/1780 Training loss: 1.4541 10.3193 sec/batch
Epoch 8/10  Iteration 1400/1780 Training loss: 1.4539 10.0951 sec/batch
Validation loss: 1.3172 Saving checkpoint!
Epoch 8/10  Iteration 1401/1780 Training loss: 1.4544 10.7280 sec/batch
Epoch 8/10  Iteration 1402/1780 Training loss: 1.4543 10.1900 sec/batch
Epoch 8/10  Iteration 1403/1780 Training loss: 1.4543 10.1662 sec/batch
Epoch 8/10  Iteration 1404/1780 Training loss: 1.4541 10.3813 sec/batch
Epoch 8/10  Iteration 1405/1780 Training loss: 1.4538 10.1723 sec/batch
Epoch 8/10  Iteration 1406/1780 Training loss: 1.4537 10.2186 sec/batch
Epoch 8/10  Iteration 1407/1780 Training loss: 1.4538 10.1723 sec/batch
Epoch 8/10  Iteration 1408/1780 Training loss: 1.4537 10.1677 sec/batch
Epoch 8/10  Iteration 1409/1780 Training loss: 1.4535 10.3013 sec/batch
Epoch 8/10  Iteration 1410/1780 Training loss: 1.4534 10.4478 sec/batch
Epoch 8/10  Iteration 1411/1780 Training loss: 1.4532 10.2087 sec/batch
Epoch 8/10  Iteration 1412/1780 Training loss: 1.4530 10.2241 sec/batch
Epoch 8/10  Iteration 1413/1780 Training loss: 1.4531 10.3230 sec/batch
Epoch 8/10  Iteration 1414/1780 Training loss: 1.4533 10.2092 sec/batch
Epoch 8/10  Iteration 1415/1780 Training loss: 1.4532 12.0564 sec/batch
Epoch 8/10  Iteration 1416/1780 Training loss: 1.4530 14.7925 sec/batch
Epoch 8/10  Iteration 1417/1780 Training loss: 1.4528 21.3670 sec/batch
Epoch 8/10  Iteration 1418/1780 Training loss: 1.4525 23.4783 sec/batch
Epoch 8/10  Iteration 1419/1780 Training loss: 1.4526 683.7424 sec/batch
Epoch 8/10  Iteration 1420/1780 Training loss: 1.4524 7.5248 sec/batch
Epoch 8/10  Iteration 1421/1780 Training loss: 1.4524 10812.8740 sec/batch
Epoch 8/10  Iteration 1422/1780 Training loss: 1.4521 201874.1630 sec/batch
Epoch 8/10  Iteration 1423/1780 Training loss: 1.4517 8.6922 sec/batch
Epoch 8/10  Iteration 1424/1780 Training loss: 1.4517 10.3731 sec/batch
Epoch 9/10  Iteration 1425/1780 Training loss: 1.5355 12.0676 sec/batch
Epoch 9/10  Iteration 1426/1780 Training loss: 1.4888 10.5791 sec/batch
Epoch 9/10  Iteration 1427/1780 Training loss: 1.4686 12.4703 sec/batch
Epoch 9/10  Iteration 1428/1780 Training loss: 1.4637 15.0157 sec/batch
Epoch 9/10  Iteration 1429/1780 Training loss: 1.4499 18.2234 sec/batch
Epoch 9/10  Iteration 1430/1780 Training loss: 1.4363 17.0756 sec/batch
Epoch 9/10  Iteration 1431/1780 Training loss: 1.4346 17.4207 sec/batch
Epoch 9/10  Iteration 1432/1780 Training loss: 1.4307 15.6310 sec/batch
Epoch 9/10  Iteration 1433/1780 Training loss: 1.4297 16.9228 sec/batch
Epoch 9/10  Iteration 1434/1780 Training loss: 1.4289 16.4001 sec/batch
Epoch 9/10  Iteration 1435/1780 Training loss: 1.4261 14.9297 sec/batch
Epoch 9/10  Iteration 1436/1780 Training loss: 1.4245 14.0343 sec/batch
Epoch 9/10  Iteration 1437/1780 Training loss: 1.4247 13.7871 sec/batch
Epoch 9/10  Iteration 1438/1780 Training loss: 1.4262 11.2972 sec/batch
Epoch 9/10  Iteration 1439/1780 Training loss: 1.4248 13.0787 sec/batch
Epoch 9/10  Iteration 1440/1780 Training loss: 1.4229 15.0281 sec/batch
Epoch 9/10  Iteration 1441/1780 Training loss: 1.4226 7.2577 sec/batch
Epoch 9/10  Iteration 1442/1780 Training loss: 1.4242 7.6742 sec/batch
Epoch 9/10  Iteration 1443/1780 Training loss: 1.4239 8.6115 sec/batch
Epoch 9/10  Iteration 1444/1780 Training loss: 1.4245 7.5296 sec/batch
Epoch 9/10  Iteration 1445/1780 Training loss: 1.4237 8.3592 sec/batch
Epoch 9/10  Iteration 1446/1780 Training loss: 1.4243 7.7294 sec/batch
Epoch 9/10  Iteration 1447/1780 Training loss: 1.4236 8.0343 sec/batch
Epoch 9/10  Iteration 1448/1780 Training loss: 1.4232 7.9523 sec/batch
Epoch 9/10  Iteration 1449/1780 Training loss: 1.4229 10.5590 sec/batch
Epoch 9/10  Iteration 1450/1780 Training loss: 1.4210 7.6158 sec/batch
Epoch 9/10  Iteration 1451/1780 Training loss: 1.4200 8.0314 sec/batch
Epoch 9/10  Iteration 1452/1780 Training loss: 1.4204 7.9009 sec/batch
Epoch 9/10  Iteration 1453/1780 Training loss: 1.4206 8.3786 sec/batch
Epoch 9/10  Iteration 1454/1780 Training loss: 1.4207 7.5825 sec/batch
Epoch 9/10  Iteration 1455/1780 Training loss: 1.4201 7.0710 sec/batch
Epoch 9/10  Iteration 1456/1780 Training loss: 1.4189 7.5881 sec/batch
Epoch 9/10  Iteration 1457/1780 Training loss: 1.4189 7.1285 sec/batch
Epoch 9/10  Iteration 1458/1780 Training loss: 1.4193 7.1657 sec/batch
Epoch 9/10  Iteration 1459/1780 Training loss: 1.4188 7.4618 sec/batch
Epoch 9/10  Iteration 1460/1780 Training loss: 1.4184 7.1779 sec/batch
Epoch 9/10  Iteration 1461/1780 Training loss: 1.4175 6.9334 sec/batch
Epoch 9/10  Iteration 1462/1780 Training loss: 1.4161 6.8084 sec/batch
Epoch 9/10  Iteration 1463/1780 Training loss: 1.4146 6.8493 sec/batch
Epoch 9/10  Iteration 1464/1780 Training loss: 1.4138 6.7519 sec/batch
Epoch 9/10  Iteration 1465/1780 Training loss: 1.4131 7.0547 sec/batch
Epoch 9/10  Iteration 1466/1780 Training loss: 1.4136 7.7093 sec/batch
Epoch 9/10  Iteration 1467/1780 Training loss: 1.4131 7.3062 sec/batch
Epoch 9/10  Iteration 1468/1780 Training loss: 1.4121 7.0799 sec/batch
Epoch 9/10  Iteration 1469/1780 Training loss: 1.4122 6.9400 sec/batch
Epoch 9/10  Iteration 1470/1780 Training loss: 1.4115 6.6981 sec/batch
Epoch 9/10  Iteration 1471/1780 Training loss: 1.4114 6.6767 sec/batch
Epoch 9/10  Iteration 1472/1780 Training loss: 1.4111 6.8856 sec/batch
Epoch 9/10  Iteration 1473/1780 Training loss: 1.4110 6.7164 sec/batch
Epoch 9/10  Iteration 1474/1780 Training loss: 1.4113 6.6986 sec/batch
Epoch 9/10  Iteration 1475/1780 Training loss: 1.4109 6.8865 sec/batch
Epoch 9/10  Iteration 1476/1780 Training loss: 1.4116 6.6878 sec/batch
Epoch 9/10  Iteration 1477/1780 Training loss: 1.4113 6.6813 sec/batch
Epoch 9/10  Iteration 1478/1780 Training loss: 1.4115 6.9104 sec/batch
Epoch 9/10  Iteration 1479/1780 Training loss: 1.4113 6.7237 sec/batch
Epoch 9/10  Iteration 1480/1780 Training loss: 1.4112 6.7303 sec/batch
Epoch 9/10  Iteration 1481/1780 Training loss: 1.4115 6.8447 sec/batch
Epoch 9/10  Iteration 1482/1780 Training loss: 1.4111 6.7738 sec/batch
Epoch 9/10  Iteration 1483/1780 Training loss: 1.4106 6.6707 sec/batch
Epoch 9/10  Iteration 1484/1780 Training loss: 1.4110 6.8206 sec/batch
Epoch 9/10  Iteration 1485/1780 Training loss: 1.4110 6.8071 sec/batch
Epoch 9/10  Iteration 1486/1780 Training loss: 1.4120 10.1424 sec/batch
Epoch 9/10  Iteration 1487/1780 Training loss: 1.4123 9.8992 sec/batch
Epoch 9/10  Iteration 1488/1780 Training loss: 1.4123 10.1765 sec/batch
Epoch 9/10  Iteration 1489/1780 Training loss: 1.4121 9.8154 sec/batch
Epoch 9/10  Iteration 1490/1780 Training loss: 1.4122 8.4776 sec/batch
Epoch 9/10  Iteration 1491/1780 Training loss: 1.4124 849.0691 sec/batch
Epoch 9/10  Iteration 1492/1780 Training loss: 1.4119 11.5784 sec/batch
Epoch 9/10  Iteration 1493/1780 Training loss: 1.4120 37187.8072 sec/batch
Epoch 9/10  Iteration 1494/1780 Training loss: 1.4119 16.4187 sec/batch
Epoch 9/10  Iteration 1495/1780 Training loss: 1.4124 15.2647 sec/batch
Epoch 9/10  Iteration 1496/1780 Training loss: 1.4125 14.6919 sec/batch
Epoch 9/10  Iteration 1497/1780 Training loss: 1.4130 17.3874 sec/batch
Epoch 9/10  Iteration 1498/1780 Training loss: 1.4126 15.1464 sec/batch
Epoch 9/10  Iteration 1499/1780 Training loss: 1.4123 19.5734 sec/batch
Epoch 9/10  Iteration 1500/1780 Training loss: 1.4122 13.8119 sec/batch
Validation loss: 1.29573 Saving checkpoint!
Epoch 9/10  Iteration 1501/1780 Training loss: 1.4135 18.5947 sec/batch
Epoch 9/10  Iteration 1502/1780 Training loss: 1.4138 19.7609 sec/batch
Epoch 9/10  Iteration 1503/1780 Training loss: 1.4134 13.1369 sec/batch
Epoch 9/10  Iteration 1504/1780 Training loss: 1.4134 9.1526 sec/batch
Epoch 9/10  Iteration 1505/1780 Training loss: 1.4128 9.2004 sec/batch
Epoch 9/10  Iteration 1506/1780 Training loss: 1.4128 9.3859 sec/batch
Epoch 9/10  Iteration 1507/1780 Training loss: 1.4124 14.0805 sec/batch
Epoch 9/10  Iteration 1508/1780 Training loss: 1.4124 13.9362 sec/batch
Epoch 9/10  Iteration 1509/1780 Training loss: 1.4121 16.5871 sec/batch
Epoch 9/10  Iteration 1510/1780 Training loss: 1.4120 26.7015 sec/batch
Epoch 9/10  Iteration 1511/1780 Training loss: 1.4118 19.3252 sec/batch
Epoch 9/10  Iteration 1512/1780 Training loss: 1.4116 19.5476 sec/batch
Epoch 9/10  Iteration 1513/1780 Training loss: 1.4111 21.1422 sec/batch
Epoch 9/10  Iteration 1514/1780 Training loss: 1.4112 14.8662 sec/batch
Epoch 9/10  Iteration 1515/1780 Training loss: 1.4110 15.9085 sec/batch
Epoch 9/10  Iteration 1516/1780 Training loss: 1.4109 15.1481 sec/batch
Epoch 9/10  Iteration 1517/1780 Training loss: 1.4105 12.2644 sec/batch
Epoch 9/10  Iteration 1518/1780 Training loss: 1.4102 14.5676 sec/batch
Epoch 9/10  Iteration 1519/1780 Training loss: 1.4099 23.0462 sec/batch
Epoch 9/10  Iteration 1520/1780 Training loss: 1.4099 26.0345 sec/batch
Epoch 9/10  Iteration 1521/1780 Training loss: 1.4097 24.6665 sec/batch
Epoch 9/10  Iteration 1522/1780 Training loss: 1.4092 18.0476 sec/batch
Epoch 9/10  Iteration 1523/1780 Training loss: 1.4088 14.8670 sec/batch
Epoch 9/10  Iteration 1524/1780 Training loss: 1.4084 8.8598 sec/batch
Epoch 9/10  Iteration 1525/1780 Training loss: 1.4082 8.8866 sec/batch
Epoch 9/10  Iteration 1526/1780 Training loss: 1.4080 8.1928 sec/batch
Epoch 9/10  Iteration 1527/1780 Training loss: 1.4078 8.3688 sec/batch
Epoch 9/10  Iteration 1528/1780 Training loss: 1.4075 9.0204 sec/batch
Epoch 9/10  Iteration 1529/1780 Training loss: 1.4073 9.4800 sec/batch
Epoch 9/10  Iteration 1530/1780 Training loss: 1.4072 9.9188 sec/batch
Epoch 9/10  Iteration 1531/1780 Training loss: 1.4071 9.2514 sec/batch
Epoch 9/10  Iteration 1532/1780 Training loss: 1.4070 8.7493 sec/batch
Epoch 9/10  Iteration 1533/1780 Training loss: 1.4066 12.7626 sec/batch
Epoch 9/10  Iteration 1534/1780 Training loss: 1.4067 12.9848 sec/batch
Epoch 9/10  Iteration 1535/1780 Training loss: 1.4064 7.9900 sec/batch
Epoch 9/10  Iteration 1536/1780 Training loss: 1.4062 8.2355 sec/batch
Epoch 9/10  Iteration 1537/1780 Training loss: 1.4061 8.0008 sec/batch
Epoch 9/10  Iteration 1538/1780 Training loss: 1.4058 8.5283 sec/batch
Epoch 9/10  Iteration 1539/1780 Training loss: 1.4054 8.5918 sec/batch
Epoch 9/10  Iteration 1540/1780 Training loss: 1.4051 7.9472 sec/batch
Epoch 9/10  Iteration 1541/1780 Training loss: 1.4049 7.8744 sec/batch
Epoch 9/10  Iteration 1542/1780 Training loss: 1.4048 8.0814 sec/batch
Epoch 9/10  Iteration 1543/1780 Training loss: 1.4047 8.7059 sec/batch
Epoch 9/10  Iteration 1544/1780 Training loss: 1.4045 12.5248 sec/batch
Epoch 9/10  Iteration 1545/1780 Training loss: 1.4043 10.1950 sec/batch
Epoch 9/10  Iteration 1546/1780 Training loss: 1.4038 8.3936 sec/batch
Epoch 9/10  Iteration 1547/1780 Training loss: 1.4033 8.0170 sec/batch
Epoch 9/10  Iteration 1548/1780 Training loss: 1.4033 8.7817 sec/batch
Epoch 9/10  Iteration 1549/1780 Training loss: 1.4032 10.0585 sec/batch
Epoch 9/10  Iteration 1550/1780 Training loss: 1.4027 9.3355 sec/batch
Epoch 9/10  Iteration 1551/1780 Training loss: 1.4027 10.7979 sec/batch
Epoch 9/10  Iteration 1552/1780 Training loss: 1.4025 9.2093 sec/batch
Epoch 9/10  Iteration 1553/1780 Training loss: 1.4023 11.6731 sec/batch
Epoch 9/10  Iteration 1554/1780 Training loss: 1.4020 4.2838 sec/batch
Epoch 9/10  Iteration 1555/1780 Training loss: 1.4014 10.6136 sec/batch
Epoch 9/10  Iteration 1556/1780 Training loss: 1.4011 8.3245 sec/batch
Epoch 9/10  Iteration 1557/1780 Training loss: 1.4011 8.0814 sec/batch
Epoch 9/10  Iteration 1558/1780 Training loss: 1.4010 8.4070 sec/batch
Epoch 9/10  Iteration 1559/1780 Training loss: 1.4009 8.4782 sec/batch
Epoch 9/10  Iteration 1560/1780 Training loss: 1.4009 8.0937 sec/batch
Epoch 9/10  Iteration 1561/1780 Training loss: 1.4009 8.0944 sec/batch
Epoch 9/10  Iteration 1562/1780 Training loss: 1.4009 8.1018 sec/batch
Epoch 9/10  Iteration 1563/1780 Training loss: 1.4009 7.9816 sec/batch
Epoch 9/10  Iteration 1564/1780 Training loss: 1.4008 8.0765 sec/batch
Epoch 9/10  Iteration 1565/1780 Training loss: 1.4010 7.8997 sec/batch
Epoch 9/10  Iteration 1566/1780 Training loss: 1.4009 12.8338 sec/batch
Epoch 9/10  Iteration 1567/1780 Training loss: 1.4008 14.9542 sec/batch
Epoch 9/10  Iteration 1568/1780 Training loss: 1.4008 20.8743 sec/batch
Epoch 9/10  Iteration 1569/1780 Training loss: 1.4007 18.6036 sec/batch
Epoch 9/10  Iteration 1570/1780 Training loss: 1.4007 13.8989 sec/batch
Epoch 9/10  Iteration 1571/1780 Training loss: 1.4006 14.0300 sec/batch
Epoch 9/10  Iteration 1572/1780 Training loss: 1.4007 12.4315 sec/batch
Epoch 9/10  Iteration 1573/1780 Training loss: 1.4007 10.1598 sec/batch
Epoch 9/10  Iteration 1574/1780 Training loss: 1.4004 9.0600 sec/batch
Epoch 9/10  Iteration 1575/1780 Training loss: 1.4000 8.5540 sec/batch
Epoch 9/10  Iteration 1576/1780 Training loss: 1.3999 8.6969 sec/batch
Epoch 9/10  Iteration 1577/1780 Training loss: 1.3999 8.7298 sec/batch
Epoch 9/10  Iteration 1578/1780 Training loss: 1.3998 8.7093 sec/batch
Epoch 9/10  Iteration 1579/1780 Training loss: 1.3997 8.9463 sec/batch
Epoch 9/10  Iteration 1580/1780 Training loss: 1.3995 8.8325 sec/batch
Epoch 9/10  Iteration 1581/1780 Training loss: 1.3995 8.7653 sec/batch
Epoch 9/10  Iteration 1582/1780 Training loss: 1.3993 8.8083 sec/batch
Epoch 9/10  Iteration 1583/1780 Training loss: 1.3989 8.7529 sec/batch
Epoch 9/10  Iteration 1584/1780 Training loss: 1.3990 8.6481 sec/batch
Epoch 9/10  Iteration 1585/1780 Training loss: 1.3991 9.2831 sec/batch
Epoch 9/10  Iteration 1586/1780 Training loss: 1.3990 8.6783 sec/batch
Epoch 9/10  Iteration 1587/1780 Training loss: 1.3989 8.8290 sec/batch
Epoch 9/10  Iteration 1588/1780 Training loss: 1.3989 9.1492 sec/batch
Epoch 9/10  Iteration 1589/1780 Training loss: 1.3988 8.7948 sec/batch
Epoch 9/10  Iteration 1590/1780 Training loss: 1.3986 8.8173 sec/batch
Epoch 9/10  Iteration 1591/1780 Training loss: 1.3988 8.6557 sec/batch
Epoch 9/10  Iteration 1592/1780 Training loss: 1.3991 8.6222 sec/batch
Epoch 9/10  Iteration 1593/1780 Training loss: 1.3992 8.6901 sec/batch
Epoch 9/10  Iteration 1594/1780 Training loss: 1.3990 8.2359 sec/batch
Epoch 9/10  Iteration 1595/1780 Training loss: 1.3988 8.5705 sec/batch
Epoch 9/10  Iteration 1596/1780 Training loss: 1.3987 8.7887 sec/batch
Epoch 9/10  Iteration 1597/1780 Training loss: 1.3987 8.4359 sec/batch
Epoch 9/10  Iteration 1598/1780 Training loss: 1.3986 8.4603 sec/batch
Epoch 9/10  Iteration 1599/1780 Training loss: 1.3987 8.6960 sec/batch
Epoch 9/10  Iteration 1600/1780 Training loss: 1.3985 8.7712 sec/batch
Validation loss: 1.27353 Saving checkpoint!
Epoch 9/10  Iteration 1601/1780 Training loss: 1.3991 9.2248 sec/batch
Epoch 9/10  Iteration 1602/1780 Training loss: 1.3992 8.7380 sec/batch
Epoch 10/10  Iteration 1603/1780 Training loss: 1.5073 8.4103 sec/batch
Epoch 10/10  Iteration 1604/1780 Training loss: 1.4502 8.4888 sec/batch
Epoch 10/10  Iteration 1605/1780 Training loss: 1.4300 8.5725 sec/batch
Epoch 10/10  Iteration 1606/1780 Training loss: 1.4230 8.4557 sec/batch
Epoch 10/10  Iteration 1607/1780 Training loss: 1.4114 8.6307 sec/batch
Epoch 10/10  Iteration 1608/1780 Training loss: 1.3982 8.6628 sec/batch
Epoch 10/10  Iteration 1609/1780 Training loss: 1.3974 8.9856 sec/batch
Epoch 10/10  Iteration 1610/1780 Training loss: 1.3933 8.3352 sec/batch
Epoch 10/10  Iteration 1611/1780 Training loss: 1.3919 8.3806 sec/batch
Epoch 10/10  Iteration 1612/1780 Training loss: 1.3902 8.4408 sec/batch
Epoch 10/10  Iteration 1613/1780 Training loss: 1.3868 8.3970 sec/batch
Epoch 10/10  Iteration 1614/1780 Training loss: 1.3857 8.3455 sec/batch
Epoch 10/10  Iteration 1615/1780 Training loss: 1.3856 8.5064 sec/batch
Epoch 10/10  Iteration 1616/1780 Training loss: 1.3867 8.6262 sec/batch
Epoch 10/10  Iteration 1617/1780 Training loss: 1.3846 8.5943 sec/batch
Epoch 10/10  Iteration 1618/1780 Training loss: 1.3827 8.4441 sec/batch
Epoch 10/10  Iteration 1619/1780 Training loss: 1.3827 10.1350 sec/batch
Epoch 10/10  Iteration 1620/1780 Training loss: 1.3836 7.9556 sec/batch
Epoch 10/10  Iteration 1621/1780 Training loss: 1.3830 7.9108 sec/batch
Epoch 10/10  Iteration 1622/1780 Training loss: 1.3837 7.8719 sec/batch
Epoch 10/10  Iteration 1623/1780 Training loss: 1.3829 8.2177 sec/batch
Epoch 10/10  Iteration 1624/1780 Training loss: 1.3829 11.0109 sec/batch
Epoch 10/10  Iteration 1625/1780 Training loss: 1.3819 8.8644 sec/batch
Epoch 10/10  Iteration 1626/1780 Training loss: 1.3813 7.3569 sec/batch
Epoch 10/10  Iteration 1627/1780 Training loss: 1.3809 8.6428 sec/batch
Epoch 10/10  Iteration 1628/1780 Training loss: 1.3791 8.9616 sec/batch
Epoch 10/10  Iteration 1629/1780 Training loss: 1.3778 8.8860 sec/batch
Epoch 10/10  Iteration 1630/1780 Training loss: 1.3780 12.0084 sec/batch
Epoch 10/10  Iteration 1631/1780 Training loss: 1.3781 10.2033 sec/batch
Epoch 10/10  Iteration 1632/1780 Training loss: 1.3781 8.3607 sec/batch
Epoch 10/10  Iteration 1633/1780 Training loss: 1.3776 7.9833 sec/batch
Epoch 10/10  Iteration 1634/1780 Training loss: 1.3766 10812.2612 sec/batch
Epoch 10/10  Iteration 1635/1780 Training loss: 1.3771 8.2603 sec/batch
Epoch 10/10  Iteration 1636/1780 Training loss: 1.3772 37741.4812 sec/batch
Epoch 10/10  Iteration 1637/1780 Training loss: 1.3770 12.8822 sec/batch
Epoch 10/10  Iteration 1638/1780 Training loss: 1.3766 11.3938 sec/batch
Epoch 10/10  Iteration 1639/1780 Training loss: 1.3757 12.8909 sec/batch
Epoch 10/10  Iteration 1640/1780 Training loss: 1.3745 12.3643 sec/batch
Epoch 10/10  Iteration 1641/1780 Training loss: 1.3729 10.7177 sec/batch
Epoch 10/10  Iteration 1642/1780 Training loss: 1.3723 8.6718 sec/batch
Epoch 10/10  Iteration 1643/1780 Training loss: 1.3715 11.3204 sec/batch
Epoch 10/10  Iteration 1644/1780 Training loss: 1.3718 12.0003 sec/batch
Epoch 10/10  Iteration 1645/1780 Training loss: 1.3715 14.6808 sec/batch
Epoch 10/10  Iteration 1646/1780 Training loss: 1.3706 13.5463 sec/batch
Epoch 10/10  Iteration 1647/1780 Training loss: 1.3707 9.9471 sec/batch
Epoch 10/10  Iteration 1648/1780 Training loss: 1.3699 14.1256 sec/batch
Epoch 10/10  Iteration 1649/1780 Training loss: 1.3696 11.4782 sec/batch
Epoch 10/10  Iteration 1650/1780 Training loss: 1.3693 11.0043 sec/batch
Epoch 10/10  Iteration 1651/1780 Training loss: 1.3693 9.8595 sec/batch
Epoch 10/10  Iteration 1652/1780 Training loss: 1.3694 10.1712 sec/batch
Epoch 10/10  Iteration 1653/1780 Training loss: 1.3691 11.5103 sec/batch
Epoch 10/10  Iteration 1654/1780 Training loss: 1.3697 10.7366 sec/batch
Epoch 10/10  Iteration 1655/1780 Training loss: 1.3697 10.1723 sec/batch
Epoch 10/10  Iteration 1656/1780 Training loss: 1.3700 7.7409 sec/batch
Epoch 10/10  Iteration 1657/1780 Training loss: 1.3698 8.3729 sec/batch
Epoch 10/10  Iteration 1658/1780 Training loss: 1.3698 9.0865 sec/batch
Epoch 10/10  Iteration 1659/1780 Training loss: 1.3701 8.9893 sec/batch
Epoch 10/10  Iteration 1660/1780 Training loss: 1.3698 7.3527 sec/batch
Epoch 10/10  Iteration 1661/1780 Training loss: 1.3693 6.8524 sec/batch
Epoch 10/10  Iteration 1662/1780 Training loss: 1.3699 14.0105 sec/batch
Epoch 10/10  Iteration 1663/1780 Training loss: 1.3698 12.4719 sec/batch
Epoch 10/10  Iteration 1664/1780 Training loss: 1.3708 14.3377 sec/batch
Epoch 10/10  Iteration 1665/1780 Training loss: 1.3711 6.9231 sec/batch
Epoch 10/10  Iteration 1666/1780 Training loss: 1.3711 8.3342 sec/batch
Epoch 10/10  Iteration 1667/1780 Training loss: 1.3710 7.2408 sec/batch
Epoch 10/10  Iteration 1668/1780 Training loss: 1.3711 6.9696 sec/batch
Epoch 10/10  Iteration 1669/1780 Training loss: 1.3712 6.9907 sec/batch
Epoch 10/10  Iteration 1670/1780 Training loss: 1.3710 8.2764 sec/batch
Epoch 10/10  Iteration 1671/1780 Training loss: 1.3710 11.5343 sec/batch
Epoch 10/10  Iteration 1672/1780 Training loss: 1.3707 9.7316 sec/batch
Epoch 10/10  Iteration 1673/1780 Training loss: 1.3712 17.9990 sec/batch
Epoch 10/10  Iteration 1674/1780 Training loss: 1.3714 23.0988 sec/batch
Epoch 10/10  Iteration 1675/1780 Training loss: 1.3718 28.5821 sec/batch
Epoch 10/10  Iteration 1676/1780 Training loss: 1.3715 23.6305 sec/batch
Epoch 10/10  Iteration 1677/1780 Training loss: 1.3713 21.4116 sec/batch
Epoch 10/10  Iteration 1678/1780 Training loss: 1.3713 22.5986 sec/batch
Epoch 10/10  Iteration 1679/1780 Training loss: 1.3711 17.9546 sec/batch
Epoch 10/10  Iteration 1680/1780 Training loss: 1.3710 20.7322 sec/batch
Epoch 10/10  Iteration 1681/1780 Training loss: 1.3704 20.4356 sec/batch
Epoch 10/10  Iteration 1682/1780 Training loss: 1.3702 23.2758 sec/batch
Epoch 10/10  Iteration 1683/1780 Training loss: 1.3697 21.6841 sec/batch
Epoch 10/10  Iteration 1684/1780 Training loss: 1.3695 17.3037 sec/batch
Epoch 10/10  Iteration 1685/1780 Training loss: 1.3690 18.2822 sec/batch
Epoch 10/10  Iteration 1686/1780 Training loss: 1.3690 17.8266 sec/batch
Epoch 10/10  Iteration 1687/1780 Training loss: 1.3686 18.1766 sec/batch
Epoch 10/10  Iteration 1688/1780 Training loss: 1.3684 16.6308 sec/batch
Epoch 10/10  Iteration 1689/1780 Training loss: 1.3682 18.4774 sec/batch
Epoch 10/10  Iteration 1690/1780 Training loss: 1.3678 15.4262 sec/batch
Epoch 10/10  Iteration 1691/1780 Training loss: 1.3674 19.4691 sec/batch
Epoch 10/10  Iteration 1692/1780 Training loss: 1.3675 15.7545 sec/batch
Epoch 10/10  Iteration 1693/1780 Training loss: 1.3673 13.8856 sec/batch
Epoch 10/10  Iteration 1694/1780 Training loss: 1.3671 17.7293 sec/batch
Epoch 10/10  Iteration 1695/1780 Training loss: 1.3668 15.6934 sec/batch
Epoch 10/10  Iteration 1696/1780 Training loss: 1.3664 17.4341 sec/batch
Epoch 10/10  Iteration 1697/1780 Training loss: 1.3661 16.3547 sec/batch
Epoch 10/10  Iteration 1698/1780 Training loss: 1.3662 16.1766 sec/batch
Epoch 10/10  Iteration 1699/1780 Training loss: 1.3662 15.9921 sec/batch
Epoch 10/10  Iteration 1700/1780 Training loss: 1.3657 12.9797 sec/batch
Validation loss: 1.25689 Saving checkpoint!
Epoch 10/10  Iteration 1701/1780 Training loss: 1.3669 22.2634 sec/batch
Epoch 10/10  Iteration 1702/1780 Training loss: 1.3667 17.2118 sec/batch
Epoch 10/10  Iteration 1703/1780 Training loss: 1.3667 23.6767 sec/batch
Epoch 10/10  Iteration 1704/1780 Training loss: 1.3666 23.5828 sec/batch
Epoch 10/10  Iteration 1705/1780 Training loss: 1.3665 17.8249 sec/batch
Epoch 10/10  Iteration 1706/1780 Training loss: 1.3665 20.1574 sec/batch
Epoch 10/10  Iteration 1707/1780 Training loss: 1.3663 21.9347 sec/batch
Epoch 10/10  Iteration 1708/1780 Training loss: 1.3663 20.6567 sec/batch
Epoch 10/10  Iteration 1709/1780 Training loss: 1.3663 21.2810 sec/batch
Epoch 10/10  Iteration 1710/1780 Training loss: 1.3663 20.6186 sec/batch
Epoch 10/10  Iteration 1711/1780 Training loss: 1.3661 22.7698 sec/batch
Epoch 10/10  Iteration 1712/1780 Training loss: 1.3662 18.0226 sec/batch
Epoch 10/10  Iteration 1713/1780 Training loss: 1.3660 19.1658 sec/batch
Epoch 10/10  Iteration 1714/1780 Training loss: 1.3660 23.8427 sec/batch
Epoch 10/10  Iteration 1715/1780 Training loss: 1.3659 21.7284 sec/batch
Epoch 10/10  Iteration 1716/1780 Training loss: 1.3657 21.6143 sec/batch
Epoch 10/10  Iteration 1717/1780 Training loss: 1.3654 11.2756 sec/batch
Epoch 10/10  Iteration 1718/1780 Training loss: 1.3651 18.2411 sec/batch
Epoch 10/10  Iteration 1719/1780 Training loss: 1.3650 14.2341 sec/batch
Epoch 10/10  Iteration 1720/1780 Training loss: 1.3649 8.3441 sec/batch
Epoch 10/10  Iteration 1721/1780 Training loss: 1.3647 11.2680 sec/batch
Epoch 10/10  Iteration 1722/1780 Training loss: 1.3646 22.2165 sec/batch
Epoch 10/10  Iteration 1723/1780 Training loss: 1.3644 13.7111 sec/batch
Epoch 10/10  Iteration 1724/1780 Training loss: 1.3640 14.6515 sec/batch
Epoch 10/10  Iteration 1725/1780 Training loss: 1.3635 14.8900 sec/batch
Epoch 10/10  Iteration 1726/1780 Training loss: 1.3635 19.7535 sec/batch
Epoch 10/10  Iteration 1727/1780 Training loss: 1.3633 17.4293 sec/batch
Epoch 10/10  Iteration 1728/1780 Training loss: 1.3629 23.3810 sec/batch
Epoch 10/10  Iteration 1729/1780 Training loss: 1.3628 19.6694 sec/batch
Epoch 10/10  Iteration 1730/1780 Training loss: 1.3628 16.8234 sec/batch
Epoch 10/10  Iteration 1731/1780 Training loss: 1.3625 15.6856 sec/batch
Epoch 10/10  Iteration 1732/1780 Training loss: 1.3621 20.4446 sec/batch
Epoch 10/10  Iteration 1733/1780 Training loss: 1.3616 18.2818 sec/batch
Epoch 10/10  Iteration 1734/1780 Training loss: 1.3613 20.2853 sec/batch
Epoch 10/10  Iteration 1735/1780 Training loss: 1.3615 22.2340 sec/batch
Epoch 10/10  Iteration 1736/1780 Training loss: 1.3614 15.7794 sec/batch
Epoch 10/10  Iteration 1737/1780 Training loss: 1.3613 15.4632 sec/batch
Epoch 10/10  Iteration 1738/1780 Training loss: 1.3613 16.3355 sec/batch
Epoch 10/10  Iteration 1739/1780 Training loss: 1.3614 16.2120 sec/batch
Epoch 10/10  Iteration 1740/1780 Training loss: 1.3615 23.4453 sec/batch
Epoch 10/10  Iteration 1741/1780 Training loss: 1.3615 22.2910 sec/batch
Epoch 10/10  Iteration 1742/1780 Training loss: 1.3614 19.1404 sec/batch
Epoch 10/10  Iteration 1743/1780 Training loss: 1.3616 12.9988 sec/batch
Epoch 10/10  Iteration 1744/1780 Training loss: 1.3616 19.4338 sec/batch
Epoch 10/10  Iteration 1745/1780 Training loss: 1.3615 16.5083 sec/batch
Epoch 10/10  Iteration 1746/1780 Training loss: 1.3616 15.4224 sec/batch
Epoch 10/10  Iteration 1747/1780 Training loss: 1.3613 14.5853 sec/batch
Epoch 10/10  Iteration 1748/1780 Training loss: 1.3614 15.5691 sec/batch
Epoch 10/10  Iteration 1749/1780 Training loss: 1.3613 16.7046 sec/batch
Epoch 10/10  Iteration 1750/1780 Training loss: 1.3615 15.5961 sec/batch
Epoch 10/10  Iteration 1751/1780 Training loss: 1.3615 15.1282 sec/batch
Epoch 10/10  Iteration 1752/1780 Training loss: 1.3613 13.2484 sec/batch
Epoch 10/10  Iteration 1753/1780 Training loss: 1.3609 16.3708 sec/batch
Epoch 10/10  Iteration 1754/1780 Training loss: 1.3608 12.7531 sec/batch
Epoch 10/10  Iteration 1755/1780 Training loss: 1.3607 11.4577 sec/batch
Epoch 10/10  Iteration 1756/1780 Training loss: 1.3607 11.4650 sec/batch
Epoch 10/10  Iteration 1757/1780 Training loss: 1.3606 15.8221 sec/batch
Epoch 10/10  Iteration 1758/1780 Training loss: 1.3605 19.0987 sec/batch
Epoch 10/10  Iteration 1759/1780 Training loss: 1.3605 19.3942 sec/batch
Epoch 10/10  Iteration 1760/1780 Training loss: 1.3604 15.9489 sec/batch
Epoch 10/10  Iteration 1761/1780 Training loss: 1.3600 21.1150 sec/batch
Epoch 10/10  Iteration 1762/1780 Training loss: 1.3600 17.4902 sec/batch
Epoch 10/10  Iteration 1763/1780 Training loss: 1.3602 18.6988 sec/batch
Epoch 10/10  Iteration 1764/1780 Training loss: 1.3602 13.6648 sec/batch
Epoch 10/10  Iteration 1765/1780 Training loss: 1.3601 9.8737 sec/batch
Epoch 10/10  Iteration 1766/1780 Training loss: 1.3600 18.3233 sec/batch
Epoch 10/10  Iteration 1767/1780 Training loss: 1.3600 20.3229 sec/batch
Epoch 10/10  Iteration 1768/1780 Training loss: 1.3599 16.9430 sec/batch
Epoch 10/10  Iteration 1769/1780 Training loss: 1.3600 17.2586 sec/batch
Epoch 10/10  Iteration 1770/1780 Training loss: 1.3604 17.7381 sec/batch
Epoch 10/10  Iteration 1771/1780 Training loss: 1.3604 15.7351 sec/batch
Epoch 10/10  Iteration 1772/1780 Training loss: 1.3604 13.0037 sec/batch
Epoch 10/10  Iteration 1773/1780 Training loss: 1.3602 11.0918 sec/batch
Epoch 10/10  Iteration 1774/1780 Training loss: 1.3600 11.5442 sec/batch
Epoch 10/10  Iteration 1775/1780 Training loss: 1.3602 16.7632 sec/batch
Epoch 10/10  Iteration 1776/1780 Training loss: 1.3601 17.2007 sec/batch
Epoch 10/10  Iteration 1777/1780 Training loss: 1.3602 11.0171 sec/batch
Epoch 10/10  Iteration 1778/1780 Training loss: 1.3600 10.4032 sec/batch
Epoch 10/10  Iteration 1779/1780 Training loss: 1.3598 11.5864 sec/batch
Epoch 10/10  Iteration 1780/1780 Training loss: 1.3599 10.2447 sec/batch
Validation loss: 1.24253 Saving checkpoint!



In [14]:

    
tf.train.get_checkpoint_state('checkpoints/anna')

Sampling

Now that the network is trained, we'll can use it to generate new text. The idea is that we pass in a character, then the network will predict the next character. We can use the new one, to predict the next one. And we keep doing this to generate all new text. I also included some functionality to prime the network with some text by passing in a string and building up a state from that.

The network gives us predictions for each character. To reduce noise and make things a little less random, I'm going to only choose a new character from the top N most likely characters.



In [15]:

    
def pick_top_n(preds, vocab_size, top_n=5):
    p = np.squeeze(preds)
    p[np.argsort(p)[:-top_n]] = 0
    p = p / np.sum(p)
    c = np.random.choice(vocab_size, 1, p=p)[0]
    return c



In [16]:

    
def sample(checkpoint, n_samples, lstm_size, vocab_size, prime="The "):
    prime = "Far"
    samples = [c for c in prime]
    model = build_rnn(vocab_size, lstm_size=lstm_size, sampling=True)
    saver = tf.train.Saver()
    with tf.Session() as sess:
        saver.restore(sess, checkpoint)
        new_state = sess.run(model.initial_state)
        for c in prime:
            x = np.zeros((1, 1))
            x[0,0] = vocab_to_int[c]
            feed = {model.inputs: x,
                    model.keep_prob: 1.,
                    model.initial_state: new_state}
            preds, new_state = sess.run([model.preds, model.final_state], 
                                         feed_dict=feed)

        c = pick_top_n(preds, len(vocab))
        samples.append(int_to_vocab[c])

        for i in range(n_samples):
            x[0,0] = c
            feed = {model.inputs: x,
                    model.keep_prob: 1.,
                    model.initial_state: new_state}
            preds, new_state = sess.run([model.preds, model.final_state], 
                                         feed_dict=feed)

            c = pick_top_n(preds, len(vocab))
            samples.append(int_to_vocab[c])
        
    return ''.join(samples)



In [44]:

    
checkpoint = "checkpoints/anna/i3560_l512_1.122.ckpt"
samp = sample(checkpoint, 2000, lstm_size, len(vocab), prime="Far")
print(samp)









    



Farlathit that if had so
like it that it were. He could not trouble to his wife, and there was
anything in them of the side of his weaky in the creature at his forteren
to him.

"What is it? I can't bread to those," said Stepan Arkadyevitch. "It's not
my children, and there is an almost this arm, true it mays already,
and tell you what I have say to you, and was not looking at the peasant,
why is, I don't know him out, and she doesn't speak to me immediately, as
you would say the countess and the more frest an angelembre, and time and
things's silent, but I was not in my stand that is in my head. But if he
say, and was so feeling with his soul. A child--in his soul of his
soul of his soul. He should not see that any of that sense of. Here he
had not been so composed and to speak for as in a whole picture, but
all the setting and her excellent and society, who had been delighted
and see to anywing had been being troed to thousand words on them,
we liked him.

That set in her money at the table, he came into the party. The capable
of his she could not be as an old composure.

"That's all something there will be down becime by throe is
such a silent, as in a countess, I should state it out and divorct.
The discussion is not for me. I was that something was simply they are
all three manshess of a sensitions of mind it all."

"No," he thought, shouted and lifting his soul. "While it might see your
honser and she, I could burst. And I had been a midelity. And I had a
marnief are through the countess," he said, looking at him, a chosing
which they had been carried out and still solied, and there was a sen that
was to be completely, and that this matter of all the seconds of it, and
a concipation were to her husband, who came up and conscaously, that he
was not the station. All his fourse she was always at the country,,
to speak oft, and though they were to hear the delightful throom and
whether they came towards the morning, and his living and a coller and
hold--the children.



In [43]:

    
checkpoint = "checkpoints/anna/i200_l512_2.432.ckpt"
samp = sample(checkpoint, 1000, lstm_size, len(vocab), prime="Far")
print(samp)









    



Farnt him oste wha sorind thans tout thint asd an sesand an hires on thime sind thit aled, ban thand and out hore as the ter hos ton ho te that, was tis tart al the hand sostint him sore an tit an son thes, win he se ther san ther hher tas tarereng,.

Anl at an ades in ond hesiln, ad hhe torers teans, wast tar arering tho this sos alten sorer has hhas an siton ther him he had sin he ard ate te anling the sosin her ans and
arins asd and ther ale te tot an tand tanginge wath and ho ald, so sot th asend sat hare sother horesinnd, he hesense wing ante her so tith tir sherinn, anded and to the toul anderin he sorit he torsith she se atere an ting ot hand and thit hhe so the te wile har
ens ont in the sersise, and we he seres tar aterer, to ato tat or has he he wan ton here won and sen heren he sosering, to to theer oo adent har herere the wosh oute, was serild ward tous hed astend..

I's sint on alt in har tor tit her asd hade shithans ored he talereng an soredendere tim tot hees. Tise sor and



In [46]:

    
checkpoint = "checkpoints/anna/i600_l512_1.750.ckpt"
samp = sample(checkpoint, 1000, lstm_size, len(vocab), prime="Far")
print(samp)









    



Fard as astice her said he celatice of to seress in the raice, and to be the some and sere allats to that said to that the sark and a cast a the wither ald the pacinesse of her had astition, he said to the sount as she west at hissele. Af the cond it he was a fact onthis astisarianing.


"Or a ton to to be that's a more at aspestale as the sont of anstiring as
thours and trey.

The same wo dangring the
raterst, who sore and somethy had ast out an of his book. "We had's beane were that, and a morted a thay he had to tere. Then to
her homent andertersed his his ancouted to the pirsted, the soution for of the pirsice inthirgest and stenciol, with the hard and and
a colrice of to be oneres,
the song to this anderssad.
The could ounterss the said to serom of
soment a carsed of sheres of she
torded
har and want in their of hould, but
her told in that in he tad a the same to her. Serghing an her has and with the seed, and the camt ont his about of the
sail, the her then all houg ant or to hus to



In [47]:

    
checkpoint = "checkpoints/anna/i1000_l512_1.484.ckpt"
samp = sample(checkpoint, 1000, lstm_size, len(vocab), prime="Far")
print(samp)









    



Farrat, his felt has at it.

"When the pose ther hor exceed
to his sheant was," weat a sime of his sounsed. The coment and the facily that which had began terede a marilicaly whice whether the pose of his hand, at she was alligated herself the same on she had to
taiking to his forthing and streath how to hand
began in a lang at some at it, this he cholded not set all her. "Wo love that is setthing. Him anstering as seen that."

"Yes in the man that say the mare a crances is it?" said Sergazy Ivancatching. "You doon think were somether is ifficult of a mone of
though the most at the countes that the
mean on the come to say the most, to
his feesing of
a man she, whilo he
sained and well, that he would still at to said. He wind at his for the sore in the most
of hoss and almoved to see him. They have betine the sumper into at he his stire, and what he was that at the so steate of the
sound, and shin should have a geest of shall feet on the conderation to she had been at that imporsing the dre