Large-Scale Stochastic Variational GP Regression (CUDA) (w/ KISS-GP)

Overview

In this notebook, we'll give an overview of how to use Deep Kernel Learning with SKI and stochastic variational regression to rapidly train using minibatches on the song UCI dataset, which has hundreds of thousands of training examples.

Stochastic variational inference has several major advantages over the standard regression setting. Most notably, the ELBO used for optimization decomposes in such a way that stochastic gradient descent techniques can be used. See https://arxiv.org/pdf/1411.2005.pdf and https://arxiv.org/pdf/1611.00336.pdf for more technical details of this.



In [1]:

    
import math
import torch
import gpytorch
from matplotlib import pyplot as plt

# Make plots inline
%matplotlib inline

Loading Data

For this example notebook, we'll be using the 3droad UCI dataset used in the paper. Running the next cell downloads a copy of the dataset that has already been scaled and normalized appropriately. For this notebook, we'll simply be splitting the data using the first 80% of the data as training and the last 20% as testing.

Note: Running the next cell will attempt to download a ~136 MB file to the current directory.



In [2]:

    
import urllib.request
import os.path
from scipy.io import loadmat
from math import floor

if not os.path.isfile('3droad.mat'):
    print('Downloading \'3droad\' UCI dataset...')
    urllib.request.urlretrieve('https://www.dropbox.com/s/f6ow1i59oqx05pl/3droad.mat?dl=1', '3droad.mat')
    
data = torch.Tensor(loadmat('3droad.mat')['data'])
X = data[:, :-1]
X = X - X.min(0)[0]
X = 2 * (X / X.max(0)[0]) - 1
y = data[:, -1]

# Use the first 80% of the data for training, and the last 20% for testing.
train_n = int(floor(0.8*len(X)))

train_x = X[:train_n, :].contiguous().cuda()
train_y = y[:train_n].contiguous().cuda()

test_x = X[train_n:, :].contiguous().cuda()
test_y = y[train_n:].contiguous().cuda()

Creating a DataLoader

The next step is to create a torch DataLoader that will handle getting us random minibatches of data. This involves using the standard TensorDataset and DataLoader modules provided by PyTorch.

In this notebook we'll be using a fairly large batch size of 1024 just to make optimization run faster, but you could of course change this as you so choose.



In [3]:

    
from torch.utils.data import TensorDataset, DataLoader
train_dataset = TensorDataset(train_x, train_y)
train_loader = DataLoader(train_dataset, batch_size=2028, shuffle=True)

Defining the DKL Feature Extractor

Next, we define the neural network feature extractor used to define the deep kernel. In this case, we use a fully connected network with the architecture d -> 1000 -> 500 -> 50 -> 2, as described in the original DKL paper. All of the code below uses standard PyTorch implementations of neural network layers.



In [4]:

    
data_dim = train_x.size(-1)

class LargeFeatureExtractor(torch.nn.Sequential):           
    def __init__(self):                                      
        super(LargeFeatureExtractor, self).__init__()        
        self.add_module('linear1', torch.nn.Linear(data_dim, 1000))
        self.add_module('bn1', torch.nn.BatchNorm1d(1000))
        self.add_module('relu1', torch.nn.ReLU())
        self.add_module('linear2', torch.nn.Linear(1000, 1000))
        self.add_module('bn2', torch.nn.BatchNorm1d(1000))
        self.add_module('relu2', torch.nn.ReLU())                       
        self.add_module('linear3', torch.nn.Linear(1000, 500))
        self.add_module('bn3', torch.nn.BatchNorm1d(500))
        self.add_module('relu3', torch.nn.ReLU())                  
        self.add_module('linear4', torch.nn.Linear(500, 50))       
        self.add_module('bn4', torch.nn.BatchNorm1d(50))
        self.add_module('relu4', torch.nn.ReLU())                  
        self.add_module('linear5', torch.nn.Linear(50, 2))         
                                                             
feature_extractor = LargeFeatureExtractor().cuda()
# num_features is the number of final features extracted by the neural network, in this case 2.
num_features = 2

Defining the GP Regression Layer

We now define the GP regression module that, intuitvely, will act as the final "layer" of our neural network. In this case, because we are doing variational inference and not exact inference, we will be using an AbstractVariationalGP. To use grid interpolation for variational inference, we'll be using a GridInterpolationVariationalStrategy.

Because the feature extractor we defined above extracts two features, we'll need to define our grid bounds over two dimensions.

See the CIFAR example for using an AbstractVariationalGP with an AdditiveGridInterpolationVariationalStrategy, which additionally assumes the kernel decomposes additively, which is a strong modelling assumption but allows us to use many more output features from the neural network.



In [5]:

    
from gpytorch.models import AbstractVariationalGP
from gpytorch.variational import CholeskyVariationalDistribution, GridInterpolationVariationalStrategy

softplus = torch.functional.F.softplus

class GPRegressionLayer(AbstractVariationalGP):
    def __init__(self, grid_size=32, grid_bounds=[(-1, 1), (-1, 1)]):
        variational_distribution = CholeskyVariationalDistribution(num_inducing_points=grid_size*grid_size)
        variational_strategy = GridInterpolationVariationalStrategy(self,
                                                                    grid_size=grid_size,
                                                                    grid_bounds=grid_bounds,
                                                                    variational_distribution=variational_distribution)
        super(GPRegressionLayer, self).__init__(variational_strategy)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(
            gpytorch.kernels.RBFKernel(param_transform=softplus), param_transform=softplus
        )

    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

Defining the DKL Model

With the feature extractor and GP regression layer defined, we can now define our full model. To do this, we simply create a module whose forward() method passes the data first through the feature extractor, and then through the GP regression layer.

The only other interesting feature of the model below is that we use a helper function, scale_to_bounds, to ensure that the features extracted by the neural network fit within the grid bounds used for SKI.



In [6]:

    
class DKLModel(gpytorch.Module):
    def __init__(self, feature_extractor, num_features, grid_bounds=(-1., 1.)):
        super(DKLModel, self).__init__()
        self.feature_extractor = feature_extractor
        self.gp_layer = GPRegressionLayer()
        self.grid_bounds = grid_bounds
        self.num_features = num_features

    def forward(self, x):
        features = self.feature_extractor(x)
        features = gpytorch.utils.grid.scale_to_bounds(features, self.grid_bounds[0], self.grid_bounds[1])
        res = self.gp_layer(features)
        return res

model = DKLModel(feature_extractor, num_features=num_features).cuda()
likelihood = gpytorch.likelihoods.GaussianLikelihood(param_transform=softplus).cuda()

Training the Model

The cell below trains the DKL model above, learning both the hyperparameters of the Gaussian process and the parameters of the neural network in an end-to-end fashion using Type-II MLE.

Unlike when using the exact GP marginal log likelihood, performing variational inference allows us to make use of stochastic optimization techniques. For this example, we'll do one epoch of training. Given the small size of the neural network relative to the size of the dataset, this should be sufficient to achieve comparable accuracy to what was observed in the DKL paper.

The optimization loop differs from the one seen in our more simple tutorials in that it involves looping over both a number of training iterations (epochs) and minibatches of the data. However, the basic process is the same: for each minibatch, we forward through the model, compute the loss (the VariationalELBO), call backwards, and do a step of optimization.



In [7]:

    
model.train()
likelihood.train()

# We'll do 6 epochs of training in this tutorial
num_epochs = 7

# We use SGD here, rather than Adam. Emperically, we find that SGD is better for variational regression
optimizer = torch.optim.Adam([
    {'params': model.feature_extractor.parameters(), 'weight_decay': 1e-3},
    {'params': model.gp_layer.parameters()},
    {'params': likelihood.parameters()},
], lr=0.1)

scheduler = torch.optim.lr_scheduler.MultiStepLR(optimizer, milestones=[3, 5], gamma=0.1)

# Our loss object. We're using the VariationalELBO, which essentially just computes the ELBO
mll = gpytorch.mlls.VariationalELBO(likelihood, model.gp_layer, num_data=train_y.size(0), combine_terms=False)

for i in range(num_epochs):
    scheduler.step()
    # Within each iteration, we will go over each minibatch of data
    for minibatch_i, (x_batch, y_batch) in enumerate(train_loader):        
        optimizer.zero_grad()
        
        # Because the grid is relatively small, we turn off the Toeplitz matrix multiplication and just perform them directly
        # We find this to be more efficient when the grid is very small.
        with gpytorch.settings.use_toeplitz(False):
            output = model(x_batch)
            log_lik, kl_div, log_prior = mll(output, y_batch)
            loss = -(log_lik - kl_div + log_prior)
            print('Epoch %d [%d/%d] - Loss: %.3f [%.3f, %.3f, %.3f]' % (i + 1, minibatch_i, len(train_loader), loss.item(), log_lik.item(), kl_div.item(), log_prior.item()))

        # The actual optimization step
        loss.backward()
        optimizer.step()









    



Epoch 1 [0/172] - Loss: 256.829 [-256.829, 0.000, 0.000]
Epoch 1 [1/172] - Loss: 237.597 [-237.228, 0.369, 0.000]
Epoch 1 [2/172] - Loss: 226.816 [-226.364, 0.452, 0.000]
Epoch 1 [3/172] - Loss: 201.177 [-200.882, 0.295, 0.000]
Epoch 1 [4/172] - Loss: 184.058 [-183.559, 0.499, 0.000]
Epoch 1 [5/172] - Loss: 182.979 [-182.470, 0.509, 0.000]
Epoch 1 [6/172] - Loss: 187.616 [-187.123, 0.493, 0.000]
Epoch 1 [7/172] - Loss: 169.386 [-168.979, 0.407, 0.000]
Epoch 1 [8/172] - Loss: 145.485 [-145.099, 0.386, 0.000]
Epoch 1 [9/172] - Loss: 144.235 [-143.850, 0.384, 0.000]
Epoch 1 [10/172] - Loss: 139.877 [-139.527, 0.349, 0.000]
Epoch 1 [11/172] - Loss: 133.140 [-132.759, 0.381, 0.000]
Epoch 1 [12/172] - Loss: 120.182 [-119.813, 0.369, 0.000]
Epoch 1 [13/172] - Loss: 117.984 [-117.599, 0.385, 0.000]
Epoch 1 [14/172] - Loss: 111.975 [-111.598, 0.377, 0.000]
Epoch 1 [15/172] - Loss: 102.585 [-102.230, 0.355, 0.000]
Epoch 1 [16/172] - Loss: 111.100 [-110.779, 0.322, 0.000]
Epoch 1 [17/172] - Loss: 98.811 [-98.510, 0.301, 0.000]
Epoch 1 [18/172] - Loss: 99.875 [-99.592, 0.283, 0.000]
Epoch 1 [19/172] - Loss: 92.438 [-92.177, 0.261, 0.000]
Epoch 1 [20/172] - Loss: 95.095 [-94.879, 0.217, 0.000]
Epoch 1 [21/172] - Loss: 89.734 [-89.566, 0.168, 0.000]
Epoch 1 [22/172] - Loss: 89.488 [-89.362, 0.126, 0.000]
Epoch 1 [23/172] - Loss: 89.191 [-89.098, 0.093, 0.000]
Epoch 1 [24/172] - Loss: 84.126 [-84.060, 0.067, 0.000]
Epoch 1 [25/172] - Loss: 83.090 [-83.045, 0.045, 0.000]
Epoch 1 [26/172] - Loss: 81.729 [-81.698, 0.031, 0.000]
Epoch 1 [27/172] - Loss: 75.172 [-75.150, 0.023, 0.000]
Epoch 1 [28/172] - Loss: 70.712 [-70.695, 0.017, 0.000]
Epoch 1 [29/172] - Loss: 77.441 [-77.427, 0.014, 0.000]
Epoch 1 [30/172] - Loss: 67.502 [-67.490, 0.012, 0.000]
Epoch 1 [31/172] - Loss: 73.067 [-73.056, 0.011, 0.000]
Epoch 1 [32/172] - Loss: 71.902 [-71.891, 0.011, 0.000]
Epoch 1 [33/172] - Loss: 69.254 [-69.243, 0.011, 0.000]
Epoch 1 [34/172] - Loss: 68.644 [-68.634, 0.011, 0.000]
Epoch 1 [35/172] - Loss: 63.679 [-63.668, 0.011, 0.000]
Epoch 1 [36/172] - Loss: 62.348 [-62.337, 0.011, 0.000]
Epoch 1 [37/172] - Loss: 60.129 [-60.118, 0.011, 0.000]
Epoch 1 [38/172] - Loss: 61.133 [-61.121, 0.011, 0.000]
Epoch 1 [39/172] - Loss: 58.680 [-58.669, 0.011, 0.000]
Epoch 1 [40/172] - Loss: 58.826 [-58.815, 0.011, 0.000]
Epoch 1 [41/172] - Loss: 58.208 [-58.197, 0.011, 0.000]
Epoch 1 [42/172] - Loss: 59.898 [-59.887, 0.012, 0.000]
Epoch 1 [43/172] - Loss: 58.657 [-58.645, 0.012, 0.000]
Epoch 1 [44/172] - Loss: 56.731 [-56.720, 0.012, 0.000]
Epoch 1 [45/172] - Loss: 59.812 [-59.800, 0.012, 0.000]
Epoch 1 [46/172] - Loss: 60.670 [-60.658, 0.012, 0.000]
Epoch 1 [47/172] - Loss: 60.445 [-60.433, 0.012, 0.000]
Epoch 1 [48/172] - Loss: 56.949 [-56.937, 0.012, 0.000]
Epoch 1 [49/172] - Loss: 59.060 [-59.048, 0.012, 0.000]
Epoch 1 [50/172] - Loss: 55.551 [-55.539, 0.012, 0.000]
Epoch 1 [51/172] - Loss: 53.587 [-53.575, 0.012, 0.000]
Epoch 1 [52/172] - Loss: 53.161 [-53.149, 0.012, 0.000]
Epoch 1 [53/172] - Loss: 51.820 [-51.808, 0.012, 0.000]
Epoch 1 [54/172] - Loss: 51.784 [-51.771, 0.012, 0.000]
Epoch 1 [55/172] - Loss: 55.737 [-55.725, 0.012, 0.000]
Epoch 1 [56/172] - Loss: 51.258 [-51.246, 0.012, 0.000]
Epoch 1 [57/172] - Loss: 50.776 [-50.764, 0.012, 0.000]
Epoch 1 [58/172] - Loss: 52.634 [-52.622, 0.012, 0.000]
Epoch 1 [59/172] - Loss: 52.315 [-52.303, 0.012, 0.000]
Epoch 1 [60/172] - Loss: 51.028 [-51.016, 0.012, 0.000]
Epoch 1 [61/172] - Loss: 49.471 [-49.459, 0.012, 0.000]
Epoch 1 [62/172] - Loss: 51.543 [-51.531, 0.012, 0.000]
Epoch 1 [63/172] - Loss: 44.998 [-44.986, 0.012, 0.000]
Epoch 1 [64/172] - Loss: 50.519 [-50.507, 0.012, 0.000]
Epoch 1 [65/172] - Loss: 45.577 [-45.565, 0.012, 0.000]
Epoch 1 [66/172] - Loss: 48.021 [-48.009, 0.012, 0.000]
Epoch 1 [67/172] - Loss: 49.987 [-49.975, 0.012, 0.000]
Epoch 1 [68/172] - Loss: 48.026 [-48.014, 0.012, 0.000]
Epoch 1 [69/172] - Loss: 43.552 [-43.541, 0.012, 0.000]
Epoch 1 [70/172] - Loss: 46.572 [-46.561, 0.012, 0.000]
Epoch 1 [71/172] - Loss: 45.619 [-45.607, 0.012, 0.000]
Epoch 1 [72/172] - Loss: 44.535 [-44.523, 0.012, 0.000]
Epoch 1 [73/172] - Loss: 41.512 [-41.501, 0.011, 0.000]
Epoch 1 [74/172] - Loss: 43.729 [-43.718, 0.011, 0.000]
Epoch 1 [75/172] - Loss: 42.864 [-42.852, 0.011, 0.000]
Epoch 1 [76/172] - Loss: 44.146 [-44.135, 0.011, 0.000]
Epoch 1 [77/172] - Loss: 39.930 [-39.919, 0.011, 0.000]
Epoch 1 [78/172] - Loss: 41.369 [-41.357, 0.011, 0.000]
Epoch 1 [79/172] - Loss: 44.265 [-44.254, 0.011, 0.000]
Epoch 1 [80/172] - Loss: 43.240 [-43.229, 0.011, 0.000]
Epoch 1 [81/172] - Loss: 43.147 [-43.136, 0.011, 0.000]
Epoch 1 [82/172] - Loss: 42.395 [-42.384, 0.011, 0.000]
Epoch 1 [83/172] - Loss: 40.624 [-40.613, 0.011, 0.000]
Epoch 1 [84/172] - Loss: 39.747 [-39.736, 0.011, 0.000]
Epoch 1 [85/172] - Loss: 37.813 [-37.803, 0.011, 0.000]
Epoch 1 [86/172] - Loss: 39.206 [-39.196, 0.011, 0.000]
Epoch 1 [87/172] - Loss: 44.632 [-44.622, 0.011, 0.000]
Epoch 1 [88/172] - Loss: 42.226 [-42.216, 0.010, 0.000]
Epoch 1 [89/172] - Loss: 37.774 [-37.763, 0.010, 0.000]
Epoch 1 [90/172] - Loss: 40.036 [-40.026, 0.010, 0.000]
Epoch 1 [91/172] - Loss: 40.957 [-40.947, 0.010, 0.000]
Epoch 1 [92/172] - Loss: 40.093 [-40.083, 0.010, 0.000]
Epoch 1 [93/172] - Loss: 44.472 [-44.462, 0.010, 0.000]
Epoch 1 [94/172] - Loss: 40.993 [-40.983, 0.010, 0.000]
Epoch 1 [95/172] - Loss: 37.381 [-37.371, 0.010, 0.000]
Epoch 1 [96/172] - Loss: 35.961 [-35.951, 0.010, 0.000]
Epoch 1 [97/172] - Loss: 39.019 [-39.010, 0.010, 0.000]
Epoch 1 [98/172] - Loss: 36.973 [-36.964, 0.010, 0.000]
Epoch 1 [99/172] - Loss: 39.148 [-39.138, 0.010, 0.000]
Epoch 1 [100/172] - Loss: 42.111 [-42.102, 0.010, 0.000]
Epoch 1 [101/172] - Loss: 40.401 [-40.392, 0.010, 0.000]
Epoch 1 [102/172] - Loss: 42.556 [-42.547, 0.009, 0.000]
Epoch 1 [103/172] - Loss: 36.869 [-36.860, 0.009, 0.000]
Epoch 1 [104/172] - Loss: 38.346 [-38.337, 0.009, 0.000]
Epoch 1 [105/172] - Loss: 35.110 [-35.101, 0.009, 0.000]
Epoch 1 [106/172] - Loss: 37.441 [-37.432, 0.009, 0.000]
Epoch 1 [107/172] - Loss: 38.215 [-38.205, 0.009, 0.000]
Epoch 1 [108/172] - Loss: 37.865 [-37.856, 0.009, 0.000]
Epoch 1 [109/172] - Loss: 34.126 [-34.117, 0.009, 0.000]
Epoch 1 [110/172] - Loss: 34.951 [-34.942, 0.009, 0.000]
Epoch 1 [111/172] - Loss: 33.475 [-33.466, 0.009, 0.000]
Epoch 1 [112/172] - Loss: 34.453 [-34.444, 0.009, 0.000]
Epoch 1 [113/172] - Loss: 35.123 [-35.115, 0.009, 0.000]
Epoch 1 [114/172] - Loss: 34.945 [-34.937, 0.009, 0.000]
Epoch 1 [115/172] - Loss: 35.046 [-35.037, 0.009, 0.000]
Epoch 1 [116/172] - Loss: 31.318 [-31.309, 0.009, 0.000]
Epoch 1 [117/172] - Loss: 33.993 [-33.984, 0.009, 0.000]
Epoch 1 [118/172] - Loss: 34.109 [-34.100, 0.009, 0.000]
Epoch 1 [119/172] - Loss: 31.387 [-31.378, 0.009, 0.000]
Epoch 1 [120/172] - Loss: 32.500 [-32.492, 0.009, 0.000]
Epoch 1 [121/172] - Loss: 34.633 [-34.625, 0.009, 0.000]
Epoch 1 [122/172] - Loss: 32.961 [-32.953, 0.009, 0.000]
Epoch 1 [123/172] - Loss: 31.323 [-31.314, 0.009, 0.000]
Epoch 1 [124/172] - Loss: 34.432 [-34.424, 0.008, 0.000]
Epoch 1 [125/172] - Loss: 31.181 [-31.173, 0.008, 0.000]
Epoch 1 [126/172] - Loss: 35.480 [-35.471, 0.008, 0.000]
Epoch 1 [127/172] - Loss: 32.101 [-32.093, 0.008, 0.000]
Epoch 1 [128/172] - Loss: 31.932 [-31.924, 0.008, 0.000]
Epoch 1 [129/172] - Loss: 35.933 [-35.925, 0.008, 0.000]
Epoch 1 [130/172] - Loss: 35.822 [-35.814, 0.008, 0.000]
Epoch 1 [131/172] - Loss: 34.983 [-34.975, 0.008, 0.000]
Epoch 1 [132/172] - Loss: 32.552 [-32.544, 0.008, 0.000]
Epoch 1 [133/172] - Loss: 36.627 [-36.619, 0.008, 0.000]
Epoch 1 [134/172] - Loss: 34.070 [-34.062, 0.008, 0.000]
Epoch 1 [135/172] - Loss: 29.094 [-29.086, 0.008, 0.000]
Epoch 1 [136/172] - Loss: 32.431 [-32.423, 0.008, 0.000]
Epoch 1 [137/172] - Loss: 32.702 [-32.695, 0.008, 0.000]
Epoch 1 [138/172] - Loss: 35.281 [-35.273, 0.008, 0.000]
Epoch 1 [139/172] - Loss: 31.476 [-31.468, 0.008, 0.000]
Epoch 1 [140/172] - Loss: 29.852 [-29.844, 0.008, 0.000]
Epoch 1 [141/172] - Loss: 30.571 [-30.563, 0.008, 0.000]
Epoch 1 [142/172] - Loss: 29.364 [-29.356, 0.008, 0.000]
Epoch 1 [143/172] - Loss: 28.489 [-28.481, 0.008, 0.000]
Epoch 1 [144/172] - Loss: 30.965 [-30.957, 0.008, 0.000]
Epoch 1 [145/172] - Loss: 29.741 [-29.733, 0.008, 0.000]
Epoch 1 [146/172] - Loss: 32.072 [-32.065, 0.008, 0.000]
Epoch 1 [147/172] - Loss: 31.924 [-31.916, 0.008, 0.000]
Epoch 1 [148/172] - Loss: 30.508 [-30.501, 0.008, 0.000]
Epoch 1 [149/172] - Loss: 29.303 [-29.296, 0.008, 0.000]
Epoch 1 [150/172] - Loss: 29.396 [-29.388, 0.008, 0.000]
Epoch 1 [151/172] - Loss: 30.438 [-30.431, 0.008, 0.000]
Epoch 1 [152/172] - Loss: 27.519 [-27.511, 0.008, 0.000]
Epoch 1 [153/172] - Loss: 27.981 [-27.974, 0.008, 0.000]
Epoch 1 [154/172] - Loss: 28.627 [-28.619, 0.008, 0.000]
Epoch 1 [155/172] - Loss: 29.085 [-29.077, 0.008, 0.000]
Epoch 1 [156/172] - Loss: 27.740 [-27.733, 0.008, 0.000]
Epoch 1 [157/172] - Loss: 26.016 [-26.008, 0.008, 0.000]
Epoch 1 [158/172] - Loss: 27.154 [-27.146, 0.008, 0.000]
Epoch 1 [159/172] - Loss: 28.167 [-28.159, 0.008, 0.000]
Epoch 1 [160/172] - Loss: 29.723 [-29.715, 0.008, 0.000]
Epoch 1 [161/172] - Loss: 30.179 [-30.171, 0.008, 0.000]
Epoch 1 [162/172] - Loss: 28.494 [-28.486, 0.008, 0.000]
Epoch 1 [163/172] - Loss: 28.793 [-28.786, 0.008, 0.000]
Epoch 1 [164/172] - Loss: 29.919 [-29.912, 0.007, 0.000]
Epoch 1 [165/172] - Loss: 29.677 [-29.670, 0.007, 0.000]
Epoch 1 [166/172] - Loss: 27.379 [-27.371, 0.007, 0.000]
Epoch 1 [167/172] - Loss: 27.328 [-27.321, 0.007, 0.000]
Epoch 1 [168/172] - Loss: 26.571 [-26.563, 0.007, 0.000]
Epoch 1 [169/172] - Loss: 26.630 [-26.622, 0.007, 0.000]
Epoch 1 [170/172] - Loss: 24.753 [-24.746, 0.007, 0.000]
Epoch 1 [171/172] - Loss: 26.142 [-26.134, 0.007, 0.000]
Epoch 2 [0/172] - Loss: 27.047 [-27.040, 0.007, 0.000]
Epoch 2 [1/172] - Loss: 26.027 [-26.020, 0.007, 0.000]
Epoch 2 [2/172] - Loss: 26.738 [-26.730, 0.007, 0.000]
Epoch 2 [3/172] - Loss: 26.605 [-26.598, 0.007, 0.000]
Epoch 2 [4/172] - Loss: 27.893 [-27.886, 0.007, 0.000]
Epoch 2 [5/172] - Loss: 25.183 [-25.176, 0.007, 0.000]
Epoch 2 [6/172] - Loss: 25.722 [-25.715, 0.007, 0.000]
Epoch 2 [7/172] - Loss: 25.956 [-25.949, 0.007, 0.000]
Epoch 2 [8/172] - Loss: 27.345 [-27.338, 0.007, 0.000]
Epoch 2 [9/172] - Loss: 26.076 [-26.069, 0.007, 0.000]
Epoch 2 [10/172] - Loss: 25.698 [-25.691, 0.007, 0.000]
Epoch 2 [11/172] - Loss: 26.591 [-26.584, 0.007, 0.000]
Epoch 2 [12/172] - Loss: 27.529 [-27.522, 0.007, 0.000]
Epoch 2 [13/172] - Loss: 25.255 [-25.248, 0.007, 0.000]
Epoch 2 [14/172] - Loss: 28.086 [-28.079, 0.007, 0.000]
Epoch 2 [15/172] - Loss: 25.967 [-25.960, 0.007, 0.000]
Epoch 2 [16/172] - Loss: 26.988 [-26.981, 0.007, 0.000]
Epoch 2 [17/172] - Loss: 26.392 [-26.385, 0.007, 0.000]
Epoch 2 [18/172] - Loss: 25.773 [-25.766, 0.007, 0.000]
Epoch 2 [19/172] - Loss: 25.323 [-25.316, 0.007, 0.000]
Epoch 2 [20/172] - Loss: 28.923 [-28.916, 0.007, 0.000]
Epoch 2 [21/172] - Loss: 28.624 [-28.618, 0.007, 0.000]
Epoch 2 [22/172] - Loss: 25.547 [-25.540, 0.007, 0.000]
Epoch 2 [23/172] - Loss: 24.150 [-24.143, 0.007, 0.000]
Epoch 2 [24/172] - Loss: 25.994 [-25.987, 0.007, 0.000]
Epoch 2 [25/172] - Loss: 27.702 [-27.695, 0.007, 0.000]
Epoch 2 [26/172] - Loss: 24.966 [-24.959, 0.007, 0.000]
Epoch 2 [27/172] - Loss: 25.692 [-25.685, 0.007, 0.000]
Epoch 2 [28/172] - Loss: 26.954 [-26.947, 0.007, 0.000]
Epoch 2 [29/172] - Loss: 25.189 [-25.183, 0.007, 0.000]
Epoch 2 [30/172] - Loss: 24.081 [-24.075, 0.007, 0.000]
Epoch 2 [31/172] - Loss: 29.248 [-29.242, 0.007, 0.000]
Epoch 2 [32/172] - Loss: 24.731 [-24.724, 0.007, 0.000]
Epoch 2 [33/172] - Loss: 24.397 [-24.391, 0.007, 0.000]
Epoch 2 [34/172] - Loss: 24.792 [-24.785, 0.007, 0.000]
Epoch 2 [35/172] - Loss: 25.668 [-25.661, 0.007, 0.000]
Epoch 2 [36/172] - Loss: 23.440 [-23.434, 0.007, 0.000]
Epoch 2 [37/172] - Loss: 24.450 [-24.444, 0.007, 0.000]
Epoch 2 [38/172] - Loss: 24.462 [-24.455, 0.007, 0.000]
Epoch 2 [39/172] - Loss: 22.700 [-22.694, 0.007, 0.000]
Epoch 2 [40/172] - Loss: 25.682 [-25.675, 0.007, 0.000]
Epoch 2 [41/172] - Loss: 23.182 [-23.175, 0.007, 0.000]
Epoch 2 [42/172] - Loss: 28.360 [-28.353, 0.007, 0.000]
Epoch 2 [43/172] - Loss: 25.143 [-25.136, 0.007, 0.000]
Epoch 2 [44/172] - Loss: 24.773 [-24.766, 0.007, 0.000]
Epoch 2 [45/172] - Loss: 24.935 [-24.928, 0.007, 0.000]
Epoch 2 [46/172] - Loss: 22.950 [-22.943, 0.007, 0.000]
Epoch 2 [47/172] - Loss: 25.704 [-25.698, 0.007, 0.000]
Epoch 2 [48/172] - Loss: 23.325 [-23.318, 0.007, 0.000]
Epoch 2 [49/172] - Loss: 25.097 [-25.090, 0.007, 0.000]
Epoch 2 [50/172] - Loss: 21.785 [-21.779, 0.007, 0.000]
Epoch 2 [51/172] - Loss: 23.411 [-23.404, 0.007, 0.000]
Epoch 2 [52/172] - Loss: 25.332 [-25.326, 0.007, 0.000]
Epoch 2 [53/172] - Loss: 23.468 [-23.461, 0.007, 0.000]
Epoch 2 [54/172] - Loss: 23.583 [-23.577, 0.007, 0.000]
Epoch 2 [55/172] - Loss: 24.095 [-24.088, 0.007, 0.000]
Epoch 2 [56/172] - Loss: 24.564 [-24.558, 0.006, 0.000]
Epoch 2 [57/172] - Loss: 22.862 [-22.856, 0.006, 0.000]
Epoch 2 [58/172] - Loss: 20.969 [-20.963, 0.006, 0.000]
Epoch 2 [59/172] - Loss: 21.373 [-21.367, 0.006, 0.000]
Epoch 2 [60/172] - Loss: 24.505 [-24.498, 0.006, 0.000]
Epoch 2 [61/172] - Loss: 22.940 [-22.933, 0.006, 0.000]
Epoch 2 [62/172] - Loss: 22.760 [-22.753, 0.006, 0.000]
Epoch 2 [63/172] - Loss: 22.576 [-22.569, 0.006, 0.000]
Epoch 2 [64/172] - Loss: 23.079 [-23.072, 0.006, 0.000]
Epoch 2 [65/172] - Loss: 21.922 [-21.916, 0.006, 0.000]
Epoch 2 [66/172] - Loss: 20.594 [-20.588, 0.006, 0.000]
Epoch 2 [67/172] - Loss: 22.275 [-22.269, 0.006, 0.000]
Epoch 2 [68/172] - Loss: 22.295 [-22.289, 0.006, 0.000]
Epoch 2 [69/172] - Loss: 21.589 [-21.583, 0.006, 0.000]
Epoch 2 [70/172] - Loss: 22.227 [-22.221, 0.006, 0.000]
Epoch 2 [71/172] - Loss: 20.594 [-20.587, 0.006, 0.000]
Epoch 2 [72/172] - Loss: 23.079 [-23.073, 0.006, 0.000]
Epoch 2 [73/172] - Loss: 22.630 [-22.624, 0.006, 0.000]
Epoch 2 [74/172] - Loss: 22.850 [-22.844, 0.006, 0.000]
Epoch 2 [75/172] - Loss: 23.234 [-23.228, 0.006, 0.000]
Epoch 2 [76/172] - Loss: 22.631 [-22.625, 0.006, 0.000]
Epoch 2 [77/172] - Loss: 25.441 [-25.435, 0.006, 0.000]
Epoch 2 [78/172] - Loss: 22.454 [-22.448, 0.006, 0.000]
Epoch 2 [79/172] - Loss: 22.793 [-22.787, 0.006, 0.000]
Epoch 2 [80/172] - Loss: 22.729 [-22.723, 0.006, 0.000]
Epoch 2 [81/172] - Loss: 22.457 [-22.451, 0.006, 0.000]
Epoch 2 [82/172] - Loss: 23.463 [-23.457, 0.006, 0.000]
Epoch 2 [83/172] - Loss: 22.269 [-22.263, 0.006, 0.000]
Epoch 2 [84/172] - Loss: 21.279 [-21.273, 0.006, 0.000]
Epoch 2 [85/172] - Loss: 22.317 [-22.311, 0.006, 0.000]
Epoch 2 [86/172] - Loss: 21.855 [-21.848, 0.006, 0.000]
Epoch 2 [87/172] - Loss: 21.216 [-21.210, 0.006, 0.000]
Epoch 2 [88/172] - Loss: 22.743 [-22.737, 0.006, 0.000]
Epoch 2 [89/172] - Loss: 20.002 [-19.996, 0.006, 0.000]
Epoch 2 [90/172] - Loss: 22.237 [-22.231, 0.006, 0.000]
Epoch 2 [91/172] - Loss: 19.951 [-19.945, 0.006, 0.000]
Epoch 2 [92/172] - Loss: 20.837 [-20.831, 0.006, 0.000]
Epoch 2 [93/172] - Loss: 24.461 [-24.455, 0.006, 0.000]
Epoch 2 [94/172] - Loss: 19.335 [-19.329, 0.006, 0.000]
Epoch 2 [95/172] - Loss: 20.455 [-20.449, 0.006, 0.000]
Epoch 2 [96/172] - Loss: 20.722 [-20.716, 0.006, 0.000]
Epoch 2 [97/172] - Loss: 20.442 [-20.436, 0.006, 0.000]
Epoch 2 [98/172] - Loss: 19.421 [-19.415, 0.006, 0.000]
Epoch 2 [99/172] - Loss: 19.933 [-19.927, 0.006, 0.000]
Epoch 2 [100/172] - Loss: 20.284 [-20.278, 0.006, 0.000]
Epoch 2 [101/172] - Loss: 20.610 [-20.604, 0.006, 0.000]
Epoch 2 [102/172] - Loss: 19.757 [-19.751, 0.006, 0.000]
Epoch 2 [103/172] - Loss: 24.770 [-24.764, 0.006, 0.000]
Epoch 2 [104/172] - Loss: 21.265 [-21.259, 0.006, 0.000]
Epoch 2 [105/172] - Loss: 21.869 [-21.863, 0.006, 0.000]
Epoch 2 [106/172] - Loss: 21.135 [-21.129, 0.006, 0.000]
Epoch 2 [107/172] - Loss: 19.652 [-19.646, 0.006, 0.000]
Epoch 2 [108/172] - Loss: 22.278 [-22.272, 0.006, 0.000]
Epoch 2 [109/172] - Loss: 19.185 [-19.179, 0.006, 0.000]
Epoch 2 [110/172] - Loss: 21.195 [-21.189, 0.006, 0.000]
Epoch 2 [111/172] - Loss: 22.765 [-22.759, 0.006, 0.000]
Epoch 2 [112/172] - Loss: 23.563 [-23.557, 0.006, 0.000]
Epoch 2 [113/172] - Loss: 23.492 [-23.486, 0.006, 0.000]
Epoch 2 [114/172] - Loss: 29.524 [-29.518, 0.006, 0.000]
Epoch 2 [115/172] - Loss: 27.156 [-27.150, 0.006, 0.000]
Epoch 2 [116/172] - Loss: 24.903 [-24.897, 0.006, 0.000]
Epoch 2 [117/172] - Loss: 25.934 [-25.928, 0.006, 0.000]
Epoch 2 [118/172] - Loss: 23.794 [-23.788, 0.006, 0.000]
Epoch 2 [119/172] - Loss: 22.374 [-22.369, 0.006, 0.000]
Epoch 2 [120/172] - Loss: 21.478 [-21.472, 0.006, 0.000]
Epoch 2 [121/172] - Loss: 23.689 [-23.683, 0.006, 0.000]
Epoch 2 [122/172] - Loss: 21.226 [-21.220, 0.006, 0.000]
Epoch 2 [123/172] - Loss: 24.248 [-24.242, 0.006, 0.000]
Epoch 2 [124/172] - Loss: 20.942 [-20.936, 0.006, 0.000]
Epoch 2 [125/172] - Loss: 24.795 [-24.789, 0.006, 0.000]
Epoch 2 [126/172] - Loss: 19.729 [-19.723, 0.006, 0.000]
Epoch 2 [127/172] - Loss: 23.454 [-23.448, 0.006, 0.000]
Epoch 2 [128/172] - Loss: 21.089 [-21.084, 0.006, 0.000]
Epoch 2 [129/172] - Loss: 21.803 [-21.797, 0.006, 0.000]
Epoch 2 [130/172] - Loss: 21.630 [-21.624, 0.006, 0.000]
Epoch 2 [131/172] - Loss: 21.022 [-21.016, 0.006, 0.000]
Epoch 2 [132/172] - Loss: 19.342 [-19.336, 0.006, 0.000]
Epoch 2 [133/172] - Loss: 19.359 [-19.353, 0.006, 0.000]
Epoch 2 [134/172] - Loss: 20.994 [-20.988, 0.006, 0.000]
Epoch 2 [135/172] - Loss: 20.218 [-20.213, 0.006, 0.000]
Epoch 2 [136/172] - Loss: 19.204 [-19.198, 0.006, 0.000]
Epoch 2 [137/172] - Loss: 19.247 [-19.242, 0.006, 0.000]
Epoch 2 [138/172] - Loss: 20.682 [-20.676, 0.006, 0.000]
Epoch 2 [139/172] - Loss: 18.650 [-18.644, 0.006, 0.000]
Epoch 2 [140/172] - Loss: 20.235 [-20.230, 0.006, 0.000]
Epoch 2 [141/172] - Loss: 17.869 [-17.864, 0.006, 0.000]
Epoch 2 [142/172] - Loss: 19.388 [-19.382, 0.006, 0.000]
Epoch 2 [143/172] - Loss: 18.812 [-18.806, 0.006, 0.000]
Epoch 2 [144/172] - Loss: 20.186 [-20.180, 0.006, 0.000]
Epoch 2 [145/172] - Loss: 19.438 [-19.432, 0.006, 0.000]
Epoch 2 [146/172] - Loss: 19.803 [-19.797, 0.006, 0.000]
Epoch 2 [147/172] - Loss: 21.874 [-21.868, 0.006, 0.000]
Epoch 2 [148/172] - Loss: 19.743 [-19.737, 0.006, 0.000]
Epoch 2 [149/172] - Loss: 18.784 [-18.779, 0.006, 0.000]
Epoch 2 [150/172] - Loss: 20.092 [-20.087, 0.006, 0.000]
Epoch 2 [151/172] - Loss: 21.202 [-21.196, 0.006, 0.000]
Epoch 2 [152/172] - Loss: 21.936 [-21.930, 0.006, 0.000]
Epoch 2 [153/172] - Loss: 19.533 [-19.527, 0.006, 0.000]
Epoch 2 [154/172] - Loss: 20.077 [-20.071, 0.006, 0.000]
Epoch 2 [155/172] - Loss: 20.510 [-20.504, 0.006, 0.000]
Epoch 2 [156/172] - Loss: 20.769 [-20.763, 0.006, 0.000]
Epoch 2 [157/172] - Loss: 17.613 [-17.607, 0.006, 0.000]
Epoch 2 [158/172] - Loss: 18.276 [-18.271, 0.006, 0.000]
Epoch 2 [159/172] - Loss: 20.414 [-20.408, 0.006, 0.000]
Epoch 2 [160/172] - Loss: 21.771 [-21.765, 0.006, 0.000]
Epoch 2 [161/172] - Loss: 19.757 [-19.751, 0.006, 0.000]
Epoch 2 [162/172] - Loss: 20.286 [-20.280, 0.006, 0.000]
Epoch 2 [163/172] - Loss: 18.707 [-18.701, 0.006, 0.000]
Epoch 2 [164/172] - Loss: 17.722 [-17.716, 0.006, 0.000]
Epoch 2 [165/172] - Loss: 19.088 [-19.083, 0.006, 0.000]
Epoch 2 [166/172] - Loss: 19.906 [-19.900, 0.006, 0.000]
Epoch 2 [167/172] - Loss: 19.175 [-19.169, 0.006, 0.000]
Epoch 2 [168/172] - Loss: 19.188 [-19.183, 0.006, 0.000]
Epoch 2 [169/172] - Loss: 17.643 [-17.638, 0.006, 0.000]
Epoch 2 [170/172] - Loss: 18.700 [-18.694, 0.006, 0.000]
Epoch 2 [171/172] - Loss: 19.076 [-19.070, 0.006, 0.000]
Epoch 3 [0/172] - Loss: 18.389 [-18.383, 0.006, 0.000]
Epoch 3 [1/172] - Loss: 19.460 [-19.454, 0.006, 0.000]
Epoch 3 [2/172] - Loss: 20.014 [-20.009, 0.006, 0.000]
Epoch 3 [3/172] - Loss: 17.737 [-17.732, 0.006, 0.000]
Epoch 3 [4/172] - Loss: 18.653 [-18.647, 0.006, 0.000]
Epoch 3 [5/172] - Loss: 19.476 [-19.471, 0.006, 0.000]
Epoch 3 [6/172] - Loss: 19.264 [-19.258, 0.006, 0.000]
Epoch 3 [7/172] - Loss: 19.647 [-19.641, 0.006, 0.000]
Epoch 3 [8/172] - Loss: 18.267 [-18.261, 0.006, 0.000]
Epoch 3 [9/172] - Loss: 20.326 [-20.321, 0.006, 0.000]
Epoch 3 [10/172] - Loss: 19.299 [-19.294, 0.006, 0.000]
Epoch 3 [11/172] - Loss: 20.284 [-20.278, 0.006, 0.000]
Epoch 3 [12/172] - Loss: 18.636 [-18.630, 0.006, 0.000]
Epoch 3 [13/172] - Loss: 19.779 [-19.773, 0.006, 0.000]
Epoch 3 [14/172] - Loss: 19.013 [-19.007, 0.006, 0.000]
Epoch 3 [15/172] - Loss: 18.195 [-18.190, 0.006, 0.000]
Epoch 3 [16/172] - Loss: 19.122 [-19.116, 0.006, 0.000]
Epoch 3 [17/172] - Loss: 19.706 [-19.700, 0.006, 0.000]
Epoch 3 [18/172] - Loss: 20.487 [-20.482, 0.006, 0.000]
Epoch 3 [19/172] - Loss: 19.363 [-19.358, 0.006, 0.000]
Epoch 3 [20/172] - Loss: 18.510 [-18.505, 0.006, 0.000]
Epoch 3 [21/172] - Loss: 18.427 [-18.421, 0.006, 0.000]
Epoch 3 [22/172] - Loss: 17.943 [-17.937, 0.006, 0.000]
Epoch 3 [23/172] - Loss: 17.806 [-17.800, 0.006, 0.000]
Epoch 3 [24/172] - Loss: 19.276 [-19.270, 0.006, 0.000]
Epoch 3 [25/172] - Loss: 18.677 [-18.671, 0.006, 0.000]
Epoch 3 [26/172] - Loss: 19.329 [-19.324, 0.005, 0.000]
Epoch 3 [27/172] - Loss: 19.195 [-19.189, 0.005, 0.000]
Epoch 3 [28/172] - Loss: 20.030 [-20.024, 0.006, 0.000]
Epoch 3 [29/172] - Loss: 18.287 [-18.281, 0.005, 0.000]
Epoch 3 [30/172] - Loss: 17.153 [-17.147, 0.005, 0.000]
Epoch 3 [31/172] - Loss: 18.750 [-18.744, 0.005, 0.000]
Epoch 3 [32/172] - Loss: 17.995 [-17.990, 0.006, 0.000]
Epoch 3 [33/172] - Loss: 17.200 [-17.194, 0.006, 0.000]
Epoch 3 [34/172] - Loss: 18.743 [-18.737, 0.006, 0.000]
Epoch 3 [35/172] - Loss: 18.838 [-18.833, 0.006, 0.000]
Epoch 3 [36/172] - Loss: 17.860 [-17.854, 0.006, 0.000]
Epoch 3 [37/172] - Loss: 19.799 [-19.793, 0.006, 0.000]
Epoch 3 [38/172] - Loss: 18.889 [-18.883, 0.006, 0.000]
Epoch 3 [39/172] - Loss: 18.708 [-18.703, 0.005, 0.000]
Epoch 3 [40/172] - Loss: 18.257 [-18.252, 0.005, 0.000]
Epoch 3 [41/172] - Loss: 19.569 [-19.563, 0.005, 0.000]
Epoch 3 [42/172] - Loss: 20.357 [-20.352, 0.005, 0.000]
Epoch 3 [43/172] - Loss: 17.849 [-17.844, 0.005, 0.000]
Epoch 3 [44/172] - Loss: 17.954 [-17.948, 0.005, 0.000]
Epoch 3 [45/172] - Loss: 17.766 [-17.760, 0.005, 0.000]
Epoch 3 [46/172] - Loss: 18.178 [-18.173, 0.005, 0.000]
Epoch 3 [47/172] - Loss: 17.936 [-17.931, 0.005, 0.000]
Epoch 3 [48/172] - Loss: 16.954 [-16.949, 0.005, 0.000]
Epoch 3 [49/172] - Loss: 17.951 [-17.945, 0.005, 0.000]
Epoch 3 [50/172] - Loss: 16.385 [-16.380, 0.005, 0.000]
Epoch 3 [51/172] - Loss: 19.350 [-19.344, 0.005, 0.000]
Epoch 3 [52/172] - Loss: 18.870 [-18.864, 0.005, 0.000]
Epoch 3 [53/172] - Loss: 17.491 [-17.485, 0.005, 0.000]
Epoch 3 [54/172] - Loss: 19.887 [-19.881, 0.005, 0.000]
Epoch 3 [55/172] - Loss: 18.081 [-18.076, 0.005, 0.000]
Epoch 3 [56/172] - Loss: 19.547 [-19.542, 0.005, 0.000]
Epoch 3 [57/172] - Loss: 17.707 [-17.701, 0.005, 0.000]
Epoch 3 [58/172] - Loss: 18.495 [-18.489, 0.005, 0.000]
Epoch 3 [59/172] - Loss: 18.207 [-18.202, 0.005, 0.000]
Epoch 3 [60/172] - Loss: 17.620 [-17.614, 0.005, 0.000]
Epoch 3 [61/172] - Loss: 17.275 [-17.270, 0.005, 0.000]
Epoch 3 [62/172] - Loss: 18.588 [-18.582, 0.005, 0.000]
Epoch 3 [63/172] - Loss: 18.558 [-18.553, 0.005, 0.000]
Epoch 3 [64/172] - Loss: 18.692 [-18.686, 0.005, 0.000]
Epoch 3 [65/172] - Loss: 17.892 [-17.887, 0.005, 0.000]
Epoch 3 [66/172] - Loss: 18.145 [-18.140, 0.005, 0.000]
Epoch 3 [67/172] - Loss: 16.896 [-16.890, 0.005, 0.000]
Epoch 3 [68/172] - Loss: 18.442 [-18.436, 0.005, 0.000]
Epoch 3 [69/172] - Loss: 16.822 [-16.817, 0.005, 0.000]
Epoch 3 [70/172] - Loss: 16.627 [-16.622, 0.005, 0.000]
Epoch 3 [71/172] - Loss: 18.848 [-18.842, 0.005, 0.000]
Epoch 3 [72/172] - Loss: 16.431 [-16.426, 0.005, 0.000]
Epoch 3 [73/172] - Loss: 18.501 [-18.496, 0.005, 0.000]
Epoch 3 [74/172] - Loss: 17.351 [-17.346, 0.005, 0.000]
Epoch 3 [75/172] - Loss: 17.214 [-17.209, 0.005, 0.000]
Epoch 3 [76/172] - Loss: 18.722 [-18.717, 0.005, 0.000]
Epoch 3 [77/172] - Loss: 18.774 [-18.769, 0.005, 0.000]
Epoch 3 [78/172] - Loss: 16.797 [-16.792, 0.005, 0.000]
Epoch 3 [79/172] - Loss: 17.152 [-17.147, 0.005, 0.000]
Epoch 3 [80/172] - Loss: 16.451 [-16.445, 0.005, 0.000]
Epoch 3 [81/172] - Loss: 17.808 [-17.803, 0.005, 0.000]
Epoch 3 [82/172] - Loss: 16.825 [-16.820, 0.005, 0.000]
Epoch 3 [83/172] - Loss: 19.027 [-19.022, 0.005, 0.000]
Epoch 3 [84/172] - Loss: 19.477 [-19.472, 0.005, 0.000]
Epoch 3 [85/172] - Loss: 19.719 [-19.713, 0.005, 0.000]
Epoch 3 [86/172] - Loss: 19.490 [-19.484, 0.005, 0.000]
Epoch 3 [87/172] - Loss: 19.093 [-19.087, 0.005, 0.000]
Epoch 3 [88/172] - Loss: 19.149 [-19.144, 0.005, 0.000]
Epoch 3 [89/172] - Loss: 19.512 [-19.507, 0.005, 0.000]
Epoch 3 [90/172] - Loss: 18.738 [-18.732, 0.005, 0.000]
Epoch 3 [91/172] - Loss: 18.866 [-18.860, 0.005, 0.000]
Epoch 3 [92/172] - Loss: 18.126 [-18.120, 0.005, 0.000]
Epoch 3 [93/172] - Loss: 17.196 [-17.191, 0.005, 0.000]
Epoch 3 [94/172] - Loss: 17.871 [-17.865, 0.005, 0.000]
Epoch 3 [95/172] - Loss: 16.200 [-16.194, 0.005, 0.000]
Epoch 3 [96/172] - Loss: 18.464 [-18.458, 0.005, 0.000]
Epoch 3 [97/172] - Loss: 16.661 [-16.655, 0.005, 0.000]
Epoch 3 [98/172] - Loss: 17.575 [-17.570, 0.005, 0.000]
Epoch 3 [99/172] - Loss: 17.622 [-17.617, 0.005, 0.000]
Epoch 3 [100/172] - Loss: 18.544 [-18.539, 0.005, 0.000]
Epoch 3 [101/172] - Loss: 18.757 [-18.751, 0.005, 0.000]
Epoch 3 [102/172] - Loss: 17.213 [-17.208, 0.005, 0.000]
Epoch 3 [103/172] - Loss: 17.495 [-17.490, 0.005, 0.000]
Epoch 3 [104/172] - Loss: 17.658 [-17.653, 0.005, 0.000]
Epoch 3 [105/172] - Loss: 17.331 [-17.325, 0.005, 0.000]
Epoch 3 [106/172] - Loss: 17.369 [-17.364, 0.005, 0.000]
Epoch 3 [107/172] - Loss: 16.952 [-16.947, 0.005, 0.000]
Epoch 3 [108/172] - Loss: 17.033 [-17.027, 0.005, 0.000]
Epoch 3 [109/172] - Loss: 17.710 [-17.704, 0.005, 0.000]
Epoch 3 [110/172] - Loss: 16.958 [-16.953, 0.005, 0.000]
Epoch 3 [111/172] - Loss: 16.872 [-16.866, 0.005, 0.000]
Epoch 3 [112/172] - Loss: 17.300 [-17.295, 0.005, 0.000]
Epoch 3 [113/172] - Loss: 17.915 [-17.910, 0.005, 0.000]
Epoch 3 [114/172] - Loss: 17.622 [-17.617, 0.005, 0.000]
Epoch 3 [115/172] - Loss: 17.768 [-17.763, 0.005, 0.000]
Epoch 3 [116/172] - Loss: 18.476 [-18.471, 0.005, 0.000]
Epoch 3 [117/172] - Loss: 18.819 [-18.814, 0.005, 0.000]
Epoch 3 [118/172] - Loss: 17.615 [-17.610, 0.005, 0.000]
Epoch 3 [119/172] - Loss: 17.313 [-17.307, 0.005, 0.000]
Epoch 3 [120/172] - Loss: 18.919 [-18.914, 0.005, 0.000]
Epoch 3 [121/172] - Loss: 16.251 [-16.246, 0.005, 0.000]
Epoch 3 [122/172] - Loss: 18.292 [-18.287, 0.005, 0.000]
Epoch 3 [123/172] - Loss: 17.875 [-17.870, 0.005, 0.000]
Epoch 3 [124/172] - Loss: 17.943 [-17.938, 0.005, 0.000]
Epoch 3 [125/172] - Loss: 16.229 [-16.223, 0.005, 0.000]
Epoch 3 [126/172] - Loss: 17.597 [-17.592, 0.005, 0.000]
Epoch 3 [127/172] - Loss: 16.388 [-16.383, 0.005, 0.000]
Epoch 3 [128/172] - Loss: 17.271 [-17.265, 0.005, 0.000]
Epoch 3 [129/172] - Loss: 17.930 [-17.925, 0.005, 0.000]
Epoch 3 [130/172] - Loss: 19.158 [-19.153, 0.005, 0.000]
Epoch 3 [131/172] - Loss: 17.759 [-17.754, 0.005, 0.000]
Epoch 3 [132/172] - Loss: 19.288 [-19.283, 0.005, 0.000]
Epoch 3 [133/172] - Loss: 20.205 [-20.200, 0.005, 0.000]
Epoch 3 [134/172] - Loss: 18.289 [-18.284, 0.005, 0.000]
Epoch 3 [135/172] - Loss: 17.201 [-17.196, 0.005, 0.000]
Epoch 3 [136/172] - Loss: 17.038 [-17.032, 0.005, 0.000]
Epoch 3 [137/172] - Loss: 20.162 [-20.156, 0.005, 0.000]
Epoch 3 [138/172] - Loss: 20.328 [-20.323, 0.005, 0.000]
Epoch 3 [139/172] - Loss: 19.472 [-19.467, 0.005, 0.000]
Epoch 3 [140/172] - Loss: 18.155 [-18.150, 0.005, 0.000]
Epoch 3 [141/172] - Loss: 17.928 [-17.923, 0.005, 0.000]
Epoch 3 [142/172] - Loss: 17.010 [-17.005, 0.005, 0.000]
Epoch 3 [143/172] - Loss: 16.708 [-16.703, 0.005, 0.000]
Epoch 3 [144/172] - Loss: 17.418 [-17.413, 0.005, 0.000]
Epoch 3 [145/172] - Loss: 16.589 [-16.584, 0.005, 0.000]
Epoch 3 [146/172] - Loss: 16.570 [-16.565, 0.005, 0.000]
Epoch 3 [147/172] - Loss: 17.861 [-17.856, 0.005, 0.000]
Epoch 3 [148/172] - Loss: 17.947 [-17.942, 0.005, 0.000]
Epoch 3 [149/172] - Loss: 16.184 [-16.178, 0.005, 0.000]
Epoch 3 [150/172] - Loss: 17.241 [-17.236, 0.005, 0.000]
Epoch 3 [151/172] - Loss: 16.728 [-16.723, 0.005, 0.000]
Epoch 3 [152/172] - Loss: 15.393 [-15.388, 0.005, 0.000]
Epoch 3 [153/172] - Loss: 16.113 [-16.108, 0.005, 0.000]
Epoch 3 [154/172] - Loss: 15.992 [-15.987, 0.005, 0.000]
Epoch 3 [155/172] - Loss: 16.245 [-16.240, 0.005, 0.000]
Epoch 3 [156/172] - Loss: 15.961 [-15.956, 0.005, 0.000]
Epoch 3 [157/172] - Loss: 16.330 [-16.325, 0.005, 0.000]
Epoch 3 [158/172] - Loss: 16.138 [-16.133, 0.005, 0.000]
Epoch 3 [159/172] - Loss: 16.700 [-16.695, 0.005, 0.000]
Epoch 3 [160/172] - Loss: 15.248 [-15.243, 0.005, 0.000]
Epoch 3 [161/172] - Loss: 17.414 [-17.409, 0.005, 0.000]
Epoch 3 [162/172] - Loss: 15.587 [-15.582, 0.005, 0.000]
Epoch 3 [163/172] - Loss: 16.390 [-16.385, 0.005, 0.000]
Epoch 3 [164/172] - Loss: 16.359 [-16.354, 0.005, 0.000]
Epoch 3 [165/172] - Loss: 16.076 [-16.071, 0.005, 0.000]
Epoch 3 [166/172] - Loss: 16.197 [-16.192, 0.005, 0.000]
Epoch 3 [167/172] - Loss: 16.925 [-16.920, 0.005, 0.000]
Epoch 3 [168/172] - Loss: 16.311 [-16.306, 0.005, 0.000]
Epoch 3 [169/172] - Loss: 17.644 [-17.639, 0.005, 0.000]
Epoch 3 [170/172] - Loss: 16.767 [-16.762, 0.005, 0.000]
Epoch 3 [171/172] - Loss: 16.057 [-16.052, 0.005, 0.000]
Epoch 4 [0/172] - Loss: 19.838 [-19.833, 0.005, 0.000]
Epoch 4 [1/172] - Loss: 19.956 [-19.951, 0.005, 0.000]
Epoch 4 [2/172] - Loss: 18.349 [-18.344, 0.005, 0.000]
Epoch 4 [3/172] - Loss: 18.129 [-18.124, 0.005, 0.000]
Epoch 4 [4/172] - Loss: 16.619 [-16.614, 0.005, 0.000]
Epoch 4 [5/172] - Loss: 16.600 [-16.595, 0.005, 0.000]
Epoch 4 [6/172] - Loss: 17.351 [-17.346, 0.005, 0.000]
Epoch 4 [7/172] - Loss: 17.683 [-17.678, 0.005, 0.000]
Epoch 4 [8/172] - Loss: 16.586 [-16.581, 0.005, 0.000]
Epoch 4 [9/172] - Loss: 16.137 [-16.132, 0.005, 0.000]
Epoch 4 [10/172] - Loss: 16.576 [-16.571, 0.005, 0.000]
Epoch 4 [11/172] - Loss: 17.940 [-17.935, 0.005, 0.000]
Epoch 4 [12/172] - Loss: 15.691 [-15.686, 0.005, 0.000]
Epoch 4 [13/172] - Loss: 17.654 [-17.649, 0.005, 0.000]
Epoch 4 [14/172] - Loss: 16.932 [-16.927, 0.005, 0.000]
Epoch 4 [15/172] - Loss: 15.200 [-15.195, 0.005, 0.000]
Epoch 4 [16/172] - Loss: 15.217 [-15.212, 0.005, 0.000]
Epoch 4 [17/172] - Loss: 15.572 [-15.567, 0.005, 0.000]
Epoch 4 [18/172] - Loss: 15.659 [-15.654, 0.005, 0.000]
Epoch 4 [19/172] - Loss: 14.812 [-14.807, 0.005, 0.000]
Epoch 4 [20/172] - Loss: 15.999 [-15.994, 0.005, 0.000]
Epoch 4 [21/172] - Loss: 16.943 [-16.938, 0.005, 0.000]
Epoch 4 [22/172] - Loss: 15.436 [-15.431, 0.005, 0.000]
Epoch 4 [23/172] - Loss: 15.697 [-15.692, 0.005, 0.000]
Epoch 4 [24/172] - Loss: 16.396 [-16.391, 0.005, 0.000]
Epoch 4 [25/172] - Loss: 16.207 [-16.202, 0.005, 0.000]
Epoch 4 [26/172] - Loss: 17.183 [-17.178, 0.005, 0.000]
Epoch 4 [27/172] - Loss: 16.119 [-16.114, 0.005, 0.000]
Epoch 4 [28/172] - Loss: 14.882 [-14.877, 0.005, 0.000]
Epoch 4 [29/172] - Loss: 15.067 [-15.061, 0.005, 0.000]
Epoch 4 [30/172] - Loss: 15.462 [-15.457, 0.005, 0.000]
Epoch 4 [31/172] - Loss: 16.324 [-16.319, 0.005, 0.000]
Epoch 4 [32/172] - Loss: 15.489 [-15.484, 0.005, 0.000]
Epoch 4 [33/172] - Loss: 15.253 [-15.247, 0.005, 0.000]
Epoch 4 [34/172] - Loss: 15.481 [-15.476, 0.005, 0.000]
Epoch 4 [35/172] - Loss: 15.229 [-15.224, 0.005, 0.000]
Epoch 4 [36/172] - Loss: 15.464 [-15.459, 0.005, 0.000]
Epoch 4 [37/172] - Loss: 15.162 [-15.157, 0.005, 0.000]
Epoch 4 [38/172] - Loss: 14.898 [-14.893, 0.005, 0.000]
Epoch 4 [39/172] - Loss: 15.417 [-15.412, 0.005, 0.000]
Epoch 4 [40/172] - Loss: 15.745 [-15.740, 0.005, 0.000]
Epoch 4 [41/172] - Loss: 14.932 [-14.927, 0.005, 0.000]
Epoch 4 [42/172] - Loss: 15.387 [-15.382, 0.005, 0.000]
Epoch 4 [43/172] - Loss: 15.090 [-15.085, 0.005, 0.000]
Epoch 4 [44/172] - Loss: 14.678 [-14.673, 0.005, 0.000]
Epoch 4 [45/172] - Loss: 14.912 [-14.907, 0.005, 0.000]
Epoch 4 [46/172] - Loss: 15.643 [-15.638, 0.005, 0.000]
Epoch 4 [47/172] - Loss: 15.523 [-15.518, 0.005, 0.000]
Epoch 4 [48/172] - Loss: 15.582 [-15.577, 0.005, 0.000]
Epoch 4 [49/172] - Loss: 14.934 [-14.929, 0.005, 0.000]
Epoch 4 [50/172] - Loss: 15.005 [-15.000, 0.005, 0.000]
Epoch 4 [51/172] - Loss: 14.667 [-14.662, 0.005, 0.000]
Epoch 4 [52/172] - Loss: 15.148 [-15.143, 0.005, 0.000]
Epoch 4 [53/172] - Loss: 15.834 [-15.829, 0.005, 0.000]
Epoch 4 [54/172] - Loss: 15.054 [-15.049, 0.005, 0.000]
Epoch 4 [55/172] - Loss: 14.907 [-14.902, 0.005, 0.000]
Epoch 4 [56/172] - Loss: 15.882 [-15.877, 0.005, 0.000]
Epoch 4 [57/172] - Loss: 14.631 [-14.626, 0.005, 0.000]
Epoch 4 [58/172] - Loss: 14.201 [-14.196, 0.005, 0.000]
Epoch 4 [59/172] - Loss: 16.303 [-16.298, 0.005, 0.000]
Epoch 4 [60/172] - Loss: 15.058 [-15.053, 0.005, 0.000]
Epoch 4 [61/172] - Loss: 15.470 [-15.465, 0.005, 0.000]
Epoch 4 [62/172] - Loss: 15.801 [-15.796, 0.005, 0.000]
Epoch 4 [63/172] - Loss: 15.337 [-15.332, 0.005, 0.000]
Epoch 4 [64/172] - Loss: 16.658 [-16.652, 0.005, 0.000]
Epoch 4 [65/172] - Loss: 16.369 [-16.364, 0.005, 0.000]
Epoch 4 [66/172] - Loss: 14.620 [-14.615, 0.005, 0.000]
Epoch 4 [67/172] - Loss: 15.195 [-15.190, 0.005, 0.000]
Epoch 4 [68/172] - Loss: 14.849 [-14.844, 0.005, 0.000]
Epoch 4 [69/172] - Loss: 13.902 [-13.897, 0.005, 0.000]
Epoch 4 [70/172] - Loss: 15.349 [-15.344, 0.005, 0.000]
Epoch 4 [71/172] - Loss: 16.424 [-16.419, 0.005, 0.000]
Epoch 4 [72/172] - Loss: 14.878 [-14.873, 0.005, 0.000]
Epoch 4 [73/172] - Loss: 16.017 [-16.012, 0.005, 0.000]
Epoch 4 [74/172] - Loss: 14.951 [-14.946, 0.005, 0.000]
Epoch 4 [75/172] - Loss: 15.057 [-15.052, 0.005, 0.000]
Epoch 4 [76/172] - Loss: 15.156 [-15.151, 0.005, 0.000]
Epoch 4 [77/172] - Loss: 15.764 [-15.759, 0.005, 0.000]
Epoch 4 [78/172] - Loss: 14.770 [-14.765, 0.005, 0.000]
Epoch 4 [79/172] - Loss: 14.672 [-14.667, 0.005, 0.000]
Epoch 4 [80/172] - Loss: 14.766 [-14.761, 0.005, 0.000]
Epoch 4 [81/172] - Loss: 14.994 [-14.989, 0.005, 0.000]
Epoch 4 [82/172] - Loss: 14.294 [-14.289, 0.005, 0.000]
Epoch 4 [83/172] - Loss: 15.211 [-15.206, 0.005, 0.000]
Epoch 4 [84/172] - Loss: 14.892 [-14.887, 0.005, 0.000]
Epoch 4 [85/172] - Loss: 15.003 [-14.998, 0.005, 0.000]
Epoch 4 [86/172] - Loss: 14.874 [-14.869, 0.005, 0.000]
Epoch 4 [87/172] - Loss: 15.171 [-15.166, 0.005, 0.000]
Epoch 4 [88/172] - Loss: 16.078 [-16.073, 0.005, 0.000]
Epoch 4 [89/172] - Loss: 15.450 [-15.445, 0.005, 0.000]
Epoch 4 [90/172] - Loss: 15.208 [-15.203, 0.005, 0.000]
Epoch 4 [91/172] - Loss: 15.636 [-15.631, 0.005, 0.000]
Epoch 4 [92/172] - Loss: 15.270 [-15.265, 0.005, 0.000]
Epoch 4 [93/172] - Loss: 14.629 [-14.624, 0.005, 0.000]
Epoch 4 [94/172] - Loss: 15.901 [-15.895, 0.005, 0.000]
Epoch 4 [95/172] - Loss: 15.852 [-15.847, 0.005, 0.000]
Epoch 4 [96/172] - Loss: 15.550 [-15.545, 0.005, 0.000]
Epoch 4 [97/172] - Loss: 15.935 [-15.930, 0.005, 0.000]
Epoch 4 [98/172] - Loss: 15.955 [-15.950, 0.005, 0.000]
Epoch 4 [99/172] - Loss: 14.376 [-14.371, 0.005, 0.000]
Epoch 4 [100/172] - Loss: 16.127 [-16.122, 0.005, 0.000]
Epoch 4 [101/172] - Loss: 15.998 [-15.993, 0.005, 0.000]
Epoch 4 [102/172] - Loss: 15.518 [-15.513, 0.005, 0.000]
Epoch 4 [103/172] - Loss: 15.071 [-15.065, 0.005, 0.000]
Epoch 4 [104/172] - Loss: 15.190 [-15.185, 0.005, 0.000]
Epoch 4 [105/172] - Loss: 14.876 [-14.871, 0.005, 0.000]
Epoch 4 [106/172] - Loss: 15.732 [-15.727, 0.005, 0.000]
Epoch 4 [107/172] - Loss: 14.226 [-14.221, 0.005, 0.000]
Epoch 4 [108/172] - Loss: 15.267 [-15.262, 0.005, 0.000]
Epoch 4 [109/172] - Loss: 15.066 [-15.061, 0.005, 0.000]
Epoch 4 [110/172] - Loss: 15.194 [-15.189, 0.005, 0.000]
Epoch 4 [111/172] - Loss: 15.331 [-15.326, 0.005, 0.000]
Epoch 4 [112/172] - Loss: 15.691 [-15.686, 0.005, 0.000]
Epoch 4 [113/172] - Loss: 15.022 [-15.017, 0.005, 0.000]
Epoch 4 [114/172] - Loss: 14.585 [-14.580, 0.005, 0.000]
Epoch 4 [115/172] - Loss: 15.560 [-15.555, 0.005, 0.000]
Epoch 4 [116/172] - Loss: 14.763 [-14.758, 0.005, 0.000]
Epoch 4 [117/172] - Loss: 14.615 [-14.610, 0.005, 0.000]
Epoch 4 [118/172] - Loss: 15.928 [-15.923, 0.005, 0.000]
Epoch 4 [119/172] - Loss: 15.635 [-15.630, 0.005, 0.000]
Epoch 4 [120/172] - Loss: 14.294 [-14.289, 0.005, 0.000]
Epoch 4 [121/172] - Loss: 14.525 [-14.520, 0.005, 0.000]
Epoch 4 [122/172] - Loss: 14.800 [-14.795, 0.005, 0.000]
Epoch 4 [123/172] - Loss: 15.532 [-15.527, 0.005, 0.000]
Epoch 4 [124/172] - Loss: 16.101 [-16.096, 0.005, 0.000]
Epoch 4 [125/172] - Loss: 15.111 [-15.106, 0.005, 0.000]
Epoch 4 [126/172] - Loss: 16.013 [-16.008, 0.005, 0.000]
Epoch 4 [127/172] - Loss: 15.485 [-15.480, 0.005, 0.000]
Epoch 4 [128/172] - Loss: 14.731 [-14.726, 0.005, 0.000]
Epoch 4 [129/172] - Loss: 15.976 [-15.971, 0.005, 0.000]
Epoch 4 [130/172] - Loss: 15.369 [-15.364, 0.005, 0.000]
Epoch 4 [131/172] - Loss: 14.908 [-14.903, 0.005, 0.000]
Epoch 4 [132/172] - Loss: 15.311 [-15.306, 0.005, 0.000]
Epoch 4 [133/172] - Loss: 15.947 [-15.942, 0.005, 0.000]
Epoch 4 [134/172] - Loss: 15.415 [-15.410, 0.005, 0.000]
Epoch 4 [135/172] - Loss: 15.115 [-15.110, 0.005, 0.000]
Epoch 4 [136/172] - Loss: 15.560 [-15.555, 0.005, 0.000]
Epoch 4 [137/172] - Loss: 14.686 [-14.681, 0.005, 0.000]
Epoch 4 [138/172] - Loss: 15.618 [-15.613, 0.005, 0.000]
Epoch 4 [139/172] - Loss: 14.521 [-14.516, 0.005, 0.000]
Epoch 4 [140/172] - Loss: 15.191 [-15.186, 0.005, 0.000]
Epoch 4 [141/172] - Loss: 14.637 [-14.632, 0.005, 0.000]
Epoch 4 [142/172] - Loss: 15.051 [-15.046, 0.005, 0.000]
Epoch 4 [143/172] - Loss: 15.471 [-15.466, 0.005, 0.000]
Epoch 4 [144/172] - Loss: 15.221 [-15.216, 0.005, 0.000]
Epoch 4 [145/172] - Loss: 15.223 [-15.218, 0.005, 0.000]
Epoch 4 [146/172] - Loss: 15.534 [-15.529, 0.005, 0.000]
Epoch 4 [147/172] - Loss: 15.250 [-15.245, 0.005, 0.000]
Epoch 4 [148/172] - Loss: 15.029 [-15.024, 0.005, 0.000]
Epoch 4 [149/172] - Loss: 14.518 [-14.513, 0.005, 0.000]
Epoch 4 [150/172] - Loss: 14.807 [-14.802, 0.005, 0.000]
Epoch 4 [151/172] - Loss: 15.236 [-15.231, 0.005, 0.000]
Epoch 4 [152/172] - Loss: 15.194 [-15.189, 0.005, 0.000]
Epoch 4 [153/172] - Loss: 14.624 [-14.619, 0.005, 0.000]
Epoch 4 [154/172] - Loss: 14.860 [-14.855, 0.005, 0.000]
Epoch 4 [155/172] - Loss: 14.241 [-14.236, 0.005, 0.000]
Epoch 4 [156/172] - Loss: 15.475 [-15.470, 0.005, 0.000]
Epoch 4 [157/172] - Loss: 16.648 [-16.643, 0.005, 0.000]
Epoch 4 [158/172] - Loss: 15.458 [-15.453, 0.005, 0.000]
Epoch 4 [159/172] - Loss: 15.550 [-15.545, 0.005, 0.000]
Epoch 4 [160/172] - Loss: 14.938 [-14.933, 0.005, 0.000]
Epoch 4 [161/172] - Loss: 15.364 [-15.359, 0.005, 0.000]
Epoch 4 [162/172] - Loss: 15.391 [-15.386, 0.005, 0.000]
Epoch 4 [163/172] - Loss: 15.788 [-15.783, 0.005, 0.000]
Epoch 4 [164/172] - Loss: 14.974 [-14.969, 0.005, 0.000]
Epoch 4 [165/172] - Loss: 15.422 [-15.417, 0.005, 0.000]
Epoch 4 [166/172] - Loss: 15.486 [-15.481, 0.005, 0.000]
Epoch 4 [167/172] - Loss: 14.606 [-14.601, 0.005, 0.000]
Epoch 4 [168/172] - Loss: 15.538 [-15.533, 0.005, 0.000]
Epoch 4 [169/172] - Loss: 16.148 [-16.143, 0.005, 0.000]
Epoch 4 [170/172] - Loss: 14.367 [-14.362, 0.005, 0.000]
Epoch 4 [171/172] - Loss: 14.796 [-14.791, 0.005, 0.000]
Epoch 5 [0/172] - Loss: 15.053 [-15.048, 0.005, 0.000]
Epoch 5 [1/172] - Loss: 15.078 [-15.073, 0.005, 0.000]
Epoch 5 [2/172] - Loss: 14.307 [-14.302, 0.005, 0.000]
Epoch 5 [3/172] - Loss: 15.318 [-15.313, 0.005, 0.000]
Epoch 5 [4/172] - Loss: 15.002 [-14.997, 0.005, 0.000]
Epoch 5 [5/172] - Loss: 14.891 [-14.886, 0.005, 0.000]
Epoch 5 [6/172] - Loss: 16.028 [-16.023, 0.005, 0.000]
Epoch 5 [7/172] - Loss: 14.396 [-14.391, 0.005, 0.000]
Epoch 5 [8/172] - Loss: 14.776 [-14.771, 0.005, 0.000]
Epoch 5 [9/172] - Loss: 16.599 [-16.594, 0.005, 0.000]
Epoch 5 [10/172] - Loss: 14.371 [-14.366, 0.005, 0.000]
Epoch 5 [11/172] - Loss: 15.217 [-15.212, 0.005, 0.000]
Epoch 5 [12/172] - Loss: 15.141 [-15.136, 0.005, 0.000]
Epoch 5 [13/172] - Loss: 15.635 [-15.630, 0.005, 0.000]
Epoch 5 [14/172] - Loss: 15.422 [-15.416, 0.005, 0.000]
Epoch 5 [15/172] - Loss: 14.486 [-14.481, 0.005, 0.000]
Epoch 5 [16/172] - Loss: 15.168 [-15.163, 0.005, 0.000]
Epoch 5 [17/172] - Loss: 15.462 [-15.457, 0.005, 0.000]
Epoch 5 [18/172] - Loss: 14.518 [-14.513, 0.005, 0.000]
Epoch 5 [19/172] - Loss: 14.984 [-14.979, 0.005, 0.000]
Epoch 5 [20/172] - Loss: 15.399 [-15.394, 0.005, 0.000]
Epoch 5 [21/172] - Loss: 14.948 [-14.942, 0.005, 0.000]
Epoch 5 [22/172] - Loss: 14.696 [-14.691, 0.005, 0.000]
Epoch 5 [23/172] - Loss: 14.579 [-14.574, 0.005, 0.000]
Epoch 5 [24/172] - Loss: 14.525 [-14.520, 0.005, 0.000]
Epoch 5 [25/172] - Loss: 14.170 [-14.165, 0.005, 0.000]
Epoch 5 [26/172] - Loss: 15.070 [-15.065, 0.005, 0.000]
Epoch 5 [27/172] - Loss: 15.146 [-15.141, 0.005, 0.000]
Epoch 5 [28/172] - Loss: 15.200 [-15.195, 0.005, 0.000]
Epoch 5 [29/172] - Loss: 14.514 [-14.509, 0.005, 0.000]
Epoch 5 [30/172] - Loss: 14.688 [-14.683, 0.005, 0.000]
Epoch 5 [31/172] - Loss: 15.111 [-15.106, 0.005, 0.000]
Epoch 5 [32/172] - Loss: 15.781 [-15.776, 0.005, 0.000]
Epoch 5 [33/172] - Loss: 15.353 [-15.348, 0.005, 0.000]
Epoch 5 [34/172] - Loss: 15.111 [-15.106, 0.005, 0.000]
Epoch 5 [35/172] - Loss: 14.683 [-14.678, 0.005, 0.000]
Epoch 5 [36/172] - Loss: 15.027 [-15.022, 0.005, 0.000]
Epoch 5 [37/172] - Loss: 14.414 [-14.409, 0.005, 0.000]
Epoch 5 [38/172] - Loss: 15.376 [-15.371, 0.005, 0.000]
Epoch 5 [39/172] - Loss: 15.088 [-15.083, 0.005, 0.000]
Epoch 5 [40/172] - Loss: 14.598 [-14.593, 0.005, 0.000]
Epoch 5 [41/172] - Loss: 14.583 [-14.578, 0.005, 0.000]
Epoch 5 [42/172] - Loss: 14.736 [-14.731, 0.005, 0.000]
Epoch 5 [43/172] - Loss: 14.310 [-14.305, 0.005, 0.000]
Epoch 5 [44/172] - Loss: 14.406 [-14.401, 0.005, 0.000]
Epoch 5 [45/172] - Loss: 15.732 [-15.727, 0.005, 0.000]
Epoch 5 [46/172] - Loss: 15.326 [-15.321, 0.005, 0.000]
Epoch 5 [47/172] - Loss: 14.843 [-14.838, 0.005, 0.000]
Epoch 5 [48/172] - Loss: 14.207 [-14.202, 0.005, 0.000]
Epoch 5 [49/172] - Loss: 15.258 [-15.253, 0.005, 0.000]
Epoch 5 [50/172] - Loss: 14.153 [-14.148, 0.005, 0.000]
Epoch 5 [51/172] - Loss: 15.142 [-15.137, 0.005, 0.000]
Epoch 5 [52/172] - Loss: 14.871 [-14.866, 0.005, 0.000]
Epoch 5 [53/172] - Loss: 15.065 [-15.060, 0.005, 0.000]
Epoch 5 [54/172] - Loss: 15.160 [-15.155, 0.005, 0.000]
Epoch 5 [55/172] - Loss: 14.170 [-14.165, 0.005, 0.000]
Epoch 5 [56/172] - Loss: 14.545 [-14.540, 0.005, 0.000]
Epoch 5 [57/172] - Loss: 16.148 [-16.143, 0.005, 0.000]
Epoch 5 [58/172] - Loss: 15.529 [-15.524, 0.005, 0.000]
Epoch 5 [59/172] - Loss: 16.636 [-16.631, 0.005, 0.000]
Epoch 5 [60/172] - Loss: 15.653 [-15.648, 0.005, 0.000]
Epoch 5 [61/172] - Loss: 15.160 [-15.155, 0.005, 0.000]
Epoch 5 [62/172] - Loss: 15.225 [-15.220, 0.005, 0.000]
Epoch 5 [63/172] - Loss: 16.252 [-16.247, 0.005, 0.000]
Epoch 5 [64/172] - Loss: 14.766 [-14.761, 0.005, 0.000]
Epoch 5 [65/172] - Loss: 15.931 [-15.926, 0.005, 0.000]
Epoch 5 [66/172] - Loss: 15.230 [-15.225, 0.005, 0.000]
Epoch 5 [67/172] - Loss: 15.340 [-15.335, 0.005, 0.000]
Epoch 5 [68/172] - Loss: 15.502 [-15.497, 0.005, 0.000]
Epoch 5 [69/172] - Loss: 14.817 [-14.812, 0.005, 0.000]
Epoch 5 [70/172] - Loss: 15.265 [-15.260, 0.005, 0.000]
Epoch 5 [71/172] - Loss: 15.312 [-15.307, 0.005, 0.000]
Epoch 5 [72/172] - Loss: 14.834 [-14.829, 0.005, 0.000]
Epoch 5 [73/172] - Loss: 15.003 [-14.998, 0.005, 0.000]
Epoch 5 [74/172] - Loss: 15.061 [-15.056, 0.005, 0.000]
Epoch 5 [75/172] - Loss: 14.909 [-14.904, 0.005, 0.000]
Epoch 5 [76/172] - Loss: 14.726 [-14.721, 0.005, 0.000]
Epoch 5 [77/172] - Loss: 15.786 [-15.781, 0.005, 0.000]
Epoch 5 [78/172] - Loss: 15.808 [-15.803, 0.005, 0.000]
Epoch 5 [79/172] - Loss: 15.633 [-15.627, 0.005, 0.000]
Epoch 5 [80/172] - Loss: 16.392 [-16.387, 0.005, 0.000]
Epoch 5 [81/172] - Loss: 15.167 [-15.162, 0.005, 0.000]
Epoch 5 [82/172] - Loss: 14.936 [-14.931, 0.005, 0.000]
Epoch 5 [83/172] - Loss: 14.908 [-14.903, 0.005, 0.000]
Epoch 5 [84/172] - Loss: 15.287 [-15.282, 0.005, 0.000]
Epoch 5 [85/172] - Loss: 16.057 [-16.052, 0.005, 0.000]
Epoch 5 [86/172] - Loss: 15.162 [-15.157, 0.005, 0.000]
Epoch 5 [87/172] - Loss: 14.978 [-14.973, 0.005, 0.000]
Epoch 5 [88/172] - Loss: 14.448 [-14.443, 0.005, 0.000]
Epoch 5 [89/172] - Loss: 14.208 [-14.203, 0.005, 0.000]
Epoch 5 [90/172] - Loss: 15.002 [-14.997, 0.005, 0.000]
Epoch 5 [91/172] - Loss: 15.196 [-15.191, 0.005, 0.000]
Epoch 5 [92/172] - Loss: 15.354 [-15.349, 0.005, 0.000]
Epoch 5 [93/172] - Loss: 15.703 [-15.698, 0.005, 0.000]
Epoch 5 [94/172] - Loss: 15.594 [-15.589, 0.005, 0.000]
Epoch 5 [95/172] - Loss: 14.444 [-14.439, 0.005, 0.000]
Epoch 5 [96/172] - Loss: 14.835 [-14.830, 0.005, 0.000]
Epoch 5 [97/172] - Loss: 15.187 [-15.182, 0.005, 0.000]
Epoch 5 [98/172] - Loss: 16.005 [-16.000, 0.005, 0.000]
Epoch 5 [99/172] - Loss: 15.265 [-15.260, 0.005, 0.000]
Epoch 5 [100/172] - Loss: 14.204 [-14.199, 0.005, 0.000]
Epoch 5 [101/172] - Loss: 14.662 [-14.657, 0.005, 0.000]
Epoch 5 [102/172] - Loss: 16.339 [-16.334, 0.005, 0.000]
Epoch 5 [103/172] - Loss: 14.643 [-14.638, 0.005, 0.000]
Epoch 5 [104/172] - Loss: 14.504 [-14.499, 0.005, 0.000]
Epoch 5 [105/172] - Loss: 15.290 [-15.285, 0.005, 0.000]
Epoch 5 [106/172] - Loss: 14.761 [-14.756, 0.005, 0.000]
Epoch 5 [107/172] - Loss: 15.015 [-15.010, 0.005, 0.000]
Epoch 5 [108/172] - Loss: 14.418 [-14.413, 0.005, 0.000]
Epoch 5 [109/172] - Loss: 14.570 [-14.565, 0.005, 0.000]
Epoch 5 [110/172] - Loss: 14.903 [-14.898, 0.005, 0.000]
Epoch 5 [111/172] - Loss: 14.560 [-14.555, 0.005, 0.000]
Epoch 5 [112/172] - Loss: 14.826 [-14.821, 0.005, 0.000]
Epoch 5 [113/172] - Loss: 15.989 [-15.984, 0.005, 0.000]
Epoch 5 [114/172] - Loss: 13.949 [-13.944, 0.005, 0.000]
Epoch 5 [115/172] - Loss: 15.398 [-15.393, 0.005, 0.000]
Epoch 5 [116/172] - Loss: 14.408 [-14.403, 0.005, 0.000]
Epoch 5 [117/172] - Loss: 15.592 [-15.587, 0.005, 0.000]
Epoch 5 [118/172] - Loss: 14.540 [-14.535, 0.005, 0.000]
Epoch 5 [119/172] - Loss: 15.124 [-15.119, 0.005, 0.000]
Epoch 5 [120/172] - Loss: 14.821 [-14.816, 0.005, 0.000]
Epoch 5 [121/172] - Loss: 15.166 [-15.161, 0.005, 0.000]
Epoch 5 [122/172] - Loss: 15.902 [-15.897, 0.005, 0.000]
Epoch 5 [123/172] - Loss: 15.648 [-15.643, 0.005, 0.000]
Epoch 5 [124/172] - Loss: 14.400 [-14.395, 0.005, 0.000]
Epoch 5 [125/172] - Loss: 15.138 [-15.133, 0.005, 0.000]
Epoch 5 [126/172] - Loss: 14.667 [-14.662, 0.005, 0.000]
Epoch 5 [127/172] - Loss: 15.415 [-15.410, 0.005, 0.000]
Epoch 5 [128/172] - Loss: 14.513 [-14.508, 0.005, 0.000]
Epoch 5 [129/172] - Loss: 15.750 [-15.745, 0.005, 0.000]
Epoch 5 [130/172] - Loss: 15.792 [-15.787, 0.005, 0.000]
Epoch 5 [131/172] - Loss: 15.149 [-15.144, 0.005, 0.000]
Epoch 5 [132/172] - Loss: 14.776 [-14.771, 0.005, 0.000]
Epoch 5 [133/172] - Loss: 16.644 [-16.639, 0.005, 0.000]
Epoch 5 [134/172] - Loss: 15.261 [-15.256, 0.005, 0.000]
Epoch 5 [135/172] - Loss: 16.963 [-16.958, 0.005, 0.000]
Epoch 5 [136/172] - Loss: 14.868 [-14.863, 0.005, 0.000]
Epoch 5 [137/172] - Loss: 14.737 [-14.732, 0.005, 0.000]
Epoch 5 [138/172] - Loss: 15.825 [-15.820, 0.005, 0.000]
Epoch 5 [139/172] - Loss: 15.341 [-15.336, 0.005, 0.000]
Epoch 5 [140/172] - Loss: 15.051 [-15.046, 0.005, 0.000]
Epoch 5 [141/172] - Loss: 15.161 [-15.156, 0.005, 0.000]
Epoch 5 [142/172] - Loss: 16.101 [-16.096, 0.005, 0.000]
Epoch 5 [143/172] - Loss: 15.073 [-15.068, 0.005, 0.000]
Epoch 5 [144/172] - Loss: 15.145 [-15.140, 0.005, 0.000]
Epoch 5 [145/172] - Loss: 15.004 [-14.999, 0.005, 0.000]
Epoch 5 [146/172] - Loss: 15.632 [-15.627, 0.005, 0.000]
Epoch 5 [147/172] - Loss: 14.741 [-14.736, 0.005, 0.000]
Epoch 5 [148/172] - Loss: 14.547 [-14.542, 0.005, 0.000]
Epoch 5 [149/172] - Loss: 14.477 [-14.472, 0.005, 0.000]
Epoch 5 [150/172] - Loss: 15.135 [-15.130, 0.005, 0.000]
Epoch 5 [151/172] - Loss: 15.409 [-15.404, 0.005, 0.000]
Epoch 5 [152/172] - Loss: 14.337 [-14.332, 0.005, 0.000]
Epoch 5 [153/172] - Loss: 14.756 [-14.751, 0.005, 0.000]
Epoch 5 [154/172] - Loss: 15.052 [-15.047, 0.005, 0.000]
Epoch 5 [155/172] - Loss: 15.376 [-15.371, 0.005, 0.000]
Epoch 5 [156/172] - Loss: 15.284 [-15.279, 0.005, 0.000]
Epoch 5 [157/172] - Loss: 15.471 [-15.466, 0.005, 0.000]
Epoch 5 [158/172] - Loss: 14.486 [-14.481, 0.005, 0.000]
Epoch 5 [159/172] - Loss: 14.668 [-14.663, 0.005, 0.000]
Epoch 5 [160/172] - Loss: 16.408 [-16.403, 0.005, 0.000]
Epoch 5 [161/172] - Loss: 14.588 [-14.583, 0.005, 0.000]
Epoch 5 [162/172] - Loss: 14.097 [-14.092, 0.005, 0.000]
Epoch 5 [163/172] - Loss: 15.578 [-15.573, 0.005, 0.000]
Epoch 5 [164/172] - Loss: 14.208 [-14.203, 0.005, 0.000]
Epoch 5 [165/172] - Loss: 14.491 [-14.486, 0.005, 0.000]
Epoch 5 [166/172] - Loss: 15.768 [-15.763, 0.005, 0.000]
Epoch 5 [167/172] - Loss: 14.824 [-14.820, 0.005, 0.000]
Epoch 5 [168/172] - Loss: 15.442 [-15.437, 0.005, 0.000]
Epoch 5 [169/172] - Loss: 15.126 [-15.121, 0.005, 0.000]
Epoch 5 [170/172] - Loss: 15.110 [-15.105, 0.005, 0.000]
Epoch 5 [171/172] - Loss: 16.119 [-16.114, 0.005, 0.000]
Epoch 6 [0/172] - Loss: 14.593 [-14.588, 0.005, 0.000]
Epoch 6 [1/172] - Loss: 14.739 [-14.734, 0.005, 0.000]
Epoch 6 [2/172] - Loss: 14.575 [-14.570, 0.005, 0.000]
Epoch 6 [3/172] - Loss: 14.614 [-14.609, 0.005, 0.000]
Epoch 6 [4/172] - Loss: 15.555 [-15.550, 0.005, 0.000]
Epoch 6 [5/172] - Loss: 13.709 [-13.704, 0.005, 0.000]
Epoch 6 [6/172] - Loss: 16.044 [-16.039, 0.005, 0.000]
Epoch 6 [7/172] - Loss: 14.089 [-14.084, 0.005, 0.000]
Epoch 6 [8/172] - Loss: 14.290 [-14.285, 0.005, 0.000]
Epoch 6 [9/172] - Loss: 15.184 [-15.179, 0.005, 0.000]
Epoch 6 [10/172] - Loss: 15.323 [-15.318, 0.005, 0.000]
Epoch 6 [11/172] - Loss: 15.281 [-15.276, 0.005, 0.000]
Epoch 6 [12/172] - Loss: 14.796 [-14.791, 0.005, 0.000]
Epoch 6 [13/172] - Loss: 14.469 [-14.464, 0.005, 0.000]
Epoch 6 [14/172] - Loss: 15.509 [-15.504, 0.005, 0.000]
Epoch 6 [15/172] - Loss: 14.525 [-14.520, 0.005, 0.000]
Epoch 6 [16/172] - Loss: 14.377 [-14.372, 0.005, 0.000]
Epoch 6 [17/172] - Loss: 14.744 [-14.739, 0.005, 0.000]
Epoch 6 [18/172] - Loss: 14.835 [-14.830, 0.005, 0.000]
Epoch 6 [19/172] - Loss: 15.042 [-15.037, 0.005, 0.000]
Epoch 6 [20/172] - Loss: 14.750 [-14.745, 0.005, 0.000]
Epoch 6 [21/172] - Loss: 14.758 [-14.753, 0.005, 0.000]
Epoch 6 [22/172] - Loss: 15.541 [-15.536, 0.005, 0.000]
Epoch 6 [23/172] - Loss: 15.430 [-15.425, 0.005, 0.000]
Epoch 6 [24/172] - Loss: 15.219 [-15.214, 0.005, 0.000]
Epoch 6 [25/172] - Loss: 14.020 [-14.015, 0.005, 0.000]
Epoch 6 [26/172] - Loss: 15.153 [-15.148, 0.005, 0.000]
Epoch 6 [27/172] - Loss: 14.537 [-14.532, 0.005, 0.000]
Epoch 6 [28/172] - Loss: 14.698 [-14.693, 0.005, 0.000]
Epoch 6 [29/172] - Loss: 14.308 [-14.303, 0.005, 0.000]
Epoch 6 [30/172] - Loss: 14.975 [-14.970, 0.005, 0.000]
Epoch 6 [31/172] - Loss: 14.486 [-14.481, 0.005, 0.000]
Epoch 6 [32/172] - Loss: 14.628 [-14.623, 0.005, 0.000]
Epoch 6 [33/172] - Loss: 14.744 [-14.739, 0.005, 0.000]
Epoch 6 [34/172] - Loss: 14.793 [-14.788, 0.005, 0.000]
Epoch 6 [35/172] - Loss: 15.050 [-15.045, 0.005, 0.000]
Epoch 6 [36/172] - Loss: 14.431 [-14.426, 0.005, 0.000]
Epoch 6 [37/172] - Loss: 14.232 [-14.227, 0.005, 0.000]
Epoch 6 [38/172] - Loss: 15.110 [-15.105, 0.005, 0.000]
Epoch 6 [39/172] - Loss: 13.803 [-13.798, 0.005, 0.000]
Epoch 6 [40/172] - Loss: 14.059 [-14.054, 0.005, 0.000]
Epoch 6 [41/172] - Loss: 14.976 [-14.971, 0.005, 0.000]
Epoch 6 [42/172] - Loss: 14.273 [-14.268, 0.005, 0.000]
Epoch 6 [43/172] - Loss: 14.185 [-14.180, 0.005, 0.000]
Epoch 6 [44/172] - Loss: 14.304 [-14.299, 0.005, 0.000]
Epoch 6 [45/172] - Loss: 14.110 [-14.104, 0.005, 0.000]
Epoch 6 [46/172] - Loss: 15.164 [-15.159, 0.005, 0.000]
Epoch 6 [47/172] - Loss: 14.951 [-14.946, 0.005, 0.000]
Epoch 6 [48/172] - Loss: 14.039 [-14.034, 0.005, 0.000]
Epoch 6 [49/172] - Loss: 14.711 [-14.706, 0.005, 0.000]
Epoch 6 [50/172] - Loss: 14.525 [-14.520, 0.005, 0.000]
Epoch 6 [51/172] - Loss: 14.162 [-14.157, 0.005, 0.000]
Epoch 6 [52/172] - Loss: 13.926 [-13.921, 0.005, 0.000]
Epoch 6 [53/172] - Loss: 15.228 [-15.223, 0.005, 0.000]
Epoch 6 [54/172] - Loss: 14.304 [-14.299, 0.005, 0.000]
Epoch 6 [55/172] - Loss: 15.643 [-15.638, 0.005, 0.000]
Epoch 6 [56/172] - Loss: 13.710 [-13.705, 0.005, 0.000]
Epoch 6 [57/172] - Loss: 14.770 [-14.765, 0.005, 0.000]
Epoch 6 [58/172] - Loss: 14.791 [-14.786, 0.005, 0.000]
Epoch 6 [59/172] - Loss: 14.400 [-14.395, 0.005, 0.000]
Epoch 6 [60/172] - Loss: 14.192 [-14.187, 0.005, 0.000]
Epoch 6 [61/172] - Loss: 14.885 [-14.880, 0.005, 0.000]
Epoch 6 [62/172] - Loss: 13.722 [-13.717, 0.005, 0.000]
Epoch 6 [63/172] - Loss: 14.487 [-14.482, 0.005, 0.000]
Epoch 6 [64/172] - Loss: 14.833 [-14.828, 0.005, 0.000]
Epoch 6 [65/172] - Loss: 14.774 [-14.769, 0.005, 0.000]
Epoch 6 [66/172] - Loss: 14.847 [-14.842, 0.005, 0.000]
Epoch 6 [67/172] - Loss: 13.917 [-13.912, 0.005, 0.000]
Epoch 6 [68/172] - Loss: 15.467 [-15.462, 0.005, 0.000]
Epoch 6 [69/172] - Loss: 15.035 [-15.030, 0.005, 0.000]
Epoch 6 [70/172] - Loss: 14.944 [-14.939, 0.005, 0.000]
Epoch 6 [71/172] - Loss: 14.865 [-14.860, 0.005, 0.000]
Epoch 6 [72/172] - Loss: 14.984 [-14.979, 0.005, 0.000]
Epoch 6 [73/172] - Loss: 14.458 [-14.453, 0.005, 0.000]
Epoch 6 [74/172] - Loss: 14.653 [-14.648, 0.005, 0.000]
Epoch 6 [75/172] - Loss: 14.183 [-14.178, 0.005, 0.000]
Epoch 6 [76/172] - Loss: 14.416 [-14.411, 0.005, 0.000]
Epoch 6 [77/172] - Loss: 14.655 [-14.650, 0.005, 0.000]
Epoch 6 [78/172] - Loss: 15.142 [-15.137, 0.005, 0.000]
Epoch 6 [79/172] - Loss: 14.415 [-14.410, 0.005, 0.000]
Epoch 6 [80/172] - Loss: 15.303 [-15.298, 0.005, 0.000]
Epoch 6 [81/172] - Loss: 14.382 [-14.377, 0.005, 0.000]
Epoch 6 [82/172] - Loss: 14.732 [-14.727, 0.005, 0.000]
Epoch 6 [83/172] - Loss: 15.132 [-15.127, 0.005, 0.000]
Epoch 6 [84/172] - Loss: 15.251 [-15.246, 0.005, 0.000]
Epoch 6 [85/172] - Loss: 14.793 [-14.789, 0.005, 0.000]
Epoch 6 [86/172] - Loss: 14.946 [-14.941, 0.005, 0.000]
Epoch 6 [87/172] - Loss: 14.398 [-14.393, 0.005, 0.000]
Epoch 6 [88/172] - Loss: 14.994 [-14.989, 0.005, 0.000]
Epoch 6 [89/172] - Loss: 14.735 [-14.730, 0.005, 0.000]
Epoch 6 [90/172] - Loss: 15.014 [-15.009, 0.005, 0.000]
Epoch 6 [91/172] - Loss: 15.582 [-15.577, 0.005, 0.000]
Epoch 6 [92/172] - Loss: 15.287 [-15.282, 0.005, 0.000]
Epoch 6 [93/172] - Loss: 14.566 [-14.561, 0.005, 0.000]
Epoch 6 [94/172] - Loss: 14.856 [-14.851, 0.005, 0.000]
Epoch 6 [95/172] - Loss: 15.119 [-15.114, 0.005, 0.000]
Epoch 6 [96/172] - Loss: 14.910 [-14.905, 0.005, 0.000]
Epoch 6 [97/172] - Loss: 14.679 [-14.674, 0.005, 0.000]
Epoch 6 [98/172] - Loss: 15.024 [-15.019, 0.005, 0.000]
Epoch 6 [99/172] - Loss: 14.800 [-14.795, 0.005, 0.000]
Epoch 6 [100/172] - Loss: 14.504 [-14.499, 0.005, 0.000]
Epoch 6 [101/172] - Loss: 14.314 [-14.310, 0.005, 0.000]
Epoch 6 [102/172] - Loss: 15.266 [-15.261, 0.005, 0.000]
Epoch 6 [103/172] - Loss: 15.010 [-15.005, 0.005, 0.000]
Epoch 6 [104/172] - Loss: 14.208 [-14.203, 0.005, 0.000]
Epoch 6 [105/172] - Loss: 14.424 [-14.419, 0.005, 0.000]
Epoch 6 [106/172] - Loss: 14.871 [-14.866, 0.005, 0.000]
Epoch 6 [107/172] - Loss: 15.081 [-15.076, 0.005, 0.000]
Epoch 6 [108/172] - Loss: 14.843 [-14.838, 0.005, 0.000]
Epoch 6 [109/172] - Loss: 14.597 [-14.592, 0.005, 0.000]
Epoch 6 [110/172] - Loss: 14.882 [-14.877, 0.005, 0.000]
Epoch 6 [111/172] - Loss: 15.087 [-15.082, 0.005, 0.000]
Epoch 6 [112/172] - Loss: 14.206 [-14.201, 0.005, 0.000]
Epoch 6 [113/172] - Loss: 15.266 [-15.261, 0.005, 0.000]
Epoch 6 [114/172] - Loss: 14.659 [-14.654, 0.005, 0.000]
Epoch 6 [115/172] - Loss: 14.297 [-14.292, 0.005, 0.000]
Epoch 6 [116/172] - Loss: 14.359 [-14.354, 0.005, 0.000]
Epoch 6 [117/172] - Loss: 14.753 [-14.748, 0.005, 0.000]
Epoch 6 [118/172] - Loss: 14.689 [-14.684, 0.005, 0.000]
Epoch 6 [119/172] - Loss: 15.090 [-15.085, 0.005, 0.000]
Epoch 6 [120/172] - Loss: 15.196 [-15.191, 0.005, 0.000]
Epoch 6 [121/172] - Loss: 14.893 [-14.888, 0.005, 0.000]
Epoch 6 [122/172] - Loss: 13.895 [-13.890, 0.005, 0.000]
Epoch 6 [123/172] - Loss: 14.459 [-14.454, 0.005, 0.000]
Epoch 6 [124/172] - Loss: 15.197 [-15.192, 0.005, 0.000]
Epoch 6 [125/172] - Loss: 14.615 [-14.610, 0.005, 0.000]
Epoch 6 [126/172] - Loss: 14.310 [-14.305, 0.005, 0.000]
Epoch 6 [127/172] - Loss: 15.134 [-15.129, 0.005, 0.000]
Epoch 6 [128/172] - Loss: 14.718 [-14.713, 0.005, 0.000]
Epoch 6 [129/172] - Loss: 14.758 [-14.753, 0.005, 0.000]
Epoch 6 [130/172] - Loss: 14.127 [-14.123, 0.005, 0.000]
Epoch 6 [131/172] - Loss: 14.796 [-14.791, 0.005, 0.000]
Epoch 6 [132/172] - Loss: 15.060 [-15.055, 0.005, 0.000]
Epoch 6 [133/172] - Loss: 14.665 [-14.660, 0.005, 0.000]
Epoch 6 [134/172] - Loss: 14.627 [-14.622, 0.005, 0.000]
Epoch 6 [135/172] - Loss: 14.035 [-14.031, 0.005, 0.000]
Epoch 6 [136/172] - Loss: 14.895 [-14.890, 0.005, 0.000]
Epoch 6 [137/172] - Loss: 15.048 [-15.043, 0.005, 0.000]
Epoch 6 [138/172] - Loss: 13.968 [-13.963, 0.005, 0.000]
Epoch 6 [139/172] - Loss: 14.613 [-14.608, 0.005, 0.000]
Epoch 6 [140/172] - Loss: 15.142 [-15.137, 0.005, 0.000]
Epoch 6 [141/172] - Loss: 14.985 [-14.980, 0.005, 0.000]
Epoch 6 [142/172] - Loss: 14.993 [-14.988, 0.005, 0.000]
Epoch 6 [143/172] - Loss: 13.759 [-13.754, 0.005, 0.000]
Epoch 6 [144/172] - Loss: 14.942 [-14.937, 0.005, 0.000]
Epoch 6 [145/172] - Loss: 15.130 [-15.125, 0.005, 0.000]
Epoch 6 [146/172] - Loss: 14.192 [-14.187, 0.005, 0.000]
Epoch 6 [147/172] - Loss: 14.897 [-14.892, 0.005, 0.000]
Epoch 6 [148/172] - Loss: 14.828 [-14.823, 0.005, 0.000]
Epoch 6 [149/172] - Loss: 13.951 [-13.946, 0.005, 0.000]
Epoch 6 [150/172] - Loss: 15.377 [-15.372, 0.005, 0.000]
Epoch 6 [151/172] - Loss: 14.390 [-14.385, 0.005, 0.000]
Epoch 6 [152/172] - Loss: 15.477 [-15.472, 0.005, 0.000]
Epoch 6 [153/172] - Loss: 14.373 [-14.368, 0.005, 0.000]
Epoch 6 [154/172] - Loss: 15.339 [-15.334, 0.005, 0.000]
Epoch 6 [155/172] - Loss: 14.773 [-14.768, 0.005, 0.000]
Epoch 6 [156/172] - Loss: 14.250 [-14.245, 0.005, 0.000]
Epoch 6 [157/172] - Loss: 14.076 [-14.071, 0.005, 0.000]
Epoch 6 [158/172] - Loss: 13.819 [-13.814, 0.005, 0.000]
Epoch 6 [159/172] - Loss: 14.202 [-14.197, 0.005, 0.000]
Epoch 6 [160/172] - Loss: 14.444 [-14.439, 0.005, 0.000]
Epoch 6 [161/172] - Loss: 14.803 [-14.798, 0.005, 0.000]
Epoch 6 [162/172] - Loss: 14.807 [-14.802, 0.005, 0.000]
Epoch 6 [163/172] - Loss: 15.351 [-15.346, 0.005, 0.000]
Epoch 6 [164/172] - Loss: 14.114 [-14.109, 0.005, 0.000]
Epoch 6 [165/172] - Loss: 15.096 [-15.091, 0.005, 0.000]
Epoch 6 [166/172] - Loss: 14.819 [-14.814, 0.005, 0.000]
Epoch 6 [167/172] - Loss: 14.602 [-14.597, 0.005, 0.000]
Epoch 6 [168/172] - Loss: 15.289 [-15.284, 0.005, 0.000]
Epoch 6 [169/172] - Loss: 15.052 [-15.047, 0.005, 0.000]
Epoch 6 [170/172] - Loss: 14.700 [-14.695, 0.005, 0.000]
Epoch 6 [171/172] - Loss: 14.735 [-14.730, 0.005, 0.000]
Epoch 7 [0/172] - Loss: 16.148 [-16.143, 0.005, 0.000]
Epoch 7 [1/172] - Loss: 15.029 [-15.024, 0.005, 0.000]
Epoch 7 [2/172] - Loss: 15.121 [-15.116, 0.005, 0.000]
Epoch 7 [3/172] - Loss: 14.881 [-14.876, 0.005, 0.000]
Epoch 7 [4/172] - Loss: 15.741 [-15.736, 0.005, 0.000]
Epoch 7 [5/172] - Loss: 14.679 [-14.674, 0.005, 0.000]
Epoch 7 [6/172] - Loss: 14.327 [-14.322, 0.005, 0.000]
Epoch 7 [7/172] - Loss: 14.861 [-14.856, 0.005, 0.000]
Epoch 7 [8/172] - Loss: 14.087 [-14.082, 0.005, 0.000]
Epoch 7 [9/172] - Loss: 14.838 [-14.833, 0.005, 0.000]
Epoch 7 [10/172] - Loss: 15.112 [-15.107, 0.005, 0.000]
Epoch 7 [11/172] - Loss: 14.695 [-14.690, 0.005, 0.000]
Epoch 7 [12/172] - Loss: 15.467 [-15.463, 0.005, 0.000]
Epoch 7 [13/172] - Loss: 14.927 [-14.922, 0.005, 0.000]
Epoch 7 [14/172] - Loss: 14.833 [-14.828, 0.005, 0.000]
Epoch 7 [15/172] - Loss: 14.802 [-14.797, 0.005, 0.000]
Epoch 7 [16/172] - Loss: 14.955 [-14.950, 0.005, 0.000]
Epoch 7 [17/172] - Loss: 14.079 [-14.074, 0.005, 0.000]
Epoch 7 [18/172] - Loss: 15.065 [-15.060, 0.005, 0.000]
Epoch 7 [19/172] - Loss: 14.817 [-14.812, 0.005, 0.000]
Epoch 7 [20/172] - Loss: 14.651 [-14.646, 0.005, 0.000]
Epoch 7 [21/172] - Loss: 15.141 [-15.136, 0.005, 0.000]
Epoch 7 [22/172] - Loss: 14.495 [-14.490, 0.005, 0.000]
Epoch 7 [23/172] - Loss: 15.345 [-15.340, 0.005, 0.000]
Epoch 7 [24/172] - Loss: 14.847 [-14.842, 0.005, 0.000]
Epoch 7 [25/172] - Loss: 15.960 [-15.955, 0.005, 0.000]
Epoch 7 [26/172] - Loss: 15.011 [-15.006, 0.005, 0.000]
Epoch 7 [27/172] - Loss: 14.514 [-14.509, 0.005, 0.000]
Epoch 7 [28/172] - Loss: 14.853 [-14.848, 0.005, 0.000]
Epoch 7 [29/172] - Loss: 14.712 [-14.707, 0.005, 0.000]
Epoch 7 [30/172] - Loss: 14.992 [-14.987, 0.005, 0.000]
Epoch 7 [31/172] - Loss: 14.127 [-14.123, 0.005, 0.000]
Epoch 7 [32/172] - Loss: 15.545 [-15.540, 0.005, 0.000]
Epoch 7 [33/172] - Loss: 14.341 [-14.336, 0.005, 0.000]
Epoch 7 [34/172] - Loss: 14.190 [-14.185, 0.005, 0.000]
Epoch 7 [35/172] - Loss: 15.118 [-15.113, 0.005, 0.000]
Epoch 7 [36/172] - Loss: 14.179 [-14.174, 0.005, 0.000]
Epoch 7 [37/172] - Loss: 14.898 [-14.893, 0.005, 0.000]
Epoch 7 [38/172] - Loss: 14.387 [-14.382, 0.005, 0.000]
Epoch 7 [39/172] - Loss: 14.485 [-14.480, 0.005, 0.000]
Epoch 7 [40/172] - Loss: 14.794 [-14.789, 0.005, 0.000]
Epoch 7 [41/172] - Loss: 15.307 [-15.302, 0.005, 0.000]
Epoch 7 [42/172] - Loss: 13.888 [-13.883, 0.005, 0.000]
Epoch 7 [43/172] - Loss: 14.815 [-14.810, 0.005, 0.000]
Epoch 7 [44/172] - Loss: 13.641 [-13.636, 0.005, 0.000]
Epoch 7 [45/172] - Loss: 15.359 [-15.354, 0.005, 0.000]
Epoch 7 [46/172] - Loss: 16.592 [-16.587, 0.005, 0.000]
Epoch 7 [47/172] - Loss: 15.061 [-15.056, 0.005, 0.000]
Epoch 7 [48/172] - Loss: 15.526 [-15.521, 0.005, 0.000]
Epoch 7 [49/172] - Loss: 15.359 [-15.354, 0.005, 0.000]
Epoch 7 [50/172] - Loss: 14.892 [-14.887, 0.005, 0.000]
Epoch 7 [51/172] - Loss: 14.269 [-14.264, 0.005, 0.000]
Epoch 7 [52/172] - Loss: 15.285 [-15.280, 0.005, 0.000]
Epoch 7 [53/172] - Loss: 14.628 [-14.623, 0.005, 0.000]
Epoch 7 [54/172] - Loss: 15.017 [-15.012, 0.005, 0.000]
Epoch 7 [55/172] - Loss: 15.346 [-15.341, 0.005, 0.000]
Epoch 7 [56/172] - Loss: 13.883 [-13.878, 0.005, 0.000]
Epoch 7 [57/172] - Loss: 14.428 [-14.423, 0.005, 0.000]
Epoch 7 [58/172] - Loss: 14.459 [-14.454, 0.005, 0.000]
Epoch 7 [59/172] - Loss: 15.377 [-15.372, 0.005, 0.000]
Epoch 7 [60/172] - Loss: 14.914 [-14.909, 0.005, 0.000]
Epoch 7 [61/172] - Loss: 14.027 [-14.022, 0.005, 0.000]
Epoch 7 [62/172] - Loss: 15.376 [-15.371, 0.005, 0.000]
Epoch 7 [63/172] - Loss: 14.962 [-14.957, 0.005, 0.000]
Epoch 7 [64/172] - Loss: 14.605 [-14.600, 0.005, 0.000]
Epoch 7 [65/172] - Loss: 14.400 [-14.395, 0.005, 0.000]
Epoch 7 [66/172] - Loss: 14.445 [-14.440, 0.005, 0.000]
Epoch 7 [67/172] - Loss: 14.225 [-14.220, 0.005, 0.000]
Epoch 7 [68/172] - Loss: 14.421 [-14.416, 0.005, 0.000]
Epoch 7 [69/172] - Loss: 15.658 [-15.653, 0.005, 0.000]
Epoch 7 [70/172] - Loss: 14.437 [-14.432, 0.005, 0.000]
Epoch 7 [71/172] - Loss: 15.189 [-15.184, 0.005, 0.000]
Epoch 7 [72/172] - Loss: 15.159 [-15.154, 0.005, 0.000]
Epoch 7 [73/172] - Loss: 14.119 [-14.114, 0.005, 0.000]
Epoch 7 [74/172] - Loss: 14.961 [-14.956, 0.005, 0.000]
Epoch 7 [75/172] - Loss: 14.744 [-14.739, 0.005, 0.000]
Epoch 7 [76/172] - Loss: 14.488 [-14.483, 0.005, 0.000]
Epoch 7 [77/172] - Loss: 14.543 [-14.538, 0.005, 0.000]
Epoch 7 [78/172] - Loss: 14.944 [-14.939, 0.005, 0.000]
Epoch 7 [79/172] - Loss: 14.497 [-14.492, 0.005, 0.000]
Epoch 7 [80/172] - Loss: 15.280 [-15.275, 0.005, 0.000]
Epoch 7 [81/172] - Loss: 14.205 [-14.200, 0.005, 0.000]
Epoch 7 [82/172] - Loss: 14.834 [-14.829, 0.005, 0.000]
Epoch 7 [83/172] - Loss: 15.288 [-15.283, 0.005, 0.000]
Epoch 7 [84/172] - Loss: 15.167 [-15.162, 0.005, 0.000]
Epoch 7 [85/172] - Loss: 14.843 [-14.838, 0.005, 0.000]
Epoch 7 [86/172] - Loss: 14.799 [-14.794, 0.005, 0.000]
Epoch 7 [87/172] - Loss: 15.775 [-15.770, 0.005, 0.000]
Epoch 7 [88/172] - Loss: 14.614 [-14.609, 0.005, 0.000]
Epoch 7 [89/172] - Loss: 14.073 [-14.068, 0.005, 0.000]
Epoch 7 [90/172] - Loss: 13.946 [-13.941, 0.005, 0.000]
Epoch 7 [91/172] - Loss: 13.908 [-13.903, 0.005, 0.000]
Epoch 7 [92/172] - Loss: 14.854 [-14.849, 0.005, 0.000]
Epoch 7 [93/172] - Loss: 15.066 [-15.061, 0.005, 0.000]
Epoch 7 [94/172] - Loss: 14.189 [-14.184, 0.005, 0.000]
Epoch 7 [95/172] - Loss: 15.861 [-15.856, 0.005, 0.000]
Epoch 7 [96/172] - Loss: 14.445 [-14.440, 0.005, 0.000]
Epoch 7 [97/172] - Loss: 15.190 [-15.185, 0.005, 0.000]
Epoch 7 [98/172] - Loss: 14.862 [-14.857, 0.005, 0.000]
Epoch 7 [99/172] - Loss: 15.083 [-15.078, 0.005, 0.000]
Epoch 7 [100/172] - Loss: 14.469 [-14.464, 0.005, 0.000]
Epoch 7 [101/172] - Loss: 13.945 [-13.940, 0.005, 0.000]
Epoch 7 [102/172] - Loss: 15.299 [-15.294, 0.005, 0.000]
Epoch 7 [103/172] - Loss: 14.442 [-14.437, 0.005, 0.000]
Epoch 7 [104/172] - Loss: 14.831 [-14.826, 0.005, 0.000]
Epoch 7 [105/172] - Loss: 14.375 [-14.370, 0.005, 0.000]
Epoch 7 [106/172] - Loss: 14.661 [-14.656, 0.005, 0.000]
Epoch 7 [107/172] - Loss: 14.847 [-14.842, 0.005, 0.000]
Epoch 7 [108/172] - Loss: 15.227 [-15.222, 0.005, 0.000]
Epoch 7 [109/172] - Loss: 14.800 [-14.795, 0.005, 0.000]
Epoch 7 [110/172] - Loss: 14.150 [-14.145, 0.005, 0.000]
Epoch 7 [111/172] - Loss: 15.060 [-15.055, 0.005, 0.000]
Epoch 7 [112/172] - Loss: 14.662 [-14.657, 0.005, 0.000]
Epoch 7 [113/172] - Loss: 14.523 [-14.518, 0.005, 0.000]
Epoch 7 [114/172] - Loss: 13.550 [-13.545, 0.005, 0.000]
Epoch 7 [115/172] - Loss: 14.695 [-14.690, 0.005, 0.000]
Epoch 7 [116/172] - Loss: 13.834 [-13.829, 0.005, 0.000]
Epoch 7 [117/172] - Loss: 14.817 [-14.812, 0.005, 0.000]
Epoch 7 [118/172] - Loss: 14.689 [-14.684, 0.005, 0.000]
Epoch 7 [119/172] - Loss: 14.879 [-14.874, 0.005, 0.000]
Epoch 7 [120/172] - Loss: 15.138 [-15.133, 0.005, 0.000]
Epoch 7 [121/172] - Loss: 14.649 [-14.644, 0.005, 0.000]
Epoch 7 [122/172] - Loss: 15.051 [-15.046, 0.005, 0.000]
Epoch 7 [123/172] - Loss: 14.429 [-14.424, 0.005, 0.000]
Epoch 7 [124/172] - Loss: 14.744 [-14.739, 0.005, 0.000]
Epoch 7 [125/172] - Loss: 14.583 [-14.578, 0.005, 0.000]
Epoch 7 [126/172] - Loss: 13.667 [-13.662, 0.005, 0.000]
Epoch 7 [127/172] - Loss: 14.536 [-14.531, 0.005, 0.000]
Epoch 7 [128/172] - Loss: 14.548 [-14.543, 0.005, 0.000]
Epoch 7 [129/172] - Loss: 14.699 [-14.694, 0.005, 0.000]
Epoch 7 [130/172] - Loss: 14.909 [-14.904, 0.005, 0.000]
Epoch 7 [131/172] - Loss: 14.525 [-14.520, 0.005, 0.000]
Epoch 7 [132/172] - Loss: 14.627 [-14.622, 0.005, 0.000]
Epoch 7 [133/172] - Loss: 14.092 [-14.087, 0.005, 0.000]
Epoch 7 [134/172] - Loss: 14.619 [-14.614, 0.005, 0.000]
Epoch 7 [135/172] - Loss: 14.725 [-14.720, 0.005, 0.000]
Epoch 7 [136/172] - Loss: 14.868 [-14.863, 0.005, 0.000]
Epoch 7 [137/172] - Loss: 14.392 [-14.387, 0.005, 0.000]
Epoch 7 [138/172] - Loss: 14.566 [-14.561, 0.005, 0.000]
Epoch 7 [139/172] - Loss: 14.665 [-14.660, 0.005, 0.000]
Epoch 7 [140/172] - Loss: 13.815 [-13.810, 0.005, 0.000]
Epoch 7 [141/172] - Loss: 15.246 [-15.241, 0.005, 0.000]
Epoch 7 [142/172] - Loss: 15.023 [-15.018, 0.005, 0.000]
Epoch 7 [143/172] - Loss: 14.906 [-14.901, 0.005, 0.000]
Epoch 7 [144/172] - Loss: 14.576 [-14.571, 0.005, 0.000]
Epoch 7 [145/172] - Loss: 14.708 [-14.703, 0.005, 0.000]
Epoch 7 [146/172] - Loss: 14.376 [-14.371, 0.005, 0.000]
Epoch 7 [147/172] - Loss: 15.070 [-15.065, 0.005, 0.000]
Epoch 7 [148/172] - Loss: 14.989 [-14.984, 0.005, 0.000]
Epoch 7 [149/172] - Loss: 14.475 [-14.470, 0.005, 0.000]
Epoch 7 [150/172] - Loss: 15.362 [-15.357, 0.005, 0.000]
Epoch 7 [151/172] - Loss: 13.407 [-13.402, 0.005, 0.000]
Epoch 7 [152/172] - Loss: 13.776 [-13.771, 0.005, 0.000]
Epoch 7 [153/172] - Loss: 14.092 [-14.087, 0.005, 0.000]
Epoch 7 [154/172] - Loss: 14.999 [-14.994, 0.005, 0.000]
Epoch 7 [155/172] - Loss: 14.373 [-14.368, 0.005, 0.000]
Epoch 7 [156/172] - Loss: 14.579 [-14.574, 0.005, 0.000]
Epoch 7 [157/172] - Loss: 15.382 [-15.377, 0.005, 0.000]
Epoch 7 [158/172] - Loss: 14.712 [-14.707, 0.005, 0.000]
Epoch 7 [159/172] - Loss: 14.842 [-14.837, 0.005, 0.000]
Epoch 7 [160/172] - Loss: 13.738 [-13.733, 0.005, 0.000]
Epoch 7 [161/172] - Loss: 14.870 [-14.865, 0.005, 0.000]
Epoch 7 [162/172] - Loss: 14.984 [-14.979, 0.005, 0.000]
Epoch 7 [163/172] - Loss: 14.710 [-14.705, 0.005, 0.000]
Epoch 7 [164/172] - Loss: 14.710 [-14.705, 0.005, 0.000]
Epoch 7 [165/172] - Loss: 14.040 [-14.035, 0.005, 0.000]
Epoch 7 [166/172] - Loss: 14.501 [-14.496, 0.005, 0.000]
Epoch 7 [167/172] - Loss: 14.216 [-14.211, 0.005, 0.000]
Epoch 7 [168/172] - Loss: 13.846 [-13.841, 0.005, 0.000]
Epoch 7 [169/172] - Loss: 14.141 [-14.136, 0.005, 0.000]
Epoch 7 [170/172] - Loss: 14.890 [-14.885, 0.005, 0.000]
Epoch 7 [171/172] - Loss: 15.541 [-15.536, 0.005, 0.000]

Making Predictions

The next cell gets the predictive covariance for the test set (and also technically gets the predictive mean, stored in preds.mean()) using the standard SKI testing code, with no acceleration or precomputation. Because the test set is substantially smaller than the training set, we don't need to make predictions in mini batches here, although our other tutorials demonstrate how to do this (for example, see the CIFAR tutorial).



In [8]:

    
model.eval()
likelihood.eval()
with torch.no_grad(), gpytorch.settings.use_toeplitz(False):
    preds = model(test_x)



In [9]:

    
print('Test MAE: {}'.format(torch.mean(torch.abs(preds.mean - test_y))))









    



Test MAE: 9.143630027770996



In [ ]: