pylearn2 tutorial: Multilayer Perceptron

Introduction

This ipython notebook will teach you the basics of how multilayer perceptrons work, and show you how to use multilayer perceptrons in pylearn2.

To do this, we will go over several concepts:

Part 1: What pylearn2 is doing for you in this example

Review of softmax regression, and how MLPs are similar
The multilayer perceptron model
Some beneficial properties of MLPs
Some detrimental properties of MLPs

Part 2: How to use pylearn2 to train an MLP

Part 3: A deeper MLP, and pylearn2 polymorphism

Part 4: Regularization, and pylearn2 costs

Note that this won't explain in detail how the individual classes are implemented. The classes follow pretty good naming conventions and have pretty good docstrings, but if you have trouble understanding them, write to me and I might add a part 3 explaining how some of the parts work under the hood.

Please write to pylearn-dev@googlegroups.com if you encounter any problem with this tutorial.

Requirements

Before running this notebook, you must have installed pylearn2. Follow the download and installation instructions if you have not yet done so.

This tutorial also assumes you already know about softmax regression, and know how to train and evaluate a softmax regression model in pylearn2. If not, work through softmax_regression.ipynb before starting this tutorial.

It's also strongly recommend that you run this notebook with THEANO_FLAGS="device=gpu". This is a processing intensive example and the GPU will make it run a lot faster, if you have one available. Execute the next cell to verify that you are using the GPU.



In [1]:

    
import theano
print theano.config.device









    



gpu






    



Using gpu device 0: GeForce GTX 285

Part 1: What pylearn2 is doing for you in this example

In this part, we won't get into any specifics of pylearn2 yet. We'll just discuss how to train a multilayer perceptron (MLP). If you already know about MLPs, feel free to skip straight to part 2, where we show how to do all of this in pylearn2.

Review of softmax regression, and how MLPs are similar

In softmax_regression.ipynb, we saw how softmax regression is a classification model that learns to map an input vector $x$ to a probability distribution $p(y\mid x)$ where $y$ is a categorical value with $k$ different values. We then described how a dataset $\mathcal{D}$ of $(x, y)$ tuples could be used to train a softmax regression model by maximizing the log likelihood,

$$ \sum_{x,y \in \mathcal{D} } \log P(y \mid x). $$

A multilayer perceptron is a very general machine learning model. In many cases, we can think of it as mapping $x$ to $P(y\mid x)$, and train it by maximizing the log likelihood. We'll start with that basic perspective, because of its similarity to softmax regression. (It is, however, possible to interpret the output of a multiplayer perceptron non-probabilistically, to use it for regression rather than classification, and to train it by optimizing functions other than the log likelihood)

Everything we described above is still relevant to the MLP. However, there is one more fact about softmax regression that does not apply to the MLP. Specifically, softmax regression assumes that

$$ p(y \mid x) = \frac { \exp( x^T W + b ) } { \sum_i \exp(x^T W + b)_i } = \text{softmax}( x^T W + b). $$

The MLP makes a different assumption about the functional form of $p(y \mid x)$.

The multilayer perceptron model

The multilayer perceptron model assumption is very weak. Essentially, the assumption is that the relationship between inputs and outputs can be represented by the composition of several simpler functions. Each function being composed can be thought of as another "layer" or stage of processing. The number of compositions determines the "depth" of the model.

Suppose we have a sequence of functions implementing the layers, $g_1, g_2, \dots, g_L$. Then the output of our MLP is

$$f(x) = g_L(g_{L-1}( \dots g_2( g_1 ( x )) \dots )).$$

In the first example for this tutorial, we will use just two layers. The final layer will be

$ g_2(g_1) = \text{softmax}( g_1^T W^{(2)} + b^{(2)}),$

so we can think of this model as using $g_1$ to transform $x$ into a different space, then doing softmax regression in that space.

For the first layer, we will use an affine transform followed by elementwise-application of the logistic sigmoid function, $\sigma(z) = \frac {1 } { 1 + \exp(-z) }.$ This is a very commonly used type of layer in multilayer perceptrons. Putting it all together, we get

$ g_1(x) = \sigma ( x^T W^{(1)} + b^{(1)} ).$

The full model is thus

$$ f(x) = \text{softmax}( \sigma ( x^T W^{(1)} + b^{(1)} )^T W^{(2)} + b^{(2)}). $$

If we interpret $f(x)$ as defining $p(y \mid x)$, it makes sense to train the parameters $W^{(1)}$, $W^{(2)}$, $b^{(1)}$, and $b^{(2)}$ by maximizing the log likelihood of the training data.

Some beneficial properties of MLPs

An obvious problem with softmax regression and other linear classifiers is that linear functions are very simple. They prevent solutions to even very simple classification problems, such as the class of 2 bit patterns whose XOR is true. XOR is true when $x=[1,0]$ or $x=[0,1]$ but not when $x=[0,0]$ or $x=[1,1]$. Suppose we draw a line that separates $[0,0]$ from $[0,1]$. Then it must pass through some point $[0,p]$. We require that this line also pass through $[q,1]$ in order to separate $[0,1]$ from $[1,1]$. But this means it slope must be negative and its $x$-intercept must be negative. Since a line only has one $x$ intercept, it does not pass between $[0,0]$ and $[1,0]$. Those two points belong to different classes, so any linear classifier must fail.

An MLP solves this problem by introducing extra stages of processing. In our two layer example, suppose the dimensionality of the first layer is 2. We call the outputs of this layer "hidden units" because they are neither inputs nor outputs of the system; they are unobserved variables that the network must decide what to do with. The MLP can set one of these hidden units to be active when the sum of the two input variables is less than 1. It can set the other to be active when the sum of the two input variables is greater than 1. It can then set the output unit to be active by default, and to deactivate when either of the two hidden variables is active.

More generally, an MLP with one sufficient large hidden layer can represent any function. This result is known as the "universal approximator theorem."

Another advantage of MLPs is that they can be made deeper and deeper, rather than just wider and wider. Many functions can be represented more efficiently (using fewer parameters) with a deep architecture than with a wide one. Using fewer parameters is beneficial both because the MLP takes less memory to represent, but also because the parameters may be estimated more accurately from a smaller amount of data.

Some detrimental properties of MLPs

Unfortunately, just because an MLP can represent any function does not mean that it will learn to represent the right function. The problem of overfitting can still make the MLP perform badly on the test set even if it classifies the training set perfectly. While larger MLPs are capable of fitting more complicated training sets, they are also likely to overfit worse than smaller MLPs.

A related issue with MLPs is that they have many configuration options. The model itself imposes design decisions such as what type of function to use for each layer, the dimensionality of each layer. Also, the log likelihood is no longer generally concave, so the choice of optimization procedure matters more than it did with softmax regression. These configuration options are known as "hyperparameters." Choosing the right hyperparameters is an open and exciting research problem.

Most of the hyperparameters in this tutorial were not chosen particularly carefully. Feel free to play with all of the settings in this notebook. If you find better ones, write to me and I'll put your settings and your name in the tutorial!

Part 2: How to use pylearn2 to train an MLP

Now that we've described the theory of what we're going to do, it's time to do it! This part describes how to use pylearn2 to run the algorithms described above.

As in the softmax regression tutorial, we will use the MLP to do optical character recognition on the MNIST dataset. The yaml string we construct is similar ot the one we use before. The main difference is that the MLP model class takes a "layers" argument describing the various layers of the model.

Note that for each layer, we need to specify what class to load. The identity of this class determines what type of layer appears at each position in the network. Here, we use a sigmoid hidden layer followed by a softmax output layer.

Every layer of the MLP needs a unique name. Here we name the first hidden layer 'h0' and the output label representing the prediction of the class $y$ 'y'. These layer names are used to generate monitor channel names later so that we can track properties of each layer separately.

The hidden layer needs some configuration that is pretty similar to the configuration for the output layer. Much as we need to tell the output layer its size (10 classes) we also need to tell the hidden layer its dimension, or the number of hidden units to go in that layer. In this case we use 500. We also need to tell it how to initialize its weights. The Sigmoid class supports the irange argument that we demonstrated for Softmax in the softmax regression tutorial, and we could use that here. Instead, we demonstrate a different argument, sparse_init. When sparse_init is specified, each unit gets exactly sparse_init non-zero weights initially. These weights are drawn from $N(0,1)$, so they are quite large compared to how weights are usually initialized.



In [2]:

    
import os
import pylearn2
path = os.path.join(pylearn2.__path__[0], 'scripts', 'tutorials', 'multilayer_perceptron', 'mlp_tutorial_part_2.yaml')
with open(path, 'r') as f:
    train = f.read()
hyper_params = {'train_stop' : 50000,
                'valid_stop' : 60000,
                'dim_h0' : 500,
                'max_epochs' : 10000,
                'save_path' : '.'}
train = train % (hyper_params)
print train









    



!obj:pylearn2.train.Train {
    dataset: &train !obj:pylearn2.datasets.mnist.MNIST {
        which_set: 'train',
        start: 0,
        stop: 50000
    },
    model: !obj:pylearn2.models.mlp.MLP {
        layers: [
                 !obj:pylearn2.models.mlp.Sigmoid {
                     layer_name: 'h0',
                     dim: 500,
                     sparse_init: 15,
                 }, !obj:pylearn2.models.mlp.Softmax {
                     layer_name: 'y',
                     n_classes: 10,
                     irange: 0.
                 }
                ],
        nvis: 784,
    },
    algorithm: !obj:pylearn2.training_algorithms.bgd.BGD {
        batch_size: 10000,
        line_search_mode: 'exhaustive',
        conjugate: 1,
        updates_per_batch: 10,
        monitoring_dataset:
            {
                'train' : *train,
                'valid' : !obj:pylearn2.datasets.mnist.MNIST {
                              which_set: 'train',
                              start: 50000,
                              stop: 60000
                          },
                'test'  : !obj:pylearn2.datasets.mnist.MNIST {
                              which_set: 'test',
                          }
            },
        termination_criterion: !obj:pylearn2.termination_criteria.And {
            criteria: [
                !obj:pylearn2.termination_criteria.MonitorBased {
                    channel_name: "valid_y_misclass"
                },
                !obj:pylearn2.termination_criteria.EpochCounter {
                    max_epochs: 10000
                }
            ]
        }
    },
    extensions: [
        !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest {
             channel_name: 'valid_y_misclass',
             save_path: "mlp_best.pkl"
        },
    ]
}

Note that we still do not specify a cost to be minimized. In the case of LogisticRegression, the model requested the negative log likelihood by default. In the case of the MLP, it is up to the final layer of the MLP to specify the default cost if the user does not provide one. In this case, since the final layer is a Softmax layer, we still have the same objective function as in the SoftmaxRegression tutorial.

Now, we use pylearn2's yaml_parse.load to construct the Train object, and run its main loop. The same thing could be accomplished by running pylearn2's train.py script on a file containing the yaml string.

Execute the next cell to train the model. This will take several minutes and possible as much as a few hours depending on how fast your computer is.



In [3]:

    
from pylearn2.config import yaml_parse
train = yaml_parse.load(train)
train.main_loop()









    



compiling begin_record_entry...






    



/u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit.
  warnings.warn("MLP changing the recursion limit.")






    



compiling begin_record_entry done. Time elapsed: 0.479222 seconds
Monitored channels: 
	ave_grad_mult
	ave_grad_size
	ave_step_size
	test_h0_col_norms_max
	test_h0_col_norms_mean
	test_h0_col_norms_min
	test_h0_max_x_max_u
	test_h0_max_x_mean_u
	test_h0_max_x_min_u
	test_h0_mean_x_max_u
	test_h0_mean_x_mean_u
	test_h0_mean_x_min_u
	test_h0_min_x_max_u
	test_h0_min_x_mean_u
	test_h0_min_x_min_u
	test_h0_row_norms_max
	test_h0_row_norms_mean
	test_h0_row_norms_min
	test_objective
	test_y_col_norms_max
	test_y_col_norms_mean
	test_y_col_norms_min
	test_y_max_max_class
	test_y_mean_max_class
	test_y_min_max_class
	test_y_misclass
	test_y_nll
	test_y_row_norms_max
	test_y_row_norms_mean
	test_y_row_norms_min
	train_h0_col_norms_max
	train_h0_col_norms_mean
	train_h0_col_norms_min
	train_h0_max_x_max_u
	train_h0_max_x_mean_u
	train_h0_max_x_min_u
	train_h0_mean_x_max_u
	train_h0_mean_x_mean_u
	train_h0_mean_x_min_u
	train_h0_min_x_max_u
	train_h0_min_x_mean_u
	train_h0_min_x_min_u
	train_h0_row_norms_max
	train_h0_row_norms_mean
	train_h0_row_norms_min
	train_objective
	train_y_col_norms_max
	train_y_col_norms_mean
	train_y_col_norms_min
	train_y_max_max_class
	train_y_mean_max_class
	train_y_min_max_class
	train_y_misclass
	train_y_nll
	train_y_row_norms_max
	train_y_row_norms_mean
	train_y_row_norms_min
	valid_h0_col_norms_max
	valid_h0_col_norms_mean
	valid_h0_col_norms_min
	valid_h0_max_x_max_u
	valid_h0_max_x_mean_u
	valid_h0_max_x_min_u
	valid_h0_mean_x_max_u
	valid_h0_mean_x_mean_u
	valid_h0_mean_x_min_u
	valid_h0_min_x_max_u
	valid_h0_min_x_mean_u
	valid_h0_min_x_min_u
	valid_h0_row_norms_max
	valid_h0_row_norms_mean
	valid_h0_row_norms_min
	valid_objective
	valid_y_col_norms_max
	valid_y_col_norms_mean
	valid_y_col_norms_min
	valid_y_max_max_class
	valid_y_mean_max_class
	valid_y_min_max_class
	valid_y_misclass
	valid_y_nll
	valid_y_row_norms_max
	valid_y_row_norms_mean
	valid_y_row_norms_min
Compiling accum...
graph size: 160
graph size: 157
graph size: 157
Compiling accum done. Time elapsed: 11.082528 seconds
Monitoring step:
	Epochs seen: 0
	Batches seen: 0
	Examples seen: 0
	ave_grad_mult: 0.0
	ave_grad_size: 0.0
	ave_step_size: 0.0
	test_h0_col_norms_max: 6.23503398895
	test_h0_col_norms_mean: 3.82355618477
	test_h0_col_norms_min: 2.06193995476
	test_h0_max_x_max_u: 0.999900639057
	test_h0_max_x_mean_u: 0.909942150116
	test_h0_max_x_min_u: 0.508436858654
	test_h0_mean_x_max_u: 0.901069939137
	test_h0_mean_x_mean_u: 0.476713299751
	test_h0_mean_x_min_u: 0.152832776308
	test_h0_min_x_max_u: 0.480607658625
	test_h0_min_x_mean_u: 0.0718067958951
	test_h0_min_x_min_u: 0.000174344575498
	test_h0_row_norms_max: 5.89326095581
	test_h0_row_norms_mean: 2.98549151421
	test_h0_row_norms_min: 0.0
	test_objective: 2.30258440971
	test_y_col_norms_max: 0.0
	test_y_col_norms_mean: 0.0
	test_y_col_norms_min: 0.0
	test_y_max_max_class: 0.0999999940395
	test_y_mean_max_class: 0.099990285933
	test_y_min_max_class: 0.0999999940395
	test_y_misclass: 0.901999950409
	test_y_nll: 2.30258440971
	test_y_row_norms_max: 0.0
	test_y_row_norms_mean: 0.0
	test_y_row_norms_min: 0.0
	train_h0_col_norms_max: 6.23503303528
	train_h0_col_norms_mean: 3.82355594635
	train_h0_col_norms_min: 2.06193971634
	train_h0_max_x_max_u: 0.999884188175
	train_h0_max_x_mean_u: 0.910601377487
	train_h0_max_x_min_u: 0.542480230331
	train_h0_mean_x_max_u: 0.899177610874
	train_h0_mean_x_mean_u: 0.477026820183
	train_h0_mean_x_min_u: 0.158626437187
	train_h0_min_x_max_u: 0.458495438099
	train_h0_min_x_mean_u: 0.0697233080864
	train_h0_min_x_min_u: 0.000107248379209
	train_h0_row_norms_max: 5.89326000214
	train_h0_row_norms_mean: 2.98549151421
	train_h0_row_norms_min: 0.0
	train_objective: 2.30258440971
	train_y_col_norms_max: 0.0
	train_y_col_norms_mean: 0.0
	train_y_col_norms_min: 0.0
	train_y_max_max_class: 0.0999999940395
	train_y_mean_max_class: 0.0999902933836
	train_y_min_max_class: 0.0999999940395
	train_y_misclass: 0.901359915733
	train_y_nll: 2.30258440971
	train_y_row_norms_max: 0.0
	train_y_row_norms_mean: 0.0
	train_y_row_norms_min: 0.0
	valid_h0_col_norms_max: 6.23503398895
	valid_h0_col_norms_mean: 3.82355618477
	valid_h0_col_norms_min: 2.06193995476
	valid_h0_max_x_max_u: 0.999902307987
	valid_h0_max_x_mean_u: 0.910734891891
	valid_h0_max_x_min_u: 0.505713641644
	valid_h0_mean_x_max_u: 0.897212743759
	valid_h0_mean_x_mean_u: 0.477113306522
	valid_h0_mean_x_min_u: 0.159442692995
	valid_h0_min_x_max_u: 0.474104195833
	valid_h0_min_x_mean_u: 0.0706818476319
	valid_h0_min_x_min_u: 0.000110276472697
	valid_h0_row_norms_max: 5.89326095581
	valid_h0_row_norms_mean: 2.98549151421
	valid_h0_row_norms_min: 0.0
	valid_objective: 2.30258440971
	valid_y_col_norms_max: 0.0
	valid_y_col_norms_mean: 0.0
	valid_y_col_norms_min: 0.0
	valid_y_max_max_class: 0.0999999940395
	valid_y_mean_max_class: 0.099990285933
	valid_y_min_max_class: 0.0999999940395
	valid_y_misclass: 0.900900006294
	valid_y_nll: 2.30258440971
	valid_y_row_norms_max: 0.0
	valid_y_row_norms_mean: 0.0
	valid_y_row_norms_min: 0.0
Time this epoch: 35.338505 seconds
Monitoring step:
	Epochs seen: 1
	Batches seen: 5
	Examples seen: 50000
	ave_grad_mult: 0.566698908806
	ave_grad_size: 0.567735552788
	ave_step_size: 0.291175425053
	test_h0_col_norms_max: 6.24065446854
	test_h0_col_norms_mean: 3.83268666267
	test_h0_col_norms_min: 2.0723836422
	test_h0_max_x_max_u: 0.999798893929
	test_h0_max_x_mean_u: 0.930105090141
	test_h0_max_x_min_u: 0.600322246552
	test_h0_mean_x_max_u: 0.863031387329
	test_h0_mean_x_mean_u: 0.476889610291
	test_h0_mean_x_min_u: 0.171247333288
	test_h0_min_x_max_u: 0.412737071514
	test_h0_min_x_mean_u: 0.0536084063351
	test_h0_min_x_min_u: 0.000199288566364
	test_h0_row_norms_max: 5.89763784409
	test_h0_row_norms_mean: 2.99287319183
	test_h0_row_norms_min: 0.0068221190013
	test_objective: 0.350786328316
	test_y_col_norms_max: 2.74948716164
	test_y_col_norms_mean: 2.56346487999
	test_y_col_norms_min: 2.34412789345
	test_y_max_max_class: 0.999794960022
	test_y_mean_max_class: 0.840726792812
	test_y_min_max_class: 0.207839608192
	test_y_misclass: 0.0983999967575
	test_y_nll: 0.350786328316
	test_y_row_norms_max: 0.701220929623
	test_y_row_norms_mean: 0.34330791235
	test_y_row_norms_min: 0.0764839723706
	train_h0_col_norms_max: 6.24065446854
	train_h0_col_norms_mean: 3.83268642426
	train_h0_col_norms_min: 2.07238340378
	train_h0_max_x_max_u: 0.999829530716
	train_h0_max_x_mean_u: 0.930867910385
	train_h0_max_x_min_u: 0.617025732994
	train_h0_mean_x_max_u: 0.860394179821
	train_h0_mean_x_mean_u: 0.477169722319
	train_h0_mean_x_min_u: 0.177841931581
	train_h0_min_x_max_u: 0.386521846056
	train_h0_min_x_mean_u: 0.0524694435298
	train_h0_min_x_min_u: 0.000151637359522
	train_h0_row_norms_max: 5.89763736725
	train_h0_row_norms_mean: 2.99287295341
	train_h0_row_norms_min: 0.0068221190013
	train_objective: 0.372914284468
	train_y_col_norms_max: 2.74948716164
	train_y_col_norms_mean: 2.56346464157
	train_y_col_norms_min: 2.34412789345
	train_y_max_max_class: 0.999826908112
	train_y_mean_max_class: 0.833846986294
	train_y_min_max_class: 0.198893502355
	train_y_misclass: 0.106319993734
	train_y_nll: 0.372914284468
	train_y_row_norms_max: 0.701220929623
	train_y_row_norms_mean: 0.343307882547
	train_y_row_norms_min: 0.0764839798212
	valid_h0_col_norms_max: 6.24065446854
	valid_h0_col_norms_mean: 3.83268666267
	valid_h0_col_norms_min: 2.0723836422
	valid_h0_max_x_max_u: 0.999864041805
	valid_h0_max_x_mean_u: 0.930580854416
	valid_h0_max_x_min_u: 0.638543665409
	valid_h0_mean_x_max_u: 0.858349621296
	valid_h0_mean_x_mean_u: 0.477255016565
	valid_h0_mean_x_min_u: 0.177810654044
	valid_h0_min_x_max_u: 0.361713379622
	valid_h0_min_x_mean_u: 0.0531250722706
	valid_h0_min_x_min_u: 0.000215846084757
	valid_h0_row_norms_max: 5.89763784409
	valid_h0_row_norms_mean: 2.99287319183
	valid_h0_row_norms_min: 0.0068221190013
	valid_objective: 0.339448153973
	valid_y_col_norms_max: 2.74948716164
	valid_y_col_norms_mean: 2.56346487999
	valid_y_col_norms_min: 2.34412789345
	valid_y_max_max_class: 0.999945104122
	valid_y_mean_max_class: 0.845010101795
	valid_y_min_max_class: 0.196165680885
	valid_y_misclass: 0.0965999960899
	valid_y_nll: 0.339448153973
	valid_y_row_norms_max: 0.701220929623
	valid_y_row_norms_mean: 0.34330791235
	valid_y_row_norms_min: 0.0764839723706
Time this epoch: 35.029214 seconds
Monitoring step:
	Epochs seen: 2
	Batches seen: 10
	Examples seen: 100000
	ave_grad_mult: 0.648920476437
	ave_grad_size: 0.385089039803
	ave_step_size: 0.205155700445
	test_h0_col_norms_max: 6.2453122139
	test_h0_col_norms_mean: 3.8378276825
	test_h0_col_norms_min: 2.07804393768
	test_h0_max_x_max_u: 0.999864637852
	test_h0_max_x_mean_u: 0.93498313427
	test_h0_max_x_min_u: 0.613258361816
	test_h0_mean_x_max_u: 0.847131431103
	test_h0_mean_x_mean_u: 0.476234823465
	test_h0_mean_x_min_u: 0.172577545047
	test_h0_min_x_max_u: 0.381593316793
	test_h0_min_x_mean_u: 0.0493729114532
	test_h0_min_x_min_u: 0.000119279786304
	test_h0_row_norms_max: 5.90795898438
	test_h0_row_norms_mean: 2.99731445312
	test_h0_row_norms_min: 0.0140750305727
	test_objective: 0.296338170767
	test_y_col_norms_max: 3.20915484428
	test_y_col_norms_mean: 3.00029850006
	test_y_col_norms_min: 2.73683047295
	test_y_max_max_class: 0.9999589324
	test_y_mean_max_class: 0.878535091877
	test_y_min_max_class: 0.236884206533
	test_y_misclass: 0.0850000008941
	test_y_nll: 0.296338170767
	test_y_row_norms_max: 0.839111089706
	test_y_row_norms_mean: 0.403169810772
	test_y_row_norms_min: 0.0928392037749
	train_h0_col_norms_max: 6.24531269073
	train_h0_col_norms_mean: 3.83782744408
	train_h0_col_norms_min: 2.07804393768
	train_h0_max_x_max_u: 0.999843478203
	train_h0_max_x_mean_u: 0.935774207115
	train_h0_max_x_min_u: 0.630811154842
	train_h0_mean_x_max_u: 0.843988478184
	train_h0_mean_x_mean_u: 0.476507484913
	train_h0_mean_x_min_u: 0.179330348969
	train_h0_min_x_max_u: 0.372446238995
	train_h0_min_x_mean_u: 0.048459071666
	train_h0_min_x_min_u: 0.000123051402625
	train_h0_row_norms_max: 5.90795898438
	train_h0_row_norms_mean: 2.99731445312
	train_h0_row_norms_min: 0.014075031504
	train_objective: 0.310930907726
	train_y_col_norms_max: 3.20915460587
	train_y_col_norms_mean: 3.00029873848
	train_y_col_norms_min: 2.73683071136
	train_y_max_max_class: 0.999969184399
	train_y_mean_max_class: 0.872422754765
	train_y_min_max_class: 0.206743046641
	train_y_misclass: 0.0889399945736
	train_y_nll: 0.310930907726
	train_y_row_norms_max: 0.839111089706
	train_y_row_norms_mean: 0.403169810772
	train_y_row_norms_min: 0.0928391963243
	valid_h0_col_norms_max: 6.2453122139
	valid_h0_col_norms_mean: 3.8378276825
	valid_h0_col_norms_min: 2.07804393768
	valid_h0_max_x_max_u: 0.999864220619
	valid_h0_max_x_mean_u: 0.935237765312
	valid_h0_max_x_min_u: 0.672344446182
	valid_h0_mean_x_max_u: 0.842247903347
	valid_h0_mean_x_mean_u: 0.476582586765
	valid_h0_mean_x_min_u: 0.178887397051
	valid_h0_min_x_max_u: 0.358671993017
	valid_h0_min_x_mean_u: 0.0488182529807
	valid_h0_min_x_min_u: 0.000185967219295
	valid_h0_row_norms_max: 5.90795898438
	valid_h0_row_norms_mean: 2.99731445312
	valid_h0_row_norms_min: 0.0140750305727
	valid_objective: 0.286341637373
	valid_y_col_norms_max: 3.20915484428
	valid_y_col_norms_mean: 3.00029850006
	valid_y_col_norms_min: 2.73683047295
	valid_y_max_max_class: 0.999980926514
	valid_y_mean_max_class: 0.880788624287
	valid_y_min_max_class: 0.193636313081
	valid_y_misclass: 0.0813999921083
	valid_y_nll: 0.286341637373
	valid_y_row_norms_max: 0.839111089706
	valid_y_row_norms_mean: 0.403169810772
	valid_y_row_norms_min: 0.0928392037749
Time this epoch: 35.009148 seconds
Monitoring step:
	Epochs seen: 3
	Batches seen: 15
	Examples seen: 150000
	ave_grad_mult: 0.747792065144
	ave_grad_size: 0.265085607767
	ave_step_size: 0.150685995817
	test_h0_col_norms_max: 6.24948835373
	test_h0_col_norms_mean: 3.84261131287
	test_h0_col_norms_min: 2.08266615868
	test_h0_max_x_max_u: 0.99994790554
	test_h0_max_x_mean_u: 0.937485575676
	test_h0_max_x_min_u: 0.633630394936
	test_h0_mean_x_max_u: 0.859075248241
	test_h0_mean_x_mean_u: 0.475113451481
	test_h0_mean_x_min_u: 0.166715249419
	test_h0_min_x_max_u: 0.368945479393
	test_h0_min_x_mean_u: 0.0472293719649
	test_h0_min_x_min_u: 5.30257530045e-05
	test_h0_row_norms_max: 5.91970491409
	test_h0_row_norms_mean: 3.00150084496
	test_h0_row_norms_min: 0.0220027510077
	test_objective: 0.269680500031
	test_y_col_norms_max: 3.56634759903
	test_y_col_norms_mean: 3.29666876793
	test_y_col_norms_min: 3.00721621513
	test_y_max_max_class: 0.999979376793
	test_y_mean_max_class: 0.893490552902
	test_y_min_max_class: 0.250094264746
	test_y_misclass: 0.0763000026345
	test_y_nll: 0.269680500031
	test_y_row_norms_max: 0.959613263607
	test_y_row_norms_mean: 0.443394243717
	test_y_row_norms_min: 0.103941932321
	train_h0_col_norms_max: 6.24948787689
	train_h0_col_norms_mean: 3.84261083603
	train_h0_col_norms_min: 2.08266615868
	train_h0_max_x_max_u: 0.99988758564
	train_h0_max_x_mean_u: 0.938323676586
	train_h0_max_x_min_u: 0.649454653263
	train_h0_mean_x_max_u: 0.846590101719
	train_h0_mean_x_mean_u: 0.475384742022
	train_h0_mean_x_min_u: 0.171920359135
	train_h0_min_x_max_u: 0.365952074528
	train_h0_min_x_mean_u: 0.0464779213071
	train_h0_min_x_min_u: 6.07749607298e-05
	train_h0_row_norms_max: 5.91970491409
	train_h0_row_norms_mean: 3.00150060654
	train_h0_row_norms_min: 0.022002749145
	train_objective: 0.278353452682
	train_y_col_norms_max: 3.56634736061
	train_y_col_norms_mean: 3.29666852951
	train_y_col_norms_min: 3.00721621513
	train_y_max_max_class: 0.999987363815
	train_y_mean_max_class: 0.889036417007
	train_y_min_max_class: 0.227912455797
	train_y_misclass: 0.0788599997759
	train_y_nll: 0.278353452682
	train_y_row_norms_max: 0.959613204002
	train_y_row_norms_mean: 0.443394213915
	train_y_row_norms_min: 0.103941932321
	valid_h0_col_norms_max: 6.24948835373
	valid_h0_col_norms_mean: 3.84261131287
	valid_h0_col_norms_min: 2.08266615868
	valid_h0_max_x_max_u: 0.999919652939
	valid_h0_max_x_mean_u: 0.937573850155
	valid_h0_max_x_min_u: 0.684871912003
	valid_h0_mean_x_max_u: 0.850003778934
	valid_h0_mean_x_mean_u: 0.475453108549
	valid_h0_mean_x_min_u: 0.170857235789
	valid_h0_min_x_max_u: 0.353432744741
	valid_h0_min_x_mean_u: 0.0467779003084
	valid_h0_min_x_min_u: 6.80360026308e-05
	valid_h0_row_norms_max: 5.91970491409
	valid_h0_row_norms_mean: 3.00150084496
	valid_h0_row_norms_min: 0.0220027510077
	valid_objective: 0.26020783186
	valid_y_col_norms_max: 3.56634759903
	valid_y_col_norms_mean: 3.29666876793
	valid_y_col_norms_min: 3.00721621513
	valid_y_max_max_class: 0.999977052212
	valid_y_mean_max_class: 0.896274268627
	valid_y_min_max_class: 0.17623616755
	valid_y_misclass: 0.0750000029802
	valid_y_nll: 0.26020783186
	valid_y_row_norms_max: 0.959613263607
	valid_y_row_norms_mean: 0.443394243717
	valid_y_row_norms_min: 0.103941932321
Time this epoch: 35.058853 seconds
Monitoring step:
	Epochs seen: 4
	Batches seen: 20
	Examples seen: 200000
	ave_grad_mult: 0.788351774216
	ave_grad_size: 0.187993511558
	ave_step_size: 0.113317854702
	test_h0_col_norms_max: 6.25235366821
	test_h0_col_norms_mean: 3.84656834602
	test_h0_col_norms_min: 2.08510184288
	test_h0_max_x_max_u: 0.999974727631
	test_h0_max_x_mean_u: 0.938515424728
	test_h0_max_x_min_u: 0.650707960129
	test_h0_mean_x_max_u: 0.87255191803
	test_h0_mean_x_mean_u: 0.474163293839
	test_h0_mean_x_min_u: 0.160470247269
	test_h0_min_x_max_u: 0.364907234907
	test_h0_min_x_mean_u: 0.0464833118021
	test_h0_min_x_min_u: 2.23769111471e-05
	test_h0_row_norms_max: 5.93058395386
	test_h0_row_norms_mean: 3.0049738884
	test_h0_row_norms_min: 0.0284670460969
	test_objective: 0.252513170242
	test_y_col_norms_max: 3.77643465996
	test_y_col_norms_mean: 3.49576759338
	test_y_col_norms_min: 3.21715569496
	test_y_max_max_class: 0.999990880489
	test_y_mean_max_class: 0.902969479561
	test_y_min_max_class: 0.223742827773
	test_y_misclass: 0.0724000036716
	test_y_nll: 0.252513170242
	test_y_row_norms_max: 1.04190921783
	test_y_row_norms_mean: 0.47004455328
	test_y_row_norms_min: 0.109351947904
	train_h0_col_norms_max: 6.25235366821
	train_h0_col_norms_mean: 3.84656858444
	train_h0_col_norms_min: 2.08510160446
	train_h0_max_x_max_u: 0.999940037727
	train_h0_max_x_mean_u: 0.939188420773
	train_h0_max_x_min_u: 0.661542713642
	train_h0_mean_x_max_u: 0.85992783308
	train_h0_mean_x_mean_u: 0.474434643984
	train_h0_mean_x_min_u: 0.163209468126
	train_h0_min_x_max_u: 0.358978569508
	train_h0_min_x_mean_u: 0.0456797704101
	train_h0_min_x_min_u: 3.15167126246e-05
	train_h0_row_norms_max: 5.93058395386
	train_h0_row_norms_mean: 3.00497412682
	train_h0_row_norms_min: 0.0284670442343
	train_objective: 0.257761448622
	train_y_col_norms_max: 3.77643465996
	train_y_col_norms_mean: 3.49576735497
	train_y_col_norms_min: 3.21715545654
	train_y_max_max_class: 0.999995172024
	train_y_mean_max_class: 0.898737490177
	train_y_min_max_class: 0.233332633972
	train_y_misclass: 0.0732599943876
	train_y_nll: 0.257761448622
	train_y_row_norms_max: 1.04190921783
	train_y_row_norms_mean: 0.470044583082
	train_y_row_norms_min: 0.109351947904
	valid_h0_col_norms_max: 6.25235366821
	valid_h0_col_norms_mean: 3.84656834602
	valid_h0_col_norms_min: 2.08510184288
	valid_h0_max_x_max_u: 0.999963521957
	valid_h0_max_x_mean_u: 0.938330054283
	valid_h0_max_x_min_u: 0.685399234295
	valid_h0_mean_x_max_u: 0.864110708237
	valid_h0_mean_x_mean_u: 0.474497437477
	valid_h0_mean_x_min_u: 0.161501988769
	valid_h0_min_x_max_u: 0.347681999207
	valid_h0_min_x_mean_u: 0.0459976904094
	valid_h0_min_x_min_u: 2.87672100967e-05
	valid_h0_row_norms_max: 5.93058395386
	valid_h0_row_norms_mean: 3.0049738884
	valid_h0_row_norms_min: 0.0284670460969
	valid_objective: 0.242218419909
	valid_y_col_norms_max: 3.77643465996
	valid_y_col_norms_mean: 3.49576759338
	valid_y_col_norms_min: 3.21715569496
	valid_y_max_max_class: 0.999983727932
	valid_y_mean_max_class: 0.90525239706
	valid_y_min_max_class: 0.237812787294
	valid_y_misclass: 0.070799998939
	valid_y_nll: 0.242218419909
	valid_y_row_norms_max: 1.04190921783
	valid_y_row_norms_mean: 0.47004455328
	valid_y_row_norms_min: 0.109351947904
Time this epoch: 34.824181 seconds
Monitoring step:
	Epochs seen: 5
	Batches seen: 25
	Examples seen: 250000
	ave_grad_mult: 0.822910606861
	ave_grad_size: 0.140246614814
	ave_step_size: 0.0910708159208
	test_h0_col_norms_max: 6.2554602623
	test_h0_col_norms_mean: 3.85085010529
	test_h0_col_norms_min: 2.08709287643
	test_h0_max_x_max_u: 0.999985814095
	test_h0_max_x_mean_u: 0.939129829407
	test_h0_max_x_min_u: 0.667058110237
	test_h0_mean_x_max_u: 0.881521999836
	test_h0_mean_x_mean_u: 0.473096251488
	test_h0_mean_x_min_u: 0.148683413863
	test_h0_min_x_max_u: 0.366505622864
	test_h0_min_x_mean_u: 0.0459363907576
	test_h0_min_x_min_u: 9.1133879323e-06
	test_h0_row_norms_max: 5.94399118423
	test_h0_row_norms_mean: 3.00873041153
	test_h0_row_norms_min: 0.0347110852599
	test_objective: 0.236052155495
	test_y_col_norms_max: 3.98437142372
	test_y_col_norms_mean: 3.68210268021
	test_y_col_norms_min: 3.41360712051
	test_y_max_max_class: 0.99999153614
	test_y_mean_max_class: 0.909221351147
	test_y_min_max_class: 0.227106332779
	test_y_misclass: 0.0672999992967
	test_y_nll: 0.236052155495
	test_y_row_norms_max: 1.12676775455
	test_y_row_norms_mean: 0.494562119246
	test_y_row_norms_min: 0.114525236189
	train_h0_col_norms_max: 6.2554602623
	train_h0_col_norms_mean: 3.85085010529
	train_h0_col_norms_min: 2.08709263802
	train_h0_max_x_max_u: 0.999965369701
	train_h0_max_x_mean_u: 0.939886808395
	train_h0_max_x_min_u: 0.672379374504
	train_h0_mean_x_max_u: 0.869372367859
	train_h0_mean_x_mean_u: 0.473366141319
	train_h0_mean_x_min_u: 0.151700764894
	train_h0_min_x_max_u: 0.357233524323
	train_h0_min_x_mean_u: 0.0450618416071
	train_h0_min_x_min_u: 1.45595986396e-05
	train_h0_row_norms_max: 5.9439907074
	train_h0_row_norms_mean: 3.00873041153
	train_h0_row_norms_min: 0.0347110852599
	train_objective: 0.239308148623
	train_y_col_norms_max: 3.9843711853
	train_y_col_norms_mean: 3.68210220337
	train_y_col_norms_min: 3.41360712051
	train_y_max_max_class: 0.999996185303
	train_y_mean_max_class: 0.905649185181
	train_y_min_max_class: 0.236008346081
	train_y_misclass: 0.0679599940777
	train_y_nll: 0.239308148623
	train_y_row_norms_max: 1.12676763535
	train_y_row_norms_mean: 0.494562089443
	train_y_row_norms_min: 0.114525228739
	valid_h0_col_norms_max: 6.2554602623
	valid_h0_col_norms_mean: 3.85085010529
	valid_h0_col_norms_min: 2.08709287643
	valid_h0_max_x_max_u: 0.999980926514
	valid_h0_max_x_mean_u: 0.939110815525
	valid_h0_max_x_min_u: 0.683836042881
	valid_h0_mean_x_max_u: 0.873598277569
	valid_h0_mean_x_mean_u: 0.473425507545
	valid_h0_mean_x_min_u: 0.149841591716
	valid_h0_min_x_max_u: 0.346154510975
	valid_h0_min_x_mean_u: 0.0454438403249
	valid_h0_min_x_min_u: 1.18227362691e-05
	valid_h0_row_norms_max: 5.94399118423
	valid_h0_row_norms_mean: 3.00873041153
	valid_h0_row_norms_min: 0.0347110852599
	valid_objective: 0.22658072412
	valid_y_col_norms_max: 3.98437142372
	valid_y_col_norms_mean: 3.68210268021
	valid_y_col_norms_min: 3.41360712051
	valid_y_max_max_class: 0.999987483025
	valid_y_mean_max_class: 0.911411643028
	valid_y_min_max_class: 0.217763110995
	valid_y_misclass: 0.0644000023603
	valid_y_nll: 0.22658072412
	valid_y_row_norms_max: 1.12676775455
	valid_y_row_norms_mean: 0.494562119246
	valid_y_row_norms_min: 0.114525236189
Time this epoch: 35.012249 seconds
Monitoring step:
	Epochs seen: 6
	Batches seen: 30
	Examples seen: 300000
	ave_grad_mult: 0.849331319332
	ave_grad_size: 0.110973127186
	ave_step_size: 0.0771789103746
	test_h0_col_norms_max: 6.25832700729
	test_h0_col_norms_mean: 3.85529947281
	test_h0_col_norms_min: 2.08869576454
	test_h0_max_x_max_u: 0.999991595745
	test_h0_max_x_mean_u: 0.93943220377
	test_h0_max_x_min_u: 0.680398881435
	test_h0_mean_x_max_u: 0.887371778488
	test_h0_mean_x_mean_u: 0.472293674946
	test_h0_mean_x_min_u: 0.139431104064
	test_h0_min_x_max_u: 0.367107391357
	test_h0_min_x_mean_u: 0.0457468703389
	test_h0_min_x_min_u: 3.69549866264e-06
	test_h0_row_norms_max: 5.9600777626
	test_h0_row_norms_mean: 3.01261997223
	test_h0_row_norms_min: 0.0412151031196
	test_objective: 0.222071394324
	test_y_col_norms_max: 4.16519927979
	test_y_col_norms_mean: 3.85762476921
	test_y_col_norms_min: 3.61017894745
	test_y_max_max_class: 0.999991238117
	test_y_mean_max_class: 0.913735508919
	test_y_min_max_class: 0.246407344937
	test_y_misclass: 0.0631999969482
	test_y_nll: 0.222071394324
	test_y_row_norms_max: 1.19918644428
	test_y_row_norms_mean: 0.517221450806
	test_y_row_norms_min: 0.117476500571
	train_h0_col_norms_max: 6.25832748413
	train_h0_col_norms_mean: 3.85529899597
	train_h0_col_norms_min: 2.08869576454
	train_h0_max_x_max_u: 0.999979615211
	train_h0_max_x_mean_u: 0.94024169445
	train_h0_max_x_min_u: 0.675026059151
	train_h0_mean_x_max_u: 0.87550008297
	train_h0_mean_x_mean_u: 0.472564071417
	train_h0_mean_x_min_u: 0.142730906606
	train_h0_min_x_max_u: 0.356041908264
	train_h0_min_x_mean_u: 0.044754832983
	train_h0_min_x_min_u: 6.11660334471e-06
	train_h0_row_norms_max: 5.96007823944
	train_h0_row_norms_mean: 3.01261997223
	train_h0_row_norms_min: 0.0412151031196
	train_objective: 0.222275063396
	train_y_col_norms_max: 4.16519880295
	train_y_col_norms_mean: 3.85762453079
	train_y_col_norms_min: 3.61017894745
	train_y_max_max_class: 0.999996602535
	train_y_mean_max_class: 0.910623729229
	train_y_min_max_class: 0.235357835889
	train_y_misclass: 0.062839999795
	train_y_nll: 0.222275063396
	train_y_row_norms_max: 1.19918644428
	train_y_row_norms_mean: 0.517221450806
	train_y_row_norms_min: 0.11747649312
	valid_h0_col_norms_max: 6.25832700729
	valid_h0_col_norms_mean: 3.85529947281
	valid_h0_col_norms_min: 2.08869576454
	valid_h0_max_x_max_u: 0.999989330769
	valid_h0_max_x_mean_u: 0.939590632915
	valid_h0_max_x_min_u: 0.678366243839
	valid_h0_mean_x_max_u: 0.879810392857
	valid_h0_mean_x_mean_u: 0.472620040178
	valid_h0_mean_x_min_u: 0.140709280968
	valid_h0_min_x_max_u: 0.344533830881
	valid_h0_min_x_mean_u: 0.0452971383929
	valid_h0_min_x_min_u: 4.94029472975e-06
	valid_h0_row_norms_max: 5.9600777626
	valid_h0_row_norms_mean: 3.01261997223
	valid_h0_row_norms_min: 0.0412151031196
	valid_objective: 0.213480621576
	valid_y_col_norms_max: 4.16519927979
	valid_y_col_norms_mean: 3.85762476921
	valid_y_col_norms_min: 3.61017894745
	valid_y_max_max_class: 0.999992728233
	valid_y_mean_max_class: 0.915528953075
	valid_y_min_max_class: 0.230840429664
	valid_y_misclass: 0.0590999983251
	valid_y_nll: 0.213480621576
	valid_y_row_norms_max: 1.19918644428
	valid_y_row_norms_mean: 0.517221450806
	valid_y_row_norms_min: 0.117476500571
Time this epoch: 34.796789 seconds
Monitoring step:
	Epochs seen: 7
	Batches seen: 35
	Examples seen: 350000
	ave_grad_mult: 0.921035170555
	ave_grad_size: 0.0949304848909
	ave_step_size: 0.0732585340738
	test_h0_col_norms_max: 6.26188564301
	test_h0_col_norms_mean: 3.86070275307
	test_h0_col_norms_min: 2.09020781517
	test_h0_max_x_max_u: 0.999995708466
	test_h0_max_x_mean_u: 0.940146625042
	test_h0_max_x_min_u: 0.672576725483
	test_h0_mean_x_max_u: 0.892456889153
	test_h0_mean_x_mean_u: 0.47117972374
	test_h0_mean_x_min_u: 0.127655550838
	test_h0_min_x_max_u: 0.367071986198
	test_h0_min_x_mean_u: 0.0451025255024
	test_h0_min_x_min_u: 1.38111693104e-06
	test_h0_row_norms_max: 5.97794675827
	test_h0_row_norms_mean: 3.01733326912
	test_h0_row_norms_min: 0.0475185476243
	test_objective: 0.2069362849
	test_y_col_norms_max: 4.37119436264
	test_y_col_norms_mean: 4.05648756027
	test_y_col_norms_min: 3.72235488892
	test_y_max_max_class: 0.999992549419
	test_y_mean_max_class: 0.920760273933
	test_y_min_max_class: 0.212535321712
	test_y_misclass: 0.0597999989986
	test_y_nll: 0.2069362849
	test_y_row_norms_max: 1.28081488609
	test_y_row_norms_mean: 0.54237049818
	test_y_row_norms_min: 0.120768107474
	train_h0_col_norms_max: 6.26188564301
	train_h0_col_norms_mean: 3.86070251465
	train_h0_col_norms_min: 2.09020781517
	train_h0_max_x_max_u: 0.999989151955
	train_h0_max_x_mean_u: 0.941006839275
	train_h0_max_x_min_u: 0.670265555382
	train_h0_mean_x_max_u: 0.880909919739
	train_h0_mean_x_mean_u: 0.471454769373
	train_h0_mean_x_min_u: 0.130571871996
	train_h0_min_x_max_u: 0.354819297791
	train_h0_min_x_mean_u: 0.0440064184368
	train_h0_min_x_min_u: 2.32596198657e-06
	train_h0_row_norms_max: 5.97794628143
	train_h0_row_norms_mean: 3.0173330307
	train_h0_row_norms_min: 0.0475185438991
	train_objective: 0.205675914884
	train_y_col_norms_max: 4.3711938858
	train_y_col_norms_mean: 4.05648708344
	train_y_col_norms_min: 3.72235488892
	train_y_max_max_class: 0.999997496605
	train_y_mean_max_class: 0.917994856834
	train_y_min_max_class: 0.242114007473
	train_y_misclass: 0.0586799941957
	train_y_nll: 0.205675914884
	train_y_row_norms_max: 1.2808150053
	train_y_row_norms_mean: 0.542370438576
	train_y_row_norms_min: 0.120768100023
	valid_h0_col_norms_max: 6.26188564301
	valid_h0_col_norms_mean: 3.86070275307
	valid_h0_col_norms_min: 2.09020781517
	valid_h0_max_x_max_u: 0.999994754791
	valid_h0_max_x_mean_u: 0.940389454365
	valid_h0_max_x_min_u: 0.653915822506
	valid_h0_mean_x_max_u: 0.885270357132
	valid_h0_mean_x_mean_u: 0.471503049135
	valid_h0_mean_x_min_u: 0.129038855433
	valid_h0_min_x_max_u: 0.343496620655
	valid_h0_min_x_mean_u: 0.0445692464709
	valid_h0_min_x_min_u: 1.89789943761e-06
	valid_h0_row_norms_max: 5.97794675827
	valid_h0_row_norms_mean: 3.01733326912
	valid_h0_row_norms_min: 0.0475185476243
	valid_objective: 0.199690312147
	valid_y_col_norms_max: 4.37119436264
	valid_y_col_norms_mean: 4.05648756027
	valid_y_col_norms_min: 3.72235488892
	valid_y_max_max_class: 0.999996244907
	valid_y_mean_max_class: 0.922058641911
	valid_y_min_max_class: 0.22336602211
	valid_y_misclass: 0.055799998343
	valid_y_nll: 0.199690312147
	valid_y_row_norms_max: 1.28081488609
	valid_y_row_norms_mean: 0.54237049818
	valid_y_row_norms_min: 0.120768107474
Time this epoch: 34.805092 seconds
Monitoring step:
	Epochs seen: 8
	Batches seen: 40
	Examples seen: 400000
	ave_grad_mult: 0.991648554802
	ave_grad_size: 0.0825677365065
	ave_step_size: 0.0698289051652
	test_h0_col_norms_max: 6.26615095139
	test_h0_col_norms_mean: 3.86642217636
	test_h0_col_norms_min: 2.0920112133
	test_h0_max_x_max_u: 0.999997377396
	test_h0_max_x_mean_u: 0.940795004368
	test_h0_max_x_min_u: 0.66545778513
	test_h0_mean_x_max_u: 0.901528179646
	test_h0_mean_x_mean_u: 0.470299869776
	test_h0_mean_x_min_u: 0.121718779206
	test_h0_min_x_max_u: 0.370387345552
	test_h0_min_x_mean_u: 0.0449309423566
	test_h0_min_x_min_u: 5.55576320949e-07
	test_h0_row_norms_max: 5.99863862991
	test_h0_row_norms_mean: 3.02230143547
	test_h0_row_norms_min: 0.0541109740734
	test_objective: 0.1924007833
	test_y_col_norms_max: 4.68016433716
	test_y_col_norms_mean: 4.25164651871
	test_y_col_norms_min: 3.82015967369
	test_y_max_max_class: 0.999988377094
	test_y_mean_max_class: 0.924113929272
	test_y_min_max_class: 0.210057422519
	test_y_misclass: 0.0555000007153
	test_y_nll: 0.1924007833
	test_y_row_norms_max: 1.36218941212
	test_y_row_norms_mean: 0.566706836224
	test_y_row_norms_min: 0.123096778989
	train_h0_col_norms_max: 6.26615047455
	train_h0_col_norms_mean: 3.86642193794
	train_h0_col_norms_min: 2.0920112133
	train_h0_max_x_max_u: 0.999993860722
	train_h0_max_x_mean_u: 0.941651582718
	train_h0_max_x_min_u: 0.657282650471
	train_h0_mean_x_max_u: 0.89084905386
	train_h0_mean_x_mean_u: 0.470575273037
	train_h0_mean_x_min_u: 0.124335050583
	train_h0_min_x_max_u: 0.357388138771
	train_h0_min_x_mean_u: 0.0438850969076
	train_h0_min_x_min_u: 9.09530456283e-07
	train_h0_row_norms_max: 5.99863910675
	train_h0_row_norms_mean: 3.02230119705
	train_h0_row_norms_min: 0.0541109666228
	train_objective: 0.187867701054
	train_y_col_norms_max: 4.680164814
	train_y_col_norms_mean: 4.25164651871
	train_y_col_norms_min: 3.82015943527
	train_y_max_max_class: 0.999996304512
	train_y_mean_max_class: 0.922073721886
	train_y_min_max_class: 0.237471118569
	train_y_misclass: 0.0530999973416
	train_y_nll: 0.187867701054
	train_y_row_norms_max: 1.36218929291
	train_y_row_norms_mean: 0.566706776619
	train_y_row_norms_min: 0.123096778989
	valid_h0_col_norms_max: 6.26615095139
	valid_h0_col_norms_mean: 3.86642217636
	valid_h0_col_norms_min: 2.0920112133
	valid_h0_max_x_max_u: 0.999996244907
	valid_h0_max_x_mean_u: 0.940959215164
	valid_h0_max_x_min_u: 0.634269952774
	valid_h0_mean_x_max_u: 0.894827961922
	valid_h0_mean_x_mean_u: 0.470626890659
	valid_h0_mean_x_min_u: 0.123129568994
	valid_h0_min_x_max_u: 0.344170331955
	valid_h0_min_x_mean_u: 0.0444831475616
	valid_h0_min_x_min_u: 7.30816509531e-07
	valid_h0_row_norms_max: 5.99863862991
	valid_h0_row_norms_mean: 3.02230143547
	valid_h0_row_norms_min: 0.0541109740734
	valid_objective: 0.184409946203
	valid_y_col_norms_max: 4.68016433716
	valid_y_col_norms_mean: 4.25164651871
	valid_y_col_norms_min: 3.82015967369
	valid_y_max_max_class: 0.999994754791
	valid_y_mean_max_class: 0.926723182201
	valid_y_min_max_class: 0.219980046153
	valid_y_misclass: 0.047499999404
	valid_y_nll: 0.184409946203
	valid_y_row_norms_max: 1.36218941212
	valid_y_row_norms_mean: 0.566706836224
	valid_y_row_norms_min: 0.123096778989
Time this epoch: 35.663056 seconds
Monitoring step:
	Epochs seen: 9
	Batches seen: 45
	Examples seen: 450000
	ave_grad_mult: 1.00632071495
	ave_grad_size: 0.0730155408382
	ave_step_size: 0.0651284307241
	test_h0_col_norms_max: 6.27027750015
	test_h0_col_norms_mean: 3.87175488472
	test_h0_col_norms_min: 2.09271168709
	test_h0_max_x_max_u: 0.99999833107
	test_h0_max_x_mean_u: 0.941553533077
	test_h0_max_x_min_u: 0.65441852808
	test_h0_mean_x_max_u: 0.903928875923
	test_h0_mean_x_mean_u: 0.469605773687
	test_h0_mean_x_min_u: 0.114903002977
	test_h0_min_x_max_u: 0.373793333769
	test_h0_min_x_mean_u: 0.044343251735
	test_h0_min_x_min_u: 2.48894650667e-07
	test_h0_row_norms_max: 6.01675319672
	test_h0_row_norms_mean: 3.0269382
	test_h0_row_norms_min: 0.0595724433661
	test_objective: 0.178400695324
	test_y_col_norms_max: 4.93448925018
	test_y_col_norms_mean: 4.4312376976
	test_y_col_norms_min: 3.912296772
	test_y_max_max_class: 0.99998986721
	test_y_mean_max_class: 0.929982662201
	test_y_min_max_class: 0.206445708871
	test_y_misclass: 0.0520999990404
	test_y_nll: 0.178400695324
	test_y_row_norms_max: 1.42163467407
	test_y_row_norms_mean: 0.588779568672
	test_y_row_norms_min: 0.124702431262
	train_h0_col_norms_max: 6.27027750015
	train_h0_col_norms_mean: 3.87175512314
	train_h0_col_norms_min: 2.09271168709
	train_h0_max_x_max_u: 0.999995946884
	train_h0_max_x_mean_u: 0.942493140697
	train_h0_max_x_min_u: 0.638945221901
	train_h0_mean_x_max_u: 0.893475353718
	train_h0_mean_x_mean_u: 0.469883978367
	train_h0_mean_x_min_u: 0.117275975645
	train_h0_min_x_max_u: 0.360578835011
	train_h0_min_x_mean_u: 0.0432931296527
	train_h0_min_x_min_u: 3.94163265582e-07
	train_h0_row_norms_max: 6.01675271988
	train_h0_row_norms_mean: 3.0269382
	train_h0_row_norms_min: 0.0595724396408
	train_objective: 0.173733517528
	train_y_col_norms_max: 4.93448877335
	train_y_col_norms_mean: 4.4312376976
	train_y_col_norms_min: 3.91229653358
	train_y_max_max_class: 0.999996066093
	train_y_mean_max_class: 0.92810434103
	train_y_min_max_class: 0.229242756963
	train_y_misclass: 0.0490399971604
	train_y_nll: 0.173733517528
	train_y_row_norms_max: 1.42163455486
	train_y_row_norms_mean: 0.588779509068
	train_y_row_norms_min: 0.124702423811
	valid_h0_col_norms_max: 6.27027750015
	valid_h0_col_norms_mean: 3.87175488472
	valid_h0_col_norms_min: 2.09271168709
	valid_h0_max_x_max_u: 0.999997377396
	valid_h0_max_x_mean_u: 0.941749632359
	valid_h0_max_x_min_u: 0.622578442097
	valid_h0_mean_x_max_u: 0.897465348244
	valid_h0_mean_x_mean_u: 0.469932496548
	valid_h0_mean_x_min_u: 0.116939790547
	valid_h0_min_x_max_u: 0.347404718399
	valid_h0_min_x_mean_u: 0.0439214892685
	valid_h0_min_x_min_u: 3.13890211601e-07
	valid_h0_row_norms_max: 6.01675319672
	valid_h0_row_norms_mean: 3.0269382
	valid_h0_row_norms_min: 0.0595724433661
	valid_objective: 0.172197133303
	valid_y_col_norms_max: 4.93448925018
	valid_y_col_norms_mean: 4.4312376976
	valid_y_col_norms_min: 3.912296772
	valid_y_max_max_class: 0.999996781349
	valid_y_mean_max_class: 0.932501792908
	valid_y_min_max_class: 0.216077208519
	valid_y_misclass: 0.0454999953508
	valid_y_nll: 0.172197133303
	valid_y_row_norms_max: 1.42163467407
	valid_y_row_norms_mean: 0.588779568672
	valid_y_row_norms_min: 0.124702431262
Time this epoch: 35.404834 seconds
Monitoring step:
	Epochs seen: 10
	Batches seen: 50
	Examples seen: 500000
	ave_grad_mult: 1.06833612919
	ave_grad_size: 0.0678643658757
	ave_step_size: 0.0653440654278
	test_h0_col_norms_max: 6.27522420883
	test_h0_col_norms_mean: 3.87793588638
	test_h0_col_norms_min: 2.09417295456
	test_h0_max_x_max_u: 0.999998867512
	test_h0_max_x_mean_u: 0.942130804062
	test_h0_max_x_min_u: 0.645175695419
	test_h0_mean_x_max_u: 0.909636974335
	test_h0_mean_x_mean_u: 0.468845933676
	test_h0_mean_x_min_u: 0.104815065861
	test_h0_min_x_max_u: 0.378569096327
	test_h0_min_x_mean_u: 0.0440588444471
	test_h0_min_x_min_u: 1.15133666156e-07
	test_h0_row_norms_max: 6.03866481781
	test_h0_row_norms_mean: 3.03230404854
	test_h0_row_norms_min: 0.065353885293
	test_objective: 0.167283341289
	test_y_col_norms_max: 5.2253780365
	test_y_col_norms_mean: 4.62542486191
	test_y_col_norms_min: 4.01688957214
	test_y_max_max_class: 0.999992907047
	test_y_mean_max_class: 0.933511257172
	test_y_min_max_class: 0.242168530822
	test_y_misclass: 0.0492999963462
	test_y_nll: 0.167283341289
	test_y_row_norms_max: 1.50107598305
	test_y_row_norms_mean: 0.612406551838
	test_y_row_norms_min: 0.125712171197
	train_h0_col_norms_max: 6.27522373199
	train_h0_col_norms_mean: 3.87793540955
	train_h0_col_norms_min: 2.09417295456
	train_h0_max_x_max_u: 0.999997496605
	train_h0_max_x_mean_u: 0.943212330341
	train_h0_max_x_min_u: 0.628583967686
	train_h0_mean_x_max_u: 0.899803757668
	train_h0_mean_x_mean_u: 0.469121694565
	train_h0_mean_x_min_u: 0.107625767589
	train_h0_min_x_max_u: 0.36565092206
	train_h0_min_x_mean_u: 0.0430302321911
	train_h0_min_x_min_u: 1.75549445203e-07
	train_h0_row_norms_max: 6.03866481781
	train_h0_row_norms_mean: 3.03230404854
	train_h0_row_norms_min: 0.065353885293
	train_objective: 0.159167990088
	train_y_col_norms_max: 5.22537755966
	train_y_col_norms_mean: 4.62542486191
	train_y_col_norms_min: 4.01688957214
	train_y_max_max_class: 0.999997138977
	train_y_mean_max_class: 0.931973934174
	train_y_min_max_class: 0.241810530424
	train_y_misclass: 0.0449799969792
	train_y_nll: 0.159167990088
	train_y_row_norms_max: 1.50107610226
	train_y_row_norms_mean: 0.612406492233
	train_y_row_norms_min: 0.125712156296
	valid_h0_col_norms_max: 6.27522420883
	valid_h0_col_norms_mean: 3.87793588638
	valid_h0_col_norms_min: 2.09417295456
	valid_h0_max_x_max_u: 0.999998152256
	valid_h0_max_x_mean_u: 0.942497193813
	valid_h0_max_x_min_u: 0.619423508644
	valid_h0_mean_x_max_u: 0.903488636017
	valid_h0_mean_x_mean_u: 0.469177812338
	valid_h0_mean_x_min_u: 0.108095638454
	valid_h0_min_x_max_u: 0.349716216326
	valid_h0_min_x_mean_u: 0.04355686903
	valid_h0_min_x_min_u: 1.34484281489e-07
	valid_h0_row_norms_max: 6.03866481781
	valid_h0_row_norms_mean: 3.03230404854
	valid_h0_row_norms_min: 0.065353885293
	valid_objective: 0.160998404026
	valid_y_col_norms_max: 5.2253780365
	valid_y_col_norms_mean: 4.62542486191
	valid_y_col_norms_min: 4.01688957214
	valid_y_max_max_class: 0.999998152256
	valid_y_mean_max_class: 0.936175227165
	valid_y_min_max_class: 0.220791786909
	valid_y_misclass: 0.0441000014544
	valid_y_nll: 0.160998404026
	valid_y_row_norms_max: 1.50107598305
	valid_y_row_norms_mean: 0.612406551838
	valid_y_row_norms_min: 0.125712171197
Time this epoch: 35.425083 seconds
Monitoring step:
	Epochs seen: 11
	Batches seen: 55
	Examples seen: 550000
	ave_grad_mult: 1.14648592472
	ave_grad_size: 0.0634888410568
	ave_step_size: 0.0666681230068
	test_h0_col_norms_max: 6.28032588959
	test_h0_col_norms_mean: 3.88434314728
	test_h0_col_norms_min: 2.09576916695
	test_h0_max_x_max_u: 0.999999403954
	test_h0_max_x_mean_u: 0.942800343037
	test_h0_max_x_min_u: 0.63667178154
	test_h0_mean_x_max_u: 0.915984809399
	test_h0_mean_x_mean_u: 0.468008965254
	test_h0_mean_x_min_u: 0.101051539183
	test_h0_min_x_max_u: 0.390204340219
	test_h0_min_x_mean_u: 0.0434938073158
	test_h0_min_x_min_u: 5.66224080956e-08
	test_h0_row_norms_max: 6.06034469604
	test_h0_row_norms_mean: 3.03785538673
	test_h0_row_norms_min: 0.0704936757684
	test_objective: 0.15456405282
	test_y_col_norms_max: 5.50095510483
	test_y_col_norms_mean: 4.82304191589
	test_y_col_norms_min: 4.1173620224
	test_y_max_max_class: 0.999992728233
	test_y_mean_max_class: 0.936915397644
	test_y_min_max_class: 0.252786010504
	test_y_misclass: 0.0443000011146
	test_y_nll: 0.15456405282
	test_y_row_norms_max: 1.60092997551
	test_y_row_norms_mean: 0.636273026466
	test_y_row_norms_min: 0.124862372875
	train_h0_col_norms_max: 6.28032636642
	train_h0_col_norms_mean: 3.88434290886
	train_h0_col_norms_min: 2.09576892853
	train_h0_max_x_max_u: 0.999998629093
	train_h0_max_x_mean_u: 0.944033026695
	train_h0_max_x_min_u: 0.631079792976
	train_h0_mean_x_max_u: 0.90686249733
	train_h0_mean_x_mean_u: 0.468293100595
	train_h0_mean_x_min_u: 0.103928506374
	train_h0_min_x_max_u: 0.373679548502
	train_h0_min_x_mean_u: 0.0424839258194
	train_h0_min_x_min_u: 8.3652395233e-08
	train_h0_row_norms_max: 6.06034517288
	train_h0_row_norms_mean: 3.03785514832
	train_h0_row_norms_min: 0.0704936683178
	train_objective: 0.146077007055
	train_y_col_norms_max: 5.50095510483
	train_y_col_norms_mean: 4.82304239273
	train_y_col_norms_min: 4.11736249924
	train_y_max_max_class: 0.999997377396
	train_y_mean_max_class: 0.935088992119
	train_y_min_max_class: 0.235717624426
	train_y_misclass: 0.0411599949002
	train_y_nll: 0.146077007055
	train_y_row_norms_max: 1.60092973709
	train_y_row_norms_mean: 0.636273086071
	train_y_row_norms_min: 0.124862357974
	valid_h0_col_norms_max: 6.28032588959
	valid_h0_col_norms_mean: 3.88434314728
	valid_h0_col_norms_min: 2.09576916695
	valid_h0_max_x_max_u: 0.999998867512
	valid_h0_max_x_mean_u: 0.943427741528
	valid_h0_max_x_min_u: 0.627752363682
	valid_h0_mean_x_max_u: 0.910161554813
	valid_h0_mean_x_mean_u: 0.468341171741
	valid_h0_mean_x_min_u: 0.104510381818
	valid_h0_min_x_max_u: 0.357529014349
	valid_h0_min_x_mean_u: 0.0429090820253
	valid_h0_min_x_min_u: 6.19904838572e-08
	valid_h0_row_norms_max: 6.06034469604
	valid_h0_row_norms_mean: 3.03785538673
	valid_h0_row_norms_min: 0.0704936757684
	valid_objective: 0.149976089597
	valid_y_col_norms_max: 5.50095510483
	valid_y_col_norms_mean: 4.82304191589
	valid_y_col_norms_min: 4.1173620224
	valid_y_max_max_class: 0.999998509884
	valid_y_mean_max_class: 0.939062952995
	valid_y_min_max_class: 0.239928662777
	valid_y_misclass: 0.0416000001132
	valid_y_nll: 0.149976089597
	valid_y_row_norms_max: 1.60092997551
	valid_y_row_norms_mean: 0.636273026466
	valid_y_row_norms_min: 0.124862372875
Time this epoch: 35.174293 seconds
Monitoring step:
	Epochs seen: 12
	Batches seen: 60
	Examples seen: 600000
	ave_grad_mult: 1.16790962219
	ave_grad_size: 0.0593062080443
	ave_step_size: 0.0650760680437
	test_h0_col_norms_max: 6.28521823883
	test_h0_col_norms_mean: 3.89019036293
	test_h0_col_norms_min: 2.09752202034
	test_h0_max_x_max_u: 0.999999582767
	test_h0_max_x_mean_u: 0.94396853447
	test_h0_max_x_min_u: 0.629440486431
	test_h0_mean_x_max_u: 0.920006334782
	test_h0_mean_x_mean_u: 0.467411011457
	test_h0_mean_x_min_u: 0.0957048162818
	test_h0_min_x_max_u: 0.389512062073
	test_h0_min_x_mean_u: 0.0425479598343
	test_h0_min_x_min_u: 3.20807167498e-08
	test_h0_row_norms_max: 6.08012914658
	test_h0_row_norms_mean: 3.04289364815
	test_h0_row_norms_min: 0.0743318274617
	test_objective: 0.144802451134
	test_y_col_norms_max: 5.74849033356
	test_y_col_norms_mean: 5.00328540802
	test_y_col_norms_min: 4.21305179596
	test_y_max_max_class: 0.999994158745
	test_y_mean_max_class: 0.941429018974
	test_y_min_max_class: 0.231030538678
	test_y_misclass: 0.0408000014722
	test_y_nll: 0.144802451134
	test_y_row_norms_max: 1.7184125185
	test_y_row_norms_mean: 0.658156752586
	test_y_row_norms_min: 0.125041946769
	train_h0_col_norms_max: 6.28521871567
	train_h0_col_norms_mean: 3.89019012451
	train_h0_col_norms_min: 2.09752202034
	train_h0_max_x_max_u: 0.999999046326
	train_h0_max_x_mean_u: 0.945232570171
	train_h0_max_x_min_u: 0.634238958359
	train_h0_mean_x_max_u: 0.911378622055
	train_h0_mean_x_mean_u: 0.467698544264
	train_h0_mean_x_min_u: 0.0993719547987
	train_h0_min_x_max_u: 0.373709738255
	train_h0_min_x_mean_u: 0.0415380932391
	train_h0_min_x_min_u: 4.67483047828e-08
	train_h0_row_norms_max: 6.08012914658
	train_h0_row_norms_mean: 3.04289340973
	train_h0_row_norms_min: 0.0743318200111
	train_objective: 0.135217413306
	train_y_col_norms_max: 5.74849033356
	train_y_col_norms_mean: 5.00328493118
	train_y_col_norms_min: 4.21305131912
	train_y_max_max_class: 0.999997973442
	train_y_mean_max_class: 0.939604878426
	train_y_min_max_class: 0.252161383629
	train_y_misclass: 0.0378999970853
	train_y_nll: 0.135217413306
	train_y_row_norms_max: 1.71841263771
	train_y_row_norms_mean: 0.658156752586
	train_y_row_norms_min: 0.125041931868
	valid_h0_col_norms_max: 6.28521823883
	valid_h0_col_norms_mean: 3.89019036293
	valid_h0_col_norms_min: 2.09752202034
	valid_h0_max_x_max_u: 0.99999922514
	valid_h0_max_x_mean_u: 0.944651842117
	valid_h0_max_x_min_u: 0.645568966866
	valid_h0_mean_x_max_u: 0.914412498474
	valid_h0_mean_x_mean_u: 0.467748105526
	valid_h0_mean_x_min_u: 0.0972835198045
	valid_h0_min_x_max_u: 0.359061449766
	valid_h0_min_x_mean_u: 0.0419331230223
	valid_h0_min_x_min_u: 3.37856427279e-08
	valid_h0_row_norms_max: 6.08012914658
	valid_h0_row_norms_mean: 3.04289364815
	valid_h0_row_norms_min: 0.0743318274617
	valid_objective: 0.141469165683
	valid_y_col_norms_max: 5.74849033356
	valid_y_col_norms_mean: 5.00328540802
	valid_y_col_norms_min: 4.21305179596
	valid_y_max_max_class: 0.999998867512
	valid_y_mean_max_class: 0.943582773209
	valid_y_min_max_class: 0.241308540106
	valid_y_misclass: 0.0379000008106
	valid_y_nll: 0.141469165683
	valid_y_row_norms_max: 1.7184125185
	valid_y_row_norms_mean: 0.658156752586
	valid_y_row_norms_min: 0.125041946769
Time this epoch: 35.417259 seconds
Monitoring step:
	Epochs seen: 13
	Batches seen: 65
	Examples seen: 650000
	ave_grad_mult: 1.26017534733
	ave_grad_size: 0.0564817748964
	ave_step_size: 0.066411331296
	test_h0_col_norms_max: 6.29147386551
	test_h0_col_norms_mean: 3.89687585831
	test_h0_col_norms_min: 2.09867763519
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.944756031036
	test_h0_max_x_min_u: 0.627687215805
	test_h0_mean_x_max_u: 0.920987069607
	test_h0_mean_x_mean_u: 0.46668151021
	test_h0_mean_x_min_u: 0.0932114943862
	test_h0_min_x_max_u: 0.397424399853
	test_h0_min_x_mean_u: 0.041878964752
	test_h0_min_x_min_u: 1.39939251298e-08
	test_h0_row_norms_max: 6.10229206085
	test_h0_row_norms_mean: 3.04865932465
	test_h0_row_norms_min: 0.0787999555469
	test_objective: 0.135829210281
	test_y_col_norms_max: 6.01105213165
	test_y_col_norms_mean: 5.20304250717
	test_y_col_norms_min: 4.33085250854
	test_y_max_max_class: 0.999994754791
	test_y_mean_max_class: 0.945015072823
	test_y_min_max_class: 0.230172082782
	test_y_misclass: 0.0381999947131
	test_y_nll: 0.135829210281
	test_y_row_norms_max: 1.85168874264
	test_y_row_norms_mean: 0.682141900063
	test_y_row_norms_min: 0.125363498926
	train_h0_col_norms_max: 6.29147338867
	train_h0_col_norms_mean: 3.89687561989
	train_h0_col_norms_min: 2.09867739677
	train_h0_max_x_max_u: 0.999999403954
	train_h0_max_x_mean_u: 0.946107804775
	train_h0_max_x_min_u: 0.63179987669
	train_h0_mean_x_max_u: 0.912519574165
	train_h0_mean_x_mean_u: 0.466963618994
	train_h0_mean_x_min_u: 0.0961530357599
	train_h0_min_x_max_u: 0.379027783871
	train_h0_min_x_mean_u: 0.0408683530986
	train_h0_min_x_min_u: 1.94427727251e-08
	train_h0_row_norms_max: 6.10229253769
	train_h0_row_norms_mean: 3.04865932465
	train_h0_row_norms_min: 0.0787999555469
	train_objective: 0.12386597693
	train_y_col_norms_max: 6.01105213165
	train_y_col_norms_mean: 5.20304203033
	train_y_col_norms_min: 4.33085203171
	train_y_max_max_class: 0.999997973442
	train_y_mean_max_class: 0.943518102169
	train_y_min_max_class: 0.246507614851
	train_y_misclass: 0.034559994936
	train_y_nll: 0.12386597693
	train_y_row_norms_max: 1.85168862343
	train_y_row_norms_mean: 0.682141840458
	train_y_row_norms_min: 0.125363498926
	valid_h0_col_norms_max: 6.29147386551
	valid_h0_col_norms_mean: 3.89687585831
	valid_h0_col_norms_min: 2.09867763519
	valid_h0_max_x_max_u: 0.999999403954
	valid_h0_max_x_mean_u: 0.945632517338
	valid_h0_max_x_min_u: 0.651219964027
	valid_h0_mean_x_max_u: 0.915507853031
	valid_h0_mean_x_mean_u: 0.467023015022
	valid_h0_mean_x_min_u: 0.0969914197922
	valid_h0_min_x_max_u: 0.364903271198
	valid_h0_min_x_mean_u: 0.0411523580551
	valid_h0_min_x_min_u: 1.40916096569e-08
	valid_h0_row_norms_max: 6.10229206085
	valid_h0_row_norms_mean: 3.04865932465
	valid_h0_row_norms_min: 0.0787999555469
	valid_objective: 0.133389517665
	valid_y_col_norms_max: 6.01105213165
	valid_y_col_norms_mean: 5.20304250717
	valid_y_col_norms_min: 4.33085250854
	valid_y_max_max_class: 0.999999165535
	valid_y_mean_max_class: 0.946852385998
	valid_y_min_max_class: 0.214304342866
	valid_y_misclass: 0.03579999879
	valid_y_nll: 0.133389517665
	valid_y_row_norms_max: 1.85168874264
	valid_y_row_norms_mean: 0.682141900063
	valid_y_row_norms_min: 0.125363498926
Time this epoch: 35.366187 seconds
Monitoring step:
	Epochs seen: 14
	Batches seen: 70
	Examples seen: 700000
	ave_grad_mult: 1.40761697292
	ave_grad_size: 0.0550340935588
	ave_step_size: 0.0714166760445
	test_h0_col_norms_max: 6.29854393005
	test_h0_col_norms_mean: 3.90459442139
	test_h0_col_norms_min: 2.1004254818
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.94591987133
	test_h0_max_x_min_u: 0.614766418934
	test_h0_mean_x_max_u: 0.921078026295
	test_h0_mean_x_mean_u: 0.466091096401
	test_h0_mean_x_min_u: 0.0916782915592
	test_h0_min_x_max_u: 0.396448850632
	test_h0_min_x_mean_u: 0.0410171151161
	test_h0_min_x_min_u: 7.75897479599e-09
	test_h0_row_norms_max: 6.12455701828
	test_h0_row_norms_mean: 3.05531406403
	test_h0_row_norms_min: 0.0834630578756
	test_objective: 0.125504016876
	test_y_col_norms_max: 6.29601860046
	test_y_col_norms_mean: 5.42890501022
	test_y_col_norms_min: 4.46609354019
	test_y_max_max_class: 0.999994575977
	test_y_mean_max_class: 0.949410498142
	test_y_min_max_class: 0.201501131058
	test_y_misclass: 0.0355000011623
	test_y_nll: 0.125504016876
	test_y_row_norms_max: 2.01360034943
	test_y_row_norms_mean: 0.709331393242
	test_y_row_norms_min: 0.125074863434
	train_h0_col_norms_max: 6.29854393005
	train_h0_col_norms_mean: 3.90459418297
	train_h0_col_norms_min: 2.10042524338
	train_h0_max_x_max_u: 0.999999523163
	train_h0_max_x_mean_u: 0.947336554527
	train_h0_max_x_min_u: 0.624508261681
	train_h0_mean_x_max_u: 0.912684559822
	train_h0_mean_x_mean_u: 0.466372013092
	train_h0_mean_x_min_u: 0.0946839675307
	train_h0_min_x_max_u: 0.383265286684
	train_h0_min_x_mean_u: 0.0400523841381
	train_h0_min_x_min_u: 1.04573256721e-08
	train_h0_row_norms_max: 6.12455654144
	train_h0_row_norms_mean: 3.05531382561
	train_h0_row_norms_min: 0.0834630504251
	train_objective: 0.112524747849
	train_y_col_norms_max: 6.29601955414
	train_y_col_norms_mean: 5.42890501022
	train_y_col_norms_min: 4.46609306335
	train_y_max_max_class: 0.999997973442
	train_y_mean_max_class: 0.948245584965
	train_y_min_max_class: 0.237888276577
	train_y_misclass: 0.031159998849
	train_y_nll: 0.112524747849
	train_y_row_norms_max: 2.01360034943
	train_y_row_norms_mean: 0.709331333637
	train_y_row_norms_min: 0.125074848533
	valid_h0_col_norms_max: 6.29854393005
	valid_h0_col_norms_mean: 3.90459442139
	valid_h0_col_norms_min: 2.1004254818
	valid_h0_max_x_max_u: 0.999999582767
	valid_h0_max_x_mean_u: 0.946813523769
	valid_h0_max_x_min_u: 0.649647653103
	valid_h0_mean_x_max_u: 0.915705919266
	valid_h0_mean_x_mean_u: 0.466423898935
	valid_h0_mean_x_min_u: 0.0953802764416
	valid_h0_min_x_max_u: 0.369607925415
	valid_h0_min_x_mean_u: 0.0402967631817
	valid_h0_min_x_min_u: 7.3467920636e-09
	valid_h0_row_norms_max: 6.12455701828
	valid_h0_row_norms_mean: 3.05531406403
	valid_h0_row_norms_min: 0.0834630578756
	valid_objective: 0.124651312828
	valid_y_col_norms_max: 6.29601860046
	valid_y_col_norms_mean: 5.42890501022
	valid_y_col_norms_min: 4.46609354019
	valid_y_max_max_class: 0.999999046326
	valid_y_mean_max_class: 0.950519561768
	valid_y_min_max_class: 0.27420938015
	valid_y_misclass: 0.0340000018477
	valid_y_nll: 0.124651312828
	valid_y_row_norms_max: 2.01360034943
	valid_y_row_norms_mean: 0.709331393242
	valid_y_row_norms_min: 0.125074863434
Time this epoch: 35.379965 seconds
Monitoring step:
	Epochs seen: 15
	Batches seen: 75
	Examples seen: 750000
	ave_grad_mult: 1.47251427174
	ave_grad_size: 0.0522134304047
	ave_step_size: 0.071938700974
	test_h0_col_norms_max: 6.30543804169
	test_h0_col_norms_mean: 3.91161727905
	test_h0_col_norms_min: 2.10170149803
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.947175860405
	test_h0_max_x_min_u: 0.611504435539
	test_h0_mean_x_max_u: 0.926122069359
	test_h0_mean_x_mean_u: 0.466326773167
	test_h0_mean_x_min_u: 0.0923119410872
	test_h0_min_x_max_u: 0.401742935181
	test_h0_min_x_mean_u: 0.040495429188
	test_h0_min_x_min_u: 3.92714794017e-09
	test_h0_row_norms_max: 6.1450252533
	test_h0_row_norms_mean: 3.0613322258
	test_h0_row_norms_min: 0.0881084352732
	test_objective: 0.118968196213
	test_y_col_norms_max: 6.55995035172
	test_y_col_norms_mean: 5.62976980209
	test_y_col_norms_min: 4.59543800354
	test_y_max_max_class: 0.999994218349
	test_y_mean_max_class: 0.951547503471
	test_y_min_max_class: 0.228803291917
	test_y_misclass: 0.0349999964237
	test_y_nll: 0.118968196213
	test_y_row_norms_max: 2.14034724236
	test_y_row_norms_mean: 0.733418226242
	test_y_row_norms_min: 0.12729588151
	train_h0_col_norms_max: 6.30543756485
	train_h0_col_norms_mean: 3.91161704063
	train_h0_col_norms_min: 2.10170149803
	train_h0_max_x_max_u: 0.999999880791
	train_h0_max_x_mean_u: 0.948555886745
	train_h0_max_x_min_u: 0.618095517159
	train_h0_mean_x_max_u: 0.918341517448
	train_h0_mean_x_mean_u: 0.466599404812
	train_h0_mean_x_min_u: 0.0956889539957
	train_h0_min_x_max_u: 0.392006248236
	train_h0_min_x_mean_u: 0.0395230464637
	train_h0_min_x_min_u: 5.12299225264e-09
	train_h0_row_norms_max: 6.14502477646
	train_h0_row_norms_mean: 3.06133174896
	train_h0_row_norms_min: 0.088108420372
	train_objective: 0.103299617767
	train_y_col_norms_max: 6.55995082855
	train_y_col_norms_mean: 5.62976932526
	train_y_col_norms_min: 4.5954375267
	train_y_max_max_class: 0.999997377396
	train_y_mean_max_class: 0.951093494892
	train_y_min_max_class: 0.249234974384
	train_y_misclass: 0.0284799989313
	train_y_nll: 0.103299617767
	train_y_row_norms_max: 2.14034700394
	train_y_row_norms_mean: 0.733418226242
	train_y_row_norms_min: 0.12729588151
	valid_h0_col_norms_max: 6.30543804169
	valid_h0_col_norms_mean: 3.91161727905
	valid_h0_col_norms_min: 2.10170149803
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.947986066341
	valid_h0_max_x_min_u: 0.642793953419
	valid_h0_mean_x_max_u: 0.920964062214
	valid_h0_mean_x_mean_u: 0.466656267643
	valid_h0_mean_x_min_u: 0.0940045118332
	valid_h0_min_x_max_u: 0.377251118422
	valid_h0_min_x_mean_u: 0.03970798105
	valid_h0_min_x_min_u: 3.59755203405e-09
	valid_h0_row_norms_max: 6.1450252533
	valid_h0_row_norms_mean: 3.0613322258
	valid_h0_row_norms_min: 0.0881084352732
	valid_objective: 0.119057364762
	valid_y_col_norms_max: 6.55995035172
	valid_y_col_norms_mean: 5.62976980209
	valid_y_col_norms_min: 4.59543800354
	valid_y_max_max_class: 0.999998807907
	valid_y_mean_max_class: 0.953496754169
	valid_y_min_max_class: 0.279151201248
	valid_y_misclass: 0.0322999954224
	valid_y_nll: 0.119057364762
	valid_y_row_norms_max: 2.14034724236
	valid_y_row_norms_mean: 0.733418226242
	valid_y_row_norms_min: 0.12729588151
Time this epoch: 35.163641 seconds
Monitoring step:
	Epochs seen: 16
	Batches seen: 80
	Examples seen: 800000
	ave_grad_mult: 1.55044400692
	ave_grad_size: 0.0495749413967
	ave_step_size: 0.071437291801
	test_h0_col_norms_max: 6.31254959106
	test_h0_col_norms_mean: 3.91860723495
	test_h0_col_norms_min: 2.10440206528
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.948059678078
	test_h0_max_x_min_u: 0.594348907471
	test_h0_mean_x_max_u: 0.927690863609
	test_h0_mean_x_mean_u: 0.465816915035
	test_h0_mean_x_min_u: 0.0875093266368
	test_h0_min_x_max_u: 0.399484992027
	test_h0_min_x_mean_u: 0.0397286936641
	test_h0_min_x_min_u: 2.14466333581e-09
	test_h0_row_norms_max: 6.16418838501
	test_h0_row_norms_mean: 3.06732678413
	test_h0_row_norms_min: 0.091041892767
	test_objective: 0.111787736416
	test_y_col_norms_max: 6.79865264893
	test_y_col_norms_mean: 5.8274474144
	test_y_col_norms_min: 4.71656274796
	test_y_max_max_class: 0.999996483326
	test_y_mean_max_class: 0.954726696014
	test_y_min_max_class: 0.287018150091
	test_y_misclass: 0.0328000001609
	test_y_nll: 0.111787736416
	test_y_row_norms_max: 2.27131104469
	test_y_row_norms_mean: 0.757337749004
	test_y_row_norms_min: 0.12875507772
	train_h0_col_norms_max: 6.31254959106
	train_h0_col_norms_mean: 3.91860699654
	train_h0_col_norms_min: 2.10440182686
	train_h0_max_x_max_u: 0.999999880791
	train_h0_max_x_mean_u: 0.949532628059
	train_h0_max_x_min_u: 0.603366672993
	train_h0_mean_x_max_u: 0.920095324516
	train_h0_mean_x_mean_u: 0.466088950634
	train_h0_mean_x_min_u: 0.0905980989337
	train_h0_min_x_max_u: 0.391806066036
	train_h0_min_x_mean_u: 0.0387711115181
	train_h0_min_x_min_u: 2.82344658764e-09
	train_h0_row_norms_max: 6.16418838501
	train_h0_row_norms_mean: 3.06732654572
	train_h0_row_norms_min: 0.0910418853164
	train_objective: 0.0944318547845
	train_y_col_norms_max: 6.79865264893
	train_y_col_norms_mean: 5.82744646072
	train_y_col_norms_min: 4.71656322479
	train_y_max_max_class: 0.999998569489
	train_y_mean_max_class: 0.954577803612
	train_y_min_max_class: 0.255649060011
	train_y_misclass: 0.0261199977249
	train_y_nll: 0.0944318547845
	train_y_row_norms_max: 2.27131080627
	train_y_row_norms_mean: 0.757337749004
	train_y_row_norms_min: 0.128755062819
	valid_h0_col_norms_max: 6.31254959106
	valid_h0_col_norms_mean: 3.91860723495
	valid_h0_col_norms_min: 2.10440206528
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.948857724667
	valid_h0_max_x_min_u: 0.6251745224
	valid_h0_mean_x_max_u: 0.922635018826
	valid_h0_mean_x_mean_u: 0.46614703536
	valid_h0_mean_x_min_u: 0.0918302312493
	valid_h0_min_x_max_u: 0.379304587841
	valid_h0_min_x_mean_u: 0.0389546044171
	valid_h0_min_x_min_u: 1.96204941183e-09
	valid_h0_row_norms_max: 6.16418838501
	valid_h0_row_norms_mean: 3.06732678413
	valid_h0_row_norms_min: 0.091041892767
	valid_objective: 0.110771089792
	valid_y_col_norms_max: 6.79865264893
	valid_y_col_norms_mean: 5.8274474144
	valid_y_col_norms_min: 4.71656274796
	valid_y_max_max_class: 0.999999165535
	valid_y_mean_max_class: 0.95663100481
	valid_y_min_max_class: 0.264041811228
	valid_y_misclass: 0.0305000003427
	valid_y_nll: 0.110771089792
	valid_y_row_norms_max: 2.27131104469
	valid_y_row_norms_mean: 0.757337749004
	valid_y_row_norms_min: 0.12875507772
Time this epoch: 35.246666 seconds
Monitoring step:
	Epochs seen: 17
	Batches seen: 85
	Examples seen: 850000
	ave_grad_mult: 1.59982562065
	ave_grad_size: 0.0473937280476
	ave_step_size: 0.0712730288506
	test_h0_col_norms_max: 6.31961965561
	test_h0_col_norms_mean: 3.92528343201
	test_h0_col_norms_min: 2.10622811317
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.949356853962
	test_h0_max_x_min_u: 0.59099650383
	test_h0_mean_x_max_u: 0.927693426609
	test_h0_mean_x_mean_u: 0.465647280216
	test_h0_mean_x_min_u: 0.0867232903838
	test_h0_min_x_max_u: 0.39404541254
	test_h0_min_x_mean_u: 0.0387796163559
	test_h0_min_x_min_u: 1.4791411429e-09
	test_h0_row_norms_max: 6.18163251877
	test_h0_row_norms_mean: 3.07301926613
	test_h0_row_norms_min: 0.0938726961613
	test_objective: 0.106328338385
	test_y_col_norms_max: 7.01830482483
	test_y_col_norms_mean: 6.0149974823
	test_y_col_norms_min: 4.83683490753
	test_y_max_max_class: 0.999997198582
	test_y_mean_max_class: 0.95773011446
	test_y_min_max_class: 0.291382759809
	test_y_misclass: 0.0320000015199
	test_y_nll: 0.106328338385
	test_y_row_norms_max: 2.38739275932
	test_y_row_norms_mean: 0.780075967312
	test_y_row_norms_min: 0.130353063345
	train_h0_col_norms_max: 6.31961917877
	train_h0_col_norms_mean: 3.92528319359
	train_h0_col_norms_min: 2.10622787476
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.950781822205
	train_h0_max_x_min_u: 0.600085794926
	train_h0_mean_x_max_u: 0.920143485069
	train_h0_mean_x_mean_u: 0.465922415257
	train_h0_mean_x_min_u: 0.0898449495435
	train_h0_min_x_max_u: 0.391092181206
	train_h0_min_x_mean_u: 0.0378985367715
	train_h0_min_x_min_u: 1.99124361444e-09
	train_h0_row_norms_max: 6.18163204193
	train_h0_row_norms_mean: 3.07301878929
	train_h0_row_norms_min: 0.0938726961613
	train_objective: 0.088271394372
	train_y_col_norms_max: 7.01830387115
	train_y_col_norms_mean: 6.0149974823
	train_y_col_norms_min: 4.83683490753
	train_y_max_max_class: 0.999998629093
	train_y_mean_max_class: 0.957574307919
	train_y_min_max_class: 0.276376664639
	train_y_misclass: 0.023999998346
	train_y_nll: 0.088271394372
	train_y_row_norms_max: 2.3873925209
	train_y_row_norms_mean: 0.780075907707
	train_y_row_norms_min: 0.130353048444
	valid_h0_col_norms_max: 6.31961965561
	valid_h0_col_norms_mean: 3.92528343201
	valid_h0_col_norms_min: 2.10622811317
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.950090706348
	valid_h0_max_x_min_u: 0.620329141617
	valid_h0_mean_x_max_u: 0.922675073147
	valid_h0_mean_x_mean_u: 0.465976387262
	valid_h0_mean_x_min_u: 0.0912392660975
	valid_h0_min_x_max_u: 0.378798425198
	valid_h0_min_x_mean_u: 0.0380813851953
	valid_h0_min_x_min_u: 1.35891120578e-09
	valid_h0_row_norms_max: 6.18163251877
	valid_h0_row_norms_mean: 3.07301926613
	valid_h0_row_norms_min: 0.0938726961613
	valid_objective: 0.107352338731
	valid_y_col_norms_max: 7.01830482483
	valid_y_col_norms_mean: 6.0149974823
	valid_y_col_norms_min: 4.83683490753
	valid_y_max_max_class: 0.999998867512
	valid_y_mean_max_class: 0.959039092064
	valid_y_min_max_class: 0.278402447701
	valid_y_misclass: 0.0296999998391
	valid_y_nll: 0.107352338731
	valid_y_row_norms_max: 2.38739275932
	valid_y_row_norms_mean: 0.780075967312
	valid_y_row_norms_min: 0.130353063345
Time this epoch: 35.302343 seconds
Monitoring step:
	Epochs seen: 18
	Batches seen: 90
	Examples seen: 900000
	ave_grad_mult: 1.79280376434
	ave_grad_size: 0.0464615598321
	ave_step_size: 0.0771328359842
	test_h0_col_norms_max: 6.32822799683
	test_h0_col_norms_mean: 3.93359160423
	test_h0_col_norms_min: 2.10832476616
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.95045799017
	test_h0_max_x_min_u: 0.586617648602
	test_h0_mean_x_max_u: 0.929918944836
	test_h0_mean_x_mean_u: 0.465379714966
	test_h0_mean_x_min_u: 0.083888605237
	test_h0_min_x_max_u: 0.397964477539
	test_h0_min_x_mean_u: 0.0380010083318
	test_h0_min_x_min_u: 7.37118366345e-10
	test_h0_row_norms_max: 6.20447731018
	test_h0_row_norms_mean: 3.08011174202
	test_h0_row_norms_min: 0.0980293303728
	test_objective: 0.100425355136
	test_y_col_norms_max: 7.28403282166
	test_y_col_norms_mean: 6.2393155098
	test_y_col_norms_min: 4.98830795288
	test_y_max_max_class: 0.999997019768
	test_y_mean_max_class: 0.959611177444
	test_y_min_max_class: 0.283116281033
	test_y_misclass: 0.03039999865
	test_y_nll: 0.100425355136
	test_y_row_norms_max: 2.53001952171
	test_y_row_norms_mean: 0.806962490082
	test_y_row_norms_min: 0.131183430552
	train_h0_col_norms_max: 6.32822799683
	train_h0_col_norms_mean: 3.93359088898
	train_h0_col_norms_min: 2.10832476616
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.951834976673
	train_h0_max_x_min_u: 0.594883143902
	train_h0_mean_x_max_u: 0.922610759735
	train_h0_mean_x_mean_u: 0.465650081635
	train_h0_mean_x_min_u: 0.0870256572962
	train_h0_min_x_max_u: 0.393012464046
	train_h0_min_x_mean_u: 0.0370769426227
	train_h0_min_x_min_u: 9.6733221433e-10
	train_h0_row_norms_max: 6.20447683334
	train_h0_row_norms_mean: 3.0801115036
	train_h0_row_norms_min: 0.0980293378234
	train_objective: 0.0801135376096
	train_y_col_norms_max: 7.28403186798
	train_y_col_norms_mean: 6.23931598663
	train_y_col_norms_min: 4.98830747604
	train_y_max_max_class: 0.999998509884
	train_y_mean_max_class: 0.960199356079
	train_y_min_max_class: 0.269580304623
	train_y_misclass: 0.0213200002909
	train_y_nll: 0.0801135376096
	train_y_row_norms_max: 2.53001952171
	train_y_row_norms_mean: 0.806962549686
	train_y_row_norms_min: 0.131183415651
	valid_h0_col_norms_max: 6.32822799683
	valid_h0_col_norms_mean: 3.93359160423
	valid_h0_col_norms_min: 2.10832476616
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.951114416122
	valid_h0_max_x_min_u: 0.613163709641
	valid_h0_mean_x_max_u: 0.925033152103
	valid_h0_mean_x_mean_u: 0.465698361397
	valid_h0_mean_x_min_u: 0.0884924307466
	valid_h0_min_x_max_u: 0.386198699474
	valid_h0_min_x_mean_u: 0.0373685508966
	valid_h0_min_x_min_u: 6.73591848965e-10
	valid_h0_row_norms_max: 6.20447731018
	valid_h0_row_norms_mean: 3.08011174202
	valid_h0_row_norms_min: 0.0980293303728
	valid_objective: 0.101348236203
	valid_y_col_norms_max: 7.28403282166
	valid_y_col_norms_mean: 6.2393155098
	valid_y_col_norms_min: 4.98830795288
	valid_y_max_max_class: 0.99999922514
	valid_y_mean_max_class: 0.961142122746
	valid_y_min_max_class: 0.255374312401
	valid_y_misclass: 0.028299998492
	valid_y_nll: 0.101348236203
	valid_y_row_norms_max: 2.53001952171
	valid_y_row_norms_mean: 0.806962490082
	valid_y_row_norms_min: 0.131183430552
Time this epoch: 35.215917 seconds
Monitoring step:
	Epochs seen: 19
	Batches seen: 95
	Examples seen: 950000
	ave_grad_mult: 1.94697141647
	ave_grad_size: 0.0453744120896
	ave_step_size: 0.0806727781892
	test_h0_col_norms_max: 6.33764886856
	test_h0_col_norms_mean: 3.94183731079
	test_h0_col_norms_min: 2.11102938652
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.951863646507
	test_h0_max_x_min_u: 0.580549895763
	test_h0_mean_x_max_u: 0.932179749012
	test_h0_mean_x_mean_u: 0.465434730053
	test_h0_mean_x_min_u: 0.0796971917152
	test_h0_min_x_max_u: 0.390337795019
	test_h0_min_x_mean_u: 0.0372235476971
	test_h0_min_x_min_u: 6.10773209786e-10
	test_h0_row_norms_max: 6.22417736053
	test_h0_row_norms_mean: 3.08713316917
	test_h0_row_norms_min: 0.101160049438
	test_objective: 0.0948458611965
	test_y_col_norms_max: 7.54131317139
	test_y_col_norms_mean: 6.45906209946
	test_y_col_norms_min: 5.14208126068
	test_y_max_max_class: 0.999998509884
	test_y_mean_max_class: 0.962593019009
	test_y_min_max_class: 0.309717655182
	test_y_misclass: 0.0273999981582
	test_y_nll: 0.0948458611965
	test_y_row_norms_max: 2.65757870674
	test_y_row_norms_mean: 0.83378046751
	test_y_row_norms_min: 0.132128432393
	train_h0_col_norms_max: 6.3376493454
	train_h0_col_norms_mean: 3.94183754921
	train_h0_col_norms_min: 2.1110291481
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.953146517277
	train_h0_max_x_min_u: 0.591403305531
	train_h0_mean_x_max_u: 0.925151884556
	train_h0_mean_x_mean_u: 0.465704083443
	train_h0_mean_x_min_u: 0.0828539133072
	train_h0_min_x_max_u: 0.392235994339
	train_h0_min_x_mean_u: 0.0363104119897
	train_h0_min_x_min_u: 8.38429381478e-10
	train_h0_row_norms_max: 6.2241768837
	train_h0_row_norms_mean: 3.08713316917
	train_h0_row_norms_min: 0.101160041988
	train_objective: 0.073119558394
	train_y_col_norms_max: 7.54131317139
	train_y_col_norms_mean: 6.45906209946
	train_y_col_norms_min: 5.14208078384
	train_y_max_max_class: 0.999999344349
	train_y_mean_max_class: 0.963022887707
	train_y_min_max_class: 0.268300741911
	train_y_misclass: 0.0194799974561
	train_y_nll: 0.073119558394
	train_y_row_norms_max: 2.65757846832
	train_y_row_norms_mean: 0.833780527115
	train_y_row_norms_min: 0.132128432393
	valid_h0_col_norms_max: 6.33764886856
	valid_h0_col_norms_mean: 3.94183731079
	valid_h0_col_norms_min: 2.11102938652
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.952474594116
	valid_h0_max_x_min_u: 0.606046676636
	valid_h0_mean_x_max_u: 0.927380979061
	valid_h0_mean_x_mean_u: 0.465753525496
	valid_h0_mean_x_min_u: 0.0843893289566
	valid_h0_min_x_max_u: 0.386562854052
	valid_h0_min_x_mean_u: 0.0366778969765
	valid_h0_min_x_min_u: 5.66411417768e-10
	valid_h0_row_norms_max: 6.22417736053
	valid_h0_row_norms_mean: 3.08713316917
	valid_h0_row_norms_min: 0.101160049438
	valid_objective: 0.09637324512
	valid_y_col_norms_max: 7.54131317139
	valid_y_col_norms_mean: 6.45906209946
	valid_y_col_norms_min: 5.14208126068
	valid_y_max_max_class: 0.999999463558
	valid_y_mean_max_class: 0.96346116066
	valid_y_min_max_class: 0.277560830116
	valid_y_misclass: 0.0262000001967
	valid_y_nll: 0.09637324512
	valid_y_row_norms_max: 2.65757870674
	valid_y_row_norms_mean: 0.83378046751
	valid_y_row_norms_min: 0.132128432393
Time this epoch: 34.760706 seconds
Monitoring step:
	Epochs seen: 20
	Batches seen: 100
	Examples seen: 1000000
	ave_grad_mult: 2.02213191986
	ave_grad_size: 0.0437575168908
	ave_step_size: 0.081667304039
	test_h0_col_norms_max: 6.34621286392
	test_h0_col_norms_mean: 3.94933509827
	test_h0_col_norms_min: 2.11350440979
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.953083157539
	test_h0_max_x_min_u: 0.574586033821
	test_h0_mean_x_max_u: 0.934979915619
	test_h0_mean_x_mean_u: 0.465407788754
	test_h0_mean_x_min_u: 0.0830942466855
	test_h0_min_x_max_u: 0.386586099863
	test_h0_min_x_mean_u: 0.0363725870848
	test_h0_min_x_min_u: 3.24080540182e-10
	test_h0_row_norms_max: 6.2420706749
	test_h0_row_norms_mean: 3.09350514412
	test_h0_row_norms_min: 0.104648023844
	test_objective: 0.0911609381437
	test_y_col_norms_max: 7.76595830917
	test_y_col_norms_mean: 6.65801715851
	test_y_col_norms_min: 5.27815532684
	test_y_max_max_class: 0.999998688698
	test_y_mean_max_class: 0.964522898197
	test_y_min_max_class: 0.28780567646
	test_y_misclass: 0.0263999979943
	test_y_nll: 0.0911609381437
	test_y_row_norms_max: 2.76887655258
	test_y_row_norms_mean: 0.858034849167
	test_y_row_norms_min: 0.135387971997
	train_h0_col_norms_max: 6.34621238708
	train_h0_col_norms_mean: 3.94933462143
	train_h0_col_norms_min: 2.11350440979
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.95438170433
	train_h0_max_x_min_u: 0.584669828415
	train_h0_mean_x_max_u: 0.928267598152
	train_h0_mean_x_mean_u: 0.465672910213
	train_h0_mean_x_min_u: 0.0862845480442
	train_h0_min_x_max_u: 0.389768064022
	train_h0_min_x_mean_u: 0.0354867391288
	train_h0_min_x_min_u: 4.38173886064e-10
	train_h0_row_norms_max: 6.2420706749
	train_h0_row_norms_mean: 3.0935049057
	train_h0_row_norms_min: 0.104648023844
	train_objective: 0.0672194138169
	train_y_col_norms_max: 7.76595830917
	train_y_col_norms_mean: 6.65801715851
	train_y_col_norms_min: 5.27815580368
	train_y_max_max_class: 0.999999523163
	train_y_mean_max_class: 0.965664386749
	train_y_min_max_class: 0.276637971401
	train_y_misclass: 0.0176799986511
	train_y_nll: 0.0672194138169
	train_y_row_norms_max: 2.76887631416
	train_y_row_norms_mean: 0.858034789562
	train_y_row_norms_min: 0.135387957096
	valid_h0_col_norms_max: 6.34621286392
	valid_h0_col_norms_mean: 3.94933509827
	valid_h0_col_norms_min: 2.11350440979
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.953744530678
	valid_h0_max_x_min_u: 0.597306787968
	valid_h0_mean_x_max_u: 0.930288851261
	valid_h0_mean_x_mean_u: 0.465717792511
	valid_h0_mean_x_min_u: 0.087476670742
	valid_h0_min_x_max_u: 0.389485627413
	valid_h0_min_x_mean_u: 0.0357559174299
	valid_h0_min_x_min_u: 3.05218794683e-10
	valid_h0_row_norms_max: 6.2420706749
	valid_h0_row_norms_mean: 3.09350514412
	valid_h0_row_norms_min: 0.104648023844
	valid_objective: 0.0925975292921
	valid_y_col_norms_max: 7.76595830917
	valid_y_col_norms_mean: 6.65801715851
	valid_y_col_norms_min: 5.27815532684
	valid_y_max_max_class: 0.999999761581
	valid_y_mean_max_class: 0.965861082077
	valid_y_min_max_class: 0.303610026836
	valid_y_misclass: 0.0258000008762
	valid_y_nll: 0.0925975292921
	valid_y_row_norms_max: 2.76887655258
	valid_y_row_norms_mean: 0.858034849167
	valid_y_row_norms_min: 0.135387971997
Time this epoch: 35.213061 seconds
Monitoring step:
	Epochs seen: 21
	Batches seen: 105
	Examples seen: 1050000
	ave_grad_mult: 2.08118438721
	ave_grad_size: 0.0415316298604
	ave_step_size: 0.080756470561
	test_h0_col_norms_max: 6.35434007645
	test_h0_col_norms_mean: 3.95622348785
	test_h0_col_norms_min: 2.11573195457
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.953851401806
	test_h0_max_x_min_u: 0.567606449127
	test_h0_mean_x_max_u: 0.933193147182
	test_h0_mean_x_mean_u: 0.465032488108
	test_h0_mean_x_min_u: 0.0830998793244
	test_h0_min_x_max_u: 0.383445978165
	test_h0_min_x_mean_u: 0.0356372632086
	test_h0_min_x_min_u: 2.19485554731e-10
	test_h0_row_norms_max: 6.25859546661
	test_h0_row_norms_mean: 3.09933209419
	test_h0_row_norms_min: 0.107006825507
	test_objective: 0.0886002033949
	test_y_col_norms_max: 7.9637556076
	test_y_col_norms_mean: 6.83463764191
	test_y_col_norms_min: 5.3923330307
	test_y_max_max_class: 0.99999833107
	test_y_mean_max_class: 0.965270340443
	test_y_min_max_class: 0.310471683741
	test_y_misclass: 0.0262000001967
	test_y_nll: 0.0886002033949
	test_y_row_norms_max: 2.86672186852
	test_y_row_norms_mean: 0.879510939121
	test_y_row_norms_min: 0.136433556676
	train_h0_col_norms_max: 6.35433912277
	train_h0_col_norms_mean: 3.95622301102
	train_h0_col_norms_min: 2.11573171616
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.955171644688
	train_h0_max_x_min_u: 0.579390406609
	train_h0_mean_x_max_u: 0.926286578178
	train_h0_mean_x_mean_u: 0.465295374393
	train_h0_mean_x_min_u: 0.0863517001271
	train_h0_min_x_max_u: 0.384008407593
	train_h0_min_x_mean_u: 0.0348346866667
	train_h0_min_x_min_u: 2.7790120205e-10
	train_h0_row_norms_max: 6.25859498978
	train_h0_row_norms_mean: 3.09933185577
	train_h0_row_norms_min: 0.107006818056
	train_objective: 0.0625123158097
	train_y_col_norms_max: 7.96375513077
	train_y_col_norms_mean: 6.83463668823
	train_y_col_norms_min: 5.3923330307
	train_y_max_max_class: 0.99999922514
	train_y_mean_max_class: 0.967036545277
	train_y_min_max_class: 0.273270666599
	train_y_misclass: 0.0158599987626
	train_y_nll: 0.0625123158097
	train_y_row_norms_max: 2.8667216301
	train_y_row_norms_mean: 0.879510939121
	train_y_row_norms_min: 0.136433571577
	valid_h0_col_norms_max: 6.35434007645
	valid_h0_col_norms_mean: 3.95622348785
	valid_h0_col_norms_min: 2.11573195457
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.95438760519
	valid_h0_max_x_min_u: 0.590360045433
	valid_h0_mean_x_max_u: 0.928494334221
	valid_h0_mean_x_mean_u: 0.465345591307
	valid_h0_mean_x_min_u: 0.0878760442138
	valid_h0_min_x_max_u: 0.384474813938
	valid_h0_min_x_mean_u: 0.0351748354733
	valid_h0_min_x_min_u: 1.96166291544e-10
	valid_h0_row_norms_max: 6.25859546661
	valid_h0_row_norms_mean: 3.09933209419
	valid_h0_row_norms_min: 0.107006825507
	valid_objective: 0.0909144356847
	valid_y_col_norms_max: 7.9637556076
	valid_y_col_norms_mean: 6.83463764191
	valid_y_col_norms_min: 5.3923330307
	valid_y_max_max_class: 0.999999403954
	valid_y_mean_max_class: 0.966769099236
	valid_y_min_max_class: 0.282997220755
	valid_y_misclass: 0.025399999693
	valid_y_nll: 0.0909144356847
	valid_y_row_norms_max: 2.86672186852
	valid_y_row_norms_mean: 0.879510939121
	valid_y_row_norms_min: 0.136433556676
Time this epoch: 35.132773 seconds
Monitoring step:
	Epochs seen: 22
	Batches seen: 110
	Examples seen: 1100000
	ave_grad_mult: 2.14148879051
	ave_grad_size: 0.0403550490737
	ave_step_size: 0.0810787156224
	test_h0_col_norms_max: 6.3625164032
	test_h0_col_norms_mean: 3.96336507797
	test_h0_col_norms_min: 2.11782503128
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.954942882061
	test_h0_max_x_min_u: 0.564869940281
	test_h0_mean_x_max_u: 0.935991108418
	test_h0_mean_x_mean_u: 0.465213596821
	test_h0_mean_x_min_u: 0.0809470117092
	test_h0_min_x_max_u: 0.385282725096
	test_h0_min_x_mean_u: 0.0350002162158
	test_h0_min_x_min_u: 1.53522847213e-10
	test_h0_row_norms_max: 6.27728748322
	test_h0_row_norms_mean: 3.10535025597
	test_h0_row_norms_min: 0.109762132168
	test_objective: 0.0847353041172
	test_y_col_norms_max: 8.15684700012
	test_y_col_norms_mean: 7.01448202133
	test_y_col_norms_min: 5.519551754
	test_y_max_max_class: 0.999998867512
	test_y_mean_max_class: 0.967154860497
	test_y_min_max_class: 0.283250451088
	test_y_misclass: 0.0249000005424
	test_y_nll: 0.0847353041172
	test_y_row_norms_max: 2.96138525009
	test_y_row_norms_mean: 0.901844441891
	test_y_row_norms_min: 0.138287782669
	train_h0_col_norms_max: 6.3625164032
	train_h0_col_norms_mean: 3.96336531639
	train_h0_col_norms_min: 2.11782479286
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.956164836884
	train_h0_max_x_min_u: 0.575154662132
	train_h0_mean_x_max_u: 0.929373860359
	train_h0_mean_x_mean_u: 0.465472817421
	train_h0_mean_x_min_u: 0.0842097327113
	train_h0_min_x_max_u: 0.385039448738
	train_h0_min_x_mean_u: 0.0342263542116
	train_h0_min_x_min_u: 1.99991759264e-10
	train_h0_row_norms_max: 6.27728748322
	train_h0_row_norms_mean: 3.10535001755
	train_h0_row_norms_min: 0.109762117267
	train_objective: 0.0575138144195
	train_y_col_norms_max: 8.15684700012
	train_y_col_norms_mean: 7.01448202133
	train_y_col_norms_min: 5.51955223083
	train_y_max_max_class: 0.999999582767
	train_y_mean_max_class: 0.96871650219
	train_y_min_max_class: 0.287014901638
	train_y_misclass: 0.0142399985343
	train_y_nll: 0.0575138144195
	train_y_row_norms_max: 2.96138525009
	train_y_row_norms_mean: 0.901844382286
	train_y_row_norms_min: 0.138287782669
	valid_h0_col_norms_max: 6.3625164032
	valid_h0_col_norms_mean: 3.96336507797
	valid_h0_col_norms_min: 2.11782503128
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.955441474915
	valid_h0_max_x_min_u: 0.589357554913
	valid_h0_mean_x_max_u: 0.93136715889
	valid_h0_mean_x_mean_u: 0.46551990509
	valid_h0_mean_x_min_u: 0.086060911417
	valid_h0_min_x_max_u: 0.390778958797
	valid_h0_min_x_mean_u: 0.0345365256071
	valid_h0_min_x_min_u: 1.45148310038e-10
	valid_h0_row_norms_max: 6.27728748322
	valid_h0_row_norms_mean: 3.10535025597
	valid_h0_row_norms_min: 0.109762132168
	valid_objective: 0.0865774899721
	valid_y_col_norms_max: 8.15684700012
	valid_y_col_norms_mean: 7.01448202133
	valid_y_col_norms_min: 5.519551754
	valid_y_max_max_class: 0.999999761581
	valid_y_mean_max_class: 0.96779280901
	valid_y_min_max_class: 0.273192465305
	valid_y_misclass: 0.0244999974966
	valid_y_nll: 0.0865774899721
	valid_y_row_norms_max: 2.96138525009
	valid_y_row_norms_mean: 0.901844441891
	valid_y_row_norms_min: 0.138287782669
Time this epoch: 35.193111 seconds
Monitoring step:
	Epochs seen: 23
	Batches seen: 115
	Examples seen: 1150000
	ave_grad_mult: 2.29178571701
	ave_grad_size: 0.0395583026111
	ave_step_size: 0.0849489048123
	test_h0_col_norms_max: 6.37209796906
	test_h0_col_norms_mean: 3.97117829323
	test_h0_col_norms_min: 2.11957788467
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.956251323223
	test_h0_max_x_min_u: 0.561847269535
	test_h0_mean_x_max_u: 0.934819757938
	test_h0_mean_x_mean_u: 0.465812414885
	test_h0_mean_x_min_u: 0.0860762521625
	test_h0_min_x_max_u: 0.382852345705
	test_h0_min_x_mean_u: 0.0343066453934
	test_h0_min_x_min_u: 1.00234549827e-10
	test_h0_row_norms_max: 6.29464244843
	test_h0_row_norms_mean: 3.11194372177
	test_h0_row_norms_min: 0.112373262644
	test_objective: 0.0813909471035
	test_y_col_norms_max: 8.37556743622
	test_y_col_norms_mean: 7.21202421188
	test_y_col_norms_min: 5.66676425934
	test_y_max_max_class: 0.99999922514
	test_y_mean_max_class: 0.969460964203
	test_y_min_max_class: 0.304885983467
	test_y_misclass: 0.0245999991894
	test_y_nll: 0.0813909471035
	test_y_row_norms_max: 3.05758142471
	test_y_row_norms_mean: 0.926100432873
	test_y_row_norms_min: 0.141218408942
	train_h0_col_norms_max: 6.37209796906
	train_h0_col_norms_mean: 3.97117805481
	train_h0_col_norms_min: 2.11957764626
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.957390427589
	train_h0_max_x_min_u: 0.571023106575
	train_h0_mean_x_max_u: 0.928050994873
	train_h0_mean_x_mean_u: 0.466074705124
	train_h0_mean_x_min_u: 0.089665055275
	train_h0_min_x_max_u: 0.383693158627
	train_h0_min_x_mean_u: 0.0335417687893
	train_h0_min_x_min_u: 1.25948085294e-10
	train_h0_row_norms_max: 6.29464149475
	train_h0_row_norms_mean: 3.11194324493
	train_h0_row_norms_min: 0.112373247743
	train_objective: 0.0530071258545
	train_y_col_norms_max: 8.37556743622
	train_y_col_norms_mean: 7.21202325821
	train_y_col_norms_min: 5.6667637825
	train_y_max_max_class: 0.999999761581
	train_y_mean_max_class: 0.971323847771
	train_y_min_max_class: 0.274939656258
	train_y_misclass: 0.0134199988097
	train_y_nll: 0.0530071258545
	train_y_row_norms_max: 3.05758142471
	train_y_row_norms_mean: 0.926100373268
	train_y_row_norms_min: 0.141218394041
	valid_h0_col_norms_max: 6.37209796906
	valid_h0_col_norms_mean: 3.97117829323
	valid_h0_col_norms_min: 2.11957788467
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.956661045551
	valid_h0_max_x_min_u: 0.585293292999
	valid_h0_mean_x_max_u: 0.930199086666
	valid_h0_mean_x_mean_u: 0.466111898422
	valid_h0_mean_x_min_u: 0.0879896134138
	valid_h0_min_x_max_u: 0.389526426792
	valid_h0_min_x_mean_u: 0.0339160002768
	valid_h0_min_x_min_u: 9.11426628614e-11
	valid_h0_row_norms_max: 6.29464244843
	valid_h0_row_norms_mean: 3.11194372177
	valid_h0_row_norms_min: 0.112373262644
	valid_objective: 0.0844431295991
	valid_y_col_norms_max: 8.37556743622
	valid_y_col_norms_mean: 7.21202421188
	valid_y_col_norms_min: 5.66676425934
	valid_y_max_max_class: 0.999999761581
	valid_y_mean_max_class: 0.970053553581
	valid_y_min_max_class: 0.252451866865
	valid_y_misclass: 0.0244999974966
	valid_y_nll: 0.0844431295991
	valid_y_row_norms_max: 3.05758142471
	valid_y_row_norms_mean: 0.926100432873
	valid_y_row_norms_min: 0.141218408942
Time this epoch: 35.101327 seconds
Monitoring step:
	Epochs seen: 24
	Batches seen: 120
	Examples seen: 1200000
	ave_grad_mult: 2.4745285511
	ave_grad_size: 0.037980530411
	ave_step_size: 0.0874084308743
	test_h0_col_norms_max: 6.38188457489
	test_h0_col_norms_mean: 3.97928380966
	test_h0_col_norms_min: 2.12256121635
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.957348406315
	test_h0_max_x_min_u: 0.559364676476
	test_h0_mean_x_max_u: 0.934902369976
	test_h0_mean_x_mean_u: 0.465777903795
	test_h0_mean_x_min_u: 0.0820802301168
	test_h0_min_x_max_u: 0.37371224165
	test_h0_min_x_mean_u: 0.0335819907486
	test_h0_min_x_min_u: 6.84129905504e-11
	test_h0_row_norms_max: 6.31191539764
	test_h0_row_norms_mean: 3.11876320839
	test_h0_row_norms_min: 0.114646181464
	test_objective: 0.0791404470801
	test_y_col_norms_max: 8.59417057037
	test_y_col_norms_mean: 7.40912103653
	test_y_col_norms_min: 5.81003856659
	test_y_max_max_class: 0.99999922514
	test_y_mean_max_class: 0.969864010811
	test_y_min_max_class: 0.260499119759
	test_y_misclass: 0.0230999998748
	test_y_nll: 0.0791404470801
	test_y_row_norms_max: 3.15858983994
	test_y_row_norms_mean: 0.950569629669
	test_y_row_norms_min: 0.144145652652
	train_h0_col_norms_max: 6.38188409805
	train_h0_col_norms_mean: 3.97928357124
	train_h0_col_norms_min: 2.12256097794
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.958476305008
	train_h0_max_x_min_u: 0.567814290524
	train_h0_mean_x_max_u: 0.928138375282
	train_h0_mean_x_mean_u: 0.46603512764
	train_h0_mean_x_min_u: 0.0855919569731
	train_h0_min_x_max_u: 0.379186630249
	train_h0_min_x_mean_u: 0.0329259894788
	train_h0_min_x_min_u: 8.38127969804e-11
	train_h0_row_norms_max: 6.31191492081
	train_h0_row_norms_mean: 3.11876296997
	train_h0_row_norms_min: 0.114646181464
	train_objective: 0.0484027862549
	train_y_col_norms_max: 8.59417057037
	train_y_col_norms_mean: 7.40912055969
	train_y_col_norms_min: 5.81003761292
	train_y_max_max_class: 0.999999701977
	train_y_mean_max_class: 0.972274065018
	train_y_min_max_class: 0.297603964806
	train_y_misclass: 0.0116999996826
	train_y_nll: 0.0484027862549
	train_y_row_norms_max: 3.15858960152
	train_y_row_norms_mean: 0.95056951046
	train_y_row_norms_min: 0.144145637751
	valid_h0_col_norms_max: 6.38188457489
	valid_h0_col_norms_mean: 3.97928380966
	valid_h0_col_norms_min: 2.12256121635
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.957733333111
	valid_h0_max_x_min_u: 0.57974678278
	valid_h0_mean_x_max_u: 0.930305242538
	valid_h0_mean_x_mean_u: 0.466072052717
	valid_h0_mean_x_min_u: 0.0875690802932
	valid_h0_min_x_max_u: 0.382509231567
	valid_h0_min_x_mean_u: 0.0332807153463
	valid_h0_min_x_min_u: 6.19569395788e-11
	valid_h0_row_norms_max: 6.31191539764
	valid_h0_row_norms_mean: 3.11876320839
	valid_h0_row_norms_min: 0.114646181464
	valid_objective: 0.0832240283489
	valid_y_col_norms_max: 8.59417057037
	valid_y_col_norms_mean: 7.40912103653
	valid_y_col_norms_min: 5.81003856659
	valid_y_max_max_class: 0.999999761581
	valid_y_mean_max_class: 0.970567047596
	valid_y_min_max_class: 0.264748305082
	valid_y_misclass: 0.023999998346
	valid_y_nll: 0.0832240283489
	valid_y_row_norms_max: 3.15858983994
	valid_y_row_norms_mean: 0.950569629669
	valid_y_row_norms_min: 0.144145652652
Time this epoch: 35.537865 seconds
Monitoring step:
	Epochs seen: 25
	Batches seen: 125
	Examples seen: 1250000
	ave_grad_mult: 2.61537218094
	ave_grad_size: 0.0366696789861
	ave_step_size: 0.0890378654003
	test_h0_col_norms_max: 6.39218759537
	test_h0_col_norms_mean: 3.98763632774
	test_h0_col_norms_min: 2.1254658699
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.958310782909
	test_h0_max_x_min_u: 0.550610423088
	test_h0_mean_x_max_u: 0.936276137829
	test_h0_mean_x_mean_u: 0.46598726511
	test_h0_mean_x_min_u: 0.0805417820811
	test_h0_min_x_max_u: 0.375468403101
	test_h0_min_x_mean_u: 0.0328881442547
	test_h0_min_x_min_u: 7.84403653142e-11
	test_h0_row_norms_max: 6.33220767975
	test_h0_row_norms_mean: 3.1257724762
	test_h0_row_norms_min: 0.116853624582
	test_objective: 0.0754533782601
	test_y_col_norms_max: 8.81067371368
	test_y_col_norms_mean: 7.60889148712
	test_y_col_norms_min: 5.96597194672
	test_y_max_max_class: 0.999999761581
	test_y_mean_max_class: 0.971746265888
	test_y_min_max_class: 0.288777351379
	test_y_misclass: 0.0232999995351
	test_y_nll: 0.0754533782601
	test_y_row_norms_max: 3.24519085884
	test_y_row_norms_mean: 0.975342810154
	test_y_row_norms_min: 0.148321658373
	train_h0_col_norms_max: 6.3921880722
	train_h0_col_norms_mean: 3.98763632774
	train_h0_col_norms_min: 2.1254658699
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.959453582764
	train_h0_max_x_min_u: 0.561280608177
	train_h0_mean_x_max_u: 0.929621398449
	train_h0_mean_x_mean_u: 0.466242551804
	train_h0_mean_x_min_u: 0.0841432288289
	train_h0_min_x_max_u: 0.38017898798
	train_h0_min_x_mean_u: 0.0322615392506
	train_h0_min_x_min_u: 1.0068777756e-10
	train_h0_row_norms_max: 6.33220720291
	train_h0_row_norms_mean: 3.1257724762
	train_h0_row_norms_min: 0.116853624582
	train_objective: 0.043993473053
	train_y_col_norms_max: 8.81067276001
	train_y_col_norms_mean: 7.60889053345
	train_y_col_norms_min: 5.96597194672
	train_y_max_max_class: 0.999999821186
	train_y_mean_max_class: 0.974251687527
	train_y_min_max_class: 0.270618349314
	train_y_misclass: 0.0104199992493
	train_y_nll: 0.043993473053
	train_y_row_norms_max: 3.24519062042
	train_y_row_norms_mean: 0.975342690945
	train_y_row_norms_min: 0.148321658373
	valid_h0_col_norms_max: 6.39218759537
	valid_h0_col_norms_mean: 3.98763632774
	valid_h0_col_norms_min: 2.1254658699
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.958616793156
	valid_h0_max_x_min_u: 0.574836432934
	valid_h0_mean_x_max_u: 0.931737542152
	valid_h0_mean_x_mean_u: 0.466278731823
	valid_h0_mean_x_min_u: 0.0861588418484
	valid_h0_min_x_max_u: 0.383438080549
	valid_h0_min_x_mean_u: 0.032565869391
	valid_h0_min_x_min_u: 7.15359646519e-11
	valid_h0_row_norms_max: 6.33220767975
	valid_h0_row_norms_mean: 3.1257724762
	valid_h0_row_norms_min: 0.116853624582
	valid_objective: 0.0792490914464
	valid_y_col_norms_max: 8.81067371368
	valid_y_col_norms_mean: 7.60889148712
	valid_y_col_norms_min: 5.96597194672
	valid_y_max_max_class: 0.999999821186
	valid_y_mean_max_class: 0.972301781178
	valid_y_min_max_class: 0.278648257256
	valid_y_misclass: 0.0228000003844
	valid_y_nll: 0.0792490914464
	valid_y_row_norms_max: 3.24519085884
	valid_y_row_norms_mean: 0.975342810154
	valid_y_row_norms_min: 0.148321658373
Time this epoch: 35.095306 seconds
Monitoring step:
	Epochs seen: 26
	Batches seen: 130
	Examples seen: 1300000
	ave_grad_mult: 2.71106290817
	ave_grad_size: 0.0348753891885
	ave_step_size: 0.0883127823472
	test_h0_col_norms_max: 6.4017291069
	test_h0_col_norms_mean: 3.99520802498
	test_h0_col_norms_min: 2.12854385376
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.959264814854
	test_h0_max_x_min_u: 0.55078792572
	test_h0_mean_x_max_u: 0.935866773129
	test_h0_mean_x_mean_u: 0.466203004122
	test_h0_mean_x_min_u: 0.078706741333
	test_h0_min_x_max_u: 0.367448121309
	test_h0_min_x_mean_u: 0.0321592055261
	test_h0_min_x_min_u: 4.60236952715e-11
	test_h0_row_norms_max: 6.34829235077
	test_h0_row_norms_mean: 3.13210654259
	test_h0_row_norms_min: 0.118406176567
	test_objective: 0.0733289569616
	test_y_col_norms_max: 9.00574874878
	test_y_col_norms_mean: 7.78995084763
	test_y_col_norms_min: 6.10382938385
	test_y_max_max_class: 0.999999761581
	test_y_mean_max_class: 0.972537279129
	test_y_min_max_class: 0.267330288887
	test_y_misclass: 0.0230999998748
	test_y_nll: 0.0733289569616
	test_y_row_norms_max: 3.33722496033
	test_y_row_norms_mean: 0.997784733772
	test_y_row_norms_min: 0.151363104582
	train_h0_col_norms_max: 6.40172863007
	train_h0_col_norms_mean: 3.9952082634
	train_h0_col_norms_min: 2.12854361534
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.960306882858
	train_h0_max_x_min_u: 0.562210321426
	train_h0_mean_x_max_u: 0.929156780243
	train_h0_mean_x_mean_u: 0.466451466084
	train_h0_mean_x_min_u: 0.0823357179761
	train_h0_min_x_max_u: 0.377736270428
	train_h0_min_x_mean_u: 0.0315755605698
	train_h0_min_x_min_u: 5.56277697517e-11
	train_h0_row_norms_max: 6.34829139709
	train_h0_row_norms_mean: 3.13210630417
	train_h0_row_norms_min: 0.118406184018
	train_objective: 0.0409014374018
	train_y_col_norms_max: 9.00574874878
	train_y_col_norms_mean: 7.78995037079
	train_y_col_norms_min: 6.10382938385
	train_y_max_max_class: 0.999999821186
	train_y_mean_max_class: 0.975649058819
	train_y_min_max_class: 0.290484070778
	train_y_misclass: 0.00971999950707
	train_y_nll: 0.0409014374018
	train_y_row_norms_max: 3.33722496033
	train_y_row_norms_mean: 0.997784733772
	train_y_row_norms_min: 0.151363104582
	valid_h0_col_norms_max: 6.4017291069
	valid_h0_col_norms_mean: 3.99520802498
	valid_h0_col_norms_min: 2.12854385376
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.959495425224
	valid_h0_max_x_min_u: 0.576115489006
	valid_h0_mean_x_max_u: 0.931356489658
	valid_h0_mean_x_mean_u: 0.466476142406
	valid_h0_mean_x_min_u: 0.084395840764
	valid_h0_min_x_max_u: 0.37360149622
	valid_h0_min_x_mean_u: 0.0320250503719
	valid_h0_min_x_min_u: 4.11951479873e-11
	valid_h0_row_norms_max: 6.34829235077
	valid_h0_row_norms_mean: 3.13210654259
	valid_h0_row_norms_min: 0.118406176567
	valid_objective: 0.0791732370853
	valid_y_col_norms_max: 9.00574874878
	valid_y_col_norms_mean: 7.78995084763
	valid_y_col_norms_min: 6.10382938385
	valid_y_max_max_class: 0.999999821186
	valid_y_mean_max_class: 0.973247587681
	valid_y_min_max_class: 0.254454284906
	valid_y_misclass: 0.0232999995351
	valid_y_nll: 0.0791732370853
	valid_y_row_norms_max: 3.33722496033
	valid_y_row_norms_mean: 0.997784733772
	valid_y_row_norms_min: 0.151363104582
Time this epoch: 35.406078 seconds
Monitoring step:
	Epochs seen: 27
	Batches seen: 135
	Examples seen: 1350000
	ave_grad_mult: 2.80285286903
	ave_grad_size: 0.0334513224661
	ave_step_size: 0.088192678988
	test_h0_col_norms_max: 6.41177082062
	test_h0_col_norms_mean: 4.00275707245
	test_h0_col_norms_min: 2.13091373444
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.960214793682
	test_h0_max_x_min_u: 0.547308385372
	test_h0_mean_x_max_u: 0.936411142349
	test_h0_mean_x_mean_u: 0.466299444437
	test_h0_mean_x_min_u: 0.0786798894405
	test_h0_min_x_max_u: 0.366961061954
	test_h0_min_x_mean_u: 0.0315478779376
	test_h0_min_x_min_u: 4.00019496694e-11
	test_h0_row_norms_max: 6.36605072021
	test_h0_row_norms_mean: 3.13841247559
	test_h0_row_norms_min: 0.119428776205
	test_objective: 0.0725825279951
	test_y_col_norms_max: 9.19264411926
	test_y_col_norms_mean: 7.96643924713
	test_y_col_norms_min: 6.2465171814
	test_y_max_max_class: 0.999999821186
	test_y_mean_max_class: 0.974697828293
	test_y_min_max_class: 0.284473180771
	test_y_misclass: 0.0219000000507
	test_y_nll: 0.0725825279951
	test_y_row_norms_max: 3.41140413284
	test_y_row_norms_mean: 1.01977562904
	test_y_row_norms_min: 0.155052781105
	train_h0_col_norms_max: 6.41176986694
	train_h0_col_norms_mean: 4.00275659561
	train_h0_col_norms_min: 2.13091373444
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.96123111248
	train_h0_max_x_min_u: 0.556509852409
	train_h0_mean_x_max_u: 0.929732501507
	train_h0_mean_x_mean_u: 0.466540902853
	train_h0_mean_x_min_u: 0.0823666229844
	train_h0_min_x_max_u: 0.371994018555
	train_h0_min_x_mean_u: 0.0308939814568
	train_h0_min_x_min_u: 5.01665688157e-11
	train_h0_row_norms_max: 6.36605024338
	train_h0_row_norms_mean: 3.13841223717
	train_h0_row_norms_min: 0.119428783655
	train_objective: 0.0370035469532
	train_y_col_norms_max: 9.19264411926
	train_y_col_norms_mean: 7.96643972397
	train_y_col_norms_min: 6.24651670456
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.977714180946
	train_y_min_max_class: 0.2884734869
	train_y_misclass: 0.00885999947786
	train_y_nll: 0.0370035469532
	train_y_row_norms_max: 3.41140389442
	train_y_row_norms_mean: 1.01977562904
	train_y_row_norms_min: 0.155052781105
	valid_h0_col_norms_max: 6.41177082062
	valid_h0_col_norms_mean: 4.00275707245
	valid_h0_col_norms_min: 2.13091373444
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.960400640965
	valid_h0_max_x_min_u: 0.574073672295
	valid_h0_mean_x_max_u: 0.931940674782
	valid_h0_mean_x_mean_u: 0.466566413641
	valid_h0_mean_x_min_u: 0.0844538062811
	valid_h0_min_x_max_u: 0.369769692421
	valid_h0_min_x_mean_u: 0.0312857404351
	valid_h0_min_x_min_u: 3.6578275131e-11
	valid_h0_row_norms_max: 6.36605072021
	valid_h0_row_norms_mean: 3.13841247559
	valid_h0_row_norms_min: 0.119428776205
	valid_objective: 0.0765716135502
	valid_y_col_norms_max: 9.19264411926
	valid_y_col_norms_mean: 7.96643924713
	valid_y_col_norms_min: 6.2465171814
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.974825143814
	valid_y_min_max_class: 0.268961429596
	valid_y_misclass: 0.0228999983519
	valid_y_nll: 0.0765716135502
	valid_y_row_norms_max: 3.41140413284
	valid_y_row_norms_mean: 1.01977562904
	valid_y_row_norms_min: 0.155052781105
Time this epoch: 34.780491 seconds
Monitoring step:
	Epochs seen: 28
	Batches seen: 140
	Examples seen: 1400000
	ave_grad_mult: 3.07722043991
	ave_grad_size: 0.0323846936226
	ave_step_size: 0.0927985981107
	test_h0_col_norms_max: 6.42322683334
	test_h0_col_norms_mean: 4.01182746887
	test_h0_col_norms_min: 2.13467645645
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.961281061172
	test_h0_max_x_min_u: 0.549638330936
	test_h0_mean_x_max_u: 0.939934492111
	test_h0_mean_x_mean_u: 0.466522186995
	test_h0_mean_x_min_u: 0.0823039337993
	test_h0_min_x_max_u: 0.357347339392
	test_h0_min_x_mean_u: 0.030734334141
	test_h0_min_x_min_u: 5.08886266459e-11
	test_h0_row_norms_max: 6.38524675369
	test_h0_row_norms_mean: 3.1459903717
	test_h0_row_norms_min: 0.121352598071
	test_objective: 0.0716430544853
	test_y_col_norms_max: 9.41203117371
	test_y_col_norms_mean: 8.17550086975
	test_y_col_norms_min: 6.39991140366
	test_y_max_max_class: 0.999999821186
	test_y_mean_max_class: 0.974777877331
	test_y_min_max_class: 0.270264923573
	test_y_misclass: 0.0223999992013
	test_y_nll: 0.0716430544853
	test_y_row_norms_max: 3.5011806488
	test_y_row_norms_mean: 1.04629290104
	test_y_row_norms_min: 0.159884780645
	train_h0_col_norms_max: 6.42322635651
	train_h0_col_norms_mean: 4.01182699203
	train_h0_col_norms_min: 2.13467645645
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.962265074253
	train_h0_max_x_min_u: 0.556583762169
	train_h0_mean_x_max_u: 0.93360298872
	train_h0_mean_x_mean_u: 0.4667532444
	train_h0_mean_x_min_u: 0.0861736312509
	train_h0_min_x_max_u: 0.366083562374
	train_h0_min_x_mean_u: 0.0301264487207
	train_h0_min_x_min_u: 6.59544779902e-11
	train_h0_row_norms_max: 6.38524627686
	train_h0_row_norms_mean: 3.1459903717
	train_h0_row_norms_min: 0.121352590621
	train_objective: 0.0347100757062
	train_y_col_norms_max: 9.41203117371
	train_y_col_norms_mean: 8.17550086975
	train_y_col_norms_min: 6.39991092682
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.978323638439
	train_y_min_max_class: 0.29167419672
	train_y_misclass: 0.00763999950141
	train_y_nll: 0.0347100757062
	train_y_row_norms_max: 3.50118041039
	train_y_row_norms_mean: 1.04629290104
	train_y_row_norms_min: 0.159884765744
	valid_h0_col_norms_max: 6.42322683334
	valid_h0_col_norms_mean: 4.01182746887
	valid_h0_col_norms_min: 2.13467645645
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.961273550987
	valid_h0_max_x_min_u: 0.570200264454
	valid_h0_mean_x_max_u: 0.935558438301
	valid_h0_mean_x_mean_u: 0.466782003641
	valid_h0_mean_x_min_u: 0.0883660390973
	valid_h0_min_x_max_u: 0.355543404818
	valid_h0_min_x_mean_u: 0.0304867494851
	valid_h0_min_x_min_u: 4.62212004781e-11
	valid_h0_row_norms_max: 6.38524675369
	valid_h0_row_norms_mean: 3.1459903717
	valid_h0_row_norms_min: 0.121352598071
	valid_objective: 0.0746665000916
	valid_y_col_norms_max: 9.41203117371
	valid_y_col_norms_mean: 8.17550086975
	valid_y_col_norms_min: 6.39991140366
	valid_y_max_max_class: 0.999999821186
	valid_y_mean_max_class: 0.975300252438
	valid_y_min_max_class: 0.280865699053
	valid_y_misclass: 0.0222999975085
	valid_y_nll: 0.0746665000916
	valid_y_row_norms_max: 3.5011806488
	valid_y_row_norms_mean: 1.04629290104
	valid_y_row_norms_min: 0.159884780645
Time this epoch: 35.278322 seconds
Monitoring step:
	Epochs seen: 29
	Batches seen: 145
	Examples seen: 1450000
	ave_grad_mult: 3.31815242767
	ave_grad_size: 0.0319525785744
	ave_step_size: 0.0989938527346
	test_h0_col_norms_max: 6.43632364273
	test_h0_col_norms_mean: 4.02131462097
	test_h0_col_norms_min: 2.13684439659
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.96216905117
	test_h0_max_x_min_u: 0.548579275608
	test_h0_mean_x_max_u: 0.941554307938
	test_h0_mean_x_mean_u: 0.46652469039
	test_h0_mean_x_min_u: 0.0799239650369
	test_h0_min_x_max_u: 0.357737779617
	test_h0_min_x_mean_u: 0.0300854835659
	test_h0_min_x_min_u: 2.19761518011e-11
	test_h0_row_norms_max: 6.40747022629
	test_h0_row_norms_mean: 3.15389037132
	test_h0_row_norms_min: 0.123720750213
	test_objective: 0.0690323263407
	test_y_col_norms_max: 9.64226436615
	test_y_col_norms_mean: 8.3891248703
	test_y_col_norms_min: 6.57722139359
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.976467430592
	test_y_min_max_class: 0.293188840151
	test_y_misclass: 0.0212999973446
	test_y_nll: 0.0690323263407
	test_y_row_norms_max: 3.5816681385
	test_y_row_norms_mean: 1.07294213772
	test_y_row_norms_min: 0.162085324526
	train_h0_col_norms_max: 6.43632364273
	train_h0_col_norms_mean: 4.02131462097
	train_h0_col_norms_min: 2.13684439659
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.963165700436
	train_h0_max_x_min_u: 0.55665397644
	train_h0_mean_x_max_u: 0.935373008251
	train_h0_mean_x_mean_u: 0.466756403446
	train_h0_mean_x_min_u: 0.0838716328144
	train_h0_min_x_max_u: 0.36525374651
	train_h0_min_x_mean_u: 0.0295039452612
	train_h0_min_x_min_u: 2.68275124338e-11
	train_h0_row_norms_max: 6.40747022629
	train_h0_row_norms_mean: 3.15389037132
	train_h0_row_norms_min: 0.123720750213
	train_objective: 0.0306258164346
	train_y_col_norms_max: 9.64226341248
	train_y_col_norms_mean: 8.3891248703
	train_y_col_norms_min: 6.57722091675
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.980324864388
	train_y_min_max_class: 0.283443570137
	train_y_misclass: 0.00647999951616
	train_y_nll: 0.0306258164346
	train_y_row_norms_max: 3.58166790009
	train_y_row_norms_mean: 1.07294213772
	train_y_row_norms_min: 0.162085309625
	valid_h0_col_norms_max: 6.43632364273
	valid_h0_col_norms_mean: 4.02131462097
	valid_h0_col_norms_min: 2.13684439659
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.962217152119
	valid_h0_max_x_min_u: 0.571269631386
	valid_h0_mean_x_max_u: 0.937248468399
	valid_h0_mean_x_mean_u: 0.466767311096
	valid_h0_mean_x_min_u: 0.0859551951289
	valid_h0_min_x_max_u: 0.352124005556
	valid_h0_min_x_mean_u: 0.0299664791673
	valid_h0_min_x_min_u: 2.06817705323e-11
	valid_h0_row_norms_max: 6.40747022629
	valid_h0_row_norms_mean: 3.15389037132
	valid_h0_row_norms_min: 0.123720750213
	valid_objective: 0.0733132436872
	valid_y_col_norms_max: 9.64226436615
	valid_y_col_norms_mean: 8.3891248703
	valid_y_col_norms_min: 6.57722139359
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.976790785789
	valid_y_min_max_class: 0.291347831488
	valid_y_misclass: 0.0217000003904
	valid_y_nll: 0.0733132436872
	valid_y_row_norms_max: 3.5816681385
	valid_y_row_norms_mean: 1.07294213772
	valid_y_row_norms_min: 0.162085324526
Time this epoch: 35.082858 seconds
Monitoring step:
	Epochs seen: 30
	Batches seen: 150
	Examples seen: 1500000
	ave_grad_mult: 3.39051413536
	ave_grad_size: 0.0302216522396
	ave_step_size: 0.0965146124363
	test_h0_col_norms_max: 6.44657659531
	test_h0_col_norms_mean: 4.02922582626
	test_h0_col_norms_min: 2.14045143127
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.963280200958
	test_h0_max_x_min_u: 0.548351347446
	test_h0_mean_x_max_u: 0.940559446812
	test_h0_mean_x_mean_u: 0.466341674328
	test_h0_mean_x_min_u: 0.0799687504768
	test_h0_min_x_max_u: 0.357499152422
	test_h0_min_x_mean_u: 0.0293492469937
	test_h0_min_x_min_u: 2.20450845079e-11
	test_h0_row_norms_max: 6.4218788147
	test_h0_row_norms_mean: 3.16045355797
	test_h0_row_norms_min: 0.124831520021
	test_objective: 0.0672078579664
	test_y_col_norms_max: 9.82299423218
	test_y_col_norms_mean: 8.56633377075
	test_y_col_norms_min: 6.71553707123
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.977503836155
	test_y_min_max_class: 0.255842655897
	test_y_misclass: 0.0198999978602
	test_y_nll: 0.0672078579664
	test_y_row_norms_max: 3.65550899506
	test_y_row_norms_mean: 1.09536457062
	test_y_row_norms_min: 0.163716614246
	train_h0_col_norms_max: 6.44657611847
	train_h0_col_norms_mean: 4.02922534943
	train_h0_col_norms_min: 2.14045143127
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.964230835438
	train_h0_max_x_min_u: 0.55356913805
	train_h0_mean_x_max_u: 0.934283614159
	train_h0_mean_x_mean_u: 0.466563820839
	train_h0_mean_x_min_u: 0.0839123427868
	train_h0_min_x_max_u: 0.358219176531
	train_h0_min_x_mean_u: 0.0287974383682
	train_h0_min_x_min_u: 2.7636832059e-11
	train_h0_row_norms_max: 6.42187833786
	train_h0_row_norms_mean: 3.16045331955
	train_h0_row_norms_min: 0.12483151257
	train_objective: 0.0282621402293
	train_y_col_norms_max: 9.82299423218
	train_y_col_norms_mean: 8.56633377075
	train_y_col_norms_min: 6.71553659439
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.981470048428
	train_y_min_max_class: 0.304827183485
	train_y_misclass: 0.00591999944299
	train_y_nll: 0.0282621402293
	train_y_row_norms_max: 3.65550875664
	train_y_row_norms_mean: 1.09536445141
	train_y_row_norms_min: 0.163716599345
	valid_h0_col_norms_max: 6.44657659531
	valid_h0_col_norms_mean: 4.02922582626
	valid_h0_col_norms_min: 2.14045143127
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.963276088238
	valid_h0_max_x_min_u: 0.567677497864
	valid_h0_mean_x_max_u: 0.936259627342
	valid_h0_mean_x_mean_u: 0.466583073139
	valid_h0_mean_x_min_u: 0.0857717692852
	valid_h0_min_x_max_u: 0.344594448805
	valid_h0_min_x_mean_u: 0.0292918123305
	valid_h0_min_x_min_u: 2.15348190669e-11
	valid_h0_row_norms_max: 6.4218788147
	valid_h0_row_norms_mean: 3.16045355797
	valid_h0_row_norms_min: 0.124831520021
	valid_objective: 0.0722089111805
	valid_y_col_norms_max: 9.82299423218
	valid_y_col_norms_mean: 8.56633377075
	valid_y_col_norms_min: 6.71553707123
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.977750241756
	valid_y_min_max_class: 0.297483742237
	valid_y_misclass: 0.021099999547
	valid_y_nll: 0.0722089111805
	valid_y_row_norms_max: 3.65550899506
	valid_y_row_norms_mean: 1.09536457062
	valid_y_row_norms_min: 0.163716614246
Time this epoch: 35.012923 seconds
Monitoring step:
	Epochs seen: 31
	Batches seen: 155
	Examples seen: 1550000
	ave_grad_mult: 3.48831152916
	ave_grad_size: 0.0287408661097
	ave_step_size: 0.0940494984388
	test_h0_col_norms_max: 6.45725250244
	test_h0_col_norms_mean: 4.03711128235
	test_h0_col_norms_min: 2.14304447174
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.963537156582
	test_h0_max_x_min_u: 0.54846316576
	test_h0_mean_x_max_u: 0.940077364445
	test_h0_mean_x_mean_u: 0.466366380453
	test_h0_mean_x_min_u: 0.0763043165207
	test_h0_min_x_max_u: 0.358212590218
	test_h0_min_x_mean_u: 0.0289474800229
	test_h0_min_x_min_u: 2.33178042847e-11
	test_h0_row_norms_max: 6.44061088562
	test_h0_row_norms_mean: 3.16697835922
	test_h0_row_norms_min: 0.125991553068
	test_objective: 0.0662763118744
	test_y_col_norms_max: 10.0062093735
	test_y_col_norms_mean: 8.73837566376
	test_y_col_norms_min: 6.84926891327
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.978273510933
	test_y_min_max_class: 0.285628795624
	test_y_misclass: 0.0193999987096
	test_y_nll: 0.0662763118744
	test_y_row_norms_max: 3.72132134438
	test_y_row_norms_mean: 1.11716985703
	test_y_row_norms_min: 0.168285727501
	train_h0_col_norms_max: 6.45725250244
	train_h0_col_norms_mean: 4.03711175919
	train_h0_col_norms_min: 2.14304423332
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.964494287968
	train_h0_max_x_min_u: 0.551624715328
	train_h0_mean_x_max_u: 0.933692634106
	train_h0_mean_x_mean_u: 0.466590344906
	train_h0_mean_x_min_u: 0.0803528800607
	train_h0_min_x_max_u: 0.359871029854
	train_h0_min_x_mean_u: 0.0284454971552
	train_h0_min_x_min_u: 3.10430431361e-11
	train_h0_row_norms_max: 6.44061088562
	train_h0_row_norms_mean: 3.16697835922
	train_h0_row_norms_min: 0.125991553068
	train_objective: 0.0254283007234
	train_y_col_norms_max: 10.0062084198
	train_y_col_norms_mean: 8.73837471008
	train_y_col_norms_min: 6.84926795959
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.982455551624
	train_y_min_max_class: 0.295360028744
	train_y_misclass: 0.00481999944896
	train_y_nll: 0.0254283007234
	train_y_row_norms_max: 3.72132158279
	train_y_row_norms_mean: 1.11716985703
	train_y_row_norms_min: 0.168285742402
	valid_h0_col_norms_max: 6.45725250244
	valid_h0_col_norms_mean: 4.03711128235
	valid_h0_col_norms_min: 2.14304447174
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.96359193325
	valid_h0_max_x_min_u: 0.566685140133
	valid_h0_mean_x_max_u: 0.935809135437
	valid_h0_mean_x_mean_u: 0.466593444347
	valid_h0_mean_x_min_u: 0.0823497697711
	valid_h0_min_x_max_u: 0.342845439911
	valid_h0_min_x_mean_u: 0.0289610140026
	valid_h0_min_x_min_u: 2.35798464088e-11
	valid_h0_row_norms_max: 6.44061088562
	valid_h0_row_norms_mean: 3.16697835922
	valid_h0_row_norms_min: 0.125991553068
	valid_objective: 0.0720023438334
	valid_y_col_norms_max: 10.0062093735
	valid_y_col_norms_mean: 8.73837566376
	valid_y_col_norms_min: 6.84926891327
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.978184223175
	valid_y_min_max_class: 0.33930772543
	valid_y_misclass: 0.0208999998868
	valid_y_nll: 0.0720023438334
	valid_y_row_norms_max: 3.72132134438
	valid_y_row_norms_mean: 1.11716985703
	valid_y_row_norms_min: 0.168285727501
Time this epoch: 35.375439 seconds
Monitoring step:
	Epochs seen: 32
	Batches seen: 160
	Examples seen: 1600000
	ave_grad_mult: 3.63046574593
	ave_grad_size: 0.0268990695477
	ave_step_size: 0.091931194067
	test_h0_col_norms_max: 6.46750497818
	test_h0_col_norms_mean: 4.04480981827
	test_h0_col_norms_min: 2.1463572979
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.96414077282
	test_h0_max_x_min_u: 0.551202893257
	test_h0_mean_x_max_u: 0.938820004463
	test_h0_mean_x_mean_u: 0.466768145561
	test_h0_mean_x_min_u: 0.0774406716228
	test_h0_min_x_max_u: 0.35613399744
	test_h0_min_x_mean_u: 0.0284414924681
	test_h0_min_x_min_u: 2.09623499114e-11
	test_h0_row_norms_max: 6.45651197433
	test_h0_row_norms_mean: 3.17332720757
	test_h0_row_norms_min: 0.126796171069
	test_objective: 0.0663162916899
	test_y_col_norms_max: 10.1816034317
	test_y_col_norms_mean: 8.90622425079
	test_y_col_norms_min: 6.98152685165
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.978770077229
	test_y_min_max_class: 0.294499635696
	test_y_misclass: 0.0207000002265
	test_y_nll: 0.0663162916899
	test_y_row_norms_max: 3.78270602226
	test_y_row_norms_mean: 1.13857710361
	test_y_row_norms_min: 0.170636937022
	train_h0_col_norms_max: 6.46750450134
	train_h0_col_norms_mean: 4.04480981827
	train_h0_col_norms_min: 2.1463572979
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.965125858784
	train_h0_max_x_min_u: 0.553418159485
	train_h0_mean_x_max_u: 0.932287812233
	train_h0_mean_x_mean_u: 0.466983139515
	train_h0_mean_x_min_u: 0.0815795511007
	train_h0_min_x_max_u: 0.351988613605
	train_h0_min_x_mean_u: 0.0279593002051
	train_h0_min_x_min_u: 2.79454966112e-11
	train_h0_row_norms_max: 6.4565114975
	train_h0_row_norms_mean: 3.17332744598
	train_h0_row_norms_min: 0.126796171069
	train_objective: 0.0232511665672
	train_y_col_norms_max: 10.1816034317
	train_y_col_norms_mean: 8.90622425079
	train_y_col_norms_min: 6.98152732849
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.98363161087
	train_y_min_max_class: 0.305530905724
	train_y_misclass: 0.00421999953687
	train_y_nll: 0.0232511665672
	train_y_row_norms_max: 3.78270626068
	train_y_row_norms_mean: 1.13857698441
	train_y_row_norms_min: 0.170636937022
	valid_h0_col_norms_max: 6.46750497818
	valid_h0_col_norms_mean: 4.04480981827
	valid_h0_col_norms_min: 2.1463572979
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.964271306992
	valid_h0_max_x_min_u: 0.568406701088
	valid_h0_mean_x_max_u: 0.934549808502
	valid_h0_mean_x_mean_u: 0.466980487108
	valid_h0_mean_x_min_u: 0.0836460664868
	valid_h0_min_x_max_u: 0.336699128151
	valid_h0_min_x_mean_u: 0.028556285426
	valid_h0_min_x_min_u: 2.1281049839e-11
	valid_h0_row_norms_max: 6.45651197433
	valid_h0_row_norms_mean: 3.17332720757
	valid_h0_row_norms_min: 0.126796171069
	valid_objective: 0.0705031752586
	valid_y_col_norms_max: 10.1816034317
	valid_y_col_norms_mean: 8.90622425079
	valid_y_col_norms_min: 6.98152685165
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.978758752346
	valid_y_min_max_class: 0.291737556458
	valid_y_misclass: 0.0208999998868
	valid_y_nll: 0.0705031752586
	valid_y_row_norms_max: 3.78270602226
	valid_y_row_norms_mean: 1.13857710361
	valid_y_row_norms_min: 0.170636937022
Time this epoch: 35.379330 seconds
Monitoring step:
	Epochs seen: 33
	Batches seen: 165
	Examples seen: 1650000
	ave_grad_mult: 3.85002589226
	ave_grad_size: 0.0255950912833
	ave_step_size: 0.0920957773924
	test_h0_col_norms_max: 6.47780418396
	test_h0_col_norms_mean: 4.05291509628
	test_h0_col_norms_min: 2.14965701103
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.965208768845
	test_h0_max_x_min_u: 0.553891956806
	test_h0_mean_x_max_u: 0.941352784634
	test_h0_mean_x_mean_u: 0.467216670513
	test_h0_mean_x_min_u: 0.0769760459661
	test_h0_min_x_max_u: 0.357422113419
	test_h0_min_x_mean_u: 0.027681870386
	test_h0_min_x_min_u: 1.6821729773e-11
	test_h0_row_norms_max: 6.47196292877
	test_h0_row_norms_mean: 3.18000507355
	test_h0_row_norms_min: 0.127480790019
	test_objective: 0.0658261179924
	test_y_col_norms_max: 10.3589458466
	test_y_col_norms_mean: 9.08145141602
	test_y_col_norms_min: 7.12754154205
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.980045855045
	test_y_min_max_class: 0.275538861752
	test_y_misclass: 0.019999999553
	test_y_nll: 0.0658261179924
	test_y_row_norms_max: 3.84528589249
	test_y_row_norms_mean: 1.16080152988
	test_y_row_norms_min: 0.173613965511
	train_h0_col_norms_max: 6.47780418396
	train_h0_col_norms_mean: 4.05291461945
	train_h0_col_norms_min: 2.14965701103
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.966172575951
	train_h0_max_x_min_u: 0.551838994026
	train_h0_mean_x_max_u: 0.935074448586
	train_h0_mean_x_mean_u: 0.46742233634
	train_h0_mean_x_min_u: 0.0811282843351
	train_h0_min_x_max_u: 0.346001476049
	train_h0_min_x_mean_u: 0.0272373519838
	train_h0_min_x_min_u: 2.30680283902e-11
	train_h0_row_norms_max: 6.47196340561
	train_h0_row_norms_mean: 3.18000459671
	train_h0_row_norms_min: 0.127480790019
	train_objective: 0.02110886015
	train_y_col_norms_max: 10.3589458466
	train_y_col_norms_mean: 9.08145141602
	train_y_col_norms_min: 7.12754058838
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.984871923923
	train_y_min_max_class: 0.292335510254
	train_y_misclass: 0.00331999990158
	train_y_nll: 0.02110886015
	train_y_row_norms_max: 3.84528613091
	train_y_row_norms_mean: 1.16080152988
	train_y_row_norms_min: 0.173613965511
	valid_h0_col_norms_max: 6.47780418396
	valid_h0_col_norms_mean: 4.05291509628
	valid_h0_col_norms_min: 2.14965701103
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.965333819389
	valid_h0_max_x_min_u: 0.567090988159
	valid_h0_mean_x_max_u: 0.937143027782
	valid_h0_mean_x_mean_u: 0.467420905828
	valid_h0_mean_x_min_u: 0.0815225914121
	valid_h0_min_x_max_u: 0.317524284124
	valid_h0_min_x_mean_u: 0.0278288982809
	valid_h0_min_x_min_u: 1.7383304865e-11
	valid_h0_row_norms_max: 6.47196292877
	valid_h0_row_norms_mean: 3.18000507355
	valid_h0_row_norms_min: 0.127480790019
	valid_objective: 0.0706555917859
	valid_y_col_norms_max: 10.3589458466
	valid_y_col_norms_mean: 9.08145141602
	valid_y_col_norms_min: 7.12754154205
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.979981780052
	valid_y_min_max_class: 0.314534544945
	valid_y_misclass: 0.0206000003964
	valid_y_nll: 0.0706555917859
	valid_y_row_norms_max: 3.84528589249
	valid_y_row_norms_mean: 1.16080152988
	valid_y_row_norms_min: 0.173613965511
Time this epoch: 35.182908 seconds
Monitoring step:
	Epochs seen: 34
	Batches seen: 170
	Examples seen: 1700000
	ave_grad_mult: 4.07905960083
	ave_grad_size: 0.0242200661451
	ave_step_size: 0.0924715399742
	test_h0_col_norms_max: 6.48900747299
	test_h0_col_norms_mean: 4.06139850616
	test_h0_col_norms_min: 2.1522192955
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.965900540352
	test_h0_max_x_min_u: 0.551373183727
	test_h0_mean_x_max_u: 0.942069590092
	test_h0_mean_x_mean_u: 0.467438340187
	test_h0_mean_x_min_u: 0.0787537544966
	test_h0_min_x_max_u: 0.359593838453
	test_h0_min_x_mean_u: 0.0271878745407
	test_h0_min_x_min_u: 1.29720010775e-11
	test_h0_row_norms_max: 6.49045753479
	test_h0_row_norms_mean: 3.18700146675
	test_h0_row_norms_min: 0.128459200263
	test_objective: 0.0644877254963
	test_y_col_norms_max: 10.5396261215
	test_y_col_norms_mean: 9.26142787933
	test_y_col_norms_min: 7.278901577
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.98046040535
	test_y_min_max_class: 0.25162255764
	test_y_misclass: 0.0206000003964
	test_y_nll: 0.0644877254963
	test_y_row_norms_max: 3.90689897537
	test_y_row_norms_mean: 1.18369758129
	test_y_row_norms_min: 0.177592679858
	train_h0_col_norms_max: 6.48900747299
	train_h0_col_norms_mean: 4.06139850616
	train_h0_col_norms_min: 2.15221905708
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.966917276382
	train_h0_max_x_min_u: 0.551108419895
	train_h0_mean_x_max_u: 0.935860812664
	train_h0_mean_x_mean_u: 0.467652916908
	train_h0_mean_x_min_u: 0.0830486863852
	train_h0_min_x_max_u: 0.34869286418
	train_h0_min_x_mean_u: 0.0267758108675
	train_h0_min_x_min_u: 1.72074004351e-11
	train_h0_row_norms_max: 6.49045705795
	train_h0_row_norms_mean: 3.18700098991
	train_h0_row_norms_min: 0.128459185362
	train_objective: 0.0193602163345
	train_y_col_norms_max: 10.5396251678
	train_y_col_norms_mean: 9.26142692566
	train_y_col_norms_min: 7.27890205383
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.985767424107
	train_y_min_max_class: 0.336476325989
	train_y_misclass: 0.00289999973029
	train_y_nll: 0.0193602163345
	train_y_row_norms_max: 3.90689897537
	train_y_row_norms_mean: 1.18369758129
	train_y_row_norms_min: 0.177592664957
	valid_h0_col_norms_max: 6.48900747299
	valid_h0_col_norms_mean: 4.06139850616
	valid_h0_col_norms_min: 2.1522192955
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.966026246548
	valid_h0_max_x_min_u: 0.568945586681
	valid_h0_mean_x_max_u: 0.937908649445
	valid_h0_mean_x_mean_u: 0.467648357153
	valid_h0_mean_x_min_u: 0.0835038796067
	valid_h0_min_x_max_u: 0.32678771019
	valid_h0_min_x_mean_u: 0.0274151265621
	valid_h0_min_x_min_u: 1.33190177637e-11
	valid_h0_row_norms_max: 6.49045753479
	valid_h0_row_norms_mean: 3.18700146675
	valid_h0_row_norms_min: 0.128459200263
	valid_objective: 0.0707407668233
	valid_y_col_norms_max: 10.5396261215
	valid_y_col_norms_mean: 9.26142787933
	valid_y_col_norms_min: 7.278901577
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.980556607246
	valid_y_min_max_class: 0.298538506031
	valid_y_misclass: 0.0219000000507
	valid_y_nll: 0.0707407668233
	valid_y_row_norms_max: 3.90689897537
	valid_y_row_norms_mean: 1.18369758129
	valid_y_row_norms_min: 0.177592679858
Time this epoch: 35.400439 seconds
Monitoring step:
	Epochs seen: 35
	Batches seen: 175
	Examples seen: 1750000
	ave_grad_mult: 4.3184633255
	ave_grad_size: 0.022776318714
	ave_step_size: 0.0920493155718
	test_h0_col_norms_max: 6.49970197678
	test_h0_col_norms_mean: 4.06945180893
	test_h0_col_norms_min: 2.15374016762
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.966324806213
	test_h0_max_x_min_u: 0.55324202776
	test_h0_mean_x_max_u: 0.94215297699
	test_h0_mean_x_mean_u: 0.466998904943
	test_h0_mean_x_min_u: 0.076045922935
	test_h0_min_x_max_u: 0.358031690121
	test_h0_min_x_mean_u: 0.0268286950886
	test_h0_min_x_min_u: 1.09078423377e-11
	test_h0_row_norms_max: 6.50839042664
	test_h0_row_norms_mean: 3.19361257553
	test_h0_row_norms_min: 0.129599049687
	test_objective: 0.0635969266295
	test_y_col_norms_max: 10.7156534195
	test_y_col_norms_mean: 9.42882728577
	test_y_col_norms_min: 7.41432905197
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.980757176876
	test_y_min_max_class: 0.248226299882
	test_y_misclass: 0.0193999987096
	test_y_nll: 0.0635969266295
	test_y_row_norms_max: 3.96717524529
	test_y_row_norms_mean: 1.20508480072
	test_y_row_norms_min: 0.180099412799
	train_h0_col_norms_max: 6.4997010231
	train_h0_col_norms_mean: 4.06945180893
	train_h0_col_norms_min: 2.1537399292
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.967268288136
	train_h0_max_x_min_u: 0.551735162735
	train_h0_mean_x_max_u: 0.935940921307
	train_h0_mean_x_mean_u: 0.467215240002
	train_h0_mean_x_min_u: 0.0804425179958
	train_h0_min_x_max_u: 0.343524694443
	train_h0_min_x_mean_u: 0.0264016315341
	train_h0_min_x_min_u: 1.46074211754e-11
	train_h0_row_norms_max: 6.50839090347
	train_h0_row_norms_mean: 3.19361257553
	train_h0_row_norms_min: 0.129599064589
	train_objective: 0.0171565413475
	train_y_col_norms_max: 10.7156524658
	train_y_col_norms_mean: 9.42882633209
	train_y_col_norms_min: 7.41432905197
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.986733615398
	train_y_min_max_class: 0.337424963713
	train_y_misclass: 0.00196000002325
	train_y_nll: 0.0171565413475
	train_y_row_norms_max: 3.96717500687
	train_y_row_norms_mean: 1.20508468151
	train_y_row_norms_min: 0.1800994277
	valid_h0_col_norms_max: 6.49970197678
	valid_h0_col_norms_mean: 4.06945180893
	valid_h0_col_norms_min: 2.15374016762
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.966409444809
	valid_h0_max_x_min_u: 0.564903616905
	valid_h0_mean_x_max_u: 0.938026130199
	valid_h0_mean_x_mean_u: 0.467201501131
	valid_h0_mean_x_min_u: 0.0823206305504
	valid_h0_min_x_max_u: 0.31958258152
	valid_h0_min_x_mean_u: 0.0270514041185
	valid_h0_min_x_min_u: 1.14466379084e-11
	valid_h0_row_norms_max: 6.50839042664
	valid_h0_row_norms_mean: 3.19361257553
	valid_h0_row_norms_min: 0.129599049687
	valid_objective: 0.0689148977399
	valid_y_col_norms_max: 10.7156534195
	valid_y_col_norms_mean: 9.42882728577
	valid_y_col_norms_min: 7.41432905197
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.980640649796
	valid_y_min_max_class: 0.264637023211
	valid_y_misclass: 0.021099999547
	valid_y_nll: 0.0689148977399
	valid_y_row_norms_max: 3.96717524529
	valid_y_row_norms_mean: 1.20508480072
	valid_y_row_norms_min: 0.180099412799
Time this epoch: 35.392445 seconds
Monitoring step:
	Epochs seen: 36
	Batches seen: 180
	Examples seen: 1800000
	ave_grad_mult: 4.55049180984
	ave_grad_size: 0.0215135067701
	ave_step_size: 0.0914682373405
	test_h0_col_norms_max: 6.5103468895
	test_h0_col_norms_mean: 4.07773399353
	test_h0_col_norms_min: 2.15385961533
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.967212915421
	test_h0_max_x_min_u: 0.559629559517
	test_h0_mean_x_max_u: 0.943103969097
	test_h0_mean_x_mean_u: 0.466701477766
	test_h0_mean_x_min_u: 0.0809521302581
	test_h0_min_x_max_u: 0.355958789587
	test_h0_min_x_mean_u: 0.0261587612331
	test_h0_min_x_min_u: 8.34188967902e-12
	test_h0_row_norms_max: 6.52430438995
	test_h0_row_norms_mean: 3.20039582253
	test_h0_row_norms_min: 0.130876362324
	test_objective: 0.0621786899865
	test_y_col_norms_max: 10.8925733566
	test_y_col_norms_mean: 9.60350131989
	test_y_col_norms_min: 7.55749177933
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.981753587723
	test_y_min_max_class: 0.330662488937
	test_y_misclass: 0.0190999973565
	test_y_nll: 0.0621786899865
	test_y_row_norms_max: 4.02781057358
	test_y_row_norms_mean: 1.22741234303
	test_y_row_norms_min: 0.181874185801
	train_h0_col_norms_max: 6.51034593582
	train_h0_col_norms_mean: 4.07773399353
	train_h0_col_norms_min: 2.15385937691
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.968181371689
	train_h0_max_x_min_u: 0.555752873421
	train_h0_mean_x_max_u: 0.936968684196
	train_h0_mean_x_mean_u: 0.466913819313
	train_h0_mean_x_min_u: 0.0854497775435
	train_h0_min_x_max_u: 0.338039875031
	train_h0_min_x_mean_u: 0.0257331542671
	train_h0_min_x_min_u: 1.09208597027e-11
	train_h0_row_norms_max: 6.52430438995
	train_h0_row_norms_mean: 3.20039534569
	train_h0_row_norms_min: 0.130876347423
	train_objective: 0.0157043337822
	train_y_col_norms_max: 10.892572403
	train_y_col_norms_mean: 9.60350131989
	train_y_col_norms_min: 7.55749130249
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.987937808037
	train_y_min_max_class: 0.323045521975
	train_y_misclass: 0.00203999993391
	train_y_nll: 0.0157043337822
	train_y_row_norms_max: 4.02781057358
	train_y_row_norms_mean: 1.22741222382
	train_y_row_norms_min: 0.181874185801
	valid_h0_col_norms_max: 6.5103468895
	valid_h0_col_norms_mean: 4.07773399353
	valid_h0_col_norms_min: 2.15385961533
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.967329084873
	valid_h0_max_x_min_u: 0.567208707333
	valid_h0_mean_x_max_u: 0.939025402069
	valid_h0_mean_x_mean_u: 0.466897398233
	valid_h0_mean_x_min_u: 0.0875384286046
	valid_h0_min_x_max_u: 0.311605006456
	valid_h0_min_x_mean_u: 0.0264036990702
	valid_h0_min_x_min_u: 8.81185142215e-12
	valid_h0_row_norms_max: 6.52430438995
	valid_h0_row_norms_mean: 3.20039582253
	valid_h0_row_norms_min: 0.130876362324
	valid_objective: 0.0682094246149
	valid_y_col_norms_max: 10.8925733566
	valid_y_col_norms_mean: 9.60350131989
	valid_y_col_norms_min: 7.55749177933
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.982199847698
	valid_y_min_max_class: 0.324392050505
	valid_y_misclass: 0.0208000000566
	valid_y_nll: 0.0682094246149
	valid_y_row_norms_max: 4.02781057358
	valid_y_row_norms_mean: 1.22741234303
	valid_y_row_norms_min: 0.181874185801
Time this epoch: 34.710048 seconds
Monitoring step:
	Epochs seen: 37
	Batches seen: 185
	Examples seen: 1850000
	ave_grad_mult: 4.72839355469
	ave_grad_size: 0.0204669237137
	ave_step_size: 0.0900116711855
	test_h0_col_norms_max: 6.52083969116
	test_h0_col_norms_mean: 4.08554124832
	test_h0_col_norms_min: 2.15409636497
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.967603981495
	test_h0_max_x_min_u: 0.557713389397
	test_h0_mean_x_max_u: 0.943118810654
	test_h0_mean_x_mean_u: 0.466896891594
	test_h0_mean_x_min_u: 0.0787230879068
	test_h0_min_x_max_u: 0.356404840946
	test_h0_min_x_mean_u: 0.0257684588432
	test_h0_min_x_min_u: 9.0589402299e-12
	test_h0_row_norms_max: 6.53848934174
	test_h0_row_norms_mean: 3.206792593
	test_h0_row_norms_min: 0.131754085422
	test_objective: 0.0623081922531
	test_y_col_norms_max: 11.052611351
	test_y_col_norms_mean: 9.76351451874
	test_y_col_norms_min: 7.68663883209
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.982231199741
	test_y_min_max_class: 0.287253022194
	test_y_misclass: 0.0188999995589
	test_y_nll: 0.0623081922531
	test_y_row_norms_max: 4.08399629593
	test_y_row_norms_mean: 1.24770605564
	test_y_row_norms_min: 0.185720145702
	train_h0_col_norms_max: 6.52083921432
	train_h0_col_norms_mean: 4.08554124832
	train_h0_col_norms_min: 2.15409636497
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.968553900719
	train_h0_max_x_min_u: 0.553573608398
	train_h0_mean_x_max_u: 0.937007129192
	train_h0_mean_x_mean_u: 0.467112243176
	train_h0_mean_x_min_u: 0.0832107812166
	train_h0_min_x_max_u: 0.334134042263
	train_h0_min_x_mean_u: 0.0253749713302
	train_h0_min_x_min_u: 1.24377470129e-11
	train_h0_row_norms_max: 6.5384888649
	train_h0_row_norms_mean: 3.20679235458
	train_h0_row_norms_min: 0.131754085422
	train_objective: 0.0140552837402
	train_y_col_norms_max: 11.0526103973
	train_y_col_norms_mean: 9.76351451874
	train_y_col_norms_min: 7.68663883209
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.988768100739
	train_y_min_max_class: 0.329038023949
	train_y_misclass: 0.00163999991491
	train_y_nll: 0.0140552837402
	train_y_row_norms_max: 4.08399581909
	train_y_row_norms_mean: 1.24770605564
	train_y_row_norms_min: 0.1857201159
	valid_h0_col_norms_max: 6.52083969116
	valid_h0_col_norms_mean: 4.08554124832
	valid_h0_col_norms_min: 2.15409636497
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.967728018761
	valid_h0_max_x_min_u: 0.569734930992
	valid_h0_mean_x_max_u: 0.939063310623
	valid_h0_mean_x_mean_u: 0.467084676027
	valid_h0_mean_x_min_u: 0.0852277651429
	valid_h0_min_x_max_u: 0.310160905123
	valid_h0_min_x_mean_u: 0.0260545928031
	valid_h0_min_x_min_u: 9.77232097327e-12
	valid_h0_row_norms_max: 6.53848934174
	valid_h0_row_norms_mean: 3.206792593
	valid_h0_row_norms_min: 0.131754085422
	valid_objective: 0.0679266303778
	valid_y_col_norms_max: 11.052611351
	valid_y_col_norms_mean: 9.76351451874
	valid_y_col_norms_min: 7.68663883209
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.982333242893
	valid_y_min_max_class: 0.319318085909
	valid_y_misclass: 0.0204000007361
	valid_y_nll: 0.0679266303778
	valid_y_row_norms_max: 4.08399629593
	valid_y_row_norms_mean: 1.24770605564
	valid_y_row_norms_min: 0.185720145702
Time this epoch: 35.364850 seconds
Monitoring step:
	Epochs seen: 38
	Batches seen: 190
	Examples seen: 1900000
	ave_grad_mult: 5.14290428162
	ave_grad_size: 0.0190559756011
	ave_step_size: 0.0913925841451
	test_h0_col_norms_max: 6.53183841705
	test_h0_col_norms_mean: 4.09429168701
	test_h0_col_norms_min: 2.1546792984
	test_h0_max_x_max_u: 1.0
	test_h0_max_x_mean_u: 0.968341529369
	test_h0_max_x_min_u: 0.560349822044
	test_h0_mean_x_max_u: 0.94111353159
	test_h0_mean_x_mean_u: 0.466603428125
	test_h0_mean_x_min_u: 0.0797407329082
	test_h0_min_x_max_u: 0.351446330547
	test_h0_min_x_mean_u: 0.0251497104764
	test_h0_min_x_min_u: 7.31677132076e-12
	test_h0_row_norms_max: 6.55737257004
	test_h0_row_norms_mean: 3.21393060684
	test_h0_row_norms_min: 0.132736563683
	test_objective: 0.0633104071021
	test_y_col_norms_max: 11.2310876846
	test_y_col_norms_mean: 9.94289398193
	test_y_col_norms_min: 7.82843732834
	test_y_max_max_class: 1.0
	test_y_mean_max_class: 0.982944607735
	test_y_min_max_class: 0.318380922079
	test_y_misclass: 0.0193999987096
	test_y_nll: 0.0633104071021
	test_y_row_norms_max: 4.14330053329
	test_y_row_norms_mean: 1.27068781853
	test_y_row_norms_min: 0.189937055111
	train_h0_col_norms_max: 6.53183746338
	train_h0_col_norms_mean: 4.09429121017
	train_h0_col_norms_min: 2.15467905998
	train_h0_max_x_max_u: 0.999999940395
	train_h0_max_x_mean_u: 0.969276428223
	train_h0_max_x_min_u: 0.554496645927
	train_h0_mean_x_max_u: 0.934813499451
	train_h0_mean_x_mean_u: 0.466816186905
	train_h0_mean_x_min_u: 0.0843253731728
	train_h0_min_x_max_u: 0.332267045975
	train_h0_min_x_mean_u: 0.0247781910002
	train_h0_min_x_min_u: 9.73409200467e-12
	train_h0_row_norms_max: 6.5573720932
	train_h0_row_norms_mean: 3.21393036842
	train_h0_row_norms_min: 0.132736548781
	train_objective: 0.0125638237223
	train_y_col_norms_max: 11.231086731
	train_y_col_norms_mean: 9.94289398193
	train_y_col_norms_min: 7.82843637466
	train_y_max_max_class: 0.999999940395
	train_y_mean_max_class: 0.989765167236
	train_y_min_max_class: 0.37343031168
	train_y_misclass: 0.00133999995887
	train_y_nll: 0.0125638237223
	train_y_row_norms_max: 4.14330005646
	train_y_row_norms_mean: 1.27068758011
	train_y_row_norms_min: 0.189937055111
	valid_h0_col_norms_max: 6.53183841705
	valid_h0_col_norms_mean: 4.09429168701
	valid_h0_col_norms_min: 2.1546792984
	valid_h0_max_x_max_u: 1.0
	valid_h0_max_x_mean_u: 0.968490362167
	valid_h0_max_x_min_u: 0.566503345966
	valid_h0_mean_x_max_u: 0.93706715107
	valid_h0_mean_x_mean_u: 0.466789364815
	valid_h0_mean_x_min_u: 0.0863413140178
	valid_h0_min_x_max_u: 0.307645887136
	valid_h0_min_x_mean_u: 0.0253705345094
	valid_h0_min_x_min_u: 7.80784985277e-12
	valid_h0_row_norms_max: 6.55737257004
	valid_h0_row_norms_mean: 3.21393060684
	valid_h0_row_norms_min: 0.132736563683
	valid_objective: 0.0684154629707
	valid_y_col_norms_max: 11.2310876846
	valid_y_col_norms_mean: 9.94289398193
	valid_y_col_norms_min: 7.82843732834
	valid_y_max_max_class: 1.0
	valid_y_mean_max_class: 0.983206391335
	valid_y_min_max_class: 0.354223191738
	valid_y_misclass: 0.0201999973506
	valid_y_nll: 0.0684154629707
	valid_y_row_norms_max: 4.14330053329
	valid_y_row_norms_mean: 1.27068781853
	valid_y_row_norms_min: 0.189937055111

As the model trained, it should have printed out progress messages. Most of these are the values of the various channels being monitored throughout training.

We can use the print_monitor script to print the last monitoring entry of a saved model. By running it on "mlp_best.pkl", we can see the performance of the model at the point where it did the best on the validation set.



In [4]:

    
!print_monitor.py mlp_best.pkl | grep test_y_misclass









    



Using gpu device 2: GeForce GTX 285
/u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit.
  warnings.warn("MLP changing the recursion limit.")
test_y_misclass : 0.0193999987096

The test set error has dropped to 1.94%! This is a big improvement over softmax regression.

Another common way of analyzing trained models is to look at their weights. Here we use the show_weights script to visualize $W$:



In [5]:

    
!show_weights.py mlp_best.pkl









    



Using gpu device 0: GeForce GTX 285
making weights report
loading model
loading done
loading dataset...
...done
smallest enc weight magnitude: 0.0
mean enc weight magnitude: 0.0409141770966
max enc weight magnitude: 4.76068
min norm:  2.15468
mean norm:  4.09429199219
max norm:  6.53184

Part 3 A deeper MLP, and pylearn2 polymorphism

So far in these tutorials, there has not been much benefit to using pylearn2, rather than some other machine learning library, or even just an implementation of softmax regression or an MLP without an accompanying library.

Now it's time to see some of why pylearn2 is useful. We're going to make several changes to our experimental setup, while still re-using most of the code. The beauty of pylearn2 is that it is built from interchangeable parts, so that if you want to create a new machine learning experiment, you don't need to rewrite the whole experiment from scratch.

We're going to take the MLP example above and change it in three major ways:

-Instead of training just a two layer MLP, we'll train a three layer MLP. We can do this just by putting one more layer in the "layers" list. We don't need to change the training algorithm or the main MLP model.

-Instead of using the Sigmoid Layer class, we'll use a different kind of layer, called a rectified linear layer. The rectified linear layer uses the usual affine function $z = x^T W + b$ to compute the presynaptic inputs, then passes each element of $z$ through the function $g(z) = \mathbb{I}_{z > 0} z$. In other words, values greater than 0 are left unchanged, while negative values are replaced with zeros. In pylearn2, we can do this just by loading a different class in the layers list. We don't need to change the training algorithm or the main MLP model.

-Instead of optimizing the log likelihood using the nonlinear conjugate gradient descent algorithm, we will optimize it using a minibatch version of stochastic gradient descent. We can do this just by passing in a different TrainingAlgorithm object. No changes to the model or the code for the cost are needed.

Here is the updated YAML description of the experiment:



In [6]:

    
import os
import pylearn2
path = os.path.join(pylearn2.__path__[0], 'scripts', 'tutorials', 'multilayer_perceptron', 'mlp_tutorial_part_3.yaml')
with open(path, 'r') as f:
    train_2 = f.read()
hyper_params = {'train_stop' : 50000,
                'valid_stop' : 60000,
                'dim_h0' : 500,
                'dim_h1' : 1000,
                'sparse_init_h1' : 15,
                'max_epochs' : 10000,
                'save_path' : '.'}
train_2 = train_2 % (hyper_params)
print train_2









    



!obj:pylearn2.train.Train {
    dataset: &train !obj:pylearn2.datasets.mnist.MNIST {
        which_set: 'train',
        start: 0,
        stop: 50000
    },
    model: !obj:pylearn2.models.mlp.MLP {
        layers: [ !obj:pylearn2.models.mlp.RectifiedLinear {
                     layer_name: 'h0',
                     dim: 500,
                     sparse_init: 15
                 }, !obj:pylearn2.models.mlp.RectifiedLinear {
                     layer_name: 'h1',
                     dim: 1000,
                     sparse_init: 15
                 }, !obj:pylearn2.models.mlp.Softmax {
                     layer_name: 'y',
                     n_classes: 10,
                     irange: 0.
                 }
                ],
        nvis: 784,
    },
    algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
        batch_size: 100,
        learning_rate: .01,
        monitoring_dataset:
            {
                'train' : *train,
                'valid' : !obj:pylearn2.datasets.mnist.MNIST {
                              which_set: 'train',
                              start: 50000,
                              stop: 60000
                          },
                'test'  : !obj:pylearn2.datasets.mnist.MNIST {
                              which_set: 'test',
                          }
            },
        learning_rule: !obj:pylearn2.training_algorithms.learning_rule.Momentum {
            init_momentum: .5
        },
        termination_criterion: !obj:pylearn2.termination_criteria.And {
            criteria: [
                !obj:pylearn2.termination_criteria.MonitorBased {
                    channel_name: "valid_y_misclass",
                    prop_decrease: 0.,
                    N: 10
                },
                !obj:pylearn2.termination_criteria.EpochCounter {
                    max_epochs: 10000
                }
            ]
        }
    },
    extensions: [ !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest {
             channel_name: 'valid_y_misclass',
             save_path: "mlp_2_best.pkl"
        }, !obj:pylearn2.training_algorithms.learning_rule.MomentumAdjustor {
            start: 1,
            saturate: 10,
            final_momentum: .99
        }
    ]
}

This YAML config file also introduces another use of extensions to the Train object. Here, we add the MomentumAdjustor. It uses a callback to adjust the momentum setting of the SGD algorithm at the end of each epoch. Here, we configure it to start increasing the momentum after 1 epoch, and to continue increasing it until it reaches a value of .99 at the end of the tenth epoch. See the docstring for the SGD class for more information on what this momentum setting does.



In [7]:

    
from pylearn2.config import yaml_parse
train_2 = yaml_parse.load(train_2)
train_2.main_loop()









    



Parameter and initial learning rate summary:
	h0_W: 0.00999999977648
	h0_b: 0.00999999977648
	h1_W: 0.00999999977648
	h1_b: 0.00999999977648
	softmax_b: 0.00999999977648
	softmax_W: 0.00999999977648
Compiling sgd_update...
Compiling sgd_update done. Time elapsed: 2.516152 seconds
compiling begin_record_entry...
compiling begin_record_entry done. Time elapsed: 0.395491 seconds
Monitored channels: 
	learning_rate
	momentum
	test_h0_col_norms_max
	test_h0_col_norms_mean
	test_h0_col_norms_min
	test_h0_row_norms_max
	test_h0_row_norms_mean
	test_h0_row_norms_min
	test_h1_col_norms_max
	test_h1_col_norms_mean
	test_h1_col_norms_min
	test_h1_row_norms_max
	test_h1_row_norms_mean
	test_h1_row_norms_min
	test_objective
	test_y_col_norms_max
	test_y_col_norms_mean
	test_y_col_norms_min
	test_y_max_max_class
	test_y_mean_max_class
	test_y_min_max_class
	test_y_misclass
	test_y_nll
	test_y_row_norms_max
	test_y_row_norms_mean
	test_y_row_norms_min
	train_h0_col_norms_max
	train_h0_col_norms_mean
	train_h0_col_norms_min
	train_h0_row_norms_max
	train_h0_row_norms_mean
	train_h0_row_norms_min
	train_h1_col_norms_max
	train_h1_col_norms_mean
	train_h1_col_norms_min
	train_h1_row_norms_max
	train_h1_row_norms_mean
	train_h1_row_norms_min
	train_objective
	train_y_col_norms_max
	train_y_col_norms_mean
	train_y_col_norms_min
	train_y_max_max_class
	train_y_mean_max_class
	train_y_min_max_class
	train_y_misclass
	train_y_nll
	train_y_row_norms_max
	train_y_row_norms_mean
	train_y_row_norms_min
	valid_h0_col_norms_max
	valid_h0_col_norms_mean
	valid_h0_col_norms_min
	valid_h0_row_norms_max
	valid_h0_row_norms_mean
	valid_h0_row_norms_min
	valid_h1_col_norms_max
	valid_h1_col_norms_mean
	valid_h1_col_norms_min
	valid_h1_row_norms_max
	valid_h1_row_norms_mean
	valid_h1_row_norms_min
	valid_objective
	valid_y_col_norms_max
	valid_y_col_norms_mean
	valid_y_col_norms_min
	valid_y_max_max_class
	valid_y_mean_max_class
	valid_y_min_max_class
	valid_y_misclass
	valid_y_nll
	valid_y_row_norms_max
	valid_y_row_norms_mean
	valid_y_row_norms_min
Compiling accum...
graph size: 165
graph size: 163
graph size: 163
Compiling accum done. Time elapsed: 11.563393 seconds
Monitoring step:
	Epochs seen: 0
	Batches seen: 0
	Examples seen: 0
	learning_rate: 0.00999999046326
	momentum: 0.499999672174
	test_h0_col_norms_max: 6.23503017426
	test_h0_col_norms_mean: 3.82356023788
	test_h0_col_norms_min: 2.06193947792
	test_h0_row_norms_max: 5.89326524734
	test_h0_row_norms_mean: 2.98549389839
	test_h0_row_norms_min: 0.0
	test_h1_col_norms_max: 5.99438333511
	test_h1_col_norms_mean: 3.80721712112
	test_h1_col_norms_min: 1.71524214745
	test_h1_row_norms_max: 7.80886650085
	test_h1_row_norms_mean: 5.40815734863
	test_h1_row_norms_min: 2.97773504257
	test_objective: 2.30258488655
	test_y_col_norms_max: 0.0
	test_y_col_norms_mean: 0.0
	test_y_col_norms_min: 0.0
	test_y_max_max_class: 0.100000023842
	test_y_mean_max_class: 0.100000031292
	test_y_min_max_class: 0.100000023842
	test_y_misclass: 0.901999890804
	test_y_nll: 2.30258488655
	test_y_row_norms_max: 0.0
	test_y_row_norms_mean: 0.0
	test_y_row_norms_min: 0.0
	train_h0_col_norms_max: 6.23505115509
	train_h0_col_norms_mean: 3.82354259491
	train_h0_col_norms_min: 2.0619494915
	train_h0_row_norms_max: 5.89324569702
	train_h0_row_norms_mean: 2.98548007011
	train_h0_row_norms_min: 0.0
	train_h1_col_norms_max: 5.99438095093
	train_h1_col_norms_mean: 3.80721092224
	train_h1_col_norms_min: 1.71524274349
	train_h1_row_norms_max: 7.80887794495
	train_h1_row_norms_mean: 5.40813541412
	train_h1_row_norms_min: 2.97772955894
	train_objective: 2.30257916451
	train_y_col_norms_max: 0.0
	train_y_col_norms_mean: 0.0
	train_y_col_norms_min: 0.0
	train_y_max_max_class: 0.100000545382
	train_y_mean_max_class: 0.100000545382
	train_y_min_max_class: 0.100000545382
	train_y_misclass: 0.901360213757
	train_y_nll: 2.30257916451
	train_y_row_norms_max: 0.0
	train_y_row_norms_mean: 0.0
	train_y_row_norms_min: 0.0
	valid_h0_col_norms_max: 6.23503017426
	valid_h0_col_norms_mean: 3.82356023788
	valid_h0_col_norms_min: 2.06193947792
	valid_h0_row_norms_max: 5.89326524734
	valid_h0_row_norms_mean: 2.98549389839
	valid_h0_row_norms_min: 0.0
	valid_h1_col_norms_max: 5.99438333511
	valid_h1_col_norms_mean: 3.80721712112
	valid_h1_col_norms_min: 1.71524214745
	valid_h1_row_norms_max: 7.80886650085
	valid_h1_row_norms_mean: 5.40815734863
	valid_h1_row_norms_min: 2.97773504257
	valid_objective: 2.30258488655
	valid_y_col_norms_max: 0.0
	valid_y_col_norms_mean: 0.0
	valid_y_col_norms_min: 0.0
	valid_y_max_max_class: 0.100000023842
	valid_y_mean_max_class: 0.100000031292
	valid_y_min_max_class: 0.100000023842
	valid_y_misclass: 0.90089994669
	valid_y_nll: 2.30258488655
	valid_y_row_norms_max: 0.0
	valid_y_row_norms_mean: 0.0
	valid_y_row_norms_min: 0.0
Time this epoch: 3.343442 seconds
Monitoring step:
	Epochs seen: 1
	Batches seen: 500
	Examples seen: 50000
	learning_rate: 0.00999999046326
	momentum: 0.499999672174
	test_h0_col_norms_max: 6.23488473892
	test_h0_col_norms_mean: 3.82359194756
	test_h0_col_norms_min: 2.06265735626
	test_h0_row_norms_max: 5.89264249802
	test_h0_row_norms_mean: 2.98556685448
	test_h0_row_norms_min: 0.00163861282635
	test_h1_col_norms_max: 5.99485731125
	test_h1_col_norms_mean: 3.80723309517
	test_h1_col_norms_min: 1.71526324749
	test_h1_row_norms_max: 7.80893564224
	test_h1_row_norms_mean: 5.40817546844
	test_h1_row_norms_min: 2.97778272629
	test_objective: 0.268750548363
	test_y_col_norms_max: 0.645500898361
	test_y_col_norms_mean: 0.596350252628
	test_y_col_norms_min: 0.520334303379
	test_y_max_max_class: 0.999946475029
	test_y_mean_max_class: 0.904475390911
	test_y_min_max_class: 0.38064879179
	test_y_misclass: 0.0812000110745
	test_y_nll: 0.268750548363
	test_y_row_norms_max: 0.17966529727
	test_y_row_norms_mean: 0.0518538914621
	test_y_row_norms_min: 0.000149252169649
	train_h0_col_norms_max: 6.23488473892
	train_h0_col_norms_mean: 3.82361268997
	train_h0_col_norms_min: 2.06266713142
	train_h0_row_norms_max: 5.89267301559
	train_h0_row_norms_mean: 2.98556661606
	train_h0_row_norms_min: 0.001638607122
	train_h1_col_norms_max: 5.99485683441
	train_h1_col_norms_mean: 3.80721235275
	train_h1_col_norms_min: 1.71525621414
	train_h1_row_norms_max: 7.80892753601
	train_h1_row_norms_mean: 5.4081993103
	train_h1_row_norms_min: 2.97776818275
	train_objective: 0.264730095863
	train_y_col_norms_max: 0.645499527454
	train_y_col_norms_mean: 0.596347033978
	train_y_col_norms_min: 0.520334303379
	train_y_max_max_class: 0.999963521957
	train_y_mean_max_class: 0.899078428745
	train_y_min_max_class: 0.361695259809
	train_y_misclass: 0.0793600603938
	train_y_nll: 0.264730095863
	train_y_row_norms_max: 0.179665282369
	train_y_row_norms_mean: 0.051854070276
	train_y_row_norms_min: 0.000149251762195
	valid_h0_col_norms_max: 6.23488473892
	valid_h0_col_norms_mean: 3.82359194756
	valid_h0_col_norms_min: 2.06265735626
	valid_h0_row_norms_max: 5.89264249802
	valid_h0_row_norms_mean: 2.98556685448
	valid_h0_row_norms_min: 0.00163861282635
	valid_h1_col_norms_max: 5.99485731125
	valid_h1_col_norms_mean: 3.80723309517
	valid_h1_col_norms_min: 1.71526324749
	valid_h1_row_norms_max: 7.80893564224
	valid_h1_row_norms_mean: 5.40817546844
	valid_h1_row_norms_min: 2.97778272629
	valid_objective: 0.252131432295
	valid_y_col_norms_max: 0.645500898361
	valid_y_col_norms_mean: 0.596350252628
	valid_y_col_norms_min: 0.520334303379
	valid_y_max_max_class: 0.999965012074
	valid_y_mean_max_class: 0.907301902771
	valid_y_min_max_class: 0.362495720387
	valid_y_misclass: 0.0754000097513
	valid_y_nll: 0.252131432295
	valid_y_row_norms_max: 0.17966529727
	valid_y_row_norms_mean: 0.0518538914621
	valid_y_row_norms_min: 0.000149252169649
Time this epoch: 3.325040 seconds
Monitoring step:
	Epochs seen: 2
	Batches seen: 1000
	Examples seen: 100000
	learning_rate: 0.00999999046326
	momentum: 0.554444551468
	test_h0_col_norms_max: 6.2346944809
	test_h0_col_norms_mean: 3.82387781143
	test_h0_col_norms_min: 2.06334352493
	test_h0_row_norms_max: 5.89264249802
	test_h0_row_norms_mean: 2.98581314087
	test_h0_row_norms_min: 0.00337248062715
	test_h1_col_norms_max: 5.99546384811
	test_h1_col_norms_mean: 3.80735421181
	test_h1_col_norms_min: 1.71530222893
	test_h1_row_norms_max: 7.80887699127
	test_h1_row_norms_mean: 5.40835094452
	test_h1_row_norms_min: 2.97777676582
	test_objective: 0.209201917052
	test_y_col_norms_max: 0.849824726582
	test_y_col_norms_mean: 0.752399742603
	test_y_col_norms_min: 0.648707330227
	test_y_max_max_class: 0.999981224537
	test_y_mean_max_class: 0.928354024887
	test_y_min_max_class: 0.417280673981
	test_y_misclass: 0.0621000118554
	test_y_nll: 0.209201917052
	test_y_row_norms_max: 0.202846974134
	test_y_row_norms_mean: 0.0668164640665
	test_y_row_norms_min: 0.000276584294625
	train_h0_col_norms_max: 6.23466491699
	train_h0_col_norms_mean: 3.82387685776
	train_h0_col_norms_min: 2.06333851814
	train_h0_row_norms_max: 5.89267301559
	train_h0_row_norms_mean: 2.98582696915
	train_h0_row_norms_min: 0.00337246293202
	train_h1_col_norms_max: 5.99549293518
	train_h1_col_norms_mean: 3.80733585358
	train_h1_col_norms_min: 1.71530234814
	train_h1_row_norms_max: 7.80891132355
	train_h1_row_norms_mean: 5.4083533287
	train_h1_row_norms_min: 2.97776651382
	train_objective: 0.192548781633
	train_y_col_norms_max: 0.849820315838
	train_y_col_norms_mean: 0.752397358418
	train_y_col_norms_min: 0.648707211018
	train_y_max_max_class: 0.999981343746
	train_y_mean_max_class: 0.925991177559
	train_y_min_max_class: 0.379428476095
	train_y_misclass: 0.0572400614619
	train_y_nll: 0.192548781633
	train_y_row_norms_max: 0.202847748995
	train_y_row_norms_mean: 0.0668167173862
	train_y_row_norms_min: 0.000276583392406
	valid_h0_col_norms_max: 6.2346944809
	valid_h0_col_norms_mean: 3.82387781143
	valid_h0_col_norms_min: 2.06334352493
	valid_h0_row_norms_max: 5.89264249802
	valid_h0_row_norms_mean: 2.98581314087
	valid_h0_row_norms_min: 0.00337248062715
	valid_h1_col_norms_max: 5.99546384811
	valid_h1_col_norms_mean: 3.80735421181
	valid_h1_col_norms_min: 1.71530222893
	valid_h1_row_norms_max: 7.80887699127
	valid_h1_row_norms_mean: 5.40835094452
	valid_h1_row_norms_min: 2.97777676582
	valid_objective: 0.201314240694
	valid_y_col_norms_max: 0.849824726582
	valid_y_col_norms_mean: 0.752399742603
	valid_y_col_norms_min: 0.648707330227
	valid_y_max_max_class: 0.999982595444
	valid_y_mean_max_class: 0.93180680275
	valid_y_min_max_class: 0.40289413929
	valid_y_misclass: 0.0579000003636
	valid_y_nll: 0.201314240694
	valid_y_row_norms_max: 0.202846974134
	valid_y_row_norms_mean: 0.0668164640665
	valid_y_row_norms_min: 0.000276584294625
Time this epoch: 3.321143 seconds
Monitoring step:
	Epochs seen: 3
	Batches seen: 1500
	Examples seen: 150000
	learning_rate: 0.00999999046326
	momentum: 0.608888924122
	test_h0_col_norms_max: 6.23464679718
	test_h0_col_norms_mean: 3.82416844368
	test_h0_col_norms_min: 2.06404829025
	test_h0_row_norms_max: 5.89243221283
	test_h0_row_norms_mean: 2.98607397079
	test_h0_row_norms_min: 0.00511313043535
	test_h1_col_norms_max: 5.99604940414
	test_h1_col_norms_mean: 3.80747485161
	test_h1_col_norms_min: 1.71535277367
	test_h1_row_norms_max: 7.80883836746
	test_h1_row_norms_mean: 5.40852594376
	test_h1_row_norms_min: 2.97782230377
	test_objective: 0.18524043262
	test_y_col_norms_max: 1.00719892979
	test_y_col_norms_mean: 0.879001736641
	test_y_col_norms_min: 0.748181402683
	test_y_max_max_class: 0.999993741512
	test_y_mean_max_class: 0.939781844616
	test_y_min_max_class: 0.445061296225
	test_y_misclass: 0.0548000186682
	test_y_nll: 0.18524043262
	test_y_row_norms_max: 0.216917276382
	test_y_row_norms_mean: 0.0788432434201
	test_y_row_norms_min: 0.000395227049012
	train_h0_col_norms_max: 6.23464632034
	train_h0_col_norms_mean: 3.82414579391
	train_h0_col_norms_min: 2.06404733658
	train_h0_row_norms_max: 5.89245033264
	train_h0_row_norms_mean: 2.98607373238
	train_h0_row_norms_min: 0.00511312671006
	train_h1_col_norms_max: 5.99604892731
	train_h1_col_norms_mean: 3.80745625496
	train_h1_col_norms_min: 1.71535873413
	train_h1_row_norms_max: 7.80887460709
	train_h1_row_norms_mean: 5.40852594376
	train_h1_row_norms_min: 2.9778380394
	train_objective: 0.161898091435
	train_y_col_norms_max: 1.00719916821
	train_y_col_norms_mean: 0.87899774313
	train_y_col_norms_min: 0.748184919357
	train_y_max_max_class: 0.999991238117
	train_y_mean_max_class: 0.93733805418
	train_y_min_max_class: 0.405598640442
	train_y_misclass: 0.0483000576496
	train_y_nll: 0.161898091435
	train_y_row_norms_max: 0.216916337609
	train_y_row_norms_mean: 0.0788431763649
	train_y_row_norms_min: 0.000395228940761
	valid_h0_col_norms_max: 6.23464679718
	valid_h0_col_norms_mean: 3.82416844368
	valid_h0_col_norms_min: 2.06404829025
	valid_h0_row_norms_max: 5.89243221283
	valid_h0_row_norms_mean: 2.98607397079
	valid_h0_row_norms_min: 0.00511313043535
	valid_h1_col_norms_max: 5.99604940414
	valid_h1_col_norms_mean: 3.80747485161
	valid_h1_col_norms_min: 1.71535277367
	valid_h1_row_norms_max: 7.80883836746
	valid_h1_row_norms_mean: 5.40852594376
	valid_h1_row_norms_min: 2.97782230377
	valid_objective: 0.174453571439
	valid_y_col_norms_max: 1.00719892979
	valid_y_col_norms_mean: 0.879001736641
	valid_y_col_norms_min: 0.748181402683
	valid_y_max_max_class: 0.999995052814
	valid_y_mean_max_class: 0.94245827198
	valid_y_min_max_class: 0.418575078249
	valid_y_misclass: 0.0514000207186
	valid_y_nll: 0.174453571439
	valid_y_row_norms_max: 0.216917276382
	valid_y_row_norms_mean: 0.0788432434201
	valid_y_row_norms_min: 0.000395227049012
Time this epoch: 3.407873 seconds
Monitoring step:
	Epochs seen: 4
	Batches seen: 2000
	Examples seen: 200000
	learning_rate: 0.00999999046326
	momentum: 0.663333714008
	test_h0_col_norms_max: 6.23483276367
	test_h0_col_norms_mean: 3.82449483871
	test_h0_col_norms_min: 2.06498026848
	test_h0_row_norms_max: 5.89247989655
	test_h0_row_norms_mean: 2.98636126518
	test_h0_row_norms_min: 0.00637936964631
	test_h1_col_norms_max: 5.99670314789
	test_h1_col_norms_mean: 3.80761146545
	test_h1_col_norms_min: 1.71540987492
	test_h1_row_norms_max: 7.80886650085
	test_h1_row_norms_mean: 5.40871572495
	test_h1_row_norms_min: 2.97799134254
	test_objective: 0.167924150825
	test_y_col_norms_max: 1.14452064037
	test_y_col_norms_mean: 0.995063841343
	test_y_col_norms_min: 0.840617954731
	test_y_max_max_class: 0.99999588728
	test_y_mean_max_class: 0.946992635727
	test_y_min_max_class: 0.455186247826
	test_y_misclass: 0.0552000291646
	test_y_nll: 0.167924150825
	test_y_row_norms_max: 0.23083357513
	test_y_row_norms_mean: 0.08986672014
	test_y_row_norms_min: 0.000483248528326
	train_h0_col_norms_max: 6.2348651886
	train_h0_col_norms_mean: 3.82447862625
	train_h0_col_norms_min: 2.06498932838
	train_h0_row_norms_max: 5.89249992371
	train_h0_row_norms_mean: 2.98634982109
	train_h0_row_norms_min: 0.00637934077531
	train_h1_col_norms_max: 5.99670362473
	train_h1_col_norms_mean: 3.80763316154
	train_h1_col_norms_min: 1.71541762352
	train_h1_row_norms_max: 7.80887794495
	train_h1_row_norms_mean: 5.40874290466
	train_h1_row_norms_min: 2.97797679901
	train_objective: 0.138446286321
	train_y_col_norms_max: 1.1445235014
	train_y_col_norms_mean: 0.995067954063
	train_y_col_norms_min: 0.840613126755
	train_y_max_max_class: 0.999992251396
	train_y_mean_max_class: 0.945943057537
	train_y_min_max_class: 0.423846125603
	train_y_misclass: 0.0430600605905
	train_y_nll: 0.138446286321
	train_y_row_norms_max: 0.230833858252
	train_y_row_norms_mean: 0.0898664072156
	train_y_row_norms_min: 0.000483250943944
	valid_h0_col_norms_max: 6.23483276367
	valid_h0_col_norms_mean: 3.82449483871
	valid_h0_col_norms_min: 2.06498026848
	valid_h0_row_norms_max: 5.89247989655
	valid_h0_row_norms_mean: 2.98636126518
	valid_h0_row_norms_min: 0.00637936964631
	valid_h1_col_norms_max: 5.99670314789
	valid_h1_col_norms_mean: 3.80761146545
	valid_h1_col_norms_min: 1.71540987492
	valid_h1_row_norms_max: 7.80886650085
	valid_h1_row_norms_mean: 5.40871572495
	valid_h1_row_norms_min: 2.97799134254
	valid_objective: 0.157675400376
	valid_y_col_norms_max: 1.14452064037
	valid_y_col_norms_mean: 0.995063841343
	valid_y_col_norms_min: 0.840617954731
	valid_y_max_max_class: 0.999996602535
	valid_y_mean_max_class: 0.949966013432
	valid_y_min_max_class: 0.442742049694
	valid_y_misclass: 0.046300008893
	valid_y_nll: 0.157675400376
	valid_y_row_norms_max: 0.23083357513
	valid_y_row_norms_mean: 0.08986672014
	valid_y_row_norms_min: 0.000483248528326
Time this epoch: 3.220654 seconds
Monitoring step:
	Epochs seen: 5
	Batches seen: 2500
	Examples seen: 250000
	learning_rate: 0.00999999046326
	momentum: 0.717777192593
	test_h0_col_norms_max: 6.23521852493
	test_h0_col_norms_mean: 3.82483482361
	test_h0_col_norms_min: 2.06603121758
	test_h0_row_norms_max: 5.89207363129
	test_h0_row_norms_mean: 2.98667144775
	test_h0_row_norms_min: 0.00797319039702
	test_h1_col_norms_max: 5.99737501144
	test_h1_col_norms_mean: 3.80774116516
	test_h1_col_norms_min: 1.71550190449
	test_h1_row_norms_max: 7.80892467499
	test_h1_row_norms_mean: 5.40890693665
	test_h1_row_norms_min: 2.97820734978
	test_objective: 0.13814201951
	test_y_col_norms_max: 1.26785862446
	test_y_col_norms_mean: 1.10942089558
	test_y_col_norms_min: 0.9239538908
	test_y_max_max_class: 0.999995410442
	test_y_mean_max_class: 0.953776538372
	test_y_min_max_class: 0.461881011724
	test_y_misclass: 0.0431000031531
	test_y_nll: 0.13814201951
	test_y_row_norms_max: 0.258687496185
	test_y_row_norms_mean: 0.10072222352
	test_y_row_norms_min: 0.000603844528086
	train_h0_col_norms_max: 6.23519468307
	train_h0_col_norms_mean: 3.82483053207
	train_h0_col_norms_min: 2.06602716446
	train_h0_row_norms_max: 5.89205408096
	train_h0_row_norms_mean: 2.98667001724
	train_h0_row_norms_min: 0.0079732267186
	train_h1_col_norms_max: 5.99740314484
	train_h1_col_norms_mean: 3.80775809288
	train_h1_col_norms_min: 1.71549510956
	train_h1_row_norms_max: 7.80892419815
	train_h1_row_norms_mean: 5.40891933441
	train_h1_row_norms_min: 2.97820615768
	train_objective: 0.104295127094
	train_y_col_norms_max: 1.26785480976
	train_y_col_norms_mean: 1.109421134
	train_y_col_norms_min: 0.923955321312
	train_y_max_max_class: 0.999992787838
	train_y_mean_max_class: 0.954641282558
	train_y_min_max_class: 0.442351669073
	train_y_misclass: 0.0312000326812
	train_y_nll: 0.104295127094
	train_y_row_norms_max: 0.258685946465
	train_y_row_norms_mean: 0.100721813738
	train_y_row_norms_min: 0.000603846099693
	valid_h0_col_norms_max: 6.23521852493
	valid_h0_col_norms_mean: 3.82483482361
	valid_h0_col_norms_min: 2.06603121758
	valid_h0_row_norms_max: 5.89207363129
	valid_h0_row_norms_mean: 2.98667144775
	valid_h0_row_norms_min: 0.00797319039702
	valid_h1_col_norms_max: 5.99737501144
	valid_h1_col_norms_mean: 3.80774116516
	valid_h1_col_norms_min: 1.71550190449
	valid_h1_row_norms_max: 7.80892467499
	valid_h1_row_norms_mean: 5.40890693665
	valid_h1_row_norms_min: 2.97820734978
	valid_objective: 0.136576414108
	valid_y_col_norms_max: 1.26785862446
	valid_y_col_norms_mean: 1.10942089558
	valid_y_col_norms_min: 0.9239538908
	valid_y_max_max_class: 0.999996840954
	valid_y_mean_max_class: 0.956140458584
	valid_y_min_max_class: 0.448911756277
	valid_y_misclass: 0.0386999994516
	valid_y_nll: 0.136576414108
	valid_y_row_norms_max: 0.258687496185
	valid_y_row_norms_mean: 0.10072222352
	valid_y_row_norms_min: 0.000603844528086
Time this epoch: 3.204515 seconds
Monitoring step:
	Epochs seen: 6
	Batches seen: 3000
	Examples seen: 300000
	learning_rate: 0.00999999046326
	momentum: 0.772221684456
	test_h0_col_norms_max: 6.23541164398
	test_h0_col_norms_mean: 3.82526040077
	test_h0_col_norms_min: 2.0674469471
	test_h0_row_norms_max: 5.89197492599
	test_h0_row_norms_mean: 2.98706746101
	test_h0_row_norms_min: 0.00963484868407
	test_h1_col_norms_max: 5.9978518486
	test_h1_col_norms_mean: 3.80790233612
	test_h1_col_norms_min: 1.71558940411
	test_h1_row_norms_max: 7.80901002884
	test_h1_row_norms_mean: 5.40913200378
	test_h1_row_norms_min: 2.97820520401
	test_objective: 0.12612003088
	test_y_col_norms_max: 1.39495909214
	test_y_col_norms_mean: 1.23315572739
	test_y_col_norms_min: 1.02864944935
	test_y_max_max_class: 0.999998807907
	test_y_mean_max_class: 0.961598396301
	test_y_min_max_class: 0.503333091736
	test_y_misclass: 0.040100004524
	test_y_nll: 0.12612003088
	test_y_row_norms_max: 0.288501292467
	test_y_row_norms_mean: 0.112407810986
	test_y_row_norms_min: 0.000765459961258
	train_h0_col_norms_max: 6.23538017273
	train_h0_col_norms_mean: 3.82528162003
	train_h0_col_norms_min: 2.0674469471
	train_h0_row_norms_max: 5.89197683334
	train_h0_row_norms_mean: 2.98705887794
	train_h0_row_norms_min: 0.00963485334069
	train_h1_col_norms_max: 5.99787998199
	train_h1_col_norms_mean: 3.80790233612
	train_h1_col_norms_min: 1.7155970335
	train_h1_row_norms_max: 7.80897331238
	train_h1_row_norms_mean: 5.40915393829
	train_h1_row_norms_min: 2.97820544243
	train_objective: 0.0812869444489
	train_y_col_norms_max: 1.39496576786
	train_y_col_norms_mean: 1.23315918446
	train_y_col_norms_min: 1.02865147591
	train_y_max_max_class: 0.99999409914
	train_y_mean_max_class: 0.963725090027
	train_y_min_max_class: 0.476592302322
	train_y_misclass: 0.0230800136924
	train_y_nll: 0.0812869444489
	train_y_row_norms_max: 0.288501352072
	train_y_row_norms_mean: 0.112407691777
	train_y_row_norms_min: 0.00076545990305
	valid_h0_col_norms_max: 6.23541164398
	valid_h0_col_norms_mean: 3.82526040077
	valid_h0_col_norms_min: 2.0674469471
	valid_h0_row_norms_max: 5.89197492599
	valid_h0_row_norms_mean: 2.98706746101
	valid_h0_row_norms_min: 0.00963484868407
	valid_h1_col_norms_max: 5.9978518486
	valid_h1_col_norms_mean: 3.80790233612
	valid_h1_col_norms_min: 1.71558940411
	valid_h1_row_norms_max: 7.80901002884
	valid_h1_row_norms_mean: 5.40913200378
	valid_h1_row_norms_min: 2.97820520401
	valid_objective: 0.127863824368
	valid_y_col_norms_max: 1.39495909214
	valid_y_col_norms_mean: 1.23315572739
	valid_y_col_norms_min: 1.02864944935
	valid_y_max_max_class: 0.999999046326
	valid_y_mean_max_class: 0.964188098907
	valid_y_min_max_class: 0.480807334185
	valid_y_misclass: 0.0376999974251
	valid_y_nll: 0.127863824368
	valid_y_row_norms_max: 0.288501292467
	valid_y_row_norms_mean: 0.112407810986
	valid_y_row_norms_min: 0.000765459961258
Time this epoch: 3.235264 seconds
Monitoring step:
	Epochs seen: 7
	Batches seen: 3500
	Examples seen: 350000
	learning_rate: 0.00999999046326
	momentum: 0.826667308807
	test_h0_col_norms_max: 6.23617553711
	test_h0_col_norms_mean: 3.82576131821
	test_h0_col_norms_min: 2.06955361366
	test_h0_row_norms_max: 5.8926115036
	test_h0_row_norms_mean: 2.98752951622
	test_h0_row_norms_min: 0.011014319025
	test_h1_col_norms_max: 5.99838781357
	test_h1_col_norms_mean: 3.8080675602
	test_h1_col_norms_min: 1.71574032307
	test_h1_row_norms_max: 7.80883789062
	test_h1_row_norms_mean: 5.40936756134
	test_h1_row_norms_min: 2.97880935669
	test_objective: 0.127731248736
	test_y_col_norms_max: 1.54538154602
	test_y_col_norms_mean: 1.37167823315
	test_y_col_norms_min: 1.13854420185
	test_y_max_max_class: 0.999999046326
	test_y_mean_max_class: 0.9629342556
	test_y_min_max_class: 0.519809484482
	test_y_misclass: 0.0402000173926
	test_y_nll: 0.127731248736
	test_y_row_norms_max: 0.32344275713
	test_y_row_norms_mean: 0.125407382846
	test_y_row_norms_min: 0.000886962865479
	train_h0_col_norms_max: 6.23615169525
	train_h0_col_norms_mean: 3.82577753067
	train_h0_col_norms_min: 2.06954622269
	train_h0_row_norms_max: 5.89259195328
	train_h0_row_norms_mean: 2.98751401901
	train_h0_row_norms_min: 0.0110142948106
	train_h1_col_norms_max: 5.99837732315
	train_h1_col_norms_mean: 3.80804681778
	train_h1_col_norms_min: 1.71573352814
	train_h1_row_norms_max: 7.80887413025
	train_h1_row_norms_mean: 5.4093914032
	train_h1_row_norms_min: 2.97880387306
	train_objective: 0.0784979835153
	train_y_col_norms_max: 1.54537415504
	train_y_col_norms_mean: 1.37168061733
	train_y_col_norms_min: 1.13854324818
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.965206980705
	train_y_min_max_class: 0.486533343792
	train_y_misclass: 0.0245000198483
	train_y_nll: 0.0784979835153
	train_y_row_norms_max: 0.323444247246
	train_y_row_norms_mean: 0.125407934189
	train_y_row_norms_min: 0.000886966707185
	valid_h0_col_norms_max: 6.23617553711
	valid_h0_col_norms_mean: 3.82576131821
	valid_h0_col_norms_min: 2.06955361366
	valid_h0_row_norms_max: 5.8926115036
	valid_h0_row_norms_mean: 2.98752951622
	valid_h0_row_norms_min: 0.011014319025
	valid_h1_col_norms_max: 5.99838781357
	valid_h1_col_norms_mean: 3.8080675602
	valid_h1_col_norms_min: 1.71574032307
	valid_h1_row_norms_max: 7.80883789062
	valid_h1_row_norms_mean: 5.40936756134
	valid_h1_row_norms_min: 2.97880935669
	valid_objective: 0.126347467303
	valid_y_col_norms_max: 1.54538154602
	valid_y_col_norms_mean: 1.37167823315
	valid_y_col_norms_min: 1.13854420185
	valid_y_max_max_class: 0.999999165535
	valid_y_mean_max_class: 0.966301620007
	valid_y_min_max_class: 0.483229219913
	valid_y_misclass: 0.0362999886274
	valid_y_nll: 0.126347467303
	valid_y_row_norms_max: 0.32344275713
	valid_y_row_norms_mean: 0.125407382846
	valid_y_row_norms_min: 0.000886962865479
Time this epoch: 3.324166 seconds
Monitoring step:
	Epochs seen: 8
	Batches seen: 4000
	Examples seen: 400000
	learning_rate: 0.00999999046326
	momentum: 0.881111502647
	test_h0_col_norms_max: 6.23693847656
	test_h0_col_norms_mean: 3.8264799118
	test_h0_col_norms_min: 2.07238268852
	test_h0_row_norms_max: 5.89200305939
	test_h0_row_norms_mean: 2.98819732666
	test_h0_row_norms_min: 0.0122548062354
	test_h1_col_norms_max: 5.99879837036
	test_h1_col_norms_mean: 3.80823135376
	test_h1_col_norms_min: 1.71583795547
	test_h1_row_norms_max: 7.80892133713
	test_h1_row_norms_mean: 5.40960502625
	test_h1_row_norms_min: 2.97916102409
	test_objective: 0.121290750802
	test_y_col_norms_max: 1.74212527275
	test_y_col_norms_mean: 1.55456089973
	test_y_col_norms_min: 1.29530310631
	test_y_max_max_class: 0.999999284744
	test_y_mean_max_class: 0.970344901085
	test_y_min_max_class: 0.541184604168
	test_y_misclass: 0.0355000011623
	test_y_nll: 0.121290750802
	test_y_row_norms_max: 0.393140137196
	test_y_row_norms_mean: 0.142595127225
	test_y_row_norms_min: 0.00119761796668
	train_h0_col_norms_max: 6.23696804047
	train_h0_col_norms_mean: 3.82649302483
	train_h0_col_norms_min: 2.07238888741
	train_h0_row_norms_max: 5.89202260971
	train_h0_row_norms_mean: 2.98821163177
	train_h0_row_norms_min: 0.0122548071668
	train_h1_col_norms_max: 5.99882984161
	train_h1_col_norms_mean: 3.80823636055
	train_h1_col_norms_min: 1.71583855152
	train_h1_row_norms_max: 7.80892324448
	train_h1_row_norms_mean: 5.40962982178
	train_h1_row_norms_min: 2.979159832
	train_objective: 0.0608208738267
	train_y_col_norms_max: 1.7421246767
	train_y_col_norms_mean: 1.55455350876
	train_y_col_norms_min: 1.29530549049
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.97307318449
	train_y_min_max_class: 0.52649885416
	train_y_misclass: 0.018220026046
	train_y_nll: 0.0608208738267
	train_y_row_norms_max: 0.393138289452
	train_y_row_norms_mean: 0.142595857382
	train_y_row_norms_min: 0.00119762122631
	valid_h0_col_norms_max: 6.23693847656
	valid_h0_col_norms_mean: 3.8264799118
	valid_h0_col_norms_min: 2.07238268852
	valid_h0_row_norms_max: 5.89200305939
	valid_h0_row_norms_mean: 2.98819732666
	valid_h0_row_norms_min: 0.0122548062354
	valid_h1_col_norms_max: 5.99879837036
	valid_h1_col_norms_mean: 3.80823135376
	valid_h1_col_norms_min: 1.71583795547
	valid_h1_row_norms_max: 7.80892133713
	valid_h1_row_norms_mean: 5.40960502625
	valid_h1_row_norms_min: 2.97916102409
	valid_objective: 0.120653524995
	valid_y_col_norms_max: 1.74212527275
	valid_y_col_norms_mean: 1.55456089973
	valid_y_col_norms_min: 1.29530310631
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.971736133099
	valid_y_min_max_class: 0.502751410007
	valid_y_misclass: 0.0357999950647
	valid_y_nll: 0.120653524995
	valid_y_row_norms_max: 0.393140137196
	valid_y_row_norms_mean: 0.142595127225
	valid_y_row_norms_min: 0.00119761796668
Time this epoch: 3.219467 seconds
Monitoring step:
	Epochs seen: 9
	Batches seen: 4500
	Examples seen: 450000
	learning_rate: 0.00999999046326
	momentum: 0.935554862022
	test_h0_col_norms_max: 6.23974847794
	test_h0_col_norms_mean: 3.82828760147
	test_h0_col_norms_min: 2.07858109474
	test_h0_row_norms_max: 5.89074993134
	test_h0_row_norms_mean: 2.98990464211
	test_h0_row_norms_min: 0.0139329638332
	test_h1_col_norms_max: 6.00128126144
	test_h1_col_norms_mean: 3.80823659897
	test_h1_col_norms_min: 1.71664977074
	test_h1_row_norms_max: 7.80959177017
	test_h1_row_norms_mean: 5.40965270996
	test_h1_row_norms_min: 2.98309516907
	test_objective: 0.133454963565
	test_y_col_norms_max: 2.09113478661
	test_y_col_norms_mean: 1.89531803131
	test_y_col_norms_min: 1.55502259731
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.972993254662
	test_y_min_max_class: 0.555838704109
	test_y_misclass: 0.03900000453
	test_y_nll: 0.133454963565
	test_y_row_norms_max: 0.505987465382
	test_y_row_norms_mean: 0.174324646592
	test_y_row_norms_min: 0.00215850048698
	train_h0_col_norms_max: 6.23972511292
	train_h0_col_norms_mean: 3.82828736305
	train_h0_col_norms_min: 2.07858753204
	train_h0_row_norms_max: 5.89076900482
	train_h0_row_norms_mean: 2.98989081383
	train_h0_row_norms_min: 0.0139330253005
	train_h1_col_norms_max: 6.00125265121
	train_h1_col_norms_mean: 3.80825352669
	train_h1_col_norms_min: 1.7166570425
	train_h1_row_norms_max: 7.80962467194
	train_h1_row_norms_mean: 5.40965032578
	train_h1_row_norms_min: 2.98309373856
	train_objective: 0.0678227543831
	train_y_col_norms_max: 2.09112644196
	train_y_col_norms_mean: 1.8953114748
	train_y_col_norms_min: 1.55502521992
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.976901352406
	train_y_min_max_class: 0.541133284569
	train_y_misclass: 0.0215600207448
	train_y_nll: 0.0678227543831
	train_y_row_norms_max: 0.505986630917
	train_y_row_norms_mean: 0.174323886633
	train_y_row_norms_min: 0.00215849909
	valid_h0_col_norms_max: 6.23974847794
	valid_h0_col_norms_mean: 3.82828760147
	valid_h0_col_norms_min: 2.07858109474
	valid_h0_row_norms_max: 5.89074993134
	valid_h0_row_norms_mean: 2.98990464211
	valid_h0_row_norms_min: 0.0139329638332
	valid_h1_col_norms_max: 6.00128126144
	valid_h1_col_norms_mean: 3.80823659897
	valid_h1_col_norms_min: 1.71664977074
	valid_h1_row_norms_max: 7.80959177017
	valid_h1_row_norms_mean: 5.40965270996
	valid_h1_row_norms_min: 2.98309516907
	valid_objective: 0.14155356586
	valid_y_col_norms_max: 2.09113478661
	valid_y_col_norms_mean: 1.89531803131
	valid_y_col_norms_min: 1.55502259731
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.975651443005
	valid_y_min_max_class: 0.524011075497
	valid_y_misclass: 0.0348999835551
	valid_y_nll: 0.14155356586
	valid_y_row_norms_max: 0.505987465382
	valid_y_row_norms_mean: 0.174324646592
	valid_y_row_norms_min: 0.00215850048698
Time this epoch: 3.242812 seconds
Monitoring step:
	Epochs seen: 10
	Batches seen: 5000
	Examples seen: 500000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.33813095093
	test_h0_col_norms_mean: 4.00221395493
	test_h0_col_norms_min: 2.23122644424
	test_h0_row_norms_max: 6.13888168335
	test_h0_row_norms_mean: 3.13162064552
	test_h0_row_norms_min: 0.0540144480765
	test_h1_col_norms_max: 5.99460268021
	test_h1_col_norms_mean: 3.81764769554
	test_h1_col_norms_min: 1.72675585747
	test_h1_row_norms_max: 7.80806827545
	test_h1_row_norms_mean: 5.42556667328
	test_h1_row_norms_min: 3.22008705139
	test_objective: 0.242982923985
	test_y_col_norms_max: 4.86701011658
	test_y_col_norms_mean: 4.50406503677
	test_y_col_norms_min: 3.79116678238
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.970284223557
	test_y_min_max_class: 0.494895517826
	test_y_misclass: 0.0614000074565
	test_y_nll: 0.242982923985
	test_y_row_norms_max: 1.25091540813
	test_y_row_norms_mean: 0.422105878592
	test_y_row_norms_min: 0.00902531389147
	train_h0_col_norms_max: 6.33812093735
	train_h0_col_norms_mean: 4.00221395493
	train_h0_col_norms_min: 2.23123693466
	train_h0_row_norms_max: 6.13886117935
	train_h0_row_norms_mean: 3.13162612915
	train_h0_row_norms_min: 0.0540147125721
	train_h1_col_norms_max: 5.99457454681
	train_h1_col_norms_mean: 3.81765389442
	train_h1_col_norms_min: 1.726749897
	train_h1_row_norms_max: 7.80803012848
	train_h1_row_norms_mean: 5.42554092407
	train_h1_row_norms_min: 3.2200820446
	train_objective: 0.216101527214
	train_y_col_norms_max: 4.86700248718
	train_y_col_norms_mean: 4.50406646729
	train_y_col_norms_min: 3.7911875248
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.971834897995
	train_y_min_max_class: 0.494699120522
	train_y_misclass: 0.0546000786126
	train_y_nll: 0.216101527214
	train_y_row_norms_max: 1.25092113018
	train_y_row_norms_mean: 0.422105282545
	train_y_row_norms_min: 0.00902529340237
	valid_h0_col_norms_max: 6.33813095093
	valid_h0_col_norms_mean: 4.00221395493
	valid_h0_col_norms_min: 2.23122644424
	valid_h0_row_norms_max: 6.13888168335
	valid_h0_row_norms_mean: 3.13162064552
	valid_h0_row_norms_min: 0.0540144480765
	valid_h1_col_norms_max: 5.99460268021
	valid_h1_col_norms_mean: 3.81764769554
	valid_h1_col_norms_min: 1.72675585747
	valid_h1_row_norms_max: 7.80806827545
	valid_h1_row_norms_mean: 5.42556667328
	valid_h1_row_norms_min: 3.22008705139
	valid_objective: 0.262977838516
	valid_y_col_norms_max: 4.86701011658
	valid_y_col_norms_mean: 4.50406503677
	valid_y_col_norms_min: 3.79116678238
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.972873926163
	valid_y_min_max_class: 0.484322339296
	valid_y_misclass: 0.0602999925613
	valid_y_nll: 0.262977838516
	valid_y_row_norms_max: 1.25091540813
	valid_y_row_norms_mean: 0.422105878592
	valid_y_row_norms_min: 0.00902531389147
Time this epoch: 3.246498 seconds
Monitoring step:
	Epochs seen: 11
	Batches seen: 5500
	Examples seen: 550000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.34423732758
	test_h0_col_norms_mean: 4.09757995605
	test_h0_col_norms_min: 2.23610663414
	test_h0_row_norms_max: 6.29168701172
	test_h0_row_norms_mean: 3.20701622963
	test_h0_row_norms_min: 0.0794842615724
	test_h1_col_norms_max: 5.99344968796
	test_h1_col_norms_mean: 3.83266830444
	test_h1_col_norms_min: 1.72617077827
	test_h1_row_norms_max: 7.81531667709
	test_h1_row_norms_mean: 5.44732666016
	test_h1_row_norms_min: 3.22785973549
	test_objective: 0.149660229683
	test_y_col_norms_max: 5.28322935104
	test_y_col_norms_mean: 4.86907577515
	test_y_col_norms_min: 4.24763870239
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.971002280712
	test_y_min_max_class: 0.500847101212
	test_y_misclass: 0.0448000095785
	test_y_nll: 0.149660229683
	test_y_row_norms_max: 1.53015840054
	test_y_row_norms_mean: 0.458264380693
	test_y_row_norms_min: 0.0079955086112
	train_h0_col_norms_max: 6.34425830841
	train_h0_col_norms_mean: 4.09757804871
	train_h0_col_norms_min: 2.23611760139
	train_h0_row_norms_max: 6.29168462753
	train_h0_row_norms_mean: 3.20700359344
	train_h0_row_norms_min: 0.0794841647148
	train_h1_col_norms_max: 5.99343013763
	train_h1_col_norms_mean: 3.83267450333
	train_h1_col_norms_min: 1.72616374493
	train_h1_row_norms_max: 7.81534910202
	train_h1_row_norms_mean: 5.44732189178
	train_h1_row_norms_min: 3.22785782814
	train_objective: 0.115495532751
	train_y_col_norms_max: 5.2832069397
	train_y_col_norms_mean: 4.86906385422
	train_y_col_norms_min: 4.24765825272
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.974365890026
	train_y_min_max_class: 0.503006339073
	train_y_misclass: 0.0362600125372
	train_y_nll: 0.115495532751
	train_y_row_norms_max: 1.53016579151
	train_y_row_norms_mean: 0.458266496658
	train_y_row_norms_min: 0.00799546111375
	valid_h0_col_norms_max: 6.34423732758
	valid_h0_col_norms_mean: 4.09757995605
	valid_h0_col_norms_min: 2.23610663414
	valid_h0_row_norms_max: 6.29168701172
	valid_h0_row_norms_mean: 3.20701622963
	valid_h0_row_norms_min: 0.0794842615724
	valid_h1_col_norms_max: 5.99344968796
	valid_h1_col_norms_mean: 3.83266830444
	valid_h1_col_norms_min: 1.72617077827
	valid_h1_row_norms_max: 7.81531667709
	valid_h1_row_norms_mean: 5.44732666016
	valid_h1_row_norms_min: 3.22785973549
	valid_objective: 0.1691185534
	valid_y_col_norms_max: 5.28322935104
	valid_y_col_norms_mean: 4.86907577515
	valid_y_col_norms_min: 4.24763870239
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.974966526031
	valid_y_min_max_class: 0.529185950756
	valid_y_misclass: 0.0438999943435
	valid_y_nll: 0.1691185534
	valid_y_row_norms_max: 1.53015840054
	valid_y_row_norms_mean: 0.458264380693
	valid_y_row_norms_min: 0.0079955086112
Time this epoch: 3.224365 seconds
Monitoring step:
	Epochs seen: 12
	Batches seen: 6000
	Examples seen: 600000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.34843397141
	test_h0_col_norms_mean: 4.13394451141
	test_h0_col_norms_min: 2.23612523079
	test_h0_row_norms_max: 6.36067008972
	test_h0_row_norms_mean: 3.23545217514
	test_h0_row_norms_min: 0.111102260649
	test_h1_col_norms_max: 5.99360513687
	test_h1_col_norms_mean: 3.8399875164
	test_h1_col_norms_min: 1.72649633884
	test_h1_row_norms_max: 7.9447259903
	test_h1_row_norms_mean: 5.45721006393
	test_h1_row_norms_min: 3.23267006874
	test_objective: 0.138930052519
	test_y_col_norms_max: 5.38853263855
	test_y_col_norms_mean: 4.97749423981
	test_y_col_norms_min: 4.37515115738
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.976369380951
	test_y_min_max_class: 0.539442539215
	test_y_misclass: 0.0395999997854
	test_y_nll: 0.138930052519
	test_y_row_norms_max: 1.5151270628
	test_y_row_norms_mean: 0.468785196543
	test_y_row_norms_min: 0.00989222805947
	train_h0_col_norms_max: 6.34842920303
	train_h0_col_norms_mean: 4.13394021988
	train_h0_col_norms_min: 2.23612689972
	train_h0_row_norms_max: 6.36069536209
	train_h0_row_norms_mean: 3.23545718193
	train_h0_row_norms_min: 0.111102797091
	train_h1_col_norms_max: 5.99360513687
	train_h1_col_norms_mean: 3.83997154236
	train_h1_col_norms_min: 1.72650408745
	train_h1_row_norms_max: 7.94476556778
	train_h1_row_norms_mean: 5.45718336105
	train_h1_row_norms_min: 3.23268294334
	train_objective: 0.0762413665652
	train_y_col_norms_max: 5.38851499557
	train_y_col_norms_mean: 4.97747087479
	train_y_col_norms_min: 4.37513685226
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.980703771114
	train_y_min_max_class: 0.538486421108
	train_y_misclass: 0.0236600115895
	train_y_nll: 0.0762413665652
	train_y_row_norms_max: 1.51513409615
	train_y_row_norms_mean: 0.46878734231
	train_y_row_norms_min: 0.00989221502095
	valid_h0_col_norms_max: 6.34843397141
	valid_h0_col_norms_mean: 4.13394451141
	valid_h0_col_norms_min: 2.23612523079
	valid_h0_row_norms_max: 6.36067008972
	valid_h0_row_norms_mean: 3.23545217514
	valid_h0_row_norms_min: 0.111102260649
	valid_h1_col_norms_max: 5.99360513687
	valid_h1_col_norms_mean: 3.8399875164
	valid_h1_col_norms_min: 1.72649633884
	valid_h1_row_norms_max: 7.9447259903
	valid_h1_row_norms_mean: 5.45721006393
	valid_h1_row_norms_min: 3.23267006874
	valid_objective: 0.158047273755
	valid_y_col_norms_max: 5.38853263855
	valid_y_col_norms_mean: 4.97749423981
	valid_y_col_norms_min: 4.37515115738
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.978621006012
	valid_y_min_max_class: 0.533575236797
	valid_y_misclass: 0.0357999950647
	valid_y_nll: 0.158047273755
	valid_y_row_norms_max: 1.5151270628
	valid_y_row_norms_mean: 0.468785196543
	valid_y_row_norms_min: 0.00989222805947
Time this epoch: 3.233253 seconds
Monitoring step:
	Epochs seen: 13
	Batches seen: 6500
	Examples seen: 650000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.34697389603
	test_h0_col_norms_mean: 4.15769052505
	test_h0_col_norms_min: 2.23618888855
	test_h0_row_norms_max: 6.40273475647
	test_h0_row_norms_mean: 3.25405025482
	test_h0_row_norms_min: 0.113349400461
	test_h1_col_norms_max: 5.99226903915
	test_h1_col_norms_mean: 3.84424233437
	test_h1_col_norms_min: 1.7265651226
	test_h1_row_norms_max: 8.25644397736
	test_h1_row_norms_mean: 5.46302652359
	test_h1_row_norms_min: 3.24811220169
	test_objective: 0.126156955957
	test_y_col_norms_max: 5.49813652039
	test_y_col_norms_mean: 5.06592178345
	test_y_col_norms_min: 4.50360441208
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.983133792877
	test_y_min_max_class: 0.586351394653
	test_y_misclass: 0.0298999845982
	test_y_nll: 0.126156955957
	test_y_row_norms_max: 1.57926058769
	test_y_row_norms_mean: 0.477343022823
	test_y_row_norms_min: 0.0155787682161
	train_h0_col_norms_max: 6.34694576263
	train_h0_col_norms_mean: 4.15768814087
	train_h0_col_norms_min: 2.23619389534
	train_h0_row_norms_max: 6.40273189545
	train_h0_row_norms_mean: 3.25405526161
	train_h0_row_norms_min: 0.113349400461
	train_h1_col_norms_max: 5.99224901199
	train_h1_col_norms_mean: 3.84425520897
	train_h1_col_norms_min: 1.72656738758
	train_h1_row_norms_max: 8.25643634796
	train_h1_row_norms_mean: 5.46304035187
	train_h1_row_norms_min: 3.24809789658
	train_objective: 0.0474301576614
	train_y_col_norms_max: 5.49815416336
	train_y_col_norms_mean: 5.06591844559
	train_y_col_norms_min: 4.5035943985
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.986293017864
	train_y_min_max_class: 0.579336941242
	train_y_misclass: 0.0156600344926
	train_y_nll: 0.0474301576614
	train_y_row_norms_max: 1.57926678658
	train_y_row_norms_mean: 0.477343559265
	train_y_row_norms_min: 0.0155787058175
	valid_h0_col_norms_max: 6.34697389603
	valid_h0_col_norms_mean: 4.15769052505
	valid_h0_col_norms_min: 2.23618888855
	valid_h0_row_norms_max: 6.40273475647
	valid_h0_row_norms_mean: 3.25405025482
	valid_h0_row_norms_min: 0.113349400461
	valid_h1_col_norms_max: 5.99226903915
	valid_h1_col_norms_mean: 3.84424233437
	valid_h1_col_norms_min: 1.7265651226
	valid_h1_row_norms_max: 8.25644397736
	valid_h1_row_norms_mean: 5.46302652359
	valid_h1_row_norms_min: 3.24811220169
	valid_objective: 0.136303275824
	valid_y_col_norms_max: 5.49813652039
	valid_y_col_norms_mean: 5.06592178345
	valid_y_col_norms_min: 4.50360441208
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.983997404575
	valid_y_min_max_class: 0.568609714508
	valid_y_misclass: 0.0302999857813
	valid_y_nll: 0.136303275824
	valid_y_row_norms_max: 1.57926058769
	valid_y_row_norms_mean: 0.477343022823
	valid_y_row_norms_min: 0.0155787682161
Time this epoch: 3.243910 seconds
Monitoring step:
	Epochs seen: 14
	Batches seen: 7000
	Examples seen: 700000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.3465590477
	test_h0_col_norms_mean: 4.17470979691
	test_h0_col_norms_min: 2.23621320724
	test_h0_row_norms_max: 6.44742536545
	test_h0_row_norms_mean: 3.26748609543
	test_h0_row_norms_min: 0.117137983441
	test_h1_col_norms_max: 5.99374818802
	test_h1_col_norms_mean: 3.84760499001
	test_h1_col_norms_min: 1.7263559103
	test_h1_row_norms_max: 8.39470767975
	test_h1_row_norms_mean: 5.46778011322
	test_h1_row_norms_min: 3.26342630386
	test_objective: 0.107709117234
	test_y_col_norms_max: 5.54377269745
	test_y_col_norms_mean: 5.13376808167
	test_y_col_norms_min: 4.58503246307
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.985660552979
	test_y_min_max_class: 0.598270595074
	test_y_misclass: 0.0245999917388
	test_y_nll: 0.107709117234
	test_y_row_norms_max: 1.54413878918
	test_y_row_norms_mean: 0.484508126974
	test_y_row_norms_min: 0.0144754517823
	train_h0_col_norms_max: 6.346534729
	train_h0_col_norms_mean: 4.17470979691
	train_h0_col_norms_min: 2.23620676994
	train_h0_row_norms_max: 6.44738912582
	train_h0_row_norms_mean: 3.26749873161
	train_h0_row_norms_min: 0.117138013244
	train_h1_col_norms_max: 5.99376821518
	train_h1_col_norms_mean: 3.84760093689
	train_h1_col_norms_min: 1.72634637356
	train_h1_row_norms_max: 8.39471530914
	train_h1_row_norms_mean: 5.46780490875
	train_h1_row_norms_min: 3.26344394684
	train_objective: 0.0289139077067
	train_y_col_norms_max: 5.54377365112
	train_y_col_norms_mean: 5.13377904892
	train_y_col_norms_min: 4.58502912521
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.990230798721
	train_y_min_max_class: 0.636234402657
	train_y_misclass: 0.0095800133422
	train_y_nll: 0.0289139077067
	train_y_row_norms_max: 1.54413354397
	train_y_row_norms_mean: 0.484510302544
	train_y_row_norms_min: 0.0144755160436
	valid_h0_col_norms_max: 6.3465590477
	valid_h0_col_norms_mean: 4.17470979691
	valid_h0_col_norms_min: 2.23621320724
	valid_h0_row_norms_max: 6.44742536545
	valid_h0_row_norms_mean: 3.26748609543
	valid_h0_row_norms_min: 0.117137983441
	valid_h1_col_norms_max: 5.99374818802
	valid_h1_col_norms_mean: 3.84760499001
	valid_h1_col_norms_min: 1.7263559103
	valid_h1_row_norms_max: 8.39470767975
	valid_h1_row_norms_mean: 5.46778011322
	valid_h1_row_norms_min: 3.26342630386
	valid_objective: 0.118425898254
	valid_y_col_norms_max: 5.54377269745
	valid_y_col_norms_mean: 5.13376808167
	valid_y_col_norms_min: 4.58503246307
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.987525939941
	valid_y_min_max_class: 0.608628451824
	valid_y_misclass: 0.0258999839425
	valid_y_nll: 0.118425898254
	valid_y_row_norms_max: 1.54413878918
	valid_y_row_norms_mean: 0.484508126974
	valid_y_row_norms_min: 0.0144754517823
Time this epoch: 3.231089 seconds
Monitoring step:
	Epochs seen: 15
	Batches seen: 7500
	Examples seen: 750000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.34646034241
	test_h0_col_norms_mean: 4.18901968002
	test_h0_col_norms_min: 2.23616552353
	test_h0_row_norms_max: 6.49371194839
	test_h0_row_norms_mean: 3.27877855301
	test_h0_row_norms_min: 0.122728899121
	test_h1_col_norms_max: 5.9948592186
	test_h1_col_norms_mean: 3.85039448738
	test_h1_col_norms_min: 1.72630560398
	test_h1_row_norms_max: 8.49246692657
	test_h1_row_norms_mean: 5.47177028656
	test_h1_row_norms_min: 3.27335119247
	test_objective: 0.120883144438
	test_y_col_norms_max: 5.61785268784
	test_y_col_norms_mean: 5.21456623077
	test_y_col_norms_min: 4.61228704453
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.985354423523
	test_y_min_max_class: 0.593527436256
	test_y_misclass: 0.0276999864727
	test_y_nll: 0.120883144438
	test_y_row_norms_max: 1.59560739994
	test_y_row_norms_mean: 0.492057174444
	test_y_row_norms_min: 0.0153611358255
	train_h0_col_norms_max: 6.34646320343
	train_h0_col_norms_mean: 4.18901586533
	train_h0_col_norms_min: 2.23617053032
	train_h0_row_norms_max: 6.49373817444
	train_h0_row_norms_mean: 3.27876186371
	train_h0_row_norms_min: 0.122729450464
	train_h1_col_norms_max: 5.99485731125
	train_h1_col_norms_mean: 3.8503715992
	train_h1_col_norms_min: 1.72631311417
	train_h1_row_norms_max: 8.49246883392
	train_h1_row_norms_mean: 5.47177219391
	train_h1_row_norms_min: 3.27336573601
	train_objective: 0.0283282585442
	train_y_col_norms_max: 5.61785554886
	train_y_col_norms_mean: 5.21454381943
	train_y_col_norms_min: 4.61229038239
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.990546524525
	train_y_min_max_class: 0.649133205414
	train_y_misclass: 0.00910001061857
	train_y_nll: 0.0283282585442
	train_y_row_norms_max: 1.59561276436
	train_y_row_norms_mean: 0.492054820061
	train_y_row_norms_min: 0.0153612047434
	valid_h0_col_norms_max: 6.34646034241
	valid_h0_col_norms_mean: 4.18901968002
	valid_h0_col_norms_min: 2.23616552353
	valid_h0_row_norms_max: 6.49371194839
	valid_h0_row_norms_mean: 3.27877855301
	valid_h0_row_norms_min: 0.122728899121
	valid_h1_col_norms_max: 5.9948592186
	valid_h1_col_norms_mean: 3.85039448738
	valid_h1_col_norms_min: 1.72630560398
	valid_h1_row_norms_max: 8.49246692657
	valid_h1_row_norms_mean: 5.47177028656
	valid_h1_row_norms_min: 3.27335119247
	valid_objective: 0.126225486398
	valid_y_col_norms_max: 5.61785268784
	valid_y_col_norms_mean: 5.21456623077
	valid_y_col_norms_min: 4.61228704453
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.987080872059
	valid_y_min_max_class: 0.583371043205
	valid_y_misclass: 0.0265999827534
	valid_y_nll: 0.126225486398
	valid_y_row_norms_max: 1.59560739994
	valid_y_row_norms_mean: 0.492057174444
	valid_y_row_norms_min: 0.0153611358255
Time this epoch: 3.249726 seconds
Monitoring step:
	Epochs seen: 16
	Batches seen: 8000
	Examples seen: 800000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.34652328491
	test_h0_col_norms_mean: 4.20311164856
	test_h0_col_norms_min: 2.23617291451
	test_h0_row_norms_max: 6.52294111252
	test_h0_row_norms_mean: 3.28994369507
	test_h0_row_norms_min: 0.123597666621
	test_h1_col_norms_max: 5.99608755112
	test_h1_col_norms_mean: 3.85341596603
	test_h1_col_norms_min: 1.72634136677
	test_h1_row_norms_max: 8.52042388916
	test_h1_row_norms_mean: 5.47620201111
	test_h1_row_norms_min: 3.27072739601
	test_objective: 0.140668272972
	test_y_col_norms_max: 5.69501256943
	test_y_col_norms_mean: 5.31268548965
	test_y_col_norms_min: 4.74868249893
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.989543557167
	test_y_min_max_class: 0.626976370811
	test_y_misclass: 0.0258999932557
	test_y_nll: 0.140668272972
	test_y_row_norms_max: 1.60322284698
	test_y_row_norms_mean: 0.500980615616
	test_y_row_norms_min: 0.0168750006706
	train_h0_col_norms_max: 6.34652090073
	train_h0_col_norms_mean: 4.20310306549
	train_h0_col_norms_min: 2.23617100716
	train_h0_row_norms_max: 6.52291107178
	train_h0_row_norms_mean: 3.28993988037
	train_h0_row_norms_min: 0.123597674072
	train_h1_col_norms_max: 5.99606466293
	train_h1_col_norms_mean: 3.85343289375
	train_h1_col_norms_min: 1.72633349895
	train_h1_row_norms_max: 8.52042198181
	train_h1_row_norms_mean: 5.47621965408
	train_h1_row_norms_min: 3.27073836327
	train_objective: 0.0259083565325
	train_y_col_norms_max: 5.69503641129
	train_y_col_norms_mean: 5.31268262863
	train_y_col_norms_min: 4.74867868423
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.993756473064
	train_y_min_max_class: 0.699871182442
	train_y_misclass: 0.0079400036484
	train_y_nll: 0.0259083565325
	train_y_row_norms_max: 1.6032307148
	train_y_row_norms_mean: 0.500979840755
	train_y_row_norms_min: 0.0168750379235
	valid_h0_col_norms_max: 6.34652328491
	valid_h0_col_norms_mean: 4.20311164856
	valid_h0_col_norms_min: 2.23617291451
	valid_h0_row_norms_max: 6.52294111252
	valid_h0_row_norms_mean: 3.28994369507
	valid_h0_row_norms_min: 0.123597666621
	valid_h1_col_norms_max: 5.99608755112
	valid_h1_col_norms_mean: 3.85341596603
	valid_h1_col_norms_min: 1.72634136677
	valid_h1_row_norms_max: 8.52042388916
	valid_h1_row_norms_mean: 5.47620201111
	valid_h1_row_norms_min: 3.27072739601
	valid_objective: 0.140435069799
	valid_y_col_norms_max: 5.69501256943
	valid_y_col_norms_mean: 5.31268548965
	valid_y_col_norms_min: 4.74868249893
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.990495383739
	valid_y_min_max_class: 0.633842229843
	valid_y_misclass: 0.0265999827534
	valid_y_nll: 0.140435069799
	valid_y_row_norms_max: 1.60322284698
	valid_y_row_norms_mean: 0.500980615616
	valid_y_row_norms_min: 0.0168750006706
Time this epoch: 3.211907 seconds
Monitoring step:
	Epochs seen: 17
	Batches seen: 8500
	Examples seen: 850000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.34764194489
	test_h0_col_norms_mean: 4.21877479553
	test_h0_col_norms_min: 2.23619961739
	test_h0_row_norms_max: 6.5714468956
	test_h0_row_norms_mean: 3.30228757858
	test_h0_row_norms_min: 0.13643656671
	test_h1_col_norms_max: 5.99594020844
	test_h1_col_norms_mean: 3.85699319839
	test_h1_col_norms_min: 1.72638630867
	test_h1_row_norms_max: 8.61135101318
	test_h1_row_norms_mean: 5.48117828369
	test_h1_row_norms_min: 3.27077460289
	test_objective: 0.152983635664
	test_y_col_norms_max: 5.81860494614
	test_y_col_norms_mean: 5.40938711166
	test_y_col_norms_min: 4.81085681915
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.990412473679
	test_y_min_max_class: 0.641472399235
	test_y_misclass: 0.0277999881655
	test_y_nll: 0.152983635664
	test_y_row_norms_max: 1.66027259827
	test_y_row_norms_mean: 0.509944438934
	test_y_row_norms_min: 0.0174780637026
	train_h0_col_norms_max: 6.3476524353
	train_h0_col_norms_mean: 4.21879482269
	train_h0_col_norms_min: 2.23619699478
	train_h0_row_norms_max: 6.57147264481
	train_h0_row_norms_mean: 3.30230164528
	train_h0_row_norms_min: 0.136435881257
	train_h1_col_norms_max: 5.9959692955
	train_h1_col_norms_mean: 3.85701036453
	train_h1_col_norms_min: 1.72638809681
	train_h1_row_norms_max: 8.61137866974
	train_h1_row_norms_mean: 5.48117685318
	train_h1_row_norms_min: 3.27077269554
	train_objective: 0.0280419886112
	train_y_col_norms_max: 5.81860494614
	train_y_col_norms_mean: 5.40940761566
	train_y_col_norms_min: 4.81083345413
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.993593096733
	train_y_min_max_class: 0.69740664959
	train_y_misclass: 0.00842001195997
	train_y_nll: 0.0280419886112
	train_y_row_norms_max: 1.66027379036
	train_y_row_norms_mean: 0.509946644306
	train_y_row_norms_min: 0.0174781102687
	valid_h0_col_norms_max: 6.34764194489
	valid_h0_col_norms_mean: 4.21877479553
	valid_h0_col_norms_min: 2.23619961739
	valid_h0_row_norms_max: 6.5714468956
	valid_h0_row_norms_mean: 3.30228757858
	valid_h0_row_norms_min: 0.13643656671
	valid_h1_col_norms_max: 5.99594020844
	valid_h1_col_norms_mean: 3.85699319839
	valid_h1_col_norms_min: 1.72638630867
	valid_h1_row_norms_max: 8.61135101318
	valid_h1_row_norms_mean: 5.48117828369
	valid_h1_row_norms_min: 3.27077460289
	valid_objective: 0.156515717506
	valid_y_col_norms_max: 5.81860494614
	valid_y_col_norms_mean: 5.40938711166
	valid_y_col_norms_min: 4.81085681915
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.991046726704
	valid_y_min_max_class: 0.649928092957
	valid_y_misclass: 0.0286999810487
	valid_y_nll: 0.156515717506
	valid_y_row_norms_max: 1.66027259827
	valid_y_row_norms_mean: 0.509944438934
	valid_y_row_norms_min: 0.0174780637026
Time this epoch: 3.213883 seconds
Monitoring step:
	Epochs seen: 18
	Batches seen: 9000
	Examples seen: 900000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.34813308716
	test_h0_col_norms_mean: 4.2329621315
	test_h0_col_norms_min: 2.23619866371
	test_h0_row_norms_max: 6.60563611984
	test_h0_row_norms_mean: 3.31354284286
	test_h0_row_norms_min: 0.142215177417
	test_h1_col_norms_max: 5.99625921249
	test_h1_col_norms_mean: 3.86032938957
	test_h1_col_norms_min: 1.72629284859
	test_h1_row_norms_max: 8.70863246918
	test_h1_row_norms_mean: 5.48595952988
	test_h1_row_norms_min: 3.27107739449
	test_objective: 0.1266990453
	test_y_col_norms_max: 5.94182395935
	test_y_col_norms_mean: 5.48706197739
	test_y_col_norms_min: 4.85955810547
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.991642594337
	test_y_min_max_class: 0.680797755718
	test_y_misclass: 0.0225999932736
	test_y_nll: 0.1266990453
	test_y_row_norms_max: 1.65575671196
	test_y_row_norms_mean: 0.517508506775
	test_y_row_norms_min: 0.0219007991254
	train_h0_col_norms_max: 6.34813261032
	train_h0_col_norms_mean: 4.23297452927
	train_h0_col_norms_min: 2.23619627953
	train_h0_row_norms_max: 6.6056265831
	train_h0_row_norms_mean: 3.31354165077
	train_h0_row_norms_min: 0.142215907574
	train_h1_col_norms_max: 5.99623250961
	train_h1_col_norms_mean: 3.86034679413
	train_h1_col_norms_min: 1.72628378868
	train_h1_row_norms_max: 8.70865249634
	train_h1_row_norms_mean: 5.48598957062
	train_h1_row_norms_min: 3.27109384537
	train_objective: 0.0143134472892
	train_y_col_norms_max: 5.94185161591
	train_y_col_norms_mean: 5.48704767227
	train_y_col_norms_min: 4.85954427719
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996097743511
	train_y_min_max_class: 0.768372476101
	train_y_misclass: 0.00466000149027
	train_y_nll: 0.0143134472892
	train_y_row_norms_max: 1.6557571888
	train_y_row_norms_mean: 0.517506301403
	train_y_row_norms_min: 0.0219009146094
	valid_h0_col_norms_max: 6.34813308716
	valid_h0_col_norms_mean: 4.2329621315
	valid_h0_col_norms_min: 2.23619866371
	valid_h0_row_norms_max: 6.60563611984
	valid_h0_row_norms_mean: 3.31354284286
	valid_h0_row_norms_min: 0.142215177417
	valid_h1_col_norms_max: 5.99625921249
	valid_h1_col_norms_mean: 3.86032938957
	valid_h1_col_norms_min: 1.72629284859
	valid_h1_row_norms_max: 8.70863246918
	valid_h1_row_norms_mean: 5.48595952988
	valid_h1_row_norms_min: 3.27107739449
	valid_objective: 0.158007115126
	valid_y_col_norms_max: 5.94182395935
	valid_y_col_norms_mean: 5.48706197739
	valid_y_col_norms_min: 4.85955810547
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.99270170927
	valid_y_min_max_class: 0.685421526432
	valid_y_misclass: 0.0256999861449
	valid_y_nll: 0.158007115126
	valid_y_row_norms_max: 1.65575671196
	valid_y_row_norms_mean: 0.517508506775
	valid_y_row_norms_min: 0.0219007991254
Time this epoch: 3.216884 seconds
Monitoring step:
	Epochs seen: 19
	Batches seen: 9500
	Examples seen: 950000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.35075473785
	test_h0_col_norms_mean: 4.24279737473
	test_h0_col_norms_min: 2.23619127274
	test_h0_row_norms_max: 6.61569023132
	test_h0_row_norms_mean: 3.32117795944
	test_h0_row_norms_min: 0.160097524524
	test_h1_col_norms_max: 5.99536848068
	test_h1_col_norms_mean: 3.86252450943
	test_h1_col_norms_min: 1.72685301304
	test_h1_row_norms_max: 8.74706554413
	test_h1_row_norms_mean: 5.48911523819
	test_h1_row_norms_min: 3.27158546448
	test_objective: 0.128275766969
	test_y_col_norms_max: 6.00630426407
	test_y_col_norms_mean: 5.54901790619
	test_y_col_norms_min: 4.95159053802
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.992789447308
	test_y_min_max_class: 0.689596951008
	test_y_misclass: 0.0208999924362
	test_y_nll: 0.128275766969
	test_y_row_norms_max: 1.56810212135
	test_y_row_norms_mean: 0.523332059383
	test_y_row_norms_min: 0.0221748072654
	train_h0_col_norms_max: 6.35075521469
	train_h0_col_norms_mean: 4.24279022217
	train_h0_col_norms_min: 2.23619437218
	train_h0_row_norms_max: 6.61565685272
	train_h0_row_norms_mean: 3.32117271423
	train_h0_row_norms_min: 0.160097926855
	train_h1_col_norms_max: 5.99534845352
	train_h1_col_norms_mean: 3.86250782013
	train_h1_col_norms_min: 1.7268614769
	train_h1_row_norms_max: 8.74709033966
	train_h1_row_norms_mean: 5.48911237717
	train_h1_row_norms_min: 3.27158045769
	train_objective: 0.0107667120174
	train_y_col_norms_max: 6.00630140305
	train_y_col_norms_mean: 5.54901885986
	train_y_col_norms_min: 4.95157289505
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996949017048
	train_y_min_max_class: 0.813183248043
	train_y_misclass: 0.00347999692895
	train_y_nll: 0.0107667120174
	train_y_row_norms_max: 1.56809437275
	train_y_row_norms_mean: 0.523329675198
	train_y_row_norms_min: 0.0221747960895
	valid_h0_col_norms_max: 6.35075473785
	valid_h0_col_norms_mean: 4.24279737473
	valid_h0_col_norms_min: 2.23619127274
	valid_h0_row_norms_max: 6.61569023132
	valid_h0_row_norms_mean: 3.32117795944
	valid_h0_row_norms_min: 0.160097524524
	valid_h1_col_norms_max: 5.99536848068
	valid_h1_col_norms_mean: 3.86252450943
	valid_h1_col_norms_min: 1.72685301304
	valid_h1_row_norms_max: 8.74706554413
	valid_h1_row_norms_mean: 5.48911523819
	valid_h1_row_norms_min: 3.27158546448
	valid_objective: 0.152880609035
	valid_y_col_norms_max: 6.00630426407
	valid_y_col_norms_mean: 5.54901790619
	valid_y_col_norms_min: 4.95159053802
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.992956161499
	valid_y_min_max_class: 0.687586247921
	valid_y_misclass: 0.0238999892026
	valid_y_nll: 0.152880609035
	valid_y_row_norms_max: 1.56810212135
	valid_y_row_norms_mean: 0.523332059383
	valid_y_row_norms_min: 0.0221748072654
Time this epoch: 3.381361 seconds
Monitoring step:
	Epochs seen: 20
	Batches seen: 10000
	Examples seen: 1000000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.36764955521
	test_h0_col_norms_mean: 4.25145339966
	test_h0_col_norms_min: 2.23608016968
	test_h0_row_norms_max: 6.65068340302
	test_h0_row_norms_mean: 3.32794356346
	test_h0_row_norms_min: 0.160930916667
	test_h1_col_norms_max: 5.99686193466
	test_h1_col_norms_mean: 3.86456871033
	test_h1_col_norms_min: 1.72680532932
	test_h1_row_norms_max: 8.77167224884
	test_h1_row_norms_mean: 5.49206733704
	test_h1_row_norms_min: 3.27174091339
	test_objective: 0.135456323624
	test_y_col_norms_max: 6.06686162949
	test_y_col_norms_mean: 5.60846662521
	test_y_col_norms_min: 5.02197170258
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.992590069771
	test_y_min_max_class: 0.687869131565
	test_y_misclass: 0.0230999924242
	test_y_nll: 0.135456323624
	test_y_row_norms_max: 1.63880228996
	test_y_row_norms_mean: 0.528553962708
	test_y_row_norms_min: 0.0211624447256
	train_h0_col_norms_max: 6.36767864227
	train_h0_col_norms_mean: 4.25147294998
	train_h0_col_norms_min: 2.23607754707
	train_h0_row_norms_max: 6.65068531036
	train_h0_row_norms_mean: 3.32795858383
	train_h0_row_norms_min: 0.160931810737
	train_h1_col_norms_max: 5.9968791008
	train_h1_col_norms_mean: 3.86455130577
	train_h1_col_norms_min: 1.72680592537
	train_h1_row_norms_max: 8.77168178558
	train_h1_row_norms_mean: 5.49205350876
	train_h1_row_norms_min: 3.27172803879
	train_objective: 0.0139410560951
	train_y_col_norms_max: 6.06685829163
	train_y_col_norms_mean: 5.60847902298
	train_y_col_norms_min: 5.02197313309
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.997198402882
	train_y_min_max_class: 0.82025551796
	train_y_misclass: 0.00411999737844
	train_y_nll: 0.0139410560951
	train_y_row_norms_max: 1.63881158829
	train_y_row_norms_mean: 0.528555572033
	train_y_row_norms_min: 0.0211624447256
	valid_h0_col_norms_max: 6.36764955521
	valid_h0_col_norms_mean: 4.25145339966
	valid_h0_col_norms_min: 2.23608016968
	valid_h0_row_norms_max: 6.65068340302
	valid_h0_row_norms_mean: 3.32794356346
	valid_h0_row_norms_min: 0.160930916667
	valid_h1_col_norms_max: 5.99686193466
	valid_h1_col_norms_mean: 3.86456871033
	valid_h1_col_norms_min: 1.72680532932
	valid_h1_row_norms_max: 8.77167224884
	valid_h1_row_norms_mean: 5.49206733704
	valid_h1_row_norms_min: 3.27174091339
	valid_objective: 0.154028758407
	valid_y_col_norms_max: 6.06686162949
	valid_y_col_norms_mean: 5.60846662521
	valid_y_col_norms_min: 5.02197170258
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.993701696396
	valid_y_min_max_class: 0.705734312534
	valid_y_misclass: 0.0234999880195
	valid_y_nll: 0.154028758407
	valid_y_row_norms_max: 1.63880228996
	valid_y_row_norms_mean: 0.528553962708
	valid_y_row_norms_min: 0.0211624447256
Time this epoch: 3.224501 seconds
Monitoring step:
	Epochs seen: 21
	Batches seen: 10500
	Examples seen: 1050000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.36559724808
	test_h0_col_norms_mean: 4.25865936279
	test_h0_col_norms_min: 2.23606491089
	test_h0_row_norms_max: 6.65287876129
	test_h0_row_norms_mean: 3.33374094963
	test_h0_row_norms_min: 0.160923495889
	test_h1_col_norms_max: 5.9981341362
	test_h1_col_norms_mean: 3.866314888
	test_h1_col_norms_min: 1.72683930397
	test_h1_row_norms_max: 8.78785800934
	test_h1_row_norms_mean: 5.49455070496
	test_h1_row_norms_min: 3.27166962624
	test_objective: 0.132553175092
	test_y_col_norms_max: 6.10146903992
	test_y_col_norms_mean: 5.65123224258
	test_y_col_norms_min: 5.06749105453
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.992962539196
	test_y_min_max_class: 0.700105249882
	test_y_misclass: 0.0216999929398
	test_y_nll: 0.132553175092
	test_y_row_norms_max: 1.6686950922
	test_y_row_norms_mean: 0.532644450665
	test_y_row_norms_min: 0.0201863590628
	train_h0_col_norms_max: 6.36559391022
	train_h0_col_norms_mean: 4.25863981247
	train_h0_col_norms_min: 2.23606181145
	train_h0_row_norms_max: 6.6528468132
	train_h0_row_norms_mean: 3.33372306824
	train_h0_row_norms_min: 0.160924375057
	train_h1_col_norms_max: 5.9981341362
	train_h1_col_norms_mean: 3.86631464958
	train_h1_col_norms_min: 1.7268487215
	train_h1_row_norms_max: 8.78784656525
	train_h1_row_norms_mean: 5.494576931
	train_h1_row_norms_min: 3.27167153358
	train_objective: 0.00576127693057
	train_y_col_norms_max: 6.10144424438
	train_y_col_norms_mean: 5.65123510361
	train_y_col_norms_min: 5.06749773026
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998003363609
	train_y_min_max_class: 0.865698575974
	train_y_misclass: 0.00196000072174
	train_y_nll: 0.00576127693057
	train_y_row_norms_max: 1.66869258881
	train_y_row_norms_mean: 0.532643556595
	train_y_row_norms_min: 0.0201863981783
	valid_h0_col_norms_max: 6.36559724808
	valid_h0_col_norms_mean: 4.25865936279
	valid_h0_col_norms_min: 2.23606491089
	valid_h0_row_norms_max: 6.65287876129
	valid_h0_row_norms_mean: 3.33374094963
	valid_h0_row_norms_min: 0.160923495889
	valid_h1_col_norms_max: 5.9981341362
	valid_h1_col_norms_mean: 3.866314888
	valid_h1_col_norms_min: 1.72683930397
	valid_h1_row_norms_max: 8.78785800934
	valid_h1_row_norms_mean: 5.49455070496
	valid_h1_row_norms_min: 3.27166962624
	valid_objective: 0.149952054024
	valid_y_col_norms_max: 6.10146903992
	valid_y_col_norms_mean: 5.65123224258
	valid_y_col_norms_min: 5.06749105453
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.993861615658
	valid_y_min_max_class: 0.696114599705
	valid_y_misclass: 0.0218999926001
	valid_y_nll: 0.149952054024
	valid_y_row_norms_max: 1.6686950922
	valid_y_row_norms_mean: 0.532644450665
	valid_y_row_norms_min: 0.0201863590628
Time this epoch: 3.191485 seconds
Monitoring step:
	Epochs seen: 22
	Batches seen: 11000
	Examples seen: 1100000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.37540435791
	test_h0_col_norms_mean: 4.26554250717
	test_h0_col_norms_min: 2.23606491089
	test_h0_row_norms_max: 6.68969488144
	test_h0_row_norms_mean: 3.33926701546
	test_h0_row_norms_min: 0.15927760303
	test_h1_col_norms_max: 5.99918460846
	test_h1_col_norms_mean: 3.86793661118
	test_h1_col_norms_min: 1.72684121132
	test_h1_row_norms_max: 8.80519104004
	test_h1_row_norms_mean: 5.49690055847
	test_h1_row_norms_min: 3.27168059349
	test_objective: 0.129877910018
	test_y_col_norms_max: 6.1563615799
	test_y_col_norms_mean: 5.69634532928
	test_y_col_norms_min: 5.04322528839
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.993051469326
	test_y_min_max_class: 0.701347351074
	test_y_misclass: 0.0222999919206
	test_y_nll: 0.129877910018
	test_y_row_norms_max: 1.71757590771
	test_y_row_norms_mean: 0.536909937859
	test_y_row_norms_min: 0.019919058308
	train_h0_col_norms_max: 6.37537336349
	train_h0_col_norms_mean: 4.26554632187
	train_h0_col_norms_min: 2.23606181145
	train_h0_row_norms_max: 6.68972921371
	train_h0_row_norms_mean: 3.33928227425
	train_h0_row_norms_min: 0.159278333187
	train_h1_col_norms_max: 5.99916362762
	train_h1_col_norms_mean: 3.86795496941
	train_h1_col_norms_min: 1.72684931755
	train_h1_row_norms_max: 8.80523300171
	train_h1_row_norms_mean: 5.4969124794
	train_h1_row_norms_min: 3.27169203758
	train_objective: 0.00547823868692
	train_y_col_norms_max: 6.15638256073
	train_y_col_norms_mean: 5.69631719589
	train_y_col_norms_min: 5.04325008392
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998302519321
	train_y_min_max_class: 0.880661785603
	train_y_misclass: 0.00176000059582
	train_y_nll: 0.00547823868692
	train_y_row_norms_max: 1.71756851673
	train_y_row_norms_mean: 0.53690803051
	train_y_row_norms_min: 0.0199191085994
	valid_h0_col_norms_max: 6.37540435791
	valid_h0_col_norms_mean: 4.26554250717
	valid_h0_col_norms_min: 2.23606491089
	valid_h0_row_norms_max: 6.68969488144
	valid_h0_row_norms_mean: 3.33926701546
	valid_h0_row_norms_min: 0.15927760303
	valid_h1_col_norms_max: 5.99918460846
	valid_h1_col_norms_mean: 3.86793661118
	valid_h1_col_norms_min: 1.72684121132
	valid_h1_row_norms_max: 8.80519104004
	valid_h1_row_norms_mean: 5.49690055847
	valid_h1_row_norms_min: 3.27168059349
	valid_objective: 0.151706501842
	valid_y_col_norms_max: 6.1563615799
	valid_y_col_norms_mean: 5.69634532928
	valid_y_col_norms_min: 5.04322528839
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.99382263422
	valid_y_min_max_class: 0.683702290058
	valid_y_misclass: 0.0223999917507
	valid_y_nll: 0.151706501842
	valid_y_row_norms_max: 1.71757590771
	valid_y_row_norms_mean: 0.536909937859
	valid_y_row_norms_min: 0.019919058308
Time this epoch: 3.206554 seconds
Monitoring step:
	Epochs seen: 23
	Batches seen: 11500
	Examples seen: 1150000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.38593149185
	test_h0_col_norms_mean: 4.27048826218
	test_h0_col_norms_min: 2.23606491089
	test_h0_row_norms_max: 6.67957162857
	test_h0_row_norms_mean: 3.34312343597
	test_h0_row_norms_min: 0.159357041121
	test_h1_col_norms_max: 5.99570322037
	test_h1_col_norms_mean: 3.8691701889
	test_h1_col_norms_min: 1.72683918476
	test_h1_row_norms_max: 8.81508731842
	test_h1_row_norms_mean: 5.49859952927
	test_h1_row_norms_min: 3.27181625366
	test_objective: 0.123887695372
	test_y_col_norms_max: 6.22244215012
	test_y_col_norms_mean: 5.73378896713
	test_y_col_norms_min: 5.06025886536
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.993882775307
	test_y_min_max_class: 0.726153492928
	test_y_misclass: 0.0201999936253
	test_y_nll: 0.123887695372
	test_y_row_norms_max: 1.69931674004
	test_y_row_norms_mean: 0.540387809277
	test_y_row_norms_min: 0.020066447556
	train_h0_col_norms_max: 6.38596725464
	train_h0_col_norms_mean: 4.27046966553
	train_h0_col_norms_min: 2.23606181145
	train_h0_row_norms_max: 6.67954874039
	train_h0_row_norms_mean: 3.34311199188
	train_h0_row_norms_min: 0.159357577562
	train_h1_col_norms_max: 5.9957318306
	train_h1_col_norms_mean: 3.86917424202
	train_h1_col_norms_min: 1.72684860229
	train_h1_row_norms_max: 8.81507587433
	train_h1_row_norms_mean: 5.49862718582
	train_h1_row_norms_min: 3.27181768417
	train_objective: 0.00308265769854
	train_y_col_norms_max: 6.22247123718
	train_y_col_norms_mean: 5.73379087448
	train_y_col_norms_min: 5.06026697159
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998538374901
	train_y_min_max_class: 0.894672214985
	train_y_misclass: 0.000919999612961
	train_y_nll: 0.00308265769854
	train_y_row_norms_max: 1.69932484627
	train_y_row_norms_mean: 0.540388822556
	train_y_row_norms_min: 0.0200663488358
	valid_h0_col_norms_max: 6.38593149185
	valid_h0_col_norms_mean: 4.27048826218
	valid_h0_col_norms_min: 2.23606491089
	valid_h0_row_norms_max: 6.67957162857
	valid_h0_row_norms_mean: 3.34312343597
	valid_h0_row_norms_min: 0.159357041121
	valid_h1_col_norms_max: 5.99570322037
	valid_h1_col_norms_mean: 3.8691701889
	valid_h1_col_norms_min: 1.72683918476
	valid_h1_row_norms_max: 8.81508731842
	valid_h1_row_norms_mean: 5.49859952927
	valid_h1_row_norms_min: 3.27181625366
	valid_objective: 0.14809820056
	valid_y_col_norms_max: 6.22244215012
	valid_y_col_norms_mean: 5.73378896713
	valid_y_col_norms_min: 5.06025886536
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.993686497211
	valid_y_min_max_class: 0.684677302837
	valid_y_misclass: 0.0215999912471
	valid_y_nll: 0.14809820056
	valid_y_row_norms_max: 1.69931674004
	valid_y_row_norms_mean: 0.540387809277
	valid_y_row_norms_min: 0.020066447556
Time this epoch: 3.230241 seconds
Monitoring step:
	Epochs seen: 24
	Batches seen: 12000
	Examples seen: 1200000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.392578125
	test_h0_col_norms_mean: 4.27436256409
	test_h0_col_norms_min: 2.23606491089
	test_h0_row_norms_max: 6.68859195709
	test_h0_row_norms_mean: 3.34622907639
	test_h0_row_norms_min: 0.159570723772
	test_h1_col_norms_max: 5.99895811081
	test_h1_col_norms_mean: 3.87013435364
	test_h1_col_norms_min: 1.72682142258
	test_h1_row_norms_max: 8.82981967926
	test_h1_row_norms_mean: 5.49993467331
	test_h1_row_norms_min: 3.27214646339
	test_objective: 0.123282536864
	test_y_col_norms_max: 6.26617622375
	test_y_col_norms_mean: 5.76239967346
	test_y_col_norms_min: 5.08875703812
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.994056642056
	test_y_min_max_class: 0.738864719868
	test_y_misclass: 0.0195999927819
	test_y_nll: 0.123282536864
	test_y_row_norms_max: 1.71165382862
	test_y_row_norms_mean: 0.542974531651
	test_y_row_norms_min: 0.0200950335711
	train_h0_col_norms_max: 6.39255237579
	train_h0_col_norms_mean: 4.27436685562
	train_h0_col_norms_min: 2.23606181145
	train_h0_row_norms_max: 6.68859434128
	train_h0_row_norms_mean: 3.34621357918
	train_h0_row_norms_min: 0.159569814801
	train_h1_col_norms_max: 5.99892854691
	train_h1_col_norms_mean: 3.8701300621
	train_h1_col_norms_min: 1.72681927681
	train_h1_row_norms_max: 8.82980918884
	train_h1_row_norms_mean: 5.49992132187
	train_h1_row_norms_min: 3.27214837074
	train_objective: 0.00190907681827
	train_y_col_norms_max: 6.26617431641
	train_y_col_norms_mean: 5.76240110397
	train_y_col_norms_min: 5.08878278732
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998930156231
	train_y_min_max_class: 0.918284237385
	train_y_misclass: 0.000559999898542
	train_y_nll: 0.00190907681827
	train_y_row_norms_max: 1.71166217327
	train_y_row_norms_mean: 0.542971789837
	train_y_row_norms_min: 0.0200951248407
	valid_h0_col_norms_max: 6.392578125
	valid_h0_col_norms_mean: 4.27436256409
	valid_h0_col_norms_min: 2.23606491089
	valid_h0_row_norms_max: 6.68859195709
	valid_h0_row_norms_mean: 3.34622907639
	valid_h0_row_norms_min: 0.159570723772
	valid_h1_col_norms_max: 5.99895811081
	valid_h1_col_norms_mean: 3.87013435364
	valid_h1_col_norms_min: 1.72682142258
	valid_h1_row_norms_max: 8.82981967926
	valid_h1_row_norms_mean: 5.49993467331
	valid_h1_row_norms_min: 3.27214646339
	valid_objective: 0.146879151464
	valid_y_col_norms_max: 6.26617622375
	valid_y_col_norms_mean: 5.76239967346
	valid_y_col_norms_min: 5.08875703812
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.994406104088
	valid_y_min_max_class: 0.706291854382
	valid_y_misclass: 0.0211999956518
	valid_y_nll: 0.146879151464
	valid_y_row_norms_max: 1.71165382862
	valid_y_row_norms_mean: 0.542974531651
	valid_y_row_norms_min: 0.0200950335711
Time this epoch: 3.222738 seconds
Monitoring step:
	Epochs seen: 25
	Batches seen: 12500
	Examples seen: 1250000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.4013338089
	test_h0_col_norms_mean: 4.27870225906
	test_h0_col_norms_min: 2.2360560894
	test_h0_row_norms_max: 6.69665718079
	test_h0_row_norms_mean: 3.34976291656
	test_h0_row_norms_min: 0.160002231598
	test_h1_col_norms_max: 6.00171422958
	test_h1_col_norms_mean: 3.87124419212
	test_h1_col_norms_min: 1.72680687904
	test_h1_row_norms_max: 8.85285282135
	test_h1_row_norms_mean: 5.50152254105
	test_h1_row_norms_min: 3.27291631699
	test_objective: 0.121946468949
	test_y_col_norms_max: 6.27880191803
	test_y_col_norms_mean: 5.80026340485
	test_y_col_norms_min: 5.12123060226
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.993581533432
	test_y_min_max_class: 0.695117354393
	test_y_misclass: 0.0199999921024
	test_y_nll: 0.121946468949
	test_y_row_norms_max: 1.76450884342
	test_y_row_norms_mean: 0.546270668507
	test_y_row_norms_min: 0.0209660548717
	train_h0_col_norms_max: 6.40130519867
	train_h0_col_norms_mean: 4.2786822319
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.69668722153
	train_h0_row_norms_mean: 3.34977436066
	train_h0_row_norms_min: 0.160001769662
	train_h1_col_norms_max: 6.00171136856
	train_h1_col_norms_mean: 3.87122607231
	train_h1_col_norms_min: 1.726806283
	train_h1_row_norms_max: 8.85290527344
	train_h1_row_norms_mean: 5.5015130043
	train_h1_row_norms_min: 3.27290010452
	train_objective: 0.0036760433577
	train_y_col_norms_max: 6.27877187729
	train_y_col_norms_mean: 5.80025196075
	train_y_col_norms_min: 5.12123060226
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998859524727
	train_y_min_max_class: 0.912291646004
	train_y_misclass: 0.00121999997646
	train_y_nll: 0.0036760433577
	train_y_row_norms_max: 1.76451909542
	train_y_row_norms_mean: 0.546273350716
	train_y_row_norms_min: 0.0209659561515
	valid_h0_col_norms_max: 6.4013338089
	valid_h0_col_norms_mean: 4.27870225906
	valid_h0_col_norms_min: 2.2360560894
	valid_h0_row_norms_max: 6.69665718079
	valid_h0_row_norms_mean: 3.34976291656
	valid_h0_row_norms_min: 0.160002231598
	valid_h1_col_norms_max: 6.00171422958
	valid_h1_col_norms_mean: 3.87124419212
	valid_h1_col_norms_min: 1.72680687904
	valid_h1_row_norms_max: 8.85285282135
	valid_h1_row_norms_mean: 5.50152254105
	valid_h1_row_norms_min: 3.27291631699
	valid_objective: 0.137758076191
	valid_y_col_norms_max: 6.27880191803
	valid_y_col_norms_mean: 5.80026340485
	valid_y_col_norms_min: 5.12123060226
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.994390308857
	valid_y_min_max_class: 0.728678107262
	valid_y_misclass: 0.019999993965
	valid_y_nll: 0.137758076191
	valid_y_row_norms_max: 1.76450884342
	valid_y_row_norms_mean: 0.546270668507
	valid_y_row_norms_min: 0.0209660548717
Time this epoch: 3.272793 seconds
Monitoring step:
	Epochs seen: 26
	Batches seen: 13000
	Examples seen: 1300000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.4121389389
	test_h0_col_norms_mean: 4.28374528885
	test_h0_col_norms_min: 2.2360560894
	test_h0_row_norms_max: 6.71324443817
	test_h0_row_norms_mean: 3.35392951965
	test_h0_row_norms_min: 0.1600792557
	test_h1_col_norms_max: 6.00099658966
	test_h1_col_norms_mean: 3.87249565125
	test_h1_col_norms_min: 1.72674298286
	test_h1_row_norms_max: 8.85911655426
	test_h1_row_norms_mean: 5.50325918198
	test_h1_row_norms_min: 3.27451777458
	test_objective: 0.148935392499
	test_y_col_norms_max: 6.33092308044
	test_y_col_norms_mean: 5.83676052094
	test_y_col_norms_min: 5.21046447754
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.993810713291
	test_y_min_max_class: 0.718041598797
	test_y_misclass: 0.0221999920905
	test_y_nll: 0.148935392499
	test_y_row_norms_max: 1.7590252161
	test_y_row_norms_mean: 0.54948079586
	test_y_row_norms_min: 0.020847639069
	train_h0_col_norms_max: 6.41210317612
	train_h0_col_norms_mean: 4.2837562561
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.71320962906
	train_h0_row_norms_mean: 3.35394501686
	train_h0_row_norms_min: 0.16007861495
	train_h1_col_norms_max: 6.00099611282
	train_h1_col_norms_mean: 3.87251186371
	train_h1_col_norms_min: 1.72674548626
	train_h1_row_norms_max: 8.85914611816
	train_h1_row_norms_mean: 5.50324678421
	train_h1_row_norms_min: 3.27453041077
	train_objective: 0.00680599268526
	train_y_col_norms_max: 6.33095264435
	train_y_col_norms_mean: 5.83674097061
	train_y_col_norms_min: 5.21046924591
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.99815505743
	train_y_min_max_class: 0.87034368515
	train_y_misclass: 0.00213999999687
	train_y_nll: 0.00680599268526
	train_y_row_norms_max: 1.75903534889
	train_y_row_norms_mean: 0.549481749535
	train_y_row_norms_min: 0.0208476502448
	valid_h0_col_norms_max: 6.4121389389
	valid_h0_col_norms_mean: 4.28374528885
	valid_h0_col_norms_min: 2.2360560894
	valid_h0_row_norms_max: 6.71324443817
	valid_h0_row_norms_mean: 3.35392951965
	valid_h0_row_norms_min: 0.1600792557
	valid_h1_col_norms_max: 6.00099658966
	valid_h1_col_norms_mean: 3.87249565125
	valid_h1_col_norms_min: 1.72674298286
	valid_h1_row_norms_max: 8.85911655426
	valid_h1_row_norms_mean: 5.50325918198
	valid_h1_row_norms_min: 3.27451777458
	valid_objective: 0.157335549593
	valid_y_col_norms_max: 6.33092308044
	valid_y_col_norms_mean: 5.83676052094
	valid_y_col_norms_min: 5.21046447754
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.994030356407
	valid_y_min_max_class: 0.726344525814
	valid_y_misclass: 0.0226999893785
	valid_y_nll: 0.157335549593
	valid_y_row_norms_max: 1.7590252161
	valid_y_row_norms_mean: 0.54948079586
	valid_y_row_norms_min: 0.020847639069
Time this epoch: 3.208633 seconds
Monitoring step:
	Epochs seen: 27
	Batches seen: 13500
	Examples seen: 1350000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.41813564301
	test_h0_col_norms_mean: 4.28969669342
	test_h0_col_norms_min: 2.2360560894
	test_h0_row_norms_max: 6.7286157608
	test_h0_row_norms_mean: 3.35873889923
	test_h0_row_norms_min: 0.160087496042
	test_h1_col_norms_max: 6.00020074844
	test_h1_col_norms_mean: 3.87404108047
	test_h1_col_norms_min: 1.72669911385
	test_h1_row_norms_max: 8.87103843689
	test_h1_row_norms_mean: 5.50552749634
	test_h1_row_norms_min: 3.27386808395
	test_objective: 0.143524944782
	test_y_col_norms_max: 6.35547590256
	test_y_col_norms_mean: 5.87758922577
	test_y_col_norms_min: 5.21483325958
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.994994282722
	test_y_min_max_class: 0.740391731262
	test_y_misclass: 0.0209999959916
	test_y_nll: 0.143524944782
	test_y_row_norms_max: 1.73408651352
	test_y_row_norms_mean: 0.5533670187
	test_y_row_norms_min: 0.0205177664757
	train_h0_col_norms_max: 6.41816806793
	train_h0_col_norms_mean: 4.28971195221
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.72864484787
	train_h0_row_norms_mean: 3.35872411728
	train_h0_row_norms_min: 0.160087764263
	train_h1_col_norms_max: 6.00021934509
	train_h1_col_norms_mean: 3.87405753136
	train_h1_col_norms_min: 1.72669124603
	train_h1_row_norms_max: 8.87106800079
	train_h1_row_norms_mean: 5.50554513931
	train_h1_row_norms_min: 3.27385210991
	train_objective: 0.00366839556955
	train_y_col_norms_max: 6.3555059433
	train_y_col_norms_mean: 5.87757110596
	train_y_col_norms_min: 5.21484279633
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998865902424
	train_y_min_max_class: 0.908865869045
	train_y_misclass: 0.00126000004821
	train_y_nll: 0.00366839556955
	train_y_row_norms_max: 1.73407900333
	train_y_row_norms_mean: 0.553368866444
	train_y_row_norms_min: 0.0205178782344
	valid_h0_col_norms_max: 6.41813564301
	valid_h0_col_norms_mean: 4.28969669342
	valid_h0_col_norms_min: 2.2360560894
	valid_h0_row_norms_max: 6.7286157608
	valid_h0_row_norms_mean: 3.35873889923
	valid_h0_row_norms_min: 0.160087496042
	valid_h1_col_norms_max: 6.00020074844
	valid_h1_col_norms_mean: 3.87404108047
	valid_h1_col_norms_min: 1.72669911385
	valid_h1_row_norms_max: 8.87103843689
	valid_h1_row_norms_mean: 5.50552749634
	valid_h1_row_norms_min: 3.27386808395
	valid_objective: 0.155297890306
	valid_y_col_norms_max: 6.35547590256
	valid_y_col_norms_mean: 5.87758922577
	valid_y_col_norms_min: 5.21483325958
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.994454801083
	valid_y_min_max_class: 0.73979562521
	valid_y_misclass: 0.0205999910831
	valid_y_nll: 0.155297890306
	valid_y_row_norms_max: 1.73408651352
	valid_y_row_norms_mean: 0.5533670187
	valid_y_row_norms_min: 0.0205177664757
Time this epoch: 3.239587 seconds
Monitoring step:
	Epochs seen: 28
	Batches seen: 14000
	Examples seen: 1400000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.42320108414
	test_h0_col_norms_mean: 4.29595088959
	test_h0_col_norms_min: 2.2360560894
	test_h0_row_norms_max: 6.73588323593
	test_h0_row_norms_mean: 3.36365532875
	test_h0_row_norms_min: 0.160095050931
	test_h1_col_norms_max: 6.00109481812
	test_h1_col_norms_mean: 3.87561798096
	test_h1_col_norms_min: 1.72673380375
	test_h1_row_norms_max: 8.90102100372
	test_h1_row_norms_mean: 5.50783443451
	test_h1_row_norms_min: 3.27579259872
	test_objective: 0.176090538502
	test_y_col_norms_max: 6.37317848206
	test_y_col_norms_mean: 5.91372203827
	test_y_col_norms_min: 5.26935434341
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.994395077229
	test_y_min_max_class: 0.743200361729
	test_y_misclass: 0.0221999939531
	test_y_nll: 0.176090538502
	test_y_row_norms_max: 1.72095572948
	test_y_row_norms_mean: 0.556442499161
	test_y_row_norms_min: 0.0208181608468
	train_h0_col_norms_max: 6.4232301712
	train_h0_col_norms_mean: 4.29595375061
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.73585557938
	train_h0_row_norms_mean: 3.36363792419
	train_h0_row_norms_min: 0.160095304251
	train_h1_col_norms_max: 6.00107383728
	train_h1_col_norms_mean: 3.87561368942
	train_h1_col_norms_min: 1.72674226761
	train_h1_row_norms_max: 8.90106678009
	train_h1_row_norms_mean: 5.50785970688
	train_h1_row_norms_min: 3.27577996254
	train_objective: 0.00485403602943
	train_y_col_norms_max: 6.37316846848
	train_y_col_norms_mean: 5.91373300552
	train_y_col_norms_min: 5.26935815811
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998713254929
	train_y_min_max_class: 0.896820962429
	train_y_misclass: 0.00136000022758
	train_y_nll: 0.00485403602943
	train_y_row_norms_max: 1.72096157074
	train_y_row_norms_mean: 0.556439995766
	train_y_row_norms_min: 0.0208181329072
	valid_h0_col_norms_max: 6.42320108414
	valid_h0_col_norms_mean: 4.29595088959
	valid_h0_col_norms_min: 2.2360560894
	valid_h0_row_norms_max: 6.73588323593
	valid_h0_row_norms_mean: 3.36365532875
	valid_h0_row_norms_min: 0.160095050931
	valid_h1_col_norms_max: 6.00109481812
	valid_h1_col_norms_mean: 3.87561798096
	valid_h1_col_norms_min: 1.72673380375
	valid_h1_row_norms_max: 8.90102100372
	valid_h1_row_norms_mean: 5.50783443451
	valid_h1_row_norms_min: 3.27579259872
	valid_objective: 0.183195546269
	valid_y_col_norms_max: 6.37317848206
	valid_y_col_norms_mean: 5.91372203827
	valid_y_col_norms_min: 5.26935434341
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.994852602482
	valid_y_min_max_class: 0.74536216259
	valid_y_misclass: 0.0237999893725
	valid_y_nll: 0.183195546269
	valid_y_row_norms_max: 1.72095572948
	valid_y_row_norms_mean: 0.556442499161
	valid_y_row_norms_min: 0.0208181608468
Time this epoch: 3.306142 seconds
Monitoring step:
	Epochs seen: 29
	Batches seen: 14500
	Examples seen: 1450000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.45381164551
	test_h0_col_norms_mean: 4.30269384384
	test_h0_col_norms_min: 2.2360560894
	test_h0_row_norms_max: 6.74906110764
	test_h0_row_norms_mean: 3.36890244484
	test_h0_row_norms_min: 0.159244820476
	test_h1_col_norms_max: 6.00183820724
	test_h1_col_norms_mean: 3.87737250328
	test_h1_col_norms_min: 1.7269256115
	test_h1_row_norms_max: 8.89922237396
	test_h1_row_norms_mean: 5.51038217545
	test_h1_row_norms_min: 3.27727627754
	test_objective: 0.158995479345
	test_y_col_norms_max: 6.38246154785
	test_y_col_norms_mean: 5.95248889923
	test_y_col_norms_min: 5.29096841812
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.994712769985
	test_y_min_max_class: 0.747330605984
	test_y_misclass: 0.0207999963313
	test_y_nll: 0.158995479345
	test_y_row_norms_max: 1.74560809135
	test_y_row_norms_mean: 0.559956371784
	test_y_row_norms_min: 0.0206812545657
	train_h0_col_norms_max: 6.45380783081
	train_h0_col_norms_mean: 4.3027176857
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.74909591675
	train_h0_row_norms_mean: 3.36888813972
	train_h0_row_norms_min: 0.159244179726
	train_h1_col_norms_max: 6.00187015533
	train_h1_col_norms_mean: 3.87737822533
	train_h1_col_norms_min: 1.72692549229
	train_h1_row_norms_max: 8.89921569824
	train_h1_row_norms_mean: 5.51035165787
	train_h1_row_norms_min: 3.27729272842
	train_objective: 0.00499874725938
	train_y_col_norms_max: 6.38246393204
	train_y_col_norms_mean: 5.9525179863
	train_y_col_norms_min: 5.29098033905
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998843669891
	train_y_min_max_class: 0.907702028751
	train_y_misclass: 0.00154000031762
	train_y_nll: 0.00499874725938
	train_y_row_norms_max: 1.7455984354
	train_y_row_norms_mean: 0.559955894947
	train_y_row_norms_min: 0.0206812303513
	valid_h0_col_norms_max: 6.45381164551
	valid_h0_col_norms_mean: 4.30269384384
	valid_h0_col_norms_min: 2.2360560894
	valid_h0_row_norms_max: 6.74906110764
	valid_h0_row_norms_mean: 3.36890244484
	valid_h0_row_norms_min: 0.159244820476
	valid_h1_col_norms_max: 6.00183820724
	valid_h1_col_norms_mean: 3.87737250328
	valid_h1_col_norms_min: 1.7269256115
	valid_h1_row_norms_max: 8.89922237396
	valid_h1_row_norms_mean: 5.51038217545
	valid_h1_row_norms_min: 3.27727627754
	valid_objective: 0.161353841424
	valid_y_col_norms_max: 6.38246154785
	valid_y_col_norms_mean: 5.95248889923
	valid_y_col_norms_min: 5.29096841812
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995362341404
	valid_y_min_max_class: 0.764035582542
	valid_y_misclass: 0.0211999919266
	valid_y_nll: 0.161353841424
	valid_y_row_norms_max: 1.74560809135
	valid_y_row_norms_mean: 0.559956371784
	valid_y_row_norms_min: 0.0206812545657
Time this epoch: 3.264931 seconds
Monitoring step:
	Epochs seen: 30
	Batches seen: 15000
	Examples seen: 1500000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.45126152039
	test_h0_col_norms_mean: 4.30855321884
	test_h0_col_norms_min: 2.2360560894
	test_h0_row_norms_max: 6.77185153961
	test_h0_row_norms_mean: 3.37364006042
	test_h0_row_norms_min: 0.159440949559
	test_h1_col_norms_max: 6.00142860413
	test_h1_col_norms_mean: 3.8789036274
	test_h1_col_norms_min: 1.72696387768
	test_h1_row_norms_max: 8.92525005341
	test_h1_row_norms_mean: 5.5125746727
	test_h1_row_norms_min: 3.27923321724
	test_objective: 0.159945309162
	test_y_col_norms_max: 6.50855636597
	test_y_col_norms_mean: 5.9870095253
	test_y_col_norms_min: 5.30891561508
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995383441448
	test_y_min_max_class: 0.755910158157
	test_y_misclass: 0.0218999926001
	test_y_nll: 0.159945309162
	test_y_row_norms_max: 1.7809484005
	test_y_row_norms_mean: 0.563234627247
	test_y_row_norms_min: 0.0199234094471
	train_h0_col_norms_max: 6.45129537582
	train_h0_col_norms_mean: 4.30855226517
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.77182006836
	train_h0_row_norms_mean: 3.37362527847
	train_h0_row_norms_min: 0.159441739321
	train_h1_col_norms_max: 6.00145721436
	train_h1_col_norms_mean: 3.87892222404
	train_h1_col_norms_min: 1.72697114944
	train_h1_row_norms_max: 8.92523765564
	train_h1_row_norms_mean: 5.5125579834
	train_h1_row_norms_min: 3.27921772003
	train_objective: 0.0052194846794
	train_y_col_norms_max: 6.50858449936
	train_y_col_norms_mean: 5.98699235916
	train_y_col_norms_min: 5.3088889122
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998774111271
	train_y_min_max_class: 0.904954195023
	train_y_misclass: 0.00160000030883
	train_y_nll: 0.0052194846794
	train_y_row_norms_max: 1.78094053268
	train_y_row_norms_mean: 0.56323415041
	train_y_row_norms_min: 0.0199234373868
	valid_h0_col_norms_max: 6.45126152039
	valid_h0_col_norms_mean: 4.30855321884
	valid_h0_col_norms_min: 2.2360560894
	valid_h0_row_norms_max: 6.77185153961
	valid_h0_row_norms_mean: 3.37364006042
	valid_h0_row_norms_min: 0.159440949559
	valid_h1_col_norms_max: 6.00142860413
	valid_h1_col_norms_mean: 3.8789036274
	valid_h1_col_norms_min: 1.72696387768
	valid_h1_row_norms_max: 8.92525005341
	valid_h1_row_norms_mean: 5.5125746727
	valid_h1_row_norms_min: 3.27923321724
	valid_objective: 0.172797784209
	valid_y_col_norms_max: 6.50855636597
	valid_y_col_norms_mean: 5.9870095253
	valid_y_col_norms_min: 5.30891561508
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995368361473
	valid_y_min_max_class: 0.741488099098
	valid_y_misclass: 0.0201999936253
	valid_y_nll: 0.172797784209
	valid_y_row_norms_max: 1.7809484005
	valid_y_row_norms_mean: 0.563234627247
	valid_y_row_norms_min: 0.0199234094471
Time this epoch: 3.279603 seconds
Monitoring step:
	Epochs seen: 31
	Batches seen: 15500
	Examples seen: 1550000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.45977544785
	test_h0_col_norms_mean: 4.31496477127
	test_h0_col_norms_min: 2.23605561256
	test_h0_row_norms_max: 6.77787017822
	test_h0_row_norms_mean: 3.37872552872
	test_h0_row_norms_min: 0.167061835527
	test_h1_col_norms_max: 6.00070905685
	test_h1_col_norms_mean: 3.88056731224
	test_h1_col_norms_min: 1.7269756794
	test_h1_row_norms_max: 8.94437408447
	test_h1_row_norms_mean: 5.51490449905
	test_h1_row_norms_min: 3.27992272377
	test_objective: 0.131766811013
	test_y_col_norms_max: 6.49069547653
	test_y_col_norms_mean: 6.01968860626
	test_y_col_norms_min: 5.32379293442
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.99406349659
	test_y_min_max_class: 0.709186255932
	test_y_misclass: 0.0214999895543
	test_y_nll: 0.131766811013
	test_y_row_norms_max: 1.75881135464
	test_y_row_norms_mean: 0.566387176514
	test_y_row_norms_min: 0.0195109490305
	train_h0_col_norms_max: 6.45976924896
	train_h0_col_norms_mean: 4.31498289108
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.77783346176
	train_h0_row_norms_mean: 3.37874174118
	train_h0_row_norms_min: 0.167062133551
	train_h1_col_norms_max: 6.00073814392
	train_h1_col_norms_mean: 3.88058972359
	train_h1_col_norms_min: 1.72698163986
	train_h1_row_norms_max: 8.94434833527
	train_h1_row_norms_mean: 5.51487779617
	train_h1_row_norms_min: 3.27992391586
	train_objective: 0.00692026689649
	train_y_col_norms_max: 6.49070358276
	train_y_col_norms_mean: 6.01966762543
	train_y_col_norms_min: 5.323802948
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.99833124876
	train_y_min_max_class: 0.877075016499
	train_y_misclass: 0.00206000055186
	train_y_nll: 0.00692026689649
	train_y_row_norms_max: 1.7588135004
	train_y_row_norms_mean: 0.566390037537
	train_y_row_norms_min: 0.0195109229535
	valid_h0_col_norms_max: 6.45977544785
	valid_h0_col_norms_mean: 4.31496477127
	valid_h0_col_norms_min: 2.23605561256
	valid_h0_row_norms_max: 6.77787017822
	valid_h0_row_norms_mean: 3.37872552872
	valid_h0_row_norms_min: 0.167061835527
	valid_h1_col_norms_max: 6.00070905685
	valid_h1_col_norms_mean: 3.88056731224
	valid_h1_col_norms_min: 1.7269756794
	valid_h1_row_norms_max: 8.94437408447
	valid_h1_row_norms_mean: 5.51490449905
	valid_h1_row_norms_min: 3.27992272377
	valid_objective: 0.161748409271
	valid_y_col_norms_max: 6.49069547653
	valid_y_col_norms_mean: 6.01968860626
	valid_y_col_norms_min: 5.32379293442
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.994541585445
	valid_y_min_max_class: 0.741445958614
	valid_y_misclass: 0.0221999902278
	valid_y_nll: 0.161748409271
	valid_y_row_norms_max: 1.75881135464
	valid_y_row_norms_mean: 0.566387176514
	valid_y_row_norms_min: 0.0195109490305
Time this epoch: 3.251266 seconds
Monitoring step:
	Epochs seen: 32
	Batches seen: 16000
	Examples seen: 1600000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.45842170715
	test_h0_col_norms_mean: 4.32045173645
	test_h0_col_norms_min: 2.23605561256
	test_h0_row_norms_max: 6.78466415405
	test_h0_row_norms_mean: 3.38315415382
	test_h0_row_norms_min: 0.16744081676
	test_h1_col_norms_max: 6.00018596649
	test_h1_col_norms_mean: 3.88205099106
	test_h1_col_norms_min: 1.72608160973
	test_h1_row_norms_max: 8.9562330246
	test_h1_row_norms_mean: 5.51705217361
	test_h1_row_norms_min: 3.28056788445
	test_objective: 0.156137660146
	test_y_col_norms_max: 6.55750894547
	test_y_col_norms_mean: 6.04845666885
	test_y_col_norms_min: 5.33018064499
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995250225067
	test_y_min_max_class: 0.769113063812
	test_y_misclass: 0.0210999939591
	test_y_nll: 0.156137660146
	test_y_row_norms_max: 1.7797113657
	test_y_row_norms_mean: 0.568675458431
	test_y_row_norms_min: 0.0223224461079
	train_h0_col_norms_max: 6.45844841003
	train_h0_col_norms_mean: 4.32046604156
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.7846736908
	train_h0_row_norms_mean: 3.3831589222
	train_h0_row_norms_min: 0.167441576719
	train_h1_col_norms_max: 6.00020599365
	train_h1_col_norms_mean: 3.88205075264
	train_h1_col_norms_min: 1.7260876894
	train_h1_row_norms_max: 8.9562292099
	train_h1_row_norms_mean: 5.51702356339
	train_h1_row_norms_min: 3.280554533
	train_objective: 0.00448899809271
	train_y_col_norms_max: 6.55747938156
	train_y_col_norms_mean: 6.0484457016
	train_y_col_norms_min: 5.33017015457
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998964965343
	train_y_min_max_class: 0.915260314941
	train_y_misclass: 0.0013400001917
	train_y_nll: 0.00448899809271
	train_y_row_norms_max: 1.77971935272
	train_y_row_norms_mean: 0.568675458431
	train_y_row_norms_min: 0.0223225466907
	valid_h0_col_norms_max: 6.45842170715
	valid_h0_col_norms_mean: 4.32045173645
	valid_h0_col_norms_min: 2.23605561256
	valid_h0_row_norms_max: 6.78466415405
	valid_h0_row_norms_mean: 3.38315415382
	valid_h0_row_norms_min: 0.16744081676
	valid_h1_col_norms_max: 6.00018596649
	valid_h1_col_norms_mean: 3.88205099106
	valid_h1_col_norms_min: 1.72608160973
	valid_h1_row_norms_max: 8.9562330246
	valid_h1_row_norms_mean: 5.51705217361
	valid_h1_row_norms_min: 3.28056788445
	valid_objective: 0.185146003962
	valid_y_col_norms_max: 6.55750894547
	valid_y_col_norms_mean: 6.04845666885
	valid_y_col_norms_min: 5.33018064499
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995594918728
	valid_y_min_max_class: 0.771956503391
	valid_y_misclass: 0.0222999881953
	valid_y_nll: 0.185146003962
	valid_y_row_norms_max: 1.7797113657
	valid_y_row_norms_mean: 0.568675458431
	valid_y_row_norms_min: 0.0223224461079
Time this epoch: 3.265816 seconds
Monitoring step:
	Epochs seen: 33
	Batches seen: 16500
	Examples seen: 1650000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.48042154312
	test_h0_col_norms_mean: 4.32581949234
	test_h0_col_norms_min: 2.23605561256
	test_h0_row_norms_max: 6.79249668121
	test_h0_row_norms_mean: 3.38737988472
	test_h0_row_norms_min: 0.167504921556
	test_h1_col_norms_max: 6.0035238266
	test_h1_col_norms_mean: 3.88333916664
	test_h1_col_norms_min: 1.72610199451
	test_h1_row_norms_max: 8.94651126862
	test_h1_row_norms_mean: 5.51890897751
	test_h1_row_norms_min: 3.28360319138
	test_objective: 0.142962425947
	test_y_col_norms_max: 6.59494447708
	test_y_col_norms_mean: 6.06826543808
	test_y_col_norms_min: 5.36811923981
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995299935341
	test_y_min_max_class: 0.757121562958
	test_y_misclass: 0.0198999904096
	test_y_nll: 0.142962425947
	test_y_row_norms_max: 1.8589527607
	test_y_row_norms_mean: 0.570496380329
	test_y_row_norms_min: 0.0232647489756
	train_h0_col_norms_max: 6.48045063019
	train_h0_col_norms_mean: 4.32584047318
	train_h0_col_norms_min: 2.23605871201
	train_h0_row_norms_max: 6.79252815247
	train_h0_row_norms_mean: 3.38736534119
	train_h0_row_norms_min: 0.167504131794
	train_h1_col_norms_max: 6.00354385376
	train_h1_col_norms_mean: 3.88335561752
	train_h1_col_norms_min: 1.72609436512
	train_h1_row_norms_max: 8.94646167755
	train_h1_row_norms_mean: 5.51891183853
	train_h1_row_norms_min: 3.28361749649
	train_objective: 0.00277355127037
	train_y_col_norms_max: 6.59491348267
	train_y_col_norms_mean: 6.06824493408
	train_y_col_norms_min: 5.36814403534
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999049842358
	train_y_min_max_class: 0.921933472157
	train_y_misclass: 0.000999999581836
	train_y_nll: 0.00277355127037
	train_y_row_norms_max: 1.85895049572
	train_y_row_norms_mean: 0.570495426655
	train_y_row_norms_min: 0.0232646763325
	valid_h0_col_norms_max: 6.48042154312
	valid_h0_col_norms_mean: 4.32581949234
	valid_h0_col_norms_min: 2.23605561256
	valid_h0_row_norms_max: 6.79249668121
	valid_h0_row_norms_mean: 3.38737988472
	valid_h0_row_norms_min: 0.167504921556
	valid_h1_col_norms_max: 6.0035238266
	valid_h1_col_norms_mean: 3.88333916664
	valid_h1_col_norms_min: 1.72610199451
	valid_h1_row_norms_max: 8.94651126862
	valid_h1_row_norms_mean: 5.51890897751
	valid_h1_row_norms_min: 3.28360319138
	valid_objective: 0.179574415088
	valid_y_col_norms_max: 6.59494447708
	valid_y_col_norms_mean: 6.06826543808
	valid_y_col_norms_min: 5.36811923981
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995453417301
	valid_y_min_max_class: 0.75123167038
	valid_y_misclass: 0.0197999905795
	valid_y_nll: 0.179574415088
	valid_y_row_norms_max: 1.8589527607
	valid_y_row_norms_mean: 0.570496380329
	valid_y_row_norms_min: 0.0232647489756
Time this epoch: 3.231476 seconds
Monitoring step:
	Epochs seen: 34
	Batches seen: 17000
	Examples seen: 1700000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.4917049408
	test_h0_col_norms_mean: 4.33141994476
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.79732465744
	test_h0_row_norms_mean: 3.39186024666
	test_h0_row_norms_min: 0.171120882034
	test_h1_col_norms_max: 6.00534772873
	test_h1_col_norms_mean: 3.88460206985
	test_h1_col_norms_min: 1.72610270977
	test_h1_row_norms_max: 8.96625423431
	test_h1_row_norms_mean: 5.52066421509
	test_h1_row_norms_min: 3.28276824951
	test_objective: 0.141110450029
	test_y_col_norms_max: 6.61644887924
	test_y_col_norms_mean: 6.09203910828
	test_y_col_norms_min: 5.40572547913
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.994820356369
	test_y_min_max_class: 0.726503133774
	test_y_misclass: 0.0200999956578
	test_y_nll: 0.141110450029
	test_y_row_norms_max: 1.85092616081
	test_y_row_norms_mean: 0.572713196278
	test_y_row_norms_min: 0.0240506455302
	train_h0_col_norms_max: 6.49167537689
	train_h0_col_norms_mean: 4.33143472672
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.79731702805
	train_h0_row_norms_mean: 3.39186143875
	train_h0_row_norms_min: 0.171120166779
	train_h1_col_norms_max: 6.00534725189
	train_h1_col_norms_mean: 3.88458299637
	train_h1_col_norms_min: 1.72609496117
	train_h1_row_norms_max: 8.9662437439
	train_h1_row_norms_mean: 5.52065134048
	train_h1_row_norms_min: 3.28278303146
	train_objective: 0.00290546845645
	train_y_col_norms_max: 6.61642169952
	train_y_col_norms_mean: 6.09206676483
	train_y_col_norms_min: 5.40573072433
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999073982239
	train_y_min_max_class: 0.924475312233
	train_y_misclass: 0.000939999590628
	train_y_nll: 0.00290546845645
	train_y_row_norms_max: 1.85091614723
	train_y_row_norms_mean: 0.572711467743
	train_y_row_norms_min: 0.0240505319089
	valid_h0_col_norms_max: 6.4917049408
	valid_h0_col_norms_mean: 4.33141994476
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.79732465744
	valid_h0_row_norms_mean: 3.39186024666
	valid_h0_row_norms_min: 0.171120882034
	valid_h1_col_norms_max: 6.00534772873
	valid_h1_col_norms_mean: 3.88460206985
	valid_h1_col_norms_min: 1.72610270977
	valid_h1_row_norms_max: 8.96625423431
	valid_h1_row_norms_mean: 5.52066421509
	valid_h1_row_norms_min: 3.28276824951
	valid_objective: 0.162981122732
	valid_y_col_norms_max: 6.61644887924
	valid_y_col_norms_mean: 6.09203910828
	valid_y_col_norms_min: 5.40572547913
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995312690735
	valid_y_min_max_class: 0.743762373924
	valid_y_misclass: 0.0194999910891
	valid_y_nll: 0.162981122732
	valid_y_row_norms_max: 1.85092616081
	valid_y_row_norms_mean: 0.572713196278
	valid_y_row_norms_min: 0.0240506455302
Time this epoch: 3.214131 seconds
Monitoring step:
	Epochs seen: 35
	Batches seen: 17500
	Examples seen: 1750000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.49574804306
	test_h0_col_norms_mean: 4.3364033699
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.8160161972
	test_h0_row_norms_mean: 3.39588427544
	test_h0_row_norms_min: 0.171171665192
	test_h1_col_norms_max: 6.00441598892
	test_h1_col_norms_mean: 3.88574457169
	test_h1_col_norms_min: 1.72610199451
	test_h1_row_norms_max: 8.98808574677
	test_h1_row_norms_mean: 5.52225542068
	test_h1_row_norms_min: 3.28273797035
	test_objective: 0.170048907399
	test_y_col_norms_max: 6.62913417816
	test_y_col_norms_mean: 6.11489725113
	test_y_col_norms_min: 5.41416931152
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.994616866112
	test_y_min_max_class: 0.73312073946
	test_y_misclass: 0.0217999909073
	test_y_nll: 0.170048907399
	test_y_row_norms_max: 1.85863983631
	test_y_row_norms_mean: 0.574832618237
	test_y_row_norms_min: 0.0238261986524
	train_h0_col_norms_max: 6.49571895599
	train_h0_col_norms_mean: 4.33637952805
	train_h0_col_norms_min: 2.23605918884
	train_h0_row_norms_max: 6.81597948074
	train_h0_row_norms_mean: 3.39588832855
	train_h0_row_norms_min: 0.171171709895
	train_h1_col_norms_max: 6.00439691544
	train_h1_col_norms_mean: 3.88574552536
	train_h1_col_norms_min: 1.72609436512
	train_h1_row_norms_max: 8.98807621002
	train_h1_row_norms_mean: 5.52225255966
	train_h1_row_norms_min: 3.28275132179
	train_objective: 0.00725457724184
	train_y_col_norms_max: 6.62916135788
	train_y_col_norms_mean: 6.11490011215
	train_y_col_norms_min: 5.41417980194
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998784661293
	train_y_min_max_class: 0.90457379818
	train_y_misclass: 0.00184000050649
	train_y_nll: 0.00725457724184
	train_y_row_norms_max: 1.85864841938
	train_y_row_norms_mean: 0.574829816818
	train_y_row_norms_min: 0.0238260868937
	valid_h0_col_norms_max: 6.49574804306
	valid_h0_col_norms_mean: 4.3364033699
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.8160161972
	valid_h0_row_norms_mean: 3.39588427544
	valid_h0_row_norms_min: 0.171171665192
	valid_h1_col_norms_max: 6.00441598892
	valid_h1_col_norms_mean: 3.88574457169
	valid_h1_col_norms_min: 1.72610199451
	valid_h1_row_norms_max: 8.98808574677
	valid_h1_row_norms_mean: 5.52225542068
	valid_h1_row_norms_min: 3.28273797035
	valid_objective: 0.188135892153
	valid_y_col_norms_max: 6.62913417816
	valid_y_col_norms_mean: 6.11489725113
	valid_y_col_norms_min: 5.41416931152
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.99530762434
	valid_y_min_max_class: 0.754189014435
	valid_y_misclass: 0.0216999892145
	valid_y_nll: 0.188135892153
	valid_y_row_norms_max: 1.85863983631
	valid_y_row_norms_mean: 0.574832618237
	valid_y_row_norms_min: 0.0238261986524
Time this epoch: 3.284179 seconds
Monitoring step:
	Epochs seen: 36
	Batches seen: 18000
	Examples seen: 1800000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.50911712646
	test_h0_col_norms_mean: 4.34344434738
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.84686803818
	test_h0_row_norms_mean: 3.40156388283
	test_h0_row_norms_min: 0.171174243093
	test_h1_col_norms_max: 6.00547456741
	test_h1_col_norms_mean: 3.88733744621
	test_h1_col_norms_min: 1.72609961033
	test_h1_row_norms_max: 9.016705513
	test_h1_row_norms_mean: 5.5244436264
	test_h1_row_norms_min: 3.28328037262
	test_objective: 0.147451668978
	test_y_col_norms_max: 6.66465806961
	test_y_col_norms_mean: 6.14104557037
	test_y_col_norms_min: 5.43022489548
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.994985222816
	test_y_min_max_class: 0.730050563812
	test_y_misclass: 0.0206999927759
	test_y_nll: 0.147451668978
	test_y_row_norms_max: 1.78328752518
	test_y_row_norms_mean: 0.577396690845
	test_y_row_norms_min: 0.025094171986
	train_h0_col_norms_max: 6.50908374786
	train_h0_col_norms_mean: 4.34342718124
	train_h0_col_norms_min: 2.23605918884
	train_h0_row_norms_max: 6.8468914032
	train_h0_row_norms_mean: 3.40154623985
	train_h0_row_norms_min: 0.171174883842
	train_h1_col_norms_max: 6.00550603867
	train_h1_col_norms_mean: 3.8873193264
	train_h1_col_norms_min: 1.72609198093
	train_h1_row_norms_max: 9.0167131424
	train_h1_row_norms_mean: 5.52441453934
	train_h1_row_norms_min: 3.28326916695
	train_objective: 0.00539966486394
	train_y_col_norms_max: 6.66468572617
	train_y_col_norms_mean: 6.14102125168
	train_y_col_norms_min: 5.43022203445
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.99889010191
	train_y_min_max_class: 0.915294647217
	train_y_misclass: 0.00152000051457
	train_y_nll: 0.00539966486394
	train_y_row_norms_max: 1.78329563141
	train_y_row_norms_mean: 0.57739341259
	train_y_row_norms_min: 0.0250942651182
	valid_h0_col_norms_max: 6.50911712646
	valid_h0_col_norms_mean: 4.34344434738
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.84686803818
	valid_h0_row_norms_mean: 3.40156388283
	valid_h0_row_norms_min: 0.171174243093
	valid_h1_col_norms_max: 6.00547456741
	valid_h1_col_norms_mean: 3.88733744621
	valid_h1_col_norms_min: 1.72609961033
	valid_h1_row_norms_max: 9.016705513
	valid_h1_row_norms_mean: 5.5244436264
	valid_h1_row_norms_min: 3.28328037262
	valid_objective: 0.161581993103
	valid_y_col_norms_max: 6.66465806961
	valid_y_col_norms_mean: 6.14104557037
	valid_y_col_norms_min: 5.43022489548
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995217263699
	valid_y_min_max_class: 0.752208411694
	valid_y_misclass: 0.0202999934554
	valid_y_nll: 0.161581993103
	valid_y_row_norms_max: 1.78328752518
	valid_y_row_norms_mean: 0.577396690845
	valid_y_row_norms_min: 0.025094171986
Time this epoch: 3.277391 seconds
Monitoring step:
	Epochs seen: 37
	Batches seen: 18500
	Examples seen: 1850000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.51882982254
	test_h0_col_norms_mean: 4.34898805618
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.86316585541
	test_h0_row_norms_mean: 3.40600013733
	test_h0_row_norms_min: 0.171176031232
	test_h1_col_norms_max: 6.00360631943
	test_h1_col_norms_mean: 3.88884663582
	test_h1_col_norms_min: 1.72619795799
	test_h1_row_norms_max: 9.0371131897
	test_h1_row_norms_mean: 5.526512146
	test_h1_row_norms_min: 3.28363656998
	test_objective: 0.174357533455
	test_y_col_norms_max: 6.70250511169
	test_y_col_norms_mean: 6.17451667786
	test_y_col_norms_min: 5.43355512619
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995144307613
	test_y_min_max_class: 0.754079401493
	test_y_misclass: 0.0214999951422
	test_y_nll: 0.174357533455
	test_y_row_norms_max: 1.83495354652
	test_y_row_norms_mean: 0.580399692059
	test_y_row_norms_min: 0.0246269144118
	train_h0_col_norms_max: 6.51883935928
	train_h0_col_norms_mean: 4.34899139404
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.86313438416
	train_h0_row_norms_mean: 3.40601491928
	train_h0_row_norms_min: 0.171176567674
	train_h1_col_norms_max: 6.00361680984
	train_h1_col_norms_mean: 3.88884592056
	train_h1_col_norms_min: 1.72620582581
	train_h1_row_norms_max: 9.03706741333
	train_h1_row_norms_mean: 5.52653741837
	train_h1_row_norms_min: 3.28362202644
	train_objective: 0.00331209623255
	train_y_col_norms_max: 6.70247983932
	train_y_col_norms_mean: 6.17454624176
	train_y_col_norms_min: 5.43355798721
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999139487743
	train_y_min_max_class: 0.931698381901
	train_y_misclass: 0.000979999545962
	train_y_nll: 0.00331209623255
	train_y_row_norms_max: 1.83494448662
	train_y_row_norms_mean: 0.580400049686
	train_y_row_norms_min: 0.0246269479394
	valid_h0_col_norms_max: 6.51882982254
	valid_h0_col_norms_mean: 4.34898805618
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.86316585541
	valid_h0_row_norms_mean: 3.40600013733
	valid_h0_row_norms_min: 0.171176031232
	valid_h1_col_norms_max: 6.00360631943
	valid_h1_col_norms_mean: 3.88884663582
	valid_h1_col_norms_min: 1.72619795799
	valid_h1_row_norms_max: 9.0371131897
	valid_h1_row_norms_mean: 5.526512146
	valid_h1_row_norms_min: 3.28363656998
	valid_objective: 0.164556577802
	valid_y_col_norms_max: 6.70250511169
	valid_y_col_norms_mean: 6.17451667786
	valid_y_col_norms_min: 5.43355512619
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995738983154
	valid_y_min_max_class: 0.76286149025
	valid_y_misclass: 0.0205999910831
	valid_y_nll: 0.164556577802
	valid_y_row_norms_max: 1.83495354652
	valid_y_row_norms_mean: 0.580399692059
	valid_y_row_norms_min: 0.0246269144118
Time this epoch: 3.300500 seconds
Monitoring step:
	Epochs seen: 38
	Batches seen: 19000
	Examples seen: 1900000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.52381372452
	test_h0_col_norms_mean: 4.35216140747
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.87646770477
	test_h0_row_norms_mean: 3.40848636627
	test_h0_row_norms_min: 0.171177119017
	test_h1_col_norms_max: 6.00470304489
	test_h1_col_norms_mean: 3.88970422745
	test_h1_col_norms_min: 1.72622287273
	test_h1_row_norms_max: 9.0545091629
	test_h1_row_norms_mean: 5.52772140503
	test_h1_row_norms_min: 3.28486537933
	test_objective: 0.16956473887
	test_y_col_norms_max: 6.70925521851
	test_y_col_norms_mean: 6.2000246048
	test_y_col_norms_min: 5.47072219849
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.996227920055
	test_y_min_max_class: 0.774923741817
	test_y_misclass: 0.0192999932915
	test_y_nll: 0.16956473887
	test_y_row_norms_max: 1.87937033176
	test_y_row_norms_mean: 0.582399070263
	test_y_row_norms_min: 0.0244527608156
	train_h0_col_norms_max: 6.52384281158
	train_h0_col_norms_mean: 4.35217618942
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.87646818161
	train_h0_row_norms_mean: 3.40846681595
	train_h0_row_norms_min: 0.171177104115
	train_h1_col_norms_max: 6.00473213196
	train_h1_col_norms_mean: 3.88968753815
	train_h1_col_norms_min: 1.72621440887
	train_h1_row_norms_max: 9.05449295044
	train_h1_row_norms_mean: 5.52773332596
	train_h1_row_norms_min: 3.28484797478
	train_objective: 0.0016456496669
	train_y_col_norms_max: 6.70928049088
	train_y_col_norms_mean: 6.20005607605
	train_y_col_norms_min: 5.47074699402
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999532461166
	train_y_min_max_class: 0.958807349205
	train_y_misclass: 0.000520000001416
	train_y_nll: 0.0016456496669
	train_y_row_norms_max: 1.87937915325
	train_y_row_norms_mean: 0.582397639751
	train_y_row_norms_min: 0.0244527999312
	valid_h0_col_norms_max: 6.52381372452
	valid_h0_col_norms_mean: 4.35216140747
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.87646770477
	valid_h0_row_norms_mean: 3.40848636627
	valid_h0_row_norms_min: 0.171177119017
	valid_h1_col_norms_max: 6.00470304489
	valid_h1_col_norms_mean: 3.88970422745
	valid_h1_col_norms_min: 1.72622287273
	valid_h1_row_norms_max: 9.0545091629
	valid_h1_row_norms_mean: 5.52772140503
	valid_h1_row_norms_min: 3.28486537933
	valid_objective: 0.174608826637
	valid_y_col_norms_max: 6.70925521851
	valid_y_col_norms_mean: 6.2000246048
	valid_y_col_norms_min: 5.47072219849
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996570110321
	valid_y_min_max_class: 0.792669534683
	valid_y_misclass: 0.0185999963433
	valid_y_nll: 0.174608826637
	valid_y_row_norms_max: 1.87937033176
	valid_y_row_norms_mean: 0.582399070263
	valid_y_row_norms_min: 0.0244527608156
Time this epoch: 3.301847 seconds
Monitoring step:
	Epochs seen: 39
	Batches seen: 19500
	Examples seen: 1950000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.52631568909
	test_h0_col_norms_mean: 4.35376691818
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.87409830093
	test_h0_row_norms_mean: 3.40977239609
	test_h0_row_norms_min: 0.171177133918
	test_h1_col_norms_max: 6.00363349915
	test_h1_col_norms_mean: 3.89011406898
	test_h1_col_norms_min: 1.72623074055
	test_h1_row_norms_max: 9.06535053253
	test_h1_row_norms_mean: 5.52831077576
	test_h1_row_norms_min: 3.28474617004
	test_objective: 0.158702552319
	test_y_col_norms_max: 6.72936153412
	test_y_col_norms_mean: 6.2109913826
	test_y_col_norms_min: 5.48157644272
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995932340622
	test_y_min_max_class: 0.764656722546
	test_y_misclass: 0.0200999919325
	test_y_nll: 0.158702552319
	test_y_row_norms_max: 1.87921774387
	test_y_row_norms_mean: 0.583407759666
	test_y_row_norms_min: 0.024447273463
	train_h0_col_norms_max: 6.52629041672
	train_h0_col_norms_mean: 4.35376310349
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.87408828735
	train_h0_row_norms_mean: 3.40976953506
	train_h0_row_norms_min: 0.171177104115
	train_h1_col_norms_max: 6.00362253189
	train_h1_col_norms_mean: 3.8901321888
	train_h1_col_norms_min: 1.72622382641
	train_h1_row_norms_max: 9.06535148621
	train_h1_row_norms_mean: 5.52829360962
	train_h1_row_norms_min: 3.28472876549
	train_objective: 0.00152394291945
	train_y_col_norms_max: 6.7293639183
	train_y_col_norms_mean: 6.21102333069
	train_y_col_norms_min: 5.48156309128
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999768614769
	train_y_min_max_class: 0.979155957699
	train_y_misclass: 0.000379999983124
	train_y_nll: 0.00152394291945
	train_y_row_norms_max: 1.87921559811
	train_y_row_norms_mean: 0.583409488201
	train_y_row_norms_min: 0.0244472324848
	valid_h0_col_norms_max: 6.52631568909
	valid_h0_col_norms_mean: 4.35376691818
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.87409830093
	valid_h0_row_norms_mean: 3.40977239609
	valid_h0_row_norms_min: 0.171177133918
	valid_h1_col_norms_max: 6.00363349915
	valid_h1_col_norms_mean: 3.89011406898
	valid_h1_col_norms_min: 1.72623074055
	valid_h1_row_norms_max: 9.06535053253
	valid_h1_row_norms_mean: 5.52831077576
	valid_h1_row_norms_min: 3.28474617004
	valid_objective: 0.17522443831
	valid_y_col_norms_max: 6.72936153412
	valid_y_col_norms_mean: 6.2109913826
	valid_y_col_norms_min: 5.48157644272
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996479153633
	valid_y_min_max_class: 0.788241684437
	valid_y_misclass: 0.0187999941409
	valid_y_nll: 0.17522443831
	valid_y_row_norms_max: 1.87921774387
	valid_y_row_norms_mean: 0.583407759666
	valid_y_row_norms_min: 0.024447273463
Time this epoch: 3.268098 seconds
Monitoring step:
	Epochs seen: 40
	Batches seen: 20000
	Examples seen: 2000000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.53570699692
	test_h0_col_norms_mean: 4.35643339157
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.86570596695
	test_h0_row_norms_mean: 3.41193628311
	test_h0_row_norms_min: 0.171177208424
	test_h1_col_norms_max: 6.00472784042
	test_h1_col_norms_mean: 3.89065885544
	test_h1_col_norms_min: 1.72635400295
	test_h1_row_norms_max: 9.0626745224
	test_h1_row_norms_mean: 5.52905321121
	test_h1_row_norms_min: 3.28488898277
	test_objective: 0.16143476963
	test_y_col_norms_max: 6.73923158646
	test_y_col_norms_mean: 6.22264146805
	test_y_col_norms_min: 5.52369451523
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995911836624
	test_y_min_max_class: 0.785954415798
	test_y_misclass: 0.019199995324
	test_y_nll: 0.16143476963
	test_y_row_norms_max: 1.85353505611
	test_y_row_norms_mean: 0.584432959557
	test_y_row_norms_min: 0.0243270788342
	train_h0_col_norms_max: 6.5357131958
	train_h0_col_norms_mean: 4.35641145706
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.86573553085
	train_h0_row_norms_mean: 3.411921978
	train_h0_row_norms_min: 0.171177133918
	train_h1_col_norms_max: 6.00474691391
	train_h1_col_norms_mean: 3.89064121246
	train_h1_col_norms_min: 1.72634625435
	train_h1_row_norms_max: 9.06272411346
	train_h1_row_norms_mean: 5.52907943726
	train_h1_row_norms_min: 3.28490185738
	train_objective: 0.00306967948563
	train_y_col_norms_max: 6.73919677734
	train_y_col_norms_mean: 6.22266340256
	train_y_col_norms_min: 5.52368307114
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999223351479
	train_y_min_max_class: 0.938300907612
	train_y_misclass: 0.00105999980588
	train_y_nll: 0.00306967948563
	train_y_row_norms_max: 1.85352873802
	train_y_row_norms_mean: 0.584433317184
	train_y_row_norms_min: 0.0243270788342
	valid_h0_col_norms_max: 6.53570699692
	valid_h0_col_norms_mean: 4.35643339157
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.86570596695
	valid_h0_row_norms_mean: 3.41193628311
	valid_h0_row_norms_min: 0.171177208424
	valid_h1_col_norms_max: 6.00472784042
	valid_h1_col_norms_mean: 3.89065885544
	valid_h1_col_norms_min: 1.72635400295
	valid_h1_row_norms_max: 9.0626745224
	valid_h1_row_norms_mean: 5.52905321121
	valid_h1_row_norms_min: 3.28488898277
	valid_objective: 0.182417109609
	valid_y_col_norms_max: 6.73923158646
	valid_y_col_norms_mean: 6.22264146805
	valid_y_col_norms_min: 5.52369451523
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996435403824
	valid_y_min_max_class: 0.793238520622
	valid_y_misclass: 0.0203999951482
	valid_y_nll: 0.182417109609
	valid_y_row_norms_max: 1.85353505611
	valid_y_row_norms_mean: 0.584432959557
	valid_y_row_norms_min: 0.0243270788342
Time this epoch: 3.294892 seconds
Monitoring step:
	Epochs seen: 41
	Batches seen: 20500
	Examples seen: 2050000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.5425863266
	test_h0_col_norms_mean: 4.35914468765
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.87823629379
	test_h0_row_norms_mean: 3.41422724724
	test_h0_row_norms_min: 0.171178132296
	test_h1_col_norms_max: 6.00586032867
	test_h1_col_norms_mean: 3.89139056206
	test_h1_col_norms_min: 1.72638916969
	test_h1_row_norms_max: 9.06592273712
	test_h1_row_norms_mean: 5.53010177612
	test_h1_row_norms_min: 3.28573608398
	test_objective: 0.158061608672
	test_y_col_norms_max: 6.74868965149
	test_y_col_norms_mean: 6.23669672012
	test_y_col_norms_min: 5.50828027725
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995399415493
	test_y_min_max_class: 0.739345610142
	test_y_misclass: 0.0194999948144
	test_y_nll: 0.158061608672
	test_y_row_norms_max: 1.86322903633
	test_y_row_norms_mean: 0.585780024529
	test_y_row_norms_min: 0.0242832899094
	train_h0_col_norms_max: 6.54261350632
	train_h0_col_norms_mean: 4.3591375351
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.87820577621
	train_h0_row_norms_mean: 3.41424489021
	train_h0_row_norms_min: 0.171177610755
	train_h1_col_norms_max: 6.00583934784
	train_h1_col_norms_mean: 3.89137220383
	train_h1_col_norms_min: 1.72638905048
	train_h1_row_norms_max: 9.06593418121
	train_h1_row_norms_mean: 5.53011369705
	train_h1_row_norms_min: 3.28573846817
	train_objective: 0.00130198767874
	train_y_col_norms_max: 6.74868011475
	train_y_col_norms_mean: 6.23671960831
	train_y_col_norms_min: 5.50826644897
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999629914761
	train_y_min_max_class: 0.966445803642
	train_y_misclass: 0.000419999967562
	train_y_nll: 0.00130198767874
	train_y_row_norms_max: 1.86323726177
	train_y_row_norms_mean: 0.585781753063
	train_y_row_norms_min: 0.0242832992226
	valid_h0_col_norms_max: 6.5425863266
	valid_h0_col_norms_mean: 4.35914468765
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.87823629379
	valid_h0_row_norms_mean: 3.41422724724
	valid_h0_row_norms_min: 0.171178132296
	valid_h1_col_norms_max: 6.00586032867
	valid_h1_col_norms_mean: 3.89139056206
	valid_h1_col_norms_min: 1.72638916969
	valid_h1_row_norms_max: 9.06592273712
	valid_h1_row_norms_mean: 5.53010177612
	valid_h1_row_norms_min: 3.28573608398
	valid_objective: 0.168345704675
	valid_y_col_norms_max: 6.74868965149
	valid_y_col_norms_mean: 6.23669672012
	valid_y_col_norms_min: 5.50828027725
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995861887932
	valid_y_min_max_class: 0.767153561115
	valid_y_misclass: 0.0193999931216
	valid_y_nll: 0.168345704675
	valid_y_row_norms_max: 1.86322903633
	valid_y_row_norms_mean: 0.585780024529
	valid_y_row_norms_min: 0.0242832899094
Time this epoch: 3.283051 seconds
Monitoring step:
	Epochs seen: 42
	Batches seen: 21000
	Examples seen: 2100000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.54826259613
	test_h0_col_norms_mean: 4.36148118973
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.87971019745
	test_h0_row_norms_mean: 3.41604399681
	test_h0_row_norms_min: 0.171194016933
	test_h1_col_norms_max: 6.00196123123
	test_h1_col_norms_mean: 3.89196276665
	test_h1_col_norms_min: 1.72636771202
	test_h1_row_norms_max: 9.06931400299
	test_h1_row_norms_mean: 5.53089809418
	test_h1_row_norms_min: 3.28621292114
	test_objective: 0.152915328741
	test_y_col_norms_max: 6.76382827759
	test_y_col_norms_mean: 6.25065279007
	test_y_col_norms_min: 5.53469228745
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995481073856
	test_y_min_max_class: 0.755885243416
	test_y_misclass: 0.0184999946505
	test_y_nll: 0.152915328741
	test_y_row_norms_max: 1.87736725807
	test_y_row_norms_mean: 0.586914539337
	test_y_row_norms_min: 0.0246897321194
	train_h0_col_norms_max: 6.54823541641
	train_h0_col_norms_mean: 4.36147928238
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.87974071503
	train_h0_row_norms_mean: 3.4160592556
	train_h0_row_norms_min: 0.171194061637
	train_h1_col_norms_max: 6.00195074081
	train_h1_col_norms_mean: 3.8919467926
	train_h1_col_norms_min: 1.72637498379
	train_h1_row_norms_max: 9.06928443909
	train_h1_row_norms_mean: 5.53089904785
	train_h1_row_norms_min: 3.28621530533
	train_objective: 0.00141110678669
	train_y_col_norms_max: 6.76386547089
	train_y_col_norms_mean: 6.25062465668
	train_y_col_norms_min: 5.5346660614
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999635100365
	train_y_min_max_class: 0.967330634594
	train_y_misclass: 0.000319999962812
	train_y_nll: 0.00141110678669
	train_y_row_norms_max: 1.87736737728
	train_y_row_norms_mean: 0.58691483736
	train_y_row_norms_min: 0.0246896371245
	valid_h0_col_norms_max: 6.54826259613
	valid_h0_col_norms_mean: 4.36148118973
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.87971019745
	valid_h0_row_norms_mean: 3.41604399681
	valid_h0_row_norms_min: 0.171194016933
	valid_h1_col_norms_max: 6.00196123123
	valid_h1_col_norms_mean: 3.89196276665
	valid_h1_col_norms_min: 1.72636771202
	valid_h1_row_norms_max: 9.06931400299
	valid_h1_row_norms_mean: 5.53089809418
	valid_h1_row_norms_min: 3.28621292114
	valid_objective: 0.164742320776
	valid_y_col_norms_max: 6.76382827759
	valid_y_col_norms_mean: 6.25065279007
	valid_y_col_norms_min: 5.53469228745
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996081888676
	valid_y_min_max_class: 0.769794583321
	valid_y_misclass: 0.0189999956638
	valid_y_nll: 0.164742320776
	valid_y_row_norms_max: 1.87736725807
	valid_y_row_norms_mean: 0.586914539337
	valid_y_row_norms_min: 0.0246897321194
Time this epoch: 3.293110 seconds
Monitoring step:
	Epochs seen: 43
	Batches seen: 21500
	Examples seen: 2150000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.55259990692
	test_h0_col_norms_mean: 4.36336374283
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.8740735054
	test_h0_row_norms_mean: 3.41756176949
	test_h0_row_norms_min: 0.17119500041
	test_h1_col_norms_max: 6.0039639473
	test_h1_col_norms_mean: 3.89240264893
	test_h1_col_norms_min: 1.72636425495
	test_h1_row_norms_max: 9.07901191711
	test_h1_row_norms_mean: 5.53150510788
	test_h1_row_norms_min: 3.28636312485
	test_objective: 0.136897221208
	test_y_col_norms_max: 6.77247095108
	test_y_col_norms_mean: 6.26096439362
	test_y_col_norms_min: 5.51252508163
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995941281319
	test_y_min_max_class: 0.776527881622
	test_y_misclass: 0.0178999938071
	test_y_nll: 0.136897221208
	test_y_row_norms_max: 1.87920343876
	test_y_row_norms_mean: 0.587858736515
	test_y_row_norms_min: 0.0247891973704
	train_h0_col_norms_max: 6.55262708664
	train_h0_col_norms_mean: 4.36338043213
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.8740811348
	train_h0_row_norms_mean: 3.41757678986
	train_h0_row_norms_min: 0.171194568276
	train_h1_col_norms_max: 6.00393533707
	train_h1_col_norms_mean: 3.89241600037
	train_h1_col_norms_min: 1.72637200356
	train_h1_row_norms_max: 9.07898330688
	train_h1_row_norms_mean: 5.53153181076
	train_h1_row_norms_min: 3.28636193275
	train_objective: 0.00148291292135
	train_y_col_norms_max: 6.77250146866
	train_y_col_norms_mean: 6.26093387604
	train_y_col_norms_min: 5.51249742508
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999666690826
	train_y_min_max_class: 0.971082031727
	train_y_misclass: 0.000460000039311
	train_y_nll: 0.00148291292135
	train_y_row_norms_max: 1.87921154499
	train_y_row_norms_mean: 0.58786034584
	train_y_row_norms_min: 0.0247890818864
	valid_h0_col_norms_max: 6.55259990692
	valid_h0_col_norms_mean: 4.36336374283
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.8740735054
	valid_h0_row_norms_mean: 3.41756176949
	valid_h0_row_norms_min: 0.17119500041
	valid_h1_col_norms_max: 6.0039639473
	valid_h1_col_norms_mean: 3.89240264893
	valid_h1_col_norms_min: 1.72636425495
	valid_h1_row_norms_max: 9.07901191711
	valid_h1_row_norms_mean: 5.53150510788
	valid_h1_row_norms_min: 3.28636312485
	valid_objective: 0.161794766784
	valid_y_col_norms_max: 6.77247095108
	valid_y_col_norms_mean: 6.26096439362
	valid_y_col_norms_min: 5.51252508163
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995917260647
	valid_y_min_max_class: 0.753068387508
	valid_y_misclass: 0.0201999936253
	valid_y_nll: 0.161794766784
	valid_y_row_norms_max: 1.87920343876
	valid_y_row_norms_mean: 0.587858736515
	valid_y_row_norms_min: 0.0247891973704
Time this epoch: 3.359274 seconds
Monitoring step:
	Epochs seen: 44
	Batches seen: 22000
	Examples seen: 2200000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.55098342896
	test_h0_col_norms_mean: 4.36544847488
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.87497997284
	test_h0_row_norms_mean: 3.41930341721
	test_h0_row_norms_min: 0.171195015311
	test_h1_col_norms_max: 6.00462388992
	test_h1_col_norms_mean: 3.89291667938
	test_h1_col_norms_min: 1.72640001774
	test_h1_row_norms_max: 9.07387065887
	test_h1_row_norms_mean: 5.53226518631
	test_h1_row_norms_min: 3.28615379333
	test_objective: 0.140558704734
	test_y_col_norms_max: 6.78662919998
	test_y_col_norms_mean: 6.26840209961
	test_y_col_norms_min: 5.52506113052
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.996500074863
	test_y_min_max_class: 0.794377505779
	test_y_misclass: 0.0174999963492
	test_y_nll: 0.140558704734
	test_y_row_norms_max: 1.89163661003
	test_y_row_norms_mean: 0.58866494894
	test_y_row_norms_min: 0.0248291995376
	train_h0_col_norms_max: 6.5510134697
	train_h0_col_norms_mean: 4.36544466019
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.87498617172
	train_h0_row_norms_mean: 3.41928720474
	train_h0_row_norms_min: 0.171194568276
	train_h1_col_norms_max: 6.00459194183
	train_h1_col_norms_mean: 3.89290046692
	train_h1_col_norms_min: 1.72639226913
	train_h1_row_norms_max: 9.07389450073
	train_h1_row_norms_mean: 5.53226518631
	train_h1_row_norms_min: 3.28615093231
	train_objective: 0.000438805494923
	train_y_col_norms_max: 6.7865986824
	train_y_col_norms_mean: 6.2684264183
	train_y_col_norms_min: 5.52509069443
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999836444855
	train_y_min_max_class: 0.985349237919
	train_y_misclass: 0.000159999981406
	train_y_nll: 0.000438805494923
	train_y_row_norms_max: 1.89162778854
	train_y_row_norms_mean: 0.588665127754
	train_y_row_norms_min: 0.0248291157186
	valid_h0_col_norms_max: 6.55098342896
	valid_h0_col_norms_mean: 4.36544847488
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.87497997284
	valid_h0_row_norms_mean: 3.41930341721
	valid_h0_row_norms_min: 0.171195015311
	valid_h1_col_norms_max: 6.00462388992
	valid_h1_col_norms_mean: 3.89291667938
	valid_h1_col_norms_min: 1.72640001774
	valid_h1_row_norms_max: 9.07387065887
	valid_h1_row_norms_mean: 5.53226518631
	valid_h1_row_norms_min: 3.28615379333
	valid_objective: 0.157897502184
	valid_y_col_norms_max: 6.78662919998
	valid_y_col_norms_mean: 6.26840209961
	valid_y_col_norms_min: 5.52506113052
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995646238327
	valid_y_min_max_class: 0.742088675499
	valid_y_misclass: 0.0179999954998
	valid_y_nll: 0.157897502184
	valid_y_row_norms_max: 1.89163661003
	valid_y_row_norms_mean: 0.58866494894
	valid_y_row_norms_min: 0.0248291995376
Time this epoch: 3.258919 seconds
Monitoring step:
	Epochs seen: 45
	Batches seen: 22500
	Examples seen: 2250000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.55773639679
	test_h0_col_norms_mean: 4.36757230759
	test_h0_col_norms_min: 2.23605656624
	test_h0_row_norms_max: 6.88162136078
	test_h0_row_norms_mean: 3.42107534409
	test_h0_row_norms_min: 0.171194553375
	test_h1_col_norms_max: 6.00479459763
	test_h1_col_norms_mean: 3.89360809326
	test_h1_col_norms_min: 1.72638893127
	test_h1_row_norms_max: 9.08829307556
	test_h1_row_norms_mean: 5.53330039978
	test_h1_row_norms_min: 3.28643465042
	test_objective: 0.172753751278
	test_y_col_norms_max: 6.79764652252
	test_y_col_norms_mean: 6.28485965729
	test_y_col_norms_min: 5.55204916
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.99615073204
	test_y_min_max_class: 0.77610886097
	test_y_misclass: 0.020499991253
	test_y_nll: 0.172753751278
	test_y_row_norms_max: 1.87029504776
	test_y_row_norms_mean: 0.590070128441
	test_y_row_norms_min: 0.0248381886631
	train_h0_col_norms_max: 6.55770730972
	train_h0_col_norms_mean: 4.36758470535
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.8816576004
	train_h0_row_norms_mean: 3.4210703373
	train_h0_row_norms_min: 0.171194195747
	train_h1_col_norms_max: 6.00480556488
	train_h1_col_norms_mean: 3.89361214638
	train_h1_col_norms_min: 1.72638893127
	train_h1_row_norms_max: 9.08830928802
	train_h1_row_norms_mean: 5.53328752518
	train_h1_row_norms_min: 3.28645133972
	train_objective: 0.00231568375602
	train_y_col_norms_max: 6.79761791229
	train_y_col_norms_mean: 6.2848906517
	train_y_col_norms_min: 5.55205202103
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999481022358
	train_y_min_max_class: 0.955595433712
	train_y_misclass: 0.000539999979082
	train_y_nll: 0.00231568375602
	train_y_row_norms_max: 1.87028670311
	train_y_row_norms_mean: 0.590068638325
	train_y_row_norms_min: 0.0248383041471
	valid_h0_col_norms_max: 6.55773639679
	valid_h0_col_norms_mean: 4.36757230759
	valid_h0_col_norms_min: 2.23605656624
	valid_h0_row_norms_max: 6.88162136078
	valid_h0_row_norms_mean: 3.42107534409
	valid_h0_row_norms_min: 0.171194553375
	valid_h1_col_norms_max: 6.00479459763
	valid_h1_col_norms_mean: 3.89360809326
	valid_h1_col_norms_min: 1.72638893127
	valid_h1_row_norms_max: 9.08829307556
	valid_h1_row_norms_mean: 5.53330039978
	valid_h1_row_norms_min: 3.28643465042
	valid_objective: 0.17547737062
	valid_y_col_norms_max: 6.79764652252
	valid_y_col_norms_mean: 6.28485965729
	valid_y_col_norms_min: 5.55204916
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995465695858
	valid_y_min_max_class: 0.7330275774
	valid_y_misclass: 0.020799998194
	valid_y_nll: 0.17547737062
	valid_y_row_norms_max: 1.87029504776
	valid_y_row_norms_mean: 0.590070128441
	valid_y_row_norms_min: 0.0248381886631
Time this epoch: 3.263050 seconds
Monitoring step:
	Epochs seen: 46
	Batches seen: 23000
	Examples seen: 2300000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.59422492981
	test_h0_col_norms_mean: 4.37052488327
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.8726644516
	test_h0_row_norms_mean: 3.42359375954
	test_h0_row_norms_min: 0.171194955707
	test_h1_col_norms_max: 6.00406217575
	test_h1_col_norms_mean: 3.89443039894
	test_h1_col_norms_min: 1.72642493248
	test_h1_row_norms_max: 9.08111953735
	test_h1_row_norms_mean: 5.53444480896
	test_h1_row_norms_min: 3.28672647476
	test_objective: 0.176214575768
	test_y_col_norms_max: 6.79807567596
	test_y_col_norms_mean: 6.29843473434
	test_y_col_norms_min: 5.56106996536
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.996127128601
	test_y_min_max_class: 0.767001569271
	test_y_misclass: 0.0188999976963
	test_y_nll: 0.176214575768
	test_y_row_norms_max: 1.88375401497
	test_y_row_norms_mean: 0.591480791569
	test_y_row_norms_min: 0.0244950912893
	train_h0_col_norms_max: 6.59419536591
	train_h0_col_norms_mean: 4.37053442001
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.87265539169
	train_h0_row_norms_mean: 3.42357826233
	train_h0_row_norms_min: 0.171194553375
	train_h1_col_norms_max: 6.00408983231
	train_h1_col_norms_mean: 3.89444637299
	train_h1_col_norms_min: 1.72643446922
	train_h1_row_norms_max: 9.08109664917
	train_h1_row_norms_mean: 5.53442716599
	train_h1_row_norms_min: 3.28672146797
	train_objective: 0.00163910887204
	train_y_col_norms_max: 6.79804325104
	train_y_col_norms_mean: 6.29846715927
	train_y_col_norms_min: 5.56109666824
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999451816082
	train_y_min_max_class: 0.952225148678
	train_y_misclass: 0.00061999988975
	train_y_nll: 0.00163910887204
	train_y_row_norms_max: 1.88374614716
	train_y_row_norms_mean: 0.591483712196
	train_y_row_norms_min: 0.0244949962944
	valid_h0_col_norms_max: 6.59422492981
	valid_h0_col_norms_mean: 4.37052488327
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.8726644516
	valid_h0_row_norms_mean: 3.42359375954
	valid_h0_row_norms_min: 0.171194955707
	valid_h1_col_norms_max: 6.00406217575
	valid_h1_col_norms_mean: 3.89443039894
	valid_h1_col_norms_min: 1.72642493248
	valid_h1_row_norms_max: 9.08111953735
	valid_h1_row_norms_mean: 5.53444480896
	valid_h1_row_norms_min: 3.28672647476
	valid_objective: 0.186354964972
	valid_y_col_norms_max: 6.79807567596
	valid_y_col_norms_mean: 6.29843473434
	valid_y_col_norms_min: 5.56106996536
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.99598556757
	valid_y_min_max_class: 0.759403705597
	valid_y_misclass: 0.0206999909133
	valid_y_nll: 0.186354964972
	valid_y_row_norms_max: 1.88375401497
	valid_y_row_norms_mean: 0.591480791569
	valid_y_row_norms_min: 0.0244950912893
Time this epoch: 3.263870 seconds
Monitoring step:
	Epochs seen: 47
	Batches seen: 23500
	Examples seen: 2350000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.61253595352
	test_h0_col_norms_mean: 4.37271356583
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.87648153305
	test_h0_row_norms_mean: 3.425365448
	test_h0_row_norms_min: 0.17119538784
	test_h1_col_norms_max: 6.00418663025
	test_h1_col_norms_mean: 3.8950676918
	test_h1_col_norms_min: 1.72642803192
	test_h1_row_norms_max: 9.10107326508
	test_h1_row_norms_mean: 5.53528594971
	test_h1_row_norms_min: 3.28674340248
	test_objective: 0.160995185375
	test_y_col_norms_max: 6.79669380188
	test_y_col_norms_mean: 6.31103897095
	test_y_col_norms_min: 5.58734273911
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995669007301
	test_y_min_max_class: 0.77006238699
	test_y_misclass: 0.0187999941409
	test_y_nll: 0.160995185375
	test_y_row_norms_max: 1.89035248756
	test_y_row_norms_mean: 0.592762053013
	test_y_row_norms_min: 0.0258602239192
	train_h0_col_norms_max: 6.61253833771
	train_h0_col_norms_mean: 4.37270545959
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.87647247314
	train_h0_row_norms_mean: 3.42536139488
	train_h0_row_norms_min: 0.171194672585
	train_h1_col_norms_max: 6.00415945053
	train_h1_col_norms_mean: 3.89505052567
	train_h1_col_norms_min: 1.72643530369
	train_h1_row_norms_max: 9.10108375549
	train_h1_row_norms_mean: 5.53529548645
	train_h1_row_norms_min: 3.28672790527
	train_objective: 0.0021845579613
	train_y_col_norms_max: 6.79666471481
	train_y_col_norms_mean: 6.31101417542
	train_y_col_norms_min: 5.58734083176
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999446511269
	train_y_min_max_class: 0.954655766487
	train_y_misclass: 0.000679999880958
	train_y_nll: 0.0021845579613
	train_y_row_norms_max: 1.89036035538
	train_y_row_norms_mean: 0.59276509285
	train_y_row_norms_min: 0.0258601009846
	valid_h0_col_norms_max: 6.61253595352
	valid_h0_col_norms_mean: 4.37271356583
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.87648153305
	valid_h0_row_norms_mean: 3.425365448
	valid_h0_row_norms_min: 0.17119538784
	valid_h1_col_norms_max: 6.00418663025
	valid_h1_col_norms_mean: 3.8950676918
	valid_h1_col_norms_min: 1.72642803192
	valid_h1_row_norms_max: 9.10107326508
	valid_h1_row_norms_mean: 5.53528594971
	valid_h1_row_norms_min: 3.28674340248
	valid_objective: 0.158408492804
	valid_y_col_norms_max: 6.79669380188
	valid_y_col_norms_mean: 6.31103897095
	valid_y_col_norms_min: 5.58734273911
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995150506496
	valid_y_min_max_class: 0.737409770489
	valid_y_misclass: 0.0196999944746
	valid_y_nll: 0.158408492804
	valid_y_row_norms_max: 1.89035248756
	valid_y_row_norms_mean: 0.592762053013
	valid_y_row_norms_min: 0.0258602239192
Time this epoch: 3.246123 seconds
Monitoring step:
	Epochs seen: 48
	Batches seen: 24000
	Examples seen: 2400000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.60942840576
	test_h0_col_norms_mean: 4.37416505814
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.88673448563
	test_h0_row_norms_mean: 3.42650437355
	test_h0_row_norms_min: 0.171197414398
	test_h1_col_norms_max: 6.00638771057
	test_h1_col_norms_mean: 3.89544963837
	test_h1_col_norms_min: 1.72642791271
	test_h1_row_norms_max: 9.09871959686
	test_h1_row_norms_mean: 5.53591918945
	test_h1_row_norms_min: 3.2867603302
	test_objective: 0.175691723824
	test_y_col_norms_max: 6.78237819672
	test_y_col_norms_mean: 6.3183298111
	test_y_col_norms_min: 5.59972047806
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.996128737926
	test_y_min_max_class: 0.775415062904
	test_y_misclass: 0.0199999921024
	test_y_nll: 0.175691723824
	test_y_row_norms_max: 1.88972866535
	test_y_row_norms_mean: 0.593335032463
	test_y_row_norms_min: 0.0257790517062
	train_h0_col_norms_max: 6.60943603516
	train_h0_col_norms_mean: 4.37415838242
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.88672494888
	train_h0_row_norms_mean: 3.4265191555
	train_h0_row_norms_min: 0.171197369695
	train_h1_col_norms_max: 6.00641536713
	train_h1_col_norms_mean: 3.8954308033
	train_h1_col_norms_min: 1.72643482685
	train_h1_row_norms_max: 9.09872722626
	train_h1_row_norms_mean: 5.53590679169
	train_h1_row_norms_min: 3.28674817085
	train_objective: 0.000762883864809
	train_y_col_norms_max: 6.78235006332
	train_y_col_norms_mean: 6.31833028793
	train_y_col_norms_min: 5.59973287582
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.99974834919
	train_y_min_max_class: 0.978080093861
	train_y_misclass: 0.000239999950281
	train_y_nll: 0.000762883864809
	train_y_row_norms_max: 1.88971841335
	train_y_row_norms_mean: 0.593334615231
	train_y_row_norms_min: 0.0257790144533
	valid_h0_col_norms_max: 6.60942840576
	valid_h0_col_norms_mean: 4.37416505814
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.88673448563
	valid_h0_row_norms_mean: 3.42650437355
	valid_h0_row_norms_min: 0.171197414398
	valid_h1_col_norms_max: 6.00638771057
	valid_h1_col_norms_mean: 3.89544963837
	valid_h1_col_norms_min: 1.72642791271
	valid_h1_row_norms_max: 9.09871959686
	valid_h1_row_norms_mean: 5.53591918945
	valid_h1_row_norms_min: 3.2867603302
	valid_objective: 0.178655579686
	valid_y_col_norms_max: 6.78237819672
	valid_y_col_norms_mean: 6.3183298111
	valid_y_col_norms_min: 5.59972047806
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996117174625
	valid_y_min_max_class: 0.773514211178
	valid_y_misclass: 0.0190999954939
	valid_y_nll: 0.178655579686
	valid_y_row_norms_max: 1.88972866535
	valid_y_row_norms_mean: 0.593335032463
	valid_y_row_norms_min: 0.0257790517062
Time this epoch: 3.274107 seconds
Monitoring step:
	Epochs seen: 49
	Batches seen: 24500
	Examples seen: 2450000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.61158514023
	test_h0_col_norms_mean: 4.37596178055
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.89095163345
	test_h0_row_norms_mean: 3.42802858353
	test_h0_row_norms_min: 0.171208888292
	test_h1_col_norms_max: 6.00729322433
	test_h1_col_norms_mean: 3.89590859413
	test_h1_col_norms_min: 1.72634100914
	test_h1_row_norms_max: 9.11584568024
	test_h1_row_norms_mean: 5.53655290604
	test_h1_row_norms_min: 3.28674292564
	test_objective: 0.173922881484
	test_y_col_norms_max: 6.80417919159
	test_y_col_norms_mean: 6.32538461685
	test_y_col_norms_min: 5.59382343292
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.995864152908
	test_y_min_max_class: 0.774653494358
	test_y_misclass: 0.0195999965072
	test_y_nll: 0.173922881484
	test_y_row_norms_max: 1.90011572838
	test_y_row_norms_mean: 0.593958258629
	test_y_row_norms_min: 0.0259114392102
	train_h0_col_norms_max: 6.61158514023
	train_h0_col_norms_mean: 4.37594175339
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.89095973969
	train_h0_row_norms_mean: 3.42801046371
	train_h0_row_norms_min: 0.171208947897
	train_h1_col_norms_max: 6.00726985931
	train_h1_col_norms_mean: 3.89590215683
	train_h1_col_norms_min: 1.7263327837
	train_h1_row_norms_max: 9.11585617065
	train_h1_row_norms_mean: 5.53655576706
	train_h1_row_norms_min: 3.28672790527
	train_objective: 0.00186892366037
	train_y_col_norms_max: 6.80421066284
	train_y_col_norms_mean: 6.32541131973
	train_y_col_norms_min: 5.59379386902
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999438583851
	train_y_min_max_class: 0.950516223907
	train_y_misclass: 0.000539999979082
	train_y_nll: 0.00186892366037
	train_y_row_norms_max: 1.90012443066
	train_y_row_norms_mean: 0.59395968914
	train_y_row_norms_min: 0.025911314413
	valid_h0_col_norms_max: 6.61158514023
	valid_h0_col_norms_mean: 4.37596178055
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.89095163345
	valid_h0_row_norms_mean: 3.42802858353
	valid_h0_row_norms_min: 0.171208888292
	valid_h1_col_norms_max: 6.00729322433
	valid_h1_col_norms_mean: 3.89590859413
	valid_h1_col_norms_min: 1.72634100914
	valid_h1_row_norms_max: 9.11584568024
	valid_h1_row_norms_mean: 5.53655290604
	valid_h1_row_norms_min: 3.28674292564
	valid_objective: 0.167324125767
	valid_y_col_norms_max: 6.80417919159
	valid_y_col_norms_mean: 6.32538461685
	valid_y_col_norms_min: 5.59382343292
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.995694279671
	valid_y_min_max_class: 0.751230061054
	valid_y_misclass: 0.0211999900639
	valid_y_nll: 0.167324125767
	valid_y_row_norms_max: 1.90011572838
	valid_y_row_norms_mean: 0.593958258629
	valid_y_row_norms_min: 0.0259114392102
Time this epoch: 3.276921 seconds
Monitoring step:
	Epochs seen: 50
	Batches seen: 25000
	Examples seen: 2500000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.61482906342
	test_h0_col_norms_mean: 4.37731599808
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.90527057648
	test_h0_row_norms_mean: 3.42910242081
	test_h0_row_norms_min: 0.171211406589
	test_h1_col_norms_max: 6.01254796982
	test_h1_col_norms_mean: 3.89633321762
	test_h1_col_norms_min: 1.72635316849
	test_h1_row_norms_max: 9.1068277359
	test_h1_row_norms_mean: 5.53717756271
	test_h1_row_norms_min: 3.28689336777
	test_objective: 0.178737580776
	test_y_col_norms_max: 6.81128787994
	test_y_col_norms_mean: 6.33507156372
	test_y_col_norms_min: 5.58309650421
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.99630522728
	test_y_min_max_class: 0.788845181465
	test_y_misclass: 0.0197999924421
	test_y_nll: 0.178737580776
	test_y_row_norms_max: 1.93474268913
	test_y_row_norms_mean: 0.594771564007
	test_y_row_norms_min: 0.0260054916143
	train_h0_col_norms_max: 6.6148557663
	train_h0_col_norms_mean: 4.37733840942
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.90530347824
	train_h0_row_norms_mean: 3.42908787727
	train_h0_row_norms_min: 0.171211794019
	train_h1_col_norms_max: 6.01251840591
	train_h1_col_norms_mean: 3.89634943008
	train_h1_col_norms_min: 1.72634553909
	train_h1_row_norms_max: 9.10683345795
	train_h1_row_norms_mean: 5.53719091415
	train_h1_row_norms_min: 3.28687477112
	train_objective: 0.00155572697986
	train_y_col_norms_max: 6.81132364273
	train_y_col_norms_mean: 6.33503913879
	train_y_col_norms_min: 5.58306837082
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999542534351
	train_y_min_max_class: 0.959059894085
	train_y_misclass: 0.000539999920875
	train_y_nll: 0.00155572697986
	train_y_row_norms_max: 1.93475210667
	train_y_row_norms_mean: 0.594768106937
	train_y_row_norms_min: 0.0260053742677
	valid_h0_col_norms_max: 6.61482906342
	valid_h0_col_norms_mean: 4.37731599808
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.90527057648
	valid_h0_row_norms_mean: 3.42910242081
	valid_h0_row_norms_min: 0.171211406589
	valid_h1_col_norms_max: 6.01254796982
	valid_h1_col_norms_mean: 3.89633321762
	valid_h1_col_norms_min: 1.72635316849
	valid_h1_row_norms_max: 9.1068277359
	valid_h1_row_norms_mean: 5.53717756271
	valid_h1_row_norms_min: 3.28689336777
	valid_objective: 0.168507456779
	valid_y_col_norms_max: 6.81128787994
	valid_y_col_norms_mean: 6.33507156372
	valid_y_col_norms_min: 5.58309650421
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996147751808
	valid_y_min_max_class: 0.789148688316
	valid_y_misclass: 0.0197999924421
	valid_y_nll: 0.168507456779
	valid_y_row_norms_max: 1.93474268913
	valid_y_row_norms_mean: 0.594771564007
	valid_y_row_norms_min: 0.0260054916143
Time this epoch: 3.200028 seconds
Monitoring step:
	Epochs seen: 51
	Batches seen: 25500
	Examples seen: 2550000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.6147813797
	test_h0_col_norms_mean: 4.37828779221
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.90187358856
	test_h0_row_norms_mean: 3.42986750603
	test_h0_row_norms_min: 0.171211406589
	test_h1_col_norms_max: 6.0115852356
	test_h1_col_norms_mean: 3.89659976959
	test_h1_col_norms_min: 1.72635293007
	test_h1_row_norms_max: 9.1057882309
	test_h1_row_norms_mean: 5.53757143021
	test_h1_row_norms_min: 3.28781795502
	test_objective: 0.172010108829
	test_y_col_norms_max: 6.82110786438
	test_y_col_norms_mean: 6.33999061584
	test_y_col_norms_min: 5.59692811966
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.996686458588
	test_y_min_max_class: 0.808368086815
	test_y_misclass: 0.0198999941349
	test_y_nll: 0.172010108829
	test_y_row_norms_max: 1.93597054482
	test_y_row_norms_mean: 0.595229923725
	test_y_row_norms_min: 0.0260351337492
	train_h0_col_norms_max: 6.61475372314
	train_h0_col_norms_mean: 4.37829828262
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.9019112587
	train_h0_row_norms_mean: 3.42988300323
	train_h0_row_norms_min: 0.171211794019
	train_h1_col_norms_max: 6.01156425476
	train_h1_col_norms_mean: 3.89659571648
	train_h1_col_norms_min: 1.72634553909
	train_h1_row_norms_max: 9.10573482513
	train_h1_row_norms_mean: 5.53757476807
	train_h1_row_norms_min: 3.28780126572
	train_objective: 0.00100987718906
	train_y_col_norms_max: 6.8211388588
	train_y_col_norms_mean: 6.34001255035
	train_y_col_norms_min: 5.59689760208
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999795496464
	train_y_min_max_class: 0.98121035099
	train_y_misclass: 0.000219999958063
	train_y_nll: 0.00100987718906
	train_y_row_norms_max: 1.93596208096
	train_y_row_norms_mean: 0.59522998333
	train_y_row_norms_min: 0.0260351262987
	valid_h0_col_norms_max: 6.6147813797
	valid_h0_col_norms_mean: 4.37828779221
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.90187358856
	valid_h0_row_norms_mean: 3.42986750603
	valid_h0_row_norms_min: 0.171211406589
	valid_h1_col_norms_max: 6.0115852356
	valid_h1_col_norms_mean: 3.89659976959
	valid_h1_col_norms_min: 1.72635293007
	valid_h1_row_norms_max: 9.1057882309
	valid_h1_row_norms_mean: 5.53757143021
	valid_h1_row_norms_min: 3.28781795502
	valid_objective: 0.175070494413
	valid_y_col_norms_max: 6.82110786438
	valid_y_col_norms_mean: 6.33999061584
	valid_y_col_norms_min: 5.59692811966
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996282696724
	valid_y_min_max_class: 0.780044913292
	valid_y_misclass: 0.0196999944746
	valid_y_nll: 0.175070494413
	valid_y_row_norms_max: 1.93597054482
	valid_y_row_norms_mean: 0.595229923725
	valid_y_row_norms_min: 0.0260351337492
Time this epoch: 3.239240 seconds
Monitoring step:
	Epochs seen: 52
	Batches seen: 26000
	Examples seen: 2600000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.61600589752
	test_h0_col_norms_mean: 4.37898588181
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.90484142303
	test_h0_row_norms_mean: 3.43044447899
	test_h0_row_norms_min: 0.171211466193
	test_h1_col_norms_max: 6.01123189926
	test_h1_col_norms_mean: 3.8968091011
	test_h1_col_norms_min: 1.72635233402
	test_h1_row_norms_max: 9.11070537567
	test_h1_row_norms_mean: 5.53788709641
	test_h1_row_norms_min: 3.28822088242
	test_objective: 0.160425424576
	test_y_col_norms_max: 6.80656099319
	test_y_col_norms_mean: 6.34523868561
	test_y_col_norms_min: 5.59447908401
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.996790707111
	test_y_min_max_class: 0.826726913452
	test_y_misclass: 0.0184999965131
	test_y_nll: 0.160425424576
	test_y_row_norms_max: 1.94036662579
	test_y_row_norms_mean: 0.595700562
	test_y_row_norms_min: 0.0263105537742
	train_h0_col_norms_max: 6.61604356766
	train_h0_col_norms_mean: 4.37900781631
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.9048409462
	train_h0_row_norms_mean: 3.43045806885
	train_h0_row_norms_min: 0.171212136745
	train_h1_col_norms_max: 6.01124334335
	train_h1_col_norms_mean: 3.89682626724
	train_h1_col_norms_min: 1.72634339333
	train_h1_row_norms_max: 9.11072158813
	train_h1_row_norms_mean: 5.53790187836
	train_h1_row_norms_min: 3.28823471069
	train_objective: 0.000158851937158
	train_y_col_norms_max: 6.8065943718
	train_y_col_norms_mean: 6.34526729584
	train_y_col_norms_min: 5.59449052811
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999920845032
	train_y_min_max_class: 0.992827177048
	train_y_misclass: 7.9999997979e-05
	train_y_nll: 0.000158851937158
	train_y_row_norms_max: 1.94036877155
	train_y_row_norms_mean: 0.595700562
	train_y_row_norms_min: 0.0263105835766
	valid_h0_col_norms_max: 6.61600589752
	valid_h0_col_norms_mean: 4.37898588181
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.90484142303
	valid_h0_row_norms_mean: 3.43044447899
	valid_h0_row_norms_min: 0.171211466193
	valid_h1_col_norms_max: 6.01123189926
	valid_h1_col_norms_mean: 3.8968091011
	valid_h1_col_norms_min: 1.72635233402
	valid_h1_row_norms_max: 9.11070537567
	valid_h1_row_norms_mean: 5.53788709641
	valid_h1_row_norms_min: 3.28822088242
	valid_objective: 0.169489264488
	valid_y_col_norms_max: 6.80656099319
	valid_y_col_norms_mean: 6.34523868561
	valid_y_col_norms_min: 5.59447908401
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996821761131
	valid_y_min_max_class: 0.813496589661
	valid_y_misclass: 0.0197999961674
	valid_y_nll: 0.169489264488
	valid_y_row_norms_max: 1.94036662579
	valid_y_row_norms_mean: 0.595700562
	valid_y_row_norms_min: 0.0263105537742
Time this epoch: 3.259741 seconds
Monitoring step:
	Epochs seen: 53
	Batches seen: 26500
	Examples seen: 2650000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.6160197258
	test_h0_col_norms_mean: 4.3792681694
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.90263414383
	test_h0_row_norms_mean: 3.43065404892
	test_h0_row_norms_min: 0.171211466193
	test_h1_col_norms_max: 6.01123523712
	test_h1_col_norms_mean: 3.89689803123
	test_h1_col_norms_min: 1.72635245323
	test_h1_row_norms_max: 9.11182498932
	test_h1_row_norms_mean: 5.53798723221
	test_h1_row_norms_min: 3.2883348465
	test_objective: 0.151598215103
	test_y_col_norms_max: 6.82387685776
	test_y_col_norms_mean: 6.34804153442
	test_y_col_norms_min: 5.58777189255
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.99653673172
	test_y_min_max_class: 0.809294104576
	test_y_misclass: 0.0181999951601
	test_y_nll: 0.151598215103
	test_y_row_norms_max: 1.94308698177
	test_y_row_norms_mean: 0.595957636833
	test_y_row_norms_min: 0.0262990482152
	train_h0_col_norms_max: 6.61604738235
	train_h0_col_norms_mean: 4.3792719841
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.9026427269
	train_h0_row_norms_mean: 3.43063807487
	train_h0_row_norms_min: 0.171212136745
	train_h1_col_norms_max: 6.01124382019
	train_h1_col_norms_mean: 3.89692115784
	train_h1_col_norms_min: 1.72634339333
	train_h1_row_norms_max: 9.11186790466
	train_h1_row_norms_mean: 5.53798723221
	train_h1_row_norms_min: 3.28835225105
	train_objective: 0.000210020065424
	train_y_col_norms_max: 6.82384681702
	train_y_col_norms_mean: 6.348072052
	train_y_col_norms_min: 5.58779907227
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999953627586
	train_y_min_max_class: 0.996000170708
	train_y_misclass: 5.99999984843e-05
	train_y_nll: 0.000210020065424
	train_y_row_norms_max: 1.94309675694
	train_y_row_norms_mean: 0.595957219601
	train_y_row_norms_min: 0.0262989122421
	valid_h0_col_norms_max: 6.6160197258
	valid_h0_col_norms_mean: 4.3792681694
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.90263414383
	valid_h0_row_norms_mean: 3.43065404892
	valid_h0_row_norms_min: 0.171211466193
	valid_h1_col_norms_max: 6.01123523712
	valid_h1_col_norms_mean: 3.89689803123
	valid_h1_col_norms_min: 1.72635245323
	valid_h1_row_norms_max: 9.11182498932
	valid_h1_row_norms_mean: 5.53798723221
	valid_h1_row_norms_min: 3.2883348465
	valid_objective: 0.163225889206
	valid_y_col_norms_max: 6.82387685776
	valid_y_col_norms_mean: 6.34804153442
	valid_y_col_norms_min: 5.58777189255
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.996884763241
	valid_y_min_max_class: 0.818299531937
	valid_y_misclass: 0.0186999943107
	valid_y_nll: 0.163225889206
	valid_y_row_norms_max: 1.94308698177
	valid_y_row_norms_mean: 0.595957636833
	valid_y_row_norms_min: 0.0262990482152
Time this epoch: 3.224364 seconds
Monitoring step:
	Epochs seen: 54
	Batches seen: 27000
	Examples seen: 2700000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 6.61593580246
	test_h0_col_norms_mean: 4.37931966782
	test_h0_col_norms_min: 2.23605632782
	test_h0_row_norms_max: 6.90164899826
	test_h0_row_norms_mean: 3.43070554733
	test_h0_row_norms_min: 0.171211466193
	test_h1_col_norms_max: 6.01123189926
	test_h1_col_norms_mean: 3.89690995216
	test_h1_col_norms_min: 1.72635293007
	test_h1_row_norms_max: 9.10886287689
	test_h1_row_norms_mean: 5.53801727295
	test_h1_row_norms_min: 3.28835773468
	test_objective: 0.156616300344
	test_y_col_norms_max: 6.82711935043
	test_y_col_norms_mean: 6.34913825989
	test_y_col_norms_min: 5.58710432053
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.996679246426
	test_y_min_max_class: 0.821564376354
	test_y_misclass: 0.0183999948204
	test_y_nll: 0.156616300344
	test_y_row_norms_max: 1.94326412678
	test_y_row_norms_mean: 0.596061944962
	test_y_row_norms_min: 0.0262778773904
	train_h0_col_norms_max: 6.61593151093
	train_h0_col_norms_mean: 4.37929821014
	train_h0_col_norms_min: 2.23605895042
	train_h0_row_norms_max: 6.90168523788
	train_h0_row_norms_mean: 3.43071746826
	train_h0_row_norms_min: 0.171212136745
	train_h1_col_norms_max: 6.01124286652
	train_h1_col_norms_mean: 3.89692831039
	train_h1_col_norms_min: 1.72634553909
	train_h1_row_norms_max: 9.10885238647
	train_h1_row_norms_mean: 5.53799200058
	train_h1_row_norms_min: 3.28836083412
	train_objective: 1.18671387099e-05
	train_y_col_norms_max: 6.82711696625
	train_y_col_norms_mean: 6.34910583496
	train_y_col_norms_min: 5.5871014595
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.999985456467
	train_y_min_max_class: 0.999015629292
	train_y_misclass: 0.0
	train_y_nll: 1.18671387099e-05
	train_y_row_norms_max: 1.94327509403
	train_y_row_norms_mean: 0.596063792706
	train_y_row_norms_min: 0.0262779761106
	valid_h0_col_norms_max: 6.61593580246
	valid_h0_col_norms_mean: 4.37931966782
	valid_h0_col_norms_min: 2.23605632782
	valid_h0_row_norms_max: 6.90164899826
	valid_h0_row_norms_mean: 3.43070554733
	valid_h0_row_norms_min: 0.171211466193
	valid_h1_col_norms_max: 6.01123189926
	valid_h1_col_norms_mean: 3.89690995216
	valid_h1_col_norms_min: 1.72635293007
	valid_h1_row_norms_max: 9.10886287689
	valid_h1_row_norms_mean: 5.53801727295
	valid_h1_row_norms_min: 3.28835773468
	valid_objective: 0.168142601848
	valid_y_col_norms_max: 6.82711935043
	valid_y_col_norms_mean: 6.34913825989
	valid_y_col_norms_min: 5.58710432053
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.997302770615
	valid_y_min_max_class: 0.829375386238
	valid_y_misclass: 0.0184999927878
	valid_y_nll: 0.168142601848
	valid_y_row_norms_max: 1.94326412678
	valid_y_row_norms_mean: 0.596061944962
	valid_y_row_norms_min: 0.0262778773904



In [8]:

    
!print_monitor.py mlp_2_best.pkl | grep test_y_misclass









    



Using gpu device 2: GeForce GTX 285
/u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit.
  warnings.warn("MLP changing the recursion limit.")
test_y_misclass : 0.0174999963492

Using the deeper architecture, rectifier units, and SGD brought the test error rate down from 1.94% to 1.75%.

Part 4: Regularization and pylearn2 costs

In softmax_regression.ipynb, we discussed the problem of overfitting, and how early stopping guided by validation set performance can result in better test set performance. Another way to prevent overfitting is to explicitly change the cost function to discourage overfitting.

The best way to prevent overfitting is to use Bayesian inference to predict labels on the new data. Suppose we have been given a dataset $\mathcal{D}$, and we want to classify a new point $x'$. Call its uknown label $y'$. Suppose that we also have a probability distribution over all possible model parameters, and that we call the set of all parameters $\theta$. Then

$$p(y' \mid x', \mathcal{D} ) = \int p(y', \theta \mid x', \mathcal{D}) d \theta $$$$ = \int p( y' \mid x' , \theta ) p( \theta \mid \mathcal{D} ) d \theta $$$$ \propto \int p( y' \mid x' , \theta ) p( \mathcal{D} \mid \theta ) p(\theta) d \theta $$

(On the last line, we only worry about computing the distribution over $y'$ up to a constant, because we can easily find this constant by summing over the $k$ possible values of $y'$)

In other words, the right thing to do is to have all of the infinitely many possible values of $\theta$ vote on how to classify $x'$, with each value of $\theta$'s vote weighted by $p(\theta) p(\mathcal{D} \mid \theta)$.

Unfortunately, while conceptually straight forward, there is not an obvious way to evaluate this integral for a large multilayer perceptron. Instead, we assume that the distribution $p(\theta) p(\mathcal{D} \mid \theta)$ is very peaked, so that we can get a good prediction by using the single most likely value of $\theta$.

This suggests that we should maximize $p(\theta) p(\mathcal{D} \mid \theta)$, rather than maximizing $p(\mathcal{D} \mid \theta)$ as we have so far. Note that in log space, this is $\log p(\theta) + \log p( \mathcal{D} \mid \theta)$. We can thus add regularization to our training procedure by adding a term for $\log p(\theta)$ to our objective function.

This is very easy to do in pylearn2 using the SumOfCosts class. The following YAML string sets up the same experiment as before, but using SumOfCosts to add a regularization term. Before, we did not specify the "cost" argument to the training algorithm. The model provided the training algorithm with a default cost. Now, we specify that the cost should be the sum of two different costs. The first is the Default cost, which just asks the output layer what cost to use. This is the same cost we have implicitly been using all along, because models.mlp.MLP.get_default_cost() returns costs.mlp.Default(). The second term of our new cost function is called WeightDecay, and it implements a prior on our model parameters $\theta$.



In [9]:

    
import os
import pylearn2
path = os.path.join(pylearn2.__path__[0], 'scripts', 'tutorials', 'multilayer_perceptron', 'mlp_tutorial_part_4.yaml')
with open(path, 'r') as f:
    train_3 = f.read()
hyper_params = {'train_stop' : 50000,
                'valid_stop' : 60000,
                'dim_h0' : 500,
                'dim_h1' : 1000,
                'sparse_init_h1' : 15,
                'max_epochs' : 10000,
                'save_path' : '.'}
train_3 = train_3 % (hyper_params)
print train_3









    



!obj:pylearn2.train.Train {
    dataset: &train !obj:pylearn2.datasets.mnist.MNIST {
        which_set: 'train',
        start: 0,
        stop: 50000
    },
    model: !obj:pylearn2.models.mlp.MLP {
        layers: [ !obj:pylearn2.models.mlp.RectifiedLinear {
                     layer_name: 'h0',
                     dim: 500,
                     sparse_init: 15
                 },  !obj:pylearn2.models.mlp.RectifiedLinear {
                     layer_name: 'h1',
                     dim: 500,
                     sparse_init: 15
                 }, !obj:pylearn2.models.mlp.Softmax {
                     layer_name: 'y',
                     n_classes: 10,
                     irange: 0.
                 }
                ],
        nvis: 784,
    },
    algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
        batch_size: 100,
        learning_rate: .01,
        monitoring_dataset:
            {
                'train' : *train,
                'valid' : !obj:pylearn2.datasets.mnist.MNIST {
                              which_set: 'train',
                              start: 50000,
                              stop: 60000
                          },
                'test'  : !obj:pylearn2.datasets.mnist.MNIST {
                              which_set: 'test',
                          }
            },
        cost: !obj:pylearn2.costs.cost.SumOfCosts { costs: [
            !obj:pylearn2.costs.mlp.Default {
            }, !obj:pylearn2.costs.mlp.WeightDecay {
                coeffs: [ .00005, .00005, .00005 ]
            }
            ]
        },
        learning_rule: !obj:pylearn2.training_algorithms.learning_rule.Momentum {
            init_momentum: .5
        },
        termination_criterion: !obj:pylearn2.termination_criteria.And {
            criteria: [
                !obj:pylearn2.termination_criteria.MonitorBased {
                    channel_name: "valid_y_misclass",
                    prop_decrease: 0.,
                    N: 10
                },
                !obj:pylearn2.termination_criteria.EpochCounter {
                    max_epochs: 10000
                }
            ]
        }
    },
    extensions: [
        !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest {
             channel_name: 'valid_y_misclass',
             save_path: "mlp_3_best.pkl"
        }, !obj:pylearn2.training_algorithms.learning_rule.MomentumAdjustor {
            start: 1,
            saturate: 10,
            final_momentum: .99
        }
    ]
}

The WeightDecay class adds a cost based on the sum of the squares of the elements of $W$ for the different layers, multiplying each by a different coefficient. This corresponds to $p(\theta)$ being Gaussian distribution on $W$, with a diagonal covariance matrix. (We don't regularize $b$, which is a bit of a hack, but can be thought of as putting extremely high variance on $b$ in the prior) In other words, our prior belief about $\theta$ is that the weights should be small. This basically says that, all else being equal, the different units in our network shouldn't interact with each other. Compared to the unregularized network, a network trained with weight decay wants to see more evidence that two units should interact before it allows them to do so.

Note that the SumOfCosts class doesn't explicitly have anything to do with the MLP. There is no requirement that the cost function be closely tied to the code for a particular model in pylearn2. This gives you great flexibility in the kind of experiments pylearn2 can run. The SumOfCosts class allows you to combine several pre-existing building blocks in pylearn2. By implementing your own cost classes, you can get even greater flexibility.

Of course, some costs are tightly integrated with a specific kind of model. The costs.mlp.Default cost expects to be able to ask a model for its last layer, and ask that layer what kind of cost to apply to the target values $y$ and an estimate of them produced by calling the model's fprop method. This implies that the cost can really only be used with MLP subclasses. Likewise, the WeightDecay cost depends on the assumption that the model is organized into layers and each layer has a single weight matrix. This means that it can only be used with an MLP,and even then only with layers that are governed by a weight matrix. It's OK to make a Cost that is this tightly integrated with a specific kind of model. Doing so is inevitable. Usually in pylearn2 we put the costs for a specific model family in their own submodule of pylearn2 so it's easy to tell what models they can be used with.

We now show what happens when you train the regularized MLP:



In [10]:

    
from pylearn2.config import yaml_parse
train_3 = yaml_parse.load(train_3)
train_3.main_loop()









    



Parameter and initial learning rate summary:
	h0_W: 0.00999999977648
	h0_b: 0.00999999977648
	h1_W: 0.00999999977648
	h1_b: 0.00999999977648
	softmax_b: 0.00999999977648
	softmax_W: 0.00999999977648
Compiling sgd_update...
Compiling sgd_update done. Time elapsed: 2.973035 seconds
compiling begin_record_entry...
compiling begin_record_entry done. Time elapsed: 0.457965 seconds
Monitored channels: 
	learning_rate
	momentum
	test_h0_col_norms_max
	test_h0_col_norms_mean
	test_h0_col_norms_min
	test_h0_row_norms_max
	test_h0_row_norms_mean
	test_h0_row_norms_min
	test_h1_col_norms_max
	test_h1_col_norms_mean
	test_h1_col_norms_min
	test_h1_row_norms_max
	test_h1_row_norms_mean
	test_h1_row_norms_min
	test_objective
	test_term_0
	test_term_1_weight_decay
	test_y_col_norms_max
	test_y_col_norms_mean
	test_y_col_norms_min
	test_y_max_max_class
	test_y_mean_max_class
	test_y_min_max_class
	test_y_misclass
	test_y_nll
	test_y_row_norms_max
	test_y_row_norms_mean
	test_y_row_norms_min
	train_h0_col_norms_max
	train_h0_col_norms_mean
	train_h0_col_norms_min
	train_h0_row_norms_max
	train_h0_row_norms_mean
	train_h0_row_norms_min
	train_h1_col_norms_max
	train_h1_col_norms_mean
	train_h1_col_norms_min
	train_h1_row_norms_max
	train_h1_row_norms_mean
	train_h1_row_norms_min
	train_objective
	train_term_0
	train_term_1_weight_decay
	train_y_col_norms_max
	train_y_col_norms_mean
	train_y_col_norms_min
	train_y_max_max_class
	train_y_mean_max_class
	train_y_min_max_class
	train_y_misclass
	train_y_nll
	train_y_row_norms_max
	train_y_row_norms_mean
	train_y_row_norms_min
	valid_h0_col_norms_max
	valid_h0_col_norms_mean
	valid_h0_col_norms_min
	valid_h0_row_norms_max
	valid_h0_row_norms_mean
	valid_h0_row_norms_min
	valid_h1_col_norms_max
	valid_h1_col_norms_mean
	valid_h1_col_norms_min
	valid_h1_row_norms_max
	valid_h1_row_norms_mean
	valid_h1_row_norms_min
	valid_objective
	valid_term_0
	valid_term_1_weight_decay
	valid_y_col_norms_max
	valid_y_col_norms_mean
	valid_y_col_norms_min
	valid_y_max_max_class
	valid_y_mean_max_class
	valid_y_min_max_class
	valid_y_misclass
	valid_y_nll
	valid_y_row_norms_max
	valid_y_row_norms_mean
	valid_y_row_norms_min
Compiling accum...
graph size: 171
graph size: 169
graph size: 169
Compiling accum done. Time elapsed: 13.418733 seconds
Monitoring step:
	Epochs seen: 0
	Batches seen: 0
	Examples seen: 0
	learning_rate: 0.00999999046326
	momentum: 0.499999672174
	test_h0_col_norms_max: 6.23503017426
	test_h0_col_norms_mean: 3.82356023788
	test_h0_col_norms_min: 2.06193947792
	test_h0_row_norms_max: 5.89326524734
	test_h0_row_norms_mean: 2.98549389839
	test_h0_row_norms_min: 0.0
	test_h1_col_norms_max: 5.99438333511
	test_h1_col_norms_mean: 3.80721712112
	test_h1_col_norms_min: 1.71524214745
	test_h1_row_norms_max: 7.80886650085
	test_h1_row_norms_mean: 5.40815734863
	test_h1_row_norms_min: 2.97773504257
	test_objective: 3.4297709465
	test_term_0: 2.30258488655
	test_term_1_weight_decay: 1.12718772888
	test_y_col_norms_max: 0.0
	test_y_col_norms_mean: 0.0
	test_y_col_norms_min: 0.0
	test_y_max_max_class: 0.100000023842
	test_y_mean_max_class: 0.100000031292
	test_y_min_max_class: 0.100000023842
	test_y_misclass: 0.901999890804
	test_y_nll: 2.30258488655
	test_y_row_norms_max: 0.0
	test_y_row_norms_mean: 0.0
	test_y_row_norms_min: 0.0
	train_h0_col_norms_max: 6.23505115509
	train_h0_col_norms_mean: 3.82354259491
	train_h0_col_norms_min: 2.0619494915
	train_h0_row_norms_max: 5.89324569702
	train_h0_row_norms_mean: 2.98548007011
	train_h0_row_norms_min: 0.0
	train_h1_col_norms_max: 5.99438095093
	train_h1_col_norms_mean: 3.80721092224
	train_h1_col_norms_min: 1.71524274349
	train_h1_row_norms_max: 7.80887794495
	train_h1_row_norms_mean: 5.40813541412
	train_h1_row_norms_min: 2.97772955894
	train_objective: 3.42977070808
	train_term_0: 2.30257916451
	train_term_1_weight_decay: 1.12718474865
	train_y_col_norms_max: 0.0
	train_y_col_norms_mean: 0.0
	train_y_col_norms_min: 0.0
	train_y_max_max_class: 0.100000545382
	train_y_mean_max_class: 0.100000545382
	train_y_min_max_class: 0.100000545382
	train_y_misclass: 0.901360213757
	train_y_nll: 2.30257916451
	train_y_row_norms_max: 0.0
	train_y_row_norms_mean: 0.0
	train_y_row_norms_min: 0.0
	valid_h0_col_norms_max: 6.23503017426
	valid_h0_col_norms_mean: 3.82356023788
	valid_h0_col_norms_min: 2.06193947792
	valid_h0_row_norms_max: 5.89326524734
	valid_h0_row_norms_mean: 2.98549389839
	valid_h0_row_norms_min: 0.0
	valid_h1_col_norms_max: 5.99438333511
	valid_h1_col_norms_mean: 3.80721712112
	valid_h1_col_norms_min: 1.71524214745
	valid_h1_row_norms_max: 7.80886650085
	valid_h1_row_norms_mean: 5.40815734863
	valid_h1_row_norms_min: 2.97773504257
	valid_objective: 3.4297709465
	valid_term_0: 2.30258488655
	valid_term_1_weight_decay: 1.12718772888
	valid_y_col_norms_max: 0.0
	valid_y_col_norms_mean: 0.0
	valid_y_col_norms_min: 0.0
	valid_y_max_max_class: 0.100000023842
	valid_y_mean_max_class: 0.100000031292
	valid_y_min_max_class: 0.100000023842
	valid_y_misclass: 0.90089994669
	valid_y_nll: 2.30258488655
	valid_y_row_norms_max: 0.0
	valid_y_row_norms_mean: 0.0
	valid_y_row_norms_min: 0.0
Time this epoch: 3.310886 seconds
Monitoring step:
	Epochs seen: 1
	Batches seen: 500
	Examples seen: 50000
	learning_rate: 0.00999999046326
	momentum: 0.499999672174
	test_h0_col_norms_max: 6.22863864899
	test_h0_col_norms_mean: 3.81978034973
	test_h0_col_norms_min: 2.06060481071
	test_h0_row_norms_max: 5.88668251038
	test_h0_row_norms_mean: 2.98259210587
	test_h0_row_norms_min: 0.00163801340386
	test_h1_col_norms_max: 5.98888349533
	test_h1_col_norms_mean: 3.80343770981
	test_h1_col_norms_min: 1.71354997158
	test_h1_row_norms_max: 7.80116271973
	test_h1_row_norms_mean: 5.40278577805
	test_h1_row_norms_min: 2.97481369972
	test_objective: 1.39391481876
	test_term_0: 0.268794178963
	test_term_1_weight_decay: 1.12512099743
	test_y_col_norms_max: 0.645387113094
	test_y_col_norms_mean: 0.59630638361
	test_y_col_norms_min: 0.520404875278
	test_y_max_max_class: 0.999945759773
	test_y_mean_max_class: 0.904323577881
	test_y_min_max_class: 0.380515068769
	test_y_misclass: 0.0813000127673
	test_y_nll: 0.268794178963
	test_y_row_norms_max: 0.179665878415
	test_y_row_norms_mean: 0.0518467575312
	test_y_row_norms_min: 0.000148977691424
	train_h0_col_norms_max: 6.2286696434
	train_h0_col_norms_mean: 3.81979823112
	train_h0_col_norms_min: 2.06059765816
	train_h0_row_norms_max: 5.88671255112
	train_h0_row_norms_mean: 2.9826066494
	train_h0_row_norms_min: 0.00163802062161
	train_h1_col_norms_max: 5.9888548851
	train_h1_col_norms_mean: 3.80346035957
	train_h1_col_norms_min: 1.71355748177
	train_h1_row_norms_max: 7.80111694336
	train_h1_row_norms_mean: 5.40279817581
	train_h1_row_norms_min: 2.97482800484
	train_objective: 1.38994812965
	train_term_0: 0.264828205109
	train_term_1_weight_decay: 1.12512207031
	train_y_col_norms_max: 0.645388245583
	train_y_col_norms_mean: 0.596305251122
	train_y_col_norms_min: 0.520407259464
	train_y_max_max_class: 0.99996304512
	train_y_mean_max_class: 0.898920297623
	train_y_min_max_class: 0.361467987299
	train_y_misclass: 0.0793600603938
	train_y_nll: 0.264828205109
	train_y_row_norms_max: 0.179665371776
	train_y_row_norms_mean: 0.0518467389047
	train_y_row_norms_min: 0.000148977618665
	valid_h0_col_norms_max: 6.22863864899
	valid_h0_col_norms_mean: 3.81978034973
	valid_h0_col_norms_min: 2.06060481071
	valid_h0_row_norms_max: 5.88668251038
	valid_h0_row_norms_mean: 2.98259210587
	valid_h0_row_norms_min: 0.00163801340386
	valid_h1_col_norms_max: 5.98888349533
	valid_h1_col_norms_mean: 3.80343770981
	valid_h1_col_norms_min: 1.71354997158
	valid_h1_row_norms_max: 7.80116271973
	valid_h1_row_norms_mean: 5.40278577805
	valid_h1_row_norms_min: 2.97481369972
	valid_objective: 1.37731289864
	valid_term_0: 0.252192467451
	valid_term_1_weight_decay: 1.12512099743
	valid_y_col_norms_max: 0.645387113094
	valid_y_col_norms_mean: 0.59630638361
	valid_y_col_norms_min: 0.520404875278
	valid_y_max_max_class: 0.999964594841
	valid_y_mean_max_class: 0.907153248787
	valid_y_min_max_class: 0.362326830626
	valid_y_misclass: 0.0756999999285
	valid_y_nll: 0.252192467451
	valid_y_row_norms_max: 0.179665878415
	valid_y_row_norms_mean: 0.0518467575312
	valid_y_row_norms_min: 0.000148977691424
Time this epoch: 3.343837 seconds
Monitoring step:
	Epochs seen: 2
	Batches seen: 1000
	Examples seen: 100000
	learning_rate: 0.00999999046326
	momentum: 0.554444551468
	test_h0_col_norms_max: 6.22144937515
	test_h0_col_norms_mean: 3.81579256058
	test_h0_col_norms_min: 2.05898046494
	test_h0_row_norms_max: 5.88006973267
	test_h0_row_norms_mean: 2.9794948101
	test_h0_row_norms_min: 0.00336797139607
	test_h1_col_norms_max: 5.98277664185
	test_h1_col_norms_mean: 3.79929542542
	test_h1_col_norms_min: 1.71166646481
	test_h1_row_norms_max: 7.79234170914
	test_h1_row_norms_mean: 5.3969039917
	test_h1_row_norms_min: 2.97146487236
	test_objective: 1.3320376873
	test_term_0: 0.209235101938
	test_term_1_weight_decay: 1.12280321121
	test_y_col_norms_max: 0.849509298801
	test_y_col_norms_mean: 0.752226889133
	test_y_col_norms_min: 0.648749351501
	test_y_max_max_class: 0.999980688095
	test_y_mean_max_class: 0.928127348423
	test_y_min_max_class: 0.417017698288
	test_y_misclass: 0.0624000132084
	test_y_nll: 0.209235101938
	test_y_row_norms_max: 0.202931031585
	test_y_row_norms_mean: 0.0667919442058
	test_y_row_norms_min: 0.00027507453342
	train_h0_col_norms_max: 6.22147130966
	train_h0_col_norms_mean: 3.81577634811
	train_h0_col_norms_min: 2.0589826107
	train_h0_row_norms_max: 5.8800983429
	train_h0_row_norms_mean: 2.9795088768
	train_h0_row_norms_min: 0.00336798490025
	train_h1_col_norms_max: 5.98279714584
	train_h1_col_norms_mean: 3.7993118763
	train_h1_col_norms_min: 1.71166646481
	train_h1_row_norms_max: 7.79229545593
	train_h1_row_norms_mean: 5.39690923691
	train_h1_row_norms_min: 2.97145032883
	train_objective: 1.31553328037
	train_term_0: 0.192730411887
	train_term_1_weight_decay: 1.12280583382
	train_y_col_norms_max: 0.849513113499
	train_y_col_norms_mean: 0.752230584621
	train_y_col_norms_min: 0.648747861385
	train_y_max_max_class: 0.999980807304
	train_y_mean_max_class: 0.925747811794
	train_y_min_max_class: 0.379059791565
	train_y_misclass: 0.0572400614619
	train_y_nll: 0.192730411887
	train_y_row_norms_max: 0.202931344509
	train_y_row_norms_mean: 0.0667921230197
	train_y_row_norms_min: 0.00027507476625
	valid_h0_col_norms_max: 6.22144937515
	valid_h0_col_norms_mean: 3.81579256058
	valid_h0_col_norms_min: 2.05898046494
	valid_h0_row_norms_max: 5.88006973267
	valid_h0_row_norms_mean: 2.9794948101
	valid_h0_row_norms_min: 0.00336797139607
	valid_h1_col_norms_max: 5.98277664185
	valid_h1_col_norms_mean: 3.79929542542
	valid_h1_col_norms_min: 1.71166646481
	valid_h1_row_norms_max: 7.79234170914
	valid_h1_row_norms_mean: 5.3969039917
	valid_h1_row_norms_min: 2.97146487236
	valid_objective: 1.32417428493
	valid_term_0: 0.201371654868
	valid_term_1_weight_decay: 1.12280321121
	valid_y_col_norms_max: 0.849509298801
	valid_y_col_norms_mean: 0.752226889133
	valid_y_col_norms_min: 0.648749351501
	valid_y_max_max_class: 0.999982237816
	valid_y_mean_max_class: 0.931577861309
	valid_y_min_max_class: 0.40255895257
	valid_y_misclass: 0.0578999966383
	valid_y_nll: 0.201371654868
	valid_y_row_norms_max: 0.202931031585
	valid_y_row_norms_mean: 0.0667919442058
	valid_y_row_norms_min: 0.00027507453342
Time this epoch: 3.283221 seconds
Monitoring step:
	Epochs seen: 3
	Batches seen: 1500
	Examples seen: 150000
	learning_rate: 0.00999999046326
	momentum: 0.608888924122
	test_h0_col_norms_max: 6.21347379684
	test_h0_col_norms_mean: 3.81121587753
	test_h0_col_norms_min: 2.05705142021
	test_h0_row_norms_max: 5.87235736847
	test_h0_row_norms_mean: 2.97595834732
	test_h0_row_norms_min: 0.00510276248679
	test_h1_col_norms_max: 5.97572278976
	test_h1_col_norms_mean: 3.79457330704
	test_h1_col_norms_min: 1.70953249931
	test_h1_row_norms_max: 7.78235435486
	test_h1_row_norms_mean: 5.39019727707
	test_h1_row_norms_min: 2.96771478653
	test_objective: 1.30544030666
	test_term_0: 0.185299769044
	test_term_1_weight_decay: 1.12013947964
	test_y_col_norms_max: 1.00650155544
	test_y_col_norms_mean: 0.878560483456
	test_y_col_norms_min: 0.748090326786
	test_y_max_max_class: 0.999993503094
	test_y_mean_max_class: 0.939459979534
	test_y_min_max_class: 0.444366723299
	test_y_misclass: 0.0547000169754
	test_y_nll: 0.185299769044
	test_y_row_norms_max: 0.217191457748
	test_y_row_norms_mean: 0.0787876471877
	test_y_row_norms_min: 0.000392778747482
	train_h0_col_norms_max: 6.21344470978
	train_h0_col_norms_mean: 3.81123256683
	train_h0_col_norms_min: 2.05706167221
	train_h0_row_norms_max: 5.87232971191
	train_h0_row_norms_mean: 2.97594833374
	train_h0_row_norms_min: 0.00510273734108
	train_h1_col_norms_max: 5.97572278976
	train_h1_col_norms_mean: 3.79455709457
	train_h1_col_norms_min: 1.70952439308
	train_h1_row_norms_max: 7.78239917755
	train_h1_row_norms_mean: 5.39017248154
	train_h1_row_norms_min: 2.96771502495
	train_objective: 1.2823060751
	train_term_0: 0.162165120244
	train_term_1_weight_decay: 1.12014472485
	train_y_col_norms_max: 1.00650632381
	train_y_col_norms_mean: 0.878564417362
	train_y_col_norms_min: 0.748090386391
	train_y_max_max_class: 0.999991178513
	train_y_mean_max_class: 0.93700414896
	train_y_min_max_class: 0.404900848866
	train_y_misclass: 0.0482200570405
	train_y_nll: 0.162165120244
	train_y_row_norms_max: 0.21719174087
	train_y_row_norms_mean: 0.0787875503302
	train_y_row_norms_min: 0.000392780813854
	valid_h0_col_norms_max: 6.21347379684
	valid_h0_col_norms_mean: 3.81121587753
	valid_h0_col_norms_min: 2.05705142021
	valid_h0_row_norms_max: 5.87235736847
	valid_h0_row_norms_mean: 2.97595834732
	valid_h0_row_norms_min: 0.00510276248679
	valid_h1_col_norms_max: 5.97572278976
	valid_h1_col_norms_mean: 3.79457330704
	valid_h1_col_norms_min: 1.70953249931
	valid_h1_row_norms_max: 7.78235435486
	valid_h1_row_norms_mean: 5.39019727707
	valid_h1_row_norms_min: 2.96771478653
	valid_objective: 1.29470717907
	valid_term_0: 0.174566537142
	valid_term_1_weight_decay: 1.12013947964
	valid_y_col_norms_max: 1.00650155544
	valid_y_col_norms_mean: 0.878560483456
	valid_y_col_norms_min: 0.748090326786
	valid_y_max_max_class: 0.999994695187
	valid_y_mean_max_class: 0.942149102688
	valid_y_min_max_class: 0.417711257935
	valid_y_misclass: 0.051200017333
	valid_y_nll: 0.174566537142
	valid_y_row_norms_max: 0.217191457748
	valid_y_row_norms_mean: 0.0787876471877
	valid_y_row_norms_min: 0.000392778747482
Time this epoch: 3.301401 seconds
Monitoring step:
	Epochs seen: 4
	Batches seen: 2000
	Examples seen: 200000
	learning_rate: 0.00999999046326
	momentum: 0.663333714008
	test_h0_col_norms_max: 6.20446586609
	test_h0_col_norms_mean: 3.80589365959
	test_h0_col_norms_min: 2.05491876602
	test_h0_row_norms_max: 5.86368274689
	test_h0_row_norms_mean: 2.97183966637
	test_h0_row_norms_min: 0.00636858073995
	test_h1_col_norms_max: 5.96751737595
	test_h1_col_norms_mean: 3.78907322884
	test_h1_col_norms_min: 1.70705342293
	test_h1_row_norms_max: 7.77082681656
	test_h1_row_norms_mean: 5.38239336014
	test_h1_row_norms_min: 2.96349358559
	test_objective: 1.28483641148
	test_term_0: 0.167798668146
	test_term_1_weight_decay: 1.11703836918
	test_y_col_norms_max: 1.14337170124
	test_y_col_norms_mean: 0.994192421436
	test_y_col_norms_min: 0.840292572975
	test_y_max_max_class: 0.999995589256
	test_y_mean_max_class: 0.946651279926
	test_y_min_max_class: 0.454940706491
	test_y_misclass: 0.0549000278115
	test_y_nll: 0.167798668146
	test_y_row_norms_max: 0.231142029166
	test_y_row_norms_mean: 0.089763648808
	test_y_row_norms_min: 0.000477136200061
	train_h0_col_norms_max: 6.20444250107
	train_h0_col_norms_mean: 3.80587768555
	train_h0_col_norms_min: 2.05492663383
	train_h0_row_norms_max: 5.86367082596
	train_h0_row_norms_mean: 2.97184991837
	train_h0_row_norms_min: 0.00636860262603
	train_h1_col_norms_max: 5.96753835678
	train_h1_col_norms_mean: 3.7890689373
	train_h1_col_norms_min: 1.70706069469
	train_h1_row_norms_max: 7.77079200745
	train_h1_row_norms_mean: 5.38238239288
	train_h1_row_norms_min: 2.96350455284
	train_objective: 1.25564575195
	train_term_0: 0.138607770205
	train_term_1_weight_decay: 1.11704432964
	train_y_col_norms_max: 1.14337110519
	train_y_col_norms_mean: 0.994198083878
	train_y_col_norms_min: 0.840297460556
	train_y_max_max_class: 0.999992132187
	train_y_mean_max_class: 0.945581674576
	train_y_min_max_class: 0.42304289341
	train_y_misclass: 0.0431200563908
	train_y_nll: 0.138607770205
	train_y_row_norms_max: 0.231140971184
	train_y_row_norms_mean: 0.0897636190057
	train_y_row_norms_min: 0.000477139052236
	valid_h0_col_norms_max: 6.20446586609
	valid_h0_col_norms_mean: 3.80589365959
	valid_h0_col_norms_min: 2.05491876602
	valid_h0_row_norms_max: 5.86368274689
	valid_h0_row_norms_mean: 2.97183966637
	valid_h0_row_norms_min: 0.00636858073995
	valid_h1_col_norms_max: 5.96751737595
	valid_h1_col_norms_mean: 3.78907322884
	valid_h1_col_norms_min: 1.70705342293
	valid_h1_row_norms_max: 7.77082681656
	valid_h1_row_norms_mean: 5.38239336014
	valid_h1_row_norms_min: 2.96349358559
	valid_objective: 1.27460837364
	valid_term_0: 0.157571211457
	valid_term_1_weight_decay: 1.11703836918
	valid_y_col_norms_max: 1.14337170124
	valid_y_col_norms_mean: 0.994192421436
	valid_y_col_norms_min: 0.840292572975
	valid_y_max_max_class: 0.999996304512
	valid_y_mean_max_class: 0.949614882469
	valid_y_min_max_class: 0.442067503929
	valid_y_misclass: 0.0465000085533
	valid_y_nll: 0.157571211457
	valid_y_row_norms_max: 0.231142029166
	valid_y_row_norms_mean: 0.089763648808
	valid_y_row_norms_min: 0.000477136200061
Time this epoch: 3.266055 seconds
Monitoring step:
	Epochs seen: 5
	Batches seen: 2500
	Examples seen: 250000
	learning_rate: 0.00999999046326
	momentum: 0.717777192593
	test_h0_col_norms_max: 6.19388818741
	test_h0_col_norms_mean: 3.79951477051
	test_h0_col_norms_min: 2.05230784416
	test_h0_row_norms_max: 5.85298204422
	test_h0_row_norms_mean: 2.96690416336
	test_h0_row_norms_min: 0.00795079302043
	test_h1_col_norms_max: 5.95764780045
	test_h1_col_norms_mean: 3.78251552582
	test_h1_col_norms_min: 1.70412421227
	test_h1_row_norms_max: 7.7571387291
	test_h1_row_norms_mean: 5.37306642532
	test_h1_row_norms_min: 2.95853662491
	test_objective: 1.25132834911
	test_term_0: 0.138001933694
	test_term_1_weight_decay: 1.11332631111
	test_y_col_norms_max: 1.26581287384
	test_y_col_norms_mean: 1.10778701305
	test_y_col_norms_min: 0.922472834587
	test_y_max_max_class: 0.999994754791
	test_y_mean_max_class: 0.953354179859
	test_y_min_max_class: 0.460847198963
	test_y_misclass: 0.0430000051856
	test_y_nll: 0.138001933694
	test_y_row_norms_max: 0.258754551411
	test_y_row_norms_mean: 0.100538700819
	test_y_row_norms_min: 0.000593058066443
	train_h0_col_norms_max: 6.19387769699
	train_h0_col_norms_mean: 3.79953241348
	train_h0_col_norms_min: 2.05230736732
	train_h0_row_norms_max: 5.85295391083
	train_h0_row_norms_mean: 2.96689033508
	train_h0_row_norms_min: 0.0079507548362
	train_h1_col_norms_max: 5.95761966705
	train_h1_col_norms_mean: 3.78251123428
	train_h1_col_norms_min: 1.70413899422
	train_h1_row_norms_max: 7.75717258453
	train_h1_row_norms_mean: 5.37306880951
	train_h1_row_norms_min: 2.95853638649
	train_objective: 1.21803998947
	train_term_0: 0.104714490473
	train_term_1_weight_decay: 1.11332845688
	train_y_col_norms_max: 1.26581907272
	train_y_col_norms_mean: 1.1077862978
	train_y_col_norms_min: 0.922471702099
	train_y_max_max_class: 0.999992728233
	train_y_mean_max_class: 0.954178750515
	train_y_min_max_class: 0.440906405449
	train_y_misclass: 0.0312400292605
	train_y_nll: 0.104714490473
	train_y_row_norms_max: 0.258753240108
	train_y_row_norms_mean: 0.100538358092
	train_y_row_norms_min: 0.000593057950027
	valid_h0_col_norms_max: 6.19388818741
	valid_h0_col_norms_mean: 3.79951477051
	valid_h0_col_norms_min: 2.05230784416
	valid_h0_row_norms_max: 5.85298204422
	valid_h0_row_norms_mean: 2.96690416336
	valid_h0_row_norms_min: 0.00795079302043
	valid_h1_col_norms_max: 5.95764780045
	valid_h1_col_norms_mean: 3.78251552582
	valid_h1_col_norms_min: 1.70412421227
	valid_h1_row_norms_max: 7.7571387291
	valid_h1_row_norms_mean: 5.37306642532
	valid_h1_row_norms_min: 2.95853662491
	valid_objective: 1.24973428249
	valid_term_0: 0.136407867074
	valid_term_1_weight_decay: 1.11332631111
	valid_y_col_norms_max: 1.26581287384
	valid_y_col_norms_mean: 1.10778701305
	valid_y_col_norms_min: 0.922472834587
	valid_y_max_max_class: 0.999996542931
	valid_y_mean_max_class: 0.955720424652
	valid_y_min_max_class: 0.447657436132
	valid_y_misclass: 0.0386000014842
	valid_y_nll: 0.136407867074
	valid_y_row_norms_max: 0.258754551411
	valid_y_row_norms_mean: 0.100538700819
	valid_y_row_norms_min: 0.000593058066443
Time this epoch: 3.281634 seconds
Monitoring step:
	Epochs seen: 6
	Batches seen: 3000
	Examples seen: 300000
	learning_rate: 0.00999999046326
	momentum: 0.772221684456
	test_h0_col_norms_max: 6.18053913116
	test_h0_col_norms_mean: 3.79164195061
	test_h0_col_norms_min: 2.04931807518
	test_h0_row_norms_max: 5.84014606476
	test_h0_row_norms_mean: 2.96080875397
	test_h0_row_norms_min: 0.00960826966912
	test_h1_col_norms_max: 5.94511365891
	test_h1_col_norms_mean: 3.77440404892
	test_h1_col_norms_min: 1.70048546791
	test_h1_row_norms_max: 7.74020195007
	test_h1_row_norms_mean: 5.3615436554
	test_h1_row_norms_min: 2.95202755928
	test_objective: 1.23484170437
	test_term_0: 0.126101091504
	test_term_1_weight_decay: 1.10874140263
	test_y_col_norms_max: 1.39184403419
	test_y_col_norms_mean: 1.23041391373
	test_y_col_norms_min: 1.02565836906
	test_y_max_max_class: 0.999998748302
	test_y_mean_max_class: 0.961094081402
	test_y_min_max_class: 0.502607226372
	test_y_misclass: 0.0397000052035
	test_y_nll: 0.126101091504
	test_y_row_norms_max: 0.288574844599
	test_y_row_norms_mean: 0.112107351422
	test_y_row_norms_min: 0.000744926044717
	train_h0_col_norms_max: 6.18052864075
	train_h0_col_norms_mean: 3.79166030884
	train_h0_col_norms_min: 2.04932594299
	train_h0_row_norms_max: 5.84012889862
	train_h0_row_norms_mean: 2.96080327034
	train_h0_row_norms_min: 0.0096082771197
	train_h1_col_norms_max: 5.94514036179
	train_h1_col_norms_mean: 3.77440428734
	train_h1_col_norms_min: 1.7004776001
	train_h1_row_norms_max: 7.74021291733
	train_h1_row_norms_mean: 5.36155557632
	train_h1_row_norms_min: 2.9520175457
	train_objective: 1.19061946869
	train_term_0: 0.0818792134523
	train_term_1_weight_decay: 1.10874009132
	train_y_col_norms_max: 1.39184439182
	train_y_col_norms_mean: 1.2304173708
	train_y_col_norms_min: 1.02565360069
	train_y_max_max_class: 0.999994039536
	train_y_mean_max_class: 0.963193774223
	train_y_min_max_class: 0.475303918123
	train_y_misclass: 0.0230600107461
	train_y_nll: 0.0818792134523
	train_y_row_norms_max: 0.288575559855
	train_y_row_norms_mean: 0.112107902765
	train_y_row_norms_min: 0.000744922028389
	valid_h0_col_norms_max: 6.18053913116
	valid_h0_col_norms_mean: 3.79164195061
	valid_h0_col_norms_min: 2.04931807518
	valid_h0_row_norms_max: 5.84014606476
	valid_h0_row_norms_mean: 2.96080875397
	valid_h0_row_norms_min: 0.00960826966912
	valid_h1_col_norms_max: 5.94511365891
	valid_h1_col_norms_mean: 3.77440404892
	valid_h1_col_norms_min: 1.70048546791
	valid_h1_row_norms_max: 7.74020195007
	valid_h1_row_norms_mean: 5.3615436554
	valid_h1_row_norms_min: 2.95202755928
	valid_objective: 1.23645818233
	valid_term_0: 0.127717524767
	valid_term_1_weight_decay: 1.10874140263
	valid_y_col_norms_max: 1.39184403419
	valid_y_col_norms_mean: 1.23041391373
	valid_y_col_norms_min: 1.02565836906
	valid_y_max_max_class: 0.999998986721
	valid_y_mean_max_class: 0.963711440563
	valid_y_min_max_class: 0.479158580303
	valid_y_misclass: 0.0373999997973
	valid_y_nll: 0.127717524767
	valid_y_row_norms_max: 0.288574844599
	valid_y_row_norms_mean: 0.112107351422
	valid_y_row_norms_min: 0.000744926044717
Time this epoch: 3.285549 seconds
Monitoring step:
	Epochs seen: 7
	Batches seen: 3500
	Examples seen: 350000
	learning_rate: 0.00999999046326
	momentum: 0.826667308807
	test_h0_col_norms_max: 6.16351413727
	test_h0_col_norms_mean: 3.78127264977
	test_h0_col_norms_min: 2.04552721977
	test_h0_row_norms_max: 5.82413673401
	test_h0_row_norms_mean: 2.95279192924
	test_h0_row_norms_min: 0.0109715117142
	test_h1_col_norms_max: 5.92860794067
	test_h1_col_norms_mean: 3.7637283802
	test_h1_col_norms_min: 1.69574940205
	test_h1_row_norms_max: 7.71776247025
	test_h1_row_norms_mean: 5.34639310837
	test_h1_row_norms_min: 2.94415974617
	test_objective: 1.2293548584
	test_term_0: 0.126640558243
	test_term_1_weight_decay: 1.1027148962
	test_y_col_norms_max: 1.53999233246
	test_y_col_norms_mean: 1.36674308777
	test_y_col_norms_min: 1.134085536
	test_y_max_max_class: 0.999998986721
	test_y_mean_max_class: 0.962450027466
	test_y_min_max_class: 0.520037055016
	test_y_misclass: 0.0400000177324
	test_y_nll: 0.126640558243
	test_y_row_norms_max: 0.323384702206
	test_y_row_norms_mean: 0.124884955585
	test_y_row_norms_min: 0.000862787244841
	train_h0_col_norms_max: 6.1635351181
	train_h0_col_norms_mean: 3.78129315376
	train_h0_col_norms_min: 2.04552340508
	train_h0_row_norms_max: 5.82410860062
	train_h0_row_norms_mean: 2.95280575752
	train_h0_row_norms_min: 0.0109714772552
	train_h1_col_norms_max: 5.92858171463
	train_h1_col_norms_mean: 3.76370692253
	train_h1_col_norms_min: 1.69575130939
	train_h1_row_norms_max: 7.71779823303
	train_h1_row_norms_mean: 5.34638214111
	train_h1_row_norms_min: 2.94414997101
	train_objective: 1.18144452572
	train_term_0: 0.0787304490805
	train_term_1_weight_decay: 1.10271286964
	train_y_col_norms_max: 1.54000031948
	train_y_col_norms_mean: 1.36673867702
	train_y_col_norms_min: 1.13409137726
	train_y_max_max_class: 0.999994158745
	train_y_mean_max_class: 0.964662730694
	train_y_min_max_class: 0.485619604588
	train_y_misclass: 0.0242600161582
	train_y_nll: 0.0787304490805
	train_y_row_norms_max: 0.323384910822
	train_y_row_norms_mean: 0.124885700643
	train_y_row_norms_min: 0.000862783577759
	valid_h0_col_norms_max: 6.16351413727
	valid_h0_col_norms_mean: 3.78127264977
	valid_h0_col_norms_min: 2.04552721977
	valid_h0_row_norms_max: 5.82413673401
	valid_h0_row_norms_mean: 2.95279192924
	valid_h0_row_norms_min: 0.0109715117142
	valid_h1_col_norms_max: 5.92860794067
	valid_h1_col_norms_mean: 3.7637283802
	valid_h1_col_norms_min: 1.69574940205
	valid_h1_row_norms_max: 7.71776247025
	valid_h1_row_norms_mean: 5.34639310837
	valid_h1_row_norms_min: 2.94415974617
	valid_objective: 1.22817146778
	valid_term_0: 0.125456944108
	valid_term_1_weight_decay: 1.1027148962
	valid_y_col_norms_max: 1.53999233246
	valid_y_col_norms_mean: 1.36674308777
	valid_y_col_norms_min: 1.134085536
	valid_y_max_max_class: 0.99999910593
	valid_y_mean_max_class: 0.965774953365
	valid_y_min_max_class: 0.481605708599
	valid_y_misclass: 0.0360999889672
	valid_y_nll: 0.125456944108
	valid_y_row_norms_max: 0.323384702206
	valid_y_row_norms_mean: 0.124884955585
	valid_y_row_norms_min: 0.000862787244841
Time this epoch: 3.275973 seconds
Monitoring step:
	Epochs seen: 8
	Batches seen: 4000
	Examples seen: 400000
	learning_rate: 0.00999999046326
	momentum: 0.881111502647
	test_h0_col_norms_max: 6.13874149323
	test_h0_col_norms_mean: 3.76625037193
	test_h0_col_norms_min: 2.03984022141
	test_h0_row_norms_max: 5.79944992065
	test_h0_row_norms_mean: 2.94116210938
	test_h0_row_norms_min: 0.0121828410774
	test_h1_col_norms_max: 5.90430831909
	test_h1_col_norms_mean: 3.74820208549
	test_h1_col_norms_min: 1.68876981735
	test_h1_row_norms_max: 7.68556308746
	test_h1_row_norms_mean: 5.32432985306
	test_h1_row_norms_min: 2.93232631683
	test_objective: 1.21413767338
	test_term_0: 0.12014952302
	test_term_1_weight_decay: 1.09398806095
	test_y_col_norms_max: 1.73185801506
	test_y_col_norms_mean: 1.54484415054
	test_y_col_norms_min: 1.28760778904
	test_y_max_max_class: 0.999999284744
	test_y_mean_max_class: 0.969546198845
	test_y_min_max_class: 0.53670758009
	test_y_misclass: 0.0355999991298
	test_y_nll: 0.12014952302
	test_y_row_norms_max: 0.390541791916
	test_y_row_norms_mean: 0.141607090831
	test_y_row_norms_min: 0.00119230698328
	train_h0_col_norms_max: 6.13874340057
	train_h0_col_norms_mean: 3.76626849174
	train_h0_col_norms_min: 2.03984594345
	train_h0_row_norms_max: 5.79946660995
	train_h0_row_norms_mean: 2.94116210938
	train_h0_row_norms_min: 0.0121827786788
	train_h1_col_norms_max: 5.90427827835
	train_h1_col_norms_mean: 3.74818611145
	train_h1_col_norms_min: 1.68877720833
	train_h1_row_norms_max: 7.68560028076
	train_h1_row_norms_mean: 5.32434654236
	train_h1_row_norms_min: 2.93231272697
	train_objective: 1.15510380268
	train_term_0: 0.0611156411469
	train_term_1_weight_decay: 1.09398496151
	train_y_col_norms_max: 1.73185968399
	train_y_col_norms_mean: 1.54483699799
	train_y_col_norms_min: 1.28760266304
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.97226446867
	train_y_min_max_class: 0.523442387581
	train_y_misclass: 0.0181600283831
	train_y_nll: 0.0611156411469
	train_y_row_norms_max: 0.390543580055
	train_y_row_norms_mean: 0.141606390476
	train_y_row_norms_min: 0.00119230675045
	valid_h0_col_norms_max: 6.13874149323
	valid_h0_col_norms_mean: 3.76625037193
	valid_h0_col_norms_min: 2.03984022141
	valid_h0_row_norms_max: 5.79944992065
	valid_h0_row_norms_mean: 2.94116210938
	valid_h0_row_norms_min: 0.0121828410774
	valid_h1_col_norms_max: 5.90430831909
	valid_h1_col_norms_mean: 3.74820208549
	valid_h1_col_norms_min: 1.68876981735
	valid_h1_row_norms_max: 7.68556308746
	valid_h1_row_norms_mean: 5.32432985306
	valid_h1_row_norms_min: 2.93232631683
	valid_objective: 1.2128187418
	valid_term_0: 0.118830725551
	valid_term_1_weight_decay: 1.09398806095
	valid_y_col_norms_max: 1.73185801506
	valid_y_col_norms_mean: 1.54484415054
	valid_y_col_norms_min: 1.28760778904
	valid_y_max_max_class: 0.999999284744
	valid_y_mean_max_class: 0.971059143543
	valid_y_min_max_class: 0.500100016594
	valid_y_misclass: 0.0353999920189
	valid_y_nll: 0.118830725551
	valid_y_row_norms_max: 0.390541791916
	valid_y_row_norms_mean: 0.141607090831
	valid_y_row_norms_min: 0.00119230698328
Time this epoch: 3.273986 seconds
Monitoring step:
	Epochs seen: 9
	Batches seen: 4500
	Examples seen: 450000
	learning_rate: 0.00999999046326
	momentum: 0.935554862022
	test_h0_col_norms_max: 6.09445524216
	test_h0_col_norms_mean: 3.73940348625
	test_h0_col_norms_min: 2.03072142601
	test_h0_row_norms_max: 5.75560235977
	test_h0_row_norms_mean: 2.92046833038
	test_h0_row_norms_min: 0.014029703103
	test_h1_col_norms_max: 5.86166810989
	test_h1_col_norms_mean: 3.71971082687
	test_h1_col_norms_min: 1.67665565014
	test_h1_row_norms_max: 7.62777662277
	test_h1_row_norms_mean: 5.2838845253
	test_h1_row_norms_min: 2.91292881966
	test_objective: 1.20774161816
	test_term_0: 0.129474073648
	test_term_1_weight_decay: 1.0782674551
	test_y_col_norms_max: 2.063549757
	test_y_col_norms_mean: 1.8654705286
	test_y_col_norms_min: 1.53516829014
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.971782028675
	test_y_min_max_class: 0.541796386242
	test_y_misclass: 0.0371000058949
	test_y_nll: 0.129474073648
	test_y_row_norms_max: 0.496850013733
	test_y_row_norms_mean: 0.171486049891
	test_y_row_norms_min: 0.00181403872557
	train_h0_col_norms_max: 6.09445524216
	train_h0_col_norms_mean: 3.73938298225
	train_h0_col_norms_min: 2.03072929382
	train_h0_row_norms_max: 5.75560045242
	train_h0_row_norms_mean: 2.92047595978
	train_h0_row_norms_min: 0.0140297813341
	train_h1_col_norms_max: 5.86169338226
	train_h1_col_norms_mean: 3.71969389915
	train_h1_col_norms_min: 1.67666423321
	train_h1_row_norms_max: 7.62774133682
	train_h1_row_norms_mean: 5.2838549614
	train_h1_row_norms_min: 2.91291928291
	train_objective: 1.14386320114
	train_term_0: 0.0655960813165
	train_term_1_weight_decay: 1.07827007771
	train_y_col_norms_max: 2.06355881691
	train_y_col_norms_mean: 1.86546158791
	train_y_col_norms_min: 1.53517353535
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.976020038128
	train_y_min_max_class: 0.536927580833
	train_y_misclass: 0.0207600202411
	train_y_nll: 0.0655960813165
	train_y_row_norms_max: 0.4968495965
	train_y_row_norms_mean: 0.17148527503
	train_y_row_norms_min: 0.00181404093746
	valid_h0_col_norms_max: 6.09445524216
	valid_h0_col_norms_mean: 3.73940348625
	valid_h0_col_norms_min: 2.03072142601
	valid_h0_row_norms_max: 5.75560235977
	valid_h0_row_norms_mean: 2.92046833038
	valid_h0_row_norms_min: 0.014029703103
	valid_h1_col_norms_max: 5.86166810989
	valid_h1_col_norms_mean: 3.71971082687
	valid_h1_col_norms_min: 1.67665565014
	valid_h1_row_norms_max: 7.62777662277
	valid_h1_row_norms_mean: 5.2838845253
	valid_h1_row_norms_min: 2.91292881966
	valid_objective: 1.21526145935
	valid_term_0: 0.136994019151
	valid_term_1_weight_decay: 1.0782674551
	valid_y_col_norms_max: 2.063549757
	valid_y_col_norms_mean: 1.8654705286
	valid_y_col_norms_min: 1.53516829014
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.974301934242
	valid_y_min_max_class: 0.516560852528
	valid_y_misclass: 0.0349999815226
	valid_y_nll: 0.136994019151
	valid_y_row_norms_max: 0.496850013733
	valid_y_row_norms_mean: 0.171486049891
	valid_y_row_norms_min: 0.00181403872557
Time this epoch: 3.317775 seconds
Monitoring step:
	Epochs seen: 10
	Batches seen: 5000
	Examples seen: 500000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 5.92522621155
	test_h0_col_norms_mean: 3.73818850517
	test_h0_col_norms_min: 2.15961098671
	test_h0_row_norms_max: 5.7353053093
	test_h0_row_norms_mean: 2.92477583885
	test_h0_row_norms_min: 0.0341217853129
	test_h1_col_norms_max: 5.61352205276
	test_h1_col_norms_mean: 3.57546806335
	test_h1_col_norms_min: 1.61370325089
	test_h1_row_norms_max: 7.31059169769
	test_h1_row_norms_mean: 5.08152914047
	test_h1_row_norms_min: 2.99987840652
	test_objective: 1.26450061798
	test_term_0: 0.236082434654
	test_term_1_weight_decay: 1.02841842175
	test_y_col_norms_max: 4.73058700562
	test_y_col_norms_mean: 4.18089103699
	test_y_col_norms_min: 3.5669798851
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.968040525913
	test_y_min_max_class: 0.498749941587
	test_y_misclass: 0.059400010854
	test_y_nll: 0.236082434654
	test_y_row_norms_max: 0.891591668129
	test_y_row_norms_mean: 0.392109334469
	test_y_row_norms_min: 0.0124359438196
	train_h0_col_norms_max: 5.92519760132
	train_h0_col_norms_mean: 3.73817253113
	train_h0_col_norms_min: 2.15960621834
	train_h0_row_norms_max: 5.73533010483
	train_h0_row_norms_mean: 2.92479014397
	train_h0_row_norms_min: 0.0341217927635
	train_h1_col_norms_max: 5.61354923248
	train_h1_col_norms_mean: 3.57545208931
	train_h1_col_norms_min: 1.61369478703
	train_h1_row_norms_max: 7.31061649323
	train_h1_row_norms_mean: 5.08150863647
	train_h1_row_norms_min: 2.99989366531
	train_objective: 1.2140481472
	train_term_0: 0.185629963875
	train_term_1_weight_decay: 1.02841842175
	train_y_col_norms_max: 4.73060131073
	train_y_col_norms_mean: 4.18090629578
	train_y_col_norms_min: 3.56698012352
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.968826234341
	train_y_min_max_class: 0.484800755978
	train_y_misclass: 0.0509400516748
	train_y_nll: 0.185629963875
	train_y_row_norms_max: 0.891595542431
	train_y_row_norms_mean: 0.392109185457
	train_y_row_norms_min: 0.0124359484762
	valid_h0_col_norms_max: 5.92522621155
	valid_h0_col_norms_mean: 3.73818850517
	valid_h0_col_norms_min: 2.15961098671
	valid_h0_row_norms_max: 5.7353053093
	valid_h0_row_norms_mean: 2.92477583885
	valid_h0_row_norms_min: 0.0341217853129
	valid_h1_col_norms_max: 5.61352205276
	valid_h1_col_norms_mean: 3.57546806335
	valid_h1_col_norms_min: 1.61370325089
	valid_h1_row_norms_max: 7.31059169769
	valid_h1_row_norms_mean: 5.08152914047
	valid_h1_row_norms_min: 2.99987840652
	valid_objective: 1.27066576481
	valid_term_0: 0.242247447371
	valid_term_1_weight_decay: 1.02841842175
	valid_y_col_norms_max: 4.73058700562
	valid_y_col_norms_mean: 4.18089103699
	valid_y_col_norms_min: 3.5669798851
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.969310641289
	valid_y_min_max_class: 0.485083043575
	valid_y_misclass: 0.0584000013769
	valid_y_nll: 0.242247447371
	valid_y_row_norms_max: 0.891591668129
	valid_y_row_norms_mean: 0.392109334469
	valid_y_row_norms_min: 0.0124359438196
Time this epoch: 3.378083 seconds
Monitoring step:
	Epochs seen: 11
	Batches seen: 5500
	Examples seen: 550000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 5.70130395889
	test_h0_col_norms_mean: 3.63394594193
	test_h0_col_norms_min: 2.06413507462
	test_h0_row_norms_max: 5.63436841965
	test_h0_row_norms_mean: 2.84383249283
	test_h0_row_norms_min: 0.0585759952664
	test_h1_col_norms_max: 5.33032464981
	test_h1_col_norms_mean: 3.41074442863
	test_h1_col_norms_min: 1.54273200035
	test_h1_row_norms_max: 6.95094776154
	test_h1_row_norms_mean: 4.84889364243
	test_h1_row_norms_min: 2.85255265236
	test_objective: 1.09497404099
	test_term_0: 0.145816907287
	test_term_1_weight_decay: 0.94915664196
	test_y_col_norms_max: 4.7894949913
	test_y_col_norms_mean: 4.32798671722
	test_y_col_norms_min: 3.85334467888
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.976011812687
	test_y_min_max_class: 0.533329129219
	test_y_misclass: 0.0402000099421
	test_y_nll: 0.145816907287
	test_y_row_norms_max: 1.16048634052
	test_y_row_norms_mean: 0.407433569431
	test_y_row_norms_min: 0.0134850135073
	train_h0_col_norms_max: 5.70130395889
	train_h0_col_norms_mean: 3.63395118713
	train_h0_col_norms_min: 2.06412315369
	train_h0_row_norms_max: 5.63437128067
	train_h0_row_norms_mean: 2.84384655952
	train_h0_row_norms_min: 0.0585760846734
	train_h1_col_norms_max: 5.33032464981
	train_h1_col_norms_mean: 3.4107298851
	train_h1_col_norms_min: 1.54273247719
	train_h1_row_norms_max: 6.95091104507
	train_h1_row_norms_mean: 4.84888315201
	train_h1_row_norms_min: 2.8525583744
	train_objective: 1.04947304726
	train_term_0: 0.10031542182
	train_term_1_weight_decay: 0.949151813984
	train_y_col_norms_max: 4.78949642181
	train_y_col_norms_mean: 4.32800579071
	train_y_col_norms_min: 3.85332846642
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.977987766266
	train_y_min_max_class: 0.520123898983
	train_y_misclass: 0.0307400058955
	train_y_nll: 0.10031542182
	train_y_row_norms_max: 1.16048312187
	train_y_row_norms_mean: 0.407431900501
	train_y_row_norms_min: 0.0134850600734
	valid_h0_col_norms_max: 5.70130395889
	valid_h0_col_norms_mean: 3.63394594193
	valid_h0_col_norms_min: 2.06413507462
	valid_h0_row_norms_max: 5.63436841965
	valid_h0_row_norms_mean: 2.84383249283
	valid_h0_row_norms_min: 0.0585759952664
	valid_h1_col_norms_max: 5.33032464981
	valid_h1_col_norms_mean: 3.41074442863
	valid_h1_col_norms_min: 1.54273200035
	valid_h1_row_norms_max: 6.95094776154
	valid_h1_row_norms_mean: 4.84889364243
	valid_h1_row_norms_min: 2.85255265236
	valid_objective: 1.09732854366
	valid_term_0: 0.148171290755
	valid_term_1_weight_decay: 0.94915664196
	valid_y_col_norms_max: 4.7894949913
	valid_y_col_norms_mean: 4.32798671722
	valid_y_col_norms_min: 3.85334467888
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.977250099182
	valid_y_min_max_class: 0.51136559248
	valid_y_misclass: 0.0399999879301
	valid_y_nll: 0.148171290755
	valid_y_row_norms_max: 1.16048634052
	valid_y_row_norms_mean: 0.407433569431
	valid_y_row_norms_min: 0.0134850135073
Time this epoch: 3.333940 seconds
Monitoring step:
	Epochs seen: 12
	Batches seen: 6000
	Examples seen: 600000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 5.4267373085
	test_h0_col_norms_mean: 3.48842096329
	test_h0_col_norms_min: 1.96254551411
	test_h0_row_norms_max: 5.41478538513
	test_h0_row_norms_mean: 2.7299387455
	test_h0_row_norms_min: 0.0784849375486
	test_h1_col_norms_max: 5.06470775604
	test_h1_col_norms_mean: 3.24842214584
	test_h1_col_norms_min: 1.46678352356
	test_h1_row_norms_max: 6.60853338242
	test_h1_row_norms_mean: 4.6184220314
	test_h1_row_norms_min: 2.71205830574
	test_objective: 0.985752701759
	test_term_0: 0.119499914348
	test_term_1_weight_decay: 0.866252303123
	test_y_col_norms_max: 4.76992559433
	test_y_col_norms_mean: 4.27050018311
	test_y_col_norms_min: 3.78093886375
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.979267477989
	test_y_min_max_class: 0.543375730515
	test_y_misclass: 0.0315999910235
	test_y_nll: 0.119499914348
	test_y_row_norms_max: 0.912945210934
	test_y_row_norms_mean: 0.402993023396
	test_y_row_norms_min: 0.0216930937022
	train_h0_col_norms_max: 5.42672777176
	train_h0_col_norms_mean: 3.48842120171
	train_h0_col_norms_min: 1.96254348755
	train_h0_row_norms_max: 5.41479063034
	train_h0_row_norms_mean: 2.72992825508
	train_h0_row_norms_min: 0.0784846991301
	train_h1_col_norms_max: 5.06472110748
	train_h1_col_norms_mean: 3.24842524529
	train_h1_col_norms_min: 1.46679055691
	train_h1_row_norms_max: 6.60850334167
	train_h1_row_norms_mean: 4.61840820312
	train_h1_row_norms_min: 2.71205258369
	train_objective: 0.922969102859
	train_term_0: 0.0567165091634
	train_term_1_weight_decay: 0.866256058216
	train_y_col_norms_max: 4.76994085312
	train_y_col_norms_mean: 4.27052545547
	train_y_col_norms_min: 3.7809548378
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.982750058174
	train_y_min_max_class: 0.558061778545
	train_y_misclass: 0.018240025267
	train_y_nll: 0.0567165091634
	train_y_row_norms_max: 0.912941157818
	train_y_row_norms_mean: 0.402991384268
	train_y_row_norms_min: 0.0216932129115
	valid_h0_col_norms_max: 5.4267373085
	valid_h0_col_norms_mean: 3.48842096329
	valid_h0_col_norms_min: 1.96254551411
	valid_h0_row_norms_max: 5.41478538513
	valid_h0_row_norms_mean: 2.7299387455
	valid_h0_row_norms_min: 0.0784849375486
	valid_h1_col_norms_max: 5.06470775604
	valid_h1_col_norms_mean: 3.24842214584
	valid_h1_col_norms_min: 1.46678352356
	valid_h1_row_norms_max: 6.60853338242
	valid_h1_row_norms_mean: 4.6184220314
	valid_h1_row_norms_min: 2.71205830574
	valid_objective: 0.983159482479
	valid_term_0: 0.11690659076
	valid_term_1_weight_decay: 0.866252303123
	valid_y_col_norms_max: 4.76992559433
	valid_y_col_norms_mean: 4.27050018311
	valid_y_col_norms_min: 3.78093886375
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.981662929058
	valid_y_min_max_class: 0.533038794994
	valid_y_misclass: 0.0296999812126
	valid_y_nll: 0.11690659076
	valid_y_row_norms_max: 0.912945210934
	valid_y_row_norms_mean: 0.402993023396
	valid_y_row_norms_min: 0.0216930937022
Time this epoch: 3.286931 seconds
Monitoring step:
	Epochs seen: 13
	Batches seen: 6500
	Examples seen: 650000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 5.16044139862
	test_h0_col_norms_mean: 3.34162855148
	test_h0_col_norms_min: 1.86588740349
	test_h0_row_norms_max: 5.20599794388
	test_h0_row_norms_mean: 2.61515665054
	test_h0_row_norms_min: 0.0764672607183
	test_h1_col_norms_max: 4.8263502121
	test_h1_col_norms_mean: 3.09343934059
	test_h1_col_norms_min: 1.39710497856
	test_h1_row_norms_max: 6.2830324173
	test_h1_row_norms_mean: 4.39829969406
	test_h1_row_norms_min: 2.57847547531
	test_objective: 0.8922701478
	test_term_0: 0.102732278407
	test_term_1_weight_decay: 0.789536893368
	test_y_col_norms_max: 4.68681240082
	test_y_col_norms_mean: 4.23078680038
	test_y_col_norms_min: 3.78408479691
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.983542442322
	test_y_min_max_class: 0.593747019768
	test_y_misclass: 0.0273999907076
	test_y_nll: 0.102732278407
	test_y_row_norms_max: 0.966835141182
	test_y_row_norms_mean: 0.398707449436
	test_y_row_norms_min: 0.0218474734575
	train_h0_col_norms_max: 5.16042280197
	train_h0_col_norms_mean: 3.34164571762
	train_h0_col_norms_min: 1.86587870121
	train_h0_row_norms_max: 5.2060174942
	train_h0_row_norms_mean: 2.61516785622
	train_h0_row_norms_min: 0.0764675214887
	train_h1_col_norms_max: 4.82632637024
	train_h1_col_norms_mean: 3.09345006943
	train_h1_col_norms_min: 1.39711165428
	train_h1_row_norms_max: 6.28304100037
	train_h1_row_norms_mean: 4.39832401276
	train_h1_row_norms_min: 2.5784881115
	train_objective: 0.829474568367
	train_term_0: 0.0399360619485
	train_term_1_weight_decay: 0.789532542229
	train_y_col_norms_max: 4.6868262291
	train_y_col_norms_mean: 4.23076534271
	train_y_col_norms_min: 3.78406834602
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.987834095955
	train_y_min_max_class: 0.609752178192
	train_y_misclass: 0.0125200273469
	train_y_nll: 0.0399360619485
	train_y_row_norms_max: 0.96683973074
	train_y_row_norms_mean: 0.398709416389
	train_y_row_norms_min: 0.0218475684524
	valid_h0_col_norms_max: 5.16044139862
	valid_h0_col_norms_mean: 3.34162855148
	valid_h0_col_norms_min: 1.86588740349
	valid_h0_row_norms_max: 5.20599794388
	valid_h0_row_norms_mean: 2.61515665054
	valid_h0_row_norms_min: 0.0764672607183
	valid_h1_col_norms_max: 4.8263502121
	valid_h1_col_norms_mean: 3.09343934059
	valid_h1_col_norms_min: 1.39710497856
	valid_h1_row_norms_max: 6.2830324173
	valid_h1_row_norms_mean: 4.39829969406
	valid_h1_row_norms_min: 2.57847547531
	valid_objective: 0.903808116913
	valid_term_0: 0.114270374179
	valid_term_1_weight_decay: 0.789536893368
	valid_y_col_norms_max: 4.68681240082
	valid_y_col_norms_mean: 4.23078680038
	valid_y_col_norms_min: 3.78408479691
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.984707713127
	valid_y_min_max_class: 0.566586375237
	valid_y_misclass: 0.028899980709
	valid_y_nll: 0.114270374179
	valid_y_row_norms_max: 0.966835141182
	valid_y_row_norms_mean: 0.398707449436
	valid_y_row_norms_min: 0.0218474734575
Time this epoch: 3.373106 seconds
Monitoring step:
	Epochs seen: 14
	Batches seen: 7000
	Examples seen: 700000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 4.90614843369
	test_h0_col_norms_mean: 3.19672346115
	test_h0_col_norms_min: 1.77398645878
	test_h0_row_norms_max: 4.96580123901
	test_h0_row_norms_mean: 2.50189256668
	test_h0_row_norms_min: 0.0782802626491
	test_h1_col_norms_max: 4.59008312225
	test_h1_col_norms_mean: 2.94533538818
	test_h1_col_norms_min: 1.32907187939
	test_h1_row_norms_max: 5.97338581085
	test_h1_row_norms_mean: 4.18786859512
	test_h1_row_norms_min: 2.45148181915
	test_objective: 0.819695711136
	test_term_0: 0.100928872824
	test_term_1_weight_decay: 0.718766570091
	test_y_col_norms_max: 4.5962023735
	test_y_col_norms_mean: 4.16727113724
	test_y_col_norms_min: 3.66778349876
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.987643420696
	test_y_min_max_class: 0.622340202332
	test_y_misclass: 0.0252999924123
	test_y_nll: 0.100928872824
	test_y_row_norms_max: 0.958882212639
	test_y_row_norms_mean: 0.392418205738
	test_y_row_norms_min: 0.0207168832421
	train_h0_col_norms_max: 4.90617132187
	train_h0_col_norms_mean: 3.19671821594
	train_h0_col_norms_min: 1.77399635315
	train_h0_row_norms_max: 4.96578741074
	train_h0_row_norms_mean: 2.50188994408
	train_h0_row_norms_min: 0.0782798752189
	train_h1_col_norms_max: 4.5900592804
	train_h1_col_norms_mean: 2.94533443451
	train_h1_col_norms_min: 1.32906579971
	train_h1_row_norms_max: 5.97335529327
	train_h1_row_norms_mean: 4.18784570694
	train_h1_row_norms_min: 2.45149064064
	train_objective: 0.745359420776
	train_term_0: 0.0265923049301
	train_term_1_weight_decay: 0.718770325184
	train_y_col_norms_max: 4.5962138176
	train_y_col_norms_mean: 4.16729164124
	train_y_col_norms_min: 3.66780090332
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.990943193436
	train_y_min_max_class: 0.66484606266
	train_y_misclass: 0.00918001402169
	train_y_nll: 0.0265923049301
	train_y_row_norms_max: 0.95888376236
	train_y_row_norms_mean: 0.392418503761
	train_y_row_norms_min: 0.0207169353962
	valid_h0_col_norms_max: 4.90614843369
	valid_h0_col_norms_mean: 3.19672346115
	valid_h0_col_norms_min: 1.77398645878
	valid_h0_row_norms_max: 4.96580123901
	valid_h0_row_norms_mean: 2.50189256668
	valid_h0_row_norms_min: 0.0782802626491
	valid_h1_col_norms_max: 4.59008312225
	valid_h1_col_norms_mean: 2.94533538818
	valid_h1_col_norms_min: 1.32907187939
	valid_h1_row_norms_max: 5.97338581085
	valid_h1_row_norms_mean: 4.18786859512
	valid_h1_row_norms_min: 2.45148181915
	valid_objective: 0.827313005924
	valid_term_0: 0.108545988798
	valid_term_1_weight_decay: 0.718766570091
	valid_y_col_norms_max: 4.5962023735
	valid_y_col_norms_mean: 4.16727113724
	valid_y_col_norms_min: 3.66778349876
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.987360954285
	valid_y_min_max_class: 0.601630806923
	valid_y_misclass: 0.0260999873281
	valid_y_nll: 0.108545988798
	valid_y_row_norms_max: 0.958882212639
	valid_y_row_norms_mean: 0.392418205738
	valid_y_row_norms_min: 0.0207168832421
Time this epoch: 3.270202 seconds
Monitoring step:
	Epochs seen: 15
	Batches seen: 7500
	Examples seen: 750000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 4.66921758652
	test_h0_col_norms_mean: 3.05542969704
	test_h0_col_norms_min: 1.68660902977
	test_h0_row_norms_max: 4.75306463242
	test_h0_row_norms_mean: 2.39132237434
	test_h0_row_norms_min: 0.0765107423067
	test_h1_col_norms_max: 4.3710064888
	test_h1_col_norms_mean: 2.8040626049
	test_h1_col_norms_min: 1.26379609108
	test_h1_row_norms_max: 5.67917490005
	test_h1_row_norms_mean: 3.98712182045
	test_h1_row_norms_min: 2.33073425293
	test_objective: 0.737741053104
	test_term_0: 0.083799123764
	test_term_1_weight_decay: 0.653942167759
	test_y_col_norms_max: 4.54380941391
	test_y_col_norms_mean: 4.11158180237
	test_y_col_norms_min: 3.66944622993
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.987535178661
	test_y_min_max_class: 0.639883577824
	test_y_misclass: 0.0226999949664
	test_y_nll: 0.083799123764
	test_y_row_norms_max: 0.992673635483
	test_y_row_norms_mean: 0.386759877205
	test_y_row_norms_min: 0.0214904490858
	train_h0_col_norms_max: 4.66919612885
	train_h0_col_norms_mean: 3.05544400215
	train_h0_col_norms_min: 1.68661606312
	train_h0_row_norms_max: 4.75308895111
	train_h0_row_norms_mean: 2.39132881165
	train_h0_row_norms_min: 0.0765103250742
	train_h1_col_norms_max: 4.37101602554
	train_h1_col_norms_mean: 2.80404901505
	train_h1_col_norms_min: 1.26379692554
	train_h1_row_norms_max: 5.67914772034
	train_h1_row_norms_mean: 3.98710203171
	train_h1_row_norms_min: 2.33073854446
	train_objective: 0.66691416502
	train_term_0: 0.0129722505808
	train_term_1_weight_decay: 0.653943121433
	train_y_col_norms_max: 4.54378795624
	train_y_col_norms_mean: 4.11155748367
	train_y_col_norms_min: 3.66946792603
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.993395149708
	train_y_min_max_class: 0.715351760387
	train_y_misclass: 0.00405999692157
	train_y_nll: 0.0129722505808
	train_y_row_norms_max: 0.992678523064
	train_y_row_norms_mean: 0.386759728193
	train_y_row_norms_min: 0.0214903373271
	valid_h0_col_norms_max: 4.66921758652
	valid_h0_col_norms_mean: 3.05542969704
	valid_h0_col_norms_min: 1.68660902977
	valid_h0_row_norms_max: 4.75306463242
	valid_h0_row_norms_mean: 2.39132237434
	valid_h0_row_norms_min: 0.0765107423067
	valid_h1_col_norms_max: 4.3710064888
	valid_h1_col_norms_mean: 2.8040626049
	valid_h1_col_norms_min: 1.26379609108
	valid_h1_row_norms_max: 5.67917490005
	valid_h1_row_norms_mean: 3.98712182045
	valid_h1_row_norms_min: 2.33073425293
	valid_objective: 0.734254300594
	valid_term_0: 0.0803121104836
	valid_term_1_weight_decay: 0.653942167759
	valid_y_col_norms_max: 4.54380941391
	valid_y_col_norms_mean: 4.11158180237
	valid_y_col_norms_min: 3.66944622993
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.987278044224
	valid_y_min_max_class: 0.594924449921
	valid_y_misclass: 0.0219999905676
	valid_y_nll: 0.0803121104836
	valid_y_row_norms_max: 0.992673635483
	valid_y_row_norms_mean: 0.386759877205
	valid_y_row_norms_min: 0.0214904490858
Time this epoch: 3.281498 seconds
Monitoring step:
	Epochs seen: 16
	Batches seen: 8000
	Examples seen: 800000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 4.46374130249
	test_h0_col_norms_mean: 2.9173309803
	test_h0_col_norms_min: 1.60354030132
	test_h0_row_norms_max: 4.55304861069
	test_h0_row_norms_mean: 2.28333234787
	test_h0_row_norms_min: 0.0760971903801
	test_h1_col_norms_max: 4.15992879868
	test_h1_col_norms_mean: 2.66938233376
	test_h1_col_norms_min: 1.20336544514
	test_h1_row_norms_max: 5.3994641304
	test_h1_row_norms_mean: 3.79574894905
	test_h1_row_norms_min: 2.21593642235
	test_objective: 0.674262106419
	test_term_0: 0.0797682702541
	test_term_1_weight_decay: 0.594494223595
	test_y_col_norms_max: 4.41636514664
	test_y_col_norms_mean: 4.05042076111
	test_y_col_norms_min: 3.58171629906
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.986650168896
	test_y_min_max_class: 0.618766665459
	test_y_misclass: 0.0221999883652
	test_y_nll: 0.0797682702541
	test_y_row_norms_max: 0.987005531788
	test_y_row_norms_mean: 0.380280554295
	test_y_row_norms_min: 0.0215586218983
	train_h0_col_norms_max: 4.46375417709
	train_h0_col_norms_mean: 2.91732239723
	train_h0_col_norms_min: 1.60354280472
	train_h0_row_norms_max: 4.55305957794
	train_h0_row_norms_mean: 2.2833340168
	train_h0_row_norms_min: 0.0760968104005
	train_h1_col_norms_max: 4.159927845
	train_h1_col_norms_mean: 2.66937685013
	train_h1_col_norms_min: 1.20335972309
	train_h1_row_norms_max: 5.39946508408
	train_h1_row_norms_mean: 3.79574465752
	train_h1_row_norms_min: 2.21593785286
	train_objective: 0.604817152023
	train_term_0: 0.0103233894333
	train_term_1_weight_decay: 0.594495952129
	train_y_col_norms_max: 4.41635942459
	train_y_col_norms_mean: 4.05040311813
	train_y_col_norms_min: 3.58173537254
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.993671536446
	train_y_min_max_class: 0.732953190804
	train_y_misclass: 0.00291999848559
	train_y_nll: 0.0103233894333
	train_y_row_norms_max: 0.987000524998
	train_y_row_norms_mean: 0.380278617144
	train_y_row_norms_min: 0.0215585716069
	valid_h0_col_norms_max: 4.46374130249
	valid_h0_col_norms_mean: 2.9173309803
	valid_h0_col_norms_min: 1.60354030132
	valid_h0_row_norms_max: 4.55304861069
	valid_h0_row_norms_mean: 2.28333234787
	valid_h0_row_norms_min: 0.0760971903801
	valid_h1_col_norms_max: 4.15992879868
	valid_h1_col_norms_mean: 2.66938233376
	valid_h1_col_norms_min: 1.20336544514
	valid_h1_row_norms_max: 5.3994641304
	valid_h1_row_norms_mean: 3.79574894905
	valid_h1_row_norms_min: 2.21593642235
	valid_objective: 0.67914390564
	valid_term_0: 0.0846498459578
	valid_term_1_weight_decay: 0.594494223595
	valid_y_col_norms_max: 4.41636514664
	valid_y_col_norms_mean: 4.05042076111
	valid_y_col_norms_min: 3.58171629906
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.987098455429
	valid_y_min_max_class: 0.597771346569
	valid_y_misclass: 0.0203999932855
	valid_y_nll: 0.0846498459578
	valid_y_row_norms_max: 0.987005531788
	valid_y_row_norms_mean: 0.380280554295
	valid_y_row_norms_min: 0.0215586218983
Time this epoch: 3.317685 seconds
Monitoring step:
	Epochs seen: 17
	Batches seen: 8500
	Examples seen: 850000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 4.26407384872
	test_h0_col_norms_mean: 2.78463411331
	test_h0_col_norms_min: 1.52455866337
	test_h0_row_norms_max: 4.34796571732
	test_h0_row_norms_mean: 2.17960953712
	test_h0_row_norms_min: 0.0764012187719
	test_h1_col_norms_max: 3.95875430107
	test_h1_col_norms_mean: 2.54128909111
	test_h1_col_norms_min: 1.14473068714
	test_h1_row_norms_max: 5.13351964951
	test_h1_row_norms_mean: 3.61375975609
	test_h1_row_norms_min: 2.1067969799
	test_objective: 0.610892295837
	test_term_0: 0.0704278945923
	test_term_1_weight_decay: 0.540464937687
	test_y_col_norms_max: 4.3217663765
	test_y_col_norms_mean: 4.00428628922
	test_y_col_norms_min: 3.53744649887
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.988494455814
	test_y_min_max_class: 0.640034735203
	test_y_misclass: 0.0186999924481
	test_y_nll: 0.0704278945923
	test_y_row_norms_max: 0.994636058807
	test_y_row_norms_mean: 0.3746727705
	test_y_row_norms_min: 0.0214648172259
	train_h0_col_norms_max: 4.26409387589
	train_h0_col_norms_mean: 2.78462028503
	train_h0_col_norms_min: 1.52456390858
	train_h0_row_norms_max: 4.34795570374
	train_h0_row_norms_mean: 2.17961931229
	train_h0_row_norms_min: 0.0764016136527
	train_h1_col_norms_max: 3.95873188972
	train_h1_col_norms_mean: 2.54130077362
	train_h1_col_norms_min: 1.14473164082
	train_h1_row_norms_max: 5.13353729248
	train_h1_row_norms_mean: 3.61374282837
	train_h1_row_norms_min: 2.10678911209
	train_objective: 0.5477257967
	train_term_0: 0.007261632476
	train_term_1_weight_decay: 0.540465056896
	train_y_col_norms_max: 4.32178735733
	train_y_col_norms_mean: 4.00426435471
	train_y_col_norms_min: 3.5374417305
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.995666265488
	train_y_min_max_class: 0.789768993855
	train_y_misclass: 0.00194000091869
	train_y_nll: 0.007261632476
	train_y_row_norms_max: 0.994630157948
	train_y_row_norms_mean: 0.374674469233
	train_y_row_norms_min: 0.0214648637921
	valid_h0_col_norms_max: 4.26407384872
	valid_h0_col_norms_mean: 2.78463411331
	valid_h0_col_norms_min: 1.52455866337
	valid_h0_row_norms_max: 4.34796571732
	valid_h0_row_norms_mean: 2.17960953712
	valid_h0_row_norms_min: 0.0764012187719
	valid_h1_col_norms_max: 3.95875430107
	valid_h1_col_norms_mean: 2.54128909111
	valid_h1_col_norms_min: 1.14473068714
	valid_h1_row_norms_max: 5.13351964951
	valid_h1_row_norms_mean: 3.61375975609
	valid_h1_row_norms_min: 2.1067969799
	valid_objective: 0.617604732513
	valid_term_0: 0.0771402940154
	valid_term_1_weight_decay: 0.540464937687
	valid_y_col_norms_max: 4.3217663765
	valid_y_col_norms_mean: 4.00428628922
	valid_y_col_norms_min: 3.53744649887
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989329993725
	valid_y_min_max_class: 0.605314671993
	valid_y_misclass: 0.0208999905735
	valid_y_nll: 0.0771402940154
	valid_y_row_norms_max: 0.994636058807
	valid_y_row_norms_mean: 0.3746727705
	valid_y_row_norms_min: 0.0214648172259
Time this epoch: 3.269546 seconds
Monitoring step:
	Epochs seen: 18
	Batches seen: 9000
	Examples seen: 900000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 4.07507371902
	test_h0_col_norms_mean: 2.65655350685
	test_h0_col_norms_min: 1.44946885109
	test_h0_row_norms_max: 4.15038585663
	test_h0_row_norms_mean: 2.07941555977
	test_h0_row_norms_min: 0.0857979208231
	test_h1_col_norms_max: 3.77103662491
	test_h1_col_norms_mean: 2.41892313957
	test_h1_col_norms_min: 1.08750927448
	test_h1_row_norms_max: 4.88069534302
	test_h1_row_norms_mean: 3.43979096413
	test_h1_row_norms_min: 2.00302839279
	test_objective: 0.562726557255
	test_term_0: 0.0717355385423
	test_term_1_weight_decay: 0.490990847349
	test_y_col_norms_max: 4.28208780289
	test_y_col_norms_mean: 3.93249392509
	test_y_col_norms_min: 3.48496580124
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.990153551102
	test_y_min_max_class: 0.659395575523
	test_y_misclass: 0.0185999944806
	test_y_nll: 0.0717355385423
	test_y_row_norms_max: 0.942749202251
	test_y_row_norms_mean: 0.367405802011
	test_y_row_norms_min: 0.019349604845
	train_h0_col_norms_max: 4.07505750656
	train_h0_col_norms_mean: 2.65656781197
	train_h0_col_norms_min: 1.44946610928
	train_h0_row_norms_max: 4.15039014816
	train_h0_row_norms_mean: 2.07942199707
	train_h0_row_norms_min: 0.0857974886894
	train_h1_col_norms_max: 3.77104258537
	train_h1_col_norms_mean: 2.41892194748
	train_h1_col_norms_min: 1.08750891685
	train_h1_row_norms_max: 4.88067293167
	train_h1_row_norms_mean: 3.43978619576
	train_h1_row_norms_min: 2.00302529335
	train_objective: 0.496095150709
	train_term_0: 0.00510408030823
	train_term_1_weight_decay: 0.490992516279
	train_y_col_norms_max: 4.28208827972
	train_y_col_norms_mean: 3.93247795105
	train_y_col_norms_min: 3.48496460915
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996478378773
	train_y_min_max_class: 0.825236082077
	train_y_misclass: 0.00105999980588
	train_y_nll: 0.00510408030823
	train_y_row_norms_max: 0.942744672298
	train_y_row_norms_mean: 0.367404073477
	train_y_row_norms_min: 0.0193495322019
	valid_h0_col_norms_max: 4.07507371902
	valid_h0_col_norms_mean: 2.65655350685
	valid_h0_col_norms_min: 1.44946885109
	valid_h0_row_norms_max: 4.15038585663
	valid_h0_row_norms_mean: 2.07941555977
	valid_h0_row_norms_min: 0.0857979208231
	valid_h1_col_norms_max: 3.77103662491
	valid_h1_col_norms_mean: 2.41892313957
	valid_h1_col_norms_min: 1.08750927448
	valid_h1_row_norms_max: 4.88069534302
	valid_h1_row_norms_mean: 3.43979096413
	valid_h1_row_norms_min: 2.00302839279
	valid_objective: 0.568551659584
	valid_term_0: 0.0775607377291
	valid_term_1_weight_decay: 0.490990847349
	valid_y_col_norms_max: 4.28208780289
	valid_y_col_norms_mean: 3.93249392509
	valid_y_col_norms_min: 3.48496580124
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989968895912
	valid_y_min_max_class: 0.620238602161
	valid_y_misclass: 0.0208999887109
	valid_y_nll: 0.0775607377291
	valid_y_row_norms_max: 0.942749202251
	valid_y_row_norms_mean: 0.367405802011
	valid_y_row_norms_min: 0.019349604845
Time this epoch: 3.304628 seconds
Monitoring step:
	Epochs seen: 19
	Batches seen: 9500
	Examples seen: 950000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 3.88326621056
	test_h0_col_norms_mean: 2.53280711174
	test_h0_col_norms_min: 1.37807917595
	test_h0_row_norms_max: 3.96461653709
	test_h0_row_norms_mean: 1.98260319233
	test_h0_row_norms_min: 0.09099239856
	test_h1_col_norms_max: 3.58734297752
	test_h1_col_norms_mean: 2.30225491524
	test_h1_col_norms_min: 1.03453934193
	test_h1_row_norms_max: 4.64029741287
	test_h1_row_norms_mean: 3.27396249771
	test_h1_row_norms_min: 1.90437150002
	test_objective: 0.510573804379
	test_term_0: 0.0647685080767
	test_term_1_weight_decay: 0.445804834366
	test_y_col_norms_max: 4.17118215561
	test_y_col_norms_mean: 3.85513329506
	test_y_col_norms_min: 3.38714289665
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.989724636078
	test_y_min_max_class: 0.680286705494
	test_y_misclass: 0.0178999938071
	test_y_nll: 0.0647685080767
	test_y_row_norms_max: 0.922487914562
	test_y_row_norms_mean: 0.359323531389
	test_y_row_norms_min: 0.0180249232799
	train_h0_col_norms_max: 3.883248806
	train_h0_col_norms_mean: 2.53280425072
	train_h0_col_norms_min: 1.37807655334
	train_h0_row_norms_max: 3.96463823318
	train_h0_row_norms_mean: 1.982614398
	train_h0_row_norms_min: 0.090992718935
	train_h1_col_norms_max: 3.58735847473
	train_h1_col_norms_mean: 2.30224943161
	train_h1_col_norms_min: 1.03453481197
	train_h1_row_norms_max: 4.64032030106
	train_h1_row_norms_mean: 3.2739636898
	train_h1_row_norms_min: 1.90436935425
	train_objective: 0.449831366539
	train_term_0: 0.00402596499771
	train_term_1_weight_decay: 0.445802211761
	train_y_col_norms_max: 4.17116117477
	train_y_col_norms_mean: 3.85511660576
	train_y_col_norms_min: 3.3871281147
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996860563755
	train_y_min_max_class: 0.843070626259
	train_y_misclass: 0.000819999666419
	train_y_nll: 0.00402596499771
	train_y_row_norms_max: 0.922493100166
	train_y_row_norms_mean: 0.359325319529
	train_y_row_norms_min: 0.0180248413235
	valid_h0_col_norms_max: 3.88326621056
	valid_h0_col_norms_mean: 2.53280711174
	valid_h0_col_norms_min: 1.37807917595
	valid_h0_row_norms_max: 3.96461653709
	valid_h0_row_norms_mean: 1.98260319233
	valid_h0_row_norms_min: 0.09099239856
	valid_h1_col_norms_max: 3.58734297752
	valid_h1_col_norms_mean: 2.30225491524
	valid_h1_col_norms_min: 1.03453934193
	valid_h1_row_norms_max: 4.64029741287
	valid_h1_row_norms_mean: 3.27396249771
	valid_h1_row_norms_min: 1.90437150002
	valid_objective: 0.517447412014
	valid_term_0: 0.0716420337558
	valid_term_1_weight_decay: 0.445804834366
	valid_y_col_norms_max: 4.17118215561
	valid_y_col_norms_mean: 3.85513329506
	valid_y_col_norms_min: 3.38714289665
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.990103065968
	valid_y_min_max_class: 0.65864700079
	valid_y_misclass: 0.019399991259
	valid_y_nll: 0.0716420337558
	valid_y_row_norms_max: 0.922487914562
	valid_y_row_norms_mean: 0.359323531389
	valid_y_row_norms_min: 0.0180249232799
Time this epoch: 3.352973 seconds
Monitoring step:
	Epochs seen: 20
	Batches seen: 10000
	Examples seen: 1000000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 3.69830107689
	test_h0_col_norms_mean: 2.41490340233
	test_h0_col_norms_min: 1.31020438671
	test_h0_row_norms_max: 3.78174734116
	test_h0_row_norms_mean: 1.89037334919
	test_h0_row_norms_min: 0.0872991830111
	test_h1_col_norms_max: 3.41410470009
	test_h1_col_norms_mean: 2.19139242172
	test_h1_col_norms_min: 0.983708977699
	test_h1_row_norms_max: 4.41174936295
	test_h1_row_norms_mean: 3.11639881134
	test_h1_row_norms_min: 1.81057536602
	test_objective: 0.4696611166
	test_term_0: 0.064779728651
	test_term_1_weight_decay: 0.404881685972
	test_y_col_norms_max: 4.09565019608
	test_y_col_norms_mean: 3.78634428978
	test_y_col_norms_min: 3.3164498806
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.98881983757
	test_y_min_max_class: 0.653255581856
	test_y_misclass: 0.017599998042
	test_y_nll: 0.064779728651
	test_y_row_norms_max: 0.925839364529
	test_y_row_norms_mean: 0.351857930422
	test_y_row_norms_min: 0.0175057649612
	train_h0_col_norms_max: 3.69831848145
	train_h0_col_norms_mean: 2.41490244865
	train_h0_col_norms_min: 1.31020689011
	train_h0_row_norms_max: 3.78176569939
	train_h0_row_norms_mean: 1.89036512375
	train_h0_row_norms_min: 0.0872987210751
	train_h1_col_norms_max: 3.41411972046
	train_h1_col_norms_mean: 2.19141077995
	train_h1_col_norms_min: 0.983713150024
	train_h1_row_norms_max: 4.41172409058
	train_h1_row_norms_mean: 3.11639785767
	train_h1_row_norms_min: 1.81056690216
	train_objective: 0.409165471792
	train_term_0: 0.00428416905925
	train_term_1_weight_decay: 0.404880315065
	train_y_col_norms_max: 4.09566497803
	train_y_col_norms_mean: 3.78633141518
	train_y_col_norms_min: 3.31643605232
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996947467327
	train_y_min_max_class: 0.855619430542
	train_y_misclass: 0.000879999657627
	train_y_nll: 0.00428416905925
	train_y_row_norms_max: 0.925839066505
	train_y_row_norms_mean: 0.351859807968
	train_y_row_norms_min: 0.0175057388842
	valid_h0_col_norms_max: 3.69830107689
	valid_h0_col_norms_mean: 2.41490340233
	valid_h0_col_norms_min: 1.31020438671
	valid_h0_row_norms_max: 3.78174734116
	valid_h0_row_norms_mean: 1.89037334919
	valid_h0_row_norms_min: 0.0872991830111
	valid_h1_col_norms_max: 3.41410470009
	valid_h1_col_norms_mean: 2.19139242172
	valid_h1_col_norms_min: 0.983708977699
	valid_h1_row_norms_max: 4.41174936295
	valid_h1_row_norms_mean: 3.11639881134
	valid_h1_row_norms_min: 1.81057536602
	valid_objective: 0.475686132908
	valid_term_0: 0.0708047524095
	valid_term_1_weight_decay: 0.404881685972
	valid_y_col_norms_max: 4.09565019608
	valid_y_col_norms_mean: 3.78634428978
	valid_y_col_norms_min: 3.3164498806
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989249825478
	valid_y_min_max_class: 0.616850614548
	valid_y_misclass: 0.0192999914289
	valid_y_nll: 0.0708047524095
	valid_y_row_norms_max: 0.925839364529
	valid_y_row_norms_mean: 0.351857930422
	valid_y_row_norms_min: 0.0175057649612
Time this epoch: 3.278321 seconds
Monitoring step:
	Epochs seen: 21
	Batches seen: 10500
	Examples seen: 1050000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 3.52711653709
	test_h0_col_norms_mean: 2.30246567726
	test_h0_col_norms_min: 1.2456703186
	test_h0_row_norms_max: 3.60426926613
	test_h0_row_norms_mean: 1.80239653587
	test_h0_row_norms_min: 0.0854785442352
	test_h1_col_norms_max: 3.24995541573
	test_h1_col_norms_mean: 2.08604121208
	test_h1_col_norms_min: 0.935500979424
	test_h1_row_norms_max: 4.19445514679
	test_h1_row_norms_mean: 2.96661686897
	test_h1_row_norms_min: 1.72139751911
	test_objective: 0.435198038816
	test_term_0: 0.0674102455378
	test_term_1_weight_decay: 0.367787539959
	test_y_col_norms_max: 4.01598834991
	test_y_col_norms_mean: 3.72248363495
	test_y_col_norms_min: 3.24742627144
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.988867938519
	test_y_min_max_class: 0.670085370541
	test_y_misclass: 0.0185999963433
	test_y_nll: 0.0674102455378
	test_y_row_norms_max: 0.902759611607
	test_y_row_norms_mean: 0.344950795174
	test_y_row_norms_min: 0.0167198460549
	train_h0_col_norms_max: 3.52713274956
	train_h0_col_norms_mean: 2.30245995522
	train_h0_col_norms_min: 1.24567604065
	train_h0_row_norms_max: 3.60428500175
	train_h0_row_norms_mean: 1.80238819122
	train_h0_row_norms_min: 0.0854781419039
	train_h1_col_norms_max: 3.24996852875
	train_h1_col_norms_mean: 2.08604311943
	train_h1_col_norms_min: 0.935496866703
	train_h1_row_norms_max: 4.19447517395
	train_h1_row_norms_mean: 2.96663236618
	train_h1_row_norms_min: 1.72139537334
	train_objective: 0.372235387564
	train_term_0: 0.00444769486785
	train_term_1_weight_decay: 0.367789417505
	train_y_col_norms_max: 4.01601171494
	train_y_col_norms_mean: 3.7224612236
	train_y_col_norms_min: 3.24741005898
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996587693691
	train_y_min_max_class: 0.846141993999
	train_y_misclass: 0.000799999630544
	train_y_nll: 0.00444769486785
	train_y_row_norms_max: 0.902763843536
	train_y_row_norms_mean: 0.344951033592
	train_y_row_norms_min: 0.0167199298739
	valid_h0_col_norms_max: 3.52711653709
	valid_h0_col_norms_mean: 2.30246567726
	valid_h0_col_norms_min: 1.2456703186
	valid_h0_row_norms_max: 3.60426926613
	valid_h0_row_norms_mean: 1.80239653587
	valid_h0_row_norms_min: 0.0854785442352
	valid_h1_col_norms_max: 3.24995541573
	valid_h1_col_norms_mean: 2.08604121208
	valid_h1_col_norms_min: 0.935500979424
	valid_h1_row_norms_max: 4.19445514679
	valid_h1_row_norms_mean: 2.96661686897
	valid_h1_row_norms_min: 1.72139751911
	valid_objective: 0.439748078585
	valid_term_0: 0.0719603598118
	valid_term_1_weight_decay: 0.367787539959
	valid_y_col_norms_max: 4.01598834991
	valid_y_col_norms_mean: 3.72248363495
	valid_y_col_norms_min: 3.24742627144
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989000380039
	valid_y_min_max_class: 0.610602736473
	valid_y_misclass: 0.0192999914289
	valid_y_nll: 0.0719603598118
	valid_y_row_norms_max: 0.902759611607
	valid_y_row_norms_mean: 0.344950795174
	valid_y_row_norms_min: 0.0167198460549
Time this epoch: 3.292022 seconds
Monitoring step:
	Epochs seen: 22
	Batches seen: 11000
	Examples seen: 1100000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 3.36697125435
	test_h0_col_norms_mean: 2.19646573067
	test_h0_col_norms_min: 1.18431913853
	test_h0_row_norms_max: 3.44316983223
	test_h0_row_norms_mean: 1.71948647499
	test_h0_row_norms_min: 0.0820252001286
	test_h1_col_norms_max: 3.09565114975
	test_h1_col_norms_mean: 1.98627471924
	test_h1_col_norms_min: 0.889862000942
	test_h1_row_norms_max: 3.98788499832
	test_h1_row_norms_mean: 2.82480931282
	test_h1_row_norms_min: 1.63661336899
	test_objective: 0.394841223955
	test_term_0: 0.0604076348245
	test_term_1_weight_decay: 0.334433555603
	test_y_col_norms_max: 4.01139307022
	test_y_col_norms_mean: 3.67662405968
	test_y_col_norms_min: 3.18460655212
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.989187180996
	test_y_min_max_class: 0.643190681934
	test_y_misclass: 0.0180999971926
	test_y_nll: 0.0604076348245
	test_y_row_norms_max: 0.901866018772
	test_y_row_norms_mean: 0.339687138796
	test_y_row_norms_min: 0.01714236103
	train_h0_col_norms_max: 3.36695551872
	train_h0_col_norms_mean: 2.19647264481
	train_h0_col_norms_min: 1.18431949615
	train_h0_row_norms_max: 3.44315481186
	train_h0_row_norms_mean: 1.7194788456
	train_h0_row_norms_min: 0.0820248499513
	train_h1_col_norms_max: 3.09563612938
	train_h1_col_norms_mean: 1.98626804352
	train_h1_col_norms_min: 0.889865934849
	train_h1_row_norms_max: 3.98790216446
	train_h1_row_norms_mean: 2.82479405403
	train_h1_row_norms_min: 1.63662087917
	train_objective: 0.337020277977
	train_term_0: 0.00258666928858
	train_term_1_weight_decay: 0.334432244301
	train_y_col_norms_max: 4.01139307022
	train_y_col_norms_mean: 3.67662858963
	train_y_col_norms_min: 3.18459391594
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.997638344765
	train_y_min_max_class: 0.894991517067
	train_y_misclass: 0.000179999973625
	train_y_nll: 0.00258666928858
	train_y_row_norms_max: 0.901870131493
	train_y_row_norms_mean: 0.339686959982
	train_y_row_norms_min: 0.0171423424035
	valid_h0_col_norms_max: 3.36697125435
	valid_h0_col_norms_mean: 2.19646573067
	valid_h0_col_norms_min: 1.18431913853
	valid_h0_row_norms_max: 3.44316983223
	valid_h0_row_norms_mean: 1.71948647499
	valid_h0_row_norms_min: 0.0820252001286
	valid_h1_col_norms_max: 3.09565114975
	valid_h1_col_norms_mean: 1.98627471924
	valid_h1_col_norms_min: 0.889862000942
	valid_h1_row_norms_max: 3.98788499832
	valid_h1_row_norms_mean: 2.82480931282
	valid_h1_row_norms_min: 1.63661336899
	valid_objective: 0.399684429169
	valid_term_0: 0.065250813961
	valid_term_1_weight_decay: 0.334433555603
	valid_y_col_norms_max: 4.01139307022
	valid_y_col_norms_mean: 3.67662405968
	valid_y_col_norms_min: 3.18460655212
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989832878113
	valid_y_min_max_class: 0.622855961323
	valid_y_misclass: 0.0177999921143
	valid_y_nll: 0.065250813961
	valid_y_row_norms_max: 0.901866018772
	valid_y_row_norms_mean: 0.339687138796
	valid_y_row_norms_min: 0.01714236103
Time this epoch: 3.278771 seconds
Monitoring step:
	Epochs seen: 23
	Batches seen: 11500
	Examples seen: 1150000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 3.2081348896
	test_h0_col_norms_mean: 2.09370923042
	test_h0_col_norms_min: 1.12598621845
	test_h0_row_norms_max: 3.28446102142
	test_h0_row_norms_mean: 1.63902020454
	test_h0_row_norms_min: 0.0778670459986
	test_h1_col_norms_max: 2.94689941406
	test_h1_col_norms_mean: 1.89091038704
	test_h1_col_norms_min: 0.846523106098
	test_h1_row_norms_max: 3.79147481918
	test_h1_row_norms_mean: 2.68924212456
	test_h1_row_norms_min: 1.55600595474
	test_objective: 0.359032511711
	test_term_0: 0.0552162267268
	test_term_1_weight_decay: 0.303816497326
	test_y_col_norms_max: 3.92041349411
	test_y_col_norms_mean: 3.61436057091
	test_y_col_norms_min: 3.12086963654
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.989588081837
	test_y_min_max_class: 0.669209182262
	test_y_misclass: 0.016299996525
	test_y_nll: 0.0552162267268
	test_y_row_norms_max: 0.909698069096
	test_y_row_norms_mean: 0.333013266325
	test_y_row_norms_min: 0.0162505507469
	train_h0_col_norms_max: 3.20815110207
	train_h0_col_norms_mean: 2.09371638298
	train_h0_col_norms_min: 1.12598991394
	train_h0_row_norms_max: 3.28445625305
	train_h0_row_norms_mean: 1.63901221752
	train_h0_row_norms_min: 0.0778671503067
	train_h1_col_norms_max: 2.94688630104
	train_h1_col_norms_mean: 1.89090168476
	train_h1_col_norms_min: 0.846526682377
	train_h1_row_norms_max: 3.79145789146
	train_h1_row_norms_mean: 2.68924236298
	train_h1_row_norms_min: 1.55599808693
	train_objective: 0.306422680616
	train_term_0: 0.00260642380454
	train_term_1_weight_decay: 0.303818255663
	train_y_col_norms_max: 3.92042994499
	train_y_col_norms_mean: 3.61433935165
	train_y_col_norms_min: 3.12087059021
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.997653722763
	train_y_min_max_class: 0.897151112556
	train_y_misclass: 0.000119999996969
	train_y_nll: 0.00260642380454
	train_y_row_norms_max: 0.909693837166
	train_y_row_norms_mean: 0.333012223244
	train_y_row_norms_min: 0.0162506196648
	valid_h0_col_norms_max: 3.2081348896
	valid_h0_col_norms_mean: 2.09370923042
	valid_h0_col_norms_min: 1.12598621845
	valid_h0_row_norms_max: 3.28446102142
	valid_h0_row_norms_mean: 1.63902020454
	valid_h0_row_norms_min: 0.0778670459986
	valid_h1_col_norms_max: 2.94689941406
	valid_h1_col_norms_mean: 1.89091038704
	valid_h1_col_norms_min: 0.846523106098
	valid_h1_row_norms_max: 3.79147481918
	valid_h1_row_norms_mean: 2.68924212456
	valid_h1_row_norms_min: 1.55600595474
	valid_objective: 0.370760649443
	valid_term_0: 0.0669444948435
	valid_term_1_weight_decay: 0.303816497326
	valid_y_col_norms_max: 3.92041349411
	valid_y_col_norms_mean: 3.61436057091
	valid_y_col_norms_min: 3.12086963654
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989882349968
	valid_y_min_max_class: 0.639726042747
	valid_y_misclass: 0.0178999956697
	valid_y_nll: 0.0669444948435
	valid_y_row_norms_max: 0.909698069096
	valid_y_row_norms_mean: 0.333013266325
	valid_y_row_norms_min: 0.0162505507469
Time this epoch: 3.283699 seconds
Monitoring step:
	Epochs seen: 24
	Batches seen: 12000
	Examples seen: 1200000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 3.0553176403
	test_h0_col_norms_mean: 1.99559020996
	test_h0_col_norms_min: 1.07052779198
	test_h0_row_norms_max: 3.1363966465
	test_h0_row_norms_mean: 1.56222510338
	test_h0_row_norms_min: 0.0742355883121
	test_h1_col_norms_max: 2.80656790733
	test_h1_col_norms_mean: 1.80023908615
	test_h1_col_norms_min: 0.805699706078
	test_h1_row_norms_max: 3.60472822189
	test_h1_row_norms_mean: 2.5603313446
	test_h1_row_norms_min: 1.47936725616
	test_objective: 0.333813428879
	test_term_0: 0.0577764734626
	test_term_1_weight_decay: 0.276036947966
	test_y_col_norms_max: 3.85618805885
	test_y_col_norms_mean: 3.55425548553
	test_y_col_norms_min: 3.04648113251
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.989487826824
	test_y_min_max_class: 0.649445652962
	test_y_misclass: 0.0160999950022
	test_y_nll: 0.0577764734626
	test_y_row_norms_max: 0.918738484383
	test_y_row_norms_mean: 0.326478481293
	test_y_row_norms_min: 0.0155664272606
	train_h0_col_norms_max: 3.055331707
	train_h0_col_norms_mean: 1.99557840824
	train_h0_col_norms_min: 1.07052719593
	train_h0_row_norms_max: 3.1363966465
	train_h0_row_norms_mean: 1.56223297119
	train_h0_row_norms_min: 0.0742354020476
	train_h1_col_norms_max: 2.80655431747
	train_h1_col_norms_mean: 1.80023896694
	train_h1_col_norms_min: 0.805703580379
	train_h1_row_norms_max: 3.60474324226
	train_h1_row_norms_mean: 2.56033945084
	train_h1_row_norms_min: 1.47937119007
	train_objective: 0.278264194727
	train_term_0: 0.00222728447989
	train_term_1_weight_decay: 0.276037305593
	train_y_col_norms_max: 3.85618400574
	train_y_col_norms_mean: 3.55425071716
	train_y_col_norms_min: 3.04648280144
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.997917592525
	train_y_min_max_class: 0.919997572899
	train_y_misclass: 7.9999997979e-05
	train_y_nll: 0.00222728447989
	train_y_row_norms_max: 0.918739795685
	train_y_row_norms_mean: 0.326479077339
	train_y_row_norms_min: 0.0155664980412
	valid_h0_col_norms_max: 3.0553176403
	valid_h0_col_norms_mean: 1.99559020996
	valid_h0_col_norms_min: 1.07052779198
	valid_h0_row_norms_max: 3.1363966465
	valid_h0_row_norms_mean: 1.56222510338
	valid_h0_row_norms_min: 0.0742355883121
	valid_h1_col_norms_max: 2.80656790733
	valid_h1_col_norms_mean: 1.80023908615
	valid_h1_col_norms_min: 0.805699706078
	valid_h1_row_norms_max: 3.60472822189
	valid_h1_row_norms_mean: 2.5603313446
	valid_h1_row_norms_min: 1.47936725616
	valid_objective: 0.338611006737
	valid_term_0: 0.062574096024
	valid_term_1_weight_decay: 0.276036947966
	valid_y_col_norms_max: 3.85618805885
	valid_y_col_norms_mean: 3.55425548553
	valid_y_col_norms_min: 3.04648113251
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.98962688446
	valid_y_min_max_class: 0.629032611847
	valid_y_misclass: 0.0176999941468
	valid_y_nll: 0.062574096024
	valid_y_row_norms_max: 0.918738484383
	valid_y_row_norms_mean: 0.326478481293
	valid_y_row_norms_min: 0.0155664272606
Time this epoch: 3.319413 seconds
Monitoring step:
	Epochs seen: 25
	Batches seen: 12500
	Examples seen: 1250000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.91628909111
	test_h0_col_norms_mean: 1.90174818039
	test_h0_col_norms_min: 1.01780152321
	test_h0_row_norms_max: 2.98749780655
	test_h0_row_norms_mean: 1.48878085613
	test_h0_row_norms_min: 0.0709456577897
	test_h1_col_norms_max: 2.6732199192
	test_h1_col_norms_mean: 1.71399199963
	test_h1_col_norms_min: 0.766523241997
	test_h1_row_norms_max: 3.42718911171
	test_h1_row_norms_mean: 2.43771839142
	test_h1_row_norms_min: 1.40650296211
	test_objective: 0.305504858494
	test_term_0: 0.0546990483999
	test_term_1_weight_decay: 0.2508058846
	test_y_col_norms_max: 3.78892588615
	test_y_col_norms_mean: 3.49624156952
	test_y_col_norms_min: 3.00044965744
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.989326953888
	test_y_min_max_class: 0.664230465889
	test_y_misclass: 0.0153999980539
	test_y_nll: 0.0546990483999
	test_y_row_norms_max: 0.914359211922
	test_y_row_norms_mean: 0.320046216249
	test_y_row_norms_min: 0.0148301701993
	train_h0_col_norms_max: 2.9162979126
	train_h0_col_norms_mean: 1.90174603462
	train_h0_col_norms_min: 1.01780331135
	train_h0_row_norms_max: 2.98750209808
	train_h0_row_norms_mean: 1.48878598213
	train_h0_row_norms_min: 0.0709457397461
	train_h1_col_norms_max: 2.67322587967
	train_h1_col_norms_mean: 1.71399140358
	train_h1_col_norms_min: 0.766523063183
	train_h1_row_norms_max: 3.42717552185
	train_h1_row_norms_mean: 2.4377117157
	train_h1_row_norms_min: 1.40649604797
	train_objective: 0.252682715654
	train_term_0: 0.00187695620116
	train_term_1_weight_decay: 0.250807106495
	train_y_col_norms_max: 3.78894209862
	train_y_col_norms_mean: 3.49625706673
	train_y_col_norms_min: 3.00046014786
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.998175680637
	train_y_min_max_class: 0.935915350914
	train_y_misclass: 0.0
	train_y_nll: 0.00187695620116
	train_y_row_norms_max: 0.914363384247
	train_y_row_norms_mean: 0.320046842098
	train_y_row_norms_min: 0.0148302586749
	valid_h0_col_norms_max: 2.91628909111
	valid_h0_col_norms_mean: 1.90174818039
	valid_h0_col_norms_min: 1.01780152321
	valid_h0_row_norms_max: 2.98749780655
	valid_h0_row_norms_mean: 1.48878085613
	valid_h0_row_norms_min: 0.0709456577897
	valid_h1_col_norms_max: 2.6732199192
	valid_h1_col_norms_mean: 1.71399199963
	valid_h1_col_norms_min: 0.766523241997
	valid_h1_row_norms_max: 3.42718911171
	valid_h1_row_norms_mean: 2.43771839142
	valid_h1_row_norms_min: 1.40650296211
	valid_objective: 0.310465872288
	valid_term_0: 0.0596601851285
	valid_term_1_weight_decay: 0.2508058846
	valid_y_col_norms_max: 3.78892588615
	valid_y_col_norms_mean: 3.49624156952
	valid_y_col_norms_min: 3.00044965744
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989892303944
	valid_y_min_max_class: 0.631849765778
	valid_y_misclass: 0.0163999944925
	valid_y_nll: 0.0596601851285
	valid_y_row_norms_max: 0.914359211922
	valid_y_row_norms_mean: 0.320046216249
	valid_y_row_norms_min: 0.0148301701993
Time this epoch: 3.332601 seconds
Monitoring step:
	Epochs seen: 26
	Batches seen: 13000
	Examples seen: 1300000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.78018331528
	test_h0_col_norms_mean: 1.81279492378
	test_h0_col_norms_min: 0.967669785023
	test_h0_row_norms_max: 2.84347510338
	test_h0_row_norms_mean: 1.41918671131
	test_h0_row_norms_min: 0.0677683353424
	test_h1_col_norms_max: 2.54604840279
	test_h1_col_norms_mean: 1.63219892979
	test_h1_col_norms_min: 0.7294241786
	test_h1_row_norms_max: 3.26415896416
	test_h1_row_norms_mean: 2.32143187523
	test_h1_row_norms_min: 1.33722984791
	test_objective: 0.282132327557
	test_term_0: 0.0540966317058
	test_term_1_weight_decay: 0.228035539389
	test_y_col_norms_max: 3.75175428391
	test_y_col_norms_mean: 3.44715094566
	test_y_col_norms_min: 2.93773794174
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.989186286926
	test_y_min_max_class: 0.657333433628
	test_y_misclass: 0.0153999971226
	test_y_nll: 0.0540966317058
	test_y_row_norms_max: 0.922168970108
	test_y_row_norms_mean: 0.314452946186
	test_y_row_norms_min: 0.0141052464023
	train_h0_col_norms_max: 2.7801964283
	train_h0_col_norms_mean: 1.81280255318
	train_h0_col_norms_min: 0.967675149441
	train_h0_row_norms_max: 2.84348917007
	train_h0_row_norms_mean: 1.41918671131
	train_h0_row_norms_min: 0.0677684471011
	train_h1_col_norms_max: 2.54606103897
	train_h1_col_norms_mean: 1.63219988346
	train_h1_col_norms_min: 0.729421555996
	train_h1_row_norms_max: 3.26416134834
	train_h1_row_norms_mean: 2.3214328289
	train_h1_row_norms_min: 1.33723008633
	train_objective: 0.230251327157
	train_term_0: 0.00221558846533
	train_term_1_weight_decay: 0.228034421802
	train_y_col_norms_max: 3.75175452232
	train_y_col_norms_mean: 3.44716668129
	train_y_col_norms_min: 2.93775320053
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.997844338417
	train_y_min_max_class: 0.929183900356
	train_y_misclass: 0.0
	train_y_nll: 0.00221558846533
	train_y_row_norms_max: 0.922172784805
	train_y_row_norms_mean: 0.314453363419
	train_y_row_norms_min: 0.0141053134575
	valid_h0_col_norms_max: 2.78018331528
	valid_h0_col_norms_mean: 1.81279492378
	valid_h0_col_norms_min: 0.967669785023
	valid_h0_row_norms_max: 2.84347510338
	valid_h0_row_norms_mean: 1.41918671131
	valid_h0_row_norms_min: 0.0677683353424
	valid_h1_col_norms_max: 2.54604840279
	valid_h1_col_norms_mean: 1.63219892979
	valid_h1_col_norms_min: 0.7294241786
	valid_h1_row_norms_max: 3.26415896416
	valid_h1_row_norms_mean: 2.32143187523
	valid_h1_row_norms_min: 1.33722984791
	valid_objective: 0.287775874138
	valid_term_0: 0.0597402378917
	valid_term_1_weight_decay: 0.228035539389
	valid_y_col_norms_max: 3.75175428391
	valid_y_col_norms_mean: 3.44715094566
	valid_y_col_norms_min: 2.93773794174
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989176094532
	valid_y_min_max_class: 0.624788343906
	valid_y_misclass: 0.0166999921203
	valid_y_nll: 0.0597402378917
	valid_y_row_norms_max: 0.922168970108
	valid_y_row_norms_mean: 0.314452946186
	valid_y_row_norms_min: 0.0141052464023
Time this epoch: 3.284030 seconds
Monitoring step:
	Epochs seen: 27
	Batches seen: 13500
	Examples seen: 1350000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.65619587898
	test_h0_col_norms_mean: 1.72903525829
	test_h0_col_norms_min: 0.920009553432
	test_h0_row_norms_max: 2.71527957916
	test_h0_row_norms_mean: 1.3536605835
	test_h0_row_norms_min: 0.0648870319128
	test_h1_col_norms_max: 2.42550611496
	test_h1_col_norms_mean: 1.55485773087
	test_h1_col_norms_min: 0.694191396236
	test_h1_row_norms_max: 3.11506414413
	test_h1_row_norms_mean: 2.21147465706
	test_h1_row_norms_min: 1.27136671543
	test_objective: 0.261537849903
	test_term_0: 0.0539344884455
	test_term_1_weight_decay: 0.207603096962
	test_y_col_norms_max: 3.72367501259
	test_y_col_norms_mean: 3.41610884666
	test_y_col_norms_min: 2.90950012207
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.98896753788
	test_y_min_max_class: 0.650016546249
	test_y_misclass: 0.0154999988154
	test_y_nll: 0.0539344884455
	test_y_row_norms_max: 0.928149938583
	test_y_row_norms_mean: 0.310383200645
	test_y_row_norms_min: 0.0134189818054
	train_h0_col_norms_max: 2.6562101841
	train_h0_col_norms_mean: 1.72902774811
	train_h0_col_norms_min: 0.920005261898
	train_h0_row_norms_max: 2.71527171135
	train_h0_row_norms_mean: 1.35366332531
	train_h0_row_norms_min: 0.0648870840669
	train_h1_col_norms_max: 2.42551159859
	train_h1_col_norms_mean: 1.5548504591
	train_h1_col_norms_min: 0.694188058376
	train_h1_row_norms_max: 3.1150598526
	train_h1_row_norms_mean: 2.21146249771
	train_h1_row_norms_min: 1.27136695385
	train_objective: 0.20999661088
	train_term_0: 0.002393146744
	train_term_1_weight_decay: 0.207603022456
	train_y_col_norms_max: 3.7236533165
	train_y_col_norms_mean: 3.41609406471
	train_y_col_norms_min: 2.90950846672
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.997686505318
	train_y_min_max_class: 0.923012793064
	train_y_misclass: 1.99999994948e-05
	train_y_nll: 0.002393146744
	train_y_row_norms_max: 0.928141772747
	train_y_row_norms_mean: 0.310382217169
	train_y_row_norms_min: 0.0134189641103
	valid_h0_col_norms_max: 2.65619587898
	valid_h0_col_norms_mean: 1.72903525829
	valid_h0_col_norms_min: 0.920009553432
	valid_h0_row_norms_max: 2.71527957916
	valid_h0_row_norms_mean: 1.3536605835
	valid_h0_row_norms_min: 0.0648870319128
	valid_h1_col_norms_max: 2.42550611496
	valid_h1_col_norms_mean: 1.55485773087
	valid_h1_col_norms_min: 0.694191396236
	valid_h1_row_norms_max: 3.11506414413
	valid_h1_row_norms_mean: 2.21147465706
	valid_h1_row_norms_min: 1.27136671543
	valid_objective: 0.268219769001
	valid_term_0: 0.0606164671481
	valid_term_1_weight_decay: 0.207603096962
	valid_y_col_norms_max: 3.72367501259
	valid_y_col_norms_mean: 3.41610884666
	valid_y_col_norms_min: 2.90950012207
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989122092724
	valid_y_min_max_class: 0.619553089142
	valid_y_misclass: 0.0165999922901
	valid_y_nll: 0.0606164671481
	valid_y_row_norms_max: 0.928149938583
	valid_y_row_norms_mean: 0.310383200645
	valid_y_row_norms_min: 0.0134189818054
Time this epoch: 3.261071 seconds
Monitoring step:
	Epochs seen: 28
	Batches seen: 14000
	Examples seen: 1400000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.53800392151
	test_h0_col_norms_mean: 1.65007030964
	test_h0_col_norms_min: 0.874696552753
	test_h0_row_norms_max: 2.59223008156
	test_h0_row_norms_mean: 1.29187560081
	test_h0_row_norms_min: 0.0620772130787
	test_h1_col_norms_max: 2.31092762947
	test_h1_col_norms_mean: 1.48166322708
	test_h1_col_norms_min: 0.660871863365
	test_h1_row_norms_max: 2.96952915192
	test_h1_row_norms_mean: 2.10743236542
	test_h1_row_norms_min: 1.20874655247
	test_objective: 0.244267836213
	test_term_0: 0.0550341755152
	test_term_1_weight_decay: 0.189233824611
	test_y_col_norms_max: 3.7156329155
	test_y_col_norms_mean: 3.3953909874
	test_y_col_norms_min: 2.88775634766
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.988679349422
	test_y_min_max_class: 0.66010850668
	test_y_misclass: 0.015999995172
	test_y_nll: 0.0550341755152
	test_y_row_norms_max: 0.940686166286
	test_y_row_norms_mean: 0.307195395231
	test_y_row_norms_min: 0.0127685274929
	train_h0_col_norms_max: 2.53800177574
	train_h0_col_norms_mean: 1.65007722378
	train_h0_col_norms_min: 0.874697685242
	train_h0_row_norms_max: 2.59221696854
	train_h0_row_norms_mean: 1.29187226295
	train_h0_row_norms_min: 0.0620768405497
	train_h1_col_norms_max: 2.3109228611
	train_h1_col_norms_mean: 1.48165631294
	train_h1_col_norms_min: 0.660868704319
	train_h1_row_norms_max: 2.96951341629
	train_h1_row_norms_mean: 2.10743737221
	train_h1_row_norms_min: 1.208745718
	train_objective: 0.19228720665
	train_term_0: 0.00305363954976
	train_term_1_weight_decay: 0.18923483789
	train_y_col_norms_max: 3.71563386917
	train_y_col_norms_mean: 3.39540982246
	train_y_col_norms_min: 2.88775682449
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.99712729454
	train_y_min_max_class: 0.899743020535
	train_y_misclass: 9.99999974738e-05
	train_y_nll: 0.00305363954976
	train_y_row_norms_max: 0.940682113171
	train_y_row_norms_mean: 0.307196319103
	train_y_row_norms_min: 0.0127684678882
	valid_h0_col_norms_max: 2.53800392151
	valid_h0_col_norms_mean: 1.65007030964
	valid_h0_col_norms_min: 0.874696552753
	valid_h0_row_norms_max: 2.59223008156
	valid_h0_row_norms_mean: 1.29187560081
	valid_h0_row_norms_min: 0.0620772130787
	valid_h1_col_norms_max: 2.31092762947
	valid_h1_col_norms_mean: 1.48166322708
	valid_h1_col_norms_min: 0.660871863365
	valid_h1_row_norms_max: 2.96952915192
	valid_h1_row_norms_mean: 2.10743236542
	valid_h1_row_norms_min: 1.20874655247
	valid_objective: 0.250370264053
	valid_term_0: 0.0611366219819
	valid_term_1_weight_decay: 0.189233824611
	valid_y_col_norms_max: 3.7156329155
	valid_y_col_norms_mean: 3.3953909874
	valid_y_col_norms_min: 2.88775634766
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.98891800642
	valid_y_min_max_class: 0.613278985023
	valid_y_misclass: 0.0181999895722
	valid_y_nll: 0.0611366219819
	valid_y_row_norms_max: 0.940686166286
	valid_y_row_norms_mean: 0.307195395231
	valid_y_row_norms_min: 0.0127685274929
Time this epoch: 3.281839 seconds
Monitoring step:
	Epochs seen: 29
	Batches seen: 14500
	Examples seen: 1450000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.42693471909
	test_h0_col_norms_mean: 1.57674181461
	test_h0_col_norms_min: 0.831614494324
	test_h0_row_norms_max: 2.47894763947
	test_h0_row_norms_mean: 1.23453938961
	test_h0_row_norms_min: 0.0597868897021
	test_h1_col_norms_max: 2.20548892021
	test_h1_col_norms_mean: 1.41274940968
	test_h1_col_norms_min: 0.629146695137
	test_h1_row_norms_max: 2.83711600304
	test_h1_row_norms_mean: 2.00946569443
	test_h1_row_norms_min: 1.14921236038
	test_objective: 0.231429338455
	test_term_0: 0.0585358664393
	test_term_1_weight_decay: 0.17289365828
	test_y_col_norms_max: 3.72800803185
	test_y_col_norms_mean: 3.39454960823
	test_y_col_norms_min: 2.87621188164
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.988135576248
	test_y_min_max_class: 0.641624808311
	test_y_misclass: 0.0174999944866
	test_y_nll: 0.0585358664393
	test_y_row_norms_max: 0.94206482172
	test_y_row_norms_mean: 0.305722147226
	test_y_row_norms_min: 0.0122694317251
	train_h0_col_norms_max: 2.42693591118
	train_h0_col_norms_mean: 1.57673621178
	train_h0_col_norms_min: 0.831611156464
	train_h0_row_norms_max: 2.47895431519
	train_h0_row_norms_mean: 1.23453593254
	train_h0_row_norms_min: 0.0597870908678
	train_h1_col_norms_max: 2.20549035072
	train_h1_col_norms_mean: 1.41274940968
	train_h1_col_norms_min: 0.629147231579
	train_h1_row_norms_max: 2.83710312843
	train_h1_row_norms_mean: 2.0094628334
	train_h1_row_norms_min: 1.14921247959
	train_objective: 0.176759794354
	train_term_0: 0.00386637193151
	train_term_1_weight_decay: 0.172893241048
	train_y_col_norms_max: 3.72802448273
	train_y_col_norms_mean: 3.39456868172
	train_y_col_norms_min: 2.87620210648
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996502935886
	train_y_min_max_class: 0.867810547352
	train_y_misclass: 0.000219999958063
	train_y_nll: 0.00386637193151
	train_y_row_norms_max: 0.942059278488
	train_y_row_norms_mean: 0.305722266436
	train_y_row_norms_min: 0.012269385159
	valid_h0_col_norms_max: 2.42693471909
	valid_h0_col_norms_mean: 1.57674181461
	valid_h0_col_norms_min: 0.831614494324
	valid_h0_row_norms_max: 2.47894763947
	valid_h0_row_norms_mean: 1.23453938961
	valid_h0_row_norms_min: 0.0597868897021
	valid_h1_col_norms_max: 2.20548892021
	valid_h1_col_norms_mean: 1.41274940968
	valid_h1_col_norms_min: 0.629146695137
	valid_h1_row_norms_max: 2.83711600304
	valid_h1_row_norms_mean: 2.00946569443
	valid_h1_row_norms_min: 1.14921236038
	valid_objective: 0.233349367976
	valid_term_0: 0.0604558549821
	valid_term_1_weight_decay: 0.17289365828
	valid_y_col_norms_max: 3.72800803185
	valid_y_col_norms_mean: 3.39454960823
	valid_y_col_norms_min: 2.87621188164
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.988323628902
	valid_y_min_max_class: 0.623263895512
	valid_y_misclass: 0.0180999934673
	valid_y_nll: 0.0604558549821
	valid_y_row_norms_max: 0.94206482172
	valid_y_row_norms_mean: 0.305722147226
	valid_y_row_norms_min: 0.0122694317251
Time this epoch: 3.305740 seconds
Monitoring step:
	Epochs seen: 30
	Batches seen: 15000
	Examples seen: 1500000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.32734417915
	test_h0_col_norms_mean: 1.50859749317
	test_h0_col_norms_min: 0.790655136108
	test_h0_row_norms_max: 2.3708486557
	test_h0_row_norms_mean: 1.18130934238
	test_h0_row_norms_min: 0.0573941357434
	test_h1_col_norms_max: 2.10413718224
	test_h1_col_norms_mean: 1.34777581692
	test_h1_col_norms_min: 0.599398136139
	test_h1_row_norms_max: 2.71429800987
	test_h1_row_norms_mean: 1.91710066795
	test_h1_row_norms_min: 1.09260857105
	test_objective: 0.213433161378
	test_term_0: 0.0551191605628
	test_term_1_weight_decay: 0.158313959837
	test_y_col_norms_max: 3.74245023727
	test_y_col_norms_mean: 3.40365672112
	test_y_col_norms_min: 2.87184524536
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.988063156605
	test_y_min_max_class: 0.63681012392
	test_y_misclass: 0.0161999966949
	test_y_nll: 0.0551191605628
	test_y_row_norms_max: 0.970815420151
	test_y_row_norms_mean: 0.305047929287
	test_y_row_norms_min: 0.0117727546021
	train_h0_col_norms_max: 2.32733273506
	train_h0_col_norms_mean: 1.50859475136
	train_h0_col_norms_min: 0.790655434132
	train_h0_row_norms_max: 2.37083745003
	train_h0_row_norms_mean: 1.18130576611
	train_h0_row_norms_min: 0.0573940649629
	train_h1_col_norms_max: 2.10414910316
	train_h1_col_norms_mean: 1.34777891636
	train_h1_col_norms_min: 0.599396586418
	train_h1_row_norms_max: 2.71428489685
	train_h1_row_norms_mean: 1.91711127758
	train_h1_row_norms_min: 1.09261226654
	train_objective: 0.161947190762
	train_term_0: 0.00363326980732
	train_term_1_weight_decay: 0.158314481378
	train_y_col_norms_max: 3.74245476723
	train_y_col_norms_mean: 3.40366172791
	train_y_col_norms_min: 2.87186050415
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996790409088
	train_y_min_max_class: 0.880268752575
	train_y_misclass: 0.0002599999425
	train_y_nll: 0.00363326980732
	train_y_row_norms_max: 0.970812141895
	train_y_row_norms_mean: 0.305047690868
	train_y_row_norms_min: 0.0117728123441
	valid_h0_col_norms_max: 2.32734417915
	valid_h0_col_norms_mean: 1.50859749317
	valid_h0_col_norms_min: 0.790655136108
	valid_h0_row_norms_max: 2.3708486557
	valid_h0_row_norms_mean: 1.18130934238
	valid_h0_row_norms_min: 0.0573941357434
	valid_h1_col_norms_max: 2.10413718224
	valid_h1_col_norms_mean: 1.34777581692
	valid_h1_col_norms_min: 0.599398136139
	valid_h1_row_norms_max: 2.71429800987
	valid_h1_row_norms_mean: 1.91710066795
	valid_h1_row_norms_min: 1.09260857105
	valid_objective: 0.221332803369
	valid_term_0: 0.0630188435316
	valid_term_1_weight_decay: 0.158313959837
	valid_y_col_norms_max: 3.74245023727
	valid_y_col_norms_mean: 3.40365672112
	valid_y_col_norms_min: 2.87184524536
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.988815426826
	valid_y_min_max_class: 0.629059970379
	valid_y_misclass: 0.0181999914348
	valid_y_nll: 0.0630188435316
	valid_y_row_norms_max: 0.970815420151
	valid_y_row_norms_mean: 0.305047929287
	valid_y_row_norms_min: 0.0117727546021
Time this epoch: 3.277658 seconds
Monitoring step:
	Epochs seen: 31
	Batches seen: 15500
	Examples seen: 1550000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.22986006737
	test_h0_col_norms_mean: 1.44450759888
	test_h0_col_norms_min: 0.751711428165
	test_h0_row_norms_max: 2.27136349678
	test_h0_row_norms_mean: 1.13126826286
	test_h0_row_norms_min: 0.055449090898
	test_h1_col_norms_max: 2.01123976707
	test_h1_col_norms_mean: 1.28632330894
	test_h1_col_norms_min: 0.570827186108
	test_h1_row_norms_max: 2.59384894371
	test_h1_row_norms_mean: 1.82977592945
	test_h1_row_norms_min: 1.03879511356
	test_objective: 0.199405178428
	test_term_0: 0.0542121008039
	test_term_1_weight_decay: 0.145193070173
	test_y_col_norms_max: 3.75564265251
	test_y_col_norms_mean: 3.41556763649
	test_y_col_norms_min: 2.89411330223
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.98805475235
	test_y_min_max_class: 0.651372611523
	test_y_misclass: 0.0173999965191
	test_y_nll: 0.0542121008039
	test_y_row_norms_max: 0.984490036964
	test_y_row_norms_mean: 0.30471482873
	test_y_row_norms_min: 0.0115118613467
	train_h0_col_norms_max: 2.22986245155
	train_h0_col_norms_mean: 1.44451439381
	train_h0_col_norms_min: 0.751711428165
	train_h0_row_norms_max: 2.27136206627
	train_h0_row_norms_mean: 1.13127017021
	train_h0_row_norms_min: 0.0554490871727
	train_h1_col_norms_max: 2.01124000549
	train_h1_col_norms_mean: 1.28632640839
	train_h1_col_norms_min: 0.570830464363
	train_h1_row_norms_max: 2.59386348724
	train_h1_row_norms_mean: 1.8297867775
	train_h1_row_norms_min: 1.03879117966
	train_objective: 0.148557990789
	train_term_0: 0.00336487870663
	train_term_1_weight_decay: 0.145192667842
	train_y_col_norms_max: 3.75566291809
	train_y_col_norms_mean: 3.41558241844
	train_y_col_norms_min: 2.89412260056
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.996838271618
	train_y_min_max_class: 0.885646343231
	train_y_misclass: 7.9999997979e-05
	train_y_nll: 0.00336487870663
	train_y_row_norms_max: 0.98448997736
	train_y_row_norms_mean: 0.304714143276
	train_y_row_norms_min: 0.0115119209513
	valid_h0_col_norms_max: 2.22986006737
	valid_h0_col_norms_mean: 1.44450759888
	valid_h0_col_norms_min: 0.751711428165
	valid_h0_row_norms_max: 2.27136349678
	valid_h0_row_norms_mean: 1.13126826286
	valid_h0_row_norms_min: 0.055449090898
	valid_h1_col_norms_max: 2.01123976707
	valid_h1_col_norms_mean: 1.28632330894
	valid_h1_col_norms_min: 0.570827186108
	valid_h1_row_norms_max: 2.59384894371
	valid_h1_row_norms_mean: 1.82977592945
	valid_h1_row_norms_min: 1.03879511356
	valid_objective: 0.204451009631
	valid_term_0: 0.0592579171062
	valid_term_1_weight_decay: 0.145193070173
	valid_y_col_norms_max: 3.75564265251
	valid_y_col_norms_mean: 3.41556763649
	valid_y_col_norms_min: 2.89411330223
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.989104747772
	valid_y_min_max_class: 0.617950022221
	valid_y_misclass: 0.0168999936432
	valid_y_nll: 0.0592579171062
	valid_y_row_norms_max: 0.984490036964
	valid_y_row_norms_mean: 0.30471482873
	valid_y_row_norms_min: 0.0115118613467
Time this epoch: 3.299106 seconds
Monitoring step:
	Epochs seen: 32
	Batches seen: 16000
	Examples seen: 1600000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.14349389076
	test_h0_col_norms_mean: 1.38710844517
	test_h0_col_norms_min: 0.714687824249
	test_h0_row_norms_max: 2.18458104134
	test_h0_row_norms_mean: 1.08642613888
	test_h0_row_norms_min: 0.0531989820302
	test_h1_col_norms_max: 1.92048859596
	test_h1_col_norms_mean: 1.22870218754
	test_h1_col_norms_min: 0.54377913475
	test_h1_row_norms_max: 2.48094320297
	test_h1_row_norms_mean: 1.74789762497
	test_h1_row_norms_min: 0.987630963326
	test_objective: 0.198874086142
	test_term_0: 0.0651714801788
	test_term_1_weight_decay: 0.13370269537
	test_y_col_norms_max: 3.78167033195
	test_y_col_norms_mean: 3.43723106384
	test_y_col_norms_min: 2.910176754
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.986017048359
	test_y_min_max_class: 0.594191014767
	test_y_misclass: 0.0199999921024
	test_y_nll: 0.0651714801788
	test_y_row_norms_max: 0.997191548347
	test_y_row_norms_mean: 0.305199384689
	test_y_row_norms_min: 0.0112372441217
	train_h0_col_norms_max: 2.14350128174
	train_h0_col_norms_mean: 1.38711500168
	train_h0_col_norms_min: 0.714688956738
	train_h0_row_norms_max: 2.18457770348
	train_h0_row_norms_mean: 1.08642041683
	train_h0_row_norms_min: 0.0531990006566
	train_h1_col_norms_max: 1.92047715187
	train_h1_col_norms_mean: 1.22869598866
	train_h1_col_norms_min: 0.543776392937
	train_h1_row_norms_max: 2.4809448719
	train_h1_row_norms_mean: 1.74790513515
	train_h1_row_norms_min: 0.987626254559
	train_objective: 0.142097592354
	train_term_0: 0.00839497055858
	train_term_1_weight_decay: 0.133703291416
	train_y_col_norms_max: 3.78167462349
	train_y_col_norms_mean: 3.43724656105
	train_y_col_norms_min: 2.91016840935
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.993861615658
	train_y_min_max_class: 0.764626443386
	train_y_misclass: 0.00168000021949
	train_y_nll: 0.00839497055858
	train_y_row_norms_max: 0.997187256813
	train_y_row_norms_mean: 0.305199593306
	train_y_row_norms_min: 0.0112373000011
	valid_h0_col_norms_max: 2.14349389076
	valid_h0_col_norms_mean: 1.38710844517
	valid_h0_col_norms_min: 0.714687824249
	valid_h0_row_norms_max: 2.18458104134
	valid_h0_row_norms_mean: 1.08642613888
	valid_h0_row_norms_min: 0.0531989820302
	valid_h1_col_norms_max: 1.92048859596
	valid_h1_col_norms_mean: 1.22870218754
	valid_h1_col_norms_min: 0.54377913475
	valid_h1_row_norms_max: 2.48094320297
	valid_h1_row_norms_mean: 1.74789762497
	valid_h1_row_norms_min: 0.987630963326
	valid_objective: 0.206293180585
	valid_term_0: 0.0725905746222
	valid_term_1_weight_decay: 0.13370269537
	valid_y_col_norms_max: 3.78167033195
	valid_y_col_norms_mean: 3.43723106384
	valid_y_col_norms_min: 2.910176754
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.98618721962
	valid_y_min_max_class: 0.584758162498
	valid_y_misclass: 0.0210999920964
	valid_y_nll: 0.0725905746222
	valid_y_row_norms_max: 0.997191548347
	valid_y_row_norms_mean: 0.305199384689
	valid_y_row_norms_min: 0.0112372441217
Time this epoch: 3.316685 seconds
Monitoring step:
	Epochs seen: 33
	Batches seen: 16500
	Examples seen: 1650000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.07966470718
	test_h0_col_norms_mean: 1.34304857254
	test_h0_col_norms_min: 0.67948693037
	test_h0_row_norms_max: 2.12437868118
	test_h0_row_norms_mean: 1.05228435993
	test_h0_row_norms_min: 0.0516484305263
	test_h1_col_norms_max: 1.84021937847
	test_h1_col_norms_mean: 1.17601656914
	test_h1_col_norms_min: 0.518966257572
	test_h1_row_norms_max: 2.38222265244
	test_h1_row_norms_mean: 1.67307877541
	test_h1_row_norms_min: 0.938986539841
	test_objective: 0.1945425421
	test_term_0: 0.0701323673129
	test_term_1_weight_decay: 0.124409988523
	test_y_col_norms_max: 3.7988409996
	test_y_col_norms_mean: 3.48081755638
	test_y_col_norms_min: 2.99444794655
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.985891222954
	test_y_min_max_class: 0.610743761063
	test_y_misclass: 0.021299995482
	test_y_nll: 0.0701323673129
	test_y_row_norms_max: 1.03870010376
	test_y_row_norms_mean: 0.307827204466
	test_y_row_norms_min: 0.0110923619941
	train_h0_col_norms_max: 2.07966327667
	train_h0_col_norms_mean: 1.34305346012
	train_h0_col_norms_min: 0.679490327835
	train_h0_row_norms_max: 2.12437534332
	train_h0_row_norms_mean: 1.05228734016
	train_h0_row_norms_min: 0.0516487248242
	train_h1_col_norms_max: 1.84022164345
	train_h1_col_norms_mean: 1.17601895332
	train_h1_col_norms_min: 0.518965959549
	train_h1_row_norms_max: 2.3822286129
	train_h1_row_norms_mean: 1.67308568954
	train_h1_row_norms_min: 0.93898332119
	train_objective: 0.135569825768
	train_term_0: 0.0111596826464
	train_term_1_weight_decay: 0.124409854412
	train_y_col_norms_max: 3.79884195328
	train_y_col_norms_mean: 3.48080062866
	train_y_col_norms_min: 2.99444794655
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.992844820023
	train_y_min_max_class: 0.72961461544
	train_y_misclass: 0.00289999856614
	train_y_nll: 0.0111596826464
	train_y_row_norms_max: 1.03869926929
	train_y_row_norms_mean: 0.307827889919
	train_y_row_norms_min: 0.0110923871398
	valid_h0_col_norms_max: 2.07966470718
	valid_h0_col_norms_mean: 1.34304857254
	valid_h0_col_norms_min: 0.67948693037
	valid_h0_row_norms_max: 2.12437868118
	valid_h0_row_norms_mean: 1.05228435993
	valid_h0_row_norms_min: 0.0516484305263
	valid_h1_col_norms_max: 1.84021937847
	valid_h1_col_norms_mean: 1.17601656914
	valid_h1_col_norms_min: 0.518966257572
	valid_h1_row_norms_max: 2.38222265244
	valid_h1_row_norms_mean: 1.67307877541
	valid_h1_row_norms_min: 0.938986539841
	valid_objective: 0.197794348001
	valid_term_0: 0.0733841732144
	valid_term_1_weight_decay: 0.124409988523
	valid_y_col_norms_max: 3.7988409996
	valid_y_col_norms_mean: 3.48081755638
	valid_y_col_norms_min: 2.99444794655
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.986917078495
	valid_y_min_max_class: 0.61943089962
	valid_y_misclass: 0.0203999932855
	valid_y_nll: 0.0733841732144
	valid_y_row_norms_max: 1.03870010376
	valid_y_row_norms_mean: 0.307827204466
	valid_y_row_norms_min: 0.0110923619941
Time this epoch: 3.390813 seconds
Monitoring step:
	Epochs seen: 34
	Batches seen: 17000
	Examples seen: 1700000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.09944272041
	test_h0_col_norms_mean: 1.31166350842
	test_h0_col_norms_min: 0.646019160748
	test_h0_row_norms_max: 2.08779644966
	test_h0_row_norms_mean: 1.02819681168
	test_h0_row_norms_min: 0.0544924363494
	test_h1_col_norms_max: 1.76557374001
	test_h1_col_norms_mean: 1.12808454037
	test_h1_col_norms_min: 0.495299696922
	test_h1_row_norms_max: 2.29631304741
	test_h1_row_norms_mean: 1.60503029823
	test_h1_row_norms_min: 0.892738819122
	test_objective: 0.189725786448
	test_term_0: 0.0726886093616
	test_term_1_weight_decay: 0.117037259042
	test_y_col_norms_max: 3.87525558472
	test_y_col_norms_mean: 3.52211046219
	test_y_col_norms_min: 3.00069046021
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.983940660954
	test_y_min_max_class: 0.607474207878
	test_y_misclass: 0.0210999920964
	test_y_nll: 0.0726886093616
	test_y_row_norms_max: 1.04711163044
	test_y_row_norms_mean: 0.310424894094
	test_y_row_norms_min: 0.0110520040616
	train_h0_col_norms_max: 2.09944820404
	train_h0_col_norms_mean: 1.31166100502
	train_h0_col_norms_min: 0.646022617817
	train_h0_row_norms_max: 2.08778810501
	train_h0_row_norms_mean: 1.02820193768
	train_h0_row_norms_min: 0.0544921904802
	train_h1_col_norms_max: 1.76556527615
	train_h1_col_norms_mean: 1.12807917595
	train_h1_col_norms_min: 0.495299696922
	train_h1_row_norms_max: 2.29631876945
	train_h1_row_norms_mean: 1.60503292084
	train_h1_row_norms_min: 0.892735242844
	train_objective: 0.138252094388
	train_term_0: 0.0212149638683
	train_term_1_weight_decay: 0.117037393153
	train_y_col_norms_max: 3.87525558472
	train_y_col_norms_mean: 3.5221259594
	train_y_col_norms_min: 3.00070405006
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.989364624023
	train_y_min_max_class: 0.659551143646
	train_y_misclass: 0.00660000368953
	train_y_nll: 0.0212149638683
	train_y_row_norms_max: 1.04711127281
	train_y_row_norms_mean: 0.310425490141
	train_y_row_norms_min: 0.0110520040616
	valid_h0_col_norms_max: 2.09944272041
	valid_h0_col_norms_mean: 1.31166350842
	valid_h0_col_norms_min: 0.646019160748
	valid_h0_row_norms_max: 2.08779644966
	valid_h0_row_norms_mean: 1.02819681168
	valid_h0_row_norms_min: 0.0544924363494
	valid_h1_col_norms_max: 1.76557374001
	valid_h1_col_norms_mean: 1.12808454037
	valid_h1_col_norms_min: 0.495299696922
	valid_h1_row_norms_max: 2.29631304741
	valid_h1_row_norms_mean: 1.60503029823
	valid_h1_row_norms_min: 0.892738819122
	valid_objective: 0.204115614295
	valid_term_0: 0.0870784968138
	valid_term_1_weight_decay: 0.117037259042
	valid_y_col_norms_max: 3.87525558472
	valid_y_col_norms_mean: 3.52211046219
	valid_y_col_norms_min: 3.00069046021
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.984562575817
	valid_y_min_max_class: 0.597014904022
	valid_y_misclass: 0.0247999858111
	valid_y_nll: 0.0870784968138
	valid_y_row_norms_max: 1.04711163044
	valid_y_row_norms_mean: 0.310424894094
	valid_y_row_norms_min: 0.0110520040616
Time this epoch: 3.325348 seconds
Monitoring step:
	Epochs seen: 35
	Batches seen: 17500
	Examples seen: 1750000
	learning_rate: 0.00999999046326
	momentum: 0.989998817444
	test_h0_col_norms_max: 2.33471369743
	test_h0_col_norms_mean: 1.30764365196
	test_h0_col_norms_min: 0.614201545715
	test_h0_row_norms_max: 2.11369776726
	test_h0_row_norms_mean: 1.02605807781
	test_h0_row_norms_min: 0.0789580345154
	test_h1_col_norms_max: 1.72045576572
	test_h1_col_norms_mean: 1.08744347095
	test_h1_col_norms_min: 0.471859395504
	test_h1_row_norms_max: 2.22104668617
	test_h1_row_norms_mean: 1.54732775688
	test_h1_row_norms_min: 0.848768413067
	test_objective: 0.186729609966
	test_term_0: 0.0738602727652
	test_term_1_weight_decay: 0.112869426608
	test_y_col_norms_max: 3.81233644485
	test_y_col_norms_mean: 3.53644061089
	test_y_col_norms_min: 3.07366251945
	test_y_max_max_class: 0.999999344349
	test_y_mean_max_class: 0.982964873314
	test_y_min_max_class: 0.570746660233
	test_y_misclass: 0.0224999897182
	test_y_nll: 0.0738602727652
	test_y_row_norms_max: 1.04587638378
	test_y_row_norms_mean: 0.311368614435
	test_y_row_norms_min: 0.0108088394627
	train_h0_col_norms_max: 2.33471369743
	train_h0_col_norms_mean: 1.30764782429
	train_h0_col_norms_min: 0.614198505878
	train_h0_row_norms_max: 2.11369967461
	train_h0_row_norms_mean: 1.02606165409
	train_h0_row_norms_min: 0.0789578035474
	train_h1_col_norms_max: 1.72044575214
	train_h1_col_norms_mean: 1.08744776249
	train_h1_col_norms_min: 0.471859931946
	train_h1_row_norms_max: 2.2210419178
	train_h1_row_norms_mean: 1.54733288288
	train_h1_row_norms_min: 0.848769664764
	train_objective: 0.133081272244
	train_term_0: 0.020211936906
	train_term_1_weight_decay: 0.112869039178
	train_y_col_norms_max: 3.81231951714
	train_y_col_norms_mean: 3.53645634651
	train_y_col_norms_min: 3.07366323471
	train_y_max_max_class: 0.999994218349
	train_y_mean_max_class: 0.989763617516
	train_y_min_max_class: 0.656112134457
	train_y_misclass: 0.00610000034794
	train_y_nll: 0.020211936906
	train_y_row_norms_max: 1.04588091373
	train_y_row_norms_mean: 0.311368972063
	train_y_row_norms_min: 0.0108088953421
	valid_h0_col_norms_max: 2.33471369743
	valid_h0_col_norms_mean: 1.30764365196
	valid_h0_col_norms_min: 0.614201545715
	valid_h0_row_norms_max: 2.11369776726
	valid_h0_row_norms_mean: 1.02605807781
	valid_h0_row_norms_min: 0.0789580345154
	valid_h1_col_norms_max: 1.72045576572
	valid_h1_col_norms_mean: 1.08744347095
	valid_h1_col_norms_min: 0.471859395504
	valid_h1_row_norms_max: 2.22104668617
	valid_h1_row_norms_mean: 1.54732775688
	valid_h1_row_norms_min: 0.848768413067
	valid_objective: 0.191735550761
	valid_term_0: 0.0788661986589
	valid_term_1_weight_decay: 0.112869426608
	valid_y_col_norms_max: 3.81233644485
	valid_y_col_norms_mean: 3.53644061089
	valid_y_col_norms_min: 3.07366251945
	valid_y_max_max_class: 0.999999344349
	valid_y_mean_max_class: 0.985468804836
	valid_y_min_max_class: 0.593561589718
	valid_y_misclass: 0.0224999915808
	valid_y_nll: 0.0788661986589
	valid_y_row_norms_max: 1.04587638378
	valid_y_row_norms_mean: 0.311368614435
	valid_y_row_norms_min: 0.0108088394627



In [11]:

    
!print_monitor.py mlp_3_best.pkl | grep test_y_misclass









    



Using gpu device 2: GeForce GTX 285
/u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit.
  warnings.warn("MLP changing the recursion limit.")
test_y_misclass : 0.0153999980539

Using a simple form of regularization thus brought the test error rate for this MLP down from 1.75% to 1.54%.

pylearn2 tutorial: Multilayer Perceptron

Introduction

Requirements

Part 1: What pylearn2 is doing for you in this example

Review of softmax regression, and how MLPs are similar

The multilayer perceptron model

Some beneficial properties of MLPs

Some detrimental properties of MLPs

Part 2: How to use pylearn2 to train an MLP

Part 3 A deeper MLP, and pylearn2 polymorphism

Part 4: Regularization and pylearn2 costs

Further reading