Removing previous artifacts...
Experiment started at 17:17:57
.......................................
WARNING:tensorflow:RunConfig.uid (from tensorflow.contrib.learn.python.learn.estimators.run_config) is experimental and may change or be removed at any time, and without warning.
INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x121a74b38>, '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'local', '_is_chief': True, '_evaluation_master': '', '_tf_config': gpu_options {
per_process_gpu_memory_fraction: 1
}
, '_tf_random_seed': 19830610, '_save_summary_steps': 100, '_save_checkpoints_secs': None, '_log_step_count_steps': 100, '_session_config': None, '_save_checkpoints_steps': 2400, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': 'trained_models/reg-model-04'}
Estimator Type: <class 'tensorflow.python.estimator.estimator.Estimator'>
WARNING:tensorflow:RunConfig.uid (from tensorflow.contrib.learn.python.learn.estimators.run_config) is experimental and may change or be removed at any time, and without warning.
WARNING:tensorflow:From /Users/khalidsalama/anaconda/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/monitors.py:267: BaseMonitor.__init__ (from tensorflow.contrib.learn.python.learn.monitors) is deprecated and will be removed after 2016-12-05.
Instructions for updating:
Monitors are deprecated. Please use tf.train.SessionRunHook.
* data input_fn:
================
Input file(s): data/train-*.csv
Batch size: 500
Epoch Count: 1000
Mode: train
Shuffle: True
================
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Saving checkpoints for 1 into trained_models/reg-model-04/model.ckpt.
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:18:04
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-1
INFO:tensorflow:Finished evaluation at 2017-11-15-17:18:04
INFO:tensorflow:Saving dict for global step 1: global_step = 1, loss = 324.174, rmse = 18.0048
INFO:tensorflow:Validation (step 1): loss = 324.174, rmse = 18.0048, global_step = 1
INFO:tensorflow:loss = 305.581, step = 1
INFO:tensorflow:global_step/sec: 27.7021
INFO:tensorflow:loss = 368.79, step = 101 (0.761 sec)
INFO:tensorflow:global_step/sec: 148.981
INFO:tensorflow:loss = 269.747, step = 201 (0.671 sec)
INFO:tensorflow:global_step/sec: 131.481
INFO:tensorflow:loss = 322.637, step = 301 (0.761 sec)
INFO:tensorflow:global_step/sec: 125.216
INFO:tensorflow:loss = 254.287, step = 401 (0.798 sec)
INFO:tensorflow:global_step/sec: 120.693
INFO:tensorflow:loss = 232.263, step = 501 (0.828 sec)
INFO:tensorflow:global_step/sec: 133.346
INFO:tensorflow:loss = 210.963, step = 601 (0.752 sec)
INFO:tensorflow:global_step/sec: 105.211
INFO:tensorflow:loss = 217.493, step = 701 (0.950 sec)
INFO:tensorflow:global_step/sec: 118.339
INFO:tensorflow:loss = 161.082, step = 801 (0.845 sec)
INFO:tensorflow:global_step/sec: 96.0488
INFO:tensorflow:loss = 158.592, step = 901 (1.041 sec)
INFO:tensorflow:global_step/sec: 107.686
INFO:tensorflow:loss = 142.485, step = 1001 (0.928 sec)
INFO:tensorflow:global_step/sec: 118.178
INFO:tensorflow:loss = 141.336, step = 1101 (0.847 sec)
INFO:tensorflow:global_step/sec: 105.205
INFO:tensorflow:loss = 138.915, step = 1201 (0.951 sec)
INFO:tensorflow:global_step/sec: 127.681
INFO:tensorflow:loss = 157.873, step = 1301 (0.783 sec)
INFO:tensorflow:global_step/sec: 112.202
INFO:tensorflow:loss = 112.564, step = 1401 (0.890 sec)
INFO:tensorflow:global_step/sec: 114.96
INFO:tensorflow:loss = 125.055, step = 1501 (0.869 sec)
INFO:tensorflow:global_step/sec: 135.279
INFO:tensorflow:loss = 101.862, step = 1601 (0.739 sec)
INFO:tensorflow:global_step/sec: 132.51
INFO:tensorflow:loss = 126.73, step = 1701 (0.756 sec)
INFO:tensorflow:global_step/sec: 125.851
INFO:tensorflow:loss = 102.2, step = 1801 (0.794 sec)
INFO:tensorflow:global_step/sec: 129.619
INFO:tensorflow:loss = 119.56, step = 1901 (0.772 sec)
INFO:tensorflow:global_step/sec: 110.474
INFO:tensorflow:loss = 110.547, step = 2001 (0.905 sec)
INFO:tensorflow:global_step/sec: 116.811
INFO:tensorflow:loss = 94.6563, step = 2101 (0.856 sec)
INFO:tensorflow:global_step/sec: 129.81
INFO:tensorflow:loss = 111.647, step = 2201 (0.770 sec)
INFO:tensorflow:global_step/sec: 122.031
INFO:tensorflow:loss = 103.735, step = 2301 (0.819 sec)
INFO:tensorflow:Saving checkpoints for 2401 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 62.227
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:18:27
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-2401
INFO:tensorflow:Finished evaluation at 2017-11-15-17:18:28
INFO:tensorflow:Saving dict for global step 2401: global_step = 2401, loss = 111.019, rmse = 10.5366
INFO:tensorflow:Validation (step 2401): loss = 111.019, rmse = 10.5366, global_step = 2401
INFO:tensorflow:loss = 109.904, step = 2401 (3.406 sec)
INFO:tensorflow:global_step/sec: 38.4282
INFO:tensorflow:loss = 90.4845, step = 2501 (0.804 sec)
INFO:tensorflow:global_step/sec: 126.078
INFO:tensorflow:loss = 93.4027, step = 2601 (0.794 sec)
INFO:tensorflow:global_step/sec: 124.458
INFO:tensorflow:loss = 96.8483, step = 2701 (0.804 sec)
INFO:tensorflow:global_step/sec: 138.629
INFO:tensorflow:loss = 92.6891, step = 2801 (0.721 sec)
INFO:tensorflow:global_step/sec: 141.396
INFO:tensorflow:loss = 93.79, step = 2901 (0.707 sec)
INFO:tensorflow:global_step/sec: 143.568
INFO:tensorflow:loss = 98.7764, step = 3001 (0.696 sec)
INFO:tensorflow:global_step/sec: 121.522
INFO:tensorflow:loss = 107.321, step = 3101 (0.823 sec)
INFO:tensorflow:global_step/sec: 101.757
INFO:tensorflow:loss = 106.702, step = 3201 (0.986 sec)
INFO:tensorflow:global_step/sec: 84.4662
INFO:tensorflow:loss = 90.4981, step = 3301 (1.182 sec)
INFO:tensorflow:global_step/sec: 86.2167
INFO:tensorflow:loss = 102.711, step = 3401 (1.160 sec)
INFO:tensorflow:global_step/sec: 82.4557
INFO:tensorflow:loss = 73.9731, step = 3501 (1.213 sec)
INFO:tensorflow:global_step/sec: 104.218
INFO:tensorflow:loss = 100.379, step = 3601 (0.959 sec)
INFO:tensorflow:global_step/sec: 125.433
INFO:tensorflow:loss = 96.7336, step = 3701 (0.797 sec)
INFO:tensorflow:global_step/sec: 137.894
INFO:tensorflow:loss = 95.0096, step = 3801 (0.725 sec)
INFO:tensorflow:global_step/sec: 135.539
INFO:tensorflow:loss = 90.6274, step = 3901 (0.738 sec)
INFO:tensorflow:global_step/sec: 126.798
INFO:tensorflow:loss = 104.553, step = 4001 (0.790 sec)
INFO:tensorflow:global_step/sec: 113.883
INFO:tensorflow:loss = 95.309, step = 4101 (0.879 sec)
INFO:tensorflow:global_step/sec: 120.287
INFO:tensorflow:loss = 93.0883, step = 4201 (0.830 sec)
INFO:tensorflow:global_step/sec: 136.161
INFO:tensorflow:loss = 82.3985, step = 4301 (0.733 sec)
INFO:tensorflow:global_step/sec: 126.43
INFO:tensorflow:loss = 91.7385, step = 4401 (0.792 sec)
INFO:tensorflow:global_step/sec: 118.332
INFO:tensorflow:loss = 95.2653, step = 4501 (0.846 sec)
INFO:tensorflow:global_step/sec: 130.969
INFO:tensorflow:loss = 91.2204, step = 4601 (0.762 sec)
INFO:tensorflow:global_step/sec: 137.648
INFO:tensorflow:loss = 75.8971, step = 4701 (0.726 sec)
INFO:tensorflow:Saving checkpoints for 4801 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 60.1168
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:18:50
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-4801
INFO:tensorflow:Finished evaluation at 2017-11-15-17:18:52
INFO:tensorflow:Saving dict for global step 4801: global_step = 4801, loss = 99.1767, rmse = 9.95875
INFO:tensorflow:Validation (step 4801): loss = 99.1767, rmse = 9.95875, global_step = 4801
INFO:tensorflow:loss = 91.5243, step = 4801 (3.991 sec)
INFO:tensorflow:global_step/sec: 30.6975
INFO:tensorflow:loss = 96.4061, step = 4901 (0.932 sec)
INFO:tensorflow:global_step/sec: 116.533
INFO:tensorflow:loss = 71.5417, step = 5001 (0.859 sec)
INFO:tensorflow:global_step/sec: 113.577
INFO:tensorflow:loss = 115.22, step = 5101 (0.880 sec)
INFO:tensorflow:global_step/sec: 108.132
INFO:tensorflow:loss = 76.1541, step = 5201 (0.924 sec)
INFO:tensorflow:global_step/sec: 127.932
INFO:tensorflow:loss = 101.792, step = 5301 (0.781 sec)
INFO:tensorflow:global_step/sec: 136.327
INFO:tensorflow:loss = 115.638, step = 5401 (0.734 sec)
INFO:tensorflow:global_step/sec: 140.918
INFO:tensorflow:loss = 80.7678, step = 5501 (0.709 sec)
INFO:tensorflow:global_step/sec: 136.876
INFO:tensorflow:loss = 76.5593, step = 5601 (0.731 sec)
INFO:tensorflow:global_step/sec: 139.89
INFO:tensorflow:loss = 91.0885, step = 5701 (0.715 sec)
INFO:tensorflow:global_step/sec: 133.167
INFO:tensorflow:loss = 85.4326, step = 5801 (0.751 sec)
INFO:tensorflow:global_step/sec: 139.961
INFO:tensorflow:loss = 87.558, step = 5901 (0.715 sec)
INFO:tensorflow:global_step/sec: 136.259
INFO:tensorflow:loss = 98.2536, step = 6001 (0.734 sec)
INFO:tensorflow:global_step/sec: 133.309
INFO:tensorflow:loss = 75.2041, step = 6101 (0.749 sec)
INFO:tensorflow:global_step/sec: 140.177
INFO:tensorflow:loss = 86.4089, step = 6201 (0.713 sec)
INFO:tensorflow:global_step/sec: 142.551
INFO:tensorflow:loss = 98.8525, step = 6301 (0.702 sec)
INFO:tensorflow:global_step/sec: 140.433
INFO:tensorflow:loss = 89.3366, step = 6401 (0.712 sec)
INFO:tensorflow:global_step/sec: 136.153
INFO:tensorflow:loss = 85.1617, step = 6501 (0.735 sec)
INFO:tensorflow:global_step/sec: 142.027
INFO:tensorflow:loss = 100.363, step = 6601 (0.704 sec)
INFO:tensorflow:global_step/sec: 134.214
INFO:tensorflow:loss = 80.5897, step = 6701 (0.744 sec)
INFO:tensorflow:global_step/sec: 131.315
INFO:tensorflow:loss = 89.2196, step = 6801 (0.764 sec)
INFO:tensorflow:global_step/sec: 132.721
INFO:tensorflow:loss = 90.5747, step = 6901 (0.752 sec)
INFO:tensorflow:global_step/sec: 132.441
INFO:tensorflow:loss = 83.1952, step = 7001 (0.754 sec)
INFO:tensorflow:global_step/sec: 137.362
INFO:tensorflow:loss = 73.7089, step = 7101 (0.729 sec)
INFO:tensorflow:Saving checkpoints for 7201 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 62.8855
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:19:12
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-7201
INFO:tensorflow:Finished evaluation at 2017-11-15-17:19:13
INFO:tensorflow:Saving dict for global step 7201: global_step = 7201, loss = 96.9113, rmse = 9.84435
INFO:tensorflow:Validation (step 7201): loss = 96.9113, rmse = 9.84435, global_step = 7201
INFO:tensorflow:loss = 102.963, step = 7201 (3.709 sec)
INFO:tensorflow:global_step/sec: 32.7451
INFO:tensorflow:loss = 87.1191, step = 7301 (0.936 sec)
INFO:tensorflow:global_step/sec: 111.742
INFO:tensorflow:loss = 80.0266, step = 7401 (0.895 sec)
INFO:tensorflow:global_step/sec: 134.62
INFO:tensorflow:loss = 62.3085, step = 7501 (0.742 sec)
INFO:tensorflow:global_step/sec: 135.144
INFO:tensorflow:loss = 106.276, step = 7601 (0.740 sec)
INFO:tensorflow:global_step/sec: 135.778
INFO:tensorflow:loss = 86.2817, step = 7701 (0.737 sec)
INFO:tensorflow:global_step/sec: 131.485
INFO:tensorflow:loss = 94.816, step = 7801 (0.761 sec)
INFO:tensorflow:global_step/sec: 128.826
INFO:tensorflow:loss = 102.25, step = 7901 (0.775 sec)
INFO:tensorflow:global_step/sec: 132.348
INFO:tensorflow:loss = 91.009, step = 8001 (0.756 sec)
INFO:tensorflow:global_step/sec: 140.505
INFO:tensorflow:loss = 76.027, step = 8101 (0.711 sec)
INFO:tensorflow:global_step/sec: 97.8052
INFO:tensorflow:loss = 102.481, step = 8201 (1.025 sec)
INFO:tensorflow:global_step/sec: 78.7297
INFO:tensorflow:loss = 90.9343, step = 8301 (1.272 sec)
INFO:tensorflow:global_step/sec: 75.883
INFO:tensorflow:loss = 77.5593, step = 8401 (1.315 sec)
INFO:tensorflow:global_step/sec: 92.9067
INFO:tensorflow:loss = 73.9644, step = 8501 (1.075 sec)
INFO:tensorflow:global_step/sec: 106.569
INFO:tensorflow:loss = 99.2161, step = 8601 (0.940 sec)
INFO:tensorflow:global_step/sec: 92.4485
INFO:tensorflow:loss = 80.0712, step = 8701 (1.081 sec)
INFO:tensorflow:global_step/sec: 121.173
INFO:tensorflow:loss = 93.2751, step = 8801 (0.825 sec)
INFO:tensorflow:global_step/sec: 139.892
INFO:tensorflow:loss = 91.0283, step = 8901 (0.714 sec)
INFO:tensorflow:global_step/sec: 135.068
INFO:tensorflow:loss = 81.5481, step = 9001 (0.740 sec)
INFO:tensorflow:global_step/sec: 141.43
INFO:tensorflow:loss = 86.362, step = 9101 (0.708 sec)
INFO:tensorflow:global_step/sec: 139.548
INFO:tensorflow:loss = 102.046, step = 9201 (0.716 sec)
INFO:tensorflow:global_step/sec: 127.498
INFO:tensorflow:loss = 96.3737, step = 9301 (0.785 sec)
INFO:tensorflow:global_step/sec: 122.757
INFO:tensorflow:loss = 94.3923, step = 9401 (0.814 sec)
INFO:tensorflow:global_step/sec: 126.044
INFO:tensorflow:loss = 96.458, step = 9501 (0.794 sec)
INFO:tensorflow:Saving checkpoints for 9601 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 75.0063
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:19:35
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-9601
INFO:tensorflow:Finished evaluation at 2017-11-15-17:19:36
INFO:tensorflow:Saving dict for global step 9601: global_step = 9601, loss = 95.7524, rmse = 9.78532
INFO:tensorflow:Validation (step 9601): loss = 95.7524, rmse = 9.78532, global_step = 9601
INFO:tensorflow:loss = 106.336, step = 9601 (3.345 sec)
INFO:tensorflow:global_step/sec: 36.6991
INFO:tensorflow:loss = 107.314, step = 9701 (0.713 sec)
INFO:tensorflow:global_step/sec: 137.973
INFO:tensorflow:loss = 71.1633, step = 9801 (0.725 sec)
INFO:tensorflow:global_step/sec: 135.014
INFO:tensorflow:loss = 87.9739, step = 9901 (0.740 sec)
INFO:tensorflow:global_step/sec: 135.278
INFO:tensorflow:loss = 100.31, step = 10001 (0.740 sec)
INFO:tensorflow:global_step/sec: 126.609
INFO:tensorflow:loss = 92.408, step = 10101 (0.790 sec)
INFO:tensorflow:global_step/sec: 136.781
INFO:tensorflow:loss = 97.7729, step = 10201 (0.730 sec)
INFO:tensorflow:global_step/sec: 135.417
INFO:tensorflow:loss = 80.3, step = 10301 (0.739 sec)
INFO:tensorflow:global_step/sec: 125.671
INFO:tensorflow:loss = 83.4821, step = 10401 (0.796 sec)
INFO:tensorflow:global_step/sec: 108.391
INFO:tensorflow:loss = 72.4306, step = 10501 (0.924 sec)
INFO:tensorflow:global_step/sec: 98.8776
INFO:tensorflow:loss = 83.0199, step = 10601 (1.010 sec)
INFO:tensorflow:global_step/sec: 125.343
INFO:tensorflow:loss = 84.3509, step = 10701 (0.798 sec)
INFO:tensorflow:global_step/sec: 131.566
INFO:tensorflow:loss = 108.604, step = 10801 (0.763 sec)
INFO:tensorflow:global_step/sec: 113.84
INFO:tensorflow:loss = 81.5453, step = 10901 (0.875 sec)
INFO:tensorflow:global_step/sec: 107.895
INFO:tensorflow:loss = 85.9325, step = 11001 (0.927 sec)
INFO:tensorflow:global_step/sec: 131.404
INFO:tensorflow:loss = 90.8869, step = 11101 (0.760 sec)
INFO:tensorflow:global_step/sec: 116.025
INFO:tensorflow:loss = 87.7549, step = 11201 (0.862 sec)
INFO:tensorflow:global_step/sec: 129.617
INFO:tensorflow:loss = 90.1264, step = 11301 (0.772 sec)
INFO:tensorflow:global_step/sec: 125.43
INFO:tensorflow:loss = 85.2653, step = 11401 (0.797 sec)
INFO:tensorflow:global_step/sec: 107.464
INFO:tensorflow:loss = 60.0375, step = 11501 (0.931 sec)
INFO:tensorflow:global_step/sec: 119.667
INFO:tensorflow:loss = 101.51, step = 11601 (0.835 sec)
INFO:tensorflow:global_step/sec: 124.788
INFO:tensorflow:loss = 85.8463, step = 11701 (0.802 sec)
INFO:tensorflow:global_step/sec: 106.763
INFO:tensorflow:loss = 84.845, step = 11801 (0.936 sec)
INFO:tensorflow:global_step/sec: 126.928
INFO:tensorflow:loss = 98.2499, step = 11901 (0.787 sec)
INFO:tensorflow:Saving checkpoints for 12001 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 60.5599
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:19:58
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-12001
INFO:tensorflow:Finished evaluation at 2017-11-15-17:19:59
INFO:tensorflow:Saving dict for global step 12001: global_step = 12001, loss = 95.1853, rmse = 9.7563
INFO:tensorflow:Validation (step 12001): loss = 95.1853, rmse = 9.7563, global_step = 12001
INFO:tensorflow:loss = 97.8091, step = 12001 (4.537 sec)
INFO:tensorflow:global_step/sec: 26.0564
INFO:tensorflow:loss = 66.8572, step = 12101 (0.954 sec)
INFO:tensorflow:global_step/sec: 125.648
INFO:tensorflow:loss = 76.6806, step = 12201 (0.795 sec)
INFO:tensorflow:global_step/sec: 136.423
INFO:tensorflow:loss = 85.0077, step = 12301 (0.732 sec)
INFO:tensorflow:global_step/sec: 103.577
INFO:tensorflow:loss = 71.1743, step = 12401 (0.969 sec)
INFO:tensorflow:global_step/sec: 78.3862
INFO:tensorflow:loss = 85.8499, step = 12501 (1.275 sec)
INFO:tensorflow:global_step/sec: 83.3678
INFO:tensorflow:loss = 97.3313, step = 12601 (1.198 sec)
INFO:tensorflow:global_step/sec: 80.2644
INFO:tensorflow:loss = 74.5801, step = 12701 (1.245 sec)
INFO:tensorflow:global_step/sec: 107.831
INFO:tensorflow:loss = 96.9373, step = 12801 (0.926 sec)
INFO:tensorflow:global_step/sec: 114.27
INFO:tensorflow:loss = 62.2827, step = 12901 (0.875 sec)
INFO:tensorflow:global_step/sec: 137.679
INFO:tensorflow:loss = 86.4258, step = 13001 (0.726 sec)
INFO:tensorflow:global_step/sec: 135.604
INFO:tensorflow:loss = 75.6892, step = 13101 (0.737 sec)
INFO:tensorflow:global_step/sec: 104.43
INFO:tensorflow:loss = 90.7635, step = 13201 (0.959 sec)
INFO:tensorflow:global_step/sec: 78.3317
INFO:tensorflow:loss = 76.7654, step = 13301 (1.276 sec)
INFO:tensorflow:global_step/sec: 106.986
INFO:tensorflow:loss = 73.7493, step = 13401 (0.935 sec)
INFO:tensorflow:global_step/sec: 115.601
INFO:tensorflow:loss = 81.115, step = 13501 (0.864 sec)
INFO:tensorflow:global_step/sec: 134.839
INFO:tensorflow:loss = 93.5551, step = 13601 (0.742 sec)
INFO:tensorflow:global_step/sec: 139.796
INFO:tensorflow:loss = 89.8246, step = 13701 (0.714 sec)
INFO:tensorflow:global_step/sec: 135.746
INFO:tensorflow:loss = 110.005, step = 13801 (0.739 sec)
INFO:tensorflow:global_step/sec: 110.052
INFO:tensorflow:loss = 86.0695, step = 13901 (0.906 sec)
INFO:tensorflow:global_step/sec: 117.753
INFO:tensorflow:loss = 88.913, step = 14001 (0.850 sec)
INFO:tensorflow:global_step/sec: 125.329
INFO:tensorflow:loss = 63.7754, step = 14101 (0.797 sec)
INFO:tensorflow:global_step/sec: 120.181
INFO:tensorflow:loss = 67.4602, step = 14201 (0.834 sec)
INFO:tensorflow:global_step/sec: 120.763
INFO:tensorflow:loss = 87.6832, step = 14301 (0.826 sec)
INFO:tensorflow:Saving checkpoints for 14401 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 57.4726
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:20:23
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-14401
INFO:tensorflow:Finished evaluation at 2017-11-15-17:20:24
INFO:tensorflow:Saving dict for global step 14401: global_step = 14401, loss = 94.825, rmse = 9.73781
INFO:tensorflow:Validation (step 14401): loss = 94.825, rmse = 9.73781, global_step = 14401
INFO:tensorflow:loss = 115.338, step = 14401 (4.270 sec)
INFO:tensorflow:global_step/sec: 26.9336
INFO:tensorflow:loss = 91.855, step = 14501 (1.186 sec)
INFO:tensorflow:global_step/sec: 107.661
INFO:tensorflow:loss = 70.124, step = 14601 (0.930 sec)
INFO:tensorflow:global_step/sec: 95.6159
INFO:tensorflow:loss = 91.2174, step = 14701 (1.044 sec)
INFO:tensorflow:global_step/sec: 123.472
INFO:tensorflow:loss = 79.1345, step = 14801 (0.810 sec)
INFO:tensorflow:global_step/sec: 117.364
INFO:tensorflow:loss = 79.6455, step = 14901 (0.850 sec)
INFO:tensorflow:global_step/sec: 119.987
INFO:tensorflow:loss = 86.6081, step = 15001 (0.834 sec)
INFO:tensorflow:global_step/sec: 102.141
INFO:tensorflow:loss = 77.605, step = 15101 (0.980 sec)
INFO:tensorflow:global_step/sec: 98.9558
INFO:tensorflow:loss = 83.4992, step = 15201 (1.012 sec)
INFO:tensorflow:global_step/sec: 106.431
INFO:tensorflow:loss = 101.17, step = 15301 (0.937 sec)
INFO:tensorflow:global_step/sec: 102.454
INFO:tensorflow:loss = 60.1519, step = 15401 (0.977 sec)
INFO:tensorflow:global_step/sec: 102.958
INFO:tensorflow:loss = 94.1324, step = 15501 (0.971 sec)
INFO:tensorflow:global_step/sec: 115.696
INFO:tensorflow:loss = 94.769, step = 15601 (0.864 sec)
INFO:tensorflow:global_step/sec: 129.466
INFO:tensorflow:loss = 79.8938, step = 15701 (0.772 sec)
INFO:tensorflow:global_step/sec: 137.666
INFO:tensorflow:loss = 70.9775, step = 15801 (0.727 sec)
INFO:tensorflow:global_step/sec: 132.399
INFO:tensorflow:loss = 84.0039, step = 15901 (0.754 sec)
INFO:tensorflow:global_step/sec: 114.34
INFO:tensorflow:loss = 58.0531, step = 16001 (0.876 sec)
INFO:tensorflow:global_step/sec: 94.2941
INFO:tensorflow:loss = 84.2777, step = 16101 (1.061 sec)
INFO:tensorflow:global_step/sec: 86.65
INFO:tensorflow:loss = 83.4427, step = 16201 (1.152 sec)
INFO:tensorflow:global_step/sec: 114.596
INFO:tensorflow:loss = 79.0373, step = 16301 (0.874 sec)
INFO:tensorflow:global_step/sec: 111.071
INFO:tensorflow:loss = 80.4201, step = 16401 (0.899 sec)
INFO:tensorflow:global_step/sec: 102.022
INFO:tensorflow:loss = 92.0403, step = 16501 (0.982 sec)
INFO:tensorflow:global_step/sec: 113.703
INFO:tensorflow:loss = 90.6051, step = 16601 (0.881 sec)
INFO:tensorflow:global_step/sec: 85.5278
INFO:tensorflow:loss = 90.619, step = 16701 (1.169 sec)
INFO:tensorflow:Saving checkpoints for 16801 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 60.5597
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:20:49
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-16801
INFO:tensorflow:Finished evaluation at 2017-11-15-17:20:50
INFO:tensorflow:Saving dict for global step 16801: global_step = 16801, loss = 94.5301, rmse = 9.72266
INFO:tensorflow:Validation (step 16801): loss = 94.5301, rmse = 9.72266, global_step = 16801
INFO:tensorflow:loss = 86.0765, step = 16801 (3.651 sec)
INFO:tensorflow:global_step/sec: 35.2543
INFO:tensorflow:loss = 108.696, step = 16901 (0.836 sec)
INFO:tensorflow:global_step/sec: 133.901
INFO:tensorflow:loss = 68.3136, step = 17001 (0.745 sec)
INFO:tensorflow:global_step/sec: 133.557
INFO:tensorflow:loss = 91.7749, step = 17101 (0.749 sec)
INFO:tensorflow:global_step/sec: 139.598
INFO:tensorflow:loss = 87.6673, step = 17201 (0.716 sec)
INFO:tensorflow:global_step/sec: 142.745
INFO:tensorflow:loss = 101.521, step = 17301 (0.701 sec)
INFO:tensorflow:global_step/sec: 142.548
INFO:tensorflow:loss = 78.7295, step = 17401 (0.701 sec)
INFO:tensorflow:global_step/sec: 135.531
INFO:tensorflow:loss = 69.3208, step = 17501 (0.739 sec)
INFO:tensorflow:global_step/sec: 135.613
INFO:tensorflow:loss = 78.5277, step = 17601 (0.737 sec)
INFO:tensorflow:global_step/sec: 141.893
INFO:tensorflow:loss = 87.7595, step = 17701 (0.705 sec)
INFO:tensorflow:global_step/sec: 136.375
INFO:tensorflow:loss = 88.1816, step = 17801 (0.734 sec)
INFO:tensorflow:global_step/sec: 140.295
INFO:tensorflow:loss = 96.0901, step = 17901 (0.713 sec)
INFO:tensorflow:global_step/sec: 139.509
INFO:tensorflow:loss = 87.9287, step = 18001 (0.716 sec)
INFO:tensorflow:global_step/sec: 125.593
INFO:tensorflow:loss = 80.1272, step = 18101 (0.797 sec)
INFO:tensorflow:global_step/sec: 123.549
INFO:tensorflow:loss = 91.0642, step = 18201 (0.810 sec)
INFO:tensorflow:global_step/sec: 140.226
INFO:tensorflow:loss = 85.7089, step = 18301 (0.712 sec)
INFO:tensorflow:global_step/sec: 134.052
INFO:tensorflow:loss = 89.2168, step = 18401 (0.747 sec)
INFO:tensorflow:global_step/sec: 118.349
INFO:tensorflow:loss = 88.0649, step = 18501 (0.844 sec)
INFO:tensorflow:global_step/sec: 117.601
INFO:tensorflow:loss = 100.742, step = 18601 (0.850 sec)
INFO:tensorflow:global_step/sec: 137.84
INFO:tensorflow:loss = 84.214, step = 18701 (0.726 sec)
INFO:tensorflow:global_step/sec: 137.517
INFO:tensorflow:loss = 79.1987, step = 18801 (0.727 sec)
INFO:tensorflow:global_step/sec: 98.0977
INFO:tensorflow:loss = 89.9931, step = 18901 (1.020 sec)
INFO:tensorflow:global_step/sec: 79.5646
INFO:tensorflow:loss = 104.02, step = 19001 (1.257 sec)
INFO:tensorflow:global_step/sec: 79.8784
INFO:tensorflow:loss = 90.2135, step = 19101 (1.251 sec)
INFO:tensorflow:Saving checkpoints for 19201 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 36.8563
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:21:13
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-19201
INFO:tensorflow:Finished evaluation at 2017-11-15-17:21:15
INFO:tensorflow:Saving dict for global step 19201: global_step = 19201, loss = 94.2922, rmse = 9.71042
INFO:tensorflow:Validation (step 19201): loss = 94.2922, rmse = 9.71042, global_step = 19201
INFO:tensorflow:loss = 84.0389, step = 19201 (7.008 sec)
INFO:tensorflow:global_step/sec: 18.3376
INFO:tensorflow:loss = 79.4139, step = 19301 (1.159 sec)
INFO:tensorflow:global_step/sec: 76.9558
INFO:tensorflow:loss = 93.9544, step = 19401 (1.300 sec)
INFO:tensorflow:global_step/sec: 62.4861
INFO:tensorflow:loss = 97.5546, step = 19501 (1.600 sec)
INFO:tensorflow:global_step/sec: 72.3594
INFO:tensorflow:loss = 85.9556, step = 19601 (1.381 sec)
INFO:tensorflow:global_step/sec: 97.3398
INFO:tensorflow:loss = 92.5018, step = 19701 (1.027 sec)
INFO:tensorflow:global_step/sec: 97.3898
INFO:tensorflow:loss = 78.5447, step = 19801 (1.027 sec)
INFO:tensorflow:global_step/sec: 101.8
INFO:tensorflow:loss = 78.1952, step = 19901 (0.982 sec)
INFO:tensorflow:global_step/sec: 108.08
INFO:tensorflow:loss = 67.3773, step = 20001 (0.924 sec)
INFO:tensorflow:global_step/sec: 130.588
INFO:tensorflow:loss = 75.6347, step = 20101 (0.766 sec)
INFO:tensorflow:global_step/sec: 123.66
INFO:tensorflow:loss = 90.7395, step = 20201 (0.809 sec)
INFO:tensorflow:global_step/sec: 143.303
INFO:tensorflow:loss = 95.1072, step = 20301 (0.698 sec)
INFO:tensorflow:global_step/sec: 133.006
INFO:tensorflow:loss = 99.8141, step = 20401 (0.751 sec)
INFO:tensorflow:global_step/sec: 141.709
INFO:tensorflow:loss = 61.6359, step = 20501 (0.706 sec)
INFO:tensorflow:global_step/sec: 142.72
INFO:tensorflow:loss = 82.7771, step = 20601 (0.700 sec)
INFO:tensorflow:global_step/sec: 110.2
INFO:tensorflow:loss = 97.8497, step = 20701 (0.907 sec)
INFO:tensorflow:global_step/sec: 132.631
INFO:tensorflow:loss = 88.8605, step = 20801 (0.758 sec)
INFO:tensorflow:global_step/sec: 122.167
INFO:tensorflow:loss = 109.574, step = 20901 (0.815 sec)
INFO:tensorflow:global_step/sec: 128.525
INFO:tensorflow:loss = 89.0331, step = 21001 (0.778 sec)
INFO:tensorflow:global_step/sec: 128.299
INFO:tensorflow:loss = 78.3089, step = 21101 (0.779 sec)
INFO:tensorflow:global_step/sec: 127.963
INFO:tensorflow:loss = 83.1385, step = 21201 (0.782 sec)
INFO:tensorflow:global_step/sec: 136.898
INFO:tensorflow:loss = 88.5258, step = 21301 (0.730 sec)
INFO:tensorflow:global_step/sec: 129.286
INFO:tensorflow:loss = 92.417, step = 21401 (0.774 sec)
INFO:tensorflow:global_step/sec: 137.107
INFO:tensorflow:loss = 85.5742, step = 21501 (0.729 sec)
INFO:tensorflow:Saving checkpoints for 21601 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:global_step/sec: 69.1783
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:21:38
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-21601
INFO:tensorflow:Finished evaluation at 2017-11-15-17:21:39
INFO:tensorflow:Saving dict for global step 21601: global_step = 21601, loss = 94.121, rmse = 9.7016
INFO:tensorflow:Validation (step 21601): loss = 94.121, rmse = 9.7016, global_step = 21601
INFO:tensorflow:loss = 77.6509, step = 21601 (3.291 sec)
INFO:tensorflow:global_step/sec: 37.1814
INFO:tensorflow:loss = 85.5089, step = 21701 (0.845 sec)
INFO:tensorflow:global_step/sec: 106.687
INFO:tensorflow:loss = 93.8376, step = 21801 (0.937 sec)
INFO:tensorflow:global_step/sec: 105.935
INFO:tensorflow:loss = 81.9773, step = 21901 (0.944 sec)
INFO:tensorflow:global_step/sec: 109.704
INFO:tensorflow:loss = 103.487, step = 22001 (0.911 sec)
INFO:tensorflow:global_step/sec: 120.046
INFO:tensorflow:loss = 88.4914, step = 22101 (0.834 sec)
INFO:tensorflow:global_step/sec: 124.048
INFO:tensorflow:loss = 101.088, step = 22201 (0.805 sec)
INFO:tensorflow:global_step/sec: 127.909
INFO:tensorflow:loss = 98.1419, step = 22301 (0.782 sec)
INFO:tensorflow:global_step/sec: 133.753
INFO:tensorflow:loss = 69.4008, step = 22401 (0.748 sec)
INFO:tensorflow:global_step/sec: 137.429
INFO:tensorflow:loss = 81.0104, step = 22501 (0.727 sec)
INFO:tensorflow:global_step/sec: 138.787
INFO:tensorflow:loss = 86.5658, step = 22601 (0.720 sec)
INFO:tensorflow:global_step/sec: 138.005
INFO:tensorflow:loss = 76.74, step = 22701 (0.725 sec)
INFO:tensorflow:global_step/sec: 141.652
INFO:tensorflow:loss = 95.1791, step = 22801 (0.706 sec)
INFO:tensorflow:global_step/sec: 139.396
INFO:tensorflow:loss = 68.3581, step = 22901 (0.717 sec)
INFO:tensorflow:global_step/sec: 139.625
INFO:tensorflow:loss = 91.7207, step = 23001 (0.716 sec)
INFO:tensorflow:global_step/sec: 138.493
INFO:tensorflow:loss = 91.1131, step = 23101 (0.722 sec)
INFO:tensorflow:global_step/sec: 139.138
INFO:tensorflow:loss = 85.8931, step = 23201 (0.720 sec)
INFO:tensorflow:global_step/sec: 138.266
INFO:tensorflow:loss = 71.9149, step = 23301 (0.723 sec)
INFO:tensorflow:global_step/sec: 139.454
INFO:tensorflow:loss = 102.607, step = 23401 (0.716 sec)
INFO:tensorflow:global_step/sec: 140.845
INFO:tensorflow:loss = 101.642, step = 23501 (0.710 sec)
INFO:tensorflow:global_step/sec: 137.288
INFO:tensorflow:loss = 90.2794, step = 23601 (0.730 sec)
INFO:tensorflow:global_step/sec: 143.313
INFO:tensorflow:loss = 86.5073, step = 23701 (0.696 sec)
INFO:tensorflow:global_step/sec: 141.96
INFO:tensorflow:loss = 69.6256, step = 23801 (0.705 sec)
INFO:tensorflow:global_step/sec: 137.985
INFO:tensorflow:loss = 92.4355, step = 23901 (0.725 sec)
INFO:tensorflow:Saving checkpoints for 24000 into trained_models/reg-model-04/model.ckpt.
INFO:tensorflow:Loss for final step: 82.5325.
* data input_fn:
================
Input file(s): data/valid-*.csv
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================
INFO:tensorflow:Starting evaluation at 2017-11-15-17:21:59
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000
INFO:tensorflow:Finished evaluation at 2017-11-15-17:22:00
INFO:tensorflow:Saving dict for global step 24000: global_step = 24000, loss = 93.9864, rmse = 9.69466
INFO:tensorflow:Restoring parameters from trained_models/reg-model-04/model.ckpt-24000
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:No assets to write.
INFO:tensorflow:SavedModel written to: b"trained_models/reg-model-04/export/Servo/temp-b'1510766521'/saved_model.pb"
.......................................
Experiment finished at 17:22:01
Experiment elapsed time: 244.561297 seconds