Time series prediction using RNNs + Estimators

This notebook illustrates how to:

  1. Creating a Recurrent Neural Network in TensorFlow
  2. Creating a Custom Estimator in tf.contrib.learn

Dependecies


In [105]:
#!/usr/bin/env python

# Copyright 2017 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# original code from: https://github.com/GoogleCloudPlatform/training-data-analyst/tree/master/blogs/timeseries
# modified by: Marianne Linhares, monteirom@google.com, May 2017

# tensorflow
import tensorflow as tf
import tensorflow.contrib.learn as tflearn
import tensorflow.contrib.layers as tflayers
from tensorflow.contrib.learn.python.learn import learn_runner
import tensorflow.contrib.metrics as metrics
import tensorflow.contrib.rnn as rnn
# Rnn common functions
from tensorflow.contrib.learn.python.learn.estimators import rnn_common

# visualization
import matplotlib.pyplot as plt

# helpers
import numpy as np
import pandas as pd
import csv

# enable tensorflow logs
tf.logging.set_verbosity(tf.logging.INFO)

Describing the data set and the model

We're using a weather dataset....[DESCRIBE DATA SET, HOW THE DATA WAS GENERATED AND OTHER DETAILS].

The goal is: based on features from the past days predict the avg temperature in the next day. More specifically we'll use the data from 10 days in sequence to predict the avg temperature in the next day.

Preparing the data

First let's prepare the data for the RNN, the RNN input is:

  • x = sequence of features
  • y = what we want to predict/classify from x, in our casewe want to predict the next avg temperature

Then we have to separe trainning data from test data.


In [71]:
df = pd.read_csv('weather.csv')

number_of_rows = len(df)
print('number of rows in the dataset:', number_of_rows)

print('how a row looks like:')
print(df.head(11))

print()
print("we don't the year mo da columns, so let's forget about them")
df = df[['avg_tmp', 'avg_dewp', 'avg_slp']]
print(df.head(11))


number of rows in the dataset: 32094
how a row looks like:
    year  mo  da  avg_tmp  avg_dewp   avg_slp
0   1929   8   1   58.200    52.875   994.950
1   1929   8   2   55.925    49.500  1013.075
2   1929   8   3   56.650    51.350  1013.675
3   1929   8   4   60.625    55.350  1004.625
4   1929   8   5   58.450    48.300  1010.400
5   1929   8   6   58.700    56.525  1008.000
6   1929   8   7   58.525    54.975  1007.825
7   1929   8   8   58.200    49.925  1015.800
8   1929   8   9   58.850    52.150  1016.400
9   1929   8  10   62.225    56.175  1018.250
10  1929   8  11   62.975    57.900  1017.550

we don't the year mo da columns, so let's forget about them
    avg_tmp  avg_dewp   avg_slp
0    58.200    52.875   994.950
1    55.925    49.500  1013.075
2    56.650    51.350  1013.675
3    60.625    55.350  1004.625
4    58.450    48.300  1010.400
5    58.700    56.525  1008.000
6    58.525    54.975  1007.825
7    58.200    49.925  1015.800
8    58.850    52.150  1016.400
9    62.225    56.175  1018.250
10   62.975    57.900  1017.550

In [72]:
SEQ_LEN = 10
VALID_ROWS = number_of_rows - SEQ_LEN - 1
NUM_FEATURES = 3

# then we can use indexes to access rows easily
df = np.asarray(df)

# sequences will have shape: [VALID_ROWS, SEQ_LEN, NUM_FEATURES]
sequences = np.zeros((VALID_ROWS, SEQ_LEN, NUM_FEATURES), dtype=np.float32)
labels = np.zeros((VALID_ROWS, 1))

# if the sequence would have len < SEQ_LEN we don't want to use it
# @monteirom: but we can, just need to pass the seq_len as parameter to the dynamic RNN,
# but for now let's keep things simple
for i in range(VALID_ROWS):
    sequences[i] = df[i: i + SEQ_LEN]
    labels[i] = df[i + SEQ_LEN][0]

print('-' * 20)
print('Example')
print('-' * 20)

print('sequence:')
print(sequences[0])
print('prediction:', labels[0])


--------------------
Example
--------------------
sequence:
[[   58.20000076    52.875        994.95001221]
 [   55.92499924    49.5         1013.07501221]
 [   56.65000153    51.34999847  1013.67498779]
 [   60.625         55.34999847  1004.625     ]
 [   58.45000076    48.29999924  1010.40002441]
 [   58.70000076    56.52500153  1008.        ]
 [   58.52500153    54.97499847  1007.82501221]
 [   58.20000076    49.92499924  1015.79998779]
 [   58.84999847    52.15000153  1016.40002441]
 [   62.22499847    56.17499924  1018.25      ]]
prediction: [ 62.975]

Separating training, evaluation and a small test data

The test data is going to be very small (10 sequences) and is being used just for visualization.


In [146]:
# these values are based on the number of valid rows which is 32083
TRAIN_SIZE = 30000
EVAL_SIZE = 2073
TEST_SIZE = 10

# TODO(@monteirom): suffle

train_seq = sequences[:TRAIN_SIZE]
train_label = np.asarray(labels[:TRAIN_SIZE], dtype=np.float32)

eval_seq = sequences[TRAIN_SIZE: TRAIN_SIZE + EVAL_SIZE]
eval_label = np.asarray(labels[TRAIN_SIZE:TRAIN_SIZE + EVAL_SIZE], dtype=np.float32)

test_seq = sequences[TRAIN_SIZE + EVAL_SIZE: ]
test_label = np.asarray(labels[TRAIN_SIZE + EVAL_SIZE: ], dtype=np.float32)

print('train shape:', train_seq.shape)
print('eval shape:', eval_seq.shape)
print('test shape:', test_seq.shape)


train shape: (30000, 10, 3)
eval shape: (2073, 10, 3)
test shape: (10, 10, 3)

What we want to predict

This is the plot of the labels from the test data.


In [166]:
# getting test labels
test_plot_data = [test_label[i][0] for i in range(TEST_SIZE)]

# plotting
sns.tsplot(test_plot_data)
plt.show()


Defining Input functions


In [148]:
BATCH_SIZE = 64

FEATURE_KEY = 'x'
SEQ_LEN_KEY = 'sequence_length'
def make_dict(x):
    d = {}
    d[FEATURE_KEY] = x
    # [SIZE OF DATA SET, 1]
    # where the second dimesion contains the sequence of each
    # sequence in the data set
    d[SEQ_LEN_KEY] = np.asarray(x.shape[0] * [SEQ_LEN], dtype=np.int32)
    return d

# Make input function for training: 
#   num_epochs=None -> will cycle through input data forever
#   shuffle=True -> randomize order of input data
train_input_fn = tf.estimator.inputs.numpy_input_fn(x=make_dict(train_seq),
                                                    y=train_label,
                                                    batch_size=BATCH_SIZE,
                                                    shuffle=True,
                                                    num_epochs=None)

# Make input function for evaluation:
# shuffle=False -> do not randomize input data
eval_input_fn = tf.estimator.inputs.numpy_input_fn(x=make_dict(eval_seq),
                                                    y=eval_label,
                                                    batch_size=BATCH_SIZE,
                                                    shuffle=False)

# Make input function for testing:
# shuffle=False -> do not randomize input data
test_input_fn = tf.estimator.inputs.numpy_input_fn(x=make_dict(test_seq),
                                                    y=test_label,
                                                    batch_size=1,
                                                    shuffle=False)

RNN Model


In [140]:
N_OUTPUTS = 1 # 1 prediction
NUM_FEATURES = 3

def get_model_fn(rnn_cell_sizes,
                 label_dimension,
                 dnn_layer_sizes=[],
                 optimizer='SGD',
                 learning_rate=0.01):
    
    def model_fn(features, targets, mode, params):

        x = features[FEATURES_KEY]
        sequence_length = features[SEQ_LEN_KEY]

        # 1. configure the RNN
        
        # Each RNN layer will consist of a LSTM cell
        rnn_layers = [tf.nn.rnn_cell.LSTMCell(size) for size in rnn_cell_sizes]
        
        # Construct the layers
        multi_rnn_cell = tf.nn.rnn_cell.MultiRNNCell(rnn_layers)
        
        outputs, _ = tf.nn.dynamic_rnn(multi_rnn_cell, x, dtype=tf.float32)

        # Slice to keep only the last cell of the RNN
        last_activations = rnn_common.select_last_activations(outputs,
                                                              sequence_length)

        # Construct dense layers on top of the last cell of the RNN
        for units in dnn_layer_sizes:
            last_activations = tf.layers.dense(last_activations,
                                               units,
                                               activation=tf.nn.relu)
        
        # Final dense layer for prediction
        predictions = tf.layers.dense(last_activations, label_dimension)
        
        # 2. Define the loss function for training/evaluation
        #print 'targets={}'.format(targets)
        #print 'preds={}'.format(predictions)
        loss = tf.losses.mean_squared_error(targets, predictions)
        eval_metric_ops = {
          "rmse": tf.metrics.root_mean_squared_error(targets, predictions)
        }

        # 3. Define the training operation/optimizer
        train_op = tf.contrib.layers.optimize_loss(
          loss=loss,
          global_step=tf.contrib.framework.get_global_step(),
          learning_rate=learning_rate,
          optimizer=optimizer)

        # 4. Create predictions
        predictions_dict = {"predicted": predictions}

        # 5. return ModelFnOps
        return tflearn.ModelFnOps(
          mode=mode,
          predictions=predictions_dict,
          loss=loss,
          train_op=train_op,
          eval_metric_ops=eval_metric_ops)
    
    return model_fn

Running model


In [141]:
model_fn = get_model_fn(rnn_cell_sizes=[64], # size of the hidden layers
                        label_dimension=1, # since is just 1 prediction
                        dnn_layer_sizes=[32], # size of units in the dense layers on top of the RNN
                        optimizer='Adam',
                        learning_rate=0.001)

estimator = tf.contrib.learn.Estimator(model_fn=model_fn)


INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpx2dxdi53
INFO:tensorflow:Using config: {'_evaluation_master': '', '_session_config': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd2c4378d68>, '_keep_checkpoint_max': 5, '_environment': 'local', '_num_worker_replicas': 0, '_model_dir': '/tmp/tmpx2dxdi53', '_num_ps_replicas': 0, '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1
}
, '_master': '', '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_task_id': 0, '_save_checkpoints_secs': 600, '_tf_random_seed': None, '_is_chief': True, '_task_type': None, '_keep_checkpoint_every_n_hours': 10000}
WARNING:tensorflow:Estimator's model_fn (<function get_model_fn.<locals>.model_fn at 0x7fd2bf6a2488>) includes params argument, but params are not passed to Estimator.

Trainning


In [167]:
estimator.fit(input_fn=train_input_fn, steps=10000)


/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
INFO:tensorflow:Create CheckpointSaverHook.
WARNING:tensorflow:Error encountered when serializing LAYER_NAME_UIDS.
Type is unsupported, or the types of the items don't match field type in CollectionDef.
'dict' object has no attribute 'name'
INFO:tensorflow:Restoring parameters from /tmp/tmpx2dxdi53/model.ckpt-10000
WARNING:tensorflow:Error encountered when serializing LAYER_NAME_UIDS.
Type is unsupported, or the types of the items don't match field type in CollectionDef.
'dict' object has no attribute 'name'
INFO:tensorflow:Saving checkpoints for 10001 into /tmp/tmpx2dxdi53/model.ckpt.
WARNING:tensorflow:Error encountered when serializing LAYER_NAME_UIDS.
Type is unsupported, or the types of the items don't match field type in CollectionDef.
'dict' object has no attribute 'name'
INFO:tensorflow:loss = 0.624791, step = 10001
INFO:tensorflow:global_step/sec: 174.571
INFO:tensorflow:loss = 0.609394, step = 10101 (0.574 sec)
INFO:tensorflow:global_step/sec: 214.748
INFO:tensorflow:loss = 3.75179, step = 10201 (0.466 sec)
INFO:tensorflow:global_step/sec: 217.294
INFO:tensorflow:loss = 2.38958, step = 10301 (0.460 sec)
INFO:tensorflow:global_step/sec: 224.35
INFO:tensorflow:loss = 2.87315, step = 10401 (0.446 sec)
INFO:tensorflow:global_step/sec: 220.56
INFO:tensorflow:loss = 0.565767, step = 10501 (0.453 sec)
INFO:tensorflow:global_step/sec: 221.665
INFO:tensorflow:loss = 0.478508, step = 10601 (0.451 sec)
INFO:tensorflow:global_step/sec: 225.538
INFO:tensorflow:loss = 4.27246, step = 10701 (0.443 sec)
INFO:tensorflow:global_step/sec: 219.726
INFO:tensorflow:loss = 1.33334, step = 10801 (0.455 sec)
INFO:tensorflow:global_step/sec: 224.38
INFO:tensorflow:loss = 2.66933, step = 10901 (0.446 sec)
INFO:tensorflow:global_step/sec: 220.75
INFO:tensorflow:loss = 0.3985, step = 11001 (0.453 sec)
INFO:tensorflow:global_step/sec: 221.944
INFO:tensorflow:loss = 2.97408, step = 11101 (0.451 sec)
INFO:tensorflow:global_step/sec: 224.588
INFO:tensorflow:loss = 2.2236, step = 11201 (0.445 sec)
INFO:tensorflow:global_step/sec: 219.788
INFO:tensorflow:loss = 0.695852, step = 11301 (0.455 sec)
INFO:tensorflow:global_step/sec: 223.554
INFO:tensorflow:loss = 0.60163, step = 11401 (0.447 sec)
INFO:tensorflow:global_step/sec: 222.059
INFO:tensorflow:loss = 0.445206, step = 11501 (0.450 sec)
INFO:tensorflow:global_step/sec: 206.39
INFO:tensorflow:loss = 3.94332, step = 11601 (0.485 sec)
INFO:tensorflow:global_step/sec: 220.998
INFO:tensorflow:loss = 2.1236, step = 11701 (0.452 sec)
INFO:tensorflow:global_step/sec: 223.531
INFO:tensorflow:loss = 2.1767, step = 11801 (0.447 sec)
INFO:tensorflow:global_step/sec: 224.329
INFO:tensorflow:loss = 0.266538, step = 11901 (0.446 sec)
INFO:tensorflow:global_step/sec: 222.393
INFO:tensorflow:loss = 0.430484, step = 12001 (0.450 sec)
INFO:tensorflow:global_step/sec: 221.83
INFO:tensorflow:loss = 6.77654, step = 12101 (0.451 sec)
INFO:tensorflow:global_step/sec: 217.858
INFO:tensorflow:loss = 1.45121, step = 12201 (0.459 sec)
INFO:tensorflow:global_step/sec: 216.306
INFO:tensorflow:loss = 1.27801, step = 12301 (0.462 sec)
INFO:tensorflow:global_step/sec: 224.585
INFO:tensorflow:loss = 0.386939, step = 12401 (0.445 sec)
INFO:tensorflow:global_step/sec: 211.697
INFO:tensorflow:loss = 2.28383, step = 12501 (0.472 sec)
INFO:tensorflow:global_step/sec: 209.802
INFO:tensorflow:loss = 1.4403, step = 12601 (0.477 sec)
INFO:tensorflow:global_step/sec: 221.938
INFO:tensorflow:loss = 0.572057, step = 12701 (0.450 sec)
INFO:tensorflow:global_step/sec: 217.306
INFO:tensorflow:loss = 0.521414, step = 12801 (0.460 sec)
INFO:tensorflow:global_step/sec: 220.574
INFO:tensorflow:loss = 0.294471, step = 12901 (0.453 sec)
INFO:tensorflow:global_step/sec: 226.12
INFO:tensorflow:loss = 3.5573, step = 13001 (0.442 sec)
INFO:tensorflow:global_step/sec: 222.123
INFO:tensorflow:loss = 2.63305, step = 13101 (0.450 sec)
INFO:tensorflow:global_step/sec: 215.744
INFO:tensorflow:loss = 1.08016, step = 13201 (0.464 sec)
INFO:tensorflow:global_step/sec: 222.628
INFO:tensorflow:loss = 0.52581, step = 13301 (0.449 sec)
INFO:tensorflow:global_step/sec: 216.086
INFO:tensorflow:loss = 0.389347, step = 13401 (0.463 sec)
INFO:tensorflow:global_step/sec: 222.423
INFO:tensorflow:loss = 8.7956, step = 13501 (0.450 sec)
INFO:tensorflow:global_step/sec: 216.036
INFO:tensorflow:loss = 2.03815, step = 13601 (0.463 sec)
INFO:tensorflow:global_step/sec: 220.041
INFO:tensorflow:loss = 0.990746, step = 13701 (0.454 sec)
INFO:tensorflow:global_step/sec: 222.59
INFO:tensorflow:loss = 0.414991, step = 13801 (0.449 sec)
INFO:tensorflow:global_step/sec: 221.039
INFO:tensorflow:loss = 0.753714, step = 13901 (0.452 sec)
INFO:tensorflow:global_step/sec: 207.066
INFO:tensorflow:loss = 2.65897, step = 14001 (0.483 sec)
INFO:tensorflow:global_step/sec: 218.257
INFO:tensorflow:loss = 0.692365, step = 14101 (0.458 sec)
INFO:tensorflow:global_step/sec: 216.326
INFO:tensorflow:loss = 0.446008, step = 14201 (0.462 sec)
INFO:tensorflow:global_step/sec: 217.652
INFO:tensorflow:loss = 0.572175, step = 14301 (0.459 sec)
INFO:tensorflow:global_step/sec: 221.894
INFO:tensorflow:loss = 3.4343, step = 14401 (0.451 sec)
INFO:tensorflow:global_step/sec: 215.258
INFO:tensorflow:loss = 1.82866, step = 14501 (0.465 sec)
INFO:tensorflow:global_step/sec: 211.972
INFO:tensorflow:loss = 2.17783, step = 14601 (0.472 sec)
INFO:tensorflow:global_step/sec: 222.926
INFO:tensorflow:loss = 0.488388, step = 14701 (0.449 sec)
INFO:tensorflow:global_step/sec: 220.29
INFO:tensorflow:loss = 0.313431, step = 14801 (0.454 sec)
INFO:tensorflow:global_step/sec: 208.979
INFO:tensorflow:loss = 8.44724, step = 14901 (0.478 sec)
INFO:tensorflow:global_step/sec: 215.197
INFO:tensorflow:loss = 2.22418, step = 15001 (0.465 sec)
INFO:tensorflow:global_step/sec: 212.084
INFO:tensorflow:loss = 1.14468, step = 15101 (0.472 sec)
INFO:tensorflow:global_step/sec: 219.677
INFO:tensorflow:loss = 0.322908, step = 15201 (0.455 sec)
INFO:tensorflow:global_step/sec: 211.732
INFO:tensorflow:loss = 0.517554, step = 15301 (0.472 sec)
INFO:tensorflow:global_step/sec: 216.612
INFO:tensorflow:loss = 3.41087, step = 15401 (0.462 sec)
INFO:tensorflow:global_step/sec: 223.754
INFO:tensorflow:loss = 1.36985, step = 15501 (0.447 sec)
INFO:tensorflow:global_step/sec: 206.881
INFO:tensorflow:loss = 0.42084, step = 15601 (0.483 sec)
INFO:tensorflow:global_step/sec: 223.11
INFO:tensorflow:loss = 0.421255, step = 15701 (0.448 sec)
INFO:tensorflow:global_step/sec: 216.216
INFO:tensorflow:loss = 2.87442, step = 15801 (0.462 sec)
INFO:tensorflow:global_step/sec: 224.895
INFO:tensorflow:loss = 2.87897, step = 15901 (0.445 sec)
INFO:tensorflow:global_step/sec: 220.474
INFO:tensorflow:loss = 2.36745, step = 16001 (0.454 sec)
INFO:tensorflow:global_step/sec: 226.613
INFO:tensorflow:loss = 0.626587, step = 16101 (0.441 sec)
INFO:tensorflow:global_step/sec: 218.955
INFO:tensorflow:loss = 0.565466, step = 16201 (0.457 sec)
INFO:tensorflow:global_step/sec: 221.42
INFO:tensorflow:loss = 5.32842, step = 16301 (0.452 sec)
INFO:tensorflow:global_step/sec: 220.903
INFO:tensorflow:loss = 1.11754, step = 16401 (0.453 sec)
INFO:tensorflow:global_step/sec: 221.529
INFO:tensorflow:loss = 1.79527, step = 16501 (0.451 sec)
INFO:tensorflow:global_step/sec: 221.047
INFO:tensorflow:loss = 0.566482, step = 16601 (0.452 sec)
INFO:tensorflow:global_step/sec: 221.413
INFO:tensorflow:loss = 0.410068, step = 16701 (0.452 sec)
INFO:tensorflow:global_step/sec: 226.995
INFO:tensorflow:loss = 2.43707, step = 16801 (0.441 sec)
INFO:tensorflow:global_step/sec: 221.305
INFO:tensorflow:loss = 1.30276, step = 16901 (0.452 sec)
INFO:tensorflow:global_step/sec: 216.459
INFO:tensorflow:loss = 1.20359, step = 17001 (0.462 sec)
INFO:tensorflow:global_step/sec: 221.005
INFO:tensorflow:loss = 0.364844, step = 17101 (0.453 sec)
INFO:tensorflow:global_step/sec: 215.181
INFO:tensorflow:loss = 3.30819, step = 17201 (0.465 sec)
INFO:tensorflow:global_step/sec: 214.663
INFO:tensorflow:loss = 2.19812, step = 17301 (0.466 sec)
INFO:tensorflow:global_step/sec: 218.919
INFO:tensorflow:loss = 1.42075, step = 17401 (0.457 sec)
INFO:tensorflow:global_step/sec: 224.003
INFO:tensorflow:loss = 0.396161, step = 17501 (0.446 sec)
INFO:tensorflow:global_step/sec: 217.932
INFO:tensorflow:loss = 0.419296, step = 17601 (0.459 sec)
INFO:tensorflow:global_step/sec: 221.891
INFO:tensorflow:loss = 4.25041, step = 17701 (0.451 sec)
INFO:tensorflow:global_step/sec: 214.559
INFO:tensorflow:loss = 1.06043, step = 17801 (0.466 sec)
INFO:tensorflow:global_step/sec: 215.11
INFO:tensorflow:loss = 1.88006, step = 17901 (0.465 sec)
INFO:tensorflow:global_step/sec: 211.211
INFO:tensorflow:loss = 0.433759, step = 18001 (0.474 sec)
INFO:tensorflow:global_step/sec: 220.434
INFO:tensorflow:loss = 0.208091, step = 18101 (0.454 sec)
INFO:tensorflow:global_step/sec: 217.259
INFO:tensorflow:loss = 5.40114, step = 18201 (0.460 sec)
INFO:tensorflow:global_step/sec: 213.747
INFO:tensorflow:loss = 1.23059, step = 18301 (0.468 sec)
INFO:tensorflow:global_step/sec: 220.907
INFO:tensorflow:loss = 0.691913, step = 18401 (0.453 sec)
INFO:tensorflow:global_step/sec: 220.307
INFO:tensorflow:loss = 0.417544, step = 18501 (0.454 sec)
INFO:tensorflow:global_step/sec: 224.785
INFO:tensorflow:loss = 2.1053, step = 18601 (0.445 sec)
INFO:tensorflow:global_step/sec: 220.753
INFO:tensorflow:loss = 1.67306, step = 18701 (0.453 sec)
INFO:tensorflow:global_step/sec: 224.163
INFO:tensorflow:loss = 1.28819, step = 18801 (0.446 sec)
INFO:tensorflow:global_step/sec: 221.593
INFO:tensorflow:loss = 0.706468, step = 18901 (0.451 sec)
INFO:tensorflow:global_step/sec: 215.683
INFO:tensorflow:loss = 0.291713, step = 19001 (0.464 sec)
INFO:tensorflow:global_step/sec: 220.87
INFO:tensorflow:loss = 5.20336, step = 19101 (0.453 sec)
INFO:tensorflow:global_step/sec: 225.05
INFO:tensorflow:loss = 1.2072, step = 19201 (0.444 sec)
INFO:tensorflow:global_step/sec: 224.94
INFO:tensorflow:loss = 1.84804, step = 19301 (0.445 sec)
INFO:tensorflow:global_step/sec: 226.687
INFO:tensorflow:loss = 0.331116, step = 19401 (0.441 sec)
INFO:tensorflow:global_step/sec: 221.129
INFO:tensorflow:loss = 0.334256, step = 19501 (0.452 sec)
INFO:tensorflow:global_step/sec: 222.141
INFO:tensorflow:loss = 8.76528, step = 19601 (0.450 sec)
INFO:tensorflow:global_step/sec: 220.814
INFO:tensorflow:loss = 2.03203, step = 19701 (0.453 sec)
INFO:tensorflow:global_step/sec: 215.718
INFO:tensorflow:loss = 0.922788, step = 19801 (0.464 sec)
INFO:tensorflow:global_step/sec: 221.304
INFO:tensorflow:loss = 0.334275, step = 19901 (0.452 sec)
INFO:tensorflow:Saving checkpoints for 20000 into /tmp/tmpx2dxdi53/model.ckpt.
WARNING:tensorflow:Error encountered when serializing LAYER_NAME_UIDS.
Type is unsupported, or the types of the items don't match field type in CollectionDef.
'dict' object has no attribute 'name'
INFO:tensorflow:Loss for final step: 3.37264.
Out[167]:
Estimator(params=None)

Evaluating


In [168]:
ev = estimator.evaluate(input_fn=eval_input_fn)
print(ev)


/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
INFO:tensorflow:Starting evaluation at 2017-06-15-18:57:22
INFO:tensorflow:Restoring parameters from /tmp/tmpx2dxdi53/model.ckpt-20000
INFO:tensorflow:Finished evaluation at 2017-06-15-18:57:23
INFO:tensorflow:Saving dict for global step 20000: global_step = 20000, loss = 1.12044, rmse = 1.05857
WARNING:tensorflow:Error encountered when serializing LAYER_NAME_UIDS.
Type is unsupported, or the types of the items don't match field type in CollectionDef.
'dict' object has no attribute 'name'
{'loss': 1.120443, 'global_step': 20000, 'rmse': 1.058568}

Testing


In [169]:
preds = list(estimator.predict(input_fn=test_input_fn))
predictions = []
for p in preds:
    print(p)
    predictions.append(p["predicted"][0])


/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
INFO:tensorflow:Restoring parameters from /tmp/tmpx2dxdi53/model.ckpt-20000
{'predicted': array([ 65.95652771], dtype=float32)}
{'predicted': array([ 66.04210663], dtype=float32)}
{'predicted': array([ 66.49085236], dtype=float32)}
{'predicted': array([ 66.49862671], dtype=float32)}
{'predicted': array([ 66.5960083], dtype=float32)}
{'predicted': array([ 66.78509521], dtype=float32)}
{'predicted': array([ 67.30899048], dtype=float32)}
{'predicted': array([ 67.2804718], dtype=float32)}
{'predicted': array([ 67.24388885], dtype=float32)}
{'predicted': array([ 67.93393707], dtype=float32)}

Visualizing predictions


In [170]:
# plotting real values in black
sns.tsplot(test_plot_data, color="black")
# plotting predictions in red
sns.tsplot(predictions, color="red")

plt.show()