Import numpy pandas and matplotlib
In [1]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
Use pandas to read the csv of the monthly-milk-production.csv file and set index_col='Month'
In [2]:
data = pd.read_csv("./data/monthly-milk-production.csv", index_col = 'Month')
Check out the head of the dataframe
In [3]:
data.head()
Out[3]:
Make the index a time series by using:
milk.index = pd.to_datetime(milk.index)
In [4]:
data.index = pd.to_datetime(data.index)
Plot out the time series data.
In [5]:
data.plot()
Out[5]:
Let's attempt to predict a year's worth of data. (12 months or 12 steps into the future)
Create a test train split using indexing (hint: use .head() or tail() or .iloc[]). We don't want a random train test split, we want to specify that the test set is the last 12 months of data is the test set, with everything before it is the training.
In [6]:
data.info()
In [7]:
training_set = data.head(156)
In [8]:
test_set = data.tail(12)
In [9]:
from sklearn.preprocessing import MinMaxScaler
In [10]:
scaler = MinMaxScaler()
In [11]:
training_set = scaler.fit_transform(training_set)
In [12]:
test_set_scaled = scaler.transform(test_set)
We'll need a function that can feed batches of the training data. We'll need to do several things that are listed out as steps in the comments of the function. Remember to reference the previous batch method from the lecture for hints. Try to fill out the function template below, this is a pretty hard step, so feel free to reference the solutions!
In [13]:
def next_batch(training_data, batch_size, steps):
"""
INPUT: Data, Batch Size, Time Steps per batch
OUTPUT: A tuple of y time series results. y[:,:-1] and y[:,1:]
"""
# STEP 1: Use np.random.randint to set a random starting point index for the batch.
# Remember that each batch needs have the same number of steps in it.
# This means you should limit the starting point to len(data)-steps
random_start = np.random.randint(0, len(training_data) - steps)
# STEP 2: Now that you have a starting index you'll need to index the data from
# the random start to random start + steps + 1. Then reshape this data to be (1,steps+1)
# Create Y data for time series in the batches
y_batch = np.array(training_data[random_start : random_start + steps + 1]).reshape(1, steps+1)
# STEP 3: Return the batches. You'll have two batches to return y[:,:-1] and y[:,1:]
# You'll need to reshape these into tensors for the RNN to .reshape(-1,steps,1)
return y_batch[:, :-1].reshape(-1, steps, 1), y_batch[:, 1:].reshape(-1, steps, 1)
Import TensorFlow
In [14]:
import tensorflow as tf
Define the constants in a single cell. You'll need the following (in parenthesis are the values I used in my solution, but you can play with some of these):
In [15]:
num_inputs = 1
num_time_steps = 12
num_neurons = 100
num_outputs = 1
learning_rate = 0.03
num_train_iter = 4000
batch_size = 1
Create Placeholders for X and y. (You can change the variable names if you want). The shape for these placeholders should be [None,num_time_steps-1,num_inputs] and [None, num_time_steps-1, num_outputs] The reason we use num_time_steps-1 is because each of these will be one step shorter than the original time steps size, because we are training the RNN network to predict one point into the future based on the input sequence.
In [16]:
X = tf.placeholder(tf.float32, [None, num_time_steps, num_inputs])
y = tf.placeholder(tf.float32, [None, num_time_steps, num_outputs])
Now create the RNN Layer, you have complete freedom over this, use tf.contrib.rnn and choose anything you want, OutputProjectionWrappers, BasicRNNCells, BasicLSTMCells, MultiRNNCell, GRUCell etc... Keep in mind not every combination will work well! (If in doubt, the solutions used an Outputprojection Wrapper around a basic LSTM cell with relu activation.
In [17]:
cell = tf.contrib.rnn.OutputProjectionWrapper(tf.contrib.rnn.BasicLSTMCell(num_units = num_neurons, activation = tf.nn.relu), output_size = num_outputs)
Now pass in the cells variable into tf.nn.dynamic_rnn, along with your first placeholder (X)
In [18]:
outputs, states = tf.nn.dynamic_rnn(cell, X, dtype = tf.float32)
In [19]:
# MSE
loss = tf.reduce_mean(tf.square(outputs - y))
optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate)
train = optimizer.minimize(loss)
Initialize the global variables
In [20]:
init = tf.global_variables_initializer()
Create an instance of tf.train.Saver()
In [21]:
saver = tf.train.Saver()
In [22]:
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction = 0.75)
In [23]:
with tf.Session() as sess:
# Run
sess.run(init)
for iteration in range(num_train_iter):
X_batch, Y_batch = next_batch(training_set, batch_size, num_time_steps)
sess.run(train, feed_dict = {X: X_batch, y: Y_batch})
if iteration % 100 == 0:
mse = loss.eval(feed_dict = {X: X_batch, y: Y_batch})
print(iteration, "\tMSE:", mse)
# Save Model for Later
saver.save(sess, "./checkpoints/ex_time_series_model")
Show the test_set (the last 12 months of your original complete data set)
In [24]:
test_set
Out[24]:
Now we want to attempt to predict these 12 months of data, using only the training data we had. To do this we will feed in a seed training_instance of the last 12 months of the training_set of data to predict 12 months into the future. Then we will be able to compare our generated 12 months to our actual true historical values from the test set!
Fill out the session code below to generate 12 months of data based off the last 12 months of data from the training set. The hardest part about this is adjusting the arrays with their shapes and sizes. Reference the lecture for hints.
In [25]:
with tf.Session() as sess:
# Use your Saver instance to restore your saved rnn time series model
saver.restore(sess, "./checkpoints/ex_time_series_model")
# Create a numpy array for your genreative seed from the last 12 months of the
# training set data. Hint: Just use tail(12) and then pass it to an np.array
train_seed = list(training_set[-12:])
## Now create a for loop that
for iteration in range(12):
X_batch = np.array(train_seed[-num_time_steps:]).reshape(1, num_time_steps, 1)
y_pred = sess.run(outputs, feed_dict={X: X_batch})
train_seed.append(y_pred[0, -1, 0])
Show the result of the predictions.
In [26]:
train_seed
Out[26]:
Grab the portion of the results that are the generated values and apply inverse_transform on them to turn them back into milk production value units (lbs per cow). Also reshape the results to be (12,1) so we can easily add them to the test_set dataframe.
In [27]:
results = scaler.inverse_transform(np.array(train_seed[12:]).reshape(12, 1))
Create a new column on the test_set called "Generated" and set it equal to the generated results. You may get a warning about this, feel free to ignore it.
In [28]:
test_set['Generated'] = results
View the test_set dataframe.
In [29]:
test_set
Out[29]:
Plot out the two columns for comparison.
In [30]:
test_set.plot()
Out[30]: