Deep Nets with TF Abstractions

Let's explore a few of the various abstractions that TensorFlow offers. You can check out the tf.contrib documentation for more options.

The Data

To compare these various abstractions we'll use a dataset easily available from the SciKit Learn library. The data is comprised of the results of a chemical analysis of wines grown in the same region in Italy by three different cultivators. There are thirteen different measurements taken for different constituents found in the three types of wine. We will use the various TF Abstractions to classify the wine to one of the 3 possible labels.

First let's show you how to get the data:



In [ ]:

    
from sklearn.datasets import load_wine



In [ ]:

    
wine_data = load_wine()



In [ ]:

    
type(wine_data)



In [ ]:

    
wine_data.keys()



In [ ]:

    
print(wine_data.DESCR)



In [ ]:

    
feat_data = wine_data['data']



In [ ]:

    
labels = wine_data['target']

Train Test Split



In [ ]:

    
from sklearn.model_selection import train_test_split



In [ ]:

    
X_train, X_test, y_train, y_test = train_test_split(feat_data,
                                                    labels,
                                                    test_size = 0.3,
                                                   random_state = 101)

Scale the Data



In [ ]:

    
from sklearn.preprocessing import MinMaxScaler



In [ ]:

    
scaler = MinMaxScaler()



In [ ]:

    
scaled_x_train = scaler.fit_transform(X_train)



In [ ]:

    
scaled_x_test = scaler.transform(X_test)

Abstractions

Estimator API



In [ ]:

    
import tensorflow as tf



In [ ]:

    
from tensorflow import estimator



In [ ]:

    
X_train.shape



In [ ]:

    
feat_cols = [tf.feature_column.numeric_column("x", shape = [13])]



In [ ]:

    
deep_model = estimator.DNNClassifier(hidden_units = [13,13,13],
                            feature_columns = feat_cols,
                            n_classes = 3,
                            optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01) )



In [ ]:

    
input_fn = estimator.inputs.numpy_input_fn(x = {'x':scaled_x_train},
                                           y = y_train,
                                           shuffle = True,
                                           batch_size = 10,
                                           num_epochs = 5)



In [ ]:

    
deep_model.train(input_fn = input_fn, 
                 steps = 500)



In [ ]:

    
input_fn_eval = estimator.inputs.numpy_input_fn(x = {'x': scaled_x_test},
                                                shuffle = False)



In [ ]:

    
preds = list(deep_model.predict(input_fn = input_fn_eval))



In [ ]:

    
predictions = [p['class_ids'][0] for p in preds]



In [ ]:

    
from sklearn.metrics import confusion_matrix, classification_report



In [ ]:

    
print(classification_report(y_test,predictions))

TensorFlow Keras

Create the Model



In [ ]:

    
from tensorflow.contrib.keras import models



In [ ]:

    
dnn_keras_model = models.Sequential()

Add Layers to the model



In [ ]:

    
from tensorflow.contrib.keras import layers



In [ ]:

    
dnn_keras_model.add(layers.Dense(units = 13, 
                                 input_dim = 13,
                                 activation = 'relu'))



In [ ]:

    
dnn_keras_model.add(layers.Dense(units = 13,
                                 activation = 'relu'))
dnn_keras_model.add(layers.Dense(units = 13,
                                 activation = 'relu'))



In [ ]:

    
dnn_keras_model.add(layers.Dense(units = 3,
                                 activation = 'softmax'))

Compile the Model



In [ ]:

    
from tensorflow.contrib.keras import losses, optimizers, metrics



In [ ]:

    
losses.sparse_categorical_crossentropy



In [ ]:

    
dnn_keras_model.compile(optimizer = 'adam',
              loss = 'sparse_categorical_crossentropy',
              metrics = ['accuracy'])

Train Model



In [ ]:

    
dnn_keras_model.fit(scaled_x_train,y_train,epochs=50)



In [ ]:

    
predictions = dnn_keras_model.predict_classes(scaled_x_test)



In [ ]:

    
print(classification_report(predictions, y_test))

Layers API

https://www.tensorflow.org/tutorials/layers

Formating Data



In [ ]:

    
import pandas as pd
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler



In [ ]:

    
wine_data = load_wine()
feat_data = wine_data['data']
labels = wine_data['target']



In [ ]:

    
X_train, X_test, y_train, y_test = train_test_split(feat_data,
                                                    labels,
                                                    test_size = 0.3,
                                                   random_state = 101)



In [ ]:

    
scaler = MinMaxScaler()
scaled_x_train = scaler.fit_transform(X_train)
scaled_x_test = scaler.transform(X_test)
# ONE HOT ENCODED
onehot_y_train = pd.get_dummies(y_train).as_matrix()
one_hot_y_test = pd.get_dummies(y_test).as_matrix()

Parameters



In [ ]:

    
num_feat = 13
num_hidden1 = 13
num_hidden2 = 13
num_outputs = 3
learning_rate = 0.01



In [ ]:

    
import tensorflow as tf
from tensorflow.contrib.layers import fully_connected

Placeholder



In [ ]:

    
X = tf.placeholder(tf.float32, shape = [None,num_feat])
y_true = tf.placeholder(tf.float32, shape = [None,3])

Activation Function



In [ ]:

    
actf = tf.nn.relu

Create Layers



In [ ]:

    
hidden1 = fully_connected(X, num_hidden1, activation_fn = actf)



In [ ]:

    
hidden2 = fully_connected(hidden1,num_hidden2,activation_fn = actf)



In [ ]:

    
output = fully_connected(hidden2, num_outputs)

Loss Function



In [ ]:

    
loss = tf.losses.softmax_cross_entropy(onehot_labels = y_true, logits = output)

Optimizer



In [ ]:

    
optimizer = tf.train.AdamOptimizer(learning_rate)
train = optimizer.minimize(loss)

Init



In [ ]:

    
init = tf.global_variables_initializer()



In [ ]:

    
training_steps = 1000
with tf.Session() as sess:
    sess.run(init) 
    for i in range(training_steps):
        sess.run(train, feed_dict = {X: scaled_x_train,
                                     y_true: y_train})
        
    # Get Predictions
    logits = output.eval(feed_dict = {X: scaled_x_test})
    
    preds = tf.argmax(logits, 
                      axis = 1)
    
    results = preds.eval()



In [ ]:

    
from sklearn.metrics import confusion_matrix,classification_report
print(classification_report(results,y_test))