In this part of the tutorial, you will put on your data scientist hat and train and deploy some models to Clipper to improve your application accuracy.
Because this is a separate Python instance, you must create a new Clipper object and connect to your running Clipper instance. Make sure you enter the same information here as you did in part one.
In [ ]:
import sys
import os
from clipper_admin import Clipper
# Change the username if necessary
user = ""
# Set the path to the SSH key
key = ""
# Set the SSH host
host = ""
clipper = Clipper(host, user, key)
Because this is a new notebook, you must load the CIFAR dataset again. This time, you will be using it to train and evaluate machine learning models.
Set cifar_loc to the same location you did in the "Download the Images" section of part one of the tutorial. You will load into Python the number of training and test datapoints specified in "Extract the images" section of part one.
In [ ]:
cifar_loc = ""
import cifar_utils
train_x, train_y = cifar_utils.filter_data(
*cifar_utils.load_cifar(cifar_loc, cifar_filename="cifar_train.data", norm=True))
test_x, test_y = cifar_utils.filter_data(
*cifar_utils.load_cifar(cifar_loc, cifar_filename="cifar_test.data", norm=True))
When tackling a new problem with machine learning, it's always good to start with simple models and only add complexity when needed. Start by training a logistic regression binary classifier using Scikit-Learn. This model gets about 68% accuracy on the offline evaluation dataset if you use 10,000 training examples. It gets about 74% if you use all 50,000 examples.
In [ ]:
from sklearn import linear_model as lm
def train_sklearn_model(m, train_x, train_y):
m.fit(train_x, train_y)
return m
lr_model = train_sklearn_model(lm.LogisticRegression(), train_x, train_y)
print("Logistic Regression test score: %f" % lr_model.score(test_x, test_y))
While 68-74% accuracy on a CIFAR binary classification task is significantly below state of the art, it's already much better than the 50% accuracy your application yields right now by guessing randomly.
You can deploy your logistic regression model directly to Clipper without having to worry about how to serialize the model or integrate it with application code.
To deploy a model to Clipper, you must assign it a name ("sklearn_cifar"), a version (1), and then provide some metadata about the model itself. In this case, you are specifying that you want to run the model using the sklearn_cifar_container Docker image in the Clipper repo on Docker Hub. You can assign the model descriptive labels, and specify the input type that this model expects. Finally, you can specify how many replicas of the model (how many Docker containers) to launch. Adding more replicas increases the throughput of this model.
After completing this step, Clipper will be managing a new container in Docker with your model in it:
Once again, because you are deploying a Docker image this command may take awhile to download the image. Thanks for being patient!
In [ ]:
model_name = "birds_vs_planes_classifier"
model_added = clipper.deploy_model(
model_name,
1,
lr_model,
"clipper/sklearn_cifar_container:latest",
"doubles",
num_containers=1
)
print("Model deploy successful? {success}".format(success=model_added))
To use your newly deployed model to generate predictions, it needs to be linked to your Clipper application. Let Clipper know that your "cifar_demo" app should use the "birds_vs_planes_classifier" model to serve predictions.
In the future, when you deploy new versions of the "birds_vs_planes_classifier" model, queries to your Clipper application will route to them.
In [ ]:
clipper.link_model_to_app(app_name, model_name)
You can view which models your app is linked to by running the code below.
In [ ]:
clipper.get_linked_models(app_name)
Now that you've deployed and linked your model to your app, go ahead and check back on your running frontend application from part 1. You should see the accuracy rise from around 50% to the accuracy of your SKLearn model (68-74%), without having to stop or modify your application at all!
To improve the accuracy of your application further, you will now deploy a TensorFlow convolutional neural network. This model takes a few hours to train, so you will download the trained model parameters rather than training it from scratch. This model gets about 88% accuracy on the test dataset.
There is a pre-trained TensorFlow model stored in the repo using git-lfs. Once you install git-lfs, you can download the model with the command git lfs pull. If you don't want to deploy a TensorFlow model, you can skip this step.
In [ ]:
import os
import tensorflow as tf
import numpy as np
tf_cifar_model_path = os.path.abspath("tf_cifar_model/cifar10_model_full")
tf_session = tf.Session('', tf.Graph())
with tf_session.graph.as_default():
saver = tf.train.import_meta_graph("%s.meta" % tf_cifar_model_path)
saver.restore(tf_session, tf_cifar_model_path)
def tensorflow_score(session, test_x, test_y):
"""
NOTE: This predict method expects pre-whitened (normalized) images
"""
logits = session.run('softmax_logits:0',
feed_dict={'x:0': test_x})
relevant_activations = logits[:, [cifar_utils.negative_class, cifar_utils.positive_class]]
preds = np.argmax(relevant_activations, axis=1)
return float(np.sum(preds == test_y)) / float(len(test_y))
print("TensorFlow CNN test score: %f" % tensorflow_score(tf_session, test_x, test_y))
Similar to deploying the logistic regression model, you can now deploy your TensorFlow neural network. Note that you will specify a different model container to use this time: the tf_cifar_container. In this case, you are providing Clipper a serialized version of the model. The container has been set up to reconstruct the original model from the serialized representation.
After completing this step, Clipper will send queries to the newly-deployed TensorFlow model instead of the logistic regression Scikit-Learn model, improving the application's accuracy.
Once again, please patient while the Docker image is downloaded.
In [ ]:
model_added = clipper.deploy_model(
model_name,
2,
os.path.abspath("tf_cifar_model"),
"clipper/tf_cifar_container:latest",
"doubles",
num_containers=1
)
print("Model deploy successful? {success}".format(success=model_added))
In [ ]:
clipper.inspect_instance()
Congratulations! You've now successfully completed the tutorial. You started Clipper, created an application and queried it from a frontend client, and deployed two models trained in two different machine learning frameworks (Scikit-Learn and TensorFlow) to the running system.
Head back to the notebook from part 1. When you're done watching the accuracy of your application, stop the cell (hit the little "stop" square in the notebook toolbar).
This step will stop and remove all Clipper Docker containers running on the host. This command will not affect other Docker containers running on the host machine.
When you're completely done with the tutorial and want to shut down your Clipper instance, you can run the stop_all() command to stop all the Clipper Docker containers.
If you check the accuracy of your frontend application a final time, you should see accuracy around 88%. If the accuracy is below that, you can try sending more feedback to increase the weight on the TensorFlow model even more.
In [ ]:
clipper.stop_all()