Deploying models created using scikit-learn in a Turi Predictive Service is very easy. This notebook walks you through the step-by-step process. The notebook has three sections:
If you are deploying a model in an existing Predictive Service instance you can go to step two directly.
Apart from GraphLab Create you will, naturally, need scikit-learn installed in your current Python environment. The most straightforward way to do that is to use conda:
conda install scikit-learnYou will also need a valid AWS account in order to set up a predictive service.
This section shows you how to deploy a Predictive Service to EC2. The EC2 instances used by the Predictive Service will be launched in your own AWS account, so you will be responsible for the cost.
To create a Predictive Service in Amazon AWS, we first configure the EC2 Config object, which contains the configuration parameters required for launching a Predictive Service cluster in EC2. These fields are optional and include the region, instance type, CIDR rules etc. Predictive Service uses this configuration for service creation.
Having configured our EC2 Config object, we're ready to launch a Predictive Service Deployment, There are a few aspects of the Predictive Service that can be customized:
num_hosts) is 1. To obtain good cache utility and high availability, we recommended setting num_hosts to at least 3.The following code snippet shows you how to create a Predictive Service. You will have to replace the ps_state_path and credentials for your Predictive Service.
In [ ]:
import graphlab as gl
# make sure to replace the following with your own information
ps_state_path = 's3://<your-bucket-name>/predictive_service/ps'
# Create an EC2 config
# You can either specify your AWS credentials using environment variables, or
# set them as arguments to this object's constructor
ec2_config = gl.deploy.Ec2Config(
aws_access_key_id='<your access key>',
aws_secret_access_key='<your secret key>')
# use the EC2 config to launch a new Predictive Service
# num_hosts specifies how many hosts the Predictive Service cluster has. You can scale up and down later after initial creation.
ps = gl.deploy.predictive_service.create(
name='sklearn-predictive-service',
ec2_config=ec2_config,
state_path=ps_state_path,
num_hosts=1)
In [5]:
# once the Predictive Service is successfully created, you can query the service status
ps.get_status()
Out[5]:
Let's train a simple random forest model and deploy it in the Predictive Service
In [6]:
from sklearn.ensemble import RandomForestClassifier
X = [[0, 0], [1, 1]]
Y = [0, 1]
clf = RandomForestClassifier(n_estimators=10)
clf = clf.fit(X, Y)
We can expose the trained model as a REST endpoint in the Predictive Service. This will allow other applications to consume the predictions from the model.
In order to do that, we wrap the model object in a Python function and add it to the Predictive Service. In the function you may add your own logic for transform input to the model, ensemble different models or manipulate output before returning. Checkout our user guide for more details.
The result of the function needs to be a JSON serializable object.
In [7]:
def classify(x):
prediction = clf.predict(x)
# convert into a json serializable value
return list(prediction)
# add your predictive function that wraps scikit-learn model
ps.add('classify', classify)
You may do a test query before really deploying it to production. This will help detect errors in the function before deploying it the Predictive Service.
The response to a query is a JSON object with the following keys:
* response: is the actual response from the query;
* uuid: is the unique identifier for your query. The 'uuid' is useful when you need to correlated the query with other data you potentially have for future model tuning.
* version: is the model version. This is useful when you are updating model and you want to know exactly which version served your query
In [8]:
ps.test_query('classify', x=[[0,0],[1,1]])
Out[8]:
It is as expected, let us apply the changes and the predictive model is ready to go!
In [ ]:
# This will push the custom query to the Predictive Service. Since the update is asynchronous, you may need to wait
# a little while before the model is fully deployed.
ps.apply_changes()
Check status and make sure the deployed custom predictive object is fully operational:
In [10]:
# There are other variable way of query status, check API document for more details
ps.get_status('model')
Out[10]:
In [11]:
# test query to make sure the model works fine
ps.query('classify', x=[[0,0],[1,1]])
Out[11]:
Now other applications can interact with our model! In the next section we will illustrate how to consume the model. We can also use other APIs like ps.update() to update a model and ps.remove() to remove a model.
Turi Predictive Services includes a stand-alone Python client for those who just want to query a running service. We will show you how to use the client in the following section. The client takes a configuration file containing the endpoint of the Predictive Service and API key used by client. You can generate the Python client configuration using the following call and hand off the configuration file to your consumer.
In [ ]:
# Generate a client configuration file for Predictive Service Client to consume
# It is a good practice to config a CNAME entry in your DNS provider to have a well known endpoint
# like https://models.companyname.com to point to the Predictive Service so that the consumer of
# the Predictive Service do not need to change their code when you make modifications to your
# Predictive Service
# Here we use None only for demo purpose
ps.save_client_config(file_path='/tmp/ps_client.conf', predictive_service_cname = None)
Once generated, the ps_client.conf file may be passed along to your client side developer. We will show you how to use the file in next section.
The model query is exposed through REST API. The endpoints URL is:
http(s)://<your-ps-endpoint-base>/query/<model-name>
You can find out the endpoint URL base by simply printing the ps object, and copying the Load Balancer DNS Name.
The HTTP call for querying a model or method is a POST call, requiring a JSON-serialized string in the following format as payload:
{ "data": <parameters to model or custom method> }
You also need a valid API key, which you can retreive through ps.api_key.
Here is a sample curl command to query the classify method that we deployed in this notebook:
curl -u api_key:<your-api-key> -d '{"data": {"x": [[0,0],[1,1]]}}'
http://<your-ps-endpoint-base>/query/classifyWe also ship a Python client package that you may easily consume the model. To install the package, do:
pip install GraphLab-Service-Client
After that you may consume the Predictive Model:
In [14]:
from graphlab_service_client import PredictiveServiceClient
# the configuration is saved through ps.save_client_config()
client = PredictiveServiceClient(config_file='/tmp/ps_client.conf')
client.query('classify', x=[[0,0], [1,1]])
Out[14]:
If you don't need to keep you predictive service around for further tasks, make sure to terminate it to avoid incurring unnecessary costs:
In [15]:
ps.terminate_service()
This notebook gives you a peek at what Turi Predictive Service can offer. For a more detailed look at the functionalities in the Turi Predictive Service, checkout out user guide for more details. If you have any questions, post it in our forum and we are happy to assist you!