Introduction to ML Deployment

Deploying models created using python in a Turi Predictive Service is very easy. This notebook walks you through the step-by-step process.

Deployment Steps

The notebook has three sections:

Create a model
Create a predictive service
Query the model

If you are deploying a model in an existing Predictive Service instance you can go to step (2) directly.

Preliminaries

For the following code, you will need to install the following packages:

pip install graphlab-create
pip install sklearn
pip install numpy
pip install scipy

1. Create a model

Let's train a simple random forest model and deploy it in the Predictive Service.



In [ ]:

    
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
iris = load_iris()

model = RandomForestClassifier(n_estimators=10)
model = model.fit(iris['data'], iris['target'])
model

We can expose the trained model as a REST endpoint. This will allow other applications to consume the predictions from the model.

In order to do that, we wrap the model object in a Python function and add it to the Predictive Service. In the function you may add your own logic for transform input to the model, ensemble different models or manipulate output before returning. Checkout out user guide for more details.

The result of the function needs to be a JSON serializable object.



In [ ]:

    
def classify(x):
    prediction = model.predict(x)

    # convert into a json serializable value
    return list(prediction)

2. Create a Predictive Service (One time)

This section shows you how to deploy a Predictive Service to EC2. The EC2 instances used by the Predictive Service will be launched in your own AWS account, so you will be responsible for the cost.

To create a Predictive Service in Amazon AWS, we first configure the EC2 Config object, which contains the configuration parameters required for launching a Predictive Service cluster in EC2. These fields are optional and include the region, instance type, CIDR rules etc. Predictive Service uses this configuration for service creation.

Having configured our EC2 Config object, we're ready to launch a Predictive Service Deployment, There are a few aspects of the Predictive Service that can be customized:

Number of nodes in the service - By default the number of hosts (num_hosts) is 1. To obtain good cache utility and high availability, we recommended setting num_hosts to at least 3.
State path to persist service state and service logs. This is a s3 location.
Port to be used by the server.
Other settings, such as SSL credentials etc.

The following code snippet shows you how to create a Predictive Service. You will have to replace the ps_state_path and credentials for your Predictive Service.



In [ ]:

    
import graphlab as gl

# Replace with your path.
ps_state_path = 's3://<your-bucket-name>/predictive_service/ps'

# Set your AWS credentials.
gl.aws.set_credentials(<key>, <secret>)

# Create an EC2 config
ec2 = gl.deploy.Ec2Config()

# Launch a predictive service
ps = gl.deploy.predictive_service.create(
    name='sklearn-predictive-service', 
    ec2_config=ec2, state_path=ps_state_path, num_hosts=1)

Load an already created service



In [ ]:

    
import graphlab as gl
ps = gl.deploy.predictive_service.load('s3://gl-demo-usw2/predictive_service/demolab/ps-1.8.4')



In [ ]:

    
ps



In [ ]:

    
# ps.add('classify-sklearn', classify, description='Classify an iris based on petal and sepal dimensions')
ps.update('classify-sklearn', classify, description='Classify an iris based on petal and sepal dimensions')



In [ ]:

    
ps.apply_changes()

Query the model

You may do a test query before really deploying it to production. This will help detect errors in the function before deploying it the Predictive Service.



In [ ]:

    
ps.test_query('classify-sklearn', x=[5.1,  3.5,  1.4,  0.2])

Now, let us query the real service.



In [ ]:

    
# test query to make sure the model works fine
ps.query('classify-sklearn', x=[5.1,  3.5,  1.4,  0.2])

Query from external applications via REST

Now other applications can interact with our model! In the next section we will illustrate how to consume the model. We can also use other APIs like ps.update() to update a mode, ps.remove() to remove a model.

The model query is exposed through REST API. The url to query is:

http(s)://<your-ps-endpoint>/query/<model-name>

The predictive service uses basic access authentication to authorize the client to query. The client needs to provide the service's API key in the HTTP header as the password for user name api_key. The 'api key' may be obtained through ps.api_key

The payload is a JSON serialized string in the following format:

{ "data": <data-passed-to-custom-query> }

The data is the actual data passed to the custom predictive object in the Predictive Service. It will be passed to the query using kwargs format.

Here is a sample curl command to query your model:

curl -u api_key:b0a1c056-30b9-4468-9b8d-c07289017228 -d '{"data":{"x":[5.1,  3.5,  1.4,  0.2]}}' http://demolab-one-six-2015364754.us-west-2.elb.amazonaws.com/query/classify-sklearn

You can also query though Python using the requests module

Query through Python



In [ ]:

    
import json
import requests
from requests.auth import HTTPBasicAuth

def restful_query(x):
    headers = {'content-type': 'application/json'}
    payload = {"data":{"x": x}}
    end_point = 'http://demolab-ps-one-eight-four-810335136.us-west-2.elb.amazonaws.com/query/classify-sklearn'
    return requests.post(
        end_point,
        json.dumps(payload),
        headers=headers,
        auth=HTTPBasicAuth('api_key', '9d97391e-8be7-47a9-8b72-34ecc9f0ad60')).json()



In [ ]:

    
restful_query([5.1,  3.5,  1.4,  0.2])



In [ ]:

    
restful_query([5.1,  3.5,  1.4,  0.3])