Introduction to ML Deployment

Deploying models created using python in a Turi Predictive Service is very easy. This notebook walks you through the step-by-step process.


Deployment Steps

The notebook has three sections:

  1. Create a model
  2. Create a predictive service
  3. Query the model

If you are deploying a model in an existing Predictive Service instance you can go to step (2) directly.

1. Create a model

Let's train a simple pattern mining model


In [1]:
# In order to run this code, you need an already trianed model (see the accompanying notebook)
import graphlab as gl
model = gl.load_model('pattern_mining_model.gl')
model


[INFO] This commercial license of GraphLab Create is assigned to engr@turi.com.

[INFO] Start server at: ipc:///tmp/graphlab_server-58453 - Server binary: /Users/srikris/.graphlab/anaconda/lib/python2.7/site-packages/graphlab/unity_server - Server log: /tmp/graphlab_server_1445977952.log
[INFO] GraphLab Server Version: 1.6.1
Out[1]:
Class                         : FrequentPatternMiner

Model fields
------------
Min support                   : 1
Max patterns                  : 100
Min pattern length            : 2

Most frequent patterns
----------------------
['CherryTart', 'ApricotDanish']: 3209
['TuileCookie', 'MarzipanCookie']: 3023
['ChocolateCake', 'ChocolateCoffee']: 2652
['CherryTart', 'OperaCake']   : 2625
['GongolaisCookie', 'TruffleCake']: 2620
['StrawberryCake', 'NapoleonCake']: 2615
['ApricotDanish', 'OperaCake']: 2604
['ApricotCroissant', 'BlueberryTart']: 2599
['OrangeJuice', 'CheeseCroissant']: 2575
['CherryTart', 'ApricotDanish', 'OperaCake']: 2487

We can expose the trained model as a REST endpoint. This will allow other applications to consume the predictions from the model.

In order to do that, we wrap the model object in a Python function and add it to the Predictive Service. In the function you may add your own logic for transform input to the model, ensemble different models or manipulate output before returning. Checkout out user guide for more details.

The result of the function needs to be a JSON serializable object.


In [4]:
def predict(x):
    # Construct an SFrame
    sf = gl.SFrame(x)

    # Add your own business logic here    
    
    # Call the predict method on the model.
    predictions = model.predict(sf)
    return predictions['prediction']

2. Create a Predictive Service (One time)

This section shows you how to deploy a Predictive Service to EC2. The EC2 instances used by the Predictive Service will be launched in your own AWS account, so you will be responsible for the cost.

To create a Predictive Service in Amazon AWS, we first configure the EC2 Config object, which contains the configuration parameters required for launching a Predictive Service cluster in EC2. These fields are optional and include the region, instance type, CIDR rules etc. Predictive Service uses this configuration for service creation.

Having configured our EC2 Config object, we're ready to launch a Predictive Service Deployment, There are a few aspects of the Predictive Service that can be customized:

  • Number of nodes in the service - By default the number of hosts (num_hosts) is 1. To obtain good cache utility and high availability, we recommended setting num_hosts to at least 3.
  • State path to persist service state and service logs. This is a s3 location.
  • Port to be used by the server.
  • Other settings, such as SSL credentials etc.

The following code snippet shows you how to create a Predictive Service. You will have to replace the ps_state_path and credentials for your Predictive Service.


In [ ]:
import graphlab as gl

# Replace with your path.
ps_state_path = 's3://<your-bucket-name>/predictive_service/ps'

# Set your AWS credentials.
gl.aws.set_credentials(<key>, <secret>)

# Create an EC2 config
ec2_config = gl.deploy.Ec2Config()

# Launch a predictive service
ps = gl.deploy.predictive_service.create(name = 'sklearn-predictive-service', 
              ec2_config = ec2_config, state_path = ps_state_path, num_hosts = 1)

Load an already created service


In [5]:
import graphlab as gl
ps = gl.deploy.predictive_service.load('s3://gl-demo-usw2/predictive_service/demolab/ps-1.6')


[WARNING] Overwritting existing Predictive Service "demolab-one-six" in local session.

In [6]:
ps


Out[6]:
Name                  : demolab-one-six
State Path            : s3://gl-demo-usw2/predictive_service/demolab/ps-1.6
Description           : Demo Predictive Service for version 1.6
API Key               : b437e588-0f2b-45e1-81c8-ce3acfa81ade
CORS origin           : *
Global Cache State    : enabled
Load Balancer DNS Name: demolab-one-six-2015364754.us-west-2.elb.amazonaws.com

Deployed endpoints:
	name: freshdress_kw_search, version: 4, type: alias, cache: disabled, description: Alias for freshdress_kw_search_model
	name: freshdress_kw_search_model, version: 2, type: model, cache: enabled, description: 
	name: stratanow_speaker, version: 1, type: model, cache: enabled, description: 
	name: composite_recommender_query, version: 3, type: model, cache: disabled, description: 
	name: freshdress_more_like_image_bow, version: 4, type: model, cache: enabled, description: 
	name: credit_prediction, version: 2, type: model, cache: enabled, description: 
	name: get_ct_bks, version: 1, type: model, cache: enabled, description: 
	name: stratanow_item_sim, version: 1, type: model, cache: enabled, description: 
	name: dress_similar, version: 1, type: model, cache: enabled, description: 
	name: composite_recommender_explanation, version: 3, type: model, cache: disabled, description: 
	name: funny, version: 1, type: model, cache: enabled, description: 
	name: pattern-mining, version: 5, type: model, cache: enabled, description: 
	name: freshdress_debug_model, version: 1, type: model, cache: enabled, description: 
	name: yelp_sentiment_most_extreme, version: 3, type: model, cache: enabled, description: 
	name: mahesh_recommender, version: 1, type: model, cache: enabled, description: 
	name: swathi_recommender, version: 1, type: model, cache: enabled, description: 
	name: freshdress_more_like_image_tfidf, version: 2, type: model, cache: enabled, description: 
	name: get_ab_bks, version: 1, type: model, cache: enabled, description: 
	name: get_wei, version: 1, type: model, cache: enabled, description: 
	name: freshdress_describe, version: 3, type: alias, cache: disabled, description: Alias for freshdress_describe_image_basic
	name: is_this_a_likely_sale, version: 1, type: model, cache: enabled, description: 
	name: yelp_sentiment_predict_text, version: 3, type: model, cache: enabled, description: 
	name: get_wb_bks, version: 1, type: model, cache: enabled, description: 
	name: freshdress_more_like_image_color, version: 5, type: model, cache: enabled, description: 
	name: simi_recommender, version: 1, type: model, cache: enabled, description: 
	name: freshdress_more_like_image_bw, version: 5, type: model, cache: enabled, description: 
	name: get_basis, version: 1, type: model, cache: enabled, description: 
	name: yelp_sentiment_summary, version: 3, type: model, cache: enabled, description: 
	name: book_recommender, version: 3, type: model, cache: enabled, description: 
	name: yelp_sentiment_most_extreme_for_place, version: 3, type: model, cache: enabled, description: 
	name: freshdress_more_like_image_avg_color, version: 2, type: model, cache: enabled, description: 
	name: classify-sklearn, version: 4, type: model, cache: enabled, description: 
	name: freshdress_more_like_image, version: 29, type: alias, cache: disabled, description: Alias for freshdress_more_like_image_bw
	name: get_umang, version: 2, type: model, cache: enabled, description: 
	name: get_books_2, version: 2, type: model, cache: enabled, description: 
	name: house_similar, version: 1, type: model, cache: enabled, description: 
	name: nathan_recommender, version: 1, type: model, cache: enabled, description: 
	name: stratanow_list_page, version: 1, type: model, cache: enabled, description: 
	name: krishna_recommender, version: 1, type: model, cache: enabled, description: 
	name: freshdress_describe_image_basic, version: 1, type: model, cache: enabled, description: 

No Pending changes.

In [7]:
# ps.add('pattern-mining', predict) (When you add this for the first time)
ps.update('pattern-mining', predict)


[INFO] Endpoint 'pattern-mining' is updated. Use apply_changes to deploy all pending changes, or continue other modification.

In [8]:
ps.apply_changes()


[INFO] Uploading local path /var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse to s3 path: s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/objects.bin to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/objects.bin
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/m_4dd4eeb7c6c1758e.frame_idx to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/m_4dd4eeb7c6c1758e.frame_idx
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_25d429e4f3ecd1fb.sidx to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_25d429e4f3ecd1fb.sidx
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/dir_archive.ini to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/dir_archive.ini
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/dir_archive.ini to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/dir_archive.ini
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/m_4dd4eeb7c6c1758e.0000 to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/m_4dd4eeb7c6c1758e.0000
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/m_4dd4eeb7c6c1758e.sidx to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_249b4896e38b5f42/m_4dd4eeb7c6c1758e.sidx
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/dir_archive.ini to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/dir_archive.ini
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_25d429e4f3ecd1fb.0000 to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_25d429e4f3ecd1fb.0000
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/objects.bin to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/objects.bin
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/m_63a1d80a9ff2ff08.0000 to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/m_63a1d80a9ff2ff08.0000
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/m_63a1d80a9ff2ff08.frame_idx to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/m_63a1d80a9ff2ff08.frame_idx
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/m_63a1d80a9ff2ff08.sidx to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_547fd4ded584a8dd/m_63a1d80a9ff2ff08.sidx
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/version to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/version
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/objects.bin to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/objects.bin
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_25d429e4f3ecd1fb.frame_idx to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/d7c43331-ffdf-4089-b33c-e0cb57b6c91e/m_25d429e4f3ecd1fb.frame_idx
Completed 16 of 17 part(s) with 1 file(s) remaining
[INFO] Successfully uploaded to s3 path s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6
[INFO] Notifying: ec2-52-88-112-32.us-west-2.compute.amazonaws.com
upload: ../../../../../../var/folders/9h/1m96s7vn5z72cxt_q7g9p86h0000gn/T/predictive_object_KTxfse/pickle_archive to s3://gl-demo-usw2/predictive_service/demolab/ps-1.6/predictive_objects/pattern-mining/6/pickle_archive

Query the model

You may do a test query before really deploying it to production. This will help detect errors in the function before deploying it the Predictive Service.


In [9]:
# test query to make sure the model works fine
ps.query('pattern-mining', x={'Receipt': [1], 'StoreNum': [2], 'Item': ['CherryTart']})


Out[9]:
{u'from_cache': True,
 u'model': u'pattern-mining',
 u'response': [[u'ApricotDanish']],
 u'uuid': u'9a8aa339-0eec-4c4d-a713-be431fe1d098',
 u'version': 5}

Query from external applications via REST

Now other applications can interact with our model! In the next section we will illustrate how to consume the model. We can also use other APIs like ps.update() to update a mode, ps.remove() to remove a model.

The model query is exposed through REST API. The path is:

http(s)://<your-ps-endpoint>/data/<model-name>

And the payload is a JSON serialized string in the following format:

{"api_key": <api key>,
 "data": <data-passed-to-custom-query>}

Here the 'api key' may be obtained through ps.api_key, and data is the actual data passed to the custom predictive object in the Predictive Service. It will be passed to the query using **kwargs format

Here is a sample curl command to query your model:

curl -X POST -d '{"api_key":"b437e588-0f2b-45e1-81c8-ce3acfa81ade", "data":{"x":{"Receipt": [1], "StoreNum": [2], "Item": ["CherryTart"]}}}' http://demolab-one-six-2015364754.us-west-2.elb.amazonaws.com/query/pattern-mining


You can also query though Python using the requests module

Query through Python


In [10]:
import json
import requests

def restful_query(x):
    headers = {'content-type': 'application/json'}
    payload = {'api_key':'b437e588-0f2b-45e1-81c8-ce3acfa81ade', "data":{"x": x}}
    end_point = 'http://demolab-one-six-2015364754.us-west-2.elb.amazonaws.com/query/pattern-mining'
    return requests.post(end_point, json.dumps(payload), headers=headers).json()

In [11]:
restful_query({'Receipt': [1], 'StoreNum': [2], 'Item': ['CherryTart']})


Out[11]:
{u'from_cache': True,
 u'model': u'pattern-mining',
 u'response': [[u'ApricotDanish']],
 u'uuid': u'd8bab376-da23-4faa-af17-75e21462d3cf',
 u'version': 6}