LAB 5b: Deploy and predict with Keras model on Cloud AI Platform.

Learning Objectives

Setup up the environment
Deploy trained Keras model to Cloud AI Platform
Online predict from model on Cloud AI Platform
Batch predict from model on Cloud AI Platform

Introduction

In this notebook, we'll deploying our Keras model to Cloud AI Platform and creating predictions.

We will set up the environment, deploy a trained Keras model to Cloud AI Platform, online predict from deployed model on Cloud AI Platform, and batch predict from deployed model on Cloud AI Platform.

Each learning objective will correspond to a #TODO in this student lab notebook -- try to complete this notebook first and then review the solution notebook.

Set up environment variables and load necessary libraries

Import necessary libraries.



In [ ]:

    
import os

Lab Task #1: Set environment variables.

Set environment variables so that we can use them throughout the entire lab. We will be using our project name for our bucket, so you only need to change your project and region.



In [ ]:

    
%%bash
PROJECT=$(gcloud config list project --format "value(core.project)")
echo "Your current GCP Project Name is: "$PROJECT



In [ ]:

    
# Change these to try this notebook out
PROJECT = "cloud-training-demos"  # TODO: Replace with your PROJECT
BUCKET = PROJECT  # defaults to PROJECT
REGION = "us-central1"  # TODO: Replace with your REGION



In [ ]:

    
os.environ["BUCKET"] = BUCKET
os.environ["REGION"] = REGION
os.environ["TFVERSION"] = "2.0"



In [ ]:

    
%%bash
gcloud config set compute/region $REGION

Check our trained model files

Let's check the directory structure of our outputs of our trained model in folder we exported the model to in our last lab. We'll want to deploy the saved_model.pb within the timestamped directory as well as the variable values in the variables folder. Therefore, we need the path of the timestamped directory so that everything within it can be found by Cloud AI Platform's model deployment service.



In [ ]:

    
%%bash
gsutil ls gs://${BUCKET}/babyweight/trained_model



In [ ]:

    
%%bash
MODEL_LOCATION=$(gsutil ls -ld -- gs://${BUCKET}/babyweight/trained_model/2* \
                 | tail -1)
gsutil ls ${MODEL_LOCATION}

Lab Task #2: Deploy trained model.

Deploying the trained model to act as a REST web service is a simple gcloud call. Complete #TODO by providing location of saved_model.pb file to Cloud AI Platoform model deployment service. The deployment will take a few minutes.



In [ ]:

    
%%bash
MODEL_NAME="babyweight"
MODEL_VERSION="ml_on_gcp"
MODEL_LOCATION=# TODO: Add GCS path to saved_model.pb file.
echo "Deleting and deploying $MODEL_NAME $MODEL_VERSION from $MODEL_LOCATION"
# gcloud ai-platform versions delete ${MODEL_VERSION} --model ${MODEL_NAME}
# gcloud ai-platform models delete ${MODEL_NAME}
gcloud ai-platform models create ${MODEL_NAME} --regions ${REGION}
gcloud ai-platform versions create ${MODEL_VERSION} \
    --model=${MODEL_NAME} \
    --origin=${MODEL_LOCATION} \
    --runtime-version=1.14 \
    --python-version=3.5

Lab Task #3: Use model to make online prediction.

Complete #TODOs for both the Python and gcloud Shell API methods of calling our deployed model on Cloud AI Platform for online prediction.

Python API

We can use the Python API to send a JSON request to the endpoint of the service to make it predict a baby's weight. The order of the responses are the order of the instances.



In [ ]:

    
from oauth2client.client import GoogleCredentials
import requests
import json

MODEL_NAME = # TODO: Add model name
MODEL_VERSION = # TODO: Add model version

token = GoogleCredentials.get_application_default().get_access_token().access_token
api = "https://ml.googleapis.com/v1/projects/{}/models/{}/versions/{}:predict" \
         .format(PROJECT, MODEL_NAME, MODEL_VERSION)
headers = {"Authorization": "Bearer " + token }
data = {
  "instances": [
    {
      "is_male": "True",
      "mother_age": 26.0,
      "plurality": "Single(1)",
      "gestation_weeks": 39
    },
    {
      "is_male": "False",
      "mother_age": 29.0,
      "plurality": "Single(1)",
      "gestation_weeks": 38
    },
    {
      "is_male": "True",
      "mother_age": 26.0,
      "plurality": "Triplets(3)",
      "gestation_weeks": 39
    },
    # TODO: Create another instance
  ]
}
response = requests.post(api, json=data, headers=headers)
print(response.content)

The predictions for the four instances were: 5.33, 6.09, 2.50, and 5.86 pounds respectively when I ran it (your results might be different).

gcloud shell API

Instead we could use the gcloud shell API. Create a newline delimited JSON file with one instance per line and submit using gcloud.



In [ ]:

    
%%writefile inputs.json
{"is_male": "True", "mother_age": 26.0, "plurality": "Single(1)", "gestation_weeks": 39}
# TODO: Create another instance

Now call gcloud ai-platform predict using the JSON we just created and point to our deployed model and version.



In [ ]:

    
%%bash
gcloud ai-platform predict \
    --model=# TODO: Add model name \
    --json-instances=inputs.json \
    --version=# TODO: Add model version

Lab Task #4: Use model to make batch prediction.

Batch prediction is commonly used when you have thousands to millions of predictions. It will create an actual Cloud AI Platform job for prediction. Complete #TODOs so we can call our deployed model on Cloud AI Platform for batch prediction.



In [ ]:

    
%%bash
INPUT=gs://${BUCKET}/babyweight/batchpred/inputs.json
OUTPUT=gs://${BUCKET}/babyweight/batchpred/outputs
gsutil cp inputs.json $INPUT
gsutil -m rm -rf $OUTPUT 
gcloud ai-platform jobs submit prediction babypred_$(date -u +%y%m%d_%H%M%S) \
    --data-format=TEXT \
    --region ${REGION} \
    --input-paths=$INPUT \
    --output-path=$OUTPUT \
    --model=# TODO: Add model name \
    --version=# TODO: Add model version

Lab Summary:

In this lab, we set up the environment, deployed a trained Keras model to Cloud AI Platform, online predicted from deployed model on Cloud AI Platform, and batch predicted from deployed model on Cloud AI Platform.

Copyright 2019 Google Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License