Time Series with BentoML and Prophet

This notebook uses BentoML and Kubeflow to build, train and boot an api powered by a Prophet time series model.

The below notebook focuses on building and serving a model using:

Covid19 Data

The notebook uses the Covid19 Kaggle locality infection data as our sample data. The notebook uses my home town New York as the locality.


In [1]:
%%capture

!pip install pandas sklearn auto-sklearn kubeflow-fairing grpcio kubeflow.metadata bentoml plotly fbprophet

In [2]:
import uuid
from importlib import reload
import grpc
from kubeflow import fairing
from kubeflow.fairing import constants
import os
import pandas as pd
import logging

logging.basicConfig(level=logging.WARN)
logger = logging.getLogger(__name__)

Config

The values below will be used throughout this notebook, make sure to adjust these values to match your registry and namespace.


In [3]:
# The docker registry to store images in
DOCKER_REGISTRY = "iancoffey"
# The k8s namespace to run the experiment in
k8s_namespace = "default"
# Use local bentoml storage
!bentoml config set yatai_service.url=""

Minio

This notebook will use Minio as the development context storage for Kubeflow Fairing, but any of the supported blob storage (s3, gcp, etc) will work.

To install Minio, apply the provided Minio manifest into the provided namespace.

kubectl apply -f ./manifests/minio -n $k8s_namespace

We will use the kubernetes sdk to determine the endpoints ip address and then build our MinioContextSource.


In [4]:
from kubernetes import utils as k8s_utils
from kubernetes import client as k8s_client
from kubernetes import config as k8s_config
from kubeflow.fairing.utils import is_running_in_k8s

from kubeflow.fairing.cloud.k8s import MinioUploader
from kubeflow.fairing.builders.cluster.minio_context import MinioContextSource

if is_running_in_k8s():
    k8s_config.load_incluster_config()
else:
    k8s_config.load_kube_config()

api_client = k8s_client.CoreV1Api()
minio_service_endpoint = api_client.read_namespaced_service(name='minio-service', namespace='default').spec.cluster_ip

In [5]:
minio_endpoint = "http://"+minio_service_endpoint+":9000"
minio_username = "minio"
minio_key = "minio123"
minio_region = "us-east-1"

minio_uploader = MinioUploader(endpoint_url=minio_endpoint, minio_secret=minio_username, minio_secret_key=minio_key, region_name=minio_region)
minio_context_source = MinioContextSource(endpoint_url=minio_endpoint, minio_secret=minio_username, minio_secret_key=minio_key, region_name=minio_region)
minio_endpoint


Out[5]:
'http://10.152.183.6:9000'

Start: Preparing the Dataset

To train our data, we need to split the data into train and test datasets as normal. We cant use the normal train_test_split function due to this being timeseries data, so we will have to define our own means of splitting the data by date.


In [6]:
# fairing:include-cell
data_path="covid_19_data.csv"
dframe = pd.read_csv(data_path, sep=',')

In [7]:
cols_of_interest = ['Confirmed', 'Province/State', 'ObservationDate']

dframe['ObservationDate'] = pd.to_datetime(dframe['ObservationDate'])
dframe.sort_index(inplace=True)

trimmed_dframe=dframe[cols_of_interest]
trimmed_dframe=trimmed_dframe.dropna()

# Note the copy() here - else we would be working on a reference
state_data = trimmed_dframe.loc[trimmed_dframe['Province/State'] == 'New York'].copy()
state_data = state_data.drop('Province/State', axis=1).sort_index()

state_data.rename(columns={'Confirmed': 'y', 'ObservationDate': 'ds'}, inplace=True)

state_data.head()


Out[7]:
y ds
4549 173.0 2020-03-10
4755 220.0 2020-03-11
4964 328.0 2020-03-12
5356 421.0 2020-03-13
5411 525.0 2020-03-14

In [8]:
color_pal = ["#F8766D", "#D39200", "#93AA00",
             "#00BA38", "#00C19F", "#00B9E3",
             "#619CFF", "#DB72FB"]


_ = state_data.plot(x ='ds', y='y', kind='scatter', figsize=(15,5), title="Raw Covid19 Dataset")



In [9]:
split_date = "2020-05-15"

train_data = state_data[state_data['ds'] <= split_date].copy()
test_data = state_data[state_data['ds'] >  split_date].copy()

len(state_data)


Out[9]:
93

Model Training

Now that we have gathered and transformed our original covid19 timeseries data, we can create a new Prophet model and fit our training data.

Then, we can use make_future_dataframe to create a dataframe to contain our future timeseries and predictions. We pass periods as 10 to ensure we have 10 days predicted.


In [10]:
# Python
import pandas as pd
from fbprophet import Prophet

m = Prophet()
m.fit(train_data)
future = m.make_future_dataframe(periods=10)
future.tail()


[I 200626 12:49:35 utils:141] NumExpr defaulting to 2 threads.
[I 200626 12:49:35 forecaster:919] Disabling yearly seasonality. Run prophet with yearly_seasonality=True to override this.
[I 200626 12:49:35 forecaster:919] Disabling daily seasonality. Run prophet with daily_seasonality=True to override this.
Out[10]:
ds
72 2020-05-21
73 2020-05-22
74 2020-05-23
75 2020-05-24
76 2020-05-25

Prediction

We can use our new model to forcast our future timeseries and plot the results.


In [11]:
import numpy as np

forecast = m.predict(future)
print(forecast['yhat'].tail())


72    362470.476200
73    365496.015175
74    369703.173415
75    372333.226614
76    374064.903150
Name: yhat, dtype: float64

In [12]:
fig1 = m.plot(forecast, figsize = (15, 10))



In [14]:
for c in forecast.columns.sort_values():
    print(c)


additive_terms
additive_terms_lower
additive_terms_upper
ds
multiplicative_terms
multiplicative_terms_lower
multiplicative_terms_upper
trend
trend_lower
trend_upper
weekly
weekly_lower
weekly_upper
yhat
yhat_lower
yhat_upper

Projected vs Reality

It is helpful to plot our predictions alongside our test data set, divided into two sections of time, defined by the boundary of the training and test datasets. We can see that the predictions were a bit too aggresive.


In [15]:
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('darkgrid', {'axes.facecolor': '.9'})
sns.set_palette(palette='deep')
sns_c = sns.color_palette(palette='deep')

threshold_date = pd.to_datetime(split_date)

fig, ax = plt.subplots(figsize = (15, 10))
sns.lineplot(x='ds', y='y', label='y_train', data=train_data, ax=ax)
sns.lineplot(x='ds', y='y', label='y_test', data=test_data, ax=ax)
sns.lineplot(x='ds', y='trend', data=forecast, ax=ax)
ax.axvline(threshold_date, color=sns_c[3], linestyle='--', label='train test split')
ax.legend(loc='upper left')
ax.set(title='Confirmed Cases', ylabel='');


Define BentoML Service

We will now define the code we want to use to serve our model with BentoML.

Our class and methods will be decorated to shows its artifact type and pip dependencies.

Following that, lets reload the the new code and pack our BentoML service with the booster we have created!


In [16]:
%%writefile prophet_serve.py

import bentoml
from bentoml.handlers import DataframeHandler
from bentoml.artifact import PickleArtifact
import fbprophet

@bentoml.artifacts([PickleArtifact('model')])
@bentoml.env(pip_dependencies=['fbprophet'])
class ProphetServe(bentoml.BentoService):
    @bentoml.api(DataframeHandler)
    def predict(self, df):
        return self.artifacts.model.predict(df)


Overwriting prophet_serve.py

In [17]:
import prophet_serve
import importlib
importlib.reload(prophet_serve)


[I 200626 12:49:42 __init__:116] Loading local BentoML config file: /home/jovyan/bentoml/bentoml.cfg
[2020-06-26 12:49:43,049] WARNING - bentoml.handlers.* will be deprecated after BentoML 1.0, use bentoml.adapters.* instead
[2020-06-26 12:49:43,056] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
[2020-06-26 12:49:43,108] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
Out[17]:
<module 'prophet_serve' from '/home/jovyan/prophet_serve.py'>

Pack BentoML Service

Lets pack the new service with our model, which would allow use to reference this iteration in the future and also build it into Docker images.


In [18]:
from prophet_serve import ProphetServe

bento_service = ProphetServe()
bento_service.pack('model', m)
saved_path = bento_service.save()


[I 200626 12:49:46 migration:155] Context impl SQLiteImpl.
[I 200626 12:49:46 migration:162] Will assume non-transactional DDL.
[I 200626 12:49:46 migration:515] Running stamp_revision  -> a6b00ae45279
[2020-06-26 12:50:05,382] INFO - BentoService bundle 'ProphetServe:20200626124946_94E973' saved to: /home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973

BentoML Service

Now that we have a BentoML Service defined, lets checkout what that looks like via the cli:


In [19]:
!bentoml get ProphetServe


BENTO_SERVICE                       AGE                         APIS                                   ARTIFACTS
ProphetServe:20200626124946_94E973  1 minute and 57.03 seconds  predict<DataframeInput:DefaultOutput>  model<PickleArtifact>

Explore the Bentoml service

Lets pick a service to launch as a service. We will use the most recent one here, but you will need to edit the values below to match yours. This is purely for convience so that we can poke at the BentoML CLI quickly.

We can see inside the autogenerated bento service directory that many files now exist, including a Dockerfile and requirements.txt


In [20]:
!bentoml get ProphetServe:20200626124946_94E973


{
  "name": "ProphetServe",
  "version": "20200626124946_94E973",
  "uri": {
    "type": "LOCAL",
    "uri": "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973"
  },
  "bentoServiceMetadata": {
    "name": "ProphetServe",
    "version": "20200626124946_94E973",
    "createdAt": "2020-06-26T12:50:05.310154Z",
    "env": {
      "condaEnv": "name: bentoml-ProphetServe\nchannels:\n- defaults\ndependencies:\n- python=3.7.6\n- pip\n",
      "pipDependencies": "pandas\nbentoml==0.8.1\nfbprophet",
      "pythonVersion": "3.7.6",
      "dockerBaseImage": "bentoml/model-server:0.8.1"
    },
    "artifacts": [
      {
        "name": "model",
        "artifactType": "PickleArtifact"
      }
    ],
    "apis": [
      {
        "name": "predict",
        "inputType": "DataframeInput",
        "docs": "BentoService API",
        "inputConfig": {
          "orient": "records",
          "typ": "frame",
          "is_batch_input": true,
          "input_dtypes": null
        },
        "outputConfig": {
          "cors": "*"
        },
        "outputType": "DefaultOutput"
      }
    ]
  }
}

Run Bento Service

Now, using our BentoML service, we can make predictions like so:


In [21]:
!bentoml run ProphetServe:20200626124946_94E973 predict --input '{"ds":["2021-07-14"]}'


[2020-06-26 12:52:15,280] WARNING - bentoml.handlers.* will be deprecated after BentoML 1.0, use bentoml.adapters.* instead
[2020-06-26 12:52:15,888] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
INFO:numexpr.utils:NumExpr defaulting to 2 threads.
          ds         trend  ...  multiplicative_terms_upper          yhat
0 2021-07-14  1.471904e+06  ...                         0.0  1.471369e+06

[1 rows x 16 columns]

Explore BentoML Generated Artifacts

BentoML generates a whole environment, including Dockerfile, for generating images based on these saved services.

It is useful to ls this directory and see what it contains.


In [32]:
!ls /home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973


bentoml-init.sh       Dockerfile       ProphetServe	 setup.py
bentoml.yml	      environment.yml  README.md
docker-entrypoint.sh  MANIFEST.in      requirements.txt

Stitching BentoML to Kubeflow Fairing

We can build our models container with Kubeflow Fairing, a cool component which aims to improve data scientist life by streamlining the process of building, training, and deploying machine learning (ML) models.

First we need to do some work to get the BentoML artifacts into our build context. We do this via the preprocessing work below. We are just taking the auto-generated BentoML Docker environment and stitching it together with Kubeflow Fairing.


In [35]:
# Let build a docker image with builder using bentoml output
from kubeflow.fairing.preprocessors.base import BasePreProcessor

output_map =  {
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/Dockerfile": "Dockerfile",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/environment.yml": "environment.yml",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/requirements.txt": "requirements.txt",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/setup.py": "setup.py",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/bentoml-init.sh": "bentoml-init.sh",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/bentoml.yml": "bentoml.yml",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/ProphetServe/": "ProphetServe/",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/ProphetServe/prophet_serve.py": "ProphetServe/prophet_serve.py",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/ProphetServe/artifacts/model.pkl": "ProphetServe/artifacts/model.pkl",
    "/home/jovyan/bentoml/repository/ProphetServe/20200626124946_94E973/docker-entrypoint.sh": "docker-entrypoint.sh",    
}

preprocessor = BasePreProcessor(output_map=output_map)

preprocessor.preprocess()


Out[35]:
set()

Cluster Building

In this workflow, we want to develop our models locally, until we arrive at a model we want to deploy. From there we want to use cloud resources to build Docker images to deploy to our Kubernetes cluster.

These Docker images will be built in our private cloud, perhaps using data and artifacts which are sensitive in nature.


In [40]:
from kubeflow.fairing.builders import cluster
from kubeflow.fairing import constants

constants.constants.KANIKO_IMAGE = "gcr.io/kaniko-project/executor:v0.22.0"

cluster_builder = cluster.cluster.ClusterBuilder(registry=DOCKER_REGISTRY,
                                                 preprocessor=preprocessor,
                                                 dockerfile_path="Dockerfile",
                                                 context_source=minio_context_source)

print(cluster_builder.build())


[I 200626 13:12:41 cluster:46] Building image using cluster builder.
[W 200626 13:12:41 base:94] Dockerfile already exists in Fairing context, skipping...
[I 200626 13:12:41 base:107] Creating docker context: /tmp/fairing_context_xiphsi3n
[W 200626 13:12:41 base:94] Dockerfile already exists in Fairing context, skipping...
[W 200626 13:12:57 manager:296] Waiting for fairing-builder-2v5vn-c74m6 to start...
[W 200626 13:12:57 manager:296] Waiting for fairing-builder-2v5vn-c74m6 to start...
[W 200626 13:12:58 manager:296] Waiting for fairing-builder-2v5vn-c74m6 to start...
[I 200626 13:13:03 manager:302] Pod started running True
INFO[0005] Retrieving image manifest bentoml/model-server:0.8.1
INFO[0007] Retrieving image manifest bentoml/model-server:0.8.1
INFO[0008] Built cross stage deps: map[]
INFO[0008] Retrieving image manifest bentoml/model-server:0.8.1
INFO[0009] Retrieving image manifest bentoml/model-server:0.8.1
INFO[0010] Executing 0 build triggers
INFO[0010] Unpacking rootfs as cmd COPY . /bento requires it.
INFO[0109] COPY . /bento
INFO[0109] Resolving 17 paths
INFO[0109] Taking snapshot of files...
INFO[0110] WORKDIR /bento
INFO[0110] cmd: workdir
INFO[0110] Changed working directory to /bento
INFO[0110] ARG PIP_INDEX_URL=https://pypi.python.org/simple/
INFO[0110] ARG PIP_TRUSTED_HOST=pypi.python.org
INFO[0110] ENV PIP_INDEX_URL $PIP_INDEX_URL
INFO[0110] ENV PIP_TRUSTED_HOST $PIP_TRUSTED_HOST
INFO[0110] RUN if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi
INFO[0110] Taking snapshot of full filesystem...
INFO[0111] Resolving 30305 paths
INFO[0128] cmd: /bin/sh
INFO[0128] args: [-c if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi]
INFO[0128] Running: [/bin/sh -c if [ -f /bento/bentoml-init.sh ]; then bash -c /bento/bentoml-init.sh; fi]
+++ dirname /bento/bentoml-init.sh
++ cd /bento
++ pwd -P
+ SAVED_BUNDLE_PATH=/bento
+ cd /bento
+ '[' -f ./setup.sh ']'
+ command -v conda
+ conda env update -n base -f ./environment.yml
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

Downloading and Extracting Packages
pip-20.1.1           | 1.7 MB    | ########## | 100% 
certifi-2020.6.20    | 156 KB    | ########## | 100% 
openssl-1.1.1g       | 2.5 MB    | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done


==> WARNING: A newer version of conda exists. <==
  current version: 4.8.2
  latest version: 4.8.3

Please update conda by running

    $ conda update -n base -c defaults conda
#
# To activate this environment, use
#
#     $ conda activate base
#
# To deactivate an active environment, use
#
#     $ conda deactivate



+ pip install -r ./requirements.txt
Looking in indexes: https://pypi.python.org/simple/
Collecting pandas
  Downloading pandas-1.0.5-cp37-cp37m-manylinux1_x86_64.whl (10.1 MB)
Requirement already satisfied: bentoml==0.8.1 in /opt/conda/lib/python3.7/site-packages (from -r ./requirements.txt (line 2)) (0.8.1)
Collecting fbprophet
  Downloading fbprophet-0.6.tar.gz (54 kB)
Collecting pytz>=2017.2
  Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)
Requirement already satisfied: numpy>=1.13.3 in /opt/conda/lib/python3.7/site-packages (from pandas->-r ./requirements.txt (line 1)) (1.18.5)
Requirement already satisfied: python-dateutil>=2.6.1 in /opt/conda/lib/python3.7/site-packages (from pandas->-r ./requirements.txt (line 1)) (2.8.0)
Requirement already satisfied: grpcio<=1.27.2 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.27.2)
Requirement already satisfied: protobuf>=3.6.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (3.12.2)
Requirement already satisfied: py-zipkin in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.20.0)
Requirement already satisfied: tabulate in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.8.7)
Requirement already satisfied: cerberus in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.3.2)
Requirement already satisfied: configparser in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (5.0.0)
Requirement already satisfied: boto3 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.14.3)
Requirement already satisfied: psutil in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (5.7.0)
Requirement already satisfied: sqlalchemy-utils in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.36.6)
Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.8.0)
Requirement already satisfied: certifi in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (2020.6.20)
Requirement already satisfied: docker in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (4.2.1)
Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (2.22.0)
Requirement already satisfied: aiohttp in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (3.6.2)
Requirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.16.10)
Requirement already satisfied: flask in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.1.2)
Requirement already satisfied: python-json-logger in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.1.11)
Requirement already satisfied: packaging in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (20.4)
Requirement already satisfied: alembic in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.4.2)
Requirement already satisfied: sqlalchemy>=1.3.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.3.17)
Requirement already satisfied: humanfriendly in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (8.2)
Requirement already satisfied: gunicorn in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (20.0.4)
Requirement already satisfied: click>=7.0 in /opt/conda/lib/python3.7/site-packages (from bentoml==0.8.1->-r ./requirements.txt (line 2)) (7.1.2)
Collecting Cython>=0.22
  Downloading Cython-0.29.20-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB)
Collecting cmdstanpy==0.4
  Downloading cmdstanpy-0.4.0-py3-none-any.whl (22 kB)
Collecting pystan>=2.14
  Downloading pystan-2.19.1.1-cp37-cp37m-manylinux1_x86_64.whl (67.3 MB)
Collecting matplotlib>=2.0.0
  Downloading matplotlib-3.2.2-cp37-cp37m-manylinux1_x86_64.whl (12.4 MB)
Collecting LunarCalendar>=0.0.9
  Downloading LunarCalendar-0.0.9-py2.py3-none-any.whl (18 kB)
Collecting convertdate>=2.1.2
  Downloading convertdate-2.2.1-py2.py3-none-any.whl (43 kB)
Collecting holidays>=0.9.5
  Downloading holidays-0.10.2.tar.gz (110 kB)
Collecting setuptools-git>=1.2
  Downloading setuptools_git-1.2-py2.py3-none-any.whl (10 kB)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas->-r ./requirements.txt (line 1)) (1.14.0)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.0->bentoml==0.8.1->-r ./requirements.txt (line 2)) (45.2.0.post20200210)
Requirement already satisfied: thriftpy2>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from py-zipkin->bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.4.11)
Requirement already satisfied: botocore<1.18.0,>=1.17.3 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.17.3)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.3.3)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /opt/conda/lib/python3.7/site-packages (from boto3->bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.10.0)
Requirement already satisfied: websocket-client>=0.32.0 in /opt/conda/lib/python3.7/site-packages (from docker->bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.57.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.8.1->-r ./requirements.txt (line 2)) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.25.8)
Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->bentoml==0.8.1->-r ./requirements.txt (line 2)) (2.8)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.1->-r ./requirements.txt (line 2)) (19.3.0)
Requirement already satisfied: multidict<5.0,>=4.5 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.1->-r ./requirements.txt (line 2)) (4.7.6)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.4.2)
Requirement already satisfied: async-timeout<4.0,>=3.0 in /opt/conda/lib/python3.7/site-packages (from aiohttp->bentoml==0.8.1->-r ./requirements.txt (line 2)) (3.0.1)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.9" in /opt/conda/lib/python3.7/site-packages (from ruamel.yaml>=0.15.0->bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.2.0)
Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.0.1)
Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.1->-r ./requirements.txt (line 2)) (2.11.2)
Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/lib/python3.7/site-packages (from flask->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.1.0)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.7/site-packages (from packaging->bentoml==0.8.1->-r ./requirements.txt (line 2)) (2.4.7)
Requirement already satisfied: python-editor>=0.3 in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.0.4)
Requirement already satisfied: Mako in /opt/conda/lib/python3.7/site-packages (from alembic->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.1.3)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.2.0-cp37-cp37m-manylinux1_x86_64.whl (88 kB)
Collecting cycler>=0.10
  Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting ephem>=3.7.5.3
  Downloading ephem-3.7.7.1-cp37-cp37m-manylinux2010_x86_64.whl (1.2 MB)
Collecting pymeeus<=1,>=0.3.6
  Downloading PyMeeus-0.3.7.tar.gz (732 kB)
Collecting korean_lunar_calendar
  Downloading korean_lunar_calendar-0.2.1-py3-none-any.whl (8.0 kB)
Requirement already satisfied: ply<4.0,>=3.4 in /opt/conda/lib/python3.7/site-packages (from thriftpy2>=0.4.0->py-zipkin->bentoml==0.8.1->-r ./requirements.txt (line 2)) (3.11)
Requirement already satisfied: docutils<0.16,>=0.10 in /opt/conda/lib/python3.7/site-packages (from botocore<1.18.0,>=1.17.3->boto3->bentoml==0.8.1->-r ./requirements.txt (line 2)) (0.15.2)
Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask->bentoml==0.8.1->-r ./requirements.txt (line 2)) (1.1.1)
Building wheels for collected packages: fbprophet, holidays, pymeeus
  Building wheel for fbprophet (setup.py): started
  Building wheel for fbprophet (setup.py): finished with status 'error'
  Running setup.py clean for fbprophet
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-zh6gfvkx/fbprophet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-zh6gfvkx/fbprophet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-qwa0toa2
       cwd: /tmp/pip-install-zh6gfvkx/fbprophet/
  Complete output (40 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/fbprophet
  creating build/lib/fbprophet/stan_model
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-zh6gfvkx/fbprophet/setup.py", line 148, in <module>
      """
    File "/opt/conda/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
      return distutils.core.setup(**attrs)
    File "/opt/conda/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/opt/conda/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 223, in run
      self.run_command('build')
    File "/opt/conda/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/opt/conda/lib/python3.7/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/opt/conda/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-zh6gfvkx/fbprophet/setup.py", line 48, in run
      build_models(target_dir)
    File "/tmp/pip-install-zh6gfvkx/fbprophet/setup.py", line 36, in build_models
      from fbprophet.models import StanBackendEnum
    File "/tmp/pip-install-zh6gfvkx/fbprophet/fbprophet/__init__.py", line 8, in <module>
      from fbprophet.forecaster import Prophet
    File "/tmp/pip-install-zh6gfvkx/fbprophet/fbprophet/forecaster.py", line 15, in <module>
      import pandas as pd
  ModuleNotFoundError: No module named 'pandas'
  ----------------------------------------
  ERROR: Failed building wheel for fbprophet
  Building wheel for holidays (setup.py): started
  Building wheel for holidays (setup.py): finished with status 'done'
  Created wheel for holidays: filename=holidays-0.10.2-py3-none-any.whl size=111560 sha256=e42528ffa1b9947182d30cec4b982d0e0572c753b64acb0c200b1855399f89fd
  Stored in directory: /root/.cache/pip/wheels/90/4e/82/f4130a57eb035c4344489ca14caff692590719b5f375540f53
  Building wheel for pymeeus (setup.py): started
  Building wheel for pymeeus (setup.py): finished with status 'done'
  Created wheel for pymeeus: filename=PyMeeus-0.3.7-py3-none-any.whl size=702876 sha256=441c449f932be4813828e725a7a9db2d211bc003bf0634cc25d516f9a715c30a
  Stored in directory: /root/.cache/pip/wheels/80/32/5f/2a67880d4ce584b9cf99146f9945e46942dfb010a9382c6ff5
Successfully built holidays pymeeus
Failed to build fbprophet
Installing collected packages: pytz, pandas, Cython, cmdstanpy, pystan, kiwisolver, cycler, matplotlib, ephem, LunarCalendar, pymeeus, convertdate, korean-lunar-calendar, holidays, setuptools-git, fbprophet
    Running setup.py install for fbprophet: started
    Running setup.py install for fbprophet: still running...
    Running setup.py install for fbprophet: finished with status 'done'
Successfully installed Cython-0.29.20 LunarCalendar-0.0.9 cmdstanpy-0.4.0 convertdate-2.2.1 cycler-0.10.0 ephem-3.7.7.1 fbprophet-0.6 holidays-0.10.2 kiwisolver-1.2.0 korean-lunar-calendar-0.2.1 matplotlib-3.2.2 pandas-1.0.5 pymeeus-0.3.7 pystan-2.19.1.1 pytz-2020.1 setuptools-git-1.2
+ for filename in ./bundled_pip_dependencies/*.tar.gz
+ '[' -e './bundled_pip_dependencies/*.tar.gz' ']'
+ continue
INFO[0421] Taking snapshot of full filesystem...
INFO[0422] Resolving 57440 paths
INFO[0521] ENV PORT 5000
INFO[0521] EXPOSE $PORT
INFO[0521] cmd: EXPOSE
INFO[0521] Adding exposed port: 5000/tcp
INFO[0521] COPY docker-entrypoint.sh /usr/local/bin/
INFO[0521] Resolving 1 paths
INFO[0521] Taking snapshot of files...
INFO[0521] ENTRYPOINT [ "docker-entrypoint.sh" ]
INFO[0521] CMD ["bentoml", "serve-gunicorn", "/bento"]
None

Deploy the Service

There are several good ways we can deploy the model we have fit, trained and crafted into an image. For this example, we will deploy the model service with KFServing by creating an InferenceService. This project uses Knative Serving to deploy the service and setup its routes. Under the covers Istio is at work routing our traffic to this new service.

For local development, set the cluster local tags and set the domain to reflect svc.cluster.local as described in these docs.

Lets use the Custom Object K8S API to deploy the pickled Prophet model we have defined with BentoML and built with Fairing.

KFServing InferenceService

Below, we define and launch an inferenceservice, which in turn reconciles into a Knative service.


In [60]:
from kfserving import V1alpha2EndpointSpec,V1alpha2InferenceServiceSpec, V1alpha2InferenceService, V1alpha2CustomSpec
from kfserving import KFServingClient
from kfserving import constants

containerSpec = k8s_client.V1Container(
    name="prophet-model-api-container",
    image=cluster_builder.image_tag,
    ports=[k8s_client.V1ContainerPort(container_port=5000)])

default_custom_model_spec = V1alpha2EndpointSpec(predictor=V1alpha2PredictorSpec(custom=V1alpha2CustomSpec(container=containerSpec)))

metadata = k8s_client.V1ObjectMeta(
    name="prophet-model-api", namespace="default",
)

isvc = V1alpha2InferenceService(api_version=constants.KFSERVING_GROUP + '/' + constants.KFSERVING_VERSION,
                          kind=constants.KFSERVING_KIND,
                          metadata=metadata,
                          spec=V1alpha2InferenceServiceSpec(default=default_custom_model_spec))

KFServing = KFServingClient()
KFServing.create(isvc)


Out[60]:
{'apiVersion': 'serving.kubeflow.org/v1alpha2',
 'kind': 'InferenceService',
 'metadata': {'creationTimestamp': '2020-06-26T14:08:34Z',
  'generation': 1,
  'name': 'prophet-model-api',
  'namespace': 'default',
  'resourceVersion': '2962980',
  'selfLink': '/apis/serving.kubeflow.org/v1alpha2/namespaces/default/inferenceservices/prophet-model-api',
  'uid': '58f289ac-a802-44d1-ab0f-928dcc283ad5'},
 'spec': {'default': {'predictor': {'custom': {'container': {'image': 'iancoffey/fairing-job:485938BA',
      'name': 'prophet-model-api-container',
      'ports': [{'containerPort': 5000}],
      'resources': {'limits': {'cpu': '1', 'memory': '2Gi'},
       'requests': {'cpu': '1', 'memory': '2Gi'}}}}}}},
 'status': {}}

Querying the Inference Server

Now we are ready to use our endpoint. Lets post a test date with curl and see our shiny brand new API endpoint perform.


In [61]:
!curl -i --header "Content-Type: application/json" -X POST http://prophet-model-api-predictor-default-7z66r-private.default.svc.cluster.local/predict --data '{"ds":["2020-07-14"]}'










[{"ds":1594684800000,"trend":506162.6572053498,"yhat_lower":301556.5952542502,"yhat_upper":688911.3268348163,"trend_lower":301342.1933916567,"trend_upper":691285.190480784,"additive_terms":-645.3873742295,"additive_terms_lower":-645.3873742295,"additive_terms_upper":-645.3873742295,"weekly":-645.3873742295,"weekly_lower":-645.3873742295,"weekly_upper":-645.3873742295,"multiplicative_terms":0.0,"multiplicative_terms_lower":0.0,"multiplicative_terms_upper":0.0,"yhat":505517.2698311203}]

Model Deployed

Now we have a fitted and trained a model, defined a Service with BentoML, built an image for it using Kubeflow Fairing and deployed it to our Kubernetes dev cluster with KFServing. Party!