Batch Serving Design Pattern

This notebook demonstrates the Batch Serving design pattern using BigQuery

Simple text classification model

Let's use the same model that was used in serving_function.ipynb -- a text classification model based on IMDB reviews trained, and exported.


In [3]:
!find export/probs/


export/probs/
export/probs/sentiment_20200505_232617
export/probs/sentiment_20200505_232617/variables
export/probs/sentiment_20200505_232617/variables/variables.data-00000-of-00001
export/probs/sentiment_20200505_232617/variables/variables.index
export/probs/sentiment_20200505_232617/assets
export/probs/sentiment_20200505_232617/assets/tokens.txt
export/probs/sentiment_20200505_232617/saved_model.pb

In [8]:
%%bash
LOCAL_DIR=$(find export/probs | head -2 | tail -1)
BUCKET=ai-analytics-solutions-kfpdemo
gsutil rm -rf gs://${BUCKET}/mlpatterns/batchserving
gsutil cp -r $LOCAL_DIR gs://${BUCKET}/mlpatterns/batchserving
gsutil ls gs://${BUCKET}/mlpatterns/batchserving


gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/saved_model.pb
gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/assets/
gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/variables/
Removing gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/assets/tokens.txt#1588793354410693...
Removing gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/saved_model.pb#1588793353859489...
Removing gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/variables/variables.data-00000-of-00001#1588793354212839...
Removing gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/variables/variables.index#1588793354293034...
/ [4 objects]                                                                   
Operation completed over 4 objects.                                              
Copying file://export/probs/sentiment_20200505_232617/saved_model.pb [Content-Type=application/octet-stream]...
Copying file://export/probs/sentiment_20200505_232617/variables/variables.data-00000-of-00001 [Content-Type=application/octet-stream]...
Copying file://export/probs/sentiment_20200505_232617/variables/variables.index [Content-Type=application/octet-stream]...
Copying file://export/probs/sentiment_20200505_232617/assets/tokens.txt [Content-Type=text/plain]...
/ [4 files][  4.7 MiB/  4.7 MiB]                                                
Operation completed over 4 objects/4.7 MiB.                                      

Load model into BigQuery for batch serving


In [12]:
%%bigquery
CREATE OR REPLACE MODEL mlpatterns.imdb_sentiment
OPTIONS(model_type='tensorflow', model_path='gs://ai-analytics-solutions-kfpdemo/mlpatterns/batchserving/*')


Out[12]:

In [13]:
%%bigquery
SELECT * FROM ML.PREDICT(MODEL mlpatterns.imdb_sentiment,
  (SELECT 'This was very well done.' AS reviews)
)


Out[13]:
positive_review_logits positive_review_probability reviews
0 1.511887 0.819341 This was very well done.

Now, do it at scale, on consumer complaints about financial products and services:


In [42]:
%%bigquery preds
SELECT * FROM ML.PREDICT(MODEL mlpatterns.imdb_sentiment,
  (SELECT consumer_complaint_narrative AS reviews 
   FROM `bigquery-public-data`.cfpb_complaints.complaint_database
   WHERE consumer_complaint_narrative IS NOT NULL
   )
)

In [44]:
preds[:3]


Out[44]:
positive_review_logits positive_review_probability reviews
0 -3.725246 0.023540 ADP TotalPay card with money network upgraded ...
1 -0.889956 0.291119 I paid off my Pay Pal Credit in full except fo...
2 -6.391967 0.001672 Well, I called citi on XXXX/XXXX/16 to determi...

In [48]:
# what's does a "positive" complaint look like?
preds.sort_values(by='positive_review_probability', ascending=False).iloc[1]['reviews']


Out[48]:
'I get phone calls morning XXXX  and night. I have told them to stop so many calls but they still call even on Sunday in the morning. I had two calls in a row on a Sunday morning from XXXX XXXX. I received 9 calls on Saturday. I receive about 9 during the week day every day as well.'

In [49]:
# what's does a "typical" complaint look like?
preds.sort_values(by='positive_review_probability', ascending=False).iloc[len(preds)//2]['reviews']


Out[49]:
'I have noticed inconsistencies with the information all three credit bureaus have. As you know, this is not allowed under my rights and also a clear indication that this information was NOT properly verified upon you receiving it. XXXX has a Chapter XXXX Bankruptcy, REF # XXXX as Date Filed : XX/XX/XXXX, however XXXX   has the following info claiming to be accurate, Chapter XXXX Bankruptcy, REF # XXXX as Date Opened : XX/XX/XXXX, and Equifax is NOT reporting this information AT ALL!!! Also XXXX has XXXX XXXX, XXXX # XXXX Balance of {$3500.00} as Date Opened : XX/XX/XXXX, however XXXX has the following info claiming to be accurate, Balance of {$3500.00} as Date Opened : XX/XX/XXXX, and Equifax has the following info claiming to be accurate, Balance of {$3500.00} as Date Opened : XX/XX/XXXX. If these are the same accounts, how is the supposed verified information listed on my credit profiles different from one another? This is not possible and because of this inconsistency, please delete the following account from my credit profile immediately. I also asked if they would update my personal information in their system, they have not done so.'

Copyright 2020 Google Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License