LAB 3a: BigQuery ML Model Baseline.

Learning Objectives

Create baseline model with BQML
Evaluate baseline model
Calculate RMSE of baseline model

Introduction

In this notebook, we will create a baseline model to predict the weight of a baby before it is born. We will use BigQuery ML to build a linear babyweight prediction model with the base features and no feature engineering, yet.

We will create a baseline model with BQML, evaluate our baseline model, and calculate the its RMSE.

Each learning objective will correspond to a #TODO in this student lab notebook -- try to complete this notebook first and then review the solution notebook.

Load necessary libraries

Check that the Google BigQuery library is installed and if not, install it.



In [ ]:

    
%%bash
sudo pip freeze | grep google-cloud-bigquery==1.6.1 || \
sudo pip install google-cloud-bigquery==1.6.1

Verify tables exist

Run the following cells to verify that we previously created the dataset and data tables. If not, go back to lab 1b_prepare_data_babyweight to create them.



In [ ]:

    
%%bigquery
-- LIMIT 0 is a free query; this allows us to check that the table exists.
SELECT * FROM babyweight.babyweight_data_train
LIMIT 0



In [ ]:

    
%%bigquery
-- LIMIT 0 is a free query; this allows us to check that the table exists.
SELECT * FROM babyweight.babyweight_data_eval
LIMIT 0

Create the baseline model

Next, we'll create a linear regression baseline model with no feature engineering. We'll use this to compare our later, more complex models against.

Lab Task #1: Train the "Baseline Model".

When creating a BQML model, you must specify the model type (in our case linear regression) and the input label (weight_pounds). Note also that we are using the training data table as the data source and we don't need BQML to split the data because we have already split it ourselves.



In [ ]:

    
%%bigquery
CREATE OR REPLACE MODEL
    babyweight.baseline_model

OPTIONS (
    MODEL_TYPE=# TODO: Add model type,
    INPUT_LABEL_COLS=[# TODO: label column name],
    DATA_SPLIT_METHOD="NO_SPLIT") AS

SELECT
    # TODO: Add features and label
FROM
    # TODO: Add train table

REMINDER: The query takes several minutes to complete. After the first iteration is complete, your model (baseline_model) appears in the navigation panel of the BigQuery web UI. Because the query uses a CREATE MODEL statement to create a model, you do not see query results.

You can observe the model as it's being trained by viewing the Model stats tab in the BigQuery web UI. As soon as the first iteration completes, the tab is updated. The stats continue to update as each iteration completes.

Once the training is done, visit the BigQuery Cloud Console and look at the model that has been trained. Then, come back to this notebook.

Evaluate the baseline model

Even though BigQuery can automatically split the data it is given, and training on only a part of the data and using the rest for evaluation, to compare with our custom models later we wanted to decide the split ourselves so that it is completely reproducible.

NOTE: The results are also displayed in the BigQuery Cloud Console under the Evaluation tab.



In [ ]:

    
%%bigquery
-- Information from model training
SELECT * FROM ML.TRAINING_INFO(MODEL babyweight.baseline_model)

Lab Task #2: Get evaluation statistics for the baseline model.

After creating your model, you evaluate the performance of the regressor using the ML.EVALUATE function. The ML.EVALUATE function evaluates the predicted values against the actual data.



In [ ]:

    
%%bigquery
SELECT
    *
FROM
    ML.EVALUATE(MODEL # TODO: Add model name,
    (
    SELECT
        # TODO: Add features and label
    FROM
        # TODO: Add eval table
    ))

Resource for an explanation of the Regression Metrics.

Lab Task #3: Write a SQL query to find the RMSE of the evaluation data

Since this is regression, we typically use the RMSE, but natively this is not in the output of our evaluation metrics above. However, we can simply take the SQRT() of the mean squared error of our loss metric from evaluation of the baseline_model to get RMSE.



In [ ]:

    
%%bigquery
SELECT
    # TODO: Select just the calculated RMSE
FROM
    ML.EVALUATE(MODEL # TODO: Add model name,
    (
    SELECT
        # TODO: Add features and label
    FROM
        # TODO: Add eval table
    ))

Lab Summary:

In this lab, we created a baseline model with BQML, evaluated our baseline model, and calculated the RMSE of our baseline model.

Copyright 2019 Google Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License



In [ ]: