Question_3-1-3_Multiclass_Ridge

Janet Matsen

Code notes:

Indivudal regressions are done by instinces of RidgeRegression, defined in rige_regression.py.
- RidgeRegression gets some methods from ClassificationBase, defined in classification_base.py.
The class HyperparameterSweep in hyperparameter_sweep_base is used to tune hyperparameters on training data.



In [ ]:

    
import numpy as np
import matplotlib as mpl
%matplotlib inline
import pandas as pd
import seaborn as sns

from mnist import MNIST  # public package for making arrays out of MINST data.



In [ ]:

    
import sys
sys.path.append('../code/')



In [ ]:

    
from ridge_regression import RidgeBinary
from hyperparameter_explorer import HyperparameterExplorer



In [ ]:

    
from mnist_helpers import mnist_training, mnist_testing, mnist_training_binary



In [ ]:

    
import matplotlib.pyplot as plt
from pylab import rcParams
rcParams['figure.figsize'] = 4, 3

Prepare MNIST training data



In [ ]:

    
train_X, train_y = mnist_training_binary(2)



In [ ]:

    
print(train_X.shape, train_y.shape)

Dev: make sure a single model runs fine

tmp1 = RidgeBinary(X=train_X[1:100,], y=train_y[1:100,], lam=10) tmp1.run()

tmp1.results

tmp2 = RidgeBinary(X=train_X[1:100,], y=train_y[1:100,], lam=1) tmp2.run()

tmp2.results

Explore hyperparameters before training model on all of the training data.



In [ ]:

    
hyper_explorer = HyperparameterExplorer(X=train_X, y=train_y, model=RidgeBinary, 
                                        validation_split=0.10, score_name='RMSE')



In [ ]:

    
hyper_explorer.train_X.shape



In [ ]:

    
hyper_explorer.train_model(lam=100)



In [ ]:

    
hyper_explorer.train_model(lam=10)



In [ ]:

    
hyper_explorer.train_model(lam=.001)



In [ ]:

    
hyper_explorer.train_model(lam=1e-5)



In [ ]:

    
hyper_explorer.summary



In [ ]:

    
fig, ax = plt.subplots(1, 1, figsize=(4, 3))
plt.semilogx(hyper_explorer.summary['lambda'], hyper_explorer.summary['validation RMSE'], 
            linestyle='--', marker='o', c='g')
plt.semilogx(hyper_explorer.summary['lambda'], hyper_explorer.summary['RMSE'], 
            linestyle='--', marker='o', c='grey')
plt.legend(loc='best')
plt.xlabel('lambda')
plt.ylabel('RMSE')
ax.axhline(y=0, color='k')



In [ ]: