Title: Hyperparameter Tuning Using Grid Search
Slug: hyperparameter_tuning_using_grid_search
Summary: How to conduct grid search for hyperparameter tuning in scikit-learn for machine learning in Python. Date: 2017-09-18 12:00
Category: Machine Learning
Tags: Model Selection Authors: Chris Albon

Preliminaries


In [1]:
# Load libraries
import numpy as np
from sklearn import linear_model, datasets
from sklearn.model_selection import GridSearchCV

Load Iris Dataset


In [2]:
# Load data
iris = datasets.load_iris()
X = iris.data
y = iris.target

Create Logistic Regression


In [3]:
# Create logistic regression
logistic = linear_model.LogisticRegression()

Create Hyperparameter Search Space


In [4]:
# Create regularization penalty space
penalty = ['l1', 'l2']

# Create regularization hyperparameter space
C = np.logspace(0, 4, 10)

# Create hyperparameter options
hyperparameters = dict(C=C, penalty=penalty)

In [5]:
# Create grid search using 5-fold cross validation
clf = GridSearchCV(logistic, hyperparameters, cv=5, verbose=0)

In [6]:
# Fit grid search
best_model = clf.fit(X, y)

View Hyperparameter Values Of Best Model


In [7]:
# View best hyperparameters
print('Best Penalty:', best_model.best_estimator_.get_params()['penalty'])
print('Best C:', best_model.best_estimator_.get_params()['C'])


Best Penalty: l1
Best C: 7.74263682681

Predict Using Best Model


In [8]:
# Predict target vector
best_model.predict(X)


Out[8]:
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1,
       1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2,
       2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])