Title: Hyperparameter Tuning Using Random Search
Slug: hyperparameter_tuning_using_random_search
Summary: How to conduct random search for hyperparameter tuning in scikit-learn for machine learning in Python.
Date: 2017-09-18 12:00
Category: Machine Learning
Tags: Model Selection
Authors: Chris Albon
In [9]:
# Load libraries
from scipy.stats import uniform
from sklearn import linear_model, datasets
from sklearn.model_selection import RandomizedSearchCV
In [10]:
# Load data
iris = datasets.load_iris()
X = iris.data
y = iris.target
In [11]:
# Create logistic regression
logistic = linear_model.LogisticRegression()
In [12]:
# Create regularization penalty space
penalty = ['l1', 'l2']
# Create regularization hyperparameter distribution using uniform distribution
C = uniform(loc=0, scale=4)
# Create hyperparameter options
hyperparameters = dict(C=C, penalty=penalty)
In [13]:
# Create randomized search 5-fold cross validation and 100 iterations
clf = RandomizedSearchCV(logistic, hyperparameters, random_state=1, n_iter=100, cv=5, verbose=0, n_jobs=-1)
In [14]:
# Fit randomized search
best_model = clf.fit(X, y)
In [15]:
# View best hyperparameters
print('Best Penalty:', best_model.best_estimator_.get_params()['penalty'])
print('Best C:', best_model.best_estimator_.get_params()['C'])
In [16]:
# Predict target vector
best_model.predict(X)
Out[16]: