Entropy Search

Written by Simon Bartels, Max Planck Institute for Intelligent Systems, and Andrei Paleyes, Amazon.com

This notebook demonstrates how to use Entropy Search (ES) in GPyOpt and compares it to Expected Improvement (EI). For details on ES have a look at the original paper:

Hennig and C. J. Schuler. Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13, 2012

Imports



In [ ]:

    
import numpy as np
import GPy
import GPyOpt
from GPyOpt.models.gpmodel import GPModel
from GPyOpt.core.task.space import Design_space, bounds_to_space
from GPyOpt.util.mcmc_sampler import AffineInvariantEnsembleSampler
from GPyOpt.acquisitions.ES import AcquisitionEntropySearch
from GPyOpt.acquisitions.EI import AcquisitionEI

import matplotlib as mpl
mpl.use('Agg')

#configure plotting
%matplotlib inline
%config InlineBackend.figure_format = 'svg'

import matplotlib;matplotlib.rcParams['figure.figsize'] = (8,5)
import matplotlib;matplotlib.rcParams['text.usetex'] = True
import matplotlib;matplotlib.rcParams['font.size'] = 16
import matplotlib;matplotlib.rcParams['font.family'] = 'serif'
from matplotlib import pyplot as plt

Toy Problem

The following toy problem demonstrates the possible advantage Entropy Search can have over Expected Improvement. The observations are chosen in a way such that EI will evaluate at the minimum whose location is pretty clear. Entropy Search on the other hand exhibits a more explorative behavior.



In [ ]:

    
X = np.array([[-1], [1], [2]])
y = 2 * -np.array([[.1], [.5], [.5]])
bounds = [(-5, 5)]
input_dim = X.shape[1]

kern = GPy.kern.RBF(input_dim, variance=1., lengthscale=1.)
model = GPModel(kern, noise_var=1e-3, max_iters=0, optimize_restarts=0)

model.updateModel(X, y, None, None)

Plot of Data-Set, Model and Acquisition Functions



In [ ]:

    
Xs = np.arange(bounds[0][0], bounds[0][1], 0.01).reshape(-1, 1)
ys, vs = model.predict(Xs)

plt.fill_between(np.ndarray.flatten(Xs), 
                 np.ndarray.flatten(ys+np.sqrt(vs)), 
                 np.ndarray.flatten(ys-np.sqrt(vs)), alpha=0.1)
plt.plot(Xs, ys, color='b')
plt.plot(X, y, 'x')

space = Design_space(bounds_to_space(bounds))
def normalize(vs):
    return (vs - min(vs))/(max(vs - min(vs)))
sampler = AffineInvariantEnsembleSampler(space)

ei = AcquisitionEI(model, space)
vei = normalize(ei.acquisition_function(Xs))

es = AcquisitionEntropySearch(model, space, sampler)
ves = normalize(es.acquisition_function(Xs))

# plot Expected Improvement again
plt.plot(Xs, ves, color='r')
# plot Entropy Search values
plt.plot(Xs, vei, color='g')
plt.show()

Expected Improvement (green line) suggests to evaluate in the location of the minimum (around 1.9). In contrast, Entropy Search (red line) is more explorative, preferring points near 4 and -4. Evaluating the minimum location would not bring much insight.

Comparison on the Branin function



In [ ]:

    
# --- Function to optimize
func  = GPyOpt.objective_examples.experiments2d.branin()
func.plot()

Let's define the necessary objects for ModularBayesianOptimization.



In [ ]:

    
objective = GPyOpt.core.task.SingleObjective(func.f)
space = GPyOpt.Design_space(space =[{'name': 'var_1', 'type': 'continuous', 'domain': (-5,10)},
                                    {'name': 'var_2', 'type': 'continuous', 'domain': (1,15)}])
acquisition_optimizer = GPyOpt.optimization.AcquisitionOptimizer(space)
initial_design = GPyOpt.experiment_design.initial_design('random', space, 5)
max_iter = 10

First run Expected Improvement.



In [ ]:

    
ei_model = GPyOpt.models.GPModel(optimize_restarts=5,verbose=False)
ei = AcquisitionEI(ei_model, space, optimizer=acquisition_optimizer)
ei_evaluator = GPyOpt.core.evaluators.Sequential(ei)
bo_ei = GPyOpt.methods.ModularBayesianOptimization(ei_model, space, objective, ei, ei_evaluator, initial_design)
bo_ei.run_optimization(max_iter = max_iter)
bo_ei.plot_acquisition()
bo_ei.plot_convergence()

And now run Entropy Search.



In [ ]:

    
es_model = GPyOpt.models.GPModel(optimize_restarts=5,verbose=False)
ei = AcquisitionEI(es_model, space, optimizer=acquisition_optimizer)
proposal_function = lambda x : np.clip(np.log(ei._compute_acq(x)), 0., np.PINF)
sampler = AffineInvariantEnsembleSampler(space)
es = AcquisitionEntropySearch(es_model, space, sampler, optimizer=acquisition_optimizer, num_representer_points=10, 
                   burn_in_steps=10, num_samples=100, proposal_function = proposal_function)
es_evaluator = GPyOpt.core.evaluators.Sequential(es)
bo_es = GPyOpt.methods.ModularBayesianOptimization(es_model, space, objective, es, es_evaluator, initial_design)
bo_es.run_optimization(max_iter = max_iter)
bo_es.plot_acquisition()
bo_es.plot_convergence()

Let's plot the locations where Entropy Search (circles) and Expected Improvement (crosses) evaluated.



In [ ]:

    
bounds = func.bounds
x1 = np.linspace(bounds[0][0], bounds[0][1], 100)
x2 = np.linspace(bounds[1][0], bounds[1][1], 100)
X1, X2 = np.meshgrid(x1, x2)
X = np.hstack((X1.reshape(100*100,1),X2.reshape(100*100,1)))
Y = func.f(X)

plt.figure()    
plt.contourf(X1, X2, Y.reshape((100,100)),100)
plt.plot(np.array(func.min)[:,0], np.array(func.min)[:,1], 'w.', markersize=20, label=u'Observations')
plt.colorbar()
plt.plot(ei_model.model.X[:, 0],ei_model.model.X[:, 1], 'o')
plt.plot(es_model.model.X[:, 0],es_model.model.X[:, 1], 'x')
plt.xlabel('X1')
plt.ylabel('X2')
plt.title(func.name)
plt.show()



In [ ]: