Machine Learning Summer School, Sydney

23rd February 2015

Neil D. Lawrence

Introduction

Welcome to the lab for the Gaussian process section at the Machine Learning Summer School in Sydney.

This notebook provides you with the guide to your lab classes for Gaussian processes. The lab classes are intended to help get you familiar with modeling with Gaussian processes as

The lab classes are based on our two software packages, pods which is used for access to datasets and GPy (release 21st November 2014) for Gaussian processes. You can install the GPy framework with

pip install GPy

As well as the GPy software we use our pods software for 'open data science' for access to data sets and other resources.

pip install -pre pods

on some systems you may need to use pip install -pre pods to allow the prerelease to install.

As well as these lab classes here are a range of tutorials on how to use GPy, many of which are written by members of the Sheffield research group. GPy is under active development and is released under a BSD license, you'd also be very welcome to contribute!

Review

Before you start, if you aren't familiar with probabilistic processes, the following lab classes from the GPRS schools might be useful. The first session will allow you to become familiar with the Jupyter (the ipython notebook) and start to work with Gaussian processes.

• Welcome to Jupyter A quick introduction to Jupyter, python and numpy.
• Introduction to Probabilistic Regression A review of least squares, basis function modelling and the probabilistic interpretation of least squares.
• Introduction to Bayesian Regression Introducing priors over parameters and averaging over solutions.

Gaussian Processes

The session will focus on Gaussian process models and developing covariance functions.

• Introduction to Gaussian Processes We move from the Bayesian regression with polynomials to Gaussian process perspectives by looking at the priors over the function directly.
• GPy: Introduction through Covariance Functions GPy is a Python Gaussian process framework that implements many of the ideas we'll see in the course. In this session we introduce its covariance functions and sample from the associated Gaussian processes.
• Gaussian Process Regression with GPy In this example we show how to do a simple regression model using Gaussian processes in GPy.
• Optimizing Gaussian Processes The parameters of the covariance function can be optimized. In this example we show how to optimize the parameters of the covariance function. (TODO HMC)

Advanced Topics

Things we haven't had time to cover in the MLSS can be found below.

Structured Outputs with Gaussian Processes

Gaussian processes for learning vector valued functions.

• Multiple Output GPs
• TODO Differential Equations and Gaussian Processes

Approximations

These examples look at approximations for speeding up inference in Gaussian processes and/or making inference tractable.

• Low Rank Approximations for Gaussian Processes
• Non Gaussian Likelihoods
• Low Rank and Non Gaussian

Dimensionality Reduction

These examples look at dimensionality reduction with Gaussian processes.

• Dimensionality Reduction with Gaussian Processes