Some Definitions

1. ###Inner product space
   • ###Inner product
2. ###Hilbert space
   • ###Complete
   • ###Separable
3. ###Reproducing kernel Hilbert space
   • ###Reproducing property

Kernel Methods in Machine Learning

Advantages of RKHS in Machine Learning

• ##Powerful and flexible models can be defined.

• ##Many results and algorithms for linear models in Euclidean spaces can be generalized to RKHS.

• ##Learning theory assures that effective learning in RKHS is possible, for instance, by means of regularization.

Example - Ridge Regression

Objective : Minimize $\|A\mathbf{x}-\mathbf{b}\|^2_2+ \lambda\| \mathbf{x}\|^2_2$

Explicit solution : $\hat{x} = (A^{T}A+ \lambda I )^{-1}A^{T}\mathbf{b}$

Kernelization $\implies$ $\hat{x} = (K+ \lambda I )^{-1}A^{T}\mathbf{b}$