P1.1

Polyomial function (1.1): $y(x,w) = \sum_{j=0}^M w_j x^j$

Sum of squares (1.2): $E(w) = \frac{1}{2} \sum_{n=1}^N (y(x_n,w) - t_n)^2 = \frac{1}{2} \sum_{n=1}^N (\sum_{j=0}^M w_j x_n^j - t_n)^2 = \frac{1}{2} \sum_{n=1}^N (w^T [x_n^0 x_n^1 \ldots x_n^M]^T - t_n)^2 = \frac{1}{2} \sum_{n=1}^N (w^T X_n - t_n)^2$

$\nabla_w E(w) = \sum_{n=1}^N (w^T X_n - t_n)(X_n) = 0$

$\Rightarrow \sum_{n=1}^N (w^T X_n) X_n = \sum_{n=1}^N (t_n X_n)$

$\Rightarrow \sum_{n=1}^N \sum_{j=0}^M(w_j x_n^j) [x_n^0, x_n^1, \ldots, x_n^M]^T = \sum_{n=1}^N (t_n [x_n^0, x_n^1, \ldots, x_n^M]^T)$

Element wise:

$\Rightarrow \sum_{n=1}^N \sum_{j=0}^M(w_j x_n^j) x_n^i = \sum_{n=1}^N (t_n x_n^i)$

$= \sum_{n=1}^N \sum_{j=0}^M w_j x_n^{j+i} = \sum_{n=1}^N (t_n x_n^i)$

$= \sum_{j=0}^M w_j \sum_{n=1}^N x_n^{j+i} = \sum_{n=1}^N (t_n x_n^i)$

And finally:

$\sum_{j=0}^M A_{ij} w_j = T_i$

where

$A_{ij} = \sum_{n=1}^N (x_n)^{i+j}, T_i = \sum_{n=1}^N (x_n)^i t_n$