损失函数

最小二乘, 残差平方和, MSE

参数函数: $h(\theta_1, \theta_2, ..., \theta_m) = \theta_1x_1 + \theta_2x_2 + ... + \theta_mx_m$

损失函数: $J(\theta) = \dfrac{1}{2m} \sum\limits_{i=1}^m{\big[ h(x^i) - y^i \big]^2}$

偏导: $\dfrac{\partial{J(\theta)}}{\partial{\theta_j}} = \dfrac{1}{m}\sum\limits_{i=1}^{m}\big[h_\theta(x^i) - y^i \big] x_j^i $

m: 样本数量

i: 第i个样本,一般放到右上方标记

j: 参数的序号: $\theta_0, \theta_1$

梯度下降更新:

$\theta_j' = \theta_j - \alpha \dfrac{\partial{J(\theta)}}{\partial{\theta_j}}$ $\alpha$是learning rate