$y_i \sim \mathrm{Normal}\left(a + b x_i, \sigma_i^2\right)$
Note: this is an alternative and more compact notation for writing that $P(y_i|x_i,a,b)$ is Gaussian with mean $a+b x_i$ and width $\sigma$.
$x_i \sim \mathrm{Normal}(\mu_x, \tau_x^2)$
$\sigma_i = s \mathrm{~(constant)}$
In [41]:
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
plt.rcParams['figure.figsize'] = (8.0, 8.0)
# the model parameters
a = np.pi
b = 1.6818
# my arbitrary constants
mu_x = np.exp(1.0) # see definitions above
tau_x = 1.0
s = 1.0
N = 50 # number of data points
# get some x's and y's
x = mu_x + tau_x*np.random.randn(N)
y = a + b*x + s*np.random.randn(N)
In [42]:
plt.plot(x, y, 'o');
$\ln L = \sum_i -\frac{1}{2}\left(a+b x_i - y_i\right)^2$ + constant
$\left(\begin{array}{c}a\\b\end{array}\right) \sim \mathrm{Normal}\left[\left(X^\mathrm{T}X\right)^{-1}X^\mathrm{T}y, \left(X^\mathrm{T}X\right)^{-1}\right]$, where $X = \left(1 ~~ x\right)$.
In [68]:
X = np.matrix(np.vstack([np.ones(len(x)), x]).T)
Y = np.matrix(y).T
In [67]:
np.linalg.inv(X.T*X)*X.T*Y
Out[67]:
In [59]:
np.linalg.lstsq(X, y)[0]
Out[59]:
In [69]:
np.linalg.inv(X.T*X)
Out[69]: