Multivariate Normal $\chi^2, t, F$

Definition:

A random vector $X=(X_1, X_2, ..., X_k)^T$ is a multivariate normal if by definition, all linear combination of its components are normal random variables.

Properties:

(Recall): Let $X\in\mathbb{R}^k$.
The density of $X$ at $x$ is

$$f_X(x) = (2\pi)^{-k/2} det(\Sigma)^{-1/2} e^{-\frac{1}{2}\phi(x)}$$

where $\Sigma = \text{covariance of }X$ i.e. $\Sigma = \mathbf{Cov}(X_i, X_j)$

and $\phi(x) = (x-\mu)^T \Sigma^{-1} (x-\mu)$ and $\mu = \mathbf{E}[X]$

Notation:

$X\sim \mathcal{N_k} (\mu, \Sigma)$

  • Let $X$ as above, let $A$ be an $m\times k$ matrix. Then, $$AX\sim \mathcal{N_m}(A\mu, A\Sigma A^T)$$
  • APPLICATION OF THE PREVIUOUS POINT

Use $A = \sqrt{\Sigma}^{-1}$, in other words, pick a matrix $A$ with this property: $AA^t = A^TA = \Sigma^{-1}$

Then, construct $Y = AX - \mu$.

Easy to prove $Y \sim \mathcal{N_k}(0, I_k)$

Note: this about $Y$ is true if and only if $Y$'s components are $\mathcal{N}(0,1)$ iid.

Example

Easy calculation: let $t = (t_1, t_2, ..., t_k)$.

Let $Z=t_1 X_1 + t_2 X_2 + ... t_k X_k$

By definition, $Z$ is normal, $\mathbf{E}[X] = \sum_{i=1}^k t_i \mu_i = t\mu \text{ (vector notation) }$

$$\mathbf{Cov}(Z) = \sum_{i=1}^k\sum_{j=1}^k t_i t_j \Sigma_{ij} = t\Sigma t^T \text{vector notation} \\ \text{ where } \Sigma_{ij} = \mathbf{E}[(X_i-\mu_i)(X_j - \mu_j)] $$

Therefore, $Z \sim \mathcal{N}(t\mu, t\Sigma t^T)$

Consequently, $$\mathbf{E}[e^{t_1X_1 + t_2X_2 + ... + t_nX_n}] = \displaystyle e^{\displaystyle t\mu + \frac{1}{2} t~\Sigma ~t^T} = \text{ MGF of multivariate normal }$$

  • In particular, two normal random vectors $X_1,X_2$ with dimensions $k_1,k_2$ are independent of each other if and only if their covariance matrix of $X=\left(\begin{array}{c}X_1\\X_2\end{array}\right)$ is of the block form

    $$\left(\begin{array}{cccccc} & \Sigma_{X_1} & & 0 & &\\ & & 0 & & \Sigma_{X_2} &\end{array}\right)$$

Application:

Let $Y_1,Y_2,.. Y_n$ be iid $\sim \mathcal{N}(\mu, \sigma^2)$.

Let $X_i = Y_i - \bar{Y}$ where $\bar{Y}= \frac{1}{n} \sum_{i=1}^n Y_i$

These $X_i$'s are the deviations of the $X$'s from the sample mean.

Thus:

$$\mathbf{Cov}(\bar{Y}, X_i) = \mathbf{Cov}(\bar{Y}, Y_i - \bar{Y}) = \mathbf{Cov}(\bar{Y}, Y_i) - \mathbf{Cov}(\bar{Y},\bar{Y})$$

We can see that $\mathbf{Cov}(\bar{Y},\bar{Y}) = \mathbf{Var}[\bar{Y}] = \frac{\sigma^2}{n}$

Also, $\mathbf{Cov}(\bar{Y}, Y_i) = \frac{1}{n}\mathbf{Var}[Y_i] = \frac{\sigma^2}{n}$

Therefore, $$\mathbf{Cov}(\bar{Y}, X_i) = 0$$

The covariance calculation shows that $\bar{Y}$ and $X_i$ are uncorrelated.

But since $X$ is a Normal vector, by definition, $(\bar{Y}, X_i)$ is a 2-dimensional normal vector. So, since everyone is jointly normal, $\bar{Y},X_i$ are independent.

$\chi^2$ Distribution

Let $Z_i \sim ~iid ~\mathcal{N}(0,1)$

Let $Q = \sum_{i=1}^n Z_i^2$

Then, $$Q\sim \chi^2 ~\text{ with n degrees of freedom}$$

Let $X_i \sim ~iid~\mathcal{N}(0,\sigma^2)$

Let $\bar{X} = \frac{1}{n} \sum_{}X_i$ (sample mean)

Let $S^2 = \frac{1}{n-1}\sum_{} (X_i-\bar{X})^2$ (sample variance)

Then, $$S^2 \text{ and } \bar{X}\text{ are independent}$$

The reason for that is because of the previous example we just did.

And, moreover,

Let $W = \frac{S^2 (n-1)}{\sigma^2}$, we get $W\sim \chi^2_{n-1}$ (with $n-1$ degrees of freedom)


In [ ]: