Let $X \sim \operatorname{Expo}(1)$.

We begin by finding the MGF of $X$:

\begin{align} M(t) &= \mathbb{E}(e^{tX}) &\quad \text{ definition of MGF} \\ &= \int_{0}^{\infty} e^{-x} \, e^{tx} dx \\ &= \int_{0}^{\infty} e^{-x(1-t)} dx \\ &= \boxed{\frac{1}{1-t}} &\quad \text{ for } t < 1 \end{align}In finding the moments, by definition we have:

- $M'(0) = \mathbb{E}(X)$
- $M''(0) = \mathbb{E}(X^2)$
- $M'''(0) = \mathbb{E}(X^3)$
- ... and so on ...

Even though finding derivatives of $\frac{1}{1-t}$ is not all that bad, it is nevertheless annoying busywork. But since we know that the $n^{th}$ moment is the coefficient of the $n^{th}$ term of the Taylor Expansion of $X$, we can leverage that fact instead.

\begin{align} \frac{1}{1-t} &= \sum_{n=0}^{\infty} t^n &\quad \text{ for } |t| < 1 \\ &= \sum_{n=0}^{\infty} \frac{n! \, t^n}{n!} &\quad \text{ since we need the form } \sum_{n=0}^{\infty} \left( \frac{\mathbb{E}(X^{n}) \, t^{n}}{n!}\right) \\ \\ \Rightarrow \mathbb{E}(X^n) &= \boxed{n!} \end{align}And now we can simply *generate* arbitary moments for r.v. $X$!

- $\mathbb{E}(X) = 1! = 1$
- $\mathbb{E}(X^2) = 2! = 2$
- $\Rightarrow \operatorname{Var}(X) = \mathbb{E}(X^2) - \mathbb{E}X^2 = 2 - 1 = 1$

Let $Y \sim \operatorname{Expo}(\lambda)$.

We begin with

\begin{align} \text{let } X &= \lambda Y \\ \text{and so } X &= \lambda Y \sim \operatorname{Expo}(1) \\ \\ \text{then } Y &= \frac{X}{\lambda} \\ Y^n &= \frac{X^n}{\lambda^n} \\ \\ \Rightarrow \mathbb{E}(Y^n) &= \frac{\mathbb{E}(X^n)}{\lambda^n} \\ &= \boxed{\frac{n!}{\lambda^n}} \end{align}And as before, we now can simply *generate* arbitary moments for r.v. $Y$!

- $\mathbb{E}(Y) = \frac{1!}{\lambda^1} = \frac{1}{\lambda}$
- $\mathbb{E}(Y^2) = \frac{2!}{\lambda^2} = \frac{2}{\lambda^2}$
- $\Rightarrow \operatorname{Var}(Y) = \mathbb{E}(Y^2) - \mathbb{E}Y^2 = \frac{2}{\lambda^2} - \left(\frac{1}{\lambda}\right)^2 = \frac{1}{\lambda^2}$

Let $Z \sim \mathcal{N}(0,1)$; find **all** its moments.

We have seen before that, by symmetry, $\mathbb{E}(Z^{2n+1}) = 0$ for all odd values of $n$.

So we will focus in on the *even* values of $n$.

Now the MGF $M(t) = e^{t^2/2}$. Without taking *any* derivatives, we can immediately Taylor expand that, since it is continuous everywhere.

Let's double-check this with what know about $\mathbb{Var}(Z)$

- by symmetry, we know that $\mathbb{E}(Z) = 0$
- at $n = 1$, $\mathbb{E}(Z^2) = \frac{2!}{2 \times 1!} = 1$
- $\Rightarrow \operatorname{Var}(Z) = \mathbb{E}(Z^2) - \mathbb{E}Z^2 = 1 - 0 = 1$
- at $n = 2$, $\mathbb{E}(Z^4) = \frac{4!}{4 \times 2!} = 3$
- at $n = 3$, $\mathbb{E}(Z^6) = \frac{6!}{8 \times 3!} = 15$

And so you might have noticed a pattern here. Let us rewrite those even moments once more:

- at $n = 1$, $\mathbb{E}(Z^2) = 1$
- at $n = 2$, $\mathbb{E}(Z^4) = 1 \times 3 = 3$
- at $n = 3$, $\mathbb{E}(Z^6) = 1 \times 3 \times 5 = 15$

Let $X \sim \operatorname{Pois}(\lambda)$; now let's consider MGFs and how to use them to find sums of random variables (convolutions).

\begin{align} M(t) &= \mathbb{E}(e^{tX}) \\ &= \sum_{k=0}^{\infty} e^{tk} \, \frac{\lambda^k e^{-\lambda}}{k!} \\ &= e^{-\lambda} \sum_{k=0}^{\infty} \frac{\lambda^k e^{tk}}{k!} \\ &= e^{-\lambda} e^{\lambda e^t} &\quad \text{but the right is just another Taylor expansion} \\ &= \boxed{e^{\lambda (e^t - 1)}} \end{align}Now let's let $Y \sim \operatorname{Pois}(\mu)$, and it is independent of $X$. Find the distribution of $(X + Y)$.

You may recall that with MGFs, all we need to do is *multiply* the MGFs.

When adding a Poisson distribution $X$ with another independent Poisson distribution $Y$, the resulting convolution $X + Y$ will also be Poisson. This interesting relation only happens with Poisson distributions.

Now think about what happens with $X$ and $Y$ are *not* independent.

Let $Y = X$, so that $X + Y = 2X$, which is clearly *not* Poisson, as

- $X + Y = 2X$ is an even function which cannot be Poisson, since Poisson can take on all values both even
*and*odd - Mean $\mathbb{E}(2x) = 2\lambda$, but $\operatorname{Var}(2X) = 4\lambda$, and since the mean and variance are
*not*equal, this cannot be Poisson

In the most basic case of two r.v. in a joint distribution, consider both r.v.'s *together*:

Joint CDFIn the general case, the joint CDF fof two r.v.'s is $ F(x,y) = P(X \le x, Y \le y)$

Joint PDF$f(x, y)$ such that, in thecontinuouscase $P((X,Y) \in B) = \iint_B f(x,y)\,dx\,dy$

$f(x, y)$ such that, in the *discrete* case

We also can consider a single r.v. of a joint distribution:

$X, Y$ are independent iff $F(x,y) = F_X(x) \, F_Y(y)$.

\begin{align} P(X=x, Y=y) &= P(X=x) \, P(Y=y) &\quad \text{discrete case} \\ \\\\ f(x, y) &= f_X(x) \, f_Y(y) &\quad \text{continuous case} \end{align}... with the caveat that this must be so for *all* $\text{x, y} \in \mathbb{R}$

$P(X \le x)$ is the *marginal distribution* of $X$, where we consider one r.v. at a time.

In the case of a two-r.v. joint distribution, we can get the marginals by using the joint distribution itself:

\begin{align} P(X=x) &= \sum_y P(X=x, Y=y) &\quad \text{marginal PMF, discrete case, for } x \\ \\\\ f_Y(y) &= \int_{-\infty}^{\infty} f_{(X,Y)}(x,y) \, dx &\quad \text{marginal PDF, continuous case, for } y \end{align}Let $X, Y$ be both Bernoulli. $X$ and $Y$ may be independent; or they might be dependent. They may or may not have the same $p$. But they are both related in the form of a *joint distribution*.

We can lay out this joint distribution in a $2 \times 2$ contigency table like below:

$Y=0$ | $Y=1$ | |
---|---|---|

$X=0$ | 2/6 | 1/6 |

$X=1$ | 2/6 | 1/6 |

In order to be a joint distribution, all of the values in our contigency table must be positive; and they must all sum up to 1. The example above shows such a PMF.

Let's add the marginals for $X$ and $Y$ to our $2 \times 2$ contigency table:

$Y=0$ | $Y=1$ | ... | |
---|---|---|---|

$X=0$ | 2/6 | 1/6 | 3/6 |

$X=1$ | 2/6 | 1/6 | 3/6 |

... | 4/6 | 2/6 |

Observe how in our example, we have:

\begin{align} P(X=0,Y=0) &= P(X=0) \, P(Y=0) \\ &= 3/6 \times 4/6 = 12/36 &= \boxed{2/6} \\ \\ P(X=0,Y=1) &= P(X=0) \, P(Y=1) \\ &= 3/6 \times 2/6 = 6/36 &= \boxed{1/6} \\ P(X=1,Y=0) &= P(X=1) \, P(Y=0) \\ &= 3/6 \times 4/6 = 12/36 &= \boxed{2/6} \\ \\ P(X=1,Y=1) &= P(X=1) \, P(Y=1) \\ &= 3/6 \times 2/6 = 6/36 &= \boxed{1/6} \\ \end{align}and so you can see that $X$ and $Y$ are independent.

Now here's an example of a two r.v. joint distribution where $X$ and $Y$ are *dependent*; check it out for yourself.

$Y=0$ | $Y=1$ | |
---|---|---|

$X=0$ | 1/3 | 0 |

$X=1$ | 1/3 | 1/3 |

Now say we had Uniform distributions on a square such that $x,y \in [0,1]$.

The joint PDF would be constant on/within the square; and 0 outside.

\begin{align} \text{joint PDF} &= \begin{cases} c &\quad \text{if } 0 \le x \le 1 \text{, } 0 \le y \le 1 \\ \\ 0 &\quad \text{otherwise} \end{cases} \end{align}In 1-dimension space, if you integrate $1$ over some interval you get the *length* of that interval.

In 2-dimension space, if you integrate $1$ over some region, you get the *area* of that region.

Normalizing $c$, we know that $c = \frac{1}{area} = 1$.

Marginally, $X$ and $Y$ are independent $\mathcal{Unif}(1)$.

Now say we had Uniform distributions on a *disc* such that $x^2 + y^2 \le 1$.

In this case, the joint PDF is $\operatorname{Unif}$ over the area of a disc centered at the origin with radius 1.

\begin{align} \text{joint PDF} &= \begin{cases} \frac{1}{\pi} &\quad \text{if } x^2 + y^2 \le 1 \\ \\ 0 &\quad \text{otherwise} \end{cases} \end{align}View Lecture 18: MGFs Continued | Statistics 110 on YouTube.