Let $X \sim \operatorname{Expo}(1)$.
We begin by finding the MGF of $X$:
\begin{align} M(t) &= \mathbb{E}(e^{tX}) &\quad \text{ definition of MGF} \\ &= \int_{0}^{\infty} e^{-x} \, e^{tx} dx \\ &= \int_{0}^{\infty} e^{-x(1-t)} dx \\ &= \boxed{\frac{1}{1-t}} &\quad \text{ for } t < 1 \end{align}In finding the moments, by definition we have:
Even though finding derivatives of $\frac{1}{1-t}$ is not all that bad, it is nevertheless annoying busywork. But since we know that the $n^{th}$ moment is the coefficient of the $n^{th}$ term of the Taylor Expansion of $X$, we can leverage that fact instead.
\begin{align} \frac{1}{1-t} &= \sum_{n=0}^{\infty} t^n &\quad \text{ for } |t| < 1 \\ &= \sum_{n=0}^{\infty} \frac{n! \, t^n}{n!} &\quad \text{ since we need the form } \sum_{n=0}^{\infty} \left( \frac{\mathbb{E}(X^{n}) \, t^{n}}{n!}\right) \\ \\ \Rightarrow \mathbb{E}(X^n) &= \boxed{n!} \end{align}And now we can simply generate arbitary moments for r.v. $X$!
Let $Y \sim \operatorname{Expo}(\lambda)$.
We begin with
\begin{align} \text{let } X &= \lambda Y \\ \text{and so } X &= \lambda Y \sim \operatorname{Expo}(1) \\ \\ \text{then } Y &= \frac{X}{\lambda} \\ Y^n &= \frac{X^n}{\lambda^n} \\ \\ \Rightarrow \mathbb{E}(Y^n) &= \frac{\mathbb{E}(X^n)}{\lambda^n} \\ &= \boxed{\frac{n!}{\lambda^n}} \end{align}And as before, we now can simply generate arbitary moments for r.v. $Y$!
Let $Z \sim \mathcal{N}(0,1)$; find all its moments.
We have seen before that, by symmetry, $\mathbb{E}(Z^{2n+1}) = 0$ for all odd values of $n$.
So we will focus in on the even values of $n$.
Now the MGF $M(t) = e^{t^2/2}$. Without taking any derivatives, we can immediately Taylor expand that, since it is continuous everywhere.
\begin{align} M(t) &= e^{t^2/2} \\ &= \sum_{n=0}^{\infty} \frac{\left(t^2/2\right)^n}{n!} \\ &= \sum_{n=0}^{\infty} \frac{t^{2n}}{2^n \, n!} \\ &= \sum_{n=0}^{\infty} \frac{(2n)! \, t^{2n}}{2^n \, n! \, (2n)!} &\quad \text{ since we need the form } \sum_{n=0}^{\infty} \left( \frac{\mathbb{E}(X^{n}) \, t^{n}}{n!}\right) \\ \\ \Rightarrow \mathbb{E}(Z^{2n}) &= \boxed{\frac{(2n)!}{2^n \, n!}} \end{align}Let's double-check this with what know about $\mathbb{Var}(Z)$
And so you might have noticed a pattern here. Let us rewrite those even moments once more:
Let $X \sim \operatorname{Pois}(\lambda)$; now let's consider MGFs and how to use them to find sums of random variables (convolutions).
\begin{align} M(t) &= \mathbb{E}(e^{tX}) \\ &= \sum_{k=0}^{\infty} e^{tk} \, \frac{\lambda^k e^{-\lambda}}{k!} \\ &= e^{-\lambda} \sum_{k=0}^{\infty} \frac{\lambda^k e^{tk}}{k!} \\ &= e^{-\lambda} e^{\lambda e^t} &\quad \text{but the right is just another Taylor expansion} \\ &= \boxed{e^{\lambda (e^t - 1)}} \end{align}Now let's let $Y \sim \operatorname{Pois}(\mu)$, and it is independent of $X$. Find the distribution of $(X + Y)$.
You may recall that with MGFs, all we need to do is multiply the MGFs.
\begin{align} e^{\lambda (e^t - 1)} \, e^{\mu (e^t - 1)} &= e^{(\lambda + \mu)(e^t - 1)} \\ \\ \Rightarrow X + Y &\sim \mathcal{Pois}(\lambda + \mu) \end{align}When adding a Poisson distribution $X$ with another independent Poisson distribution $Y$, the resulting convolution $X + Y$ will also be Poisson. This interesting relation only happens with Poisson distributions.
Now think about what happens with $X$ and $Y$ are not independent.
Let $Y = X$, so that $X + Y = 2X$, which is clearly not Poisson, as
In the most basic case of two r.v. in a joint distribution, consider both r.v.'s together:
Joint CDF
In the general case, the joint CDF fof two r.v.'s is $ F(x,y) = P(X \le x, Y \le y)$
Joint PDF $f(x, y)$ such that, in the continuous case $P((X,Y) \in B) = \iint_B f(x,y)\,dx\,dy$
$f(x, y)$ such that, in the discrete case
\begin{align} P(X=x, Y=y) \end{align}We also can consider a single r.v. of a joint distribution:
$X, Y$ are independent iff $F(x,y) = F_X(x) \, F_Y(y)$.
\begin{align} P(X=x, Y=y) &= P(X=x) \, P(Y=y) &\quad \text{discrete case} \\ \\\\ f(x, y) &= f_X(x) \, f_Y(y) &\quad \text{continuous case} \end{align}... with the caveat that this must be so for all $\text{x, y} \in \mathbb{R}$
$P(X \le x)$ is the marginal distribution of $X$, where we consider one r.v. at a time.
In the case of a two-r.v. joint distribution, we can get the marginals by using the joint distribution itself:
\begin{align} P(X=x) &= \sum_y P(X=x, Y=y) &\quad \text{marginal PMF, discrete case, for } x \\ \\\\ f_Y(y) &= \int_{-\infty}^{\infty} f_{(X,Y)}(x,y) \, dx &\quad \text{marginal PDF, continuous case, for } y \end{align}Let $X, Y$ be both Bernoulli. $X$ and $Y$ may be independent; or they might be dependent. They may or may not have the same $p$. But they are both related in the form of a joint distribution.
We can lay out this joint distribution in a $2 \times 2$ contigency table like below:
| $Y=0$ | $Y=1$ | |
|---|---|---|
| $X=0$ | 2/6 | 1/6 |
| $X=1$ | 2/6 | 1/6 |
In order to be a joint distribution, all of the values in our contigency table must be positive; and they must all sum up to 1. The example above shows such a PMF.
Let's add the marginals for $X$ and $Y$ to our $2 \times 2$ contigency table:
| $Y=0$ | $Y=1$ | ... | |
|---|---|---|---|
| $X=0$ | 2/6 | 1/6 | 3/6 |
| $X=1$ | 2/6 | 1/6 | 3/6 |
| ... | 4/6 | 2/6 |
Observe how in our example, we have:
\begin{align} P(X=0,Y=0) &= P(X=0) \, P(Y=0) \\ &= 3/6 \times 4/6 = 12/36 &= \boxed{2/6} \\ \\ P(X=0,Y=1) &= P(X=0) \, P(Y=1) \\ &= 3/6 \times 2/6 = 6/36 &= \boxed{1/6} \\ P(X=1,Y=0) &= P(X=1) \, P(Y=0) \\ &= 3/6 \times 4/6 = 12/36 &= \boxed{2/6} \\ \\ P(X=1,Y=1) &= P(X=1) \, P(Y=1) \\ &= 3/6 \times 2/6 = 6/36 &= \boxed{1/6} \\ \end{align}and so you can see that $X$ and $Y$ are independent.
Now here's an example of a two r.v. joint distribution where $X$ and $Y$ are dependent; check it out for yourself.
| $Y=0$ | $Y=1$ | |
|---|---|---|
| $X=0$ | 1/3 | 0 |
| $X=1$ | 1/3 | 1/3 |
Now say we had Uniform distributions on a square such that $x,y \in [0,1]$.
The joint PDF would be constant on/within the square; and 0 outside.
\begin{align} \text{joint PDF} &= \begin{cases} c &\quad \text{if } 0 \le x \le 1 \text{, } 0 \le y \le 1 \\ \\ 0 &\quad \text{otherwise} \end{cases} \end{align}In 1-dimension space, if you integrate $1$ over some interval you get the length of that interval.
In 2-dimension space, if you integrate $1$ over some region, you get the area of that region.
Normalizing $c$, we know that $c = \frac{1}{area} = 1$.
Marginally, $X$ and $Y$ are independent $\mathcal{Unif}(1)$.
Now say we had Uniform distributions on a disc such that $x^2 + y^2 \le 1$.
In this case, the joint PDF is $\operatorname{Unif}$ over the area of a disc centered at the origin with radius 1.
\begin{align} \text{joint PDF} &= \begin{cases} \frac{1}{\pi} &\quad \text{if } x^2 + y^2 \le 1 \\ \\ 0 &\quad \text{otherwise} \end{cases} \end{align}View Lecture 18: MGFs Continued | Statistics 110 on YouTube.