Random Variables

Definition:

A random variable $X$ is a function from a probability space $\Omega$ into $I\!R$

Note: this is perfectly legitimiate if $\Omega$ is finite or countable.

Example:

The sum of the values in a roll of two dice is a function from a set $\Omega$ with $36$ outcomes to the finite set of integeres $2,3, .. 12$

Notation:

$\{X = x\}$ is an event

Example:

• with $X$ as in the previous example, $\left\{X=3 \right\} = \left\{(1,2), (2,1)\right\}$
• and similarly: $\left\{X=7\right\} = \left\{(1,6), (6,1), (2,5), (5,2), (3,4), (4,3)\right\}$
• $\mathbf{P}[\{X = 3\}] = \frac{2}{36}$
• $\mathbf{P}[\{X=7\}] = \frac{6}{36}$

More notations:

The value $\mathbf{P}[\{X = x\}]$ is usually denoted by $p_x$ or $P_X(x)$

Definition:

This $p$ or $P_X(.)$ is called the probability mass function of $X$. Some call it the mass of $X$, or the distribution of $X$.

Example

In the previous example:

How to find out if two functions are independent?

see if the probability of their intersection holds as follws:

• $X$&$Y$ are independent if : $$\mathbf{P}[\{X = x\}\cap \{Y=y\}] = \mathbf{P}[\{X=x\}] \mathbf{P}[\{Y = y\}]=P_X(x)P_Y(y)\ \ \ \forall x,y$$

Bernoulli & Binomial Random Variable

Toss a coin $n$ times independently. Probability of a head is always $p\in (0,1)$

Let $X_i =1 $ if ith toss is heads, and $X_i=0$ if toss is tails.\

So we see $P_{X_i}(1) = p$ and $P_{X_i}(0) = 1- p$

This $X_i$ is called a Bernoulli random variable. We can interpret $X_i = 1$ as a success at trial $i$. Moreover, those $X_i$'s are all independent of each other.

Now, let $X=\#\text{of successes (heads)}$ in these $n$ independet trials (tosses). X is also a random variabkle, which can take the values $0,1,2, .. n$. This $X$ is called a binomial with parameters $n$ and $p$.

Example

Let us find the binomial $X$'s prob mass $P_X(k) = \left\{\text{exactly }k\text{ successes in }n\text{ trials}\right\}$

• Intuitive reasining: there are $\left(\begin{array}{c}n\\k\end{array}\right)$ ways to have $k$ success. All such ways are incompatible (disjoint) events: $$P_X(k) = \left\{\text{exactly }k\text{ successes in }n\text{ trials}\right\}=\left(\begin{array}{c}n\\k\end{array}\right) p^k \left(1-p\right)^{n-k}$$

• Important fact: the way that we defined $X$ from the $X_i$'s we get $X=X_1 + X_2 + .. X_n$.
  This proves the following theorem

Theorem:

the sum of $n$ independent Bernoulli random variables with parameter $p$ is a binomial random variable with parameters $n$ and $p$

Geometric distribution

Let $X_1,X_2,...X_m,...$ be an infinite number of independent Bernoulli trials all with the same success probability $p$. Let $Y$ be first time that $X_i = 1$.

• For each $k=1,2,...$ we have $\mathbf{P}[Y = k] = (1-p)^{1-k} p $

Multinomial (Generalized Bernoulli) distribution

Negative Binomial Distribution

• Binomial random variables is the sum of some independent Bernoulli random variables

• Negative Binomial random variables is the sum of some geometric random variables

where $Y_i\sim Geometric()$ and therefore $Y\sim NegativeBinomial$

• For a sequence of failure/success: $001010001000$

  • $Y_1 = 3~\ ~Y_2=2~\ ~Y_3=4,...$

  • There is another convention, that they define $Y'_i=Y_i-1$, so in this other convention: $Y'_1=2, Y'_2=1, Y'_3=3$

Therefore, the negative binomial under this other convention is $Y'=Y-n$

• Under our convention: $\mathbf{E}[NegBinom(n,p)] = \frac{n}{p}$
• Under the other convention: $\mathbf{E}[NegBinom'(n,p)] = \frac{n}{p} - n = n(\frac{1}{p}-1)$