Lecture 1: Probability and Counting

Stat 110, Prof. Joe Blitzstein, Harvard University

Definitions

We start with some basic definitions:

Definition: sample space

A sample space is the set of all possible outcomes of an experiment.

Definition: event

An event is a subset of the sample space.

Definition: naïve definition of probability

Under the naïve definition of probability, the probability of a given event $A$ occurring is expressed as

\begin{align} P(A) &= \frac{ \text{# favorable outcomes}}{\text{# possible outcomes}} \end{align}

assuming all outcomes are equally likely in a finite sample space.

Counting

With the multiplication rule, if we have an experiment with $n_1$ possible outcomes; and we have a 2^nd experiment with $n_2$ possible outcomes; ..., and for the r^th experiment there are $n_r$ possible outcomes; then overall there are $n_1 n_2 ... n_r$ possible outcomes (product).

Let's say you are ordering ice cream. You can either get a cone or a cup, and the ice cream comes in three flavors. The order of choice here does not matter, and the total number of choices is $2 \times 3 = 3 \times 2 = 6$. This can be represented with a very simple probability tree.

The binomial coefficient is defined as

\begin{align} \binom{n}{k} = \begin{cases} \frac{n!}{(n-k)!k!} & \quad \text{if } 0 \le k \le n \\ 0 & \quad \text{if } k \gt n \end{cases} \end{align}

This expresses the number of ways you could choose a subset of size $k$ from $n$ items, where order doesn't matter.

Sampling

Choose $k$ objects out of $n$

	ordered	unordered
w/ replacement	$n^k$	???
w/o replacement	$n(n-1)(n-2) \ldots (n-k+1)$	$\binom{n}{k}$

ordered, w/ replacement: there are $n$ choices for each $k$, so this follows from the multiplication rule.
ordered, w/out replacement: there are $n$ choices for the 1^st position; $n-1$ for the 2^nd; $n-2$ for the 3^rd; and $n-k+1$ for the $k$^th.
unordered, w/ replacment: ???
unordered, w/out replacement: the binomial coefficient; think of choosing a hand from a deck of cards.

Out of the 4 ways of choosing $k$ objects out of $n$, the case of unordered, with replacement is perhaps not as clear-cut and easy to grasp as the other three. Move on to Lecture 2.

View Lecture 1: Probability and Counting | Statistics 110 on YouTube.