Lesson 10: t-Tests

t-distribution

So far we have assumed that we know $\mu$ and $\sigma$ for our population. But usually we don't know them. We have samples and we need to use them to see

how different sample mean is from population
how different 2 sample means are from each other
- could be dependent
- could be independent

Till now we were using $\sigma$ to find standard error. But we cannot do that as we don't have $\sigma$. So we have a different distribution that is more error prone. It is t-distribution

Degrees of Freedom

Basically the number of actual choices that you could make excluding the ones that are forced due to constraints.

For a sample taken from a population our choice of sample is constrained by the population itself. We cannot pick something that isn't there. If we a good sample (large enough, completely representative of the population and chosen by a sound methodology) then we could at most choose n - 1 points. The last point would be chosen for us (by our methodology of taking good sample rather than someone choosing it for us).

Hence, our effective sample size is (n - 1)

t-statistic

We have t table at https://s3.amazonaws.com/udacity-hosted-downloads/t-table.jpg

$t = \frac{\bar{x} - \mu_o} {S / \sqrt{n}}$

Here the $\mu_o$ is the mean of the population to which we are comparing the sample with mean $\bar{x}$ coming from a different population. The denominator represents the error that we expect between population mean and $\mu_o$ by chance

Details here. Needs subscription

p-value

p-value is probability above/below the t-statistic. We can calculate p-values here

Cohen's d

Common measure of effect size when comparing means
standardized mean difference that compares distance between means in standardized units

Cohen's d $= \frac{M - \mu_o}{S}$

Dependent t-test for paired Samples

Dependent samples occur when the same subject takes the test twice
Within-subject designs
- Two conditions
- Pre-test and post-test
- Growth over time (longitudinal study)

Effect size

In experimental studies it means size of treatment effect
In non-experimental studies it means strength of relationship between variables
Simplest measure is mean difference. Good for variables with easy to understand meanings

Type of effect size measures

Difference measures
- Mean difference
- Standardized differences
  - Cohen's d
Correlation measures
- $r^2$
  - proportion of variation in one variable that is related to ("explained by") another variable
  - also called coefficient of determination (0.0 to 1.0)

Meaningfulness of results

What was measured? Were the variables had practical, social or theoretical importance?
Effect size
Can we rule out random chance?
Can we rule out alternative explanations?

Calculating $r^2$

$r^2 = \frac{t^2}{t^2 + df}$

Report Results

Descriptive statistics
- Mean, SD
- as text, graphs, tables
inferential statistics
- hypothesis test
  - kind of test - 1-sample t-test
  - test statistic
  - degree of freedom
  - p-value
  - direction of test
  - $\alpha$
  - APA style
    - $t(df) = X.XX, p = .XX, direction$
    - $t(24) = -2.50, p < .05, one-tailed$
  - confidence interval
    - confidence level e.g. 95%
    - lower limit
    - upper limit
    - CI on what?
    - APA style - CIs
      - Confidence interval on the mean difference; 95% CI = (4, 6)
      - Confidence interval on the mean difference; 95% CI = (4 - 6)
  - effect size measures
    - d, $r^2$
    - $d = X.XX$
    - $r^2 = .XX$

Full one-sample t-test

$df = n - 1$
$SEM = \frac{S}{\sqrt{n}}$
$t = \frac{\bar{x} - \mu}{SEM}$
$CI = \bar{x} \pm Margin of error$
$Margin of error = t_{critical} * SEM$
Fact for out case - "US families spent an average of $151 per week on food in 2012"

So $\mu = 151$

We have a group that wants to bring this down
Dependent variable - amount of money spent per week
Treatment effect - the cost saving program
null hypothesis - program did not change the cost of food
alternative hypothesis - program reduced the cost of food

$H_o : \mu_{program} \ge 151$

$H_A : \mu_{program} < 151$

So we are doing one-tailed t-test in -ve direction

If we take a sample of 25 people then $df = 24$
t-critical if $\alpha = 0.05$ = -1.711
SEM = $\frac{50}{\sqrt{25}} = \frac{50}{5} = 10$

After the program it was found that $\bar{X} = 126$ the mean difference = 126 - 151 = - 25

t-statisitc = (126 - 151) / 10 = -2.50

Margin of error is for two-tailed test always as we want to check for error on both sides
Margin of error = 2.064 * SEM = 2.064 - 20.64
CI = $126 \pm 20.64$