Examining racial discrimination in the US job market

Background

Racial discrimination continues to be pervasive in cultures throughout the world. Researchers examined the level of racial discrimination in the United States labor market by randomly assigning identical résumés black-sounding or white-sounding names and observing the impact on requests for interviews from employers.

Data

In the dataset provided, each row represents a resume. The 'race' column has two values, 'b' and 'w', indicating black-sounding and white-sounding. The column 'call' has two values, 1 and 0, indicating whether the resume received a call from employers or not.

Note that the 'b' and 'w' values in race are assigned randomly to the resumes.

Exercise

You will perform a statistical analysis to establish whether race has a significant impact on the rate of callbacks for resumes.

Answer the following questions in this notebook below and submit to your Github account.

What test is appropriate for this problem? Does CLT apply?
What are the null and alternate hypotheses?
Compute margin of error, confidence interval, and p-value.
Discuss statistical significance.

You can include written notes in notebook cells using Markdown:

In the control panel at the top, choose Cell > Cell Type > Markdown
Markdown syntax: http://nestacms.com/docs/creating-content/markdown-cheat-sheet

Resources

Experiment information and data source: http://www.povertyactionlab.org/evaluation/discrimination-job-market-united-states
Scipy statistical methods: http://docs.scipy.org/doc/scipy/reference/stats.html
Markdown syntax: http://nestacms.com/docs/creating-content/markdown-cheat-sheet



In [2]:

    
import pandas as pd
import numpy as np
from scipy import stats



In [3]:

    
from math import sqrt



In [4]:

    
data = pd.io.stata.read_stata('data/us_job_market_discrimination.dta')



In [6]:

    
black=data[data.race=='b']
white=data[data.race=='w']

We will be looking at the difference in callback rate proportions for black and white sounding applicants. The data is clearly independent, and as long as the quality of the resumes was random, the data will be drawn from the same distribution, so we expect the proportion of the data to be drawn from a nearly-normal distribution.

Our null hypothosis is that there is no difference between the callback rate between black & white applicants, while our alternative hypothosis is that the callback rate for white applicants is higher than for black.



In [11]:

    
print(black.call.count())
print(white.call.count())



In [10]:

    
SE=sqrt(data['call'].mean()*(1-data['call'].mean())*(1.0/black.shape[0]+1.0/white.shape[0]))



In [12]:

    
dc=white['call'].mean()-black['call'].mean()
(1-stats.t.cdf(dc/SE,black.call.count()-1))









    Out[12]:





2.0577751404737832e-05

Using a 5% threshold, the null hypothosis is clearly rejected, as we have a p-value of 0.002%, meaning there is virtually zero probability of seeing this or a greater difference between white and black call-back rates if the null hypothosis was true.



In [16]:

    
(1-stats.norm.cdf(dc/SE))









    Out[16]:





1.991943452495093e-05



In [17]:

    
SEci=sqrt((1-white['call'].mean())*white['call'].mean()/white['call'].count()+(1-black['call'].mean())*black['call'].mean()/black['call'].count())
SEci









    Out[17]:





0.00778337058606343



In [18]:

    
stats.t.ppf(.95,black.call.count()-1)*SEci









    Out[18]:





0.012807379867044798



In [22]:

    
dc









    Out[22]:





0.032032855

We see that the callback rate is $0.032 \pm 0.013$ more for white names than black ones, with a 95% confidence interval. As we can see from our p-value of 0.002%, these results are highly statistically significant.



In [ ]: