(From Abdi, Edelman, Valentin & Dowling, 2009; Williams, Krishnan, & Abdi, 2009)
In the 1970s Elizabeth Loftus (Loftus and Palmer, 1974) conducted a series of experiments on the theme of eyewitness testimony. They wanted to find out whether the wording of a question affected the later responses of witnesses. To do this she showed subjects a film of a car accident. Following the film, she asked them a series of questions. Among the questions was one of 5 versions of a critical question concerning the speed the vehicles were travelling in miles per hour. Here are the 5 versions of the question along with the experimental condition in which they occurred:
Software Carpentry conducted a (ficticious) replication of the study in an online survey. The results are in the file loftus.csv.
We want to be able to answer the question: Does the wording of the question affect witnesses' estimation of the speed the vehicles were traveling?
Get the file loftus.csv from the class git repo (https://github.com/LJWilliams/swc_BerkeleyDLab)
Type:
git clone https://github.com/LJWilliams/swc_BerkeleyDLab
into the shell
In [25]:
datafile = open('loftus.csv', 'rU')
for loop
In [26]:
data = []
for row in datafile:
data.append(row.strip().split(','))
In [27]:
datafile.close()
Remember to document your code and call your function something memorable so you will remember what you have done later.
In [28]:
def open_csv_file(filename):
'''(str) -> str
open_csv_file opens a csv file called filename and
returns the information in the datafile in python
readable format
example:
open_csv_file('loftus.csv')
'''
# Open the datafile
datafile = open(filename, 'rU')
# Initialize an empty array
data = []
# Create data array
for row in datafile:
data.append(row.strip().split(','))
datafile.close()
return data
data = open_csv_file('loftus.csv')
In [29]:
def open_csv_file(filename):
'''(str) -> str
open_csv_file opens a csv file called filename and
returns the information in the datafile in python
readable format. Note that the first row is ALWAYS
considered to be a header row!
example:
open_csv_file('loftus.csv')
'''
# Open the datafile
datafile = open(filename, 'rU')
# Initialize an empty array
data = []
# define counter start number
rownum = 0
for row in datafile:
if rownum == 0:
# Create header
header = row
else:
# Create data array
data.append(row.strip().split(','))
rownum += 1
datafile.close()
return data, header
But this gets dangerous when we want to reuse our code because a lot of the time (unless it is our own data) you don't know if the data has a header row
Import the csv module .
To save processing time, import your modules at the beginning of your .py file. Here we are importing them as we need them for explanation purposes only.
In [30]:
import csv
In [31]:
def open_csv_file(filename):
'''(str) -> str
open_csv_file opens a csv file called filename and
returns the information in the datafile in python
readable format. It uses the csv module to determine
whether or not there is a row of column headers
example:
open_csv_file('loftus.csv')
'''
# Open the datafile
datafile = open(filename, 'rU')
# Find out whether there is a header row
hasheader = csv.Sniffer().has_header(datafile.read(1024))
datafile.seek(0)
data = []
header = []
rownum = 0
if hasheader == True:
for row in datafile:
if rownum == 0:
header = row.strip().split(',')
print 'Data file has a header row:'
print header
else:
data.append(row.strip().split(','))
rownum += 1
else:
for row in datafile:
if rownum == 0:
print 'Data file does not have a header row'
data.append(row.strip().split(','))
rownum += 1
datafile.close()
return(data, header)
data, header = open_csv_file('loftus.csv')
Now check that the output of the function makes sense.
In [32]:
header
Out[32]:
In [33]:
data[0:10]
Out[33]:
It looks like the csv module got it correct.
In [34]:
type(header)
Out[34]:
Now the data:
In [35]:
type(data)
Out[35]:
In [36]:
len(header)
Out[36]:
Now let's do the same for data.
First, let's see how many observations are in the list
In [37]:
len(data)
Out[37]:
Now let's check that each list element has a list with 7 elements in it (to go with our headers). We might want to do this again, so let's write this as a function
In [38]:
def elements_in_row(data,header):
'''(list) -> str
Elements in row
'''
h = len(header)
rownum = 0
for row in data:
r = len(row)
if r != h:
print 'Row ' + str(rownum) + ' does not have ' + str(h) + ' elements.'
print row
rownum += 1
return 'All done! All rows have ' + str(h) + ' elements.'
elements_in_row(data,header)
Out[38]:
In [39]:
colspeed = header.index('Estimated Speed (mph)')
print colspeed
In [40]:
colcondname = header.index('Condition Name')
print colcondname
In [41]:
speed = [x[colspeed] for x in data]
print speed[0:10]
In [42]:
condname = [x[colcondname] for x in data]
print condname[0:10]
In [43]:
def get_condition_names(condname):
''' list(str) -> list(str)
get_condition_names takes a list of strings and returns
a list of the unique strings contained within the
original list.
>>> get_condition_names(condname)
['Smash', 'Hit', 'Bump', 'Collide', 'Contact', 'Sash']
'''
conditions = list(set(condname))
return conditions
conditions = get_condition_names(condname)
print conditions
And now we can use len to get the number of different conditions.
In [44]:
ncond = len(conditions)
print ncond
Not so good. We have a listing for 6 conditions, when we really have 5.
HIT: About how fast were the cars going when they hit each other?
SMASH: About how fast were the cars going when they smashed into each other?
COLLIDE: About how fast were the cars going when they collided with each other?
BUMP: About how fast were the cars going when they bumped each other?
CONTACT: About how fast were the cars going when they contacted each other?
It looks like somebody mistyped the label for the Smash condition. We need to fix this before we go on.
We need to check whether our hypothesis is correct. Let's count the number of instances of each condition.
In [45]:
def count_observations_in_condition(condname):
''' list(str) list(str) -> int
count_observations_in condition takes a list of unique strings and
returns the number of observations per string
Requires:
get_condition_names
Example:
>>> count_observations_in_condition(condname):
Smash 5000
Hit 5000
Bump 5000
Collide 5000
Contact 5000
'''
conditions = get_condition_names(condname)
idx = 0
indices = []
for cond in conditions:
indices.append([i for i, x in enumerate(condname) if x == cond])
print cond + '\t' + str(len(indices[idx]))
idx += 1
count_observations_in_condition(condname)
It looks like we were correct. Let's replace the elements Sash with Smash.
In this case it is fairly easy because we have a balanced design, but watch out if you don't have an equal number of observations in each condition.
In [46]:
def change_condition_name(Incorrect_Name,Correct_Name,Vector_of_Conditions):
''' str str list(str) -> list(str)
change_codition_name takes 2 strings (the incorrect and correct
condition names) and a list of strings and returns a list of
strings with the incorrect name replaced by the correct name.
example:
condname = change_condition_name('Sash','Smash',condname)
where, condname is a list of strings.
'''
for idx, item in enumerate(Vector_of_Conditions):
if item == Incorrect_Name:
Vector_of_Conditions[idx] = Correct_Name
print "Element " + str(idx) + " (" + str(item) + ") replaced with " + Correct_Name
return Vector_of_Conditions
condname = change_condition_name('Sash','Smash',condname)
And let's check our numbers again.
In [47]:
count_observations_in_condition(condname)
In [48]:
type(speed)
Out[48]:
Now the elements:
In [49]:
def get_type_elements(variable):
''' list -> str
get_type_elements takes a list of numbers and prints
the type of each variable to stdout
example:
get_type_elements(speed)
'''
count = 0
format = []
for item in variable:
format.append(type(variable[count]))
count += 1
print list(set(format))
get_type_elements(speed)
Oops. The data is in the wrong format. We need a list of numbers, not strings. We'll use floating point numbers to accomodate any observations that have decimal points in them. You could check this too, if you wanted.
In [50]:
def change_data_format_to_float(variable):
''' list(str) -> list(float)
change_data_format_to_float takes a list of type string
and returns numbers in type float
example:
speedfloat = change_data_format_to_float(speed)
'''
count = 0
variablefloat=[]
for item in variable:
variablefloat.append(float(variable[count]))
count += 1
return variablefloat
speedfloat = change_data_format_to_float(speed)
get_type_elements(speedfloat)
In [51]:
def create_dictionary(condname):
datadict = dict()
count = 0
for name in condname:
if name in datadict:
# append the new number to the existing array at this slot
datadict[name].append(speedfloat[count])
else:
# create a new array in this slot
datadict[name] = [speedfloat[count]]
count += 1
return datadict
datadict = create_dictionary(condname)
In [52]:
import pandas as pd
import numpy as np
from scipy import stats
from patsy import dmatrices
Now, we can turn our dictionary into a pandas dataframe.
In [53]:
df = pd.DataFrame(datadict)
print df
In [54]:
print df.head()
In [55]:
print df.tail()
In [56]:
condmean = df.mean()
print condmean
In [57]:
condvar = df.var()
print condvar
In [58]:
condstd = df.std()
print condstd
Or, we can get them all together:
In [59]:
descriptives = df.describe()
print descriptives
In [60]:
%pylab inline
from pylab import *
Now, let's plot our means
In [61]:
df.mean().plot(kind='bar')
Out[61]:
and take a look at the boxplot
In [62]:
plt.figure();
bp = df.boxplot()
First, let's specify our conditions. Let's start with Bump. Because we have the data in a pandas dataframe, we can specify our conditions by name.
In [63]:
stats.normaltest(df)
Out[63]:
The first row returns the k2 value (skew^2 + kurtosis^2) and the second row are the respective p values. We are lucky. None of our conditions are skewed or violate the assumption of normality.
In [64]:
F_val, p_val = stats.f_oneway(df["Bump"],df["Collide"],df["Contact"],df["Hit"],df["Smash"])
print "One-way ANOVA F =", F_val
print "One-way ANOVA p =", p_val
So, we now know that there is a significant difference in estimated speed of the vehicles based on the wording of the question. Now, which conditions drive this finding?
In [65]:
t_bcol,p_bcol = stats.ttest_ind(df["Bump"], df["Collide"])
print "The t value for Bump vs. Collide is = ", t_bcol, ", p =", p_bcol
In [66]:
def printtvals(df,cond1,cond2):
'''PRINTVALS takes a pandas dataframe and 2 named
conditions and returns the t and p values printed
to screen.
USAGE: printtvals(df,cond1,cond2)
where,
df is a pandas dataframe
cond1 is a named condition in df
cond2 is a named condition in df
'''
t, p = stats.ttest_ind(df[cond1],df[cond2])
print "The t value for ", cond1, "vs. ", cond2, "is =\t", t, ",\t p =", p
In [67]:
printtvals(df,"Bump","Collide")
printtvals(df,"Bump","Contact")
printtvals(df,"Bump","Hit")
printtvals(df,"Bump","Smash")
printtvals(df,"Collide","Contact")
printtvals(df,"Collide","Hit")
printtvals(df,"Collide","Smash")
printtvals(df,"Contact","Hit")
printtvals(df,"Contact","Smash")
printtvals(df,"Hit","Smash")
So, we can see that all conditions differ significantly from each other. However, the greatest difference is between the Contact and Smash conditions.
If you would like to have the full data output or have a more complex design you will need to use the functions built into the statsmodels module. statsmodels allows you to use R-like syntax to explicitly create your model. We have included the analysis from above for comparison of methods.
Download and install the statsmodels 0.6 module. First clone the repository from github from git clone git://github.com/statsmodels/statsmodels.git. You will also need to install patsy. Using pip enter pip install patsy at the command line. Everything else should have come with your installation of Anaconda. Following the installation of patsy, navigate to the directory where you put the statsmodels repository. Once there, type python setup.py build and finally python setup.py install.
The statsmodels module has many useful statistical features that will benefit your work. The latest release allows you to build statistical models using R-like syntax, but with the readability of python. Documentation is available from http://statsmodels.sourceforge.net/.
In [68]:
#import statsmodels.api as sm
import statsmodels.formula.api as sm
from statsmodels.stats.anova import anova_lm
import statsmodels.sandbox as sand
from statsmodels.stats.multicomp import (pairwise_tukeyhsd,
MultiComparison)
Although the format of df is great for getting descriptive statistics, we need a different format for running the ANOVA (corresponding vectors of data).
In [69]:
df_long = pd.DataFrame(zip(condname,speedfloat), columns=['condname', 'speed'])
df_long.head()
Out[69]:
In [70]:
df_long.tail()
Out[70]:
Now to actually run the ANOVA. Just as a reminder (for those whose stats are a bit rusty) the an ANOVA is just a special case of OLS regression, so we will run the regression model here.
Let's set up the model:
In [71]:
model = sm.ols('speed ~ C(condname)',df_long)
where, speed is the dependent variable, C() means that condname is a categorical variable, and condname is the independent variable.
Now, we can run the analysis.
In [72]:
result = model.fit()
In [73]:
print result.summary()
Note that the result object has many useful attributes. For example, we can extract parameter estimates and r-squared
In [74]:
result.params
Out[74]:
In [75]:
result.rsquared
Out[75]:
To present the results as a traditional ANOVA table
In [76]:
anova_lm(result)
Out[76]:
Now to run the post-hocs.
In [77]:
conditions = pd.Categorical.from_array(df_long['condname'])
#print conditions
In [78]:
print conditions.levels
In [79]:
res2 = pairwise_tukeyhsd(df_long['speed'], df_long['condname'])
print res2
In [80]:
mc = MultiComparison(df_long['speed'],df_long['condname'])
In [81]:
posthoc = mc.tukeyhsd()
print posthoc.summary()
print zip(list((conditions.levels)), range(5))