In [2]:
# Imports
import pandas as pd
import numpy as np
import seaborn as sns
from matplotlib import pyplot as plt
from scipy.stats import ttest_rel, norm
In [3]:
# Read in data
df = pd.read_csv('stroopdata.csv')
Null hypothesis: the reaction time for the congruent and incongruent conditions is the same.
$$\mu(\text{congruent}) = \mu(\text{incongruent})$$Alternative hypothesis: the reaction time for the two conditons are different.
$$\mu(\text{congruent}) \neq \mu(\text{incongruent})$$It's hard to imagine the reverse being true; that the reaction time would be faster for the incongruent condition. On the other hand, it is not a priori completely implausible that the reaction times would be the same.
We will use the paired t-test to test whether the mean reaction times under the two conditions are equal or not. The t-test allows finding out whether the means of two sets of data are different.
We believe the use of a paired t-test is appropriate since the following conditions are met (see [1], Analysis checklist: Paired t test):
As will be shown graphically in a later section, the differences are distributed close to normally.
The pairing was part of the design of the experiment; each pair represents trials from a single participant. This is an effective pairing method.
Each pair corresponds to trials by a different participant; they are independent.
Yes.
We did not choose a one-tail P value (although we could have).
We expect the control values and differences to be of the same order of magnitude, so differences should be fine.
In [4]:
IQR_congruent = df['Congruent'].quantile(0.75) - df['Congruent'].quantile(0.25)
IQR_incongruent = df['Incongruent'].quantile(0.75) - df['Incongruent'].quantile(0.25)
print(df['Congruent'].median(), df['Incongruent'].median(), IQR_congruent, IQR_incongruent)
In [5]:
%matplotlib inline
plt.plot(df['Incongruent']-df['Congruent'], 'o')
plt.title('Difference Between Congruent and \n Incongruent Conditions on Stroop Task')
plt.xlabel('Participant')
plt.ylabel('Reaction time');
Notice that not a single participant performed better on the incongruent task than on the congruent task (all differences are above zero).
In [60]:
plt.plot(x,norm.pdf(x, 7.3, 5), 'g-', label="N(7.3, 5)")
sns.kdeplot(df['Incongruent']-df['Congruent'], label="KDE")
plt.title('Kernel Density Estimate of the Distribution\nof Differences Between the Two Conditions');
x = np.linspace(-10, 30, 1000)
plt.legend();
At a glance, the difference distribution looks roughly normally-distributed (a manually fitted normal distribution is plotted for comparison), justifying the use of a paired t-test.
In [62]:
ttest_rel(df['Congruent'], df['Incongruent'])
Out[62]:
The analysis resulted in a t statistic of -8.021, with a p-value of 4.103e-8 <<< 0.001.
We confidently reject the null hypothesis (that the means for the two conditions is the same). The p-value indicates roughly that we have only 4 in 100,000,000 chances of seeing a difference at least this extreme if the null hypothesis is true, which is roughly equal to the chance that you will die from being bitten or stung by an insect in the next year. [2]
The result matches our expectations, outlined in the hypothesis section.