The mean normal body temperature was held to be 37$^{\circ}$C or 98.6$^{\circ}$F for more than 120 years since it was first conceptualized and reported by Carl Wunderlich in a famous 1868 book. In 1992, this value was revised to 36.8$^{\circ}$C or 98.2$^{\circ}$F.
In this exercise, you will analyze a dataset of human body temperatures and employ the concepts of hypothesis testing, confidence intervals, and statistical significance.
Answer the following questions in this notebook below and submit to your Github account.
You can include written notes in notebook cells using Markdown:
In [127]:
%matplotlib inline
import pandas as pd
import numpy as np
import scipy.stats as st
from scipy.stats import norm
import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
from IPython.core.display import HTML
css = open('style-table.css').read() + open('style-notebook.css').read()
HTML('<style>{}</style>'.format(css))
Out[127]:
In [15]:
bodytemp_df = pd.read_csv('data/human_body_temperature.csv')
bodytemp_df
Out[15]:
So, we see that we have 130 data points to work with. First, we want to take a look at the overall distribution.
In [19]:
sns.distplot(bodytemp_df.temperature, bins = 25)
Out[19]:
In [116]:
st.normaltest(df['temperature'])
Out[116]:
We see that our sample distribution does look like a normal distribution, albeit slightly left skewed. Nonetheless, we feel that it is reasonable to assume the CLT holds for this data. We see from our normaltest that the p-value returned is quite high, 25%. So we cannot reject the null hypothesis of this sample coming from a normal distribution. Thus both a visual inspection and a more rigorous computational one lets us conclude that the popluation is normally distributed in this case.
Now, we put forth the hypothesis that the true population mean is 98.6. To try and check this, we first require the sample mean, and sample standard deviation. Note that the pandas DataFrame.std method normalizes by N-1 by default.
In [32]:
hyp_mean = 98.6
sample_meantemp = bodytemp_df['temperature'].mean()
sample_std = bodytemp_df['temperature'].std()
print('The sample mean is : ' , bodytemp_df['temperature'].mean(), ' degrees Farenheit')
print('The sample standard deviation is : ' , bodytemp_df['temperature'].std(), ' degrees Farenheit')
Since we have a 'large' sample size, we can estimate the population standard deviation, and population mean as being equal to the sample standard deviation and the sample mean. We use the sample standard deviation and sample size to obtain our best estimate of the standard error of the mean.
In [30]:
sem_temp = sample_std/np.sqrt(len(bodytemp_df))
sem_temp
Out[30]:
In [104]:
sample_std/np.sqrt(130)
Out[104]:
In [66]:
z_score = (sample_meantemp - hyp_mean) / (sem_temp)
z_score
Out[66]:
In [68]:
p_value=st.norm.cdf(z_score)
p_value
Out[68]:
In [81]:
new_hyp = 98.2
z_score_new = (sample_meantemp - new_hyp)/ (sem_temp)
print(z_score_new)
p_value_new = 1-st.norm.cdf(z_score_new)
p_value_new
Out[81]:
Thus from our hypothesis that our sample population mean is incorrect, and that the real mean is 98.6 degrees seems decidedly unlikely. We are confident that the probablity of finding a value at least as low in our sample population (more than 5 stds below the mean!), is only $2.45\times10^{-8}$. Thus we choose to reject the original hypothesis that the population mean is 98.6 degrees, based on the available data. Instead we shall accept the new value of 98.2, which as seen above, is well within one standard deviation of the sample mean.
To find the normal range of human body temperatures, we need a confidence interval. Let us use the usual 95% condifence interval as our threshold. That is, we will be resonably confident that there is a 95% the true popluation mean is within our confidence interval.
In [93]:
z_critical = st.norm.ppf(.975)
conf_int = z_critical*sem_temp
print ('margin of error: ', conf_int)
print('upper limit of normal: ', sample_meantemp + conf_int)
print('lower limit of normal: ', sample_meantemp - conf_int)
So if human body temperature is outside of the range given above, then we are reasonably sure that the temperature is abnormal, as our range should encompass 95% of the population.
Now we move on to testing if there is a significant differnce between males and females.
In [98]:
female_df = bodytemp_df[bodytemp_df.gender == 'F'].copy()
male_df = bodytemp_df[bodytemp_df.gender == 'M'].copy()
In [101]:
male_mean = male_df['temperature'].mean()
print('Male mean is: ', male_mean)
female_mean = female_df['temperature'].mean()
print('Female mean is: ', female_mean)
male_std = male_df['temperature'].std()
print('Male standard deviation is: ', male_std)
female_std = female_df['temperature'].std()
print('Female standard deviation is: ', female_std)
In [112]:
difference_mean = female_mean - male_mean
print('Mean difference between two populations: ', difference_mean)
difference_sem = np.sqrt(male_std**2/len(male_df) + female_std**2/len(female_df))
print( 'Standard error of the mean: ', difference_sem)
Now that we have our population mean difference, as well as the standard error of this mean, we can go ahead and look up p-values for our data, and compare it to some threshold. Let's use the standard 5% threshold.
In [131]:
z_score_diff = (difference_mean - 0)/difference_sem
z_score_diff
Out[131]:
In [133]:
p_value_diff = 1-st.norm.cdf(z_score_diff)
p_value_diff
Out[133]:
Thus since our p_value is about 1%, well below the 5% threshold we had decided on, we conclude that there does seem to be a significant difference between men and women in terms of body temperature. What this p-value represents is that if the null hypothesis was true (no difference), then we would only expect to sample such a large difference 1 out of 100 times, based on chance. So we are confident that the observed effect is real.
In [ ]: