https://archive.ics.uci.edu/ml/datasets/pima+indians+diabetes
Title: Pima Indians Diabetes Database
Sources: (a) Original owners: National Institute of Diabetes and Digestive and
Kidney Diseases
(b) Donor of database: Vincent Sigillito (vgs@aplcen.apl.jhu.edu)
Research Center, RMI Group Leader
Applied Physics Laboratory
The Johns Hopkins University
Johns Hopkins Road
Laurel, MD 20707
(301) 953-6231
(c) Date received: 9 May 1990
Past Usage:
Smith,~J.~W., Everhart,~J.~E., Dickson,~W.~C., Knowler,~W.~C., \& Johannes,~R.~S. (1988). Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In {\it Proceedings of the Symposium on Computer Applications and Medical Care} (pp. 261--265). IEEE Computer Society Press.
The diagnostic, binary-valued variable investigated is whether the patient shows signs of diabetes according to World Health Organization criteria (i.e., if the 2 hour post-load plasma glucose was at least 200 mg/dl at any survey examination or if found during routine medical care). The population lives near Phoenix, Arizona, USA.
Results: Their ADAP algorithm makes a real-valued prediction between 0 and 1. This was transformed into a binary decision using a cutoff of 0.448. Using 576 training instances, the sensitivity and specificity of their algorithm was 76% on the remaining 192 instances.
Relevant Information: Several constraints were placed on the selection of these instances from a larger database. In particular, all patients here are females at least 21 years old of Pima Indian heritage. ADAP is an adaptive learning routine that generates and executes digital analogs of perceptron-like devices. It is a unique algorithm; see the paper for details.
Number of Instances: 768
Number of Attributes: 8 plus class
Missing Attribute Values: Yes
Class Distribution: (class value 1 is interpreted as "tested positive for diabetes")
Class Value Number of instances 0 500 1 268
Brief statistical analysis:
Attribute number: Mean: Standard Deviation:
1. 3.8 3.4
2. 120.9 32.0
3. 69.1 19.4
4. 20.5 16.0
5. 79.8 115.2
6. 32.0 7.9
7. 0.5 0.3
8. 33.2 11.8
In [1]:
import pandas as pd
In [2]:
diabetes = pd.read_csv('data/pima-indians-diabetes.csv')
In [3]:
diabetes.head()
Out[3]:
In [4]:
diabetes.columns
Out[4]:
In [5]:
# Columns that will be normalized
cols_to_norm = ['Number_pregnant', 'Glucose_concentration', 'Blood_pressure', 'Triceps',
'Insulin', 'BMI', 'Pedigree']
In [6]:
# Normalizing the columns
diabetes[cols_to_norm] = diabetes[cols_to_norm].apply(lambda x: (x - x.min()) / (x.max() - x.min()))
In [7]:
diabetes.head()
Out[7]:
In [8]:
diabetes.columns
Out[8]:
In [9]:
import tensorflow as tf
In [10]:
num_preg = tf.feature_column.numeric_column('Number_pregnant')
plasma_gluc = tf.feature_column.numeric_column('Glucose_concentration')
dias_press = tf.feature_column.numeric_column('Blood_pressure')
tricep = tf.feature_column.numeric_column('Triceps')
insulin = tf.feature_column.numeric_column('Insulin')
bmi = tf.feature_column.numeric_column('BMI')
diabetes_pedigree = tf.feature_column.numeric_column('Pedigree')
age = tf.feature_column.numeric_column('Age')
If you know the set of all possible feature values of a column and there are only a few of them, you can use categorical_column_with_vocabulary_list. If you don't know the set of possible values in advance you can use categorical_column_with_hash_bucket
In [11]:
assigned_group = tf.feature_column.categorical_column_with_vocabulary_list('Group',['A','B','C','D'])
# Alternative
# assigned_group = tf.feature_column.categorical_column_with_hash_bucket('Group', hash_bucket_size=10)
In [12]:
import matplotlib.pyplot as plt
%matplotlib inline
In [13]:
diabetes['Age'].hist(bins = 20)
Out[13]:
In [14]:
age_buckets = tf.feature_column.bucketized_column(age, boundaries=[20, 30, 40, 50, 60, 70, 80])
In [15]:
feat_cols = [num_preg, plasma_gluc, dias_press, tricep, insulin,
bmi, diabetes_pedigree, assigned_group, age_buckets]
In [16]:
diabetes.head()
Out[16]:
In [17]:
diabetes.info()
In [18]:
# Dropping 'Class' to exclude the column
x_data = diabetes.drop('Class',axis = 1)
In [19]:
labels = diabetes['Class']
In [20]:
from sklearn.model_selection import train_test_split
In [21]:
# Test train split
X_train, X_test, y_train, y_test = train_test_split(x_data,
labels,
test_size = 0.33,
random_state = 101)
In [22]:
input_func = tf.estimator.inputs.pandas_input_fn(x = X_train,
y = y_train,
batch_size = 10,
num_epochs = 1000,
shuffle = True)
In [23]:
model = tf.estimator.LinearClassifier(feature_columns = feat_cols,
n_classes = 2)
In [24]:
model.train(input_fn = input_func,
steps = 1000)
Out[24]:
In [25]:
# Useful link for your own data
# https://stackoverflow.com/questions/44664285/what-are-the-contraints-for-tensorflow-scope-names
In [26]:
eval_input_func = tf.estimator.inputs.pandas_input_fn(
x = X_test,
y = y_test,
batch_size = 10,
num_epochs = 1,
shuffle = False)
In [27]:
results = model.evaluate(eval_input_func)
In [28]:
results
Out[28]:
In [29]:
pred_input_func = tf.estimator.inputs.pandas_input_fn(
x = X_test,
batch_size = 10,
num_epochs = 1,
shuffle = False)
In [30]:
# Predictions is a generator!
predictions = model.predict(pred_input_func)
In [31]:
list(predictions)[0:5]
Out[31]:
In [32]:
dnn_model = tf.estimator.DNNClassifier(hidden_units=[10, 10, 10],
feature_columns = feat_cols,
n_classes = 2)
In [33]:
# Creating an embedding columns with 4 groups (A, B, C, D)
embedded_group_column = tf.feature_column.embedding_column(assigned_group,
dimension = 4)
In [34]:
feat_cols = [num_preg, plasma_gluc, dias_press, tricep, insulin,
bmi, diabetes_pedigree, embedded_group_column, age_buckets]
In [35]:
input_func = tf.estimator.inputs.pandas_input_fn(x = X_train,
y = y_train,
batch_size = 10,
num_epochs = 1000,
shuffle = True)
In [36]:
dnn_model = tf.estimator.DNNClassifier(hidden_units=[10, 10, 10],
feature_columns = feat_cols,
n_classes = 2)
In [37]:
dnn_model.train(input_fn = input_func,
steps = 1000)
Out[37]:
In [38]:
eval_input_func = tf.estimator.inputs.pandas_input_fn(
x = X_test,
y = y_test,
batch_size = 10,
num_epochs = 1,
shuffle = False)
In [39]:
dnn_model.evaluate(eval_input_func)
Out[39]: