We covered a lot of information today and I'd like you to practice developing classification trees on your own. For each exercise, work through the problem, determine the result, and provide the requested interpretation in comments along with the code. The point is to build classifiers, not necessarily good classifiers (that will hopefully come later)

1. Load the iris dataset and create a holdout set that is 50% of the data (50% in training and 50% in test). Output the results (don't worry about creating the tree visual unless you'd like to) and discuss them briefly (are they good or not?)


In [1]:
#First, the libraries. And, make sure matplotlib shows up in jupyter notebook! hurrah
import pandas as pd
from sklearn import datasets
from pandas.tools.plotting import scatter_matrix
import matplotlib.pyplot as plt
%matplotlib inline

In [8]:
iris = datasets.load_iris()
x = iris.data[:,2:] # the attributes
y = iris.target # the target variable

from sklearn import tree

dt = tree.DecisionTreeClassifier()
dt = dt.fit(x,y)

In [2]:
from sklearn.cross_validation import train_test_split

In [94]:
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.5,train_size=0.5,random_state=42)

In [95]:
dt = dt.fit(x_train,y_train)

In [3]:
from sklearn import metrics
import numpy as np

In [4]:
def measure_performance(X,y,clf, show_accuracy=True, show_classification_report=True, show_confussion_matrix=True):
    y_pred=clf.predict(X)
    if show_accuracy:
        print("Accuracy:{0:.3f}".format(metrics.accuracy_score(y, y_pred)),"\n")
    if show_classification_report:
        print("Classification report")
        print(metrics.classification_report(y,y_pred),"\n")
    if show_confussion_matrix:
        print("Confusion matrix")
        print(metrics.confusion_matrix(y,y_pred),"\n")

In [100]:
measure_performance(x_test,y_test,dt)


Accuracy:0.974 

Classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00        15
          1       0.92      1.00      0.96        11
          2       1.00      0.92      0.96        12

avg / total       0.98      0.97      0.97        38
 

Confusion matrix
[[15  0  0]
 [ 0 11  0]
 [ 0  1 11]] 

2. Redo the model with a 75% - 25% training/test split and compare the results. Are they better or worse than before? Discuss why this may be.


In [99]:
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.25,train_size=0.75,random_state=42)
measure_performance(x_test,y_test,dt)


Accuracy:0.974 

Classification report
             precision    recall  f1-score   support

          0       1.00      1.00      1.00        15
          1       0.92      1.00      0.96        11
          2       1.00      0.92      0.96        12

avg / total       0.98      0.97      0.97        38
 

Confusion matrix
[[15  0  0]
 [ 0 11  0]
 [ 0  1 11]] 

3. Load the breast cancer dataset (datasets.load_breast_cancer()) and perform basic exploratory analysis. What attributes to we have? What are we trying to predict?

For context of the data, see the documentation here: https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29


In [5]:
bc=datasets.load_breast_cancer()

print(bc.keys())
print(bc['target_names'])
bc['DESCR']
print(bc['target'])
bc['data']
bc['feature_names']


dict_keys(['target_names', 'feature_names', 'DESCR', 'data', 'target'])
['malignant' 'benign']
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 1 0 0 0 0 0 0 0 0 1 0 1 1 1 1 1 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0
 1 0 1 0 0 1 1 1 0 0 1 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 1 1 1 1 0 1 1 0 1 1
 1 1 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 1 0 1 0 0 1 0 0 1 1 0 1 1 0 1 1 1 1 0 1
 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 0 1 1 1 1 0 1 1 0 0 0 1 0
 1 0 1 1 1 0 1 1 0 0 1 0 0 0 0 1 0 0 0 1 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 1 1
 1 0 1 1 1 1 1 0 0 1 1 0 1 1 0 0 1 0 1 1 1 1 0 1 1 1 1 1 0 1 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 1 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1
 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 1 1 1 1 0 0 0 1 1
 1 1 0 1 0 1 0 1 1 1 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 0 0
 0 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 0 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1
 1 0 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 0 1 1 1 1 1 0 1 1
 0 1 0 1 1 0 1 0 1 1 1 1 1 1 1 1 0 0 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 0 1
 1 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 1 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0
 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
 1 1 1 1 1 1 1 0 0 0 0 0 0 1]
Out[5]:
array(['mean radius', 'mean texture', 'mean perimeter', 'mean area',
       'mean smoothness', 'mean compactness', 'mean concavity',
       'mean concave points', 'mean symmetry', 'mean fractal dimension',
       'radius error', 'texture error', 'perimeter error', 'area error',
       'smoothness error', 'compactness error', 'concavity error',
       'concave points error', 'symmetry error', 'fractal dimension error',
       'worst radius', 'worst texture', 'worst perimeter', 'worst area',
       'worst smoothness', 'worst compactness', 'worst concavity',
       'worst concave points', 'worst symmetry', 'worst fractal dimension'], 
      dtype='<U23')

In [6]:
df = pd.DataFrame(bc.data, columns= bc.feature_names)
#Okay this does not work because in my data frame I only have the features, not the classes, no way to see the best predictors for the classes. :(

4. Using the breast cancer data, create a classifier to predict the type of seed. Perform the above hold out evaluation (50-50 and 75-25) and discuss the results.


In [9]:
x = bc.data[:,21:] # the attributes. I chose column 22 onward because... they have the word "worst" in them... :/
y = bc.target # the target variable. It has already been dummified. I guess. This dataset is unfriendly

dt = tree.DecisionTreeClassifier()
dt = dt.fit(x,y)

In [10]:
#With a 50/50 split
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.5,train_size=0.5,random_state=42)

In [11]:
dt = dt.fit(x_train,y_train)

In [12]:
measure_performance(x_test,y_test,dt)


Accuracy:0.930 

Classification report
             precision    recall  f1-score   support

          0       0.88      0.92      0.90        98
          1       0.96      0.94      0.95       187

avg / total       0.93      0.93      0.93       285
 

Confusion matrix
[[ 90   8]
 [ 12 175]] 


In [13]:
#With a 75/25 split
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.25,train_size=0.75,random_state=42)
dt = dt.fit(x_train,y_train)
measure_performance(x_test,y_test,dt)


Accuracy:0.951 

Classification report
             precision    recall  f1-score   support

          0       0.91      0.96      0.94        54
          1       0.98      0.94      0.96        89

avg / total       0.95      0.95      0.95       143
 

Confusion matrix
[[52  2]
 [ 5 84]] 


In [ ]: