This document describes the usage of a classification model to provide an explanation for a given prediction.
Model explanation provides the ability to interpret the effect of the predictors on the composition of an individual score. These predictors can then be ranked according to their contribution in the final score (leading to a positive or negative decision).
Model explanation has always been used in credit risk applications in presence of regulatory settings . The credit company is expected to give the customer the main (top n) reasons why the credit application was rejected (also known as reason codes).
Model explanation was also recently introduced by the European Union’s new General Data Protection Regulation (GDPR, https://arxiv.org/pdf/1606.08813.pdf) to add the possibility to control the increasing use of machine learning algorithms in routine decision-making processes.
The law will also effectively create a “right to explanation,” whereby a user can ask for an explanation of an algorithmic decision that was made about them.
The process we will use here is similar to LIME. The main difference is that LIME uses a data sampling around score value locally, while here we perform as full cross-statistics computation between the predictors and the score and use a local piece-wise linear approximation.
Here, we will use a sciki-learn classification model on a standard dataset (breast cancer detection model).
The dataset used contains 30 predictor variables (numerical features) and one binary target (dependant variable). For practical reasons, we will restrict our study to the first 4 predictors in this document.
In [1]:
from sklearn import datasets
import pandas as pd
%matplotlib inline
ds = datasets.load_breast_cancer();
NC = 4
lFeatures = ds.feature_names[0:NC]
df_orig = pd.DataFrame(ds.data[:,0:NC] , columns=lFeatures)
df_orig['TGT'] = ds.target
df_orig.sample(6, random_state=1960)
Out[1]:
For the classification task, we will build a ridge regression model, and train it on a part of the full dataset
In [2]:
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(n_estimators=120, random_state = 1960)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(df_orig[lFeatures].values,
df_orig['TGT'].values,
test_size=0.2,
random_state=1960)
df_train = pd.DataFrame(X_train , columns=lFeatures)
df_train['TGT'] = y_train
df_test = pd.DataFrame(X_test , columns=lFeatures)
df_test['TGT'] = y_test
clf.fit(X_train , y_train)
Out[2]:
In [3]:
# clf.predict_proba(df[lFeatures])[:,1]
The goal here is to be able, for a given individual, the impact of each predictor on the final score.
For our model, we will do this by analyzing cross statistics between (binned) predictors and the (binned) final score.
For each score bin, we fit a linear model locally and use it to explain the score. This is generalization of the linear case, based on the fact that any model can be approximated well enough locally be a linear function (inside each score_bin). The more score bins we use, the more data we have, the better the approximation is.
For a random forest , the score can be seen as the probability of the positive class.
In [4]:
from sklearn.linear_model import *
def create_score_stats(df, feature_bins = 4 , score_bins=30):
df_binned = df.copy()
df_binned['Score'] = clf.predict_proba(df[lFeatures].values)[:,0]
df_binned['Score_bin'] = pd.qcut(df_binned['Score'] , q=score_bins, labels=False, duplicates='drop')
df_binned['Score_bin_labels'] = pd.qcut(df_binned['Score'] , q=score_bins, labels=None, duplicates='drop')
for col in lFeatures:
df_binned[col + '_bin'] = pd.qcut(df[col] , feature_bins, labels=False, duplicates='drop')
binned_features = [col + '_bin' for col in lFeatures]
lInterpolated_Score= pd.Series(index=df_binned.index)
bin_classifiers = {}
coefficients = {}
intercepts = {}
for b in range(score_bins):
bin_clf = Ridge(random_state = 1960)
bin_indices = (df_binned['Score_bin'] == b)
# print("PER_BIN_INDICES" , b , bin_indexes)
bin_data = df_binned[bin_indices]
bin_X = bin_data[binned_features]
bin_y = bin_data['Score']
if(bin_y.shape[0] > 0):
bin_clf.fit(bin_X , bin_y)
bin_classifiers[b] = bin_clf
bin_coefficients = dict(zip(lFeatures, [bin_clf.coef_.ravel()[i] for i in range(len(lFeatures))]))
# print("PER_BIN_COEFFICIENTS" , b , bin_coefficients)
coefficients[b] = bin_coefficients
intercepts[b] = bin_clf.intercept_
predicted = bin_clf.predict(bin_X)
lInterpolated_Score[bin_indices] = predicted
df_binned['Score_interp'] = lInterpolated_Score
return (df_binned , bin_classifiers , coefficients, intercepts)
For simplicity, to describe our method, we use 5 score bins and 5 predictor bins.
We fit our local models on the training dataset, each model is fit on the values inside its score bin.
In [17]:
(df_cross_stats , per_bin_classifiers , per_bin_coefficients, per_bin_intercepts) = create_score_stats(df_train , feature_bins=5 , score_bins=10)
def debrief_score_bin_classifiers(bin_classifiers):
binned_features = [col + '_bin' for col in lFeatures]
score_classifiers_df = pd.DataFrame(index=(['intercept'] + list(binned_features)))
for (b, bin_clf) in per_bin_classifiers.items():
bin
score_classifiers_df['score_bin_' + str(b) + "_model"] = [bin_clf.intercept_] + list(bin_clf.coef_.ravel())
return score_classifiers_df
df = debrief_score_bin_classifiers(per_bin_classifiers)
df.head(10)
Out[17]:
From the table above, we see that lower score values (score_bin_0) are all around zero probability and are not impacted by the predictor values, higher score values (score_bin_5) are all around 1 and are also not impacted. This is what one expects from a good classification model.
in the score bin 3, the score values increase significantly with mean area_bin and decrease with mean radius_bin values.
Predictor effects describe the impact of specific predictor values on the final score. For example, some values of a predictor can increase or decrease the score locally by 0.10 or more points and change the negative decision to a positive one.
The predictor effect reflects how a specific predictor increases the score (above or below the mean local contribtution of this variable).
In [18]:
for col in lFeatures:
lcoef = df_cross_stats['Score_bin'].apply(lambda x : per_bin_coefficients.get(x).get(col))
lintercept = df_cross_stats['Score_bin'].apply(lambda x : per_bin_intercepts.get(x))
lContrib = lcoef * df_cross_stats[col + '_bin'] + lintercept/len(lFeatures)
df1 = pd.DataFrame();
df1['contrib'] = lContrib
df1['Score_bin'] = df_cross_stats['Score_bin']
lContribMeanDict = df1.groupby(['Score_bin'])['contrib'].mean().to_dict()
lContribMean = df1['Score_bin'].apply(lambda x : lContribMeanDict.get(x))
# print("CONTRIB_MEAN" , col, lContribMean)
df_cross_stats[col + '_Effect'] = lContrib - lContribMean
df_cross_stats.sample(6, random_state=1960)
Out[18]:
The previous sample, shows that the first individual lost 0.000000 score points due to the feature $X_1$, gained 0.003994 with the feature $X_2$, etc
In [19]:
import numpy as np
reason_codes = np.argsort(df_cross_stats[[col + '_Effect' for col in lFeatures]].values, axis=1)
df_rc = pd.DataFrame(reason_codes, columns=['reason_idx_' + str(NC-c) for c in range(NC)])
df_rc = df_rc[list(reversed(df_rc.columns))]
df_rc = pd.concat([df_cross_stats , df_rc] , axis=1)
for c in range(NC):
reason = df_rc['reason_idx_' + str(c+1)].apply(lambda x : lFeatures[x])
df_rc['reason_' + str(c+1)] = reason
# detailed_reason = df_rc['reason_idx_' + str(c+1)].apply(lambda x : lFeatures[x] + "_bin")
# df_rc['detailed_reason_' + str(c+1)] = df_rc[['reason_' + str(c+1) , ]]
df_rc.sample(6, random_state=1960)
Out[19]:
In [8]:
df_rc[['reason_' + str(NC-c) for c in range(NC)]].describe()
Out[8]:
This was an introductory document with a simple linear classifier. Deeper analysis can be made to extend this study
In [ ]: