The teller is a model-agnostic tool for Machine Learning (ML) explainability. For more details on this tool, you can visit the following introductory links: here, here, here or here.

In this post, we are going to use teller to examine model interactions. We’ll use the Boston Housing dataset, again. Here, by interactions, we mean: how does the response variable (variable to be explained) changes when both explanatory variable 1 increases of 1, and explanatory variable 2 increases of 1. On this specific dataset, it would mean for example: understanding how the median value of owner-occupied homes (in $1000’s) changes, when the index of accessibility to radial highways and the number of rooms per dwelling increase of 1 simultaneously.

Install the package and import data

Currently, the teller’s development version can be obtained from Github as:

!pip install git+https://github.com/Techtonique/teller.git

Model training and explanations

import numpy as np
from os import chdir

import teller as tr
import pandas as pd

from sklearn import datasets
import numpy as np      
from sklearn import datasets
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split


# import data
boston = datasets.load_boston()
X = np.delete(boston.data, 11, 1)
y = boston.target
col_names = np.append(np.delete(boston.feature_names, 11), 'MEDV')


# split  data into training and testing sets 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, 
                                                    random_state=123)
print(X_train.shape)
print(X_test.shape)


# fit a linear regression model 
regr = RandomForestRegressor(n_estimators=1000, random_state=123)
regr.fit(X_train, y_train)


# creating the explainer
expr = tr.Explainer(obj=regr)

Interactions with index of accessibility to radial highways (RAD):

varx = "RAD"
expr.fit(X_test, y_test, X_names=col_names[:-1], 
         y_name=col_names[-1], 
         col_inters = varx, method="inters")
print(expr.summary())

Interactions with RAD: 
            Estimate   Std. Error   95% lbound   95% ubound  Pr(>|t|)     
CRIM               0  2.22045e-16  4.40477e-16 -4.40477e-16         1    -
ZN      -3.37992e-11  2.22045e-16 -3.37988e-11 -3.37997e-11         0  ***
INDUS   -7.06339e-11  4.33823e-11  1.54248e-11 -1.56693e-10  0.106603    -
CHAS               0  2.22045e-16  4.40477e-16 -4.40477e-16         1    -
NOX     -3.80587e-10  4.33823e-10  4.80001e-10 -1.24117e-09  0.382413    -
RM           4.05396  3.23892e-11      4.05396      4.05396         0  ***
AGE      4.08624e-12  3.47213e-12   1.0974e-11 -2.80153e-12  0.242015    -
DIS                0  2.22045e-16  4.40477e-16 -4.40477e-16         1    -
RAD                0  2.22045e-16  4.40477e-16 -4.40477e-16         1    -
TAX        -0.032699  2.22045e-16    -0.032699    -0.032699         0  ***
PTRATIO    -0.889095     0.889095      0.87463     -2.65282    0.3197    -
LSTAT    5.03253e-11  2.22045e-16  5.03258e-11  5.03249e-11         0  ***

Interactions with average number of rooms per dwelling (RM):

varx = "RM"
expr.fit(X_test, y_test, X_names=col_names[:-1], 
         y_name=col_names[-1], 
         col_inters = varx, method="inters")
print(expr.summary())

Interactions with RM: 
            Estimate   Std. Error   95% lbound   95% ubound      Pr(>|t|)     
CRIM               0  2.22045e-16  4.40477e-16 -4.40477e-16             1    -
ZN      -7.41298e-12  2.22045e-16 -7.41254e-12 -7.41342e-12             0  ***
INDUS   -3.79269e-11    1.963e-11  1.01374e-12 -7.68675e-11     0.0561504    .
CHAS               0  2.22045e-16  4.40477e-16 -4.40477e-16             1    -
NOX     -3.13992e-10  2.12499e-10  1.07549e-10 -7.35532e-10      0.142622    -
RM                 0  2.22045e-16  4.40477e-16 -4.40477e-16             1    -
AGE        -0.521402  1.65828e-12    -0.521402    -0.521402             0  ***
DIS       5.2504e-11  2.22045e-16  5.25044e-11  5.25036e-11             0  ***
RAD          4.05396  3.23892e-11      4.05396      4.05396             0  ***
TAX      9.58314e-13  2.22045e-16  9.58755e-13  9.57874e-13  9.44012e-268  ***
PTRATIO  6.77272e-12  7.06388e-12  2.07856e-11 -7.24011e-12      0.339958    -
LSTAT        2.70129  3.97481e-11      2.70129      2.70129             0  ***

A notebook containing these results can be found here. Contributions/remarks are welcome as usual, you can submit a pull request on Github.

Note: I am currently looking for a gig. You can hire me on Malt or send me an email: thierry dot moudiki at pm dot me. I can do descriptive statistics, data preparation, feature engineering, model calibration, training and validation, and model outputs’ interpretation. I am fluent in Python, R, SQL, Microsoft Excel, Visual Basic (among others) and French. My résumé? Here!

Comments powered by Talkyard.