Tests for the significance of marginal effects in the teller

08 Nov 2019 | RSS | Back to list of posts | Toggle dark mode

Follow me on LinkedIn

1

1

1

1

---

Last week, I introduced the teller, a model-agnostic tool for ML explainability. The teller’s philosophy is to rely on Taylor series to explain ML models predictions: a little increase in model’s explanatory variables + a little decrease, and we can obtain approximate sensitivities of its predictions to changes in these explanatory variables.

This new version of the package (v0.2.0) improves the interface, and introduces Jackknife Student t-tests for the significance of marginal effects. For these tests, you’ll need to have at least 30 observations in the testing set.

Installation

Currently from Github, for the development version:

pip install git+https://github.com/Techtonique/teller.git

Significance of marginal effects

The following notebook will give you an introduction to this functionality:

• thierrymoudiki_081119_boston_housing.ipynb

Contributions/remarks are welcome as usual, you can submit a pull request on Github.

Note: I am currently looking for a gig. You can hire me on Malt or send me an email: thierry dot moudiki at pm dot me. I can do descriptive statistics, data preparation, feature engineering, model calibration, training and validation, and model outputs’ interpretation. I am fluent in Python, R, SQL, Microsoft Excel, Visual Basic (among others) and French. My résumé? Here!

Previous publications

1. mlsauce's `v0.12.0`: prediction intervals for LSBoostRegressor Apr 15, 2024
2. Conformalized predictive simulations for univariate time series on more than 250 data sets Apr 7, 2024
3. learningmachine v1.1.2: for Python Apr 1, 2024
4. learningmachine v1.0.0: prediction intervals around the probability of the event 'a tumor being malignant' Mar 25, 2024
5. Bayesian inference and conformal prediction (prediction intervals) in nnetsauce v0.18.1 Mar 18, 2024
6. Multiple examples of Machine Learning forecasting with ahead Mar 11, 2024
7. rtopy (v0.1.1): calling R functions in Python Mar 4, 2024
8. ahead forecasting (v0.10.0): fast time series model calibration and Python plots Feb 26, 2024
9. A plethora of datasets at your fingertips Part3: how many times do couples cheat on each other? Feb 19, 2024
10. nnetsauce's introduction as of 2024-02-11 (new version 0.17.0) Feb 11, 2024
11. Tuning Machine Learning models with GPopt's new version Part 2 Feb 5, 2024
12. Tuning Machine Learning models with GPopt's new version Jan 29, 2024
13. Subsampling continuous and discrete response variables Jan 22, 2024
14. DeepMTS, a Deep Learning Model for Multivariate Time Series Jan 15, 2024
15. A classifier that's very accurate (and deep) Pt.2: there are > 90 classifiers in nnetsauce Jan 8, 2024
16. learningmachine: prediction intervals for conformalized Kernel ridge regression and Random Forest Jan 1, 2024
17. A plethora of datasets at your fingertips Part2: how many times do couples cheat on each other? Descriptive analytics, interpretability and prediction intervals using conformal prediction Dec 25, 2023
18. Diffusion models in Python with esgtoolkit (Part2) Dec 18, 2023
19. Diffusion models in Python with esgtoolkit Dec 11, 2023
20. Julia packaging at the command line Dec 4, 2023
21. Quasi-randomized nnetworks in Julia, Python and R Nov 27, 2023
22. A plethora of datasets at your fingertips Nov 20, 2023
23. A classifier that's very accurate (and deep) Nov 12, 2023
24. mlsauce version 0.8.10: Statistical/Machine Learning with Python and R Nov 5, 2023
25. AutoML in nnetsauce (randomized and quasi-randomized nnetworks) Pt.2: multivariate time series forecasting Oct 29, 2023
26. AutoML in nnetsauce (randomized and quasi-randomized nnetworks) Oct 22, 2023
27. Version v0.14.0 of nnetsauce for R and Python Oct 16, 2023
28. A diffusion model: G2++ Oct 9, 2023
29. Diffusion models in ESGtoolkit + announcements Oct 2, 2023
30. An infinity of time series forecasting models in nnetsauce (Part 2 with uncertainty quantification) Sep 25, 2023
31. (News from) forecasting in Python with ahead (progress bars and plots) Sep 18, 2023
32. Forecasting in Python with ahead Sep 11, 2023
33. Risk-neutralize simulations Sep 4, 2023
34. Comparing cross-validation results using crossval_ml and boxplots Aug 27, 2023
35. Reminder Apr 30, 2023
36. Did you ask ChatGPT about who you are? Apr 16, 2023
37. A new version of nnetsauce (randomized and quasi-randomized 'neural' networks) Apr 2, 2023
38. Simple interfaces to the forecasting API Nov 23, 2022
39. A web application for forecasting in Python, R, Ruby, C#, JavaScript, PHP, Go, Rust, Java, MATLAB, etc. Nov 2, 2022
40. Prediction intervals (not only) for Boosted Configuration Networks in Python Oct 5, 2022
41. Boosted Configuration (neural) Networks Pt. 2 Sep 3, 2022
42. Boosted Configuration (_neural_) Networks for classification Jul 21, 2022
43. A Machine Learning workflow using Techtonique Jun 6, 2022
44. Super Mario Bros © in the browser using PyScript May 8, 2022
45. News from ESGtoolkit, ycinterextra, and nnetsauce Apr 4, 2022
46. Explaining a Keras _neural_ network predictions with the-teller Mar 11, 2022
47. New version of nnetsauce -- various quasi-randomized networks Feb 12, 2022
48. A dashboard illustrating bivariate time series forecasting with `ahead` Jan 14, 2022
49. Hundreds of Statistical/Machine Learning models for univariate time series, using ahead, ranger, xgboost, and caret Dec 20, 2021
50. Forecasting with `ahead` (Python version) Dec 13, 2021
51. Tuning and interpreting LSBoost Nov 15, 2021
52. Time series cross-validation using `crossvalidation` (Part 2) Nov 7, 2021
53. Fast and scalable forecasting with ahead::ridge2f Oct 31, 2021
54. Automatic Forecasting with `ahead::dynrmf` and Ridge regression Oct 22, 2021
55. Forecasting with `ahead` Oct 15, 2021
56. Classification using linear regression Sep 26, 2021
57. `crossvalidation` and random search for calibrating support vector machines Aug 6, 2021
58. parallel grid search cross-validation using `crossvalidation` Jul 31, 2021
59. `crossvalidation` on R-universe, plus a classification example Jul 23, 2021
60. Documentation and source code for GPopt, a package for Bayesian optimization Jul 2, 2021
61. Hyperparameters tuning with GPopt Jun 11, 2021
62. A forecasting tool (API) with examples in curl, R, Python May 28, 2021
63. Bayesian Optimization with GPopt Part 2 (save and resume) Apr 30, 2021
64. Bayesian Optimization with GPopt Apr 16, 2021
65. Compatibility of nnetsauce and mlsauce with scikit-learn Mar 26, 2021
66. Explaining xgboost predictions with the teller Mar 12, 2021
67. An infinity of time series models in nnetsauce Mar 6, 2021
68. New activation functions in mlsauce's LSBoost Feb 12, 2021
69. 2020 recap, Gradient Boosting, Generalized Linear Models, AdaOpt with nnetsauce and mlsauce Dec 29, 2020
70. A deeper learning architecture in nnetsauce Dec 18, 2020
71. Classify penguins with nnetsauce's MultitaskClassifier Dec 11, 2020
72. Bayesian forecasting for uni/multivariate time series Dec 4, 2020
73. Generalized nonlinear models in nnetsauce Nov 28, 2020
74. Boosting nonlinear penalized least squares Nov 21, 2020
75. Statistical/Machine Learning explainability using Kernel Ridge Regression surrogates Nov 6, 2020
76. NEWS Oct 30, 2020
77. A glimpse into my PhD journey Oct 23, 2020
78. Submitting R package to CRAN Oct 16, 2020
79. Simulation of dependent variables in ESGtoolkit Oct 9, 2020
80. Forecasting lung disease progression Oct 2, 2020
81. New nnetsauce Sep 25, 2020
82. Technical documentation Sep 18, 2020
83. A new version of nnetsauce, and a new Techtonique website Sep 11, 2020
84. Back next week, and a few announcements Sep 4, 2020
85. Explainable 'AI' using Gradient Boosted randomized networks Pt2 (the Lasso) Jul 31, 2020
86. LSBoost: Explainable 'AI' using Gradient Boosted randomized networks (with examples in R and Python) Jul 24, 2020
87. nnetsauce version 0.5.0, randomized neural networks on GPU Jul 17, 2020
88. Maximizing your tip as a waiter (Part 2) Jul 10, 2020
89. New version of mlsauce, with Gradient Boosted randomized networks and stump decision trees Jul 3, 2020
90. Announcements Jun 26, 2020
91. Parallel AdaOpt classification Jun 19, 2020
92. Comments section and other news Jun 12, 2020
93. Maximizing your tip as a waiter Jun 5, 2020
94. AdaOpt classification on MNIST handwritten digits (without preprocessing) May 29, 2020
95. AdaOpt (a probabilistic classifier based on a mix of multivariable optimization and nearest neighbors) for R May 22, 2020
96. AdaOpt May 15, 2020
97. Custom errors for cross-validation using crossval::crossval_ml May 8, 2020
98. Documentation+Pypi for the `teller`, a model-agnostic tool for Machine Learning explainability May 1, 2020
99. Encoding your categorical variables based on the response variable and correlations Apr 24, 2020
100. Linear model, xgboost and randomForest cross-validation using crossval::crossval_ml Apr 17, 2020
101. Grid search cross-validation using crossval Apr 10, 2020
102. Documentation for the querier, a query language for Data Frames Apr 3, 2020
103. Time series cross-validation using crossval Mar 27, 2020
104. On model specification, identification, degrees of freedom and regularization Mar 20, 2020
105. Import data into the querier (now on Pypi), a query language for Data Frames Mar 13, 2020
106. R notebooks for nnetsauce Mar 6, 2020
107. Version 0.4.0 of nnetsauce, with fruits and breast cancer classification Feb 28, 2020
108. Create a specific feed in your Jekyll blog Feb 21, 2020
109. Git/Github for contributing to package development Feb 14, 2020
110. Feedback forms for contributing Feb 7, 2020
111. nnetsauce for R Jan 31, 2020
112. A new version of nnetsauce (v0.3.1) Jan 24, 2020
113. ESGtoolkit, a tool for Monte Carlo simulation (v0.2.0) Jan 17, 2020
114. Search bar, new year 2020 Jan 10, 2020
115. 2019 Recap, the nnetsauce, the teller and the querier Dec 20, 2019
116. Understanding model interactions with the `teller` Dec 13, 2019
117. Using the `teller` on a classifier Dec 6, 2019
118. Benchmarking the querier's verbs Nov 29, 2019
119. Composing the querier's verbs for data wrangling Nov 22, 2019
120. Comparing and explaining model predictions with the teller Nov 15, 2019
121. Tests for the significance of marginal effects in the teller Nov 8, 2019
122. Introducing the teller Nov 1, 2019
123. Introducing the querier Oct 25, 2019
124. Prediction intervals for nnetsauce models Oct 18, 2019
125. Using R in Python for statistical learning/data science Oct 11, 2019
126. Model calibration with `crossval` Oct 4, 2019
127. Bagging in the nnetsauce Sep 25, 2019
128. Adaboost learning with nnetsauce Sep 18, 2019
129. Change in blog's presentation Sep 4, 2019
130. nnetsauce on Pypi Jun 5, 2019
131. More nnetsauce (examples of use) May 9, 2019
132. nnetsauce Mar 13, 2019
133. crossval Mar 13, 2019
134. test Mar 10, 2019