Today, give a try to Techtonique web app, a tool designed to help you make informed, data-driven decisions using Mathematics, Statistics, Machine Learning, and Data Visualization. Here is a tutorial with audio, video, code, and slides: https://moudiki2.gumroad.com/l/nrhgb
Version v0.16.2
of nnetsauce
is now available on your favorite platforms: PyPI, conda and GitHub for Python; R universe and GitHub for R. The updated documentation will be available next week (I’ve been using keras-autodoc
so far for this purpose; it is now discontinued).
For Windows users: if you run into issues when trying to install nnetsauce
(but you shouldn’t): remember that you can use the Windows Subsystem for Linux (WSL).
The changes in this new version are:
- Regression-based classifiers
- Automated Machine Learning (AutoML) for MTS (multivariate time series forecasting) (not in R yet)
- AutoML for DeepMTS (multivariate time series forecasting) (not in R yet)
- Subsample continuous and discrete responses
This post is about AutoML in nnetsauce using its 90 base classifiers, thus including regression-based classifiers. It’s a mix of these resources:
- This PDF document describing the regression-based classifiers available in
nnetsauce
- Classification using linear regression
- A classifier that’s very accurate (and deep)
The goal is to show that, with nnetsauce
, it’s possible to calibrate a large number of classifiers and select the best one, using a few lines of code. AutoML for MTS (multivariate time series forecasting) and DeepMTS will be presented next week.
0 - Install packages
The notebook can be found on GitHub for reproducibility of the following results: https://github.com/Techtonique/nnetsauce/blob/master/nnetsauce/demo/thierrymoudiki_080124_lazy_classifier_deep_classifier.ipynb.
!pip install nnetsauce
Collecting nnetsauce
Downloading nnetsauce-0.16.2-py2.py3-none-any.whl (152 kB)
[2K [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m152.2/152.2 kB[0m [31m2.3 MB/s[0m eta [36m0:00:00[0m
[?25hRequirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.3.2)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (3.7.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.23.5)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.5.3)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.11.4)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.2.2)
Requirement already satisfied: threadpoolctl in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (3.2.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (4.66.1)
Requirement already satisfied: jax in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (0.4.23)
Requirement already satisfied: jaxlib in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (0.4.23+cuda12.cudnn89)
Requirement already satisfied: ml-dtypes>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from jax->nnetsauce) (0.2.0)
Requirement already satisfied: opt-einsum in /usr/local/lib/python3.10/dist-packages (from jax->nnetsauce) (3.3.0)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (4.47.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (1.4.5)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (23.2)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (9.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->nnetsauce) (2023.3.post1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->nnetsauce) (1.16.0)
Installing collected packages: nnetsauce
Successfully installed nnetsauce-0.16.2
import os
import nnetsauce as ns
import matplotlib.pyplot as plt
from sklearn.datasets import load_breast_cancer, load_iris, load_wine, load_digits
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, ConfusionMatrixDisplay
from time import time
1 - Lazy classifiers
1 - 1 breast cancer data
data = load_breast_cancer()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [01:21<00:00, 1.10it/s]
Elapsed: 81.99524259567261 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
CustomClassifier(RandomForestClassifier) | 0.99 | 0.99 | 0.99 | 0.99 | 1.96 |
CustomClassifier(SGDClassifier) | 0.99 | 0.99 | 0.99 | 0.99 | 0.24 |
CustomClassifier(QuadraticDiscriminantAnalysis) | 0.99 | 0.99 | 0.99 | 0.99 | 1.20 |
CustomClassifier(Perceptron) | 0.99 | 0.99 | 0.99 | 0.99 | 1.38 |
SimpleMultitaskClassifier(BaggingRegressor) | 0.99 | 0.99 | 0.99 | 0.99 | 0.58 |
... | ... | ... | ... | ... | ... |
SimpleMultitaskClassifier(Lasso) | 0.64 | 0.50 | 0.50 | 0.50 | 0.56 |
SimpleMultitaskClassifier(LarsCV) | 0.64 | 0.50 | 0.50 | 0.50 | 0.76 |
MultitaskClassifier(LassoLars) | 0.64 | 0.50 | 0.50 | 0.50 | 2.11 |
SimpleMultitaskClassifier(Lars) | 0.55 | 0.57 | 0.57 | 0.56 | 0.40 |
MultitaskClassifier(Lars) | 0.54 | 0.56 | 0.56 | 0.55 | 0.53 |
85 rows × 5 columns
model_dictionary["CustomClassifier(RandomForestClassifier)"]
CustomClassifier(obj=RandomForestClassifier(random_state=42))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=RandomForestClassifier(random_state=42))
RandomForestClassifier(random_state=42)
RandomForestClassifier(random_state=42)
print(classification_report(y_test, model_dictionary["CustomClassifier(RandomForestClassifier)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 0.98 0.99 41
1 0.99 1.00 0.99 73
accuracy 0.99 114
macro avg 0.99 0.99 0.99 114
weighted avg 0.99 0.99 0.99 114
ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(RandomForestClassifier)"], X_test, y_test)
plt.show()
1 - 2 - iris data
data = load_iris()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [00:10<00:00, 8.95it/s]
Elapsed: 10.071635246276855 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
MultitaskClassifier(AdaBoostRegressor) | 0.97 | 0.97 | None | 0.97 | 0.16 |
CustomClassifier(SGDClassifier) | 0.97 | 0.97 | None | 0.97 | 0.02 |
MultitaskClassifier(BayesianRidge) | 0.97 | 0.97 | None | 0.97 | 0.04 |
MultitaskClassifier(DecisionTreeRegressor) | 0.97 | 0.97 | None | 0.97 | 0.05 |
SimpleMultitaskClassifier(LinearRegression) | 0.97 | 0.97 | None | 0.97 | 0.02 |
... | ... | ... | ... | ... | ... |
MultitaskClassifier(Lasso) | 0.20 | 0.33 | None | 0.07 | 0.07 |
SimpleMultitaskClassifier(ElasticNet) | 0.20 | 0.33 | None | 0.07 | 0.02 |
SimpleMultitaskClassifier(DummyRegressor) | 0.20 | 0.33 | None | 0.07 | 0.02 |
CustomClassifier(DummyClassifier) | 0.20 | 0.33 | None | 0.07 | 0.02 |
MultitaskClassifier(LassoLars) | 0.20 | 0.33 | None | 0.07 | 0.06 |
85 rows × 5 columns
model_dictionary["MultitaskClassifier(AdaBoostRegressor)"]
CustomClassifier(obj=MultitaskClassifier(obj=AdaBoostRegressor()))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=MultitaskClassifier(obj=AdaBoostRegressor()))
MultitaskClassifier(obj=AdaBoostRegressor())
AdaBoostRegressor()
AdaBoostRegressor()
print(classification_report(y_test, model_dictionary["MultitaskClassifier(AdaBoostRegressor)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 1.00 1.00 13
1 0.86 1.00 0.92 6
2 1.00 0.91 0.95 11
accuracy 0.97 30
macro avg 0.95 0.97 0.96 30
weighted avg 0.97 0.97 0.97 30
ConfusionMatrixDisplay.from_estimator(model_dictionary["MultitaskClassifier(AdaBoostRegressor)"], X_test, y_test)
plt.show()
1 - 3 - wine data
data = load_wine()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [00:13<00:00, 6.66it/s]
Elapsed: 13.523086071014404 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
MultitaskClassifier(AdaBoostRegressor) | 1.00 | 1.00 | None | 1.00 | 0.51 |
SimpleMultitaskClassifier(ExtraTreesRegressor) | 1.00 | 1.00 | None | 1.00 | 0.70 |
MultitaskClassifier(ElasticNetCV) | 1.00 | 1.00 | None | 1.00 | 0.56 |
SimpleMultitaskClassifier(LassoLarsCV) | 1.00 | 1.00 | None | 1.00 | 0.09 |
SimpleMultitaskClassifier(HuberRegressor) | 1.00 | 1.00 | None | 1.00 | 0.18 |
... | ... | ... | ... | ... | ... |
MultitaskClassifier(Lars) | 0.31 | 0.28 | None | 0.31 | 0.07 |
CustomClassifier(DummyClassifier) | 0.31 | 0.33 | None | 0.14 | 0.02 |
MultitaskClassifier(Lasso) | 0.31 | 0.33 | None | 0.14 | 0.06 |
MultitaskClassifier(LassoLars) | 0.31 | 0.33 | None | 0.14 | 0.06 |
MultitaskClassifier(DummyRegressor) | 0.31 | 0.33 | None | 0.14 | 0.05 |
83 rows × 5 columns
model_dictionary["MultitaskClassifier(AdaBoostRegressor)"]
CustomClassifier(obj=MultitaskClassifier(obj=AdaBoostRegressor()))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=MultitaskClassifier(obj=AdaBoostRegressor()))
MultitaskClassifier(obj=AdaBoostRegressor())
AdaBoostRegressor()
AdaBoostRegressor()
print(classification_report(y_test, model_dictionary["MultitaskClassifier(AdaBoostRegressor)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 1.00 1.00 8
1 1.00 1.00 1.00 11
2 1.00 1.00 1.00 17
accuracy 1.00 36
macro avg 1.00 1.00 1.00 36
weighted avg 1.00 1.00 1.00 36
ConfusionMatrixDisplay.from_estimator(model_dictionary["MultitaskClassifier(AdaBoostRegressor)"], X_test, y_test)
plt.show()
1 - 4 - digits data
data = load_digits()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [03:47<00:00, 2.53s/it]
Elapsed: 227.8047935962677 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
CustomClassifier(SVC) | 0.99 | 0.99 | None | 0.99 | 0.99 |
CustomClassifier(ExtraTreesClassifier) | 0.98 | 0.98 | None | 0.98 | 1.26 |
MultitaskClassifier(ExtraTreesRegressor) | 0.98 | 0.98 | None | 0.98 | 7.42 |
SimpleMultitaskClassifier(ExtraTreesRegressor) | 0.98 | 0.98 | None | 0.98 | 6.63 |
CustomClassifier(LogisticRegressionCV) | 0.98 | 0.98 | None | 0.98 | 5.03 |
... | ... | ... | ... | ... | ... |
MultitaskClassifier(LassoLars) | 0.08 | 0.10 | None | 0.01 | 2.20 |
MultitaskClassifier(Lasso) | 0.08 | 0.10 | None | 0.01 | 2.07 |
MultitaskClassifier(DummyRegressor) | 0.08 | 0.10 | None | 0.01 | 2.39 |
MultitaskClassifier(ElasticNet) | 0.08 | 0.10 | None | 0.01 | 1.89 |
SimpleMultitaskClassifier(DummyRegressor) | 0.08 | 0.10 | None | 0.01 | 1.19 |
83 rows × 5 columns
model_dictionary["CustomClassifier(SVC)"]
CustomClassifier(obj=SVC(random_state=42))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=SVC(random_state=42))
SVC(random_state=42)
SVC(random_state=42)
print(classification_report(y_test, model_dictionary["CustomClassifier(SVC)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 1.00 1.00 39
1 0.94 1.00 0.97 34
2 1.00 0.97 0.99 36
3 1.00 1.00 1.00 33
4 0.95 1.00 0.98 42
5 1.00 0.95 0.97 37
6 1.00 1.00 1.00 43
7 1.00 1.00 1.00 31
8 1.00 0.95 0.97 37
9 0.97 1.00 0.98 28
accuracy 0.99 360
macro avg 0.99 0.99 0.99 360
weighted avg 0.99 0.99 0.99 360
ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(SVC)"], X_test, y_test)
plt.show()
ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(LogisticRegressionCV)"], X_test, y_test)
plt.show()
2 - Lazy Deep classifiers
2 - 1 breast cancer data
data = load_breast_cancer()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [03:43<00:00, 2.48s/it]
Elapsed: 223.3978340625763 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
SimpleMultitaskClassifier(MLPRegressor) | 0.99 | 0.99 | 0.99 | 0.99 | 3.77 |
MultitaskClassifier(BaggingRegressor) | 0.99 | 0.99 | 0.99 | 0.99 | 3.45 |
SimpleMultitaskClassifier(BaggingRegressor) | 0.99 | 0.99 | 0.99 | 0.99 | 1.94 |
CustomClassifier(Perceptron) | 0.99 | 0.99 | 0.99 | 0.99 | 1.54 |
CustomClassifier(MLPClassifier) | 0.99 | 0.99 | 0.99 | 0.99 | 3.19 |
... | ... | ... | ... | ... | ... |
MultitaskClassifier(LassoLars) | 0.64 | 0.50 | 0.50 | 0.50 | 2.25 |
SimpleMultitaskClassifier(Lars) | 0.54 | 0.54 | 0.54 | 0.55 | 3.24 |
MultitaskClassifier(Lars) | 0.51 | 0.51 | 0.51 | 0.52 | 2.02 |
SimpleMultitaskClassifier(RANSACRegressor) | 0.39 | 0.52 | 0.52 | 0.26 | 2.86 |
MultitaskClassifier(RANSACRegressor) | 0.36 | 0.50 | 0.50 | 0.19 | 5.04 |
85 rows × 5 columns
model_dictionary["SimpleMultitaskClassifier(MLPRegressor)"]
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=SimpleMultitaskClassifier(obj=MLPRegressor()))))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=SimpleMultitaskClassifier(obj=MLPRegressor()))))
CustomClassifier(obj=CustomClassifier(obj=SimpleMultitaskClassifier(obj=MLPRegressor())))
CustomClassifier(obj=SimpleMultitaskClassifier(obj=MLPRegressor()))
SimpleMultitaskClassifier(obj=MLPRegressor())
MLPRegressor()
MLPRegressor()
print(classification_report(y_test, model_dictionary["SimpleMultitaskClassifier(MLPRegressor)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 0.98 0.99 41
1 0.99 1.00 0.99 73
accuracy 0.99 114
macro avg 0.99 0.99 0.99 114
weighted avg 0.99 0.99 0.99 114
ConfusionMatrixDisplay.from_estimator(model_dictionary["SimpleMultitaskClassifier(MLPRegressor)"], X_test, y_test)
plt.show()
2 - 2 - iris data
data = load_iris()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [00:25<00:00, 3.54it/s]
Elapsed: 25.443820476531982 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
CustomClassifier(BaggingClassifier) | 1.00 | 1.00 | None | 1.00 | 0.12 |
CustomClassifier(DecisionTreeClassifier) | 1.00 | 1.00 | None | 1.00 | 0.08 |
MultitaskClassifier(MLPRegressor) | 1.00 | 1.00 | None | 1.00 | 0.85 |
MultitaskClassifier(BaggingRegressor) | 0.97 | 0.97 | None | 0.97 | 0.25 |
MultitaskClassifier(ElasticNetCV) | 0.97 | 0.97 | None | 0.97 | 1.34 |
... | ... | ... | ... | ... | ... |
SimpleMultitaskClassifier(Lasso) | 0.20 | 0.33 | None | 0.07 | 0.09 |
MultitaskClassifier(Lasso) | 0.20 | 0.33 | None | 0.07 | 0.15 |
SimpleMultitaskClassifier(ElasticNet) | 0.20 | 0.33 | None | 0.07 | 0.10 |
SimpleMultitaskClassifier(DummyRegressor) | 0.20 | 0.33 | None | 0.07 | 0.08 |
MultitaskClassifier(LassoLars) | 0.20 | 0.33 | None | 0.07 | 0.13 |
85 rows × 5 columns
model_dictionary["CustomClassifier(BaggingClassifier)"]
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=BaggingClassifier(random_state=42))))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=BaggingClassifier(random_state=42))))
CustomClassifier(obj=CustomClassifier(obj=BaggingClassifier(random_state=42)))
CustomClassifier(obj=BaggingClassifier(random_state=42))
BaggingClassifier(random_state=42)
BaggingClassifier(random_state=42)
print(classification_report(y_test, model_dictionary["CustomClassifier(BaggingClassifier)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 1.00 1.00 13
1 1.00 1.00 1.00 6
2 1.00 1.00 1.00 11
accuracy 1.00 30
macro avg 1.00 1.00 1.00 30
weighted avg 1.00 1.00 1.00 30
ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(BaggingClassifier)"], X_test, y_test)
plt.show()
2 - 3 - wine data
data = load_wine()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [00:27<00:00, 3.26it/s]
Elapsed: 27.655547857284546 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
MultitaskClassifier(LassoCV) | 1.00 | 1.00 | None | 1.00 | 1.36 |
SimpleMultitaskClassifier(LinearRegression) | 1.00 | 1.00 | None | 1.00 | 0.10 |
MultitaskClassifier(PassiveAggressiveRegressor) | 1.00 | 1.00 | None | 1.00 | 0.19 |
MultitaskClassifier(SGDRegressor) | 1.00 | 1.00 | None | 1.00 | 0.20 |
SimpleMultitaskClassifier(AdaBoostRegressor) | 1.00 | 1.00 | None | 1.00 | 0.52 |
... | ... | ... | ... | ... | ... |
MultitaskClassifier(Lars) | 0.31 | 0.38 | None | 0.27 | 0.21 |
CustomClassifier(DummyClassifier) | 0.31 | 0.33 | None | 0.14 | 0.10 |
MultitaskClassifier(Lasso) | 0.31 | 0.33 | None | 0.14 | 0.16 |
MultitaskClassifier(LassoLars) | 0.31 | 0.33 | None | 0.14 | 0.18 |
MultitaskClassifier(DummyRegressor) | 0.31 | 0.33 | None | 0.14 | 0.45 |
83 rows × 5 columns
model_dictionary["MultitaskClassifier(LassoCV)"]
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=MultitaskClassifier(obj=LassoCV()))))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=MultitaskClassifier(obj=LassoCV()))))
CustomClassifier(obj=CustomClassifier(obj=MultitaskClassifier(obj=LassoCV())))
CustomClassifier(obj=MultitaskClassifier(obj=LassoCV()))
MultitaskClassifier(obj=LassoCV())
LassoCV()
LassoCV()
print(classification_report(y_test, model_dictionary["MultitaskClassifier(LassoCV)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 1.00 1.00 8
1 1.00 1.00 1.00 11
2 1.00 1.00 1.00 17
accuracy 1.00 36
macro avg 1.00 1.00 1.00 36
weighted avg 1.00 1.00 1.00 36
ConfusionMatrixDisplay.from_estimator(model_dictionary["MultitaskClassifier(LassoCV)"], X_test, y_test)
plt.show()
2 - 4 - digits data
data = load_digits()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)
clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)
100%|██████████| 90/90 [10:05<00:00, 6.72s/it]
Elapsed: 605.1929349899292 seconds
Accuracy | Balanced Accuracy | ROC AUC | F1 Score | Time Taken | |
---|---|---|---|---|---|
Model | |||||
CustomClassifier(SVC) | 0.99 | 0.99 | None | 0.99 | 4.41 |
SimpleMultitaskClassifier(AdaBoostRegressor) | 0.98 | 0.98 | None | 0.98 | 12.88 |
MultitaskClassifier(ExtraTreesRegressor) | 0.98 | 0.98 | None | 0.98 | 17.08 |
SimpleMultitaskClassifier(ExtraTreesRegressor) | 0.98 | 0.98 | None | 0.98 | 19.48 |
CustomClassifier(LogisticRegressionCV) | 0.98 | 0.98 | None | 0.98 | 15.02 |
... | ... | ... | ... | ... | ... |
SimpleMultitaskClassifier(DummyRegressor) | 0.08 | 0.10 | None | 0.01 | 3.48 |
MultitaskClassifier(DummyRegressor) | 0.08 | 0.10 | None | 0.01 | 6.17 |
MultitaskClassifier(ElasticNet) | 0.08 | 0.10 | None | 0.01 | 5.31 |
CustomClassifier(DummyClassifier) | 0.08 | 0.10 | None | 0.01 | 4.52 |
MultitaskClassifier(LassoLars) | 0.08 | 0.10 | None | 0.01 | 4.03 |
83 rows × 5 columns
model_dictionary["CustomClassifier(SVC)"]
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=SVC(random_state=42))))In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=SVC(random_state=42))))
CustomClassifier(obj=CustomClassifier(obj=SVC(random_state=42)))
CustomClassifier(obj=SVC(random_state=42))
SVC(random_state=42)
SVC(random_state=42)
print(classification_report(y_test, model_dictionary["CustomClassifier(SVC)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 1.00 1.00 39
1 0.94 1.00 0.97 34
2 1.00 0.97 0.99 36
3 1.00 1.00 1.00 33
4 0.95 1.00 0.98 42
5 1.00 0.95 0.97 37
6 1.00 1.00 1.00 43
7 1.00 1.00 1.00 31
8 1.00 0.95 0.97 37
9 0.97 1.00 0.98 28
accuracy 0.99 360
macro avg 0.99 0.99 0.99 360
weighted avg 0.99 0.99 0.99 360
print(classification_report(y_test, model_dictionary["CustomClassifier(LogisticRegressionCV)"].fit(X_train, y_train).predict(X_test)))
precision recall f1-score support
0 1.00 1.00 1.00 39
1 0.87 1.00 0.93 34
2 1.00 1.00 1.00 36
3 1.00 1.00 1.00 33
4 0.98 0.98 0.98 42
5 1.00 0.97 0.99 37
6 1.00 0.95 0.98 43
7 0.97 1.00 0.98 31
8 1.00 0.89 0.94 37
9 0.97 1.00 0.98 28
accuracy 0.98 360
macro avg 0.98 0.98 0.98 360
weighted avg 0.98 0.98 0.98 360
ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(SVC)"], X_test, y_test)
plt.show()
ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(LogisticRegressionCV)"], X_test, y_test)
plt.show()
Comments powered by Talkyard.