Version v0.16.2 of nnetsauce is now available on your favorite platforms: PyPI, conda and GitHub for Python; R universe and GitHub for R. The updated documentation will be available next week (I’ve been using keras-autodoc so far for this purpose; it is now discontinued).

For Windows users: if you run into issues when trying to install nnetsauce (but you shouldn’t): remember that you can use the Windows Subsystem for Linux (WSL).

The changes in this new version are:

  • Regression-based classifiers
  • Automated Machine Learning (AutoML) for MTS (multivariate time series forecasting) (not in R yet)
  • AutoML for DeepMTS (multivariate time series forecasting) (not in R yet)
  • Subsample continuous and discrete responses

This post is about AutoML in nnetsauce using its 90 base classifiers, thus including regression-based classifiers. It’s a mix of these resources:

The goal is to show that, with nnetsauce, it’s possible to calibrate a large number of classifiers and select the best one, using a few lines of code. AutoML for MTS (multivariate time series forecasting) and DeepMTS will be presented next week.

0 - Install packages

The notebook can be found on GitHub for reproducibility of the following results: https://github.com/Techtonique/nnetsauce/blob/master/nnetsauce/demo/thierrymoudiki_080124_lazy_classifier_deep_classifier.ipynb.

!pip install nnetsauce

Collecting nnetsauce
  Downloading nnetsauce-0.16.2-py2.py3-none-any.whl (152 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 152.2/152.2 kB 2.3 MB/s eta 0:00:00
[?25hRequirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.3.2)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (3.7.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.23.5)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.5.3)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.11.4)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (1.2.2)
Requirement already satisfied: threadpoolctl in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (3.2.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (4.66.1)
Requirement already satisfied: jax in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (0.4.23)
Requirement already satisfied: jaxlib in /usr/local/lib/python3.10/dist-packages (from nnetsauce) (0.4.23+cuda12.cudnn89)
Requirement already satisfied: ml-dtypes>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from jax->nnetsauce) (0.2.0)
Requirement already satisfied: opt-einsum in /usr/local/lib/python3.10/dist-packages (from jax->nnetsauce) (3.3.0)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (4.47.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (1.4.5)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (23.2)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (9.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->nnetsauce) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->nnetsauce) (2023.3.post1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->nnetsauce) (1.16.0)
Installing collected packages: nnetsauce
Successfully installed nnetsauce-0.16.2



import os
import nnetsauce as ns
import matplotlib.pyplot as plt
from sklearn.datasets import load_breast_cancer, load_iris, load_wine, load_digits
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, ConfusionMatrixDisplay
from time import time

1 - Lazy classifiers

1 - 1 breast cancer data

data = load_breast_cancer()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)

display(models)

100%|██████████| 90/90 [01:21<00:00,  1.10it/s]



 Elapsed: 81.99524259567261 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
CustomClassifier(RandomForestClassifier) 0.99 0.99 0.99 0.99 1.96
CustomClassifier(SGDClassifier) 0.99 0.99 0.99 0.99 0.24
CustomClassifier(QuadraticDiscriminantAnalysis) 0.99 0.99 0.99 0.99 1.20
CustomClassifier(Perceptron) 0.99 0.99 0.99 0.99 1.38
SimpleMultitaskClassifier(BaggingRegressor) 0.99 0.99 0.99 0.99 0.58
... ... ... ... ... ...
SimpleMultitaskClassifier(Lasso) 0.64 0.50 0.50 0.50 0.56
SimpleMultitaskClassifier(LarsCV) 0.64 0.50 0.50 0.50 0.76
MultitaskClassifier(LassoLars) 0.64 0.50 0.50 0.50 2.11
SimpleMultitaskClassifier(Lars) 0.55 0.57 0.57 0.56 0.40
MultitaskClassifier(Lars) 0.54 0.56 0.56 0.55 0.53

85 rows × 5 columns

model_dictionary["CustomClassifier(RandomForestClassifier)"]

CustomClassifier(obj=RandomForestClassifier(random_state=42))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["CustomClassifier(RandomForestClassifier)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      0.98      0.99        41
           1       0.99      1.00      0.99        73

    accuracy                           0.99       114
   macro avg       0.99      0.99      0.99       114
weighted avg       0.99      0.99      0.99       114

ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(RandomForestClassifier)"], X_test, y_test)
plt.show()

xxx

1 - 2 - iris data

data = load_iris()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)

100%|██████████| 90/90 [00:10<00:00,  8.95it/s]



 Elapsed: 10.071635246276855 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
MultitaskClassifier(AdaBoostRegressor) 0.97 0.97 None 0.97 0.16
CustomClassifier(SGDClassifier) 0.97 0.97 None 0.97 0.02
MultitaskClassifier(BayesianRidge) 0.97 0.97 None 0.97 0.04
MultitaskClassifier(DecisionTreeRegressor) 0.97 0.97 None 0.97 0.05
SimpleMultitaskClassifier(LinearRegression) 0.97 0.97 None 0.97 0.02
... ... ... ... ... ...
MultitaskClassifier(Lasso) 0.20 0.33 None 0.07 0.07
SimpleMultitaskClassifier(ElasticNet) 0.20 0.33 None 0.07 0.02
SimpleMultitaskClassifier(DummyRegressor) 0.20 0.33 None 0.07 0.02
CustomClassifier(DummyClassifier) 0.20 0.33 None 0.07 0.02
MultitaskClassifier(LassoLars) 0.20 0.33 None 0.07 0.06

85 rows × 5 columns

model_dictionary["MultitaskClassifier(AdaBoostRegressor)"]

CustomClassifier(obj=MultitaskClassifier(obj=AdaBoostRegressor()))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["MultitaskClassifier(AdaBoostRegressor)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      1.00      1.00        13
           1       0.86      1.00      0.92         6
           2       1.00      0.91      0.95        11

    accuracy                           0.97        30
   macro avg       0.95      0.97      0.96        30
weighted avg       0.97      0.97      0.97        30

ConfusionMatrixDisplay.from_estimator(model_dictionary["MultitaskClassifier(AdaBoostRegressor)"], X_test, y_test)
plt.show()

xxx

1 - 3 - wine data

data = load_wine()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)

100%|██████████| 90/90 [00:13<00:00,  6.66it/s]



 Elapsed: 13.523086071014404 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
MultitaskClassifier(AdaBoostRegressor) 1.00 1.00 None 1.00 0.51
SimpleMultitaskClassifier(ExtraTreesRegressor) 1.00 1.00 None 1.00 0.70
MultitaskClassifier(ElasticNetCV) 1.00 1.00 None 1.00 0.56
SimpleMultitaskClassifier(LassoLarsCV) 1.00 1.00 None 1.00 0.09
SimpleMultitaskClassifier(HuberRegressor) 1.00 1.00 None 1.00 0.18
... ... ... ... ... ...
MultitaskClassifier(Lars) 0.31 0.28 None 0.31 0.07
CustomClassifier(DummyClassifier) 0.31 0.33 None 0.14 0.02
MultitaskClassifier(Lasso) 0.31 0.33 None 0.14 0.06
MultitaskClassifier(LassoLars) 0.31 0.33 None 0.14 0.06
MultitaskClassifier(DummyRegressor) 0.31 0.33 None 0.14 0.05

83 rows × 5 columns

model_dictionary["MultitaskClassifier(AdaBoostRegressor)"]

CustomClassifier(obj=MultitaskClassifier(obj=AdaBoostRegressor()))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["MultitaskClassifier(AdaBoostRegressor)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      1.00      1.00         8
           1       1.00      1.00      1.00        11
           2       1.00      1.00      1.00        17

    accuracy                           1.00        36
   macro avg       1.00      1.00      1.00        36
weighted avg       1.00      1.00      1.00        36

ConfusionMatrixDisplay.from_estimator(model_dictionary["MultitaskClassifier(AdaBoostRegressor)"], X_test, y_test)
plt.show()

xxx

1 - 4 - digits data

data = load_digits()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyClassifier(verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)

100%|██████████| 90/90 [03:47<00:00,  2.53s/it]



 Elapsed: 227.8047935962677 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
CustomClassifier(SVC) 0.99 0.99 None 0.99 0.99
CustomClassifier(ExtraTreesClassifier) 0.98 0.98 None 0.98 1.26
MultitaskClassifier(ExtraTreesRegressor) 0.98 0.98 None 0.98 7.42
SimpleMultitaskClassifier(ExtraTreesRegressor) 0.98 0.98 None 0.98 6.63
CustomClassifier(LogisticRegressionCV) 0.98 0.98 None 0.98 5.03
... ... ... ... ... ...
MultitaskClassifier(LassoLars) 0.08 0.10 None 0.01 2.20
MultitaskClassifier(Lasso) 0.08 0.10 None 0.01 2.07
MultitaskClassifier(DummyRegressor) 0.08 0.10 None 0.01 2.39
MultitaskClassifier(ElasticNet) 0.08 0.10 None 0.01 1.89
SimpleMultitaskClassifier(DummyRegressor) 0.08 0.10 None 0.01 1.19

83 rows × 5 columns

model_dictionary["CustomClassifier(SVC)"]

CustomClassifier(obj=SVC(random_state=42))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["CustomClassifier(SVC)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      1.00      1.00        39
           1       0.94      1.00      0.97        34
           2       1.00      0.97      0.99        36
           3       1.00      1.00      1.00        33
           4       0.95      1.00      0.98        42
           5       1.00      0.95      0.97        37
           6       1.00      1.00      1.00        43
           7       1.00      1.00      1.00        31
           8       1.00      0.95      0.97        37
           9       0.97      1.00      0.98        28

    accuracy                           0.99       360
   macro avg       0.99      0.99      0.99       360
weighted avg       0.99      0.99      0.99       360

ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(SVC)"], X_test, y_test)
plt.show()

xxx

ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(LogisticRegressionCV)"], X_test, y_test)
plt.show()

xxx

2 - Lazy Deep classifiers

2 - 1 breast cancer data

data = load_breast_cancer()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)

display(models)

100%|██████████| 90/90 [03:43<00:00,  2.48s/it]



 Elapsed: 223.3978340625763 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
SimpleMultitaskClassifier(MLPRegressor) 0.99 0.99 0.99 0.99 3.77
MultitaskClassifier(BaggingRegressor) 0.99 0.99 0.99 0.99 3.45
SimpleMultitaskClassifier(BaggingRegressor) 0.99 0.99 0.99 0.99 1.94
CustomClassifier(Perceptron) 0.99 0.99 0.99 0.99 1.54
CustomClassifier(MLPClassifier) 0.99 0.99 0.99 0.99 3.19
... ... ... ... ... ...
MultitaskClassifier(LassoLars) 0.64 0.50 0.50 0.50 2.25
SimpleMultitaskClassifier(Lars) 0.54 0.54 0.54 0.55 3.24
MultitaskClassifier(Lars) 0.51 0.51 0.51 0.52 2.02
SimpleMultitaskClassifier(RANSACRegressor) 0.39 0.52 0.52 0.26 2.86
MultitaskClassifier(RANSACRegressor) 0.36 0.50 0.50 0.19 5.04

85 rows × 5 columns

model_dictionary["SimpleMultitaskClassifier(MLPRegressor)"]

CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=SimpleMultitaskClassifier(obj=MLPRegressor()))))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["SimpleMultitaskClassifier(MLPRegressor)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      0.98      0.99        41
           1       0.99      1.00      0.99        73

    accuracy                           0.99       114
   macro avg       0.99      0.99      0.99       114
weighted avg       0.99      0.99      0.99       114

ConfusionMatrixDisplay.from_estimator(model_dictionary["SimpleMultitaskClassifier(MLPRegressor)"], X_test, y_test)
plt.show()

xxx

2 - 2 - iris data

data = load_iris()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)

100%|██████████| 90/90 [00:25<00:00,  3.54it/s]



 Elapsed: 25.443820476531982 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
CustomClassifier(BaggingClassifier) 1.00 1.00 None 1.00 0.12
CustomClassifier(DecisionTreeClassifier) 1.00 1.00 None 1.00 0.08
MultitaskClassifier(MLPRegressor) 1.00 1.00 None 1.00 0.85
MultitaskClassifier(BaggingRegressor) 0.97 0.97 None 0.97 0.25
MultitaskClassifier(ElasticNetCV) 0.97 0.97 None 0.97 1.34
... ... ... ... ... ...
SimpleMultitaskClassifier(Lasso) 0.20 0.33 None 0.07 0.09
MultitaskClassifier(Lasso) 0.20 0.33 None 0.07 0.15
SimpleMultitaskClassifier(ElasticNet) 0.20 0.33 None 0.07 0.10
SimpleMultitaskClassifier(DummyRegressor) 0.20 0.33 None 0.07 0.08
MultitaskClassifier(LassoLars) 0.20 0.33 None 0.07 0.13

85 rows × 5 columns

model_dictionary["CustomClassifier(BaggingClassifier)"]

CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=BaggingClassifier(random_state=42))))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["CustomClassifier(BaggingClassifier)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      1.00      1.00        13
           1       1.00      1.00      1.00         6
           2       1.00      1.00      1.00        11

    accuracy                           1.00        30
   macro avg       1.00      1.00      1.00        30
weighted avg       1.00      1.00      1.00        30

ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(BaggingClassifier)"], X_test, y_test)
plt.show()

xxx

2 - 3 - wine data

data = load_wine()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)

100%|██████████| 90/90 [00:27<00:00,  3.26it/s]



 Elapsed: 27.655547857284546 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
MultitaskClassifier(LassoCV) 1.00 1.00 None 1.00 1.36
SimpleMultitaskClassifier(LinearRegression) 1.00 1.00 None 1.00 0.10
MultitaskClassifier(PassiveAggressiveRegressor) 1.00 1.00 None 1.00 0.19
MultitaskClassifier(SGDRegressor) 1.00 1.00 None 1.00 0.20
SimpleMultitaskClassifier(AdaBoostRegressor) 1.00 1.00 None 1.00 0.52
... ... ... ... ... ...
MultitaskClassifier(Lars) 0.31 0.38 None 0.27 0.21
CustomClassifier(DummyClassifier) 0.31 0.33 None 0.14 0.10
MultitaskClassifier(Lasso) 0.31 0.33 None 0.14 0.16
MultitaskClassifier(LassoLars) 0.31 0.33 None 0.14 0.18
MultitaskClassifier(DummyRegressor) 0.31 0.33 None 0.14 0.45

83 rows × 5 columns

model_dictionary["MultitaskClassifier(LassoCV)"]

CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=MultitaskClassifier(obj=LassoCV()))))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["MultitaskClassifier(LassoCV)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      1.00      1.00         8
           1       1.00      1.00      1.00        11
           2       1.00      1.00      1.00        17

    accuracy                           1.00        36
   macro avg       1.00      1.00      1.00        36
weighted avg       1.00      1.00      1.00        36

ConfusionMatrixDisplay.from_estimator(model_dictionary["MultitaskClassifier(LassoCV)"], X_test, y_test)
plt.show()

xxx

2 - 4 - digits data

data = load_digits()
X = data.data
y= data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .2, random_state = 123)

clf = ns.LazyDeepClassifier(n_layers=3, verbose=0, ignore_warnings=True)
start = time()
models, predictions = clf.fit(X_train, X_test, y_train, y_test)
print(f"\n\n Elapsed: {time()-start} seconds \n")
model_dictionary = clf.provide_models(X_train, X_test, y_train, y_test)
display(models)

100%|██████████| 90/90 [10:05<00:00,  6.72s/it]



 Elapsed: 605.1929349899292 seconds
Accuracy Balanced Accuracy ROC AUC F1 Score Time Taken
Model
CustomClassifier(SVC) 0.99 0.99 None 0.99 4.41
SimpleMultitaskClassifier(AdaBoostRegressor) 0.98 0.98 None 0.98 12.88
MultitaskClassifier(ExtraTreesRegressor) 0.98 0.98 None 0.98 17.08
SimpleMultitaskClassifier(ExtraTreesRegressor) 0.98 0.98 None 0.98 19.48
CustomClassifier(LogisticRegressionCV) 0.98 0.98 None 0.98 15.02
... ... ... ... ... ...
SimpleMultitaskClassifier(DummyRegressor) 0.08 0.10 None 0.01 3.48
MultitaskClassifier(DummyRegressor) 0.08 0.10 None 0.01 6.17
MultitaskClassifier(ElasticNet) 0.08 0.10 None 0.01 5.31
CustomClassifier(DummyClassifier) 0.08 0.10 None 0.01 4.52
MultitaskClassifier(LassoLars) 0.08 0.10 None 0.01 4.03

83 rows × 5 columns

model_dictionary["CustomClassifier(SVC)"]

CustomClassifier(obj=CustomClassifier(obj=CustomClassifier(obj=SVC(random_state=42))))
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
print(classification_report(y_test, model_dictionary["CustomClassifier(SVC)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      1.00      1.00        39
           1       0.94      1.00      0.97        34
           2       1.00      0.97      0.99        36
           3       1.00      1.00      1.00        33
           4       0.95      1.00      0.98        42
           5       1.00      0.95      0.97        37
           6       1.00      1.00      1.00        43
           7       1.00      1.00      1.00        31
           8       1.00      0.95      0.97        37
           9       0.97      1.00      0.98        28

    accuracy                           0.99       360
   macro avg       0.99      0.99      0.99       360
weighted avg       0.99      0.99      0.99       360

print(classification_report(y_test, model_dictionary["CustomClassifier(LogisticRegressionCV)"].fit(X_train, y_train).predict(X_test)))

              precision    recall  f1-score   support

           0       1.00      1.00      1.00        39
           1       0.87      1.00      0.93        34
           2       1.00      1.00      1.00        36
           3       1.00      1.00      1.00        33
           4       0.98      0.98      0.98        42
           5       1.00      0.97      0.99        37
           6       1.00      0.95      0.98        43
           7       0.97      1.00      0.98        31
           8       1.00      0.89      0.94        37
           9       0.97      1.00      0.98        28

    accuracy                           0.98       360
   macro avg       0.98      0.98      0.98       360
weighted avg       0.98      0.98      0.98       360

ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(SVC)"], X_test, y_test)
plt.show()

xxx

ConfusionMatrixDisplay.from_estimator(model_dictionary["CustomClassifier(LogisticRegressionCV)"], X_test, y_test)
plt.show()

xxx

Comments powered by Talkyard.