AttributeError: 'super' object has no attribute '__sklearn_tags__' when executing quickstart examples using Pipeline GridSearchCV

[bastian-f]

Hello,

When executing the quickstart which use Pipeline or GridSearchCV from https://adriangb.com/scikeras/stable/quickstart.html, I get the error shown below.

"Training a model" example with following code works:

import numpy as np
from sklearn.datasets import make_classification
import keras

from scikeras.wrappers import KerasClassifier


X, y = make_classification(1000, 20, n_informative=10, random_state=0)
X = X.astype(np.float32)
y = y.astype(np.int64)

def get_model(hidden_layer_dim, meta):
    # note that meta is a special argument that will be
    # handed a dict containing input metadata
    n_features_in_ = meta["n_features_in_"]
    X_shape_ = meta["X_shape_"]
    n_classes_ = meta["n_classes_"]

    model = keras.models.Sequential()
    model.add(keras.layers.Dense(n_features_in_, input_shape=X_shape_[1:]))
    model.add(keras.layers.Activation("relu"))
    model.add(keras.layers.Dense(hidden_layer_dim))
    model.add(keras.layers.Activation("relu"))
    model.add(keras.layers.Dense(n_classes_))
    model.add(keras.layers.Activation("softmax"))
    return model

clf = KerasClassifier(
    get_model,
    loss="sparse_categorical_crossentropy",
    hidden_layer_dim=100,
)

clf.fit(X, y)
y_proba = clf.predict_proba(X)

But the "In an sklearn pipeline" and the "Grid search" example raise the error.

"In an sklearn pipeline" code:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler


pipe = Pipeline([
    ('scale', StandardScaler()),
    ('clf', clf),
])

pipe.fit(X, y)
y_proba = pipe.predict_proba(X)

"Grid search" code:

from sklearn.model_selection import GridSearchCV


params = {
    "hidden_layer_dim": [50, 100, 200],
    "loss": ["sparse_categorical_crossentropy"],
    "optimizer": ["adam", "sgd"],
    "optimizer__learning_rate": [0.0001, 0.001, 0.1],
}
gs = GridSearchCV(clf, params, refit=False, cv=3, scoring='accuracy')

gs.fit(X, y)
print(gs.best_score_, gs.best_params_)

The error:

/home/bastian/keras_test_venv/lib/python3.12/site-packages/sklearn/utils/_tags.py:354: FutureWarning: The KerasClassifier or classes from which it inherits use `_get_tags` and `_more_tags`. Please define the `__sklearn_tags__` method, or inherit from `sklearn.base.BaseEstimator` and/or other appropriate mixins such as `sklearn.base.TransformerMixin`, `sklearn.base.ClassifierMixin`, `sklearn.base.RegressorMixin`, and `sklearn.base.OutlierMixin`. From scikit-learn 1.7, not defining `__sklearn_tags__` will raise an error.
  warnings.warn(
Traceback (most recent call last):
  File "/home/bastian/keras_test_venv/test.py", line 69, in <module>
    gs.fit(X, y)
  File "/home/bastian/keras_test_venv/lib/python3.12/site-packages/sklearn/base.py", line 1389, in wrapper
    return fit_method(estimator, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bastian/keras_test_venv/lib/python3.12/site-packages/sklearn/model_selection/_search.py", line 932, in fit
    cv_orig = check_cv(self.cv, y, classifier=is_classifier(estimator))
                                              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bastian/keras_test_venv/lib/python3.12/site-packages/sklearn/base.py", line 1237, in is_classifier
    return get_tags(estimator).estimator_type == "classifier"
           ^^^^^^^^^^^^^^^^^^^
  File "/home/bastian/keras_test_venv/lib/python3.12/site-packages/sklearn/utils/_tags.py", line 405, in get_tags
    sklearn_tags_provider[klass] = klass.__sklearn_tags__(estimator)  # type: ignore[attr-defined]
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bastian/keras_test_venv/lib/python3.12/site-packages/sklearn/base.py", line 540, in __sklearn_tags__
    tags = super().__sklearn_tags__()
           ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'super' object has no attribute '__sklearn_tags__'

Any help would be appreciated! Thanks!!

[adriangb]

Hi, what version of Scikit-Learn are you using?

[bastian-f]

I created a new virtual environment using venv and then installed scikeras and tensorflow with:

pip install scikeras tensorflow

The scikit-learn version is 1.6.0.

[bastian-f]

I downgraded scikit-learn to 1.4.2 and the error is not raised anymore.

[adriangb]

Yep I was going to suggest the Scikit-Learn version.
Sadly I have not been able to keep up with updates in Scikit-Learn and Keras.
Please see #329