Encoder that combines One-hot and target encoding #2065

Problem Description

A suggestion from @ogrisel : an encoder that combines a one-hot encoding of the most frequent categories with target-encoding to also capture some information for less-frequent categories.

currently this has to be done manually in a dataop or FeatureUnion:

one_hot_encoded = X.skb.apply(
    OneHotEncoder(
        sparse_output=False,
        dtype=np.float32,
        handle_unknown="ignore",
        max_categories=5,
    )
)
target_encoded = X.skb.apply(TargetEncoder(random_state=0), y=y)
encoded = one_hot_encoded.skb.concat([target_encoded], axis=1)

import numpy as np
import skrub
import pandas as pd
from sklearn.preprocessing import OneHotEncoder, TargetEncoder

csv = skrub.var("csv", skrub.datasets.fetch_employee_salaries().path)
df = csv.skb.apply_func(pd.read_csv)
y = df["current_annual_salary"]
X = df[["department", "employee_position_title"]]


one_hot_encoded = X.skb.apply(
    OneHotEncoder(
        sparse_output=False,
        dtype=np.float32,
        handle_unknown="ignore",
        max_categories=5,
    )
)
target_encoded = X.skb.apply(TargetEncoder(random_state=0), y=y)
encoded = one_hot_encoded.skb.concat([target_encoded], axis=1)


encoded.skb.full_report()

which is quite verbose, and sorts the outputs by encoder whereas we would like them to be sorted by input column. it would be much nicer to have something like

encoded = X.skb.apply(skrub.CategoricalEncoder())

if using a scikit-learn pipeline instead of dataops it is also doable but quite inconvenient too, we have to use a scikit-learn FeatureUnion inside of a skrub ApplyToCols or chain 2 ApplyToCols and leverage keep_original and rename_columns to allow processing the columns twice.

Feature Description

an encoder that combines one-hot for the most important categories and a fallback targetencoding representation for the rare ones. the inner encoders could be hardcoded or configurable:

CategoricalEncoder(frequent=OrdinalEncoder(...), rare=TargetEncoder(), max_frequent=10)

(all names TBD)

Alternative Solutions

No response

Additional Context

No response