Bootstrap optimism-corrected performance

Bootstrap optimism-corrected performance

Usage

detect_overfitting(
  fit_fn,
  predict_fn,
  X,
  y,
  scoring = "roc_auc",
  n_bootstrap = 200L,
  random_state = 42L
)

Arguments

fit_fn

A function (X, y) -> model.

predict_fn

A function (model, X) -> probability vector.

X

Matrix or data frame of features.

y

Vector of targets.

scoring

"roc_auc", "accuracy", "brier".

n_bootstrap

Integer; number of bootstrap resamples used to estimate the optimism correction (default 200).

random_state

Seed.