Cross-validate a model using a user-supplied fit/predict pair

Usage

cross_validate(
  fit_fn,
  predict_fn,
  X,
  y,
  method = "stratified_kfold",
  n_folds = 5L,
  n_repeats = 10L,
  scoring = "roc_auc",
  groups = NULL,
  confidence = 0.95,
  random_state = 42L
)

Arguments

fit_fn: A function (X, y) -> model.
predict_fn: A function (model, X) -> probability vector.
X: Matrix or data frame of features.
y: Vector of targets.
method: Resampling strategy: "kfold", "stratified_kfold", "grouped_kfold", "loo", "monte_carlo", "time_series".
n_folds: Number of folds.
n_repeats: Repeats for monte_carlo.
scoring: "roc_auc", "accuracy", "brier".
groups: Group labels for grouped_kfold.
confidence: Confidence level for the score CI.
random_state: Seed.