.632 and .632+ bootstrap estimators for prediction error

Computes apparent error, mean OOB bootstrap error, the .632 estimator (Efron 1983), and the .632+ no-information-adjusted estimator (Efron and Tibshirani 1997). ipred::errorest(..., estimator = "632plus") implements the same family in ipred; it is cross-referenced for users who already work with ipred's predict.\<learner\> ecosystem. The inline implementation is retained because rmorie's API takes naked model_fn / score_fn callables and is consumed by downstream MRM code.

Usage

bootstrap_632(X, y, model_fn, score_fn, n_boot = 200L, seed = 42L)

Arguments

X

Numeric design matrix (n x p).

y

Numeric response (length n).

model_fn

Function model_fn(X_train, y_train) returning a model object that supports predict(model, X_test).

score_fn

Function score_fn(y_true, y_pred) -> scalar.

n_boot

Number of bootstrap replicates.

seed

Random seed.

Value

Named numeric list with apparent_error, bootstrap_error, error_632, error_632plus.

See also

ipred::errorest.

Examples

set.seed(1)
X <- matrix(rnorm(25 * 2), ncol = 2)
y <- rnorm(25)
model_fn <- function(Xt, yt) structure(
  list(coef = drop(solve(crossprod(Xt), crossprod(Xt, yt)))), class = "lm_lite")
predict.lm_lite <- function(object, newdata, ...) drop(newdata %*% object$coef)
registerS3method("predict", "lm_lite", predict.lm_lite)
score_fn <- function(yt, yp) mean((yt - yp)^2)
res <- bootstrap_632(X, y, model_fn, score_fn, n_boot = 10L)
res$error_632
#> [1] 1.278641