Skip to contents

Leave-one-out cross-validation

Source: R/bootstrap_methods.R
leave_one_out_cv.Rd

Convenience wrapper that calls .boot_cross_validate() with n_folds = length(y). rsample::loo_cv is the tidymodels equivalent (cross-referenced).

Usage

leave_one_out_cv(X, y, model_fn, score_fn)

Arguments

X

Numeric matrix or data.frame of predictors.

y

Numeric or factor outcome vector aligned with rows of X.

model_fn

Function (X, y) -> fitted-model used on each training fold.

score_fn

Function (y_true, y_pred) -> numeric returning a single performance metric.

Value

A morie_cv_result.

See also

rsample::loo_cv, caret::trainControl.

Examples

set.seed(1)
X <- matrix(rnorm(40), ncol = 2); y <- rnorm(20)
model_fn <- function(Xt, yt) stats::lm.fit(cbind(1, Xt), yt)
score_fn <- function(yt, yp) mean((yt - yp)^2)
predict.lm_lite2 <- function(object, newdata, ...) {
  drop(cbind(1, newdata) %*% object$coef)
}
registerS3method("predict", "lm_lite2", predict.lm_lite2)
res <- leave_one_out_cv(X, y,
  model_fn = function(Xt, yt) {
    fit <- stats::lm.fit(cbind(1, Xt), yt)
    structure(list(coef = fit$coefficients), class = "lm_lite2")
  },
  score_fn = score_fn)
str(res, max.level = 1)
#> List of 8
#>  $ scores    : num [1:20] 0.33886 0.62281 0.00728 1.03257 0.25964 ...
#>  $ mean_score: num 0.718
#>  $ se_score  : num 0.208
#>  $ ci_lower  : num 0.311
#>  $ ci_upper  : num 1.12
#>  $ n_folds   : int 20
#>  $ metric    : chr "custom"
#>  $ fold_sizes: int [1:20] 1 1 1 1 1 1 1 1 1 1 ...
#>  - attr(*, "class")= chr [1:2] "morie_cv_result" "list"