Expected prediction error of least squares at a test point (ESL eq. 2.27)

Hastie, Tibshirani and Friedman (2009), The Elements of Statistical Learning, 2nd ed., Springer, Section 2.5, book pp. 24-26 (PDF pp. 43, 45). For Y = X'beta + eps of equation (2.26), EPE(x0) = Var(y0|x0) + Var_T(yhat0) + Bias^2(yhat0) = sigma^2 + E_T x0'(X'X)^-1 x0 sigma^2 + 0^2 (2.27), and for large N with E(X) = 0, E_x0 EPE(x0) ~ sigma^2 (p/N) + sigma^2 (2.28).

Usage

Epeols(X, y, x0, sigma2 = NULL)

Arguments

X

N-by-p training design; (2.26) has no intercept.

y

N-vector of responses, used only to estimate sigma^2.

x0

p-vector test point.

sigma2

optional noise variance.

Value

list: estimate, epe, variance, bias2, sigma2, leverage, approx, n, p, method.

Details

The bias term is identically zero because least squares is unbiased under (2.26); it is returned rather than dropped. sigma^2 defaults to the residual estimate RSS/(N - p) from the same design.

Examples

Epeols(cbind(rep(1, 5), 1:5), c(1, 3, 2, 5, 4), c(1, 3))$epe
#> [1] 1.44