Linear model fit by least squares (ESL eq. 2.6)

Hastie, Tibshirani and Friedman (2009), The Elements of Statistical Learning, 2nd ed., Springer, Chapter 2, Section 2.3.1, book pp. 11-12 (PDF pp. 30-31).

Usage

Olsfit(X, y, intercept = TRUE)

Arguments

X

N-by-p matrix of inputs, one observation per row.

y

N-vector of responses.

intercept

logical; include the constant column of (2.1).

Value

list: estimate, coefficients, intercept, fitted, residuals, rss, tss, r2, sigma2, df, n, p, method.

Details

Yhat = beta0 + sum_j X_j betahat_j (2.1); Yhat = X' betahat (2.2); RSS(beta) = (y - X beta)'(y - X beta) (2.4); the normal equations X'(y - X beta) = 0 (2.5); and betahat = (X'X)^-1 X'y (2.6).

Equation (2.15) f(x) ~= x'beta is the same linear form; (2.26) Y = X'beta + eps and (2.29) Y = f(X) + eps are the additive-error models this fit estimates; (2.32) RSS(theta) is (2.4) for a general parameterised f, which for a linear f_theta is exactly this computation. The normal equations are solved directly, so the answer is (2.6) itself.

Examples

Olsfit(cbind(c(1, 2, 3, 4)), c(2, 4, 6, 8))$coefficients
#> [1] 0 2