Linear basis expansion fitted by least squares (ESL eqs. 2.30, 2.43)

Hastie, Tibshirani and Friedman (2009), The Elements of Statistical Learning, 2nd ed., Springer, Sections 2.6.1 and 2.8.3, book pp. 30 and 35-36 (PDF pp. 49, 54-55): f_theta(x) = sum_k h_k(x) theta_k (2.30) = sum_m theta_m h_m(x) (2.43), minimising RSS(theta) = sum_i (y_i - f_theta(x_i))^2 (2.32).

Usage

Basisexp(x, y, kind = "poly", M = 3, knots = NULL)

Arguments

x

N-vector of scalar inputs.

y

N-vector of responses.

kind

one of "poly", "trig", "spline".

M

polynomial degree, or number of harmonics for "trig".

knots

knots t_m for the linear spline basis.

Value

list: estimate, theta, basis, fitted, residuals, rss, tss, r2, K, n, method.

Details

The dictionaries are the ones the book names: polynomial (x^k), trigonometric (cos/sin harmonics), and the linear spline basis b1 = 1, b2 = x, b_{m+2}(x) = (x - t_m)_+. The basis functions carry no hidden parameters, so the minimisation is closed form and is solved by the normal equations rather than by search.

Examples

Basisexp(1:6, c(1, 4, 9, 16, 25, 36), kind = "poly", M = 2)$theta
#> [1] -5.511479e-14  4.331231e-14  1.000000e+00