Nadaraya-Watson and local linear kernel regression (ESL eqs. 2.40-2.42)

Hastie, Tibshirani and Friedman (2009), The Elements of Statistical Learning, 2nd ed., Springer, Section 2.8.2, book p. 35 (PDF p. 54): K_lambda(x0, x) = (1/lambda) exp(-||x - x0||^2 / (2 lambda)) (2.40), fhat(x0) = sum_i K y_i / sum_i K (2.41), and the locally weighted criterion RSS(f_theta, x0) = sum_i K (y_i - f_theta(x_i))^2 (2.42) with f_theta(x) = theta0 (which reproduces 2.41) or theta0 + theta1 x, the popular local linear regression model.

Usage

Nadwat(x, y, x0, lam, degree = 0)

Arguments

x

N-by-p matrix of inputs.

y

N-vector of responses.

x0

target point.

lam

lambda of (2.40), the Gaussian variance.

degree

0 for Nadaraya-Watson, 1 for local linear.

Value

list: estimate, fit, weights, wsum, coef, lam, n, p, degree, method.

Details

In (2.40) lambda is the VARIANCE of the Gaussian, not its standard deviation, so the exponent divides by 2 lambda. The leading 1/lambda cancels in (2.41) but is kept in the returned weights.

Examples

Nadwat(1:5, c(1, 2, 3, 4, 5), 3, lam = 1)$fit
#> [1] 3