Kernel ridge regression in its dual form

Written in the primal, a kernel fit needs the feature map; written in the dual it needs only inner products, so the feature space can be infinite-dimensional at no cost. The ridge penalty is not optional here: the Gram matrix of a smooth kernel is numerically singular for any sample worth fitting, so lambda is what makes the solve exist at all rather than a tuning nicety.

Usage

Krreg(x, y, x_eval = NULL, bandwidth = NULL, penalty = 1, kernel = "gaussian")

Arguments

x, y: Predictor and response, length n.
x_eval: Evaluation points; defaults to x.
bandwidth: Kernel bandwidth; Silverman's rule when NULL.
penalty: Ridge penalty, strictly positive.
kernel: One of "gaussian", "epanechnikov", "uniform".

Value

List with x_eval, y_hat, alpha, bandwidth, penalty, n_obs.

Details

Formula: alpha = (K + lambda I)^{-1} y with K_ij = K_h(x_i - x_j); prediction m(x0) = sum_i alpha_i K_h(x0 - x_i).

References

Saunders, C., Gammerman, A. and Vovk, V. (1998). Ridge regression learning algorithm in dual variables. Proceedings of the 15th International Conference on Machine Learning, 515-521.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
Krreg(V, V)
#> $x_eval
#> [1] 1 2 3 4 5 6 7 8
#> 
#> $y_hat
#> [1] 0.7550764 1.2735254 1.8811459 2.5419210 3.2280257 3.8108440 3.9828942
#> [8] 3.4640157
#> 
#> $alpha
#> [1] 0.2449236 0.7264746 1.1188541 1.4580790 1.7719743 2.1891560 3.0171058
#> [8] 4.5359843
#> 
#> $bandwidth
#> [1] 1.713024
#> 
#> $penalty
#> [1] 1
#> 
#> $n_obs
#> [1] 8
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"