Epsilon-insensitive support vector regression

MVSML (2022) sec. 9.6 p.369 introduces SVR but states that detailed SVR theory is not covered, referring to Burges (1998). The equations here are therefore Smola & Scholkopf (2004): the \(\epsilon\)-insensitive loss (4), the dual (10), the support vector expansion (11) and the offset interval (16). Fixed iteration count, no tolerance-driven early exit.

Usage

Svmep(
  X,
  y,
  C,
  eps,
  n_iter = 4000L,
  kernel = "linear",
  gamma = NULL,
  degree = 2,
  coef0 = 1
)

Arguments

X

Inputs, n by p.

y

Continuous responses.

C

Trade-off constant of the primal.

eps

Half-width of the insensitive tube.

n_iter

Projected-gradient steps.

kernel, gamma, degree, coef0

Passed to the shared Gram builder.

Value

Named list with estimate (mean epsilon-insensitive loss), w, b, alpha, alpha_star, theta, support_vectors, fitted, loss, objective, method.

References

Smola & Scholkopf (2004) Stat Comput 14:199-222.

Examples

Svmep(matrix(1:5, ncol = 1), c(1.2, 1.9, 3.2, 3.9, 5.1), 1, 0.1)
#> $estimate
#> [1] 0.03542857
#> 
#> $w
#> [1] 0.9614286
#> 
#> $b
#> [1] 0.1735714
#> 
#> $alpha
#> [1] 0.000000 0.000000 1.000000 0.000000 0.612363
#> 
#> $alpha_star
#> [1] 0.0000000 1.0000000 0.0000000 0.7750966 0.0000000
#> 
#> $theta
#> [1]  0.0000000 -1.0000000  1.0000000 -0.7750966  0.6123630
#> 
#> $support_vectors
#> [1] 1 2 3 4
#> 
#> $fitted
#> [1] 1.135000 2.096429 3.057857 4.019286 4.980714
#> 
#> $loss
#> [1] 0.00000000 0.09642857 0.04214286 0.01928571 0.01928571
#> 
#> $objective
#> [1] 0.5992561
#> 
#> $method
#> [1] "epsilon-insensitive SVR (Smola & Scholkopf 2004 eq. 10)"
#>