Kernel support vector machine — morie_esl_svm

Fits the dual soft-margin SVM by sequential minimal optimisation. Only the kernel enters, never the feature map. Observations with alpha = 0 do not appear in the decision function at all, which is what makes the solution sparse in the training set; small C gives a wide margin and many support vectors, large C a narrow one and few.

Usage

morie_esl_svm_kernel(
  X,
  y,
  C = 1,
  kernel = "rbf",
  gamma = NULL,
  degree = 3,
  coef0 = 1,
  newdata = NULL,
  tol = 0.001,
  max_passes = 50L,
  seed = 0L
)

Arguments

X: Training predictors, n by p.
y: Labels; any two distinct values, mapped internally to -1/+1.
C: Box constraint, positive.
kernel: "rbf", "linear", "poly" or "sigmoid".
gamma: Kernel width for rbf/sigmoid; defaults to 1/p.
degree, coef0: Polynomial and sigmoid kernel parameters.
newdata: Points to classify; defaults to X.
tol, max_passes, seed: SMO controls.

Value

List with alpha, b, support (1-based indices), decision, class, accuracy, and dual_gap_check (which the equality constraint forces to zero).

References

Platt, J. (1998). Sequential minimal optimization. MSR-TR-98-14. Hastie, T., et al. (2009). ESL (2nd ed.), Sec 12.3. Springer.

Examples

set.seed(1)
X <- rbind(matrix(rnorm(120, -2), ncol = 2), matrix(rnorm(120, 2), ncol = 2))
y <- rep(c(-1, 1), each = 60)
fit <- morie_esl_svm_kernel(X, y, C = 1, kernel = "rbf", seed = 1L)
abs(fit$dual_gap_check) < 1e-9
#> [1] TRUE