Gaussian process classification by the Laplace approximation

With the probit likelihood, Newton's method on Psi(f) = log p(y|f) - 0.5 f' K^{-1} f gives the posterior mode; the averaged predictive probability is Phi(mu*/sqrt(1 + var*)). The mode is where the gradient vanishes, and the tests check that directly by differentiating Psi numerically rather than trusting the iteration.

Usage

Gpcla(
  X,
  y,
  X_test = NULL,
  kernel = NULL,
  lengthscale = 1,
  variance = 1,
  iters = 40
)

Arguments

X: Training inputs, one row per point.
y: Binary labels, 0 or 1.
X_test: Test inputs; the training inputs by default.
kernel: Unused placeholder kept for the interface.
lengthscale, variance: RBF kernel hyperparameters.
iters: Newton iterations.

Value

List with estimate, p, predicted, f_mode, latent_mean, latent_var, objective, n, method.

Details

Formula: f <- (I + K W)^{-1} K (W f + grad log p(y|f)); p* = Phi(mu* / sqrt(1 + var*)).

References

Williams and Barber (1998), Bayesian classification with Gaussian processes, IEEE TPAMI 20(12):1342-1351, doi:10.1109/34.735807 ; Rasmussen and Williams (2006), ch. 3, Algorithm 3.1 and eq. (3.82).

Examples

set.seed(1)
X <- matrix(rnorm(20), 10, 2)
y <- rbinom(10, 1, 0.5)
Gpcla(X, y)
#> $estimate
#> [1] 0.75409
#> 
#> $p
#>  [1] 0.7540900 0.6866827 0.7211266 0.6473575 0.6957122 0.7798748 0.5465700
#>  [8] 0.5784877 0.6158356 0.7956360
#> 
#> $predicted
#>  [1] 1 1 1 1 1 1 1 1 1 1
#> 
#> $f_mode
#>  [1] 0.8697810 0.5572947 0.7333531 0.4870682 0.5943978 0.9457134 0.1394422
#>  [8] 0.2301181 0.3361253 0.9763945
#> 
#> $latent_mean
#>  [1] 0.8697810 0.5572947 0.7333531 0.4870682 0.5943978 0.9457134 0.1394422
#>  [8] 0.2301181 0.3361253 0.9763945
#> 
#> $latent_var
#>  [1] 0.6009563 0.3123786 0.5651178 0.6586164 0.3471995 0.5015608 0.4204208
#>  [8] 0.3503811 0.3021162 0.3968493
#> 
#> $objective
#>  [1] -4.974344 -4.957895 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891
#>  [8] -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891
#> [15] -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891
#> [22] -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891
#> [29] -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891 -4.957891
#> [36] -4.957891 -4.957891 -4.957891 -4.957891 -4.957891
#> 
#> $n
#> [1] 10
#> 
#> $method
#> [1] "Newton mode of Psi(f) with probit likelihood; averaged prediction R&W eq. (3.82)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"