The variational bound over M inducing values, with the expectation taken by Gauss-Hermite quadrature whose nodes come from the Golub-Welsch tridiagonal (so both language arms generate the same nodes rather than carrying a hard-coded table). The variational parameters move by fixed-step gradient ascent with central-difference gradients – deterministic, unlike the stochastic natural gradients of the paper, so the two arms follow the same path. At the starting point q(u) = p(u) the KL term is exactly zero.
Usage
Gpcgs(
X,
y,
X_test = NULL,
M = 3,
lengthscale = 1,
variance = 1,
iters = 40,
step = 0.05,
nodes = 11,
jitter = 1e-08
)Arguments
- X
Training inputs, one row per point.
- y
Binary labels, 0 or 1.
- X_test
Test inputs; the training inputs by default.
- M
Number of inducing points, between 1 and n.
- lengthscale, variance
RBF hyperparameters.
- iters, step
Gradient ascent budget and step size.
- nodes
Gauss-Hermite nodes.
- jitter
Ridge added to K_mm.
Details
Formula: L = sum_i E_q[log p(y_i | f_i)] - KL(q(u) || p(u)), mu_i = a_i' m, var_i = k_ii - a_i'(K_mm - S) a_i.
References
Hensman, Matthews and Ghahramani (2015), Scalable variational Gaussian process classification, AISTATS 2015, PMLR 38:351-360. arXiv:1411.2005
Examples
set.seed(1)
X <- matrix(rnorm(20), 10, 2)
y <- rbinom(10, 1, 0.5)
Gpcgs(X, y)
#> $estimate
#> [1] 0.7523799
#>
#> $p
#> [1] 0.7523799 0.7222854 0.6246192 0.5006972 0.6664670 0.7243150 0.6328051
#> [8] 0.5791062 0.6252133 0.8032119
#>
#> $predicted
#> [1] 1 1 1 1 1 1 1 1 1 1
#>
#> $latent_mean
#> [1] 0.862478187 0.699224525 0.435465243 0.002471667 0.495020215 0.771629009
#> [7] 0.440978516 0.238906564 0.375471896 0.994255902
#>
#> $elbo
#> [1] -7.327594
#>
#> $kl
#> [1] 0.9610834
#>
#> $elbo_path
#> [1] -10.000000 -9.248986 -8.793380 -8.495062 -8.284688 -8.127042
#> [7] -8.003564 -7.903864 -7.821695 -7.753013 -7.695031 -7.645722
#> [13] -7.603554 -7.567334 -7.536108 -7.509107 -7.485697 -7.465353
#> [19] -7.447639 -7.432186 -7.418682 -7.406864 -7.396507 -7.387418
#> [25] -7.379433 -7.372410 -7.366226 -7.360777 -7.355971 -7.351728
#> [31] -7.347979 -7.344665 -7.341734 -7.339139 -7.336841 -7.334804
#> [37] -7.332999 -7.331397 -7.329976 -7.328714 -7.327594
#>
#> $n
#> [1] 10
#>
#> $method
#> [1] "variational bound of Hensman, Matthews & Ghahramani (2015) with Gauss-Hermite quadrature"
#>
#> attr(,"class")
#> [1] "morie_rich_result" "list"