SOURCE. Hensman, J., Matthews, A. and Ghahramani, Z. (2015), "Scalable Variational Gaussian Process Classification", Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, PMLR 38:351-360.
Usage
Vargpc(
X,
y,
X_test = NULL,
m_inducing = 4,
lengthscale = 1,
variance = 1,
n_quad = 20,
steps = 40,
step_size = 0.1,
jitter = 1e-08
)Arguments
- X
n-by-d training inputs.
- y
Binary labels in {0, 1}.
- X_test
Test inputs, or NULL to predict at X.
- m_inducing
M in 1..n.
- lengthscale, variance
RBF hyperparameters, positive, fixed.
- n_quad
Gauss-Hermite nodes, at least 2.
- steps
Ascent steps.
- step_size
Initial step, positive.
- jitter
Added to the diagonal of K_mm.
Value
List with prob, pred, fit_prob,
fit_pred, elbo, elbo_path,
elbo_monotone, kl, m, S, Z,
quad_check, n, d, m_inducing.
Details
The inducing-point posterior q(u) = N(m, S) at inputs Z induces q(f_n) = N(a_n' m, k_nn - a_n' k_mn + a_n' S a_n) with a_n = K_mm^{-1} k_mn (Eqs. 6-8), and the bound maximised is ELBO = sum_n E_{q(f_n)}[log p(y_n|f_n)] - KL(q(u) || p(u)), the Bernoulli expectation done by Gauss-Hermite quadrature. The Gaussian KL is closed form: (1/2)[tr(K^{-1} S) + m' K^{-1} m - M + log|K| - log|S|].
QUADRATURE. Nodes and weights are not tabulated: they come from
Golub-Welsch, the eigenvalues of the symmetric tridiagonal Jacobi
matrix with zero diagonal and off-diagonal sqrt(k/2), weights
sqrt(pi) v_{1i}^2. quad_check is the error in
E_{N(0,1)}[f^2] = 1, which a wrong node set cannot pass.
OPTIMISATION. m and the Cholesky factor of S are fitted by gradient ascent with central finite differences and deterministic backtracking. The paper uses analytic gradients and stochastic optimisation; a deterministic derivative-free ascent is used here so both language arms perform the same arithmetic. That substitution is this implementation's choice, stated rather than attributed. ELBO monotonicity is asserted.
INDUCING POINTS Z sit at evenly spaced ranks of the data ordered by its first coordinate, not sampled.