Gaussian process regression with the hyperparameters integrated out

Conditional on the lengthscale and noise the predictive moments and the marginal likelihood are closed form; the hyperparameter average is taken by DETERMINISTIC quadrature over a grid weighted by that marginal likelihood, not by slice sampling, so both language arms land on the same numbers. A one-point grid collapses to the plain point-estimate GP, which is the identity the tests check.

Usage

Gpregb(
  X,
  y,
  kernel = NULL,
  X_test = NULL,
  lengthscales = NULL,
  noises = NULL,
  variance = 1
)

Arguments

X

Input matrix, one row per training point.

y

Training responses.

kernel

Unused placeholder kept for the interface.

X_test

Test inputs; the training inputs by default.

lengthscales

Grid of positive lengthscales.

noises

Grid of positive noise variances.

variance

Signal variance.

Value

List with estimate, mean, variance, weights, loglik, n, method.

Details

Formula: mu* = k*'(K + s2 I)^{-1} y; var* = k** - k*'(K + s2 I)^{-1} k*; log p(y) = -0.5 y'(K + s2 I)^{-1} y - 0.5 log|K + s2 I| - (n/2) log 2 pi.

References

Rasmussen and Williams (2006), Gaussian Processes for Machine Learning, MIT Press, eqs. (2.23)-(2.26) and (5.8); Murray and Adams (2010), Slice sampling covariance hyperparameters of latent Gaussian models, NIPS 23, arXiv:1006.0868.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
Gpregb(V, V)
#> $estimate
#> [1] 1.03375
#> 
#> $mean
#> [1] 1.033750 1.948732 2.955997 3.917180 4.882509 5.992567 7.066425 7.422079
#> 
#> $variance
#> [1] 0.05704030 0.03590985 0.03621433 0.03756869 0.03624703 0.03580711 0.03625938
#> [8] 0.11736106
#> 
#> $weights
#> [1] 7.971723e-24 1.667282e-21 5.718796e-09 4.535053e-08 2.353843e-01
#> [6] 7.646157e-01
#> 
#> $loglik
#> [1] -90.37081 -85.02776 -56.16417 -54.09351 -38.63120 -37.45305
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "GP posterior (R&W eqs. 2.23-2.26) averaged over a marginal-likelihood-weighted hyperparameter grid; Murray & Adams (2010)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"