Deep kernel learning

A base kernel applied to the output of a neural network rather than to the raw inputs. The network is a fixed one-hidden-layer tanh map whose weights are supplied by the caller, so there is no training loop and both language arms follow the same deterministic map; an identity map recovers the plain RBF GP exactly, which is the identity the tests check.

Usage

Gpdkl(
  X,
  y,
  X_test = NULL,
  nn = NULL,
  lengthscale = 1,
  variance = 1,
  noise = 0.01
)

Arguments

X

Training inputs.

y

Training responses.

X_test

Test inputs; the training inputs by default.

nn

List with W1, b1, W2, b2 and optionally tanh; NULL means the identity map.

lengthscale, variance, noise

GP hyperparameters.

Value

List with estimate, mean, variance, loglik, features, n, method.

Details

Formula: k(x, x') -> k(g(x, w), g(x', w)).

References

Wilson, Hu, Salakhutdinov and Xing (2016), Deep kernel learning, AISTATS 2016, PMLR 51:370-378, eq. (1). arXiv:1511.02222

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
Gpdkl(V, V)
#> $estimate
#> [1] 0.9977513
#> 
#> $mean
#> [1] 0.9977513 1.9894557 2.9920056 3.9776475 4.9911686 5.9573435 7.0028346
#> [8] 7.9249102
#> 
#> $variance
#> [1] 0.009811305 0.009651150 0.009580999 0.009558199 0.009558199 0.009580999
#> [7] 0.009651150 0.009811305
#> 
#> $loglik
#> [1] -56.16417
#> 
#> $features
#> [1] 1
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "k(g(x), g(x')) with a fixed tanh feature map, Wilson et al. (2016) eq. (1)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"