Predict from K Gaussian-process experts under a softmax gate

One GP over the whole input is both slow and wrong when different regions want different smoothness. Tresp fits several GPs and lets a gate decide per test point. The predictive variance picks up both the experts own uncertainty and their disagreement.

Usage

Gpmoe(X, y, X_test, K, ell = 1, noise = 1e-06)

Arguments

X: Training inputs.
y: Training targets.
X_test: Test inputs.
K: Number of experts.
ell: Squared-exponential length scale.
noise: Observation noise on each expert diagonal.

Value

List with estimate, mean, var, gate, n, K.

Details

Determinism: experts partition the training set into K contiguous blocks along the sorted first coordinate and the gate is a softmax of negative squared distance to each block centroid. No EM, no random initialisation.

Formula: mu(x) = sum_k pi_k(x) mu_k(x), var(x) = sum_k pi_k(x) [s_k^2(x) + mu_k(x)^2] - mu(x)^2.

References

Tresp, V. (2001). Mixtures of Gaussian processes. NIPS 13, 654-660.

Examples

Gpmoe(X = c(1, 2, 3, 4, 5, 6, 7, 8), y = c(1, 2, 3, 4, 5, 6, 7, 8),
  X_test = c(1, 2, 3, 4, 5, 6, 7, 8), K = 5L)
#> $estimate
#> [1] 4.077944
#> 
#> $mean
#> [1] 0.9979261 1.8826220 2.7533031 3.6784930 4.5310092 4.9782088 6.2223440
#> [8] 7.5796481
#> 
#> $var
#> [1] 0.004144379 0.167093914 0.473400041 0.445592170 1.140080445 2.378099066
#> [7] 1.695750070 1.728041521
#> 
#> $gate
#>              [,1]         [,2]         [,3]         [,4]         [,5]
#> [1,] 9.975272e-01 2.472623e-03 1.441407e-07 9.334484e-14 6.715352e-22
#> [2,] 8.806742e-01 1.191863e-01 1.395528e-04 1.815205e-09 2.622935e-16
#> [3,] 1.167832e-01 8.629176e-01 2.029388e-02 5.301949e-06 1.538796e-11
#> [4,] 1.677866e-03 6.768995e-01 3.197447e-01 1.677866e-03 9.781064e-08
#> [5,] 3.951616e-06 8.704014e-02 8.258139e-01 8.704014e-02 1.019135e-04
#> [6,] 1.375604e-09 1.654307e-03 3.152551e-01 6.673950e-01 1.569563e-02
#> [7,] 6.255554e-14 4.107395e-06 1.572157e-02 6.684982e-01 3.157762e-01
#> [8,] 4.050331e-19 1.452008e-09 1.116303e-04 9.533882e-02 9.045495e-01
#> 
#> $n
#> [1] 8
#> 
#> $K
#> [1] 5
#> 
#> $method
#> [1] "Mixture of GP experts (Tresp)"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"