Skip to contents

Hierarchical mixture of experts fitted by EM

Source: R/esl_native_hme.R
morie_esl_hme.Rd

Two-level HME (ESL eqs 9.25-9.30): softmax gates g_j(x), g_l|j(x) and Gaussian linear-regression (or logistic) experts; EM alternates branch posteriors with weighted least squares (weighted IRLS) for the experts and ridge-stabilised Newton fits of the gates on the posterior weights (Jordan & Jacobs 1994). An intercept is added; the start assigns softened blocks of the first input to the K^2 experts.

Usage

morie_esl_hme(
  X,
  y,
  K = 2,
  task = c("regression", "classification"),
  max_iter = 500,
  tol = 1e-10
)

Arguments

X

Predictors, N by p.

y

Response (0/1 for classification).

K

Branching factor.

task

"regression" or "classification".

max_iter, tol

EM controls.

Value

Named list: loglik, loglik_path, experts, sigma2, top_gate, sub_gates, fitted, iterations, converged.

References

Jordan, M. I. & Jacobs, R. A. (1994). Neural Computation 6, 181-214.

Examples

i <- 1:60
x <- ((7 * i) %% 59) / 59 * 4 - 2
morie_esl_hme(cbind(x), ifelse(x < 0, 2 * x, 0.5 - x), max_iter = 20)$loglik
#> [1] 773.7943