EM for a Gaussian mixture

Fits a k-component Gaussian mixture by expectation-maximisation. The observed-data log-likelihood is checked to be non-decreasing at run time – a decrease is a bug, not slow convergence. reg ridges each covariance diagonal, which is what stops a component collapsing onto a single point and driving the likelihood to infinity.

Usage

morie_esl_em_gmm(
  X,
  k = 2,
  max_iter = 200L,
  tol = 1e-06,
  reg = 1e-06,
  seed = 0L
)

Arguments

X

Data matrix, n by p (a vector is treated as one column).

k

Number of components.

max_iter

Maximum EM iterations.

tol

Stop when the log-likelihood improves by less than this.

reg

Ridge added to each covariance diagonal.

seed

Seed for the k-means++ style initialisation.

Value

List with pi, mu, sigma, resp, labels, loglik, loglik_path, n_iter, converged, aic, bic.

References

Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.), Sec 8.5. Springer.

Examples

set.seed(1)
x <- c(rnorm(200, -4, 0.5), rnorm(200, 4, 0.5))
fit <- morie_esl_em_gmm(x, k = 2)
sort(round(as.numeric(fit$mu), 1))
#> [1] -4  4