Score-matching objective — morie_esl_score

Fitting an energy model needs log Z, which is usually intractable. Score matching sidesteps it by matching the gradient of the log density with respect to x, in which Z – constant in x – has already cancelled. Hyvarinen's result is that the objective then depends only on the model, so the DATA's own score is never needed. That is the entire trick, and it is what lets an unnormalised model be fitted at all.

Usage

morie_esl_score_match(score, X, grad_score = NULL, eps = 1e-05)

Arguments

score: Function returning the model score, (n, d) to (n, d).
X: Samples, n by d.
grad_score: Optional function returning the diagonal of the score Jacobian; a central difference is used when absent.
eps: Finite-difference step.

Value

List with objective, trace_term, norm_term, per_point.

Details

The Jacobian trace costs d extra evaluations per point when grad_score is not supplied, which is what makes plain score matching expensive in high dimension and motivated the sliced and denoising variants.

References

Hyvarinen, A. (2005). Estimation of non-normalized statistical models by score matching. JMLR 6, 695-709.

Examples

set.seed(1)
X <- matrix(rnorm(400, 2), ncol = 1)
J <- sapply(c(0, 1, 2, 3), function(m) {
  morie_esl_score_match(function(Z) -(Z - m), X)$objective
})
which.min(J)
#> [1] 3