Skip to contents

Variational autoencoder evidence lower bound (SGVB estimator)

Source: R/vaeber.R
Vaeber.Rd

SOURCE. Kingma, D.P. and Welling, M. (2014), "Auto-Encoding Variational Bayes", ICLR 2014; arXiv:1312.6114.

Usage

Vaeber(
  x,
  encoder = NULL,
  decoder = NULL,
  latent_dim = 2,
  n_samples = 64,
  decoder_scale = 1,
  skip = 0
)

Arguments

x

n-by-d data matrix.

encoder

list with mu and logvar, or NULL.

decoder

list with W and b, or NULL.

latent_dim

m, used only when encoder is NULL.

n_samples

Reparameterised draws per data point.

decoder_scale

s, the decoder standard deviation.

skip

Offset into the shared deterministic stream.

Value

List with elbo, kl, recon, recon_analytic, elbo_analytic, mc_error, mu, logvar, elbo_per_point, kl_per_point, recon_per_point, n, d, latent_dim, n_samples.

Details

The bound is Eq. (3), L = -KL(q_phi(z|x) || p_theta(z)) + E_{q_phi}[log p_theta(x|z)], estimated by the SGVB estimator of Eq. (7) with the reparameterisation z = mu + sigma * eps, eps ~ N(0, I). With a Gaussian encoder and a standard normal prior the KL is closed form (Appendix B / Eq. 10): -KL = (1/2) sum_j (1 + log sigma_j^2 - mu_j^2 - sigma_j^2).

DETERMINISM. eps comes from the shared deterministic normal stream (base-2 van der Corput through AS 241), not from a pseudo-random generator, so both language arms hold the SAME draws. That is this implementation's choice; the paper draws eps at random.

DECODER. p(x|z) = N(x; W z + b, s^2 I), the Gaussian decoder of Appendix C.2, for which the reconstruction term is also closed form: E_q[log p(x|z)] = -(1/2) sum_k [log(2 pi s^2) + ((x_k - (W mu + b)_k)^2

  • sum_j sigma_j^2 W[j,k]^2)/s^2], returned as recon_analytic. The Monte Carlo estimate must approach it as n_samples grows – an anchor that does not run through the other language arm.

References

Kingma, D.P. and Welling, M. (2014). Auto-Encoding Variational Bayes. ICLR 2014; arXiv:1312.6114.

Examples

Vaeber(matrix(c(0.5, -0.2, 0.3, 1.1), 2, 2), latent_dim = 1)$kl
#> [1] 0.07511324