Sparse autoencoder objective (Geron Ch 17, morie.fn grsae)

L = ||x - dec||^2 + lam ||h||_1; the L1 zeroes units outright.

Usage

morie_geron_sparse_autoencoder(x, hidden, decoded, lam = 0.001)

Arguments

x

Input batch.

hidden

Code activations.

decoded

Reconstruction.

lam

Non-negative L1 weight.

Value

List with loss, reconstruction_loss, l1_penalty, sparsity, mean_activation, code_size.

Examples

morie_geron_sparse_autoencoder(x = c(1, 2, 3, 4, 5, 6, 7, 8),
  hidden = c(1, 2, 3, 4, 5, 6, 7, 8), decoded = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $loss
#> [1] 0.036
#> 
#> $reconstruction_loss
#> [1] 0
#> 
#> $l1_penalty
#> [1] 0.036
#> 
#> $sparsity
#> [1] 0
#> 
#> $mean_activation
#> [1] 4.5
#> 
#> $code_size
#> [1] 8
#> 
#> $estimate
#> [1] 0.036
#> 
#> $n
#> [1] 1
#>