Linear undercomplete autoencoder solved by SVD (Geron Ch 18, hmaen)

For a linear autoencoder the optimum is spanned by the top principal components, so the encoder is the leading right singular vectors and the decoder their transpose. SVD sign is arbitrary and differs between LAPACK builds, so codes and encoder carry a per-column sign convention; reconstruction, recon_error and explained_variance_ratio are sign-invariant and are what parity tests should assert.

Usage

morie_geron_autoencoder(X, bottleneck, center = TRUE)

Arguments

X

Training data (n, d).

bottleneck

Code width in [1, d].

center

Subtract feature means (this is what makes it PCA).

Value

List with encoder, decoder, codes, reconstruction, recon_error, explained_variance_ratio, mean, encode, decode.

Examples

morie_geron_autoencoder(matrix(c(0, 0, 1, 1.1, 2, 2.3, 3, 2.9),
    4, 2, byrow = TRUE), 1)
#> $encoder
#>          [,1]      [,2]
#> [1,] 0.707553 0.7066602
#> 
#> $decoder
#>           [,1]
#> [1,] 0.7075530
#> [2,] 0.7066602
#> 
#> $codes
#>            [,1]
#> [1,] -2.1743194
#> [2,] -0.6894401
#> [3,]  0.8661052
#> [4,]  1.9976544
#> 
#> $reconstruction
#>             [,1]       [,2]
#> [1,] -0.03844634 0.03849492
#> [2,]  1.01218453 1.08780007
#> [3,]  2.11281537 2.18704210
#> [4,]  2.91344644 2.98666291
#> 
#> $recon_error
#> [1] 0.01093651
#> 
#> $explained_variance_ratio
#> [1] 0.995619919 0.004380081
#> 
#> $mean
#> [1] 1.500 1.575
#> 
#> $encode
#> function (Xnew) 
#> {
#>     B <- if (is.matrix(Xnew)) 
#>         Xnew
#>     else matrix(as.numeric(Xnew), nrow = 1)
#>     .morie_gr_need(ncol(B) == d, "encode: feature count mismatch")
#>     sweep(B, 2, mean_, "-") %*% W2
#> }
#> <environment: 0x562184212280>
#> 
#> $decode
#> function (C) 
#> {
#>     Cm <- if (is.matrix(C)) 
#>         C
#>     else matrix(as.numeric(C), nrow = 1)
#>     .morie_gr_need(ncol(Cm) == k, "decode: code width mismatch")
#>     sweep(Cm %*% W1, 2, mean_, "+")
#> }
#> <environment: 0x562184212280>
#> 
#> $estimate
#> [1] 0.01093651
#> 
#> $n
#> [1] 4
#> 
#> $method
#> [1] "Linear undercomplete autoencoder solved in closed form by SVD (equals PCA)"
#>