Unsupervised pretraining: learn representation via reconstruction before labels

Fits an autoencoder on X_unlab (via morie_geron_autoencoder), pushes X_lab through the frozen encoder, fits a linear head on codes, compares exact leave-one-out MSE against a raw-feature control.

Usage

morie_geron_unsupervised_pretraining(X_unlab, X_lab, y_lab, bottleneck = 1)

Arguments

X_unlab

Unlabeled pool (m, d).

X_lab

Labeled inputs (n, d).

y_lab

Labels, length n.

bottleneck

Code width.

Value

list with encoder, encode, codes, theta, theta_control, pretrained_loo, control_loo, pretrained_train_mse, control_train_mse, gain, recon_error, explained_variance_ratio, estimate, n, method.

Examples

morie_geron_unsupervised_pretraining(X_unlab = c(1, 2, 3, 4, 5, 6, 7, 8),
  X_lab = c(1, 2, 3, 4, 5, 6, 7, 8), y_lab = c(1, 2, 3, 4, 5, 6, 7, 8))
#> $encoder
#>      [,1]
#> [1,]    1
#> 
#> $encode
#> function (Xnew) 
#> {
#>     B <- if (is.matrix(Xnew)) 
#>         Xnew
#>     else matrix(as.numeric(Xnew), nrow = 1)
#>     .morie_gr_need(ncol(B) == d, "encode: feature count mismatch")
#>     sweep(B, 2, mean_, "-") %*% W2
#> }
#> <bytecode: 0x56216ab7b568>
#> <environment: 0x562159906128>
#> 
#> $codes
#>      [,1]
#> [1,] -3.5
#> [2,] -2.5
#> [3,] -1.5
#> [4,] -0.5
#> [5,]  0.5
#> [6,]  1.5
#> [7,]  2.5
#> [8,]  3.5
#> 
#> $theta
#> [1] 4.5 1.0
#> 
#> $theta_control
#> [1] 4.940492e-15 1.000000e+00
#> 
#> $pretrained_loo
#> [1] 0
#> 
#> $control_loo
#> [1] 1.232577e-29
#> 
#> $pretrained_train_mse
#> [1] 0
#> 
#> $control_train_mse
#> [1] 5.676101e-30
#> 
#> $gain
#> [1] 1.232577e-29
#> 
#> $recon_error
#> [1] 0
#> 
#> $explained_variance_ratio
#> [1] 1
#> 
#> $estimate
#> [1] 0
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Autoencoder pretraining on the unlabeled pool (hmaen), frozen encoder + linear head, LOO-scored against a raw-feature control"
#>