Classification MLP (Geron Ch 10, morie.fn hmclsn) — morie_geron_classification

ReLU hidden stack, softmax output, trained by full backpropagation. Weights come from the shared LCG with He scaling sqrt(2/fan_in), one layer after another out of a SINGLE continuing stream, so the init matches Python draw-for-draw. The final loss is delegated to morie_geron_cross_entropy_cost on the augmented last layer, as in the Python module.

Usage

morie_geron_classification_mlp(
  X,
  y,
  hidden_sizes = c(4),
  epochs = 100,
  lr = 0.1,
  seed = 0
)

Arguments

X, y: Design and labels.
hidden_sizes: Integer vector.
epochs: Training epochs.
lr: Learning rate.
seed: LCG seed.

Value

List with weights, biases, loss_history, final_loss, predictions, probabilities, logits, accuracy, classes, layer_sizes, n_params, estimate, n.

Examples

XC <- matrix(c(1, 5, 2, 4, 3, 6, 6, 1, 7, 2, 8, 0.5), nrow = 6,
    byrow = TRUE)
YC <- c(0, 0, 1, 1, 2, 2)
morie_geron_classification_mlp(XC, YC, hidden_sizes = 3, epochs = 40,
    lr = 0.2, seed = 5)
#> $weights
#> $weights[[1]]
#>           [,1]        [,2]       [,3]
#> [1,] -1.107114 -0.87840672 -0.8900139
#> [2,]  1.200233  0.05251963 -0.4837208
#> 
#> $weights[[2]]
#>              [,1]       [,2]       [,3]
#> [1,] -0.073317979 -0.4320191 -1.5564017
#> [2,]  0.163239643 -0.8132548 -0.4926123
#> [3,] -0.005274596  0.4078709 -0.9945241
#> 
#> 
#> $biases
#> $biases[[1]]
#> [1]  0.107516 -0.284934  0.000000
#> 
#> $biases[[2]]
#> [1] -0.75599081  0.06091393  0.69507689
#> 
#> 
#> $loss_history
#>  [1] 9.9808378 0.8905009 0.8579683 0.8466659 0.8387949 0.8318409 0.8253663
#>  [8] 0.8192761 0.8135368 0.8081261 0.8030246 0.7982136 0.7936756 0.7893938
#> [15] 0.7853522 0.7815357 0.7779301 0.7745218 0.7712983 0.7682476 0.7653584
#> [22] 0.7626203 0.7600233 0.7575583 0.7552166 0.7529901 0.7508712 0.7488528
#> [29] 0.7469283 0.7450914 0.7433365 0.7416580 0.7400509 0.7385105 0.7370324
#> [36] 0.7356125 0.7342468 0.7329318 0.7316642 0.7304408
#> 
#> $final_loss
#> [1] 0.7292586
#> 
#> $predictions
#> [1] 0 0 0 2 2 2
#> 
#> $probabilities
#>           [,1]      [,2]        [,3]
#> [1,] 0.7251946 0.2729469 0.001858491
#> [2,] 0.5155640 0.4439654 0.040470591
#> [3,] 0.6439093 0.3486667 0.007424036
#> [4,] 0.1327818 0.3005494 0.566668805
#> [5,] 0.1327818 0.3005494 0.566668805
#> [6,] 0.1327818 0.3005494 0.566668805
#> 
#> $logits
#>            [,1]        [,2]       [,3]
#> [1,] -1.1226956 -2.09985834 -7.0893705
#> [2,] -0.9535256 -1.10304038 -3.4982115
#> [3,] -1.0483515 -1.66179306 -5.5111866
#> [4,] -0.7559908  0.06091393  0.6950769
#> [5,] -0.7559908  0.06091393  0.6950769
#> [6,] -0.7559908  0.06091393  0.6950769
#> 
#> $accuracy
#> [1] 0.6666667
#> 
#> $classes
#> [1] 0 1 2
#> 
#> $layer_sizes
#> [1] 2 3 3
#> 
#> $n_params
#> [1] 21
#> 
#> $estimate
#> [1] 0.6666667
#> 
#> $n
#> [1] 6
#> 
#> $method
#> [1] "ReLU MLP trained by backpropagation; loss delegated to hmcec"
#>