Learning-rate finder heuristic: divergence point / safety (Geron Ch 9, morie.fn hmlrh)

Learning-rate finder heuristic: divergence point / safety (Geron Ch 9, morie.fn hmlrh)

Usage

morie_geron_learning_rate_heuristic(
  lr_curve,
  divergence_factor = 4,
  safety = 10
)

Arguments

lr_curve

Matrix/list of (lr, loss) pairs.

divergence_factor

Multiple of running min.

safety

Divisor.

Value

List with lr, lr_diverge, lr_min_loss, min_loss, diverged, estimate, n, method.

Examples

M <- matrix(c(1, 2, 3, 4, 5, 6), nrow = 2)
morie_geron_learning_rate_heuristic(M)
#> $lr
#> [1] 1
#> 
#> $lr_diverge
#> [1] NaN
#> 
#> $lr_min_loss
#> [1] 1
#> 
#> $min_loss
#> [1] 2
#> 
#> $diverged
#> [1] FALSE
#> 
#> $lrs
#> [1] 1 3 5
#> 
#> $losses
#> [1] 2 4 6
#> 
#> $estimate
#> [1] 1
#> 
#> $n
#> [1] 3
#> 
#> $method
#> [1] "LR-finder heuristic (divergence point / 10)"
#>