1cycle learning-rate policy (Geron Ch 11, morie.fn hml1c)

1cycle learning-rate policy (Geron Ch 11, morie.fn hml1c)

Usage

morie_geron_one_cycle(t, T, lr_max, lr_min, mom_max = 0.95, mom_min = 0.85)

Arguments

t

Step (0-based).

T

Cycle length.

lr_max, lr_min

Rate bounds.

mom_max, mom_min

Momentum bounds.

Value

List with lr, momentum, lr_schedule, momentum_schedule, peak_step, phase, estimate, n, method.

Examples

morie_geron_one_cycle(t = 0, T = 5, lr_max = 0.5, lr_min = 0.1)
#> $lr
#> [1] 0.1
#> 
#> $momentum
#> [1] 0.95
#> 
#> $lr_schedule
#> [1] 0.1 0.3 0.5 0.3 0.1
#> 
#> $momentum_schedule
#> [1] 0.95 0.90 0.85 0.90 0.95
#> 
#> $peak_step
#> [1] 2
#> 
#> $phase
#> [1] "warmup"
#> 
#> $estimate
#> [1] 0.1
#> 
#> $n
#> [1] 5
#> 
#> $method
#> [1] "1cycle learning-rate policy"
#>