Cosine annealing LR schedule (Geron Ch 11, morie.fn grlrco)

eta_t = eta_min + 0.5 (eta_max - eta_min)(1 + cos(pi t / T)); the full 0..T curve is returned and its monotonicity checked.

Usage

morie_geron_lr_cosine_annealing(eta_min, eta_max, t, T)

Arguments

eta_min, eta_max

Non-negative rates with eta_min <= eta_max.

t

Step in [0, T].

T

Positive horizon.

Value

List with eta, schedule, halfway_value, is_monotone_decreasing.

Examples

morie_geron_lr_cosine_annealing(0.001, 0.1, 3, 10)
#> $eta
#> [1] 0.07959537
#> 
#> $schedule
#>  [1] 0.100000000 0.097577298 0.090546341 0.079595370 0.065796341 0.050500000
#>  [7] 0.035203659 0.021404630 0.010453659 0.003422702 0.001000000
#> 
#> $eta_min
#> [1] 0.001
#> 
#> $eta_max
#> [1] 0.1
#> 
#> $halfway_value
#> [1] 0.0505
#> 
#> $is_monotone_decreasing
#> [1] TRUE
#> 
#> $t
#> [1] 3
#> 
#> $T
#> [1] 10
#> 
#> $estimate
#> [1] 0.07959537
#> 
#> $n
#> [1] 11
#>