Cosine annealing schedule (Geron Ch 11, morie.fn hmlcos)

eta_t = eta_min + 0.5 (eta_max - eta_min) (1 + cos(pi t / T)): smooth at both ends, leaving eta_max and reaching eta_min with zero slope.

Usage

morie_geron_cosine_annealing(t, T, eta_max, eta_min = 0)

Arguments

t

Step(s) in 0..T.

T

Cycle length.

eta_max, eta_min

Rate bounds.

Value

List with eta, schedule, T, eta_max, eta_min, estimate, n.

Examples

morie_geron_cosine_annealing(t = 5L, T = 5L, eta_max = 5L)
#> $eta
#> [1] 0
#> 
#> $schedule
#> [1] 5.0000000 4.5225425 3.2725425 1.7274575 0.4774575 0.0000000
#> 
#> $T
#> [1] 5
#> 
#> $eta_max
#> [1] 5
#> 
#> $eta_min
#> [1] 0
#> 
#> $estimate
#> [1] 0
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "Cosine annealing learning-rate schedule"
#>