Power (1/t) learning-rate schedule (Geron Ch 4, morie.fn hmlrs)

eta_t = eta0 / (t + t0). The Robbins-Monro partial sums over 0..max(t) are returned so the divergence of sum eta and the convergence of sum eta^2 can be checked directly.

Usage

morie_geron_learning_rate_schedule(t, eta0, t0)

Arguments

t

Step or vector of steps.

eta0

Positive rate.

t0

Positive offset.

Value

List with eta, schedule, sum_eta, sum_eta_squared, eta0, t0, estimate, n.

Examples

morie_geron_learning_rate_schedule(c(0, 1, 4), 1, 2)
#> $eta
#> [1] 0.5000000 0.3333333 0.1666667
#> 
#> $schedule
#> [1] 0.5000000 0.3333333 0.2500000 0.2000000 0.1666667
#> 
#> $sum_eta
#> [1] 1.45
#> 
#> $sum_eta_squared
#> [1] 0.4913889
#> 
#> $eta0
#> [1] 1
#> 
#> $t0
#> [1] 2
#> 
#> $estimate
#> [1] 0.5
#> 
#> $n
#> [1] 3
#> 
#> $method
#> [1] "Power (1/t) learning-rate schedule"
#>