Momentum optimizer step (Geron Ch 11, morie.fn grmom)

v = beta v + g; theta -= eta v; terminal speedup 1/(1-beta).

Usage

morie_geron_momentum_update(theta, grad, v, eta, beta = 0.9)

Arguments

theta, grad, v

Same-shaped numeric objects.

eta

Positive learning rate.

beta

Momentum in [0, 1).

Value

List with theta_new, v_new, step, terminal_speedup.

Examples

morie_geron_momentum_update(c(1, -2), c(0.5, 0.25), c(0.1, -0.1),
    0.05, 0.8)
#> $theta_new
#> [1]  0.9710 -2.0085
#> 
#> $v_new
#> [1] 0.58 0.17
#> 
#> $step
#> [1] 0.0290 0.0085
#> 
#> $terminal_speedup
#> [1] 5
#> 
#> $beta
#> [1] 0.8
#> 
#> $eta
#> [1] 0.05
#> 
#> $estimate
#> [1] 0.03022003
#> 
#> $n
#> [1] 2
#>