Nesterov accelerated gradient (Geron Ch 11, morie.fn hmnag)

The gradient is read at the look-ahead point theta - eta beta v. Pass grads as a function to get the true look-ahead evaluation; pass a vector for the pre-evaluated form (the look-ahead point is still reported).

Usage

morie_geron_nesterov(grads, v = NULL, beta = 0.9, eta = 0.001, theta = NULL)

Arguments

grads

Gradient vector, or function(theta) -> gradient.

v

Velocity; default zeros.

beta

Decay in [0, 1).

eta

Positive rate.

theta

Parameters.

Value

List with theta, theta_next, v, lookahead, gradient, step, estimate, n.

Examples

V <- c(1, 2, 3, 4, 5, 6, 7, 8)
morie_geron_nesterov(V)
#> $theta
#> [1] -0.001 -0.002 -0.003 -0.004 -0.005 -0.006 -0.007 -0.008
#> 
#> $theta_next
#> [1] -0.001 -0.002 -0.003 -0.004 -0.005 -0.006 -0.007 -0.008
#> 
#> $v
#> [1] 1 2 3 4 5 6 7 8
#> 
#> $lookahead
#> [1] 0 0 0 0 0 0 0 0
#> 
#> $gradient
#> [1] 1 2 3 4 5 6 7 8
#> 
#> $step
#> [1] -0.001 -0.002 -0.003 -0.004 -0.005 -0.006 -0.007 -0.008
#> 
#> $estimate
#> [1] -0.001 -0.002 -0.003 -0.004 -0.005 -0.006 -0.007 -0.008
#> 
#> $n
#> [1] 8
#> 
#> $method
#> [1] "Nesterov accelerated gradient with look-ahead evaluation"
#>