Adam update, theta-first signature (Geron Ch 11, morie.fn gradmo)

Same kernel as morie_geron_adam(); this mirrors gradmo's argument order and its POSITIVE step convention (theta_new = theta - step).

Usage

morie_geron_adam_update(
  theta,
  grad,
  m,
  s,
  t,
  eta,
  b1 = 0.9,
  b2 = 0.999,
  eps = 1e-08
)

Arguments

theta, grad, m, s

Equal-length numeric vectors.

t

1-based step counter.

eta

Positive learning rate.

b1, b2

Decay rates in [0, 1).

eps

Positive floor.

Value

List with theta_new, m_new, s_new, m_hat, s_hat, step.

Examples

morie_geron_adam_update(1, 0.1, 0, 0, t = 1, eta = 0.001)
#> $theta_new
#> [1] 0.999
#> 
#> $m_new
#> [1] 0.01
#> 
#> $s_new
#> [1] 1e-05
#> 
#> $m_hat
#> [1] 0.1
#> 
#> $s_hat
#> [1] 0.01
#> 
#> $step
#> [1] 0.0009999999
#> 
#> $t
#> [1] 1
#> 
#> $estimate
#> [1] 0.0009999999
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "Adam optimizer step with bias correction"
#>