Reverse-mode autograd over a scalar tape (Geron Ch 10, morie.fn hmagrd)

loss receives a list of tape leaves and must return a tape node – returning a plain number means the tape was bypassed and no gradient exists, which is an error rather than a silent zero. Leaves support + - * / ^ plus exp log sqrt tanh sin cos and morie_gvar_relu() / morie_gvar_sigmoid(). One backward sweep fills every leaf's gradient at the cost of one forward pass.

Usage

morie_geron_autograd(loss, params)

Arguments

loss

Function(list of leaves) -> tape node.

params

Leaf values (numeric vector).

Value

List with grad, value, tape_size, params.

Examples

loss <- function(p) p[[1]] * p[[1]] + 3 * p[[2]]
r <- morie_geron_autograd(loss, c(2, 1))
str(r, max.level = 1)
#> List of 7
#>  $ grad     : num [1:2] 4 3
#>  $ value    : num 7
#>  $ tape_size: int 6
#>  $ params   : num [1:2] 2 1
#>  $ estimate : num 7
#>  $ n        : int 2
#>  $ method   : chr "Reverse-mode automatic differentiation over a scalar tape"