Reverse-mode autodiff over a chain of Jacobians (Geron Ch 10, graut)

Walks the chain from the output end, one vector-Jacobian product per node. Nodes are matrices (out_dim, in_dim) or functions g -> g J. intermediate_grads is output-end first, as in Python.

Usage

morie_geron_autograd_chain_rule(graph, grad_output)

Arguments

graph

List of nodes in FORWARD order.

grad_output

Seed gradient.

Value

List with grad_input, intermediate_grads, depth, grad_norm.

Examples

morie_geron_autograd_chain_rule(graph = c(1, 2, 3, 4, 5, 6, 7, 8), grad_output = 5L)
#> $grad_input
#> [1] 201600
#> 
#> $intermediate_grads
#> $intermediate_grads[[1]]
#> [1] 5
#> 
#> $intermediate_grads[[2]]
#> [1] 40
#> 
#> $intermediate_grads[[3]]
#> [1] 280
#> 
#> $intermediate_grads[[4]]
#> [1] 1680
#> 
#> $intermediate_grads[[5]]
#> [1] 8400
#> 
#> $intermediate_grads[[6]]
#> [1] 33600
#> 
#> $intermediate_grads[[7]]
#> [1] 100800
#> 
#> $intermediate_grads[[8]]
#> [1] 201600
#> 
#> $intermediate_grads[[9]]
#> [1] 201600
#> 
#> 
#> $depth
#> [1] 8
#> 
#> $grad_norm
#> [1] 201600
#> 
#> $estimate
#> [1] 201600
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "Reverse-mode automatic differentiation (chain rule)"
#>