Backpropagation from stored activations (Geron Ch 9/10, morie.fn grbp)

Squared-error loss L = 0.5 sum ||a_L - y||^2, derivatives taken from the POST-activation values, which is why only activations are needed.

Usage

morie_geron_backpropagation_gradient(
  activations,
  weights,
  y_true,
  activation = "sigmoid",
  output_activation = NULL
)

Arguments

activations

List of matrices, one longer than weights.

weights

List of weight matrices.

y_true

Targets matching the last activation.

activation

Hidden nonlinearity.

output_activation

Output nonlinearity; defaults to activation.

Value

List with grad_weights, grad_biases, deltas, loss, grad_norm.

Examples

morie_geron_backpropagation_gradient(list(matrix(c(1, 2), 1,
    2), matrix(3, 1, 1)), list(matrix(c(1, 1), 2, 1)), matrix(1,
    1, 1), activation = "sigmoid")
#> $grad_weights
#> $grad_weights[[1]]
#>      [,1]
#> [1,]  -12
#> [2,]  -24
#> 
#> 
#> $grad_biases
#> $grad_biases[[1]]
#> [1] -12
#> 
#> 
#> $deltas
#> $deltas[[1]]
#>      [,1]
#> [1,]  -12
#> 
#> 
#> $loss
#> [1] 2
#> 
#> $grad_norm
#> [1] 26.83282
#> 
#> $activation
#> [1] "sigmoid"
#> 
#> $output_activation
#> [1] "sigmoid"
#> 
#> $estimate
#> [1] 2
#> 
#> $n
#> [1] 1
#> 
#> $method
#> [1] "Backpropagation through a feed-forward net"
#>