Backpropagation through a feed-forward net (Geron Ch 9, morie.fn hmbp)

delta_L = grad_a L * phi'(z_L); delta_l = (W_{l+1} delta_{l+1}) phi'(z_l). weights entries are either a matrix W or a list(W, b). Class labels for loss = "ce" are 0-based, as in Python.

Usage

morie_geron_backpropagation(X, y, weights, activations, loss = "mse")

Arguments

X

Inputs (n, d0).

y

Targets or class labels.

weights

List of matrices or list(W, b) pairs.

activations

Character vector, one per layer.

loss

"mse" or "ce".

Value

List with grads_W, grads_b, deltas, loss, output.

Examples

W1 <- matrix(c(1, -1), 1, 2)
W2 <- matrix(c(2, 3), 2, 1)
morie_geron_backpropagation(matrix(c(1, 2), ncol = 1), matrix(c(0,
    1), ncol = 1), list(W1, W2), c("relu", "identity"))
#> $grads_W
#> $grads_W[[1]]
#>      [,1] [,2]
#> [1,]   16    0
#> 
#> $grads_W[[2]]
#>      [,1]
#> [1,]    8
#> [2,]    0
#> 
#> 
#> $grads_b
#> $grads_b[[1]]
#> [1] 10  0
#> 
#> $grads_b[[2]]
#> [1] 5
#> 
#> 
#> $deltas
#> $deltas[[1]]
#>      [,1] [,2]
#> [1,]    4    0
#> [2,]    6    0
#> 
#> $deltas[[2]]
#>      [,1]
#> [1,]    2
#> [2,]    3
#> 
#> 
#> $loss
#> [1] 6.5
#> 
#> $output
#>      [,1]
#> [1,]    2
#> [2,]    4
#> 
#> $activations_out
#> $activations_out[[1]]
#>      [,1]
#> [1,]    1
#> [2,]    2
#> 
#> $activations_out[[2]]
#>      [,1] [,2]
#> [1,]    1    0
#> [2,]    2    0
#> 
#> $activations_out[[3]]
#>      [,1]
#> [1,]    2
#> [2,]    4
#> 
#> 
#> $pre_activations
#> $pre_activations[[1]]
#>      [,1] [,2]
#> [1,]    1   -1
#> [2,]    2   -2
#> 
#> $pre_activations[[2]]
#>      [,1]
#> [1,]    2
#> [2,]    4
#> 
#> 
#> $estimate
#> [1] 6.5
#> 
#> $n
#> [1] 2
#> 
#> $method
#> [1] "Backpropagation of the mse loss through 2 layers"
#>