Forward pass, then the gradients by the ESL eq. (11.5) backward recurrence.
The content of backprop is the chain rule REUSING the output error: it is
computed once and propagated, rather than recomputed per weight, which turns
an O(number of weights) finite-difference cost into a single extra sweep.
Usage
morie_esl_backprop(X, y, weights, task = "regression")
Arguments
- X
Inputs, n by p, already scaled as the network expects.
- y
Targets; numeric for regression, 1-based class indices otherwise.
- weights
List with alpha (p by M), alpha0, beta (M by K), beta0.
- task
"regression" or "classification".
Value
List with grad_alpha, grad_alpha0, grad_beta, grad_beta0,
delta, hidden, output, loss.
Details
Returns the gradients rather than applying them, so the choice of optimiser
stays separate from the derivative.
References
Hastie, T., et al. (2009). ESL (2nd ed.), Sec 11.4. Springer.
Examples
set.seed(1)
X <- matrix(rnorm(120), ncol = 3)
y <- rnorm(40)
W <- list(
alpha = matrix(rnorm(12) * 0.5, 3, 4), alpha0 = numeric(4),
beta = matrix(rnorm(4) * 0.5, 4, 1), beta0 = 0
)
dim(morie_esl_backprop(X, y, W)$grad_alpha)
#> [1] 3 4
