The penalty is written lambda * sum(w^2), so the gradient is
2 * lambda * w – twice what frameworks using the lambda/2 convention
apply for the same nominal lambda. The gradient is returned explicitly so
the convention in use is visible rather than assumed.
Usage
morie_esl_weight_decay(weights, lambda_ = 0.01, loss = 0, norm = "l2")
Arguments
- weights
Weight vector, excluding bias terms.
- lambda_
Penalty strength, non-negative.
- loss
Unpenalised loss, added to give objective.
- norm
"l2" (ridge) or "l1" (lasso; subgradient reported as 0 at zero).
Value
List with penalty, gradient, objective, effective_lambda.
References
Hastie, T., et al. (2009). ESL (2nd ed.), Sec 11.5.2. Springer.
Examples
morie_esl_weight_decay(c(3, -4), lambda_ = 0.5)$gradient
#> [1] 3 -4
