Penalised logistic regression on a basis

Maximises the log-likelihood minus (lambda / 2) theta' Omega theta by penalised IRLS, theta = (N'WN + lambda Omega)^-1 N'Wz (ESL eqs 5.28-5.33); df is the trace of N (N'WN + lambda Omega)^-1 N'W.

Usage

morie_esl_penalized_logistic(N, y, Omega, lambda, max_iter = 100, tol = 1e-10)

Arguments

N

Basis matrix (include a constant column for an intercept).

y

0/1 response.

Omega

Penalty matrix.

lambda

Penalty, >= 0.

max_iter, tol

Newton controls (convergence on the linear predictor N theta, identified even when theta is not, or on a relative change below 1e-14 in the penalised log-likelihood for an ill-conditioned basis).

Value

Named list: theta, prob, loglik, penalized_loglik, df, iterations, converged.

References

Hastie, Tibshirani & Friedman (2009), sec. 5.6.

Examples

x <- sin(1:40)
N <- cbind(1, x, x^2)
morie_esl_penalized_logistic(N, as.integer(sin(3 * (1:40)) + x > 0.3), diag(c(0, 0, 1)), 0.5)$theta
#> [1] -0.09343721  3.69675937 -2.59459072