Lasso by cyclic coordinate descent

The L1 penalty is not differentiable at zero, and that is the point: the subgradient condition lets a coefficient sit exactly at zero over a whole range of correlations, so the fit selects as it shrinks. Cyclic coordinate descent exploits the fact that each one-dimensional subproblem has the closed-form soft-threshold solution, so no quadratic program is needed.

Usage

Esllso(X, y, lambda_, max_iter = 10000, tol = 1e-12)

Arguments

X

Design matrix, n by p.

y

Response of length n.

lambda_

Penalty, non-negative.

max_iter

Maximum sweeps.

tol

Convergence tolerance on the largest coefficient move.

Value

List with estimate, beta, n_nonzero, active_set, objective, iterations, converged, lambda, n, p.

Details

Columns with no variation are left unpenalised: shrinking an intercept towards zero is a statement about the origin of the response scale, not about model complexity.

Formula: min_beta 0.5 ||y - X beta||^2 + lambda ||beta||_1, cycled as \(beta_j <- S(x_j primer + ||x_j||^2 beta_j, lambda) / ||x_j||^2\) with S the soft-threshold – Tibshirani (1996); Hastie, Tibshirani and Friedman, The Elements of Statistical Learning, section 3.4.2. The penalty is on the same scale as n times glmnet's

References

Tibshirani, R. (1996). Journal of the Royal Statistical Society Series B 58(1):267-288. doi:10.1111/j.2517-6161.1996.tb02080.x .

Examples

Esllso(X = c(1, 2, 3, 4, 5, 6, 7, 8), y = c(1, 2, 3, 4, 5, 6, 7, 8), lambda_ = 5L)
#> $estimate
#> [1] 0.9754902
#> 
#> $beta
#> [1] 0.9754902
#> 
#> $n_nonzero
#> [1] 1
#> 
#> $active_set
#> [1] 0
#> 
#> $objective
#> [1] 4.938725
#> 
#> $iterations
#> [1] 2
#> 
#> $converged
#> [1] TRUE
#> 
#> $lambda
#> [1] 5
#> 
#> $n
#> [1] 8
#> 
#> $p
#> [1] 1
#> 
#> $method
#> [1] "lasso coordinate descent; constant columns unpenalised; lambda is n x glmnet's"
#> 
#> attr(,"class")
#> [1] "morie_rich_result" "list"