Lasso, elastic net and principal component front ends

R arm of morie.fn.elnetr, lasr and pcaprx. Elnetr minimises \((1/(2n))||y - b_0 - Xb||^2 + \alpha\rho||b||_1 + \alpha(1-\rho)||b||^2/2\) by cyclic coordinate descent on centred columns (soft-threshold updates, sweeps stop when the largest coefficient change is below 1e-8); Lasr is the case \(\rho = 1\). Pcaprx eigendecomposes the sample covariance (or correlation, when standardize = TRUE) matrix, orders components by decreasing eigenvalue and signs each so its largest-magnitude loading is positive.

Usage

Elnetr(X, y, alpha = 1, l1_ratio = 0.5, fit_intercept = TRUE, max_iter = 10000)

Lasr(X, y, alpha = 1, fit_intercept = TRUE, max_iter = 10000)

Pcaprx(X, n_components = NULL, standardize = TRUE)

Arguments

X

Numeric matrix (n by p).

y

Response vector.

alpha

Overall penalty.

l1_ratio

L1 share of the penalty (1 = lasso, 0 = ridge).

fit_intercept

Centre and fit an unpenalised intercept.

max_iter

Maximum number of coordinate-descent sweeps.

n_components

Components retained (all when NULL).

standardize

Scale columns by their sample standard deviation first.

Value

A named list (the Python result's payload).

References

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society B 58(1), 267-288.

Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society B 67(2), 301-320.

Friedman, J., Hastie, T. and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33(1), 1-22.

Jolliffe, I. T. (2002). Principal Component Analysis, 2nd ed. Springer.

Examples

X <- cbind(1:6, c(0.5, -1, 0.2, 1.5, -0.3, 0.8))
y <- c(1.1, 2.3, 2.8, 4.4, 4.9, 6.2)
Elnetr(X, y, alpha = 0.1, l1_ratio = 0.5)$coef
#> [1] 0.9601392 0.0242949
Lasr(X, y, alpha = 0.2)$coef
#> [1] 0.9285714 0.0000000
Pcaprx(cbind(X, y))$explained_variance
#> [1] 2.166482676 0.826603520 0.006913804