Fits the lasso, ridge and elastic net path by cyclical coordinate
descent, for a numeric (gaussian) or a 0/1 (binomial) outcome, and
cross-validates it over folds the caller supplies. The objective, the
standardisation of the columns and the scale of lambda are those of
'glmnet', so that with the same lambda and the same folds the two give
the same coefficients, up to their convergence thresholds: the gaussian
family minimises (1 / 2n) |y - a0 - x b|^2 + lambda pen(b), the
binomial family the mean negative log-likelihood plus lambda pen(b),
with pen(b) = sum((1 - alpha) / 2 (s b)^2 + alpha |s b|), s the
population standard deviations of the columns. For the gaussian family
the outcome is scaled to unit variance while fitting, as 'glmnet' does.
The intercept is never penalised; a constant column keeps a zero
coefficient.
Usage
enpath(
x,
y,
fam = "gaussian",
alpha = 1,
lambda = NULL,
nlam = 100,
lmr = NULL,
fold = NULL,
tol = 1e-07,
maxit = 1e+05
)Arguments
- x
The predictors: a numeric matrix, one row per observation, with no missing values.
- y
The outcome: one finite number per row of
x; 0 or 1 for the binomial family.- fam
"gaussian"or"binomial".- alpha
The mixing parameter, from 0 (ridge) to 1 (lasso).
- lambda
NULL, or the penalties to fit: non-negative numbers in decreasing order.- nlam
When
lambdaisNULL, the number of penalties on the path: one whole number, at least 1.- lmr
When
lambdaisNULL, the smallest penalty as a share of the largest: one number strictly between 0 and 1, orNULLfor 1e-4 whenxhas more rows than columns and 0.01 otherwise. The path runs from the smallest penalty that zeroes every coefficient down tolmrtimes it, evenly spaced on the log scale, and stops early as 'glmnet' does: from the fifth penalty on, once the share of the deviance explained grows by less than 1e-5 of itself or exceeds 0.999.- fold
NULL, or the fold of each row: whole numbers from 1 to the number of folds (at least 2), each fold used.- tol
The convergence threshold: a pass that moves no standardised coefficient by more than
sqrt(tol)(weighted by the curvature, for the binomial family) ends the descent. One positive number.- maxit
The most passes over the columns, in all: one whole number.
Value
A list: lambda, the penalties fitted; a0, the intercepts;
beta, a matrix of coefficients with one column per penalty and one
row per column of x; and, when fold is given, cvm and cvsd, the
mean held-out loss (squared error, or the binomial deviance with
probabilities kept within 1e-5 of 0 and 1) and its standard error over
folds, computed as 'glmnet' computes them, lmin, the penalty at the
minimum, and l1se, the largest penalty within one standard error of
it; these four are NULL without folds.
Details
The binomial fit is a proximal Newton method: iteratively reweighted
least squares around coordinate descent over the active columns, with
the optimality conditions checked over every column before a lambda
is left. The gaussian fit updates the
gradient through the column inner products, formed only for the
columns that enter the model.
References
Friedman, J., Hastie, T. and Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1-22. doi:10.18637/jss.v033.i01