|
|||||
|
|
On the LASSO and its Dual |
|
| Authors: | Michael R Osborne Brett Presnell Berwin A Turlach |
| Institution: | 1. Centre for Mathematics and its Applications , Australian National University , Canberra , ACT , 0200 , Australia;2. Department of Statistics , University of Florida , Gainesville , FL , 32611-8545 , USA;3. Department of Mathematics and Statistics , University of Western Australia , Nedlands , WA , 6907 , Australia |
| Abstract: | Abstract Proposed by Tibshirani, the least absolute shrinkage and selection operator (LASSO) estimates a vector of regression coefficients by minimizing the residual sum of squares subject to a constraint on the l 1-norm of the coefficient vector. The LASSO estimator typically has one or more zero elements and thus shares characteristics of both shrinkage estimation and variable selection. In this article we treat the LASSO as a convex programming problem and derive its dual. Consideration of the primal and dual problems together leads to important new insights into the characteristics of the LASSO estimator and to an improved method for estimating its covariance matrix. Using these results we also develop an efficient algorithm for computing LASSO estimates which is usable even in cases where the number of regressors exceeds the number of observations. An S-Plus library based on this algorithm is available from StatLib. |
| Keywords: | Convex programming Dual problem Partial least squares Penalized regression Quadratic programming Regression Shrinkage Subset selection Variable selection |