A new approach to variable selection in least squares problems

The title Lasso has been suggested by Tibshirani (1996) as acolourful name for a technique of variable selection which requiresthe minimization of a sum of squares subject to an l₁ bound on the solution. This forces zero components in the minimizingsolution for small values of . Thus this bound can functionas a selection parameter. This paper makes two contributionsto computational problems associated with implementing the Lasso:(1) a compact descent method for solving the constrained problemfor a particular value of is formulated, and (2) a homotopymethod, in which the constraint bound becomes the homotopyparameter, is developed to completely describe the possibleselection regimes. Both algorithms have a finite terminationproperty. It is suggested that modified Gram-Schmidt orthogonalizationapplied to an augmented design matrix provides an effectivebasis for implementing the algorithms.