How to cite this WIREs title:
WIREs Comp Stat

Least angle regression for model selection

Advanced Review

Hongyang Zhang, Ruben H. Zamar

Published Online: Jan 08 2014

DOI: 10.1002/wics.1288

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Model selection using least angle regression (LARS) is an interesting approach proposed by Efron B, Hastie T, Johnstone L, Tibshirani R. Least angle regression. The Annals of statistics 2004, 320:407–499. In this paper we first review the LARS algorithm and its relationships with other popular methods such as stagewise regression and the lasso. Then we conduct a survey of recent developments and extensions of LARS. WIREs Comput Stat 2014, 6:116–123. doi: 10.1002/wics.1288 This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical Models > Model Selection Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical Models > Linear Models

References

Tibshirani, R. Regression shrinkage and selection via the lasso. J R Stat Soc [Ser B] 1996, 58:267–288.
Hastie, T, Tibshirani, R, Friedman, J, Franklin, J. The elements of statistical learning: data mining, inference and prediction. Math Intell 2005, 27:83–85.
Akaike, H. A new look at the statistical model identification. IEEE Trans Automat Control 1974, 19:716–723.
Schwarz, G. Estimating the dimension of a model. Ann Stat 1978, 6:461–464.
Stone, M. Cross‐validatory choice and assessment of statistical predictions. J R Stat Soc [Ser B] 1974, 36:111–147.
Breiman, L. Heuristics of instability and stabilization in model selection. Ann Stat 1996, 24:2350–2383.
Khan, JA, Van Aelst, S, Zamar, RH. Robust linear model selection based on least angle regression. J Am Stat Assoc 2007, 102:1289–1299.
Efron, B, Hastie, T, Johnstone, L, Tibshirani, R. Least angle regression. Ann Stat 2004, 32:407–499.
Ishwaran, H. Discussion‐least angle regression. Ann Stat 2004, 32:452–457.
Massart, P, Loubes, J‐M. Discussion‐least angle regression. Ann Stat 2004, 32:476–482.
Madigan, D, Ridgeway, G. Discussion‐least angle regression. Ann Stat 2004, 32:465–469.
Stine, RA. Discussion‐least angle regression. Ann Stat 2004, 32:475–480.
Hoerl, AE, Kennard, RW. Ridge regression: biased estimation for nonorthogonal problems. Technometrics 1970, 12:55–67.
Yuan, M, Lin, Y. Model selection and estimation in regression with grouped variables. J R Stat Soc [Ser B] 2006, 68:49–67.
Lin, Y, Zhang, HH. Component selection and smoothing in smoothing spline analysis of variance models. Ann Stat 2006, 34:2272–2297.
Park, MY, Hastie, T. Regularization path algorithms for detecting gene interactions. Department of Statistics, Stanford University, 2006.
Nelder, JA. A reformulation of linear models. J R Stat Soc [Ser A] 1977, 140:48–77.
McCullagh, P, Nelder, JA. Generalized linear models (monographs on statistics and applied probability 37). London: Chapman Hall; 1989.
Turlach, B. Discussion‐least angle regression. Ann Stat 2004, 32:481–490.
Yuan, M, Joseph, VR, Lin, Y. An efficient variable selection approach for analyzing designed experiments. Technometrics 2007, 49:430–439.
Choi, NH, Li, W, Zhu, J. Variable selection with the strong heredity constraint and its oracle property. J Am Stat Assoc 2010, 105:354–364.
Zhao, P, Rocha, G, Yu, B. The composite absolute penalties family for grouped and hierarchical variable selection. Ann Stat 2009, 37:3468–3497.
Simila, T, Tikka, J. Common subset selection of inputs in multiresponse regression. In: Neural Networks, 2006. IJCNN`06. International Joint Conference on, IEEE, 2006, 1908–1915.
Meier, L, Bühlmann, P. Smoothing L1‐penalized estimators for high‐dimensional time‐course data. Electron J Stat 2007, 1:597–615.
Turlach, BA, Venables, WN, Wright, SJ. Simultaneous variable selection. Technometrics 2005, 47:349–363.
Silva, J, Marques, J, Lemos, J. Selecting landmark points for sparse manifold learning. In: Advances in Neural Information Processing Systems, 2005, 1241–1248.
Hirose, Y, Komaki, F. An extension of least angle regression based on the information geometry of dually flat spaces. J Comput Graph Stat 2010, 19:1007–1023.
Augugliaro, L, Mineo, AM, Wit, EC. Differential geometric least angle regression: a differential geometric approach to sparse generalized linear models. J R Stat Soc [Ser B] 2013, 75:471–498.
Park, MY, Hastie, T. L1‐regularization path algorithm for generalized linear models. J R Stat Soc [Ser B] 2007, 69:659–677.
Wu, TT, Lange, K. Coordinate descent algorithms for lasso penalized regression. Ann Stat 2008, 2:224–244.
Friedman, J, Hastie, T, Tibshirani, R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw 2010, 33:1.
Goeman, JJ. L1 penalized estimation in the cox proportional hazards model. Biom J 2010, 52:70–84.
Meier, L, Van de Geer, S, Bühlmann, P. High‐dimensional additive modeling. Ann Stat 2009, 37:3779–3821.
Zhang, H. Linear model selection based on extended robust least angle regression. Master`s thesis, The University of British Columbia, Canada, 2012.
Zou, H, Hastie, T. Regularization and variable selection via the elastic net. J R Stat Soc [Ser B] 2005, 67:301–320.
Frank, LE, Friedman, JH. A statistical view of some chemometrics regression tools. Technometrics 1993, 35:109–135.
Fan, J, Li, R. Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 2001, 96:1348–1360.
Zou, H, Hastie, T, Tibshirani, R. Sparse principal component analysis. J Comput Graph Stat 2006, 15:265–286.
Zass, R, Shashua, A. Nonnegative sparse PCA. In: Advances in Neural Information Processing Systems, 2006, 1561–1568.
Witten, DM, Tibshirani, R, Hastie, T. A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 2009, 10:515–534.
Clemmensen, L, Hastie, T, Witten, D, Ersbøll, B. Sparse discriminant analysis. Technometrics 2011, 53:406–413.
Shao, J, Wang, Y, Deng, X, Wang, S. Sparse linear discriminant analysis by thresholding for high dimensional data. Ann Stat 2011, 39:1241–1265.
Mai, Q, Zou, H, Yuan, M. A direct approach to sparse discriminant analysis in ultra‐high dimensions. Biometrika 2012, 99:29–42.
Meinshausen, N, Bühlmann, P. High‐dimensional graphs and variable selection with the lasso. Ann Stat 2006, 34:1436–1462.
Yuan, M. Efficient computation of l1 regularized estimates in Gaussian graphical models. J Comput Graph Stat 2008, 17:809–826.
Banerjee, O, El Ghaoui, L, d`Aspremont, A. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J Mach Learn Res 2008, 9:485–516.
Dahl, J, Vandenberghe, L, Roychowdhury, V. Covariance selection for nonchordal graphs via chordal embedding. Optim Method Softw 2008, 23:501–520.
Yuan, M, Lin, Y. Model selection and estimation in the Gaussian graphical model. Biometrika 2007, 94:19–35.
Friedman, J, Hastie, T, Tibshirani, R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics 2008, 9:432–441.
Honorio, J, Samaras, D, Rish, I, Cecchi, GA. Variable selection for Gaussian graphical models. In: International Conference on Artificial Intelligence and Statistics, 2012, 538–546.
Hastie, T, Efron, B. lars: Least Angle Regression, Lasso and Forward Stagewise, R package version 1.2. 2013. Available at: http://CRAN.R‐project.org/package=lars. (Accessed December 23, 2013).
Zou, H, Hastie, T. elasticnet: Elastic‐Net for Sparse Estimation and Sparse PCA, R package version 1.1. 2012. Available at: http://CRAN.R‐project.org/package=elasticnet. (Accessed December 23, 2013).
Alfons, A. robustHD: Robust methods for high‐dimensional data, R package version 0.3.2. 2013. Available at: http://CRAN.R‐project.org/package=robustHD. (Accessed December 23, 2013).
Meier, L. grplasso: Fitting user specified models with Group Lasso penalty, R package version 0.4‐3. 2013. Available at: http://CRAN.R‐project.org/package=grplasso. (Accessed December 23, 2013).
Park, MY, Hastie, T. glmpath: L1 regularization path for generalized linear models and Cox proportional hazards model, R package version 0.97. 2013. Available at: http://CRAN.R‐project.org/package=glmpath. (Accessed December 23, 2013).
Friedman, J, Hastie, T, Tibshirani, R. glasso: Graphical lasso‐ estimation of Gaussian graphical models, R package version 1.7. 2011. Available at: http://CRAN.R‐project.org/package=glasso. (Accessed December 23, 2013).

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Least angle regression for model selection

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox