Variable selection using Lq penalties

High‐dimensional data analysis emerges in many different areas of scientific research and social practice. Variable selection is an important task in statistical analysis for high‐dimensional data. The traditional all‐subset‐selection method can be ideal for variable selection, but it is computationally intractable in the high‐dimensional setting. In both regression and classification contexts, sparsity penalties are commonly used for the purpose of defining the complexity of the resulting model so as to achieve variable selection. The current article reviews a few important milestones and some recent works in the area of variable selection, especially those methods which use Lq norm and their variants as the penalty term. In particular, we review the Lq penalty, nonconvex penalties, among others. In ultrahigh‐dimensional data analysis, independence learning is often used for the purpose of dimension reduction. Theoretical results, methodological developments and computational innovations in regard to these methods are discussed. These variable selection techniques can be easily extended to problems beyond linear regression, such as classification, quantile regression, etc. Lastly, an interesting and promising research trend is the combination of multiple methods and we review several successful methods which fall into this category. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical Models > Model Selection Statistical Models > Linear Models