How to cite this WIREs title:
WIREs Comp Stat

Bayesian variable selection using the hyper‐g prior in WinBUGS

Advanced Review

Konstantinos Perrakis, Ioannis Ntzoufras

Published Online: Jul 17 2018

DOI: 10.1002/wics.1442

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

The hyper‐g prior is a default choice for Bayesian variable selection in normal linear regression models. In this article we provide an overview of the Bayesian variable selection framework and explain in detail the specification for the hyper‐g prior setup. The practical implementation of the methods under consideration is demonstrated through the use of WinBUGS software explaining the correspondence between code and theoretical setup. An illustration of results is considered through a simulated data example. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory Statistical Models > Model Selection Statistical Models > Linear Models

Posterior inclusion probabilities for each method (covariate index is ordered by the size inclusion probabilities)

[ Normal View | Magnified View ]

Bayesian model averaged posterior distributions of coefficients (on the left) and posterior inclusion probabilities (on the right) for covariates X₄, X₅, X₁₀, and X₁₂

[ Normal View | Magnified View ]

References

Barbieri,, M., & Berger,, J. (2004). Optimal predictive model selection. Annals of Statistics, 32, 870–897.
Bartlett,, M. (1957). Comment on D.V. Lindley`s statistical paradox. Biometrika, 44, 533–534.
Bayarri,, M. J., Berger,, J. O., Forte,, A., & García‐Donato,, G. (2012). Criteria for Bayesian model choice with application to variable selection. The Annals of Statistics, 40, 1550–1577.
Bondell,, H., & Reich,, B. (2012). Consistent high‐dimensional Bayesian variable selection via penalized credible regions. Journal of the American Statistical Association, 107, 1610–1624.
Clyde,, M. (2018). Using the Bayesian adaptive sampling (BAS) package for Bayesian model averaging and variable selection. R package vignette 11‐3‐2018.
Clyde,, M., Littman,, M., Wang,, Q., Ghosh,, J., & Li,, Y. (2018). BAS: Bayesian variable selection and model averaging using Bayesian adaptive sampling. R package, version 1.4.8.
Consonni,, G., Fouskakis,, D., Liseo,, B., & Ntzoufras,, I. (2018). Prior distributions for objective Bayesian analysis. Bayesian Analysis, 13, 627–679.
de Valpine,, P., Turek,, D., Paciorek,, C. J., Anderson‐Bergman,, C., Lang,, D. T., & Bodik,, R. (2017). Programming with models: Writing statistical algorithms for general model structures with NIMBLE. Journal of Computational and Graphical Statistics, 26(2), 403–413.
Dellaportas,, P., Forster,, J. J., & Ntzoufras,, I. (2002). On Bayesian model and variable selection using MCMC. Statistics and Computing, 12, 27–36.
Dellaportas,, P., Forster,, J. J., & Ntzoufras,, I. (2012). Joint specification of model space and parameter space prior distributions. Statistical Science, 27, 232–246.
Fernández,, C., Ley,, E., & Steel,, M. F. J. (2001). Benchmark priors for Bayesian model averaging. Journal of Econometrics, 100, 381–427.
Fouskakis,, D., & Ntzoufras,, I. (2016). Power‐conditional‐expected priors: Using g‐priors with random imaginary data for variable selection. Journal of Computational and Graphical Statistics, 25(3), 647–664.
Fouskakis,, D., Ntzoufras,, I., & Draper,, D. (2015). Power‐expected‐posterior priors for variable selection in Gaussian linear models. Bayesian Analysis, 10, 75–107.
Fouskakis,, D., Ntzoufras,, I., & Perrakis,, K. (2018). Power‐Expected‐Posterior priors for generalized linear models. Bayesian Analysis, 13, 721–748.
Garcia‐Donato,, G. & Forte,, A. (2016). BayesVarSel: Bayesian testing, variable selection and model averaging in linear models using R. arXiv 1611.08118.
Garcia‐Donato,, G., Forte,, A., & Vergara‐Hernández,, C. (2017). BayesVarSel: Bayes factors, model choice and variable selection in linear models. R package, version 1.8.0.
Garcia‐Donato,, G. & Paulo,, R. (2017). Handling factors in variable selection problems. arXiv preprint 1709.07238.
George,, E. I., & McCulloch,, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88, 881–889.
Goudie,, R. J. B., Turner,, R. M., De Angelis,, D., & Thomas,, A. (2017). MultiBUGS: Massively parallel MCMC for Bayesian hierarchical models. arXiv preprint 1704.03216:1–25.
Hahn,, P., & Carvalho,, C. (2015). Decoupled shrinkage and selection in bayesian linear models: A posterior summary perspective. Journal of the American Statistical Association, 110, 435–448.
Hansen,, M., & Yu,, B. (2003). Minimum description length model selection criteria for generalized linear models. Lecture Notes‐Monograph Series, 6, 145–163.
Hoeting,, J., Madigan,, D., Raftery,, A., & Volinsky,, C. (1999). Bayesian model averaging: A tutorial. Statistical Science, 14, 382–401.
JASP Team. (2018). JASP (Version 0.8.6) (Computer software).
Jeffreys,, H. (1961). Theory of probability (3rd ed.). Oxford: Oxford University Press.
Kass,, R., & Raftery,, A. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795.
Kass,, R. E., & Wasserman,, L. (1995). A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. Journal of the American Statistical Association, 90, 928–934.
Kuo,, L., & Mallick,, B. (1998). Variable selection for regression models. Sankhya B, 60, 65–81.
Li,, Y. & Clyde,, M. A. (2015). Mixtures of g‐priors in generalized linear models. arXiv:1503.06913v1 [stat.ME].
Liang,, F., Paulo,, R., Molina,, G., Clyde,, M. A., & Berger,, J. O. (2008). Mixtures of g‐priors for Bayesian variable selection. Journal of the American Statistical Association, 103, 410–423.
Lindley,, D. (1957). A statistical paradox. Biometrika, 44, 187–192.
Lunn,, D., Thomas,, A., Best,, N., & Spiegelhalter,, D. (2000). Winbugs: A bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337.
Madigan,, D., & York,, J. (1995). Bayesian graphical models for discrete data. International Statistical Review, 63, 215–232.
Malesios,, C., Demiris,, N., Kalogeropoulos,, K., & Ntzoufras,, I. (2018). Bayesian spatio‐temporal epidemic models with applications to sheep pox. Statistics in Medicine. In press.
Maruyama,, Y., & George,, E. I. (2011). Fully Bayes factors with a generalized g‐prior. The Annals of Statistics, 39, 2740–2765.
Ntzoufras,, I. (2009). Bayesian Modeling Using WinBUGS. Hoboken, NJ: Wiley Series in Computational Statistics.
Ntzoufras,, I., Dellaportas,, P., & Forster,, J. J. (2003). Bayesian variable and link determination for generalized linear models. Journal of Statistical Planning and Inference, 111, 165–180.
Raftery,, A., Madigan,, D., & Hoeting,, J. (1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association, 92, 179–191.
Ročková,, V., & George,, E. I. (2014). EMVS: The EM approach to Bayesian variable selection. Journal of the American Statistical Association, 109, 828–846.
Sabanés Bové,, D., & Held,, L. (2011). Hyper‐g priors for generalized linear models. Bayesian Analysis, 6, 387–410.
Scott,, J. G., & Berger,, J. O. (2010). Bayes and empirical‐Bayes multiplicity adjustment in the variable‐selection problem. The Annals of Statistics, 38, 2587–2,619.
Wang,, M. (2017). Mixtures of g‐priors for analysis of variance models with a diverging number of parameters. Bayesian Analysis, 12, 511–532.
Wang,, X., & George,, E. I. (2007). Adaptive Bayesian criteria in variable selection for generalized linear models. Statistica Sinica, 17, 667–690.
Wetzels,, R., Grasman,, R. P. P., & Wagenmakers,, E.‐J. (2012). A default Bayesian hypothesis test for ANOVA designs. The American Statistician, 66, 104–111.
Zellner,, A. (1986). On assessing prior distributions and Bayesian regression analysis using g‐prior distributions. In P. Goel, & A. Zellner, (Eds.), Bayesian inference and decision techniques: Essays in honor of Bruno de Finetti (pp. 233–243). Amsterdam, The Netherlands: North‐Holland.
Zellner,, A., & Siow,, A. (1980). Posterior odds ratios for selected regression hypothesis (with discussion). In J. M. Bernardo,, M. H. DeGroot,, D. V. Lindley,, & A. F. M. Smith, (Eds.), Bayesian statistics (Vol. 1, pages 585–606 & 618–647 (discussion)). Oxford: Oxford University Press.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

Bayesian variable selection using the hyper‐g prior in WinBUGS

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox