How to cite this WIREs title:
WIREs Comp Stat

A review of approximate Bayesian computation methods via density estimation: Inference for simulator‐models

Overview

Clara Grazian, Yanan Fan

Published Online: Oct 29 2019

DOI: 10.1002/wics.1486

Full article on Wiley Online Library: HTML PDF

Can't access this content? Tell your librarian.

Abstract This paper provides a review of approximate Bayesian computation (ABC) methods for carrying out Bayesian posterior inference, through the lens of density estimation. We describe several recent algorithms and make connection with traditional approaches. We show advantages and limitations of models based on parametric approaches and we then draw attention to developments in machine learning, which we believe have the potential to make ABC scalable to higher dimensions and may be the future direction for research in this area. This article is categorized under: Algorithms and Computational Methods < Algorithms Statistical and Graphical Methods of Data Analysis < Bayesian Methods and Theory Statistical Models < Simulation Models

Approximations of the posterior distributions for θ based on synthetic likelihood (left), standard approximate Bayesian computation (center) and Bayesian optimization for likelihood‐free inference (right) for the Gaussian example, with unknown location parameter and known variance. Solid line represents the true posterior distribution, dotted lines are the averages from 250 repetitions of the experiment, and the shaded area shows corresponding 95% pointwise variability intervals

[ Normal View | Magnified View ]

Approximations of the posterior distributions of log(r) based on synthetic likelihood (left), standard approximate Bayesian computation (center) and Bayesian optimization for likelihood‐free inference (right) for the Ricker model example. Top row obtained from 5 summary statistics, and bottom row 13 summary statistics. Solid vertical lines represents the true value, dotted lines are the averages from 250 repetitions of the experiment and the shaded areas show the 95% pointwise variability intervals

[ Normal View | Magnified View ]

An example of simulated data from the Ricker model with logr = 4, ϕ = 10, and σ = 0.3

[ Normal View | Magnified View ]

References

Aeschbacher,, S., Beaumont,, M. A., & Futschik,, A. (2012). A novel approach for choosing summary statistics in approximate Bayesian computation. Genetics, 192(3), 1027–1047.
Alsing,, J., Charnock,, T., Feeney,, S., & Wandelt,, B. (2019). Fast likelihood‐free cosmology with neural density estimators and active learning. arXiv preprint arXiv:1903.00007v1.
An,, Z., Nott,, D. J., & Drovandi,, C. (2018). Robust Bayesian synthetic likelihood via a semi‐parametric approach. arXiv preprint arXiv:1809.05800.
Beaumont,, M. A. (2010). Approximate Bayesian computation in evolution and ecology. Annual Review of Ecology, Evolution, and Systematics, 41, 379–406.
Beaumont,, M. A., Zhang,, W., & Balding,, D. J. (2002). Approximate Bayesian computation in population genetics. Genetics, 162(4), 2025–2035.
Blum,, M. G. (2010). Approximate Bayesian computation: A nonparametric perspective. Journal of the American Statistical Association, 105(491), 1178–1187.
Blum,, M. G., Nunes,, M. A., Prangle,, D., & Sisson,, S. A. (2013). A comparative review of dimension reduction methods in approximate Bayesian computation. Statistical Science, 28(2), 189–208.
Bonassi,, F. V., You,, L., & West,, M. (2011). Bayesian learning from marginal data in bionetwork models. Statistical Applications in Genetics and Molecular Biology, 10(1).
Bortot,, P., Coles,, S. G., & Sisson,, S. A. (2007). Inference for stereological extremes. Journal of the American Statistical Association, 102(477), 84–92.
Cameron,, E., & Pettitt,, A. (2012). Approximate Bayesian Computation for astronomical model analysis: A case study in galaxy demographics and morphological transformation at high redshift. Monthly Notices of the Royal Astronomical Society, 425(1), 44–65.
Cornuet,, J.‐M., Santos,, F., Beaumont,, M. A., Robert,, C. P., Marin,, J.‐M., Balding,, D. J., … Estoup,, A. (2008). Inferring population history with DIY ABC: A user‐friendly approach to approximate Bayesian computation. Bioinformatics, 24(23), 2713–2719.
Creel,, M., & Kristensen,, D. (2015). ABC of SV: Limited information likelihood inference in stochastic volatility jump‐diffusion models. Journal of Empirical Finance, 31, 85–108.
Drovandi,, C. C., Moores,, M. T., & Boys,, R. J. (2018). Accelerating pseudo‐marginal MCMC using Gaussian processes. Computational Statistics %26 Data Analysis, 118, 1–17.
Everitt,, R. G. (2017). Bootstrapped synthetic likelihood. arXiv preprint arXiv:1711.05825.
Fan,, Y., Nott,, D. J., & Sisson,, S. A. (2013). Approximate Bayesian computation via regression density estimation. Stat, 2(1), 34–48. https://doi.org/10.1002/sta4.15
Fasiolo,, M., Pya,, N., & Wood,, S. N. (2016). A comparison of inferential methods for highly nonlinear state space models in ecology and epidemiology. Statistical Science, 31(1), 96–118.
Fearnhead,, P., & Prangle,, D. (2012). Constructing summary statistics for approximate Bayesian computation: Semi‐automatic approximate Bayesian computation. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(3), 419–474.
Ghurye,, S., & Olkin,, I. (1969). Unbiased estimation of some multivariate probability densities and related functions. The Annals of Mathematical Statistics, 40(4), 1261–1271.
Gourieroux,, C., Monfort,, A., & Renault,, E. (1993). Indirect inference. Journal of Applied Econometrics, 8(Suppl. 1), S85–S118.
Grazian,, C., & Liseo,, B. (2017). Approximate Bayesian inference in semiparametric copula models. Bayesian Analysis, 12(4), 991–1016.
Greenberg,, D. S., Nonnenmacher,, M., & Macke,, J. H. (2019). Automatic posterior transformation for likelihood‐free inference. arXiv preprint arXiv:1905.07488v1.
Grendár,, M., & Judge,, G. (2009). Asymptotic equivalence of empirical likelihood and Bayesian MAP. The Annals of Statistics, 37(5A), 2445–2457.
Gutmann,, M. U., & Corander,, J. (2016). Bayesian optimization for likelihood‐free inference of simulator‐based statistical models. The Journal of Machine Learning Research, 17(1), 4256–4302.
Gutmann,, M. U., Dutta,, R., Kaski,, S., & Corander,, J. (2018). Likelihood‐free inference via classification. Statistics and Computing, 28(2), 411–425.
Izbicki,, R., Lee,, A. B., & Pospisil,, T. (2019). ABC‐CDE: Toward approximate Bayesian computation with complex high‐dimensional data and limited simulations. Journal of Computational and Graphical Statistics, 1–20. https://doi.org/10.1080/10618600.2018.1546594
Järvenpää,, M., Gutmann,, M. U., Pleska,, A., Vehtari,, A., & Marttinen,, P. (2019). Efficient acquisition rules for model‐based approximate Bayesian computation. Bayesian Analysis, 14(2), 595–622.
Järvenpää,, M., Gutmann,, M. U., Vehtari,, A., & Marttinen,, P. (2018). Gaussian process modelling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria. The Annals of Applied Statistics, 12(4), 2228–2251.
Li,, W., & Fearnhead,, P. (2016). Improved convergence of regression adjusted approximate Bayesian computation. arXiv preprint arXiv:1609.07135.
Luciani,, F., Sisson,, S. A., Jiang,, H., Francis,, A. R., & Tanaka,, M. M. (2009). The epidemiological fitness cost of drug resistance in Mycobacterium tuberculosis. Proceedings of the National Academy of Sciences of the United States of America, 106(34), 14711–14715.
Lueckmann,, J.‐M., Bassetto,, G., Karaletsos,, T., & Macke,, J. H. (2018). Likelihood‐free inference with emulator networks. arXiv preprint arXiv:1805.09294.
Lueckmann,, J.‐M., Goncalves,, P. J., Bassetto,, G., Öcal,, K., Nonnenmacher,, M., & Macke,, J. H. (2017). Flexible statistical inference for mechanistic models of neural dynamics. In Advances in neural information processing systems 30 (pp. 1289–1299), Long Beach, CA.
Marin,, J.‐M., Pudlo,, P., Robert,, C. P., & Ryder,, R. J. (2012). Approximate Bayesian computational methods. Statistics and Computing, 22(6), 1167–1180.
Marjoram,, P., Molitor,, J., Plagnol,, V., & Tavaré,, S. (2003). Markov chain Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences of the United States of America, 100, 15324–15328.
Meeds,, E., & Welling,, M. (2014). GPS‐ABC: Gaussian process surrogate approximate Bayesian computation. arXiv preprint arXiv:1401.2838.
Mengersen,, K. L., Pudlo,, P., & Robert,, C. P. (2013). Bayesian computation via empirical likelihood. Proceedings of the National Academy of Sciences of the United States of America, 110(4), 1321–1326.
Numminen,, E., Gutmann,, M., Shubin,, M., Marttinen,, P., Meric,, G., van Schaik,, W., … Sheppard,, S. K. (2016). The impact of host metapopulation structure on the population genetics of colonizing bacteria. Journal of Theoretical Biology, 396, 53–62.
Nunes,, M. A., & Balding,, D. J. (2010). On optimal selection of summary statistics for approximate Bayesian computation. Statistical Applications in Genetics and Molecular Biology, 9(1).
Owen,, A. B. (1988). Empirical likelihood ratio confidence intervals for a single functional. Biometrika, 75(2), 237–249.
Owen,, A. B. (2001). Empirical likelihood. Chapman and Hall/CRC.
Papamakarios,, G., & Murray,, I. (2016). Fast ε‐free inference of simulation models with Bayesian conditional density estimation. Advances in neural information processing systems, 1028–1036.
Papamakarios,, G., Pavlakou,, T., & Murray,, I. (2017). Masked autoregressive flow for density estimation. In Neural information processing systems 30 (pp. 2338–2347), Long Beach, CA.
Papamakarios,, G., Sterratt,, D. C., & Murray,, I. (2019). Sequential neural likelihood: Fast likelihood‐free inference with autoregressive flows. arXiv preprint arXiv:1805.07226v2.
Price,, L. F., Drovandi,, C. C., Lee,, A., & Nott,, D. J. (2018). Bayesian synthetic likelihood. Journal of Computational and Graphical Statistics, 27(1), 1–11.
Pritchard,, J. K., Seielstad,, M. T., Perez‐Lezaun,, A., & Feldman,, M. W. (1999). Population growth of human Y chromosomes: A study of Y chromosome microsatellites. Molecular Biology and Evolution, 16(12), 1791–1798.
Radev,, S. T., Mertens,, U. K., Voss,, A., & Köthe,, U. (2019). Towards end‐to‐end likelihood‐free inference with convolutional neural networks. British Journal of Mathematical and Statistical Psychology. https://doi.org/10.1111/bmsp.12159
Ricker,, W. E. (1954). Stock and recruitment. Journal of the Fisheries Board of Canada, 11(5), 559–623.
Schennach,, S. M. (2005). Bayesian exponentially tilted empirical likelihood. Biometrika, 92(1), 31–46.
Shah,, A., Wilson,, A., & Ghahramani,, Z. (2014). Student‐t processes as alternatives to Gaussian processes. In Artificial intelligence and statistics (pp. 877–885), Reykjavik, Iceland.
Sisson,, S. A., & Fan,, Y. (2018). ABC samplers. In S. A. Sisson,, Y. Fan,, & M. A. Beaumont, (Eds.), Handbook of approximate Bayesian Computation (pp. 87–124). Chapman %26 Hall/CRC Press.
Sisson,, S. A., Fan,, Y., & Beaumont,, M. A. (2018a). Handbook of approximate Bayesian computation. Chapman %26 Hall/CRC Press.
Sisson,, S. A., Fan,, Y., & Beaumont,, M. A. (2018b). Overview of ABC. In S. A. Sisson,, Y. Fan,, & M. A. Beaumont, (Eds.), Handbook of approximate Bayesian computation (pp. 3–54). Chapman %26 Hall/CRC Press.
Sisson,, S. A., Fan,, Y., & Tanaka,, M. M. (2007). Sequential Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences of the United States of America, 104(6), 1760–1765.
Tavaré,, S., Balding,, D. J., Griffiths,, R. C., & Donnelly,, P. (1997). Inferring coalescence times from DNA sequence data. Genetics, 145(2), 505–518.
Vehtari,, A., & Ojanen,, J. (2012). A survey of Bayesian predictive methods for model assessment, selection and comparison. Statistics Surveys, 6, 142–228.
Wilkinson,, R. D. (2013). Approximate Bayesian computation (ABC) gives exact results under the assumption of model error. Statistical Applications in Genetics and Molecular Biology, 12(2), 129–141.
Wilkinson,, R. D. (2014). Accelerating ABC methods using Gaussian processes. arXiv preprint arXiv:1401.1436.
Williams,, C. K., & Rasmussen,, C. E. (2006). Gaussian processes for machine learning (Vol. 2(3)). Cambridge, MA: MIT Press.
Wood,, S. N. (2010). Statistical inference for noisy nonlinear ecological dynamic systems. Nature, 466(7310), 1102–1104.

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

A review of approximate Bayesian computation methods via density estimation: Inference for simulator‐models

Browse by Topic

Access to this WIREs title is by subscription only.

The latest WIREs articles in your inbox