This Title All WIREs
How to cite this WIREs title:
WIREs Data Mining Knowl Discov
Impact Factor: 1.939

# Bayesian treed response surface models

Can't access this content? Tell your librarian.

Tree‐based regression and classification, popularized in the 1980s with the advent of the classification and regression trees (CART) has seen a recent resurgence in popularity alongside a boom in modern computing power. The new methodologies take advantage of simulation‐based inference, and ensemble methods, to produce higher fidelity response surfaces with competitive out‐of‐sample predictive performance while retaining many of the attractive features of classic trees: thrifty divide‐and‐conquer nonparametric inference, variable selection and sensitivity analysis, and nonstationary modeling features. In this paper, we review recent advances in Bayesian modeling for trees, from simple Bayesian CART models, treed Gaussian process, sequential inference via dynamic trees, to ensemble modeling via Bayesian additive regression trees (BART). We outline open source R packages supporting these methods and illustrate their use.

• Algorithmic Development > Statistics
• Algorithmic Development > Structure Discovery
• Application Areas > Data Mining Software Tools
• Technologies > Statistical Fundamentals
Illustrating trees (top row) diagrammatically (left) and geographically (right). Each predictive location x falls in a leaf node $η(x)$. Tree operations (bottom row) show possible perturbations of trees that are possible steps in a stochastic search MCMC algorithm. xy data.
[ Normal View | Magnified View ]
Left: BART fit to the motorcycle data with 90% intervals for the unknown function; right: a comparison between the methods discussed here and some other modern alternatives on the Friedman data. TC, treed constants; DTC, dynamic treed constants; TL, treed regression; DTL, dynamic treed regression; GP, Gaussian processes; MARS, multivariate adaptive splines; RF, random forests; BART, bart; NNET1, neural nets; NNET2, neural nets. For more details, see the friedman.1.data in the tgp package for R.
[ Normal View | Magnified View ]
Predictive surfaces for the DT models.
[ Normal View | Magnified View ]
Predictive surfaces for the treed GP model (left: posterior mean in bold, and mode dashed), and for the nontreed GP (right).
[ Normal View | Magnified View ]
Predictive surfaces for the treed constant model (left) and treed linear model (right); posterior mean in bold, and mode dashed.
[ Normal View | Magnified View ]