Home
This Title All WIREs
WIREs RSS Feed
How to cite this WIREs title:
WIREs Comp Stat

The role of margins in boosting and ensemble performance

Full article on Wiley Online Library:   HTML PDF

Can't access this content? Tell your librarian.

Boosting refers to methods that create a sequence of classifiers that perform at least slightly better than random (weak learners) and combine them into a highly accurate ensemble model (strong learners) through weighted voting. There is sufficient empirical evidence to suggest that the performance of boosting methods is superior to that of individual classifiers. In the bias–variance decomposition framework, it has been demonstrated that boosting algorithms typically reduce bias for learning problems, and in some instances reduce the variance. In addition, even when combining a large number of weak learners, boosting algorithms can be very robust to overfitting, in most instances having lower generalization error than other competing ensemble methodologies, such as bagging and random forests. To explain the successful performance of boosting methods, Schapire et al. (Boosting the margin: a new explanation for the effectiveness of voting methods. Ann Stat 1998, 26:1651–1686) developed a bound based on the margins of the training data, from which they concluded that larger margins should lead to lower generalization error. In this article, we examine the role of margins in boosting and ensemble method performance. WIREs Comput Stat 2014, 6:124–131. doi: 10.1002/wics.1292 This article is categorized under: Statistical and Graphical Methods of Data Analysis > Bootstrap and Resampling Statistical Learning and Exploratory Methods of the Data Sciences > Classification and Regression Trees (CART)
Comparisons of AdaBoost and Max–Min (Margin Improvement) algorithm for the Sonar data. The left panel compares the cumulative margin distributions (CMD) for AdaBoost and the Max–Min algorithm. The right panel is a scatterplot of the AdaBoost margins and Max–Min margins. The line in the graph indicates equality of the margins.
[ Normal View | Magnified View ]

Browse by Topic

Statistical Learning and Exploratory Methods of the Data Sciences > Classification and Regression Trees (CART)
Statistical and Graphical Methods of Data Analysis > Bootstrap and Resampling

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts