Home
This Title All WIREs
WIREs RSS Feed
How to cite this WIREs title:
WIREs Comp Stat

Aggregating predictions from experts: A review of statistical methods, experiments, and applications

Full article on Wiley Online Library:   HTML PDF

Can't access this content? Tell your librarian.

Abstract Forecasts support decision making in a variety of applications. Statistical models can produce accurate forecasts given abundant training data, but when data is sparse or rapidly changing, statistical models may not be able to make accurate predictions. Expert judgmental forecasts—models that combine expert‐generated predictions into a single forecast—can make predictions when training data is limited by relying on human intuition. Researchers have proposed a wide array of algorithms to combine expert predictions into a single forecast, but there is no consensus on an optimal aggregation model. This review surveyed recent literature on aggregating expert‐elicited predictions. We gathered common terminology, aggregation methods, and forecasting performance metrics, and offer guidance to strengthen future work that is growing at an accelerated pace. This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Multivariate Analysis
A consort diagram that describes the path from collected to analysis‐set article. The search term used to collect the initial set of articles is reported and all intermediate steps between initial and analysis‐set articles
[ Normal View | Magnified View ]
Complementary cumulative distribution of the total number of forecasts made per article (a), and the proportion of articles eliciting less than 10, 100, 103, 104, and 105 forecasts. Articles collecting more than 104 forecasts were simulations
[ Normal View | Magnified View ]
The complementary cumulative distribution (CCDF) of the number of experts elicited per article (a). The proportion of articles enrolling less than 10, less than 100, less than 103, and less than 104 expert forecasters (b). The small number of articles enrolling more than 103 were simulation studies
[ Normal View | Magnified View ]
The annual proportion of the top 12 most prevalent word stems among all abstract text. Note: The words probability and probabilities were stemmed to probabl). For each year, word w frequency was divided by the frequency of all words present in all abstracts
[ Normal View | Magnified View ]
The top 5% most frequent words used in all analysis‐set abstracts. Expert, forecast, and judgment are the most frequent and likely related to the search words used to collect these articles
[ Normal View | Magnified View ]
The cumulative proportion (a) and individual number (b) of articles published per year. The earliest analysis‐set article was published in 1992 and most recent in 2018. A sharp increase in publication occurred at or near 2010
[ Normal View | Magnified View ]

Browse by Topic

Statistical and Graphical Methods of Data Analysis > Multivariate Analysis
Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods
Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis
Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts