Decision Analysis Working Paper Abstract
Archive
WP040001
|
Title: A Generative Bayesian Model for Aggregating Probabilities
of Categorical Events
Author: Joseph
M. Kahn, Stanford University
Date: July 2004
Status: Working Paper
In order to improve forecasts, a decision-maker often combines probabilities
given by various sources, such as human experts and machine learning classifiers.
When few training data are available, aggregation can be improved by incorporating
prior knowledge about the event being forecasted and about salient properties
of the experts. To this end, we develop a generative Bayesian
aggregation model for probabilistic classification. The model
includes an event-specific prior, measures of individual experts' bias,
calibration and accuracy, and also a measure of dependence between experts.
Rather than require absolute measures, we show that aggregation may be
expressed in terms of
relative accuracy between experts.
The model results in a weighted logarithmic opinion pool (LogOps) that
satisfies consistency criteria such as the external Bayesian property.
We derive analytic solutions for independent and for exchangeable experts.
Empirical tests demonstrate the model's use, showing its accuracy to be
either on par with or better than other commonly-used aggregation methods.
Click
here to access a pdf version of the paper.
Back to the Decision Analysis Working Paper
Index
