Principles of Forecasting

Volume 30 of the series International Series in Operations Research & Management Science pp 171-192

Judgmental Bootstrapping: Inferring Experts’ Rules for Forecasting

  • J. Scott ArmstrongAffiliated withThe Wharton School, University of Pennsylvania

* Final gross prices may vary according to local VAT.

Get Access


Judgmental bootstrapping is a type of expert system. It translates an expert’s rules into a quantitative model by regressing the expert’s forecasts against the information that he used. Bootstrapping models apply an expert’s rules consistently, and many studies have shown that decisions and predictions from bootstrapping models are similar to those from the experts. Three studies showed that bootstrapping improved the quality of production decisions in companies. To date, research on forecasting with judgmental bootstrapping has been restricted primarily to cross-sectional data, not time-series data. Studies from psychology, education, personnel, marketing, and finance showed that bootstrapping forecasts were more accurate than forecasts made by experts using unaided judgment. They were more accurate for eight of eleven comparisons, less accurate in one, and there were two ties. The gains in accuracy were generally substantial. Bootstrapping can be useful when historical data on the variable to be forecast are lacking or of poor quality; otherwise, econometric models should be used. Bootstrapping is most appropriate for complex situations, where judgments are unreliable, and where experts’ judgments have some validity. When many forecasts are needed, bootstrapping is cost-effective. If experts differ greatly in expertise, bootstrapping can draw upon the forecasts made by the best experts. Bootstrapping aids learning; it can help to identify biases in the way experts make predictions, and it can reveal how the best experts make predictions. Finally, judgmental bootstrapping offers the possibility of conducting “experiments” when the historical data for causal variables have not varied over time. Thus, it can serve as a supplement for econometric models.


Conjoint analysis expert systems protocols regression reliability