Advertisement

Diffusion of Forecasting Principles through Software

  • Leonard J. Tashman
  • Jim Hoover
Part of the International Series in Operations Research & Management Science book series (ISOR, volume 30)

Abstract

Do forecasting software programs facilitate good practices in the selection, evaluation, and presentation of appropriate forecasting methods? Using representative programs from each of four market categories, we evaluate the effectiveness of forecasting software in implementing relevant principles of forecasting. The categories are (1) spreadsheet add-ins, (2) forecasting modules of general statistical programs, (3) neural network programs, and (4) dedicated business-forecasting programs. We omitted one important category—demand planning software—because software developers in that market declined to submit their products for review.

In the aggregate, forecasting software is attending to about 50 percent of the basic principles of forecasting. The steepest shortfall occurs in assessment of uncertainty: programs are often secretive about how they calculate prediction intervals and uninformative about the sources of uncertainty in the forecasts. For the remaining areas of evaluation—preparing data, selecting and implementing methods, evaluating forecast accuracy, and presenting forecasts—we rated the packages as achieving 42 to 51 percent of the maximum possible ratings (the ratings assigned for best practices).

Spreadsheet add-ins (16% of best-practices rating) have made rudimentary regression tools and some extrapolative forecasting techniques accessible to the spreadsheet analyst; however, they do not incorporate best practices in data preparation, method selection, forecast accuracy evaluation, or presentation of forecasts.

Forecasting modules of general statistical programs (42% of best-practices rating) provide effective data preparation tools; however, with the exception of one of these programs, they do not adequately help users to select, evaluate, and present a forecasting method. To implement best practices, the forecaster must perform macro programming and multiple-step processing.

Neural network packages (38% of best-practices rating) facilitate many best practices in preparing data for modeling and in evaluating neural network models. They do not use the more traditional models as comparative benchmarks, however, to test whether the neural net improves accuracy enough to justify its added complexity and lack of transparency.

Dedicated business-forecasting programs (60% of best-practices rating) have the best record for implementation of forecasting principles. Data preparation is generally good, although it could be more effectively automated. The programs are strong in method selection, implementation, and evaluation. However, they lack transparency in their assessments of uncertainty and offer forecasters little help in presenting the forecasts. Three of the dedicated business-forecasting programs contain features designed to reconcile forecasts across a product hierarchy, a task this group performs so commendably it can serve as a role model for forecasting engines in demand-planning systems.

Keywords

Automatic forecasting batch forecasting combining forecasts fit period forecast horizon intermittent demand judgmental override method evaluation method selection out-of-sample test prediction interval product hierarchy trading day variation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Allen, P. G. and R. Fildes (2001), “Econometric forecasting,” in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Publishers.Google Scholar
  2. Armstrong, J. S. ( 2001 a), “ Introduction,” in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Publishers.Google Scholar
  3. Armstrong, J. S. ( 2001 b), Standards and practices for forecasting, in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Publishers.Google Scholar
  4. Armstrong, J. S. (2001c), Combining forecasts, in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Publishers.Google Scholar
  5. Armstrong, J. S. (2001d), Evaluating forecasting methods, in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Publishers.Google Scholar
  6. Broze, L. G. Melard (1990), Exponential smoothing: Estimation by maximum likelihood, Journal of Forecasting, 9, 445 - 455.CrossRefGoogle Scholar
  7. Chatfield, C. M. Yar (1991), Prediction intervals for multiplicative Holt-Winters, International Journal of Forecasting, 7, 31 - 37.CrossRefGoogle Scholar
  8. Cox, J. D. Loomis (2001), Diffusion of forecasting principles through books, in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Publishers.Google Scholar
  9. Makridakis, S. M. Hibon (2000), The M3-Competition, International Journal of Forecasting, 16, 451 - 476.CrossRefGoogle Scholar
  10. Levenbach, H. J. Cleary (1981), The Beginning Forecaster. Belmont, CA: Lifetime Learning Publications.Google Scholar
  11. Newbold, P. T. Bos (1989), On exponential smoothing and the assumption of deterministic trend plus white noise data-generating models, International Journal ofForecasting, 5, 523 - 527.CrossRefGoogle Scholar
  12. Rycroft, R. S. (1999), Microcomputer software of interest to forecasters in comparative review: Updated again, International Journal of Forecasting, 15, 93 - 120.CrossRefGoogle Scholar
  13. Tashman, L.J. M. L. Leach (1991), Automatic forecasting software: A survey and evaluation, International Journal of Forecasting, 7, 209 - 230.CrossRefGoogle Scholar
  14. Tashman, L. J., T. Bakken J. Buzas (2001), Effect of regressor forecast error on the variance of regression forecasts, Journal of Forecasting 19, 587 - 600.CrossRefGoogle Scholar
  15. Willemain, T. R., C. N. Smart, J. H. Shocker P. A. De Sautels (1994), Forecasting intermittent demand: A comparative evaluation of Croston’s method, International Journal of Forecasting, 10, 529 - 538.CrossRefGoogle Scholar
  16. Williams, D.W. D. Miller (1999), Level-adjusted exponential smoothing for modeling planned discontinuities, International Journal of Forecasting, 15, 273 - 289.CrossRefGoogle Scholar
  17. Wittink, D. R T. Bergestuen (2001), Forecasting with conjoint analysis, in J.S. Armstrong (ed.), Principles of Forecasting. Norwell, MA: Kluwer Academic Publishers.Google Scholar
  18. Yar, M. C. Chatfield (1990), Prediction intervals for the Holt-Winters forecasting procedure, International Journal of Forecasting 6, 127 - 137.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2001

Authors and Affiliations

  • Leonard J. Tashman
    • 1
  • Jim Hoover
    • 2
  1. 1.School of Business AdministrationUniversity of VermontUSA
  2. 2.United States Department of the NavyUSA

Personalised recommendations