Population Research and Policy Review

, Volume 18, Issue 5, pp 387–410 | Cite as

In search of the ideal measure of accuracy for subnational demographic forecasts

  • Jeff Swanson
  • David A. Swanson
  • Charles F. Barr

Abstract

We examine nonlinear transformations of the forecasterror distribution in hopes of finding a summary errormeasure that is not prone to an upward bias and usesmost of the information about that error. MAPE, thecurrent standard for measuring error, often overstatesthe error represented by most of the values becausethe distribution underlying the MAPE is right skewedand truncated at zero. Using a modification to theBox-Cox family of nonlinear transformations, wetransform these skewed forecast error distributionsinto symmetrical distributions for a wide range ofsize and growth rate conditions. We verify thissymmetry using graphical devices and statisticaltests; examine the transformed errors to determine ifre-expression to the scale of the untransformed errorsis necessary; and develop and implement a procedurefor the re-expression. The MAPE-R developed by ourprocess is lower than the MAPE based on theuntransformed errors and is more consistent with arobust estimator of location.

Accuracy Demographic Forecast Nonlinear transformation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ahlburg, D. (1995). Simple versus complex models: evaluation, accuracy, and combining, Mathematical Population Studies 5: 281–290.Google Scholar
  2. Box, G. & Cox, D. (1964). An analysis of transformations, Journal of the Royal Statistical Society Series B 26: 211–252.Google Scholar
  3. Dorn, H. (1950). Pitfalls in population forecasts and projections, Journal of the American Statistical Association 45: 311–334.Google Scholar
  4. Draper, N. & Smith, H. (1981). Applied Regression Analysis, 2nd edn. New York: John Wiley.Google Scholar
  5. Emerson, J. & Stoto, M. (1983). Transforming data, pp. 97–128, in: D. Hoaglin, F. Mosteller & J. Tukey (eds.), Understanding Robust and Exploratory Data Analysis. New York: John Wiley.Google Scholar
  6. Emerson, J. & Strenio, J. (1983). Boxplots and batch comparison, pp. 58–96, in: D. Hoaglin, F. Mosteller & J. Tukey (eds.), Understanding Robust and Exploratory Data Analysis. New York: John Wiley.Google Scholar
  7. Goodall, C. (1983). M-Estimators of location: an outline of the theory, pp. 339–403, in: D. Hoaglin, F. Mosteller & J. Tukey (eds.), Understanding Robust and Exploratory Data Analysis. New York: John Wiley.Google Scholar
  8. Hajnal, J. (1955). The prospects of population forecasts, Journal of the American Statistical Association 50: 309–322.Google Scholar
  9. Hampel, F., Ronchetti, E., Rousseeuw, P. & Stahel,W. (1986). Robust Statistics: An Approach Based on Influence Functions. New York: John Wiley.Google Scholar
  10. Hoaglin, D. (1983). Letter values a set of order statistics, pp. 33–57, in: D. Hoaglin, F. Mosteller & J. Tukey (eds.), Understanding Robust and Exploratory Data Analysis. NewYork: John Wiley.Google Scholar
  11. Huber, P. (1977). Robust Statistical Procedures. Philadelphia: Society for Industrial and Applied Mathematics.Google Scholar
  12. Isserman, A. (1977). The accuracy of population projections for subcounty areas, Journal of the American Institute for Planners 43: 247–259.Google Scholar
  13. Levy, P. & Lemeshow, S. (1991). Sampling of Populations: Methods and Applications. New York: John Wiley.Google Scholar
  14. Makridakis, S. & Hibon, M. (1995). Evaluating accuracy (or error) measures. Fontainebleau, France: INSEADWorking Papers Series 95/18/TM.Google Scholar
  15. Morrison, P. (1971). Demographic Information for Cities: A Manual for Estimating and Projecting Local Population Characteristics. Report R-618, Santa Monica, CA: The Rand Corporation.Google Scholar
  16. Murdock, S., Leistritz, L., Hamm, R., Hwang, S. & Parpia, B. (1984). An assessment of the accuracy of a regional economic-demographic projection model, Demography 21: 383–404.Google Scholar
  17. NRC, National Research Council (1980). Estimating Population and Income for Small Places. Washington, DC: National Academy Press.Google Scholar
  18. Putman, S. (1983). Integrated Urban Models. London: Pion.Google Scholar
  19. Rohatgi, V. (1976). An Introduction to Probability Theory and Mathematical Statistics. New York: John Wiley.Google Scholar
  20. Rosenberger, J. & Gasko, M. (1983). Comparing location estimators: trimmed means, medians, and trimean, pp. 297–338, in: D. Hoaglin, F. Mosteller & J. Tukey (eds.), Understanding Robust and Exploratory Data Analysis. New York: John Wiley.Google Scholar
  21. Smith, S. (1987). Tests of accuracy and bias for county population projections, Journal of the American Statistical Association 82: 991–1003.Google Scholar
  22. Smith, S. & Sincich, T. (1988). Stability over time in the distribution of forecast errors, Demography 25: 461–474.Google Scholar
  23. Smith, S. & Sincich, T. (1990). On the relationship between length of base period and population forecast errors, Journal of The American Statistical Association 85: 367–375.Google Scholar
  24. Smith, S. & Sincich, T. (1992). Evaluating the forecast accuracy and bias of alternative projections for states, International Journal of Forecasting 8: 495–508.Google Scholar
  25. Smith, S. & Shahidullah, M. (1995). An evaluation of population projection errors for census tracts, Journal of the American Statistical Association 90(429): 64–71.Google Scholar
  26. Snedecor, G. & Cochran, W. (1980). Statistical Methods, 7th edn. Ames, IA: Iowa State Press.Google Scholar
  27. Swanson, D. & Tayman, J. (1995). Between a rock and a hard place: the evaluation of demographic forecasts, Population Research and Policy Review 14: 233–249.Google Scholar
  28. Tayman, J. (1996). The accuracy of small area population forecasts based on a spatial interaction land use modeling system, Journal of the American Planning Association 62: 85–98.Google Scholar
  29. Tayman, J. & Swanson, D. (1996). On the utility of population forecasts, Demography 33: 523–528.Google Scholar
  30. Tayman, J. & Swanson, D. (1999). On the validity of MAPE as a measure of population forecast accuracy, Population Research and Policy Review 18: 299–322.Google Scholar
  31. Tayman, J. & Kunkel, S. (1989) Improvements to the projective land use model for producing small area forecasts. Paper presented at the annual meeting of the Population Association of America, Baltimore, Maryland.Google Scholar
  32. Tayman, J., Schafer, E. & Carter, L. (1998). The role of population size in the determination and prediction of population forecast errors: an evaluation using confidence intervals for subcounty areas, Population Research and Policy Review 17: 1–20.Google Scholar
  33. Yokum, J. & Armstrong, J. (1995). Beyond accuracy: comparison of criteria used to select forecasting methods, International Journal of Forecasting 11: 591–597.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Jeff Swanson
    • 1
  • David A. Swanson
    • 2
  • Charles F. Barr
    • 3
  1. 1.San Diego Association of GovernmentsSan Diego
  2. 2.Helsinki School of Economics and Business AdministrationMikkeliFinland
  3. 3.Department of EconomicsUniversity of Nevada Las VegasLas VegasUSA

Personalised recommendations