Skip to main content
SpringerLink
Log in

Assessing Uncertainty in Small Area Forecasts: State of the Practice and Implementation Strategy

  • Published:
Population Research and Policy Review Aims and scope Submit manuscript

Abstract

Forecasts are needed for everyday decisions and must be in the form of numbers. Yet forecasts invariably turn out to be different than the numbers that actually occur. Yet, most producers of forecasts only present a deterministic view of the future in the form of point predictions. However, the presence of uncertainty is inherent in management or policy decisions and there is often concern that benefits are overstated and risks are understated. Such concerns are difficult to address by providing only point forecasts with no assessment of their uncertainty. Having a better understanding of uncertainty can enhance the usefulness of forecasts and make the work of forecasting agencies an even more valuable product for planners, policy makers, and the public. The purpose of this paper is twofold. First, it presents an overview of the current state-of-the-practice is assessing forecast uncertainty. Second, it offers a guidelines and options for implementing and building uncertainty into small area forecasting processes. There are options for assessing forecasting uncertainty that can and should be implemented by most, if not all, producers of forecasts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. This material is based on Smith et al. (2001, Chap. 13).

  2. Accuracy is an important characteristic of a forecast; however, it is not the only criterion on which a forecast should be judged. Forecasts can also be evaluated, for example, on their overall “utility” or their value in improving the quality of information on which decisions are based (Isserman 1984; Moen 1984; Tayman 1996a).

  3. The general ARIMA model is expressed as ARIMA (p, d, q), where p is the order of the autoregressive term; d is the degree of differencing, and q is the order of the moving average term. Model identification refers to the process for determining the best values for p, d, and q. To identify ARIMA parameters, we examined patterns of the autocorrelation and partial autocorrelation functions and their standard errors (Box and Jenkins 1976, Chap. 6), statistical tests for stationarity (Dickey et al. 1986), and the Akaike Information Criterion and Bayesian Information Criterion to avoid an over-parameterized model (Brockwell and Davis 2002, pp. 187–192). We also performed the Portmanteau test on the residuals to ensure the “white noise” or randomness requirement was satisfied for the identified models (Granger and Newbold 1986, pp. 243–244). Details of the identification process are available.

References

  • Ahlburg, D., Lutz, W., & Vaupel, J. (1998). Ways to improve population forecasting: What should be done differently in the future. Population and Development Review, 24, 191–198. (Supplement: Frontiers of Population Forecasting).

    Article  Google Scholar 

  • Alho, J. (1990). Stochastic methods in population forecasting. International Journal of Forecasting, 6, 521–530.

    Article  Google Scholar 

  • Bongaarts, J., & Bulatao, R. (2000). The uncertainty of population forecasts. In J. Bongaarts & R. Bulatao (Eds.), Beyond six billion: Forecasting the world’s population (pp. 118–217). Washington, DC: National Academies Press.

  • Box, G., & Jenkins, G. (1976). Time series analysis: Forecasting and control. San Francisco: Holden Day.

    Google Scholar 

  • Brockwell, P., & Davis, R. (2002). Introduction to time series and forecasting (2nd ed.). New York: Springer-Verlag.

    Book  Google Scholar 

  • Bureau of Economic and Business Research. (2010). Projections of Florida population by county, 2009–2035. Florida population studies Vol. 43 Bulletin 156. Gainesville: University of Florida.

  • Cohen, J. (1986). Population forecasts and confidence intervals for Sweden: A comparison of model-based and empirical approaches. Demography, 23, 105–126.

    Article  Google Scholar 

  • Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9, 456–467.

    Article  Google Scholar 

  • Dickey, D., Bell, W., & Miller, R. (1986). Unit roots in time series models: Tests and implications. American Statistician, 74, 427–431.

    Google Scholar 

  • Granger, C., & Newbold, P. (1986). Forecasting economic time series (2nd ed.). San Diego: Academic Press.

    Google Scholar 

  • Hunt, J., & Abraham, J. (2003). Design and application of the PECAS land use modeling system. Paper presented at the 8th Computers in Urban Planning and Urban Management Conference, Sendai, Japan.

  • Isserman, A. (1984). Population, forecast, and plan: On the future of population forecasting. Journal of the American Planning Association, 50, 208–221.

    Article  Google Scholar 

  • Johnson, R., & McCoy, M. (2006). Assessment of integrated transportation/land use models: Final report. Davis, CA: Information Center for the Environment, University of California Davis.

    Google Scholar 

  • Keilman, N., Pham, D., & Hetland, A. (2002). Why population forecasts should be probabilistic—illustrated by the case of Norway. Demographic Research, 6, 409–453.

    Article  Google Scholar 

  • Keyfitz, N. (1972). On future population. Journal of the American Statistical Association, 67, 347–362.

    Article  Google Scholar 

  • Keyfitz, N. (1981). The limits of population forecasting. Population and Development Review, 7(4), 579–593.

    Article  Google Scholar 

  • Keyfitz, N. (1987). The social and political context of population forecasting. In W. Alonso & P. Starr (Eds.), The politics of numbers (pp. 235–258). New York: Russell Sage Foundation.

    Google Scholar 

  • Lutz, W., & Goldstein, J. (2004). Introduction: how to deal with uncertainty in population forecasting. International Statistical Review, 72, 1–4.

    Article  Google Scholar 

  • Lutz, W., Sanderson, W., & Scherbov, S. (1998). Expert-based probabilistic population projections. Population and Development Review, 24, 139–155. (Supplement: Frontiers of Population Forecasting).

    Article  Google Scholar 

  • McNees, S. (1992). The use and abuses on ‘consensus’ forecasts. Journal of Forecasting, 11, 703–710.

    Article  Google Scholar 

  • Moen, E. (1984). Voodoo forecasting: Technical, political, and ethical issues regarding the projection of local population growth. Population Research and Policy Review, 3, 1–25.

    Article  Google Scholar 

  • Office of National Statistics. (2009). National population projections: 2008-based. Statistical Bulletin, Newport NP10 8XG.

  • Pflaumer, P. (1992). Forecasting U.S. population trends with the Box-Jenkins approach. International Journal of Forecasting, 8, 329–338.

    Article  Google Scholar 

  • Rayer, S., Smith, S., & Tayman, J. (2009). Empirical prediction intervals for county population forecasts. Population Research and Policy Review, 28, 773–793.

    Article  Google Scholar 

  • Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15, 353–375.

    Article  Google Scholar 

  • Rowe, G., & Wright, G. (2001). Expert opinions in forecasting: Role of the Delphi technique. In J. Armstrong (Ed.), Principles of forecasting: A handbook for researchers and practitioners (pp. 125–144). New York: Springer.

    Google Scholar 

  • San Diego Association of Governments. (2006a). 2030 Regional growth forecast. San Diego, CA: San Diego Association of Governments.

    Google Scholar 

  • San Diego Association of Governments. (2006b). Economic impacts of wait times on the San Diego-Baja California border. San Diego, CA: San Diego Association of Governments.

    Google Scholar 

  • San Diego County Water Authority. (2002). Regional water facilities master plan, Appendix C. Development of probabilistic water demand forecast. San Diego, CA: San Diego County Water Authority.

    Google Scholar 

  • Sanderson, W. (1995). Probability, complexity, and catastrophe in a collapsible model of population, development, and environmental interactions. Mathematical Population Studies, 5, 259–279.

    Article  Google Scholar 

  • Ševčíková, H., Raftery, A., & Waddell, P. (2007). Assessing uncertainty in urban simulations using Bayesian melding. Transportation Research Part B: Methodology, 41(6), 652–659.

    Article  Google Scholar 

  • Smith, S., & Rayer, S. (2010). Factors affecting the accuracy of subcounty population forecasts. Journal of Planning Education and Research, 30(2), 1–15.

    Google Scholar 

  • Smith, S., & Shahidullah, M. (1995). An evaluation of population projection errors for census tracts. Journal of the American Statistical Association, 90, 64–71.

    Article  Google Scholar 

  • Smith, S., & Sincich, T. (1988). Stability over time in the distribution of population forecast errors. Demography, 25, 461–474.

    Article  Google Scholar 

  • Smith, S., Tayman, J., & Swanson, D. (2001). State and local population projections: Methodology and analysis. New York: Kluwer Academic.

    Google Scholar 

  • State of Washington. (2010). Forecast of the state population: November 2010 forecast. Olympia, WA: State of Washington.

    Google Scholar 

  • Statistics Netherlands. (2008). Population forecasts 2008–2050. http://statline.cbs.nl/StatWeb/dome/?TH=5420&PA=70851ENG&LA=en.

  • Statistics New Zealand. (2010). Subnational population projections: 2006 (base)–2031 Update. Hot Off the Press. Wellington: Statistics New Zealand

  • Stoto, M. (1983). The accuracy of population projections. Journal of the American Statistical Association, 78, 13–20.

    Article  Google Scholar 

  • Swanson, D., & Tayman, J. (1995). Between and rock and a hard place: The evaluation of demographic forecasts. Population Research and Policy Review, 14(2), 233–249.

    Article  Google Scholar 

  • Tayman, J. (1996a). Forecasting, growth management and public policy decision making. Population Research and Policy Review, 15, 491–508.

    Google Scholar 

  • Tayman, J. (1996b). The accuracy of small area population forecasts based on a spatial interaction modeling system. Journal of the American Planning Association, 62, 85–98.

    Article  Google Scholar 

  • Tayman, J., Schafer, E., & Carter, L. (1998). The role of population size in the determination and prediction of population forecast errors: An evaluation using confidence intervals for subcounty areas. Population Research and Policy Review, 17(1), 1–20.

    Article  Google Scholar 

  • Tayman, J., Smith, S., & Lin, J. (2007). Precision, bias, and uncertainty for state population forecasts: An exploratory analysis of time series models. Population Research and Policy Review, 26, 347–369.

    Article  Google Scholar 

  • United Nations. (2009). World population prospects: The 2008 revision population data base. http://esa.un.org/unpp/ last updated March 11, 2009.

  • Waddell, P. (2002). UrbanSim: Modeling urban development for land use, transportation and environmental planning. Journal of the American Planning Association, 68(3), 297–314.

    Article  Google Scholar 

  • Yokum, J., & Armstrong, S. (1996). Beyond accuracy: Comparison of criteria used to select forecasting models. International Journal of Forecasting, 11, 591–597.

    Article  Google Scholar 

Download references

Acknowledgment

Anubhav Bagley, Rita Walton, and an anonymous referee provided thoughtful comments that greatly improved the manuscript.

Author information

Authors and Affiliations

  1. University of California, San Diego, 9500 Gilman Dr., La Jolla, CA, 92093-0508, USA

    Jeff Tayman

Authors
  1. Jeff Tayman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeff Tayman.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tayman, J. Assessing Uncertainty in Small Area Forecasts: State of the Practice and Implementation Strategy. Popul Res Policy Rev 30, 781–800 (2011). https://doi.org/10.1007/s11113-011-9210-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11113-011-9210-9

Keywords

Search

Navigation