Uncertainty and Error

Chapter

Abstract

Errors in input data, parameterisation, and model form cause errors and uncertainty in model outputs. This is particularly problematic in non-linear systems where small changes propagate through models to create large output differences. This chapter reviews the issues involved in understanding error, covering a broad range of methodologies and viewpoints from across the spatial modelling sciences.

Keywords

Extend Kalman Filter Output Error Monte Carlo Technique Uncertainty Assessment Bayesian Model Average 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Araújo, M., Pearson, R. G., Thuiller, W., & Erhard, M. (2005). Validation of species-climate impact models under climate change. Global Change Biology, 11, 1504–1513.Google Scholar
  2. Ballas, D., Rossiter, D., Thomas, B., Clarke, G. P., & Dorling, D. (2005). Geography matters: Simulating the local impacts of national social policies. York: Joseph Rowntree Foundation.Google Scholar
  3. Barnett, V., & Lewis, T. (1994). Outliers in statistical data (3rd ed.). Chichester: Wiley. pp.604.Google Scholar
  4. Beck, M. B. (1987). Water quality modeling: A review of the analysis of uncertainty. Water Resource Research, 23(8), 1393–1442.Google Scholar
  5. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society B, 57, 289–2300.Google Scholar
  6. Beven, K. J., & Binley, A. M. (1992). The future of distributed models: Model calibration and uncertainty prediction. Hydrological Processes, 6, 279–298.Google Scholar
  7. Björk, B-C., Roos, A., & Lauri, M. (2009). Scientific journal publishing: Yearly volume and open access availability. Information Research 14(1): Paper 391. http://InformationR.net/ir/14-1/paper391.html. Accessed 7 Feb 2011
  8. Bobashev, G. V., & Morris, R. J. (2010). Uncertainty and inference in agent-based models. In Proceedings of the 2010 Second International Conference on Advances in System Simulation. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5601895. Accessed 31 Mar 2011.
  9. Boyle, D. P., Gupta, H. V., & Sorooshian, S. (2000). Towards improved calibration of hydrologic models: Combining the strengths of manual and automatic methods. Water Resources Research, 36, 3663–3674.Google Scholar
  10. Brown, J. D. (2004). Knowledge, uncertainty and physical geography: Towards the development of methodologies for questioning belief. Transactions of the Institute of British Geographers NS, 29, 367–381.Google Scholar
  11. Brown, J. D. (2010). Prospects for the open treatment of uncertainty in environmental research. Progress in Physical Geography, 34(1), 75–100.Google Scholar
  12. Cerioli, A., & Farcomeni, A. (2011). Error rates for multivariate outlier detection. Computational Statistics & Data Analysis, 55(1), 544–553.Google Scholar
  13. Cho, W. K. T., & Gaines, B. J. (2007). Breaking the (Benford) Law: Statistical fraud detection in Campaign Finance. The American Statistician, 61(3), 218–223.Google Scholar
  14. Chowdhury, S., & Sharma, A. (2007). Mitigating parameter bias in hydrological modelling due to uncertainty in covariates. Journal of Hydrology, 340, 197–204.Google Scholar
  15. Clancy, D., Tanner, J. E., McWilliam, S., & Spencer, M. (2010). Quantifying parameter uncertainty in a coral reef model using Metropolis-Coupled Markov Chain Monte Carlo. Ecological Modelling, 221, 1337–1347.Google Scholar
  16. Clark, J. S. (2005). Why environmental scientists are becoming Bayesians. Ecological Letters, 8, 2–14.Google Scholar
  17. Clark, J. S., & Gelfand, A. E. (2006). A future for models and data in environmental science. Trends in Ecology and Evolution, 21(7), 375–380.Google Scholar
  18. Collins, M., & Knight, S. (2007). Theme issue ‘Ensembles and probabilities: A new era in the prediction of climate change’. Philosophical Transactions of the Royal Society A, 365, 1957–2191.Google Scholar
  19. Cook, J. R., & Stefanski, L. A. (1994). Simulation-extrapolation estimation in parametric measurement error models. Journal of the American Statistical Association, 89(428), 1314–1328.Google Scholar
  20. Cornélis, B., & Brunet, S. (2002). A policy-maker point of view on uncertainties in spatial decisions. In W. Shi, P. F. Fisher, & M. F. Goodchild (Eds.), Spatial data quality (pp. 168–185). London: Taylor and Frances.Google Scholar
  21. Cornford, D. (2011). Uncertainty and the OGC: An Aston University take. Presented 03/03/2011 at OGC TC Meeting, Universities DWG Session, Bonn. http://www.uncertweb.org/documents. Accessed 1 Mar 2011.
  22. Costanza, R. (1989). Model goodness of fit: A multiple resolution procedure. Ecological Modelling, 47, 199–215.Google Scholar
  23. Couclelis, H. (2003). The certainty of uncertainty: GIS and the limits of geographic knowledge. Transactions in GIS, 7(2), 165–175.Google Scholar
  24. Dassonville, L., Vauglin, F., Jakobsson, A., & Luzet, C. (2002). Quality management, data quality and users, metadata for geographical information. In W. Shi, P. F. Fisher, & M. F. Goodchild (Eds.), Spatial data quality (pp. 202–215). London: Taylor and Frances.Google Scholar
  25. Dewulf, A., Craps, M., Bouwen, R., Taillieu, T., & Pahl-Wostl, C. (2005). Integrated management of natural resources: Dealing with ambiguous issues, multiple actors and diverging frames. Water Science & Technology, 52, 115–124.Google Scholar
  26. Di Paolo, E. A, Noble, J, & Bullock, S (2000) Simulation models as opaque thought experiments. In Seventh International Conference on Artificial Life (pp. 497–506) Cambridge: MIT Press,. http://eprints.ecs.soton.ac.uk/11455/
  27. Drecki, I. (2002). Visualisation of uncertainty in geographical data. In W. Shi, P. F. Fisher, & M. F. Goodchild (Eds.), Spatial data quality (pp. 140–159). London: Taylor and Frances.Google Scholar
  28. Dubois, G., & Saisana, M. (2002). Optimal spatial declustering weights — Comparison of methods. In Proceedings of the Annual Conference of the International Association for Mathematical Geology, 15–20 Sept Berlin, 479–484. http://composite-indicators.jrc.ec.europa.eu/Document/OptimizingSpatialDeclusteringWeights-ComparisonofMethods.pdf. Accessed 31 Mar 2011.
  29. Evans, A. J., & Waters, T. (2007). Mapping vernacular geography: Web-based GIS tools for capturing “fuzzy” or “vague” entities. International Journal of Technology, Policy and Management, 7(2), 134–150.Google Scholar
  30. Evensen, G. (1992). Using the extended Kalman filter with a multilayer quasi-geostrophic ocean model. Journal of Geophysical Research, 97(C11), 17905–17924.Google Scholar
  31. Evensen, G. (1994). Sequential data assimilation with nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. Journal of Geophysical Research, 99(C5), 143–162.Google Scholar
  32. Faber, M., Manstetten, R., & Proops, J. (1992). Humankind and the environment: An anatomy of surprise and ignorance. Environmental Values, 1, 217–241.Google Scholar
  33. Farcomeni, A. (2008). A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion. Statistical Methods in Medical Research, 17, 347–388.Google Scholar
  34. Fisher, P., Wood, J., & Cheng, T. (2004). Where is Helvellyn? Fuzziness of multi-scale landscape morphometry. Transactions of the Institute of British Geographers, 29(1), 106–128.Google Scholar
  35. Funtowicz, S. O., & Ravetz, J. R. (1990). Uncertainty and quality in science for policy. Dordrecht: Kluwer Academic Publishers.Google Scholar
  36. Funtowicz, S. O., & Ravetz, J. R. (1993). Science for the post-normal age. Futures, 25, 739–755.Google Scholar
  37. Galán, J. M., Izquierdo, L. R., Izquierdo, S. S., Santos, J. I., del Olmo, R., López-Paredes, A., & Edmonds, B. (2009). Errors and artefacts in agent-based modelling. Journal of Artificial Societies and Social Simulation 12(1): 1. http://jasss.soc.surrey.ac.uk/12/1/1.html. Accessed 25 Mar 2011.
  38. Gallagher, M., & Doherty, J. (2007). Parameter estimation and uncertainty analysis for a watershed model. Environmental Modelling and Software, 22, 1000–1020.Google Scholar
  39. Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). Design patterns: Elements of reusable object-oriented software. Reading, Mass: Addison Wesley. pp.416.Google Scholar
  40. Gan, E., & Shi, W. (2002). Error metadata management systems. In W. Shi, P. F. Fisher, & M. F. Goodchild (Eds.), Spatial data quality (pp. 251–266). London: Taylor and Frances.Google Scholar
  41. George, E. I. (2000). The variable selection problem. Journal of the American Statistical Association, 95(452), 1304–1308.Google Scholar
  42. Getis, A. (2007). Reflections on spatial autocorrelation. Reg Sci Urban Econ, 37, 491–496.Google Scholar
  43. Glahn, H., & Lowry, D. (1972). The use of Model Output Statistics (MOS) in objective weather forecasting. Journal of Applied Meteorology, 11, 1203–1211.Google Scholar
  44. Gneiting, T., & Raftery, A. E. (2005). Weather forecasting with ensemble methods. Science, 310(5746), 248–249.Google Scholar
  45. Green, S. B., & Babyak, M. A. (1997). Control of type I errors with multiple tests constraints in structural equation modeling. Multivariable Behavioural Research, 32, 39–51.Google Scholar
  46. Groves, R. M., Fowler, F. J., Jr., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology. Hoboken: Wiley-Blackwell. pp.488.Google Scholar
  47. Gupta, H. V. (1999). Status of automatic calibration for hydrologic models: Comparison with multilevel expert calibration. Journal of Hydrologic Engineering, 4(2), 135–143.Google Scholar
  48. Gupta, H. V., Sorooshian, S., & Yapo, P. O. (1998). Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information. Water Resources Research, 34(4), 751–763.Google Scholar
  49. Hamby, D. M. (1994). A review of techniques for parameter sensitivity analysis of environmental models. Environmental Monitoring and Assessment, 32, 135–154.Google Scholar
  50. Hartley, S., Harris, R., & Lester, P. J. (2006). Quantifying uncertainty in the potential distribution of an invasive species: Climate and the Argentine ant. Ecology Letters, 9, 1068–1079.Google Scholar
  51. Hasings, D., & McManus, H. (2004). A framework for understanding uncertainty and its mitigation and exploitation in complex systems. In 2004 Engineering Systems Symposium, 29–31 Mar 2004, MIT. http://esd.mit.edu/symposium/pdfs/papers/hastings.pdf. Accessed 28 February 2011.
  52. Hassan, S., Garmendia, L., & Pavon, J. (2010). Introducing uncertainty into social simulation: Using fuzzy logic for agent-based modelling. International Journal of Reasoning-based Intelligent Systems, 2(2), 118–124.Google Scholar
  53. Hatton, L. (1997). The T experiments: Errors in scientific software. Computational Science & Engineering, IEEE, 4(2), 27–38.Google Scholar
  54. Hayes, B. (2003). A Lucid interval. American Scientist, 91(6), 484–488.Google Scholar
  55. Helton, J. C., Johnson, J. D., Salaberry, C. J., & Storlie, C. B. (2006). Survey of sampling based methods for uncertainty and sensitivity analysis. Reliability Engineering and System Safety, 91, 1175–1209.Google Scholar
  56. Heppenstall, A. J., Evans, A. J., & Birkin, M. H. (2007). Genetic algorithm optimisation of a multi-agent system for simulating a retail market. Environment and Planning B, 34(6), 1051–1070.Google Scholar
  57. Heuvelink, G. B. M. (1998). Error propagation in environmental modelling with GIS. London: Taylor and Francis. pp.127.Google Scholar
  58. Hornberger, G., & Spear, R. (1981). An approach to the preliminary analysis of environmental systems. Journal of Environmental Management, 7, 7–18.Google Scholar
  59. Hyde, R. (2004). Write great code 1: Understanding the machine. San Francisco: No Starch Press. pp. 440.Google Scholar
  60. Isaaks, E. H., & Srivastava, R. M. (1990). Applied geostatistics. Oxford: Oxford University Press. pp.592.Google Scholar
  61. Izquierdo, L. R., & Polhill, J. G. (2006). Is your model susceptible to floating-point errors? Journal of Artificial Societies and Social Simulation 9(4): 4. http://jasss.soc.surrey.ac.uk/9/4/4.html. Accessed 25 Mar 2011.
  62. Janssen, J. A. E. B., Krol, M. S., Schielen, R. M. J., Hoekstra, A. Y., & de Kok, J.-L. (2010). Assessment of uncertainties in expert knowledge, illustrated in fuzzy rule-based models. Ecological Modelling, 221, 1245–1251.Google Scholar
  63. JCGM (2008a) Evaluation of measurement data – Supplement 1 to the “Guide to the expression of uncertainty in measurement” – Propagation of distributions using a Monte Carlo method, Joint Committee for Guides in Metrology 100. http://www.bipm.org/utils/common/documents/jcgm/JCGM_101_2008_E.pdf. Accessed 15 Feb 2011.
  64. JCGM (2008b) Evaluation of measurement data – Guide to the expression of uncertainty in measurement, Joint Committee for Guides in Metrology 100. http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf. Accessed 15 Feb 2011.
  65. Johnson, J. B., & Omland, K. S. (2004). Model selection in ecology and evolution. Trends in Ecology & Evolution, 19, 101–108.Google Scholar
  66. Kardos, J. D., Moore, A., & Benwell, G. L. (2003). Visualising uncertainty in spatially-referenced attribute data using hierarchical spatial data structures. In Proceedings of the 7th International Conference on GeoComputation. University of Southampton, 8–10 Sept 2003. http://www.geocomputation.org/2003/. Accessed 25 Mar 2011
  67. Kavetski, D., Kuczera, G., & Franks, S. W. (2006). Bayesian analysis of input uncertainty in hydrological modeling: 1. Theory. Water Resources Research, 42, W03407.Google Scholar
  68. Keesman, K., & van Straten, G. (1989). Identification and prediction propagation of uncertainty in models with bounded noise. International Journal of Control, 49, 2259–2269.Google Scholar
  69. Keesman, K., & van Straten, G. (1990). Set-membership approach to identification and prediction of lake eutrophication. Water Resources Research, 26, 2643–2652.Google Scholar
  70. Kelejian, H., & Prucha, I. (2010). Spatial models with spatially lagged dependent variables and incomplete data. Journal of Geographical Systems, 12(3), 241–257.Google Scholar
  71. Kennedy, M. C., & O’Hagan, A. (2001). Bayesian calibration of computer models. Journal of the Royal Statistical Society B, 63(3), 425–464.Google Scholar
  72. Kiesling, J. B. (undated) Charting Electoral Fraud: Turnout Distribution Analysis as a Tool for Election Assessment. Diplomacy Lessons. http://www.bradykiesling.com/election_fraud_analysis.htm. Accessed 10 Jan 2010.
  73. King, G. (1986). How not to lie with statistics: Avoiding common mistakes in quantitative political science. American Journal of Political Science, 30, 666–687.Google Scholar
  74. Knudsen, C. D., & Fotheringham, A. S. (1986). Matrix comparison, goodness-of-fit and spatial interaction modeling. International Regional Science Review, 10(2), 127–147.Google Scholar
  75. Ku, H. (1966). Notes on the use of propagation of error formulas. Journal of Research of National Bureau of Standards C. Engineering and Instrumentation, 70(4), 263–273.Google Scholar
  76. Laffan, S. W. (1999). Spatially assessing model error using geographically weighted regression. In Proceedings of the 4th International Conference on GeoComputation. Fredericksburg: Mary Washington College, 5–28 July 1999. http://www.geocomputation.org/1999/. Accessed 25 Mar 2011.
  77. Lane, S. N. (2001). Constructive comments on D Massey ‘Space-time, “science” and the relationship between physical geography and human geography’. Transactions of the Institute of British Geographers, 6, 243–256.Google Scholar
  78. Lauridsen, J., & Kosfeld, R. (2007). Spatial cointegration and heteroscedasticity. Journal of Geographical Systems, 9(3), 253–265.Google Scholar
  79. Legendre, P. (1993). Spatial Autocorrelation: Trouble or New Paradigm? Ecology, 74, 1659–1673.Google Scholar
  80. Leith, C. E. (1974). Theoretical skill of Monte Carlo forecasts. Monthly Weather Review, 102(6), 409–418.Google Scholar
  81. Li, Y., Brimicombe, A. J., & Li, C. (2007). Agent-based services for validating multi-agent models. In Proceedings of the 9th International Conference on GeoComputation, Maynooth: Eire.Google Scholar
  82. Lodwick, W. A. (1989). Developing confidence limits on errors of suitability analyses in geographical information systems. In M. Goodchild & S. Gopal (Eds.), Accuracy of spatial databases (pp. 69–80). London: Taylor and Francis.Google Scholar
  83. López, C. (1997). Quality of geographic data – Detection of outliers and imputation of missing values (Unpublished PhD thesis, Universidad ORT, Uruguay). http://www.thedigitalmap.com/~carlos/papers/PhDthesis/phdthesis.pdf. Accessed 1 Mar 2011.
  84. Lorenz, E. N. (1963). Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20, 130–141.Google Scholar
  85. Luoto, M., Marmion, M., & Hjort, J. (2010). Assessing spatial uncertainty in predictive geomorphological mapping: A multi-modelling approach. Computers & Geosciences, 36, 355–361.Google Scholar
  86. Lutz, W., Sanderson, W., & Scherbov, S. (1996). Probabilistic population projections based on expert opinion. In W. Lutz (Ed.), The future population of the world. What can we assume today? (pp. 397–428). London: Earthscan Publications. Chapter 16.Google Scholar
  87. Lyman, R. L., & Ames, K. M. (2007). On the use of species-area curves to detect the effects of sample size. Journal of Archaeological Science, 34, 1985–1990.Google Scholar
  88. Malinarič, S., & Ďuríšek, P. (2004). Sensitivity coefficients analysis. In Proceedings of Thermophysics 2004. http://www.tpl.fpv.ukf.sk/engl_vers/thermophys/2004/Mal-Dur.pdf. Accessed 14 Feb 2011.
  89. Malleson, N. (2010). Agent-based modelling of Burglary (Unpublished PhD thesis, School of Geography, University of Leeds, Leeds). http://www.geog.leeds.ac.uk/fileadmin/downloads/school/people/postgrads/n.malleson/thesis-final.pdf. Accessed 1 Mar 2011.
  90. Marin, C. M., Guvanasen, V., & Saleem, Z. A. (2003). The 3MRA risk assessment framework – A flexible approach for performing multimedia, multipathway, and multireceptor risk assessments under uncertainty. Human and Ecological Risk Assessment: An International Journal, 9(7), 1655–1677.Google Scholar
  91. Martin, D. (2003). Extending the automated zoning procedure to reconcile incompatible zoning systems. International Journal of Geographical Information Science, 17, 181–196.Google Scholar
  92. Martin, D., Dorling, D., & Mitchell, R. (2002). Linking censuses through time: problems and solutions Area, 34, 82–91.Google Scholar
  93. Matott, L. S., Babendreier, J. E., & Parucker, S. T. (2009). Evaluating uncertainty in integrated environmental models: A review of concepts and tools. Water Resources Research 45: WO6421. http://www.agu.org/journals/wr/wr0906/2008WR007301/2008WR007301.pdf. Accessed 2 Feb 2011.
  94. McKay, M. D., Beckman, R. J., & Conover, W. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21(2), 239–245.Google Scholar
  95. Mearns, L. O. (2010). The drama of uncertainty. Climatic Change, 100, 77–85.Google Scholar
  96. Mebane, W. R Jr., & Kalinin, K. (2009). Comparative election fraud detection. In Annual Meeting of the American Political Science Association, Toronto 3–6 Sept 2009. http://www-personal.umich.edu/~wmebane/apsa09.pdf. Accessed 10 Jan 2010.
  97. Mitchell, S., Beven, K., & Freer, J. (2009). Multiple sources of predictive uncertainty in modeled estimates of net ecosystem CO2 exchange. Ecological Modelling, 220(23), 3259–3270.Google Scholar
  98. Monaco, J. H., & Anderson, R. L. (1994). Tai’s formula is the trapezoidal rule. Diabetes Care, 17(10), 1224–1225.Google Scholar
  99. Moreno, N., Wang, F., & Marceau, D. J. (2008). An object-based land-use cellular automata model to overcome cell size and neighborhood sensitivity. In Proceedings OF GEOBIA 2008 – Pixels, Objects, Intelligence GEOgraphic Object Based Image Analysis for the 21st Century. Calgary, 5–8 Aug. http://www.isprs.org/proceedings/XXXVIII/4-C1/Sessions/Session6/6753_Marceau_Proc_pap.pdf. Accessed 25 Mar 2011.
  100. Morss, R. E., Demuth, J. L., & Lazo, J. K. (2008). Communicating uncertainty in weather forecasts: a survey of the US public. Weather and Forecasting, 23,974–91.Google Scholar
  101. Nagele, P. (2001). Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. British Journal of Anaesthesia, 90(4), 514–516.Google Scholar
  102. National Research Council of the National Academies (NRC). (2006). Completing the forecast: Characterizing and communicating uncertainty for better decisions using weather and climate forecasts. Washington, DC: National Academies Press.Google Scholar
  103. Ngo, T. A., & See, L. M. (2012). Calibration and validation of agent-based models of land cover change. In A. J. Heppenstall, A. T. Crooks, L. M. See, & M. Batty (Eds.), Agent-based models of geographical systems (pp. 181–196). Dordrecht: Springer.Google Scholar
  104. NIST/SEMATECH (2010). e-Handbook of Statistical Methods, http://www.itl.nist.gov/div898/handbook/. Accessed 10 Jan 2011.
  105. O’Neill, R. V. (1973). Error analysis of ecological models. In D. J. Nelson (Ed.), Radionuclides in ecosystems. CONF-710501 (pp. 898–908). Springfield: National Technical Information Service.Google Scholar
  106. O’Neill, R. V., & Rust, B. (1979). Aggregation error in ecological models. Ecological Modelling, 7(2), 91–105.Google Scholar
  107. O’Neill, R. V., Gardner, R. H., & Mankin, J. B. (1980). Analysis of parameter error in a nonlinear model. Ecological Modelling, 8, 297–311.Google Scholar
  108. Openshaw, S. (1989). Learning to live with errors in spatial databases. In M. Goodchild & S. Gopal (Eds.), Accuracy of spatial databases (pp. 263–276). London: Taylor and Francis.Google Scholar
  109. Polhill, J. G., Izquierdo, L. R., & Gotts, N. M. (2006). What every agent-based modeller should know about floating point arithmetic. Environmental Modelling & Software, 21(3), 283–309.Google Scholar
  110. Poli, R, Langdon, W. B., and McPhee, N. F., with Koza, J. R. (2008). A Field Guide to Genetic Programming. http://www.gp-field-guide.org.uk/. Accessed 28 February 2011.
  111. Poulton, E. C. (1989). Bias in quantifying judgments. Hove: Erlbaum. pp.328.Google Scholar
  112. Powers, A. C. (2005). Simulating patterns of uncertainty in postclassification change detection. In Proceedings of the 8th International Conference on GeoComputation. University of Michigan, 31 July–3 Aug 2005. http://www.geocomputation.org/2005/. Accessed 25 Mar 2011.
  113. Raftery, A. E., Gneiting, T., Balabdaoui, F., & Polakowski, M. (2005). Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review, 133, 1155–1174.Google Scholar
  114. Refsgaard, J. C., Henriksen, H. J., Harrar, W. G., Scholten, H., & Kassahun, A. (2005). Quality assurance in model based water management – Review of existing practice and outline of new approaches. Environmental Modelling & Software, 20(10), 1201–1215.Google Scholar
  115. Refsgaard, J. C., van der Sluijs, J. P., Etejberg, A. L., & Vanrolleghem, P. A. (2007). Uncertainty in the environmental modeling process – A framework and guidance. Environmental Modelling and Software, 22, 1543–1556.Google Scholar
  116. Reinke, K., & Hunter, G. J. (2002). A theory for communicating uncertainty in spatial databases. In W. Shi, P. F. Fisher, & M. F. Goodchild (Eds.), Spatial data quality (pp. 76–101). London: Taylor and Frances.Google Scholar
  117. Rogers, J. P., Barbara, D., & Domeniconi, C. (2009). Detecting spatio-temporal outliers with kernels and statistical testing. In 17th International Conference on Geoinformatics. http://dx.doi.org/10.1109/GEOINFORMATICS.2009.5293481. Accessed 1 Mar 2011.
  118. Rowe, W. D. (1977). An anatomy of risk. New York: John Wiley and Sons.Google Scholar
  119. Saltelli, A., Chan, K., & Scott, E. M. (2000). Sensitivity analysis. Chichester: Wiley. pp.475.Google Scholar
  120. Schönfisch, B., & de Roos, A. (1999). Synchronous and asynchronous updating in cellular automata Biosystems, 51(3), 123–143.Google Scholar
  121. Shackley, S., & Wynne, B. (1996). Representing uncertainty in global climate change science and policy: Boundary ordering devices and authority. Science Technology Human Values, 21, 275–302.Google Scholar
  122. Smith, L. A., Cuéllar, M. C., Du, H., & Judd, K. (2010a). Exploiting dynamical coherence: A geometric approach to parameter estimation in nonlinear models. Physics Letters A, 374, 2618–2623.Google Scholar
  123. Smith, A. H. C., Ponci, F., & Monti, A. (2010b). Bounding the dynamic behavior of an uncertain system via polynomial chaos-based simulation. Simulation, 86, 31–40.Google Scholar
  124. Smithson, M. (1989). Ignorance and uncertainty: Emerging paradigms. New York: Springer.Google Scholar
  125. Spear, R. C. (1970). The application of Kolmogorov-Renyi statistics to problems of parameter uncertainty in systems design. International Journal of Control, 11, 771–778.Google Scholar
  126. Spetzler, C. S., & von Holstein, S. (1975). Probability encoding in decision analysis. Management Science, 22(3), 340–358.Google Scholar
  127. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & van der Linde, A. (2002). Bayesian measures of model complexity and fit (with Discussion). Journal of the Royal Statistical Society B, 64, 583–639.Google Scholar
  128. Suter, G. W., Barnthouse, L. W., & O’Neill, R. V. (1987). Treatment of risk in environmental impact assessment. Environmental Management, 11, 295–303.Google Scholar
  129. Tang, S., & Wang, Y. (2001). A parameter estimation program for error-in-variable model. Ecological Modelling, 156, 225–236.Google Scholar
  130. Tobler, W. R. (1979). Smooth pycnophylactic interpolation for geographical regions. Journal of the American Statistical Association, 74, 519–530.Google Scholar
  131. Toth, Z., & Kalnay, E. (1993). Ensemble forecasting at NMC: The generation of perturbations. Bulletin of American Meteorological Society, 74, 2317–2330.Google Scholar
  132. Urban, N. M., & Fricker, T. E. (2010). A comparison of Latin hypercube and grid ensemble designs for the multivariate emulation of an Earth system model. Computers & Geosciences, 36, 746–755.Google Scholar
  133. Van der Sluijs, J. P., Potting, J., Risbey, J., van Vuuren, D., de Vries, B., Beusen, A., Heuberger, P., Quintana, S. C., Funtowicz, S., Kloprogge, P., Nuijten, D., Petersen, A. C., & Ravetz, J. (2002). Uncertainty assessment of the IMAGE  =  TIMER B1 CO2 emissions sScenario, using the NUSAP method. (Report No. 410 200 104). Bilthoven: Dutch National Research Program on Climate Change.Google Scholar
  134. Van der Sluijs, J. P., Risbey, J., Kloprogge,P., Ravetz, J. R., Funtowicz, S. O., Quintana, S. C., Pereira, A. G., De Marchi, B., Petersen, A. C., Janssen, P. H. M., Hoppe, R., & Huijs, S. W. F. (2003). RIVM/MNP guidance for uncertainty assessment and communication. http://www.nusap.net/downloads/detailedguidance.pdf. Accessed 14 Feb 2011.
  135. van der Wel, F. J. M., van der Gaag, L. C., & Gorte, B. G. H. (1996). Visual exploration of uncertainty in remote sensing classifications. In Proceedings of the 1st International Conference on GeoComputation. Leeds: University of Leeds, 17–19 Sept 1996. http://www.geocomputation.org/1996/. Accessed 25 Mar 2011.
  136. van Straten, G. (1985). Analytical methods for parameter-space delimitation and application to shallow-lake phytoplankton-dynamics modeling. Applied Mathematics and Computation, 17, 459–482.Google Scholar
  137. Verkuilen, J. (2005). Assigning membership in a fuzzy set analysis. Sociological Methods & Research, 33(4), 462–496.Google Scholar
  138. Viard, T., Caumon, G., & Lévy, B. (2011). Adjacent versus coincident representations of geospatial uncertainty: Which promote better decisions? Computers & Geosciences, 37(4), 511–520.Google Scholar
  139. Wagener, T., McIntyre, N., Lees, M. J., Wheater, H. S., & Gupta, H. V. (2003). Towards reduced uncertainty in conceptual rainfall-runoff modelling: Dynamic identifiability analysis. Hydrological Processes, 17(2), 455–476.Google Scholar
  140. Walker, W. W. (1982). A sensitivity and error analysis framework for lake eutrophication modeling. Water Resources Bulletin, 18(1), 53–60.Google Scholar
  141. Walker, W. E., Harremoes, P., Rotmans, J., van der Sluijs, J. P., van Asselt, M. B. A., Janssen, P., et al. (2003). Defining uncertainty: A conceptual basis for uncertainty management in model-based decision support. Integrated Assessment, 4(1), 5–18.Google Scholar
  142. Warren, H. S., Jr. (2002). Hacker’s delight. Boston: Addison Wesley. pp. 306.Google Scholar
  143. Williams, M., Cornford, D., Bastin, L., & Ingram, B. (2008). UncertML: an XML schema for exchanging uncertainty. In Proceedings of GISRUK 2008. http://www.uncertml.org/publications.php. Accessed 1 Mar 2011.
  144. Wooldridge, M. (2009). An introduction to multiagent systems (2nd ed.). Chichester: Wiley. pp. 484.Google Scholar
  145. Wyss, G. D., & Jorgensen, K. H. (1998). A user’s guide to LHS: Sandia’s Latin Hypercube Sampling Software. SAND98-0210 Albuquerque: Sandia National Laboratories.Google Scholar
  146. Young, P. C. (2002). Advances in real-time flood forecasting. Philosophical Transactions of the Royal Society of London A, 360, 1433–1450.Google Scholar
  147. Young, P. C., Parkinson, S., & Lees, M. J. (1996). Simplicity out of complexity in environmental modelling: Occam’s razor revisited. Journal of Applied Statistics, 23, 165–210.Google Scholar
  148. Zadeh, L. A. (1975). The concept of a linguistic variable and its application to approximate reasoning–1. Information Sciences, 8, 199–249.Google Scholar
  149. Zadeh, L. A. (1976). A fuzzy-algorithmic approach to the definition of complex or imprecise concepts. International Journal of Man-Machine Studies, 8(3), 249–291.Google Scholar
  150. Zadeh, L. A. (2005). Toward a generalized theory of uncertainty (GTU) – An outline. Information Sciences, 172, 1–40.Google Scholar
  151. Zellner, M. L. (2008). Embracing complexity and uncertainty: The potential of agent-based modeling for environmental planning and policy. Planning Theory & Practice, 9(4), 437–457.Google Scholar
  152. Zhang, J., & Goodchild, M. (2002). Uncertainty in geographical information (p. 266). London: Taylor and Francis.Google Scholar
  153. Žižek, S. (2004). What Rumsfeld doesn’t know that he knows about Abu Ghraib. In These Times, 21 May 2004. http://www.lacan.com/zizekrumsfeld.htm. Accessed 10 Jan 2010.

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  1. 1.Centre for Applied Spatial Analysis and Policy (ASAP)University of LeedsLeedsUK

Personalised recommendations