Map Comparison Methods for Comprehensive Assessment of Geosimulation Models

  • Alex Hagen-Zanker
  • Pim Martens
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5072)


A crucial task in the calibration and validation of geosimulation models is to measure the agreement between model and reality. In recent years many map comparison methods have been developed for this purpose. This paper presents a framework to systematically assess different aspects of model performance and express the results relative to a common reference level. Application on a constrained cellular automata model of the Netherlands demonstrates that the framework gives an in-depth account of model performance. It also shows that any performance assessment that does not follow a multi-criteria approach or lacks a reference level results in an unbalanced account and ultimately false conclusions.


geosimulation calibration validation map comparison 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Benenson, I., Torrens, P.M.: Geosimulation: object-based modeling of urban phenomena. Comput. Environ. Urban Syst. 28, 1–8 (2004)CrossRefGoogle Scholar
  2. 2.
    Refsgaard, J.C., Henriksen, H.J.: Modelling guidelines: terminology and guiding principles. Adv. Water Resour. 27, 71–82 (2004)CrossRefGoogle Scholar
  3. 3.
    Costanza, R.: Model goodness of fit: a multiple resolution procedure. Ecol. Model 47, 199–215 (1989)CrossRefGoogle Scholar
  4. 4.
    Hagen, A.: Fuzzy set approach to assessing similarity of categorical maps. Int. J. Geog. Inf. Sci. 17, 235–249 (2003)CrossRefGoogle Scholar
  5. 5.
    Hagen-Zanker, A.: Map comparison methods that simultaneously address overlap and structure. J. Geogr. Syst. 8, 165–185 (2006)CrossRefGoogle Scholar
  6. 6.
    Kuhnert, M., Voinov, A., Seppelt, R.: Comparing raster map comparison algorithms for spatial modeling and analysis. Photogramm. Eng. Remote Sens. 71, 975–984 (2005)Google Scholar
  7. 7.
    Pontius Jr., R.G.: Quantification error versus location error in comparison of categorical maps. Photogramm. Eng. Remote Sens. 66, 1011–1016 (2000)Google Scholar
  8. 8.
    Power, C., Simms, A., White, R.: Hierarchical fuzzy pattern matching for the regional comparison of land use maps. Int. J. Geog. Inf. Sci. 15, 77–100 (2001)CrossRefGoogle Scholar
  9. 9.
    Remmel, T.K., Csillag, F.: Mutual information spectra for comparing categorical maps. Int. J. Remote Sens. 27, 1425–1452 (2006)CrossRefGoogle Scholar
  10. 10.
    Turner, M.G., Costanza, R., Sklar, F.H.: Methods to evaluate the performance of spatial simulation-models. Ecol. Model. 48, 1–18 (1989)CrossRefGoogle Scholar
  11. 11.
    White, R.: Pattern based map comparisons. J. Geogr. Syst. 8, 145–164 (2006)CrossRefGoogle Scholar
  12. 12.
    Batty, M., Torrens, P.M.: Modelling and prediction in a complex world. Futures 37, 745–766 (2005)CrossRefGoogle Scholar
  13. 13.
    Schelling, T.C.: Dynamic models of segregation. J. Math. Sociol. 1, 143–186 (1971)Google Scholar
  14. 14.
    Benenson, I., Omer, I., Hatna, E.: Entity-based modeling of urban residential dynamics: the case of Yaffo, Tel Aviv. Environ. Plann. B 29, 491–512 (2002)CrossRefGoogle Scholar
  15. 15.
    White, R., Engelen, G., Uljee, I.: The use of constrained cellular automata for high-resolution modelling of urban land-use dynamics. Environ. Plann. B 24, 323–343 (1997)CrossRefGoogle Scholar
  16. 16.
    Engelen, G., White, R., Uljee, I., Drazan, P.: Using cellular automata for integrated modelling of socio-environmental systems. Environ. Monit. Assess. 34, 203–214 (1995)CrossRefGoogle Scholar
  17. 17.
    White, R., Engelen, G.: High-resolution integrated modelling of the spatial dynamics of urban and regional systems. Comput. Environ. Urban Syst. 24, 383–400 (2000)CrossRefGoogle Scholar
  18. 18.
    Engelen, G., White, R., de Nijs, T.: Environment Explorer: spatial support system for the integrated assessment of socio-economic and environmental policies in the Netherlands. Integr. Assess. 4, 97–105 (2003)CrossRefGoogle Scholar
  19. 19.
    de Nijs, T.C.M., de Niet, R., Crommentuijn, L.: Constructing land-use maps of the Netherlands in 2030. J. Environ. Manage. 72, 35–42 (2004)CrossRefGoogle Scholar
  20. 20.
    Barredo, J.I., Demicheli, L.: Urban sustainability in developing countries’ megacities: modelling and predicting future urban growth in Lagos. Cities 20, 297–310 (2003)CrossRefGoogle Scholar
  21. 21.
    Takeyama, M., Couclelis, H.: Map dynamics: integrating cellular automata and GIS through geo-algebra. Int. J. Geog. Inf. Sci. 11, 73–91 (1997)CrossRefGoogle Scholar
  22. 22.
    Monserud, R.A., Leemans, R.: Comparing global vegetation maps with the Kappa statistic. Ecol. Model. 62, 275–293 (1992)CrossRefGoogle Scholar
  23. 23.
    Batty, M., Longley, P.: Fractal cities: a geometry of form and function. Academic Press Professional, Inc., San Diego (1994)zbMATHGoogle Scholar
  24. 24.
    Dungan, J.L.: Focusing on feature-based differences in map comparison. J. Geogr. Syst. 8, 131–143 (2006)CrossRefGoogle Scholar
  25. 25.
    Kok, K., Farrow, A., Veldkamp, A., Verburg, P.H.: A method and application of multi-scale validation in spatial land use models. Agricult. Ecosys. Environ. 85, 223–238 (2001)CrossRefGoogle Scholar
  26. 26.
    Pontius Jr., R.G.: Statistical methods to partition effects of quantity and location during comparison of categorical maps at multiple resolutions. Photogramm. Eng. Remote Sens. 68, 1041–1049 (2002)Google Scholar
  27. 27.
    McGarigal, K., Cushman, S.A., Neel, M.C., Ene, R.: FRAGSTATS: spatial pattern analysis program for categorical maps. Computer software program produced by the authors at the University of Massachusetts, Amherst (2002),
  28. 28.
    Foody, G.M.: Status of land cover classification accuracy assessment. Remote Sens. Environ. 80, 185–201 (2002)CrossRefGoogle Scholar
  29. 29.
    Cohen, J.A.: Coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)CrossRefGoogle Scholar
  30. 30.
    Heidke, P.: Berechnung des Erfolges und der Gute der Windstarkevorhersagen im Sturmwarnungsdienst. Geogr. Ann. 8, 301–349 (1926)CrossRefGoogle Scholar
  31. 31.
    de Keersmaecker, M.L., Frankhauser, P., Thomas, I.: Using fractal dimensions for characterizing intra-urban diversity: the example of Brussels. Geogr. Anal. 35, 310–329 (2003)CrossRefGoogle Scholar
  32. 32.
    Benguigui, L., Blumenfeld-Lieberthal, E., Czamanksi, D.: The dynamics of the Tel Aviv morphology. Environ. Plann. B 33, 269–284 (2006)CrossRefGoogle Scholar
  33. 33.
    Schweitzer, F., Steinbink, J.: Urban cluster growth: analysis and computer simulations of urban aggregations. In: Schweitzer, F. (ed.) Self-organization of complex structures: from individual to collective dynamics, pp. 501–518. Gordon & Breach, London (1997)Google Scholar
  34. 34.
    Pontius Jr., R.G., Huffaker, D., Denman, K.: Useful techniques of validation for spatially explicit land-change models. Ecol. Model. 179, 445–461 (2004)CrossRefGoogle Scholar
  35. 35.
    Hagen-Zanker, A., Lajoie, G.: Neutral models of landscape change as benchmarks in the assessment of model performance. Landscape Urban Plann (in press, 2008)Google Scholar
  36. 36.
    van Vliet, J.: Validation of land use change models: a case study on the Environment Explorer. Centre for geo-information, Master’s thesis. Universiteit Wageningen, Wageningen, 65 (2006)Google Scholar
  37. 37.
    Jantz, C.A., Goetz, S.J.: Analysis of scale dependencies in an urban land-use-change model. Int. J. Geog. Inf. Sci. 19, 217–241 (2005)CrossRefGoogle Scholar
  38. 38.
    Kocabas, V., Dragicevic, S.: Assessing cellular automata model behaviour using a sensitivity analysis approach. Comput. Environ. Urban Syst. 30, 921–953 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Alex Hagen-Zanker
    • 1
    • 2
  • Pim Martens
    • 3
  1. 1.Research Institute for Knowledge SystemsMaastrichtThe Netherlands
  2. 2.Urban Planning GroupTechnical University EindhovenEindhovenThe Netherlands
  3. 3.International Centre for Integrated assessment and Sustainable developmentMaastricht UniversityMaastrichtThe Netherlands

Personalised recommendations