Quantifying Uncertainty for Estimates Derived from Error Matrices in Land Cover Mapping Applications: The Case for a Bayesian Approach

  • Jordan PhillipsonEmail author
  • Gordon Blair
  • Peter Henrys
Conference paper
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 554)


The use of land cover mappings built using remotely sensed imagery data has become increasingly popular in recent years. However, these mappings are ultimately only models. Consequently, it is vital for one to be able to assess and verify the quality of a mapping and quantify uncertainty for any estimates that are derived from them in a reliable manner.

For this, the use of validation sets and error matrices is a long standard practice in land cover mapping applications. In this paper, we review current state of the art methods for quantifying uncertainty for estimates obtained from error matrices in a land cover mapping context. Specifically, we review methods based on their transparency, generalisability, suitability when stratified sampling and suitability in low count situations. This is done with the use of a third-party case study to act as a motivating and demonstrative example throughout the paper.

The main finding of this paper is there is a major issue of transparency for methods that quantify uncertainty in terms of confidence intervals (frequentist methods). This is primarily because of the difficulty of analysing nominal coverages in common situations. Effectively, this leaves one without the necessary tools to know when a frequentist method is reliable in all but a few niche situations. The paper then discusses how a Bayesian approach may be better suited as a default method for uncertainty quantification when judged by our criteria.


Uncertainty quantification Map assessment Bayesian Land cover maps 


  1. 1.
    Birdsey, R., et al.: Approaches to monitoring changes in carbon stocks for REDD+. Carbon Manag. 4(5), 519–537 (2013)CrossRefGoogle Scholar
  2. 2.
    DeFries, R.S., Houghton, R.A., Hansen, M.C., Field, C.B., Skole, D., Townshend, J.: Carbon emissions from tropical deforestation and regrowth based on satellite observations for the 1980 s and 1990 s. Proc. Natl. Acad. Sci. 99(22), 14256–14261 (2002)CrossRefGoogle Scholar
  3. 3.
    Myneni, R.B., et al.: A large carbon sink in the woody biomass of Northern forests. Proc. Natl. Acad. Sci. 98(26), 14784–14789 (2001)CrossRefGoogle Scholar
  4. 4.
    Schwalm, C.R., et al.: Reduction in carbon uptake during turn of the century drought in western North America. Nat. Geosci. 5(8), 551 (2012)CrossRefGoogle Scholar
  5. 5.
    Asner, G.P., Broadbent, E.N., Oliveira, P.J.C., Keller, M., Knapp, D.E., Silva, J.N.M.: Condition and fate of logged forests in the Brazilian Amazon. Proc. Natl. Acad. Sci. 103(34), 12947–12950 (2006)CrossRefGoogle Scholar
  6. 6.
    Potapov, P.V., et al.: Eastern Europe’s forest cover dynamics from 1985 to 2012 quantified from the full Landsat archive. Remote Sens. Environ. 159, 28–43 (2015)CrossRefGoogle Scholar
  7. 7.
    Shi, W., Liu, J., Du, Z., Stein, A., Yue, T.: Surface modelling of soil properties based on land use information. Geoderma 162(3–4), 347–357 (2011)CrossRefGoogle Scholar
  8. 8.
    Giustarini, L., Hostache, R., Matgen, P., Schumann, G.J.P., Bates, P.D., Mason, D.C.: A change detection approach to flood mapping in Urban areas using TerraSAR-X. IEEE Trans. Geosci. Remote Sens. 51(4), 2417–2430 (2013)CrossRefGoogle Scholar
  9. 9.
    Hussain, M., Chen, D., Cheng, A., Wei, H., Stanley, D.: Change detection from remotely sensed images: from pixel-based to object-based approaches. ISPRS J. Photogramm. Remote Sens. 80, 91–106 (2013)CrossRefGoogle Scholar
  10. 10.
    Rindfuss, R.R., Walsh, S.J., Turner, B.L., Fox, J., Mishra, V.: Developing a science of land change: challenges and methodological issues. Proc. Natl. Acad. Sci. 101(39), 13976–13981 (2004)CrossRefGoogle Scholar
  11. 11.
    Keegan, K.M., Albert, M.R., McConnell, J.R., Baker, I.: Climate change and forest fires synergistically drive widespread melt events of the Greenland Ice Sheet. Proc. Natl. Acad. Sci. 111(22), 7964–7967 (2014)CrossRefGoogle Scholar
  12. 12.
    Knyazikhin, Y., et al.: Hyperspectral remote sensing of foliar nitrogen content. Proc. Natl. Acad. Sci. 110(3), E185–E192 (2013)CrossRefGoogle Scholar
  13. 13.
    McMenamin, S.K., Hadly, E.A., Wright, C.K.: Climatic change and wetland desiccation cause amphibian decline in Yellowstone National Park. Proc. Natl. Acad. Sci. 105(44), 16988–16993 (2008)CrossRefGoogle Scholar
  14. 14.
    Syed, T.H., Famiglietti, J.S., Chambers, D.P., Willis, J.K., Hilburn, K.: Satellite-based global-ocean mass balance estimates of interannual variability and emerging trends in continental freshwater discharge. Proc. Natl. Acad. Sci. 107(42), 17916–17921 (2010)CrossRefGoogle Scholar
  15. 15.
    Fialko, Y., Sandwell, D., Simons, M., Rosen, P.: Three-dimensional deformation caused by the Bam, Iran, earthquake and the origin of shallow slip deficit. Nature 435(7040), 295 (2005)CrossRefGoogle Scholar
  16. 16.
    Khatami, R., Mountrakis, G.: Implications of classification of methodological decisions in flooding analysis from Hurricane Katrina. Remote Sens. 4(12), 3877–3891 (2012)CrossRefGoogle Scholar
  17. 17.
    Alcantara, C., Kuemmerle, T., Prishchepov, A.V., Radeloff, V.C.: Mapping abandoned agriculture with multi-temporal MODIS satellite data. Remote Sens. Environ. 124, 334–347 (2012)CrossRefGoogle Scholar
  18. 18.
    Anderson, M.C., Allen, R.G., Morse, A., Kustas, W.P.: Use of Landsat thermal imagery in monitoring evapotranspiration and managing water resources. Remote Sens. Environ. 122, 50–65 (2012)CrossRefGoogle Scholar
  19. 19.
    Asner, G.P., et al.: Large-scale impacts of herbivores on the structural diversity of African savannas. Proc. Natl. Acad. Sci. 106(12), 4947–4952 (2009)CrossRefGoogle Scholar
  20. 20.
    Mendenhall, C.D., Sekercioglu, C.H., Brenes, F.O., Ehrlich, P.R., Daily, G.C.: Predictive model for sustaining biodiversity in tropical countryside. Proc. Natl. Acad. Sci. 108(39), 16313–16316 (2011)CrossRefGoogle Scholar
  21. 21.
    Khatami, R., Mountrakis, G., Stehman, S.V.: A meta-analysis of remote sensing research on supervised pixel-based land-cover image classification processes: general guidelines for practitioners and future research. Remote Sens. Environ. 177, 89–100 (2016)CrossRefGoogle Scholar
  22. 22.
    Olofsson, P., Foody, G.M., Stehman, S.V., Woodcock, C.E.: Making better use of accuracy data in land change studies: Estimating accuracy and area and quantifying uncertainty using stratified estimation. Remote Sens. Environ. 129, 122–131 (2013)CrossRefGoogle Scholar
  23. 23.
    Olofsson, P., Foody, G.M., Herold, M., Stehman, S.V., Woodcock, C.E., Wulder, M.A.: Good practices for estimating area and assessing accuracy of land change. Remote Sens. Environ. 148, 42–57 (2014)CrossRefGoogle Scholar
  24. 24.
    DasGupta, A., Cai, T.T., Brown, L.D.: Interval estimation for a binomial proportion. Stat. Sci. 16(2), 101–133 (2001)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Wallis, S.: Binomial confidence intervals and contingency tests: mathematical fundamentals and the evaluation of alternative methods. J. Quant. Linguist. 20(3), 178–208 (2013)CrossRefGoogle Scholar
  26. 26.
    Agresti, A., Coull, B.A.: Approximate Is Better than “Exact” for Interval Estimation of Binomial Proportions. Am. Stat., vol. 52, no. 2, pp. 119–126 (1998). Published by : Taylor & Francis, Ltd. on behalf of the American Statistical Association Stable URL : Approximate is Better than “ Ex,”
  27. 27.
    Wagner, J.E., Stehman, S.V.: Optimizing sample size allocation to strata for estimating area and map accuracy. Remote Sens. Environ. 168, 126–133 (2015)CrossRefGoogle Scholar
  28. 28.
    Olofsson, P., et al.: Implications of land use change on the national terrestrial carbon budget of Georgia. Carbon Balance Manag. 5, 4 (2010)CrossRefGoogle Scholar
  29. 29.
    O.T.E.U. Council: Regulation (EU) No 2018/841 of 30 May 2018 on the inclusion of greenhouse gas emissions and removals from land use, land use change and forestry in the 2030 climate and energy framework, and amending Regulation (EU) No 525/2013 and Decision No 529/2013/EU, vol. 2018, no. October 2003, pp. 1–25 (2018)Google Scholar
  30. 30.
    Clopper, C.J., Pearson, E.S.: The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26(4), 404–413 (1934)CrossRefGoogle Scholar
  31. 31.
    Davison, A.C., Hinkley, D.V.: Bootstrap Methods and Their Application. Cambridge University Press, Cambridge (1997)CrossRefGoogle Scholar
  32. 32.
    Denham, R., Mengersen, K., Witte, C.: Bayesian analysis of thematic map accuracy data. Remote Sens. Environ. 113(2), 371–379 (2009)CrossRefGoogle Scholar
  33. 33.
    Finley, A.O., Banerjee, S., McRoberts, R.E.: A Bayesian approach to multi-source forest area estimation. Environ. Ecol. Stat. 15(2), 241–258 (2008)MathSciNetCrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2020

Authors and Affiliations

  1. 1.School of Computing and CommunicationsLancaster UniversityLancasterUK
  2. 2.Centre for Ecology and HydrologyLancasterUK

Personalised recommendations