Abstract
Assessing thematic map accuracy is a special type of map comparison that is frequently applied to remote sensing classification problems. For map comparisons in the accuracy assessment setting, one map represents the classified output and the other map represents the true or “reference” condition. Several articles in this special issue describe state-of-the-art map comparison analysis tools that could serve to quantify accuracy of a single map. However, accuracy assessment objectives generally extend beyond describing accuracy of a single map to comparing accuracy of several maps. Consequently, interest focuses on comparing map comparison measures when these measures are used to represent accuracy. The virtual workshop emphasizes the analysis component of map comparisons, but it is also important to examine the underlying study designs generating the data input into these analyses. The study designs for accuracy comparisons implemented in remote sensing practice often investigate only a single test site, thus limiting our ability to generalize the results of these accuracy comparisons. Map accuracy comparison studies can be designed to provide stronger generalizations by incorporating experimental design principles such as replication and blocking, and identifying an experimental unit appropriate for the application. It is also important to recognize the role of statistical hypothesis testing and inference for different objectives that motivate map accuracy comparisons. Deciding which of two maps to use for a particular site can be addressed by enumerative inference and does not require hypothesis testing. For the objective of a more general comparison of classification procedures, analytic inference is appropriate and hypothesis testing plays a more prominent role.
Similar content being viewed by others
References
Bradley BA, Mustard JF (2005) Identifying land cover variability distinct from land cover change: Cheatgrass in the Great Basin. Remote Sens Environ 94:204–213
Congalton RG, Green K (1999) Assessing the accuracy of remotely sensed data: principles and practices. CRC, Boca Raton
Congalton RG, Oderwald RG, Mead RA (1983) Assessing Landsat classification accuracy using discrete multivariate analysis statistical techniques. Photogram Eng Remote Sens 49:1671–1678
Deming WE (1950) Some theory of sampling. Wiley, New York
Erol H, Akdeniz F (2005) A per-field classification method based on mixture distribution models and an application to Landsat Thematic Mapper data. Int J Remote Sens 26:1229–1244
Foody GM (2002) Status of land cover classification accuracy assessment. Remote Sens Environ 80:185–201
Foody GM (2004) Thematic map comparison: evaluating the statistical significance of differences in classification accuracy. Photogram Eng Remote Sens 70:627–633
Fotheringham AS, Brunsdon C (2004) Some thoughts on inference in the analysis of spatial data. Int J Geogr Inf Sci 18:447–457
Hurlbert SH (1984) Pseudoreplication and the design of ecological field experiments. Ecol Monogr 54:187–211
Islam Z, Metternicht G (2005) The performance of fuzzy operators on fuzzy classification of urban land covers. Photogram Eng Remote Sens 71:59–68
Kelly M, Shaari D, Guo Q, Liu D (2004) A comparison of standard and hybrid classifier methods for mapping hardwood mortality in areas affected by “sudden oak death”. Photogr Eng Remote Sens 70:1229–1239
Kuehl RO (2000) Design of experiments: statistical principles of research design and analysis, 2nd edn. Brooks/Cole, Pacific Grove
Lu D, Weng Q (2004) Spectral mixture analysis of the urban landscape in Indianapolis with Landsat ETM+ imagery. Photogr Eng Remote Sens 70:1053–1062
McCullagh P, Nelder JA (1989) Generalized linear models, 2nd edn. Chapman and Hall, London
Mead R (1988) The design of experiments: statistical principles for practical application. Cambridge University Press, Cambridge
Nackaerts K, Vaesen K, Muys B, Coppin P (2005) Comparative performance of a modified change vector analysis in forest change detection. Int J Remote Sens 26:839–852
Olthof I, Buston C, Fraser R (2005) Signature extension through space for northern landcover classification: a comparison of radiometric correction methods. Remote Sens Environ 95:290–302
Overton WS (1993) Probability sampling and population inference in monitoring programs. In: Goodchild MF, Parks BO, Steyaert LT (eds) Environmental modeling with GIS. Oxford University Press, New York, pp 470–480
Özkan C, Erbek FS (2005) Comparing feature extraction techniques for urban land-use classification. Int J Remote Sens 26:747–757
Pal M, Mather PM (2005) Support vector machines for classification in remote sensing. Int J Remote Sens 26:1007–1011
Pasqualini V, Pergent-Martini C, Pergent G, Agreil M, Skoufas G, Sourbes L, Tsirika A (2005) Use of SPOT 5 for mapping seagrasses: an application to Posidonia oceanica. Remote Sens Environ 94:39–45
Petersen RG (1985) Design and analysis of experiments. Marcel-Dekker, New York
Platt RV, Goetz AFH (2004) A comparison of AVIRIS and Landsat for land use classification at the urban fringe. Photogr Eng Remote Sens 70:813–819
Prenzel B, Treitz P (2005) Comparison of function- and structure-based schemes for classification of remotely sensed data. Int J Remote Sens 26:543–561
Puissant A, Hirsch J, Weber C (2005) The utility of texture analysis to improve per-pixel classification for high to very high spatial resolution imagery. Int J Remote Sens 26:733–745
Särndal CE, Swensson B, Wretman J (1992) Model-assisted survey sampling. Springer, Berlin Heidelberg New York
Stehman SV (1997) Selecting and interpreting measures of thematic classification accuracy. Remote Sens Environ 62:77–89
Stehman SV (1999) Basic probability sampling designs for thematic map accuracy assessment. Int J Remote Sens 20:2423–2441
Stehman SV (2001) Statistical rigor and practical utility in thematic map accuracy assessment. Photogr Eng Remote Sens 67:727–734
Story M, Congalton RG (1986) Accuracy assessment: a user’s perspective. Photogr Eng Remote Sens 52:397–399
Théau J, Peddle DR, Duguay CR (2005) Mapping lichen in a caribou habitat of Northern Quebec, Canada, using an enhancement-classification method and spectral mixture analysis. Remote Sens Environ 94:232–243
Warner TA, Steinmaus K (2005) Spatial classification of orchards and vineyards with high spatial resolution panchromatic imagery. Photogr Eng Remote Sens 71:179–187
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Stehman, S.V. Design, analysis, and inference for studies comparing thematic accuracy of classified remotely sensed data: a special case of map comparison. J Geograph Syst 8, 209–226 (2006). https://doi.org/10.1007/s10109-006-0022-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10109-006-0022-0