Cohen’s Linearly Weighted Kappa is a Weighted Average of 2×2 Kappas
- 341 Downloads
An agreement table with n∈ℕ≥3 ordered categories can be collapsed into n−1 distinct 2×2 tables by combining adjacent categories. Vanbelle and Albert (Stat. Methodol. 6:157–163, 2009c) showed that the components of Cohen’s weighted kappa with linear weights can be obtained from these n−1 collapsed 2×2 tables. In this paper we consider several consequences of this result. One is that the weighted kappa with linear weights can be interpreted as a weighted arithmetic mean of the kappas corresponding to the 2×2 tables, where the weights are the denominators of the 2×2 kappas. In addition, it is shown that similar results and interpretations hold for linearly weighted kappas for multiple raters.
KeywordsCohen’s kappa merging categories linear weights quadratic weights Mielke, Berry and Johnston’s weighted kappa Hubert’s weighted kappa
Unable to display preview. Download preview PDF.
- Agresti, A. (1990). Categorical data analysis. New York: Wiley. Google Scholar
- Artstein, R., & Poesio, M. (2005). NLE technical note: Vol. 05-1. Kappa 3 = alpha (or beta). Colchester: University of Essex. Google Scholar
- Cicchetti, D., & Allison, T. (1971). A new procedure for assessing reliability of scoring EEG sleep recordings. The American Journal of EEG Technology, 11, 101–109. Google Scholar
- Fleiss, J.L. (1981). Statistical methods for rates and proportions. New York: Wiley. Google Scholar
- Heuvelmans, A.P.J.M., & Sanders, P.F. (1993). Beoordelaarsovereenstemming. In Eggen, T.J.H.M., & Sanders, P.F. (Eds.) Psychometrie in de Praktijk (pp. 443–470). Arnhem: Cito Instituut voor Toestontwikkeling. Google Scholar
- Krippendorff, K. (2004). Reliability in content analysis: some common misconceptions and recommendations. Human Communication Research, 30, 411–433. Google Scholar
- Popping, R. (1983). Overeenstemmingsmaten voor Nominale Data. Unpublished doctoral dissertation, Rijksuniversiteit Groningen, Groningen. Google Scholar
- Warrens, M.J. (2011). Weighted kappa is higher than Cohen’s kappa for tridiagonal agreement tables. Statistical Methodology, 4, 271–286. Google Scholar