Abstract
The linearly and quadratically weighted kappa coefficients are popular statistics in measuring inter-rater agreement on an ordinal scale. It has been recently demonstrated that the linearly weighted kappa is a weighted average of the kappa coefficients of the embedded 2 by 2 agreement matrices, while the quadratically weighted kappa is insensitive to the agreement matrices that are row or column reflection symmetric. A rank-one matrix decomposition approach to the weighting schemes is presented in this note such that these phenomena can be demonstrated in a concise manner.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berman, A., & Shaked-Monderer, N. (2003). Completely positive matrices. Singapore: World Scientific.
Brenner, H., & Kliebsch, U. (1996). Dependence of weighed kappa coefficients on the number of categories. Epidemiology, 7, 199–202.
Cantoni, A., & Butler, P. (1976). Eigenvalues and eigenvectors of symmetric centrosymmetric matrices. Linear Algebra and its Applications, 13, 275–288.
Cicchetti, D., & Allison, T. (1971). A new procedure for assessing reliability of scoring EEG sleep recordings. American Journal EEG Technology, 11, 101–109.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 213–220.
Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220.
Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measure of reliability. Educational and Psychological Measurement, 33, 613–619.
Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review, 51, 455–500.
Schuster, C. (2004). A note on the interpretation of weighted kappa and its relation to other rater agreement statistics for metric scales. Educational and Psychological Measurement, 64, 243–253.
Vanbelle, S. (2014). A new interpretation of the weighted kappa coefficients. Psychometrika. doi:10.1007/s11336-014-9439.
Vanbelle, S., & Albert, A. (2009). A note on the linearly weighted kappa coefficient for ordinal scales. Statistical Methodology, 6, 157–163.
Warrens, M. J. (2011). Cohen’s linearly weighted kappa is a weighted average of \(2\times 2\) kappas. Psychometrika, 76, 471–486.
Warrens, M. J. (2012a). Cohen’s linearly weighted kappa is a weighted average. Advances in Data Analysis and Classification, 6, 67–79.
Warrens, M. J. (2012b). Some paradoxical results for the quadratically weighted kappa. Psychometrika, 77, 315–323.
Warrens, M. J. (2013a). The Cicchetti–Allison weighting matrix is positive definite. Computational Statistics and Data Analysis, 59, 180–182.
Warrens, M. J. (2013b). Cohen’s weighted kappa with additive weights. Advances in Data Analysis and Classification, 7, 41–55.
Warrens, M. J. (2015). Additive kappa can be increased by combining adjacent categories. International Mathematical Forum, 10, 323–328.
Yang, J., & Chinchilli, V. M. (2011). Fixed-effects modeling of Cohen’s weighted kappa for bivariate multinomial data. Computational Statistics and Data Analysis, 55, 1061–1070.
Acknowledgments
This work was partially supported by the National Natural Science Foundation of China under Grants 61203131, 11171177, and Tsinghua University Initiative Scientific Research Program 2014z21017.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, P. A Note on the Linearly and Quadratically Weighted Kappa Coefficients. Psychometrika 81, 795–801 (2016). https://doi.org/10.1007/s11336-016-9501-5
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-016-9501-5