Abstract
Circular classifications are classification scales with categories that exhibit a certain periodicity. Since linear scales have endpoints, the standard weighted kappas used for linear scales are not appropriate for analyzing agreement between two circular classifications. A family of kappa coefficients for circular classifications is defined. The kappas differ only in one parameter. It is studied how the circular kappas are related and if the values of the circular kappas depend on the number of categories. It turns out that the values of the circular kappas can be strictly ordered in precisely two ways. The orderings suggest that the circular kappas are measuring the same thing, but to a different extent. If one accepts the use of magnitude guidelines, it is recommended to use stricter criteria for circular kappas that tend to produce higher values.
Article PDF
Similar content being viewed by others
References
ALTMAN, D.G. (1991), Practical Statistics for Medical Research, London: Chapman & Hall.
BERENS, P. (2009), “Circstat: A MATLAB Toolbox for Circular Statistics”, Journal of Statistical Software, 31, 1–21.
BROWN, M.W. (1992), “Circumplex Models for Correlation Matrices”, Psychometrika, 57, 470–479.
FLEISS, J.L., COHEN, J., and EVERITT, B.S. (1969), “Large Sample Standard Errors of Kappa and Weighted Kappa”, Psychological Bulletin, 72, 323–327.
GWET, K.L. (2012), Handbook of Inter-Rater Reliability (3rd ed.), Gaithersburg MD: Advanced Analytics LLC.
LANDIS, J.R., and KOCH, G.G. (1977), “The Measurement of Observer Agreement for Categorical Data”, Biometrics, 33, 159–174.
POSNER, J., RUSSELL, J.A., and PETERSON, B.S. (2005), “The Circumplex Model of Affect: An Integrative Approach to Affective Neuroscience, Cognitive Development, and Psychopathology”, Developmental Psychopathology, 17, 715–734.
RUEDA, C., FERNÁNDEZ, M.A., and PEDDADA, S.D. (2009), “Estimation of Parameters Subject to Order Restrictions on a Circle With Application to Estimation of Phase Angles of Cell Cycle Genes”, Journal of the American Statistical Association, 104, 338–347.
RUSSELL, J.A. (1980), “A Circumplex Model of Affect”, Journal of Personality and Social Psychology, 39, 1161–1178.
VANBELLE, S. (2015), “A New Interpretation of the Weighted Kappa Coefficients”, Psychometrika, in press.
VANBELLE, S., MUTSVARI, T., DECLERCK, D., and LESAFFRE, E. (2012), “Hierarchical Modeling of Agreement”, Statistics in Medicine, 31, 3667–3680.
WARRENS, M.J. (2008), “On the Equivalence of Cohen’s Kappa and the Hubert-Arabie Adjusted Rand Index”, Journal of Classification, 25, 177–183.
WARRENS, M.J. (2010a), “A Kraemer-Type Rescaling that Transforms the Odds Ratio Into the Weighted Kappa Coefficient”, Psychometrika, 75, 328–330.
WARRENS, M.J. (2010b), “Inequalities Between Multi-rater Kappas”, Advances in Data Analysis and Classification, 4, 271–286.
WARRENS, M.J. (2011a), “Cohen’s Kappa is a Weighted Average”, Statistical Methodology, 8, 473–484.
WARRENS, M.J. (2011b), “Weighted Kappa is Higher Than Cohen’s Kappa for Tridiagonal Agreement Tables”, Statistical Methodology, 8, 268–272.
WARRENS, M.J. (2012), “Some Paradoxical Results for the Quadratically Weighted Kappa”, Psychometrika, 77, 315–323.
WARRENS, M.J. (2013), “Cohen’s Weighted Kappa With Additive Weights”, Advances in Data Analysis and Classification, 7, 41–55.
WARRENS, M.J. (2014), “Corrected Zegers-ten Berge Coefficients Are Special Cases of Cohen’s Weighted Kappa”, Journal of Classification, 31, 179–193.
WARRENS, M.J. (2015), “Additive Kappa Can be Increased by Combining Adjacent Categories”, International Mathematical Forum, 10, 323–328.
WATSON, D., and TELLEGEN, A. (1985), “Toward a Consensual Structure of Mood”, Psychological Bulletin, 98, 219–235.
WATSON, D., WIESE, D., VAIDYA, J., and TELLEGEN, A. (1999), “The Two General Activation Systems of Affect: Structural Findings, Evolutionary Considerations, and Psychobiological Evidence”, Journal of Personality and Social Psychology, 76, 820–838.
YANG, Z., and ZHOU, M. (2014), “Kappa Statistic for Clustered Matched-pair Data”, Statistics in Medicine, 33, 2612–2633.
YANG, Z., and ZHOU, M. (2015), “Weighted Kappa Statistic for Clustered Matched-Pair Ordinal Data”, Computational Statistics and Data Analysis, 82, 1–18.
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors thank two anonymous referees for their helpful comments and valuable suggestions on an earlier version of this article.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Warrens, M.J., Pratiwi, B.C. Kappa Coefficients for Circular Classifications. J Classif 33, 507–522 (2016). https://doi.org/10.1007/s00357-016-9217-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00357-016-9217-3