Advertisement

Journal of Classification

, Volume 33, Issue 3, pp 507–522 | Cite as

Kappa Coefficients for Circular Classifications

  • Matthijs J. Warrens
  • Bunga C. Pratiwi
Open Access
Article

Abstract

Circular classifications are classification scales with categories that exhibit a certain periodicity. Since linear scales have endpoints, the standard weighted kappas used for linear scales are not appropriate for analyzing agreement between two circular classifications. A family of kappa coefficients for circular classifications is defined. The kappas differ only in one parameter. It is studied how the circular kappas are related and if the values of the circular kappas depend on the number of categories. It turns out that the values of the circular kappas can be strictly ordered in precisely two ways. The orderings suggest that the circular kappas are measuring the same thing, but to a different extent. If one accepts the use of magnitude guidelines, it is recommended to use stricter criteria for circular kappas that tend to produce higher values.

Keywords

Cohen’s kappa Weighted kappa Inter-rater agreement Linear scale Circular scale. 

References

  1. ALTMAN, D.G. (1991), Practical Statistics for Medical Research, London: Chapman & Hall.Google Scholar
  2. BERENS, P. (2009), “Circstat: A MATLAB Toolbox for Circular Statistics”, Journal of Statistical Software, 31, 1–21.Google Scholar
  3. BROWN, M.W. (1992), “Circumplex Models for Correlation Matrices”, Psychometrika, 57, 470–479.Google Scholar
  4. FLEISS, J.L., COHEN, J., and EVERITT, B.S. (1969), “Large Sample Standard Errors of Kappa and Weighted Kappa”, Psychological Bulletin, 72, 323–327.Google Scholar
  5. GWET, K.L. (2012), Handbook of Inter-Rater Reliability (3rd ed.), Gaithersburg MD: Advanced Analytics LLC.Google Scholar
  6. LANDIS, J.R., and KOCH, G.G. (1977), “The Measurement of Observer Agreement for Categorical Data”, Biometrics, 33, 159–174.Google Scholar
  7. POSNER, J., RUSSELL, J.A., and PETERSON, B.S. (2005), “The Circumplex Model of Affect: An Integrative Approach to Affective Neuroscience, Cognitive Development, and Psychopathology”, Developmental Psychopathology, 17, 715–734.Google Scholar
  8. RUEDA, C., FERNÁNDEZ, M.A., and PEDDADA, S.D. (2009), “Estimation of Parameters Subject to Order Restrictions on a Circle With Application to Estimation of Phase Angles of Cell Cycle Genes”, Journal of the American Statistical Association, 104, 338–347.Google Scholar
  9. RUSSELL, J.A. (1980), “A Circumplex Model of Affect”, Journal of Personality and Social Psychology, 39, 1161–1178.Google Scholar
  10. VANBELLE, S. (2015), “A New Interpretation of the Weighted Kappa Coefficients”, Psychometrika, in press.Google Scholar
  11. VANBELLE, S., MUTSVARI, T., DECLERCK, D., and LESAFFRE, E. (2012), “Hierarchical Modeling of Agreement”, Statistics in Medicine, 31, 3667–3680.Google Scholar
  12. WARRENS, M.J. (2008), “On the Equivalence of Cohen’s Kappa and the Hubert-Arabie Adjusted Rand Index”, Journal of Classification, 25, 177–183.Google Scholar
  13. WARRENS, M.J. (2010a), “A Kraemer-Type Rescaling that Transforms the Odds Ratio Into the Weighted Kappa Coefficient”, Psychometrika, 75, 328–330.Google Scholar
  14. WARRENS, M.J. (2010b), “Inequalities Between Multi-rater Kappas”, Advances in Data Analysis and Classification, 4, 271–286.Google Scholar
  15. WARRENS, M.J. (2011a), “Cohen’s Kappa is a Weighted Average”, Statistical Methodology, 8, 473–484.Google Scholar
  16. WARRENS, M.J. (2011b), “Weighted Kappa is Higher Than Cohen’s Kappa for Tridiagonal Agreement Tables”, Statistical Methodology, 8, 268–272.Google Scholar
  17. WARRENS, M.J. (2012), “Some Paradoxical Results for the Quadratically Weighted Kappa”, Psychometrika, 77, 315–323.Google Scholar
  18. WARRENS, M.J. (2013), “Cohen’s Weighted Kappa With Additive Weights”, Advances in Data Analysis and Classification, 7, 41–55.Google Scholar
  19. WARRENS, M.J. (2014), “Corrected Zegers-ten Berge Coefficients Are Special Cases of Cohen’s Weighted Kappa”, Journal of Classification, 31, 179–193.Google Scholar
  20. WARRENS, M.J. (2015), “Additive Kappa Can be Increased by Combining Adjacent Categories”, International Mathematical Forum, 10, 323–328.Google Scholar
  21. WATSON, D., and TELLEGEN, A. (1985), “Toward a Consensual Structure of Mood”, Psychological Bulletin, 98, 219–235.Google Scholar
  22. WATSON, D., WIESE, D., VAIDYA, J., and TELLEGEN, A. (1999), “The Two General Activation Systems of Affect: Structural Findings, Evolutionary Considerations, and Psychobiological Evidence”, Journal of Personality and Social Psychology, 76, 820–838.Google Scholar
  23. YANG, Z., and ZHOU, M. (2014), “Kappa Statistic for Clustered Matched-pair Data”, Statistics in Medicine, 33, 2612–2633.Google Scholar
  24. YANG, Z., and ZHOU, M. (2015), “Weighted Kappa Statistic for Clustered Matched-Pair Ordinal Data”, Computational Statistics and Data Analysis, 82, 1–18.Google Scholar

Copyright information

© The Author(s) 2016

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.GIONUniversity of GroningenGroningenThe Netherlands
  2. 2.Leiden UniversityLeidenThe Netherlands

Personalised recommendations