Computing Cohen’s kappa coefficients using SPSS MATRIX
 Claude A. M. Valiquette,
 Alain D. Lesage,
 Mireille Cyr,
 Jean Toupin
 … show all 4 hide
Abstract
This short paper proposes a general computing strategy to compute Kappa coefficients using the SPSS MATRIX routine. The method is based on the following rationale. If the contingency table is considered as a square matrix, then the observed proportions of agreement lie in the main diagonal’s cells, and their sum equals the trace of the matrix, whereas the proportions of agreement expected by chance are the joint product of marginals. The generalization to weighted kappa, which requires an additional square matrix of disagreement weights, both matrices having the same order, becomes possible by the use of the Hadamard productthat is, the elementwise direct product of two matrices.
This research was supported by a grant from the Quebec Health Research Fund (Canada). We thank Christopher Earls, Stéphane Sabourin, and Marc Valiquette for critical and enlightening comments on earlier drafts.
 Antonak, R. F. (1977) A computer program to compute measures of response agreement for nominal scale data obtained from two judges. Behavior Research Methods & Instrumentation 9: pp. 553553
 Berk, R. A., Campbell, K. L. (1976) A FORTRAN program for Cohen’s kappa coefficient of observer agreement. Behavior Research Methods & Instrumentation 8: pp. 396396
 Bloor, R. N. (1983) A computer program to determine interrater reliability for dichotomousordinal rating scales. Behavior Research Methods & Instrumentation 15: pp. 615615
 Burns, E., Cavallaro, C. (1982) A computer program to determine interobserver reliability statistics. Behavior Research Methods & Instrumentation 14: pp. 4242
 Chan, T. S. C. (1987) A DBase III program that performs significance testing for the kappa coefficient. Behavior Research Methods, Instruments, & Computers 19: pp. 5354
 Cicchetti, D. V., Showalter, D., McCarthy, P. (1990) A computer program for calculating subjectbysubject kappa or weighted kappa coefficients. Educational & Psychological Measurement 50: pp. 153158 CrossRef
 Cohen, J. (1960) A coefficient of agreement for nominal scales. Educational & Psychological Measurement 20: pp. 3746 CrossRef
 Cohen, J. (1968) Weighted kappa: Nomina: scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin 70: pp. 213220 CrossRef
 Collis, G. M. (1985) Kappa, measures of marginal symmetry and intraclass correlations. Educational & Psychological Measurement 45: pp. 5562 CrossRef
 Conger, A. J, Ward, D. G. (1984) Agreement among 2×2 agreement indices. Educational & Psychological Measurement 44: pp. 301313 CrossRef
 Fleiss, J. L. (1971) Measuring nominal scale agreement among many raters. Psychological Bulletin 76: pp. 378382 CrossRef
 Fleiss, J. L., Cohen, J. (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational & Psychological Measurement 33: pp. 613619 CrossRef
 Fleiss, J. L., Nee, J. C, Landis, J. R. (1979) Large sample variance of kappa in the ease of different sets of raters. Psychological Bulletin 86: pp. 974977 CrossRef
 Hubert, L. J. (1987) Assignment methods in combinatorial data analysis. Marcel Dekker, New York
 Landis, J., Koch, G. G. (1977) The measurement of observer agreement for categorical data. Biometrics 33: pp. 159174 CrossRef
 Norusis, M. J. (1990) SPSS base system user’s guide. SPSS Inc, Chicago
 Norusis, M. J. (1990) SPSS advanced statistics user’s guide. SPSS Inc., Chicago
 Rae, G. (1984) On measuring agreement among several judges on the presence or absence of a trait. Educational & Psychological Measurement 44: pp. 247253 CrossRef
 Siegel, S., Castellan, N. J (1988) Nonparametric statistics for the behavioral sciences. McGraw Hill, New York
 Watkins, M. W., Larimer, L. D. (1980) Interrater agreement statistics with the microcomputer. Behavior Research Methods & Instrumentation 12: pp. 466466
 Wixon, D. R. (1979) Cohen’s kappa coefficient of observer agreement: A BASIC program for minicomputers. Behavior Research Methods & Instrumentation 11: pp. 602602
 Title
 Computing Cohen’s kappa coefficients using SPSS MATRIX
 Journal

Behavior Research Methods, Instruments, & Computers
Volume 26, Issue 1 , pp 6061
 Cover Date
 19940301
 DOI
 10.3758/BF03204566
 Print ISSN
 07433808
 Online ISSN
 15325970
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Industry Sectors
 Authors

 Claude A. M. Valiquette ^{(1)}
 Alain D. Lesage ^{(1)}
 Mireille Cyr ^{(1)}
 Jean Toupin ^{(2)}
 Author Affiliations

 1. Department of Psychology, Université de Montréal, CP. 6128, Suce. A, H3C 317, Montréal, Québec, Canada
 2. University of Sherbrooke, Canada, Sherbrooke, Quebec