Abstract
In a recent article, Fagot proposed a generalized family of coefficients of relational agreement for multiple judges, focusing on the concept of empirically meaningful relationships. In this paper an ordinal coefficient of relational agreement, based on ranking data, is presented as a special case of the generalized family. It is shown that the proposed ordinal coefficient encompasses other ordinal coefficients, such as the Kendall coefficient of concordance, the average Spearman rank-order coefficient, and intraclass correlation based on ranks. It is also shown that the Kendall coefficient of concordance, corrected for chance agreement, is equivalent to the ordinal coefficient proposed in this paper.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cohen, J. (1960). A coefficient of agreement for nominal scales.Educational and Psychological Measurement, 20, 37–46.
Ehrenberg, A. S. C. (1952). On sampling from a population of rankers.Biometrika, 39, 82–87.
Fagot, R. F. (1993). A generalized family of coefficients of relational agreement for numerical scales.Psychometrika, 58, 357–370.
Fisher, R. A. (1924). The influence of rainfall on the yield of wheat at Rothamsted.Philosophical Transactions of the Royal Society of London, Series B, 213, 89–124.
Green, B. F. (1956). A method of scalogram analysis using summary statistics.Psychometrika, 21, 79–88.
Haggard, E. A. (1958).Intraclass correlation and the analysis of variance. New York: Dryden Press.
Kendall, M., & Gibbons, J. D. (1990).Rank correlation methods. New York: Oxford University Press.
Shrout, P. E., & Fleiss, J. E. (1979). Intraclass correlations: Uses in assessing rater reliability.Psychological Bulletin, 86, 420–428.
Siegel, S., & Castellan, N., Jr. (1988).Nonparametric statistics for the behavioral sciences (2nd ed.). New York: McGraw-Hill.
Stine, W. W. (1989). Interobserver relational agreement.Psychological Bulletin, 106, 341–347.
Tanner, M. A., & Young, M. A. (1985). Modeling ordinal scale disagreement.Psychological Bulletin, 98, 408–415.
Zegers, F. E. (1986). A family of chance-corrected association coefficients for metric scales.Psychometrika, 51, 559–562.
Zegers, F. E., & ten Berge, J. M. F. (1985). A family of association coefficients for metric scales.Psychometrika, 50, 17–24.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fagot, R.F. An ordinal coefficient of relational agreement for multiple judges. Psychometrika 59, 241–251 (1994). https://doi.org/10.1007/BF02295186
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02295186