Abstract
A family of coefficients of relational agreement for numerical scales is proposed. The theory is a generalization to multiple judges of the Zegers and ten Berge theory of association coefficients for two variables and is based on the premise that the choice of a coefficient depends on the scale type of the variables, defined by the class of admissible transformations. Coefficients of relational agreement that denote agreement with respect to empirically meaningful relationships are derived for absolute, ratio, interval, and additive scales. The proposed theory is compared to intraclass correlation, and it is shown that the coefficient of additivity is identical to one measure of intraclass correlation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adams, E. W., Fagot, R. F., & Robinson, R. E. (1965). A theory of appropriate statistics.Psychometrika, 30, 99–127.
Alper, T. M. (1985). A note on real measurement structures of scale type (m, m + 1).Journal of Mathematical Psychology, 29, 73–81.
Birnbaum, M. H. (1982). Controversies in psychological measurement. In B. Wegener (Ed.),Social attitudes and psychophysical measurement (pp. 401–485). Hillsdale, NJ: Erlbaum.
Cohen, J. (1960). A coefficient of agreement for nominal scales.Educational and Psychological Measurement, 20, 37–46.
Fagot, R. F. (1991). Reliability of ratings for multiple judges: Intraclass correlation and metric scales.Applied Psychological Measurement, 15, 1–11.
Fagot, R. F., & Mazo, R. M. (1989). Association coefficients of identity and proportionality for metric scales.Psychometrika, 54, 93–104.
Fisher, R. A. (1924). The influence of rainfall on the yield of wheat at Rothamsted.Philosophical Transactions of the Royal Society of London, Series B,213, 89–124.
Green, B. F. (1956). A method of scalogram analysis using summary statistics.Psychometrika, 21, 79–88.
Hays, W. L. (1988).Statistics (4th ed.). New York: Holt, Rinehart & Winston.
Kendall, M. G., & Stuart, A. (1961).The advanced theory of statistics (Vol. 2). London: Griffin.
Lahey, M. A., Downey, R. G., & Saal, F. E. (1983). Intraclass correlation: There's more there than meets the eye.Psychological Bulletin, 93, 586–595.
Luce, R. D., Krantz, D. H., Suppes P., & Tversky, A. (1990).Foundations of Measurement, Vol. 3: Representation, axiomatization, and invariance. New York: Academic Press.
Narens, L. (1981a). A general theory of ratio scalability with remarks about the measurement-theoretic concept of meaningfulness.Theory and Decision, 13, 1–70.
Narens, L. (1981b). On the scales of measurement.Journal of Mathematical Psychology, 24, 249–275.
Narens, L. (1985).Abstract measurement theory. Cambridge, MA: MIT Press.
Roberts, F. S. (1979).Measurement theory. Reading, MA: MIT Press.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability.Psychological Bulletin, 86, 420–428.
Stevens, S. S. (1946). On the theory of scales of measurement.Science, 103, 677–680.
Stevens, S. S. (1951). Mathematics, measurement, and psychophysics. In S. S. Stevens (Ed.),Handbook of experimental psychology (p. 1–49). New York: Wiley.
Stevens, S. S. (1968). Measurement, statistics, and the schemapiric view.Science, 161, 849–856.
Stine, W. W. (1989a). Interobserver relational agreement.Psychological Bulletin, 106, 341–347.
Stine, W. W. (1989b). Meaningful inference: The role of measurement in statistics.Psychological Bulletin, 105, 147–155.
Suppes, P., & Zinnes, J. L. (1963). Basic measurement theory. In R. D. Luce, R. R. Bush, & E. Galanter (Eds.),Handbook of mathematical psychology, Vol. I (pp. 1–76). New York: Wiley.
Winer, B. J. (1971).Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill.
Zegers, F. E. (1986). A family of chance-corrected association coefficients for metric scales.Psychometrika, 51, 559–562.
Zegers, F. E., & ten Berge, J. M. F. (1985). A family of association coefficients for metric scales.Psychometrika, 50, 17–24.
Author information
Authors and Affiliations
Additional information
The author thanks the Editor and anonymous reviewers for helpful suggestions.
Rights and permissions
About this article
Cite this article
Fagot, R.F. A generalized family of coefficients of relational agreement for numerical scales. Psychometrika 58, 357–370 (1993). https://doi.org/10.1007/BF02294581
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294581