, Volume 75, Issue 1, pp 99–119

Modeling Concordance Correlation Coefficient for Longitudinal Study Data

Theory and Methods


Measures of agreement are used in a wide range of behavioral, biomedical, psychosocial, and health-care related research to assess reliability of diagnostic test, psychometric properties of instrument, fidelity of psychosocial intervention, and accuracy of proxy outcome. The concordance correlation coefficient (CCC) is a popular measure of agreement for continuous outcomes. In modern-day applications, data are often clustered, making inference difficult to perform using existing methods. In addition, as longitudinal study designs become increasingly popular, missing data have become a serious issue, and the lack of methods to systematically address this problem has hampered the progress of research in the aforementioned fields. In this paper, we develop a novel approach to tackle the complexities involved in addressing missing data and other related issues for performing CCC analysis within a longitudinal data setting. The approach is illustrated with both real and simulated data.

diagnostic test inverse probability weighted estimates missing data monotone missing data pattern U-statistics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Barnhart, H.X., & Williamson, J.M. (2001). Modelling concordance correlation via GEE to evaluate reproducibility. Biometrics, 57, 931–940. CrossRefPubMedGoogle Scholar
  2. Barnhart, H.X., Haber, M., & Song, J. (2002). Overall concordance correlation coefficient for evaluating agreement among multiple observers. Biometrics, 58, 1020–1027. CrossRefPubMedGoogle Scholar
  3. Bauer, S., & Kennedy, J.W. (1981). Applied statistics for the clinical laboratory: II. Within-run imprecision. The Journal of Clinical Laboratory Automation, 1, 197–201. Google Scholar
  4. Chandler, J.M., Martin, A.R., Girman, C., Ross, P.D., Love-McClung, B., Lydick, E., & Yawn, B.P. (1998). Reliability of an osteoporosis-targeted quality of life survey instrument for use in the community: OPTQoL. Osteoporosis International, 8, 127–135. CrossRefPubMedGoogle Scholar
  5. Chinchilli, V.M., Martel, J.K., Kumanyika, S., & Lloyd, T. (1996). A weighted concordance correlation coefficient for repeated measures designs. Biometrics, 52, 341–353. CrossRefPubMedGoogle Scholar
  6. Costa, P., Arnould, B., Cour, F., Boyer, P., Marrel, A., Jaudinot, E.O., & Solesse de Gendre, A. (2003). Quality of Sexual Life Questionnaire (QVS): a reliable, sensitive and reproducible instrument to assess quality of life in subjects with erectile dysfunction. International Journal of Impotence Research, 15, 173–184. CrossRefPubMedGoogle Scholar
  7. Fieller, E.C. (1954). Some problems in interval estimation. Journal of the Royal Statistical Society B, 16, 175–185. Google Scholar
  8. Guo, X., Pan, W., Connett, J.E., Hannan, P.J., & French, S.A. (2005). Small-sample performance of the robust score test and its modifications in generalized estimating equations. Statistics in Medicine, 24, 3479–3495. CrossRefPubMedGoogle Scholar
  9. King, T.S., & Chinchilli, V.M. (2001). A generalized concordance correlation coefficient for continuous and categorical data. Statistics in Medicine, 20, 2131–2147. CrossRefPubMedGoogle Scholar
  10. King, T.S., Chinchilli, V.M., & Carrasco, J.L. (2007). A repeated measures concordance correlation coefficient. Statistics in Medicine, 26, 3095–3113. CrossRefPubMedGoogle Scholar
  11. Kowalski, J., & Tu, X.M. (2007). Modern applied U-statistics. New York: Wiley. CrossRefGoogle Scholar
  12. Lin, L. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45, 255–268. CrossRefPubMedGoogle Scholar
  13. Little, R.J.A., & Rubin, D.B. (1987). Statistical analysis with missing data. New York: Wiley. Google Scholar
  14. Lloyd, T., Chinchilli, V.M., Eggli, D.F., Rollings, N., & Kulin, H.E. (1998). Body composition development of adolescent white females. Archives of Pediatric Adolescence Medicine, 152, 998–1002. Google Scholar
  15. McCullagh, P., & Nelder, J.A. (1989). Generalized linear models (2nd ed.). London: Chapman and Hall. Google Scholar
  16. McGraw, K.O., & Wong, S.P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. CrossRefGoogle Scholar
  17. Morrison-Beedy, D., Carey, M.P., & Tu, X.M. (2006). Accuracy of audio computer-assisted self-interviewing (ACASI) and self-administered questionnaires for the assessment of sexual behavior. AIDS and Behavior, 10, 541–552. CrossRefPubMedGoogle Scholar
  18. Paul, I.M., Wai, K., Jewell, S.J., Shaffer, M.L., & Varadan, V.V. (2006). Evaluation of a new self-contained, ambulatory, objective cough monitor. Cough, 2, 1–7. CrossRefGoogle Scholar
  19. Prentice, R.L. (1988). Correlated binary regression with covariates specific to each binary observation. Biometrics, 44, 321–327. CrossRefGoogle Scholar
  20. Reboussin, B.A., & Liang, K.Y. (1998). An estimating equations approach for the LISCOMP model. Psychometrika, 63, 165–182. CrossRefGoogle Scholar
  21. Robins, J.M., Rotnitzky, A., & Zhao, L.P. (1995). Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. Journal of the American Statistical Association, 90, 106–121. CrossRefGoogle Scholar
  22. Schroder, K.E.E., Carey, M.P., & Vanable, P.A. (2003). Methodological challenges in research on sexual risk behavior: II. Accuracy of self-reports. Annals of Behavioral Medicine, 26, 104–123. CrossRefPubMedGoogle Scholar
  23. Seber, G.A.F. (1984). Multivariate observations. New York: Wiley. CrossRefGoogle Scholar
  24. Serfling, R.J. (1980). Approximation theorems of mathematical statistics. New York: Wiley. CrossRefGoogle Scholar
  25. Shrier, L.A., Shih, M., & Beardslee, W.R. (2005). Comparison of momentary sampling with diary and retrospective self-report methods of measurement. Pediatrics, 115, 573–581. CrossRefGoogle Scholar
  26. Shrout, P.E., & Fleiss, J.L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. CrossRefPubMedGoogle Scholar
  27. Tsiatis, A.A. (2006). Semiparametric Theory and Missing Data. New York: Springer. Google Scholar
  28. Tu, X.M., Feng, C., Kowalski, J., Tang, W., Wang, H., Wan, C., & Ma, Y. (2007). Correlation analysis for longitudinal data: applications to HIV and psychosocial research. Statistics in Medicine, 26, 4116–4138. CrossRefPubMedGoogle Scholar
  29. Westgard, J.O., & Hunt, M.R. (1973). Use and interpretation of common statistical tests on method-comparison studies. Clinical Chemistry, 19, 49–57. PubMedGoogle Scholar

Copyright information

© The Psychometric Society 2009

Authors and Affiliations

  1. 1.Department of Public HealthHospital for Special Surgery–Weill Medical College of Cornell UniversityNew YorkUSA
  2. 2.Department of Biostatistics and Computational Biology, Department of PsychiatryUniversity of RochesterRochesterUSA
  3. 3.Department of Biostatistics and Computational BiologyUniversity of RochesterRochesterUSA
  4. 4.Department of Biostatistics and Computational Biology, Department of PsychiatryUniversity of RochesterRochesterUSA

Personalised recommendations