Abstract
A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the covariance parameters obtained from a linear mixed model. The method provides a single number to express the reliability of the scale, but allows for the study of the reliability’s time evolution. The method is illustrated using a case study in schizophrenia.
Similar content being viewed by others
References
Alonso, A., Geys, H., Molenberghs, G., & Vangeneugden, T. (2002). Investigating the criterion validity of psychiatric symptom scales using surrogate marker validation methodology. Journal of Biopharmaceutical Statistics, 12, 161–179.
Bock, R.D. (1966). Contributions of multivariate experimental designs to educational research. In R.B. Cattell (Ed.), Handbook of multivariate experimental psychology. Chicago: Rand-McNally.
Bock, R.D. (1983). The discrete Bayesian. In H. Wainer & S. Messick (Eds.), Principals of modern psychological measurement. Hillsdale, NJ: Erlbaum.
Diggle, P.J., Liang, K.-Y., & Zeger, S.L. (1994). Analysis of longitudinal data. Oxford Science Publications. Oxford, UK: Clarendon Press.
Laird, N.M., & Ware, J.H. (1982). Random effects models for longitudinal data. Biometrics, 38, 963–974.
Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Vangeneugden, T., Laenen, A., Geys, H., Renard, D., & Molenberghs, G. (2004). Applying linear mixed models to estimate reliability in clinical trial data with repeated measurements. Controlled Clinical Trials, 25(1), 13–20.
Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. New York: Springer.
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors are grateful to J&J PRD for kind permission to use their data. We gratefully acknowledge support from the Belgian IUAP/PAI network “Statistical Techniques and Modeling for Complex Substantive Questions with Complex Data.”
Rights and permissions
About this article
Cite this article
Laenen, A., Alonso, A. & Molenberghs, G. A Measure for the Reliability of a Rating Scale Based on Longitudinal Clinical Trial Data. Psychometrika 72, 443–448 (2007). https://doi.org/10.1007/s11336-007-9002-7
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-007-9002-7