Abstract
In measuring outcomes of health care, information is obtained from subjects employing instruments that often use Likert scales. These instruments are typically designed using classical testing theory which assumes the errors around the true scores are normally distributed and constant. Advances in psychometric practices through the use of item response theory (IRT) models have led to more flexibility in scale development and in data analyses. In this paper, we introduce statisticians and health services researchers to IRT models through a case-study of data collected to measure subjective distress. The data consist of self-reports of symptom and problem difficulty obtained from a sample of 2,656 patients discharged with a psychiatric disorder from 13 hospitals in the United States between May 2001 and April 2002. Dimensionality of the trait is assessed using principal factor analysis. Model assessment is made using χ2 statistics and residual analyses. We select items for the scale using the Fisher Information available at selected levels of the underlying trait.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions. Dover, New York, NY (1987)
Agresti, A.: Categorical Data Analysis. John Wiley & Sons, New York, NY (1990)
Andrich, D.: A rating formulation for ordered response categories. Psychometrika 43, 561–573 (1978)
Baker, F.B.: Item Response Theory: Parameter Estimation Techniques. Marcel Dekker, New York, NY (1992)
Bock, R.D., Gibbons, R.D.: High-dimensional multivariate probit analysis. Biometrics 52, 1183–1194 (1996)
Bollen, R.A.: Structural Equations with Latent Variables. John Wiley & Sons, New York, NY (1989)
Brooks, S.P., Morgan, B.J.T.: Optimization using simulated annealing. Statistician 44, 241–257 (1995)
Carmines, E.G., Zeller, R.A.: Reliability and Validity Assessment. Sage, Beverly Hills, CA (1979)
Cattell, R.B.: The Scree test for the number of factors. Multivariate Behav. Res. 1, 245–276 (1966)
Cliff, N.: The eigenvalue-greater-than-one rule and the reliability of components. Psychol. Bull. 103(2), 276–279 (1988)
Cortina, J.M.: What is coefficient alpha? An examination of theory and applications. J. Appl. Psychol. 78, 98–104 (1993)
D’Agostino, R.B., Belanger, A.J., Markson, E.W., Kelly-Hayes, M., Wolf, P.A.: Development of health risk appraisal functions in the presence of multiple indicators: The Framingham study nursing home institutionalization model. Stat. Med. 14, 1757–1770 (1995)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Stat. Soc. B. 39, 1–38 (1977)
Doerfler, L.A., Addis, M.E., Moran, P.W.: Evaluating mental health outcomes in an inpatients setting: convergent and divergent validity of the OQ-45 and BASIS-32. J. Behav. Health Serv. Res. 29, 393–403 (2002)
Douglas, J.A.: Item response models for longitudinal quality of life data in clinical trials. Stat. Med. 18, 2917–2913 (1999)
Drasgow, F.: Polychoric and polyserial correlations. In: Kotz, L., Johnson, N.L. (eds.) Encyclopedia of Statistical Sciences, vol. 7, pp. 69–74. Wiley, New York, NY
Eisen, S.V., Dill, D.L., Grob, M.C.: Reliability and validity of a brief patient-reported instrument for psychiatric outcome evaluation. Hosp Community Psychiatry 45, 242–247 (1994)
Eisen, S.V., Normand. S-L.T., Belanger, A.J., Spiro, A., Esch, D.: The revised behavior and symptom identification scale (BASIS-24): reliability and validity. Medical Care 42(12), 1230–1241 (2004)
Embretson S.E.: The new rules of measurement. Psychol. Assess. 8, 341–349 (1996)
Embretson, S.E., Reise, S.P.: Item Response Theory for Psychologists. Lawrence Erlbaum Associates, Mahwah, NJ (2000)
Garnder, W., Kelleher, K., Pajer, K.: Multidimensional adaptive testing for mental health problems in primary care. Med. Care 40, 812–823 (2002)
Gibbons, R.D., Lavigne, J.V.: Emergence of childhood psychiatric disorders: A multivariate probit analysis. Stat. Med. 17, 2487–2499 (1998)
Gibbons, R.D., Wilcox-Gok, V.: Health service utilization and insurance coverage: A multivariate probit analysis. J. Am. Stat. Assoc. 93, 63–72 (1998)
Hambleton, R.K., Robin, R., Xing, D.: Item response models for the analysis of educational and psychological test data. In: Tinsley, H.E.A., Bronw, S.D. (eds.) Handbook of Applied Multivariate Statistics and Mathematical Modeling. Academic Press, San Diego, CA (2000)
Harris, R.J.: A Primer of Multivariate Statistics. Academic Press, Florida (1985)
Hattie, J.: Methodology review: Assessing unidimensionality of tests and items. Appl. Psychol. Meas. 9, 139–164 (1985)
Landrum, M.B., Bronskill, S.E., Normand, S.-L.T.: Analytic methods for constructing cross-sectional profiles of health care providers. Health. Serv. Outcomes Res. Methodol. 1, 23–47 (2000)
Masters, G.N.: A Rasch model for partial credit scoring. Psychometrika 47, 149–174 (1982)
McCullagh, P., Nelder, J.A.: Generalized linear Models (Chapter 5), 2nd Edition. Chapman and Hall, New York (1990)
Multilog (1991). Scientific Software International, Inc. Chicago, IL
Muraki, E.: Fitting a polytomous item response model to Likert-type data. Appl. Psychol. Meas. 14, 59–71 (1990)
Muraki, E.: A generalized partial credit model: Application of an EM algorithm. Appl. Psychol. Meas. 16, 159–176 (1992)
Muthén, L.K., Muthén, B.O.: Mplus Statistics Analysis with Latnet Variables User’s Guide. Muthén & Muthénn, Los Angeles, CA (2005)
Parscale (1997). Scientific Software International, Inc. Chicago, IL
Reckase, M.: Unifactor latent trait models applied to multifactor tests: Results and implications. J. Educ. Stat. 4, 207–230 (1979)
Samejima, F.: The graded response model. In: van der Linden W.J., Hambleton R.K. (eds.) Handbook of modern item response theory. Springer, New York, NY (1996)
Scott, S.L., Edward, H.Ip.: Empirical Bayes and item-clustering effects in a latent variable hierarchical model: A case study from the National Assessment of Educational Progress. J. Am. Stat. Assoc. 97, 409–419 (2002)
Skrondal, A., Rabe-Hesketh, S.: Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models. Chapman & Hall/CRC Press, New York, USA (2004)
Tanner, MA.: Tools for Statistical Inference. Methods for the Exploration of Posterior Distributions and Likelihood Functions. Springer-Verlag, New York, USA (1993)
Uttaro, T., Lehman, A.: Graded response modeling of the Quality of Life Interview. Eval. Program Planning 22, 41–52 (1999)
van der Linden, W.J., Hambleton, R.K.: Handbook of Modern Item Response Theory. Springer, New York, NY (1997)
von Eye, A., Clogg, C.C.: Latent Variables Analysis: Applications for Developmental Research. SAGE, London, UK (1994)
Acknowledgements
We are indebted to Ron Hambleton (University of Massachusetts) and Colleen McHorney (University of Indiana) for helpful advice on earlier versions of this manuscript. This work was supported by Grant R01-MH58240 from the National Institute of Mental Health and by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Normand, SL.T., Belanger, A.J. & Eisen, S.V. Graded response model-based item selection for behavior and symptom identification. Health Serv Outcomes Res Method 6, 1–19 (2006). https://doi.org/10.1007/s10742-006-0005-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-006-0005-0