Quality of Life Research

, Volume 24, Issue 3, pp 553–564 | Cite as

RespOnse Shift ALgorithm in Item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies

  • Alice Guilleux
  • Myriam Blanchin
  • Antoine Vanier
  • Francis Guillemin
  • Bruno Falissard
  • Carolyn E. Schwartz
  • Jean-Benoit Hardouin
  • Véronique SébilleEmail author
Response Shift and Missing Data



Some IRT models have the advantage of being robust to missing data and thus can be used with complete data as well as different patterns of missing data (informative or not). The purpose of this paper was to develop an algorithm for response shift (RS) detection using IRT models allowing for non-uniform and uniform recalibration, reprioritization RS recognition and true change estimation with these forms of RS taken into consideration if appropriate.


The algorithm is described, and its implementation is shown and compared to Oort’s structural equation modeling (SEM) procedure using data from a clinical study assessing health-related quality of life in 669 hospitalized patients with chronic conditions.


The results were quite different for the two methods. Both showed that some items of the SF-36 General Health subscale were affected by response shift, but those items usually differed between IRT and SEM. The IRT algorithm found evidence of small recalibration and reprioritization effects, whereas SEM mostly found evidence of small recalibration effects.


An algorithm has been developed for response shift analyses using IRT models and allows the investigation of non-uniform and uniform recalibration as well as reprioritization. Differences in RS detection between IRT and SEM may be due to differences between the two methods in handling missing data. However, one cannot conclude on the differences between IRT and SEM based on a single application on a dataset since the underlying truth is unknown. A next step would be to implement a simulation study to investigate those differences.


Item response theory Response shift Missing data Attrition Bias Quality of life 



This study was supported by the Institut National du Cancer, under reference “INCA_6931.” The SatisQoL cohort project (investigators: P. Auquier, F. Guillemin (PI), M. Mercier) was supported by an IRESP (Institut de recherche en santé publique) Grant from Inserm, and a PHRC (Programme Hospitalier de Recherche Clinique) National Grant from French Ministry of Health, France.


  1. 1.
    Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 14(3), 587–598.CrossRefGoogle Scholar
  2. 2.
    Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data. Hoboken: Wiley.CrossRefGoogle Scholar
  3. 3.
    Swartz, R. J., Schwartz, C., Basch, E., Cai, L., Fairclough, D. L., McLeod, L., et al. (2011). The king’s foot of patient-reported outcomes: Current practices and new developments for the measurement of change. Quality of Life Research, 20(8), 1159–1167.CrossRefPubMedCentralPubMedGoogle Scholar
  4. 4.
    Fischer, G. H., & Molenaar, I. W. (1995). Rasch models: Foundations, recent developments, and applications. New York: Springer.CrossRefGoogle Scholar
  5. 5.
    Wang, W.-C., & Chyi-In, W. (2004). Gain score in item response theory as an effect size measure. Educational and Psychological Measurement, 64(5), 758–780.CrossRefGoogle Scholar
  6. 6.
    Andrich, D. (2011). Rating scales and rasch measurement. Expert Review of Pharmacoeconomics & Outcomes Research, 11, 571–585.CrossRefGoogle Scholar
  7. 7.
    De Bock, E., Hardouin, J. -B., Blanchin, M., Le Neel, T., Kubis, G., Bonnaud-Antignac, A., et al. (2013). Rasch-family models are more valuable than score-based approaches for analysing longitudinal patient-reported outcomes with missing data. Statistical Methods in Medical Research (in press).Google Scholar
  8. 8.
    Sébille, V., Hardouin, J.-B., & Mesbah, M. (2007). Sequential analysis of latent variables using mixed-effect latent variable models: Impact of non-informative and informative missing data. Statistics in Medicine, 26(27), 4889–4904.CrossRefPubMedGoogle Scholar
  9. 9.
    Hardouin, J.-B., Conroy, R., & Sébille, V. (2011). Imputation by the mean score should be avoided when validating a Patient Reported Outcomes questionnaire by a Rasch model in presence of informative missing data. BMC Medical Research Methodology, 11(1), 105.CrossRefPubMedCentralPubMedGoogle Scholar
  10. 10.
    De Bock, É. de, Hardouin, J. -B., Blanchin, M., Neel, T. L., Kubis, G., & Sébille, V. (2014). Assessment of score- and Rasch-based methods for group comparison of longitudinal patient-reported outcomes with intermittent missing data (informative and non-informative). Quality of Life Research, 1–11.Google Scholar
  11. 11.
    Hamel J. F., Sébille V., Le Neel T., Kubis G., & Hardouin J. B. (2012) Study of different methods for comparing groups by analysis of patients reported outcomes: Item response theory based methods seem more efficient than classical test theory based methods when data is missing. Under review.Google Scholar
  12. 12.
    Schwartz, C. E., & Sprangers, M. A. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science & Medicine, 48(11), 1531–1548.Google Scholar
  13. 13.
    Glas, C. A. W. (1988). The derivation of some tests for the Rasch model from the multinomial distribution. Psychometrika, 53, 525–546.CrossRefGoogle Scholar
  14. 14.
  15. 15.
    Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Thousand Oaks, CA, US: Sage Publications, Inc.Google Scholar
  16. 16.
    Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.Google Scholar
  17. 17.
    R Development Core Team. (n.d.). R Development Core Team. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.Google Scholar
  18. 18.
    Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research-Online, 8, 23–74.Google Scholar
  19. 19.
    Bryant, F. B., & Satorra, A. (2012). Principles and practice of scaled difference Chi Square testing. Structural Equation Modeling: A Multidisciplinary Journal, 19(3), 372–398.CrossRefGoogle Scholar
  20. 20.
    Enders, C. K. (2013). Analyzing structural equation models with missing data. In Structural Equation (Ed.), Modeling : a second course (pp. 493–519). Charlotte, NC: IAP, Information Age Publ.Google Scholar
  21. 21.
    Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8(3), 430–457.CrossRefGoogle Scholar
  22. 22.
    Kepka, S., Baumann, C., Anota, A., Buron, G., Spitz, E., Auquier, P., Guillemin, F., Mercier, M. (2013). The relationship between traits optimism and anxiety and health-related quality of life in patients hospitalized for chronic diseases: data from the SATISQOL study. Health and Quality of Life Outcomes, 11(1), 134.Google Scholar
  23. 23.
    Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30(6), 473–483.CrossRefPubMedGoogle Scholar
  24. 24.
    Leplège, A., Ecosse, E., Verdier, A., & Perneger, T. V. (1998). The French SF-36 Health Survey: translation, cultural adaptation and preliminary psychometric evaluation. Journal of Clinical Epidemiology, 51(11), 1013–1023.CrossRefPubMedGoogle Scholar
  25. 25.
    Leplège, A. (2001). Le questionnaire MOS SF-36: manuel de l’utilisateur et guide d’interprétation des scores. Paris: Editions ESTEM.Google Scholar
  26. 26.
    Fairclough, D. L. (2002). Design and analysis of quality of life studies in clinical trials: Interdisciplinary statistics. London: Chapman & Hall/Crc.Google Scholar
  27. 27.
    Hamel, J.-F., Hardouin, J.-B., Le Neel, T., Kubis, G., Roquelaure, Y., & Sébille, V. (2012). Biases and power for groups comparison on subjective health measurements. PLoS ONE, 7(10), e44695.CrossRefPubMedCentralPubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Alice Guilleux
    • 1
  • Myriam Blanchin
    • 1
  • Antoine Vanier
    • 1
    • 2
    • 3
  • Francis Guillemin
    • 4
  • Bruno Falissard
    • 5
    • 6
  • Carolyn E. Schwartz
    • 7
    • 8
  • Jean-Benoit Hardouin
    • 1
    • 9
  • Véronique Sébille
    • 1
    • 9
    Email author
  1. 1.Biostatistics, Pharmacoepidemiology and Subjective Measures in Health Sciences, EA 4275University of NantesNantesFrance
  2. 2.Department of BiostatisticsUPMC, Univ. Paris 06ParisFrance
  3. 3.Department of Biostatistics Public Health and Medical Informatics, AP-HPUniversity Hospitals Pitié-Salpêtrière Charles-FoixParisFrance
  4. 4.EA 4360 Apemac, Lorraine University, Paris Descartes UniversityNancyFrance
  5. 5.INSERM 669, Université Paris-Sud, Université Paris DescartesParisFrance
  6. 6.Département de santé publique, AP-HPHôpital Paul BrousseVillejuifFrance
  7. 7.DeltaQuest Foundation, Inc.ConcordUSA
  8. 8.Department of Medicine and Orthopeadic SurgeryTufts University School of MedicineBostonUSA
  9. 9.Clinical Research Unit, Department of Methodology and BiostatisticsUniversity Hospital of NantesNantesFrance

Personalised recommendations