Quality of Life Research

, Volume 25, Issue 6, pp 1385–1393 | Cite as

The Guttman errors as a tool for response shift detection at subgroup and item levels

  • Myriam BlanchinEmail author
  • Véronique Sébille
  • Alice Guilleux
  • Jean-Benoit Hardouin
Special Section: Response Shift Effects at Item Level (by invitation only)



Statistical methods for identifying response shift (RS) at the individual level could be of great practical value in interpreting change in PRO data. Guttman errors (GE) may help to identify discrepancies in respondent’s answers to items compared to an expected response pattern and to identify subgroups of patients that are more likely to present response shift. This study explores the benefits of using a GE-based method for RS detection at the subgroup and item levels.


The analysis was performed on the SatisQoL study. The number of GE was determined for each individual at each time of measurement (at baseline T0 and 6 months after discharge M6). Individuals showing discrepancies (with many GE) were suspected to interpret the items differently from the majority of the sample. Patients having a large number of GE at M6 only and not at T0 were assumed to present RS. Patients having a small number of GE at T0 and M6 were assumed to present no RS. The RespOnse Shift ALgorithm in Item response theory (ROSALI) was then applied on the whole sample and on both groups.


Different types of RS (non-uniform recalibration, reprioritization) were more prevalent in the group composed of patients assumed to present RS based on GE. On the opposite, no RS was detected on patients having few GE.


Guttman errors and item response theory models seem to be relevant tools to discriminate individuals affected by RS from the others at the item level.


Response shift Guttman errors Item response theory Item level Individual level 



The authors gratefully acknowledge Frans J. Oort, Mirjam A. G. Sprangers and Mathilde G. E. Verdam for their comments on the manuscript. This study was supported by the Institut National du Cancer, under reference “INCA_6931.” The SatisQoL cohort project (Investigators: P Auquier, F Guillemin (PI), M Mercier) was supported by an IRESP (Institut de recherche en santé publique) grant from Inserm, and a PHRC (Programme Hospitalier de Recherche Clinique) national grant from French Ministry of Health, France.

Compliance with ethical standards

Authors declare that they have no conflict of interest. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The SatisQoL study was approved by the national Institutional Review Board and the national committee for data protection (CCTIRS 07.212 and CNIL 1248560).


  1. 1.
    Schwartz, C. E., & Sprangers, M. A. (1999). Methodological approaches for assessing response shift in longitudinal health related quality-of-life research. Social Science and Medicine, 48(11), 1531–1548.CrossRefPubMedGoogle Scholar
  2. 2.
    Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14(3), 587–598.CrossRefPubMedGoogle Scholar
  3. 3.
    Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Hardouin, J. B., & Sébille, V. (2015). RespOnse shift algorithm in item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies. Quality of Life Research, 24(3), 553–564.CrossRefPubMedGoogle Scholar
  4. 4.
    Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17(4), 627–639. doi: 10.1007/s11136-008-9329-2.CrossRefPubMedGoogle Scholar
  5. 5.
    Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory (1st ed., Vol. 5). Thousand Oaks: Sage.Google Scholar
  6. 6.
    Kepka, S., Baumann, C., Anota, A., Buron, G., Spitz, E., Auquier, P., & Mercier, M. (2013). The relationship between traits optimism and anxiety and health-related quality of life in patients hospitalized for chronic diseases: Data from the SATISQOL study. Health and Quality of Life Outcomes, 11(1), 134. doi: 10.1186/1477-7525-11-134.CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30(6), 473–483.CrossRefPubMedGoogle Scholar
  8. 8.
    Leplège, A., Ecosse, E., Verdier, A. & Perneger, T. V. (1998). The French SF-36 health survey: Translation, cultural adaptation and preliminary psychometric evaluation. Journal of Clinical Epidemiology, 51(11), 1013–1023. doi: 10.1016/S0895-4356(98)00093-6.
  9. 9.
    Beller, J., & Kliem, S. (2013). GetR: Calculate Guttman error trees in R (version 0.1) [computer software]. Hannover, Germany.
  10. 10.
    Meijer, R. R. (1994). The number of Guttman errors as a simple and powerful person-fit statistic. Applied Psychological Measurement, 18(4), 311–314.CrossRefGoogle Scholar
  11. 11.
    Tendeiro, J. N., & Meijer, R. R. (2014). Detection of invalid test scores: The usefulness of simple nonparametric statistics. Journal of Educational Measurement, 51(3), 239–259.CrossRefGoogle Scholar
  12. 12.
    Lanza, S. T., & Rhoades, B. L. (2013). Latent class analysis: An alternative perspective on subgroup analysis in prevention and treatment. Prevention Science, 14(2), 157–168.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Clogg, C. C. (1995). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 311–359). New York: Springer.CrossRefGoogle Scholar
  14. 14.
    van Leeuwen, C. M. C., Post, M. W. M., van der Woude, L. H. V., de Groot, S., Smit, C., van Kuppevelt, D., & Lindeman, E. (2012). Changes in life satisfaction in persons with spinal cord injury during and after inpatient rehabilitation: Adaptation or measurement bias? Quality of Life Research, 21(9), 1499–1508.CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    McIntosh, C. N. (2013). Pitfalls in subgroup analysis based on growth mixture models: A commentary on Van Leeuwen et al. (2012). Quality of Life Research, 22(9), 2625–2629.CrossRefPubMedGoogle Scholar
  16. 16.
    Bolck, A., Croon, M., & Hagenaars, J. (2004). Estimating latent structure models with categorical variables: One-step versus three-step estimators. Political Analysis, 12(1), 3–27.CrossRefGoogle Scholar
  17. 17.
    Vermunt, J. K. (2010). Latent class modeling with covariates: Two improved three-step approaches. Political Analysis, 18(4), 450–469.CrossRefGoogle Scholar
  18. 18.
    Kadengye, D. T., Ceulemans, E., & Van den Noortgate, W. (2014). A generalized longitudinal mixture IRT model for measuring differential growth in learning environments. Behavior Research Methods, 46(3), 823–840.PubMedGoogle Scholar
  19. 19.
    Boom, J. (2015). A new visualization and conceptualization of categorical longitudinal development: Measurement invariance and change. Frontiers in Psychology, 6, 289.CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Lu, Z. L., Zhang, Z., & Lubke, G. (2011). Bayesian inference for growth mixture models with latent class dependent missing data. Multivariate Behavioral Research, 46(4), 567–597.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Verhagen, J., & Fox, J.-P. (2013). Longitudinal measurement in health-related surveys. A Bayesian joint growth model for multivariate ordinal responses. Statistics in Medicine, 32(17), 2988–3005.CrossRefPubMedGoogle Scholar
  22. 22.
    Sprangers, M. A. G., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.CrossRefPubMedGoogle Scholar
  23. 23.
    Rapkin, B. D. & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2(14). doi: 10.1186/1477-7525-2-14.
  24. 24.
    Holland, P. W., & Wainer, H. (1993). Differential item functioning. Hillsdale: Erlbaum.Google Scholar
  25. 25.
    Osterlind, S. J., & Everson, H. T. (2009). Differential item functioning (2nd ed.). Thousand Oaks: Sage.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.EA 4275, Biostatistics, Pharmacoepidemiology and Subjective Measures in Health SciencesUniversity of NantesNantesFrance

Personalised recommendations