Quality of Life Research

, Volume 23, Issue 9, pp 2421–2430 | Cite as

Minimal evidence of response shift in the absence of a catalyst

  • Sara AhmedEmail author
  • Richard Sawatzky
  • Jean-Frédéric Levesque
  • Deborah Ehrmann-Feldman
  • Carolyn E. Schwartz



Individuals with chronic conditions experience fluctuations in health status and thus may experience response shift. We sought to test the hypothesis that response shift effects would be non-significant among individuals with chronic disease who experienced relatively small changes in their health status over a 1-year period.


This secondary analysis utilized longitudinal cohort data on a community-based sample (n = 776) representing four chronic diseases (arthritis, heart failure, diabetes, or chronic obstructive pulmonary disease). Information on health-care utilization was obtained from the provincial health insurance database. Participants completed the SF-36 twice annually. Parameter invariance over 1 year in a second-order SF-36 factor structure was evaluated by adapting Oort’s approach by fitting a second-order measurement structure with first-order factors for the SF-36 subscales and second-order factors for physical and mental health status while accommodating ordinal data.


Over 80 % of participants had no hospitalizations or emergency room visits over follow-up. The model had an acceptable fit when all measurement model parameters were constrained at both time points (RMSEA = .035, CFI = .97). There was no substantial difference in fit when measurement model parameters (item thresholds, first-order factor intercepts, and factor loadings) were allowed to vary over time.


Among chronically ill individuals with stable health, substantial response shift effects were not detected. These results support the theoretical proposition that response shift is not expected to occur in patients with relatively stable conditions.


Structural equation modeling Chronic disease Response shift Health-related quality of life 



This work was funded in part by a Catalyst grant award from the Canadian Institute of Health Research (Grant #103630), and a Career Award (Grant #13870) from the Fond de Recherche en Sante du Quebec to Dr. Ahmed. We thank Brian Quaranto, B.Sc., for assistance with data management and manuscript preparation.

Supplementary material

11136_2014_699_MOESM1_ESM.docx (158 kb)
Figure 2 (online repository): Distributions of the SF-36 items at baseline and one-year follow-up* * The numbers of each item and their response categories correspond with those of the Qualimetric SF-36® instrument (Version 1). The response categories of items GH_1, GH_11B, GH_11D, BP_7, BP_8, VI_9a, VI_9e, SF_6, MH_9d and MH_9h have been reversed so the greater category numbers indicate higher functioning (DOCX 157 kb)
11136_2014_699_MOESM2_ESM.docx (27 kb)
Supplementary material 2 (DOCX 26 kb)


  1. 1.
    Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54(4), 297–312.PubMedCrossRefGoogle Scholar
  2. 2.
    Cronbach, L. J., & Furby, L. (1970). How we should measure “change”: Or should we? Psychological Bulletin, 74, 68–80.CrossRefGoogle Scholar
  3. 3.
    Mellenbergh, G. J. (1989). Item bias and item response theory. International Journal of Educational Research, 13(2), 127–143.CrossRefGoogle Scholar
  4. 4.
    Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543.CrossRefGoogle Scholar
  5. 5.
    Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.PubMedCrossRefGoogle Scholar
  6. 6.
    Schwartz, C. E., & Sprangers, M. A. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48(11), 1531–1548.PubMedCrossRefGoogle Scholar
  7. 7.
    Schwartz, C. E., & Rapkin, B. D. (2004). Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health and Quality of Life Outcomes, 2, 16.PubMedCrossRefPubMedCentralGoogle Scholar
  8. 8.
    Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.PubMedCrossRefGoogle Scholar
  9. 9.
    Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2(1), 14.PubMedCrossRefPubMedCentralGoogle Scholar
  10. 10.
    Schwartz, C. E., Bode, R., Repucci, N., Becker, J., Sprangers, M. A., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15(9), 1533–1550.PubMedCrossRefGoogle Scholar
  11. 11.
    Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14(3), 587–598.PubMedCrossRefGoogle Scholar
  12. 12.
    King-Kallimanis, B., Oort, F., & Garst, G. (2010). Using structural equation modelling to detect measurement bias and response shift in longitudinal data. AStA Advances in Statistical Analysis, 94(2), 139–156.CrossRefGoogle Scholar
  13. 13.
    Visser, M. R., Oort, F. J., & Sprangers, M. A. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14(3), 629–639.PubMedCrossRefGoogle Scholar
  14. 14.
    Schwartz, C. E., Sprangers, M. A., Oort, F. J., Ahmed, S., Bode, R., Li, Y., et al. (2011). Response shift in patients with multiple sclerosis: An application of three statistical techniques. Quality of Life Research, 20(10), 1561–1572.PubMedCrossRefGoogle Scholar
  15. 15.
    Schwartz, C. E., Sprangers, M. A. G., Carey, A., & Reed, G. (2004). Exploring response shift in longitudinal data. Psychology and Health, 19(1), 51–69.CrossRefGoogle Scholar
  16. 16.
    Hakkennes, S. J., Brock, K., & Hill, K. D. (2011). Selection for inpatient rehabilitation after acute stroke: A systematic review of the literature. Archives of Physical Medicine and Rehabilitation, 92(12), 2057–2070.PubMedCrossRefGoogle Scholar
  17. 17.
    Ahmed, S., Bourbeau, J., Maltais, F., & Mansour, A. (2009). The Oort structural equation modeling approach detected a response shift after a COPD self-management program not detected by the Schmitt technique. Journal of Clinical Epidemiology, 62, 1165–1172.PubMedCrossRefGoogle Scholar
  18. 18.
    Razmjou, H., Schwartz, C. E., & Holtby, R. (2010). Recalibration response shift had an independent impact on perceived disability two years following rotator cuff surgery. Journal of Bone and Joint Surgery, 92, 2178–2186.PubMedCrossRefGoogle Scholar
  19. 19.
    Razmjou, H., Schwartz, C. E., Yee, A., & Finkelstein, J. A. (2009). Traditional assessment of health outcome following total knee arthroplasty was confounded by response shift phenomenon. Journal of Clinical Epidemiology, 62, 91–96.PubMedCrossRefGoogle Scholar
  20. 20.
    Razmjou, H., Yee, A., Ford, M., & Finkelstein, J. A. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery, 88(12), 2590–2595.PubMedCrossRefGoogle Scholar
  21. 21.
    Finkelstein, J. A., Razmjou, H., & Schwartz, C. E. (2009). Response shift and outcome assessment in orthopedic surgery: Is there is a difference between complete vs. partial treatment? Journal of Clinical Epidemiology, 82, 1189–1190.CrossRefGoogle Scholar
  22. 22.
    Ahmed, S., Mayo, N. E., Corbiere, M., Wood-Dauphinee, S., Hanley, J., & Cohen, R. (2005). Change in quality of life of people with stroke over time: True change or response shift? Quality of Life Research, 14(3), 611–627.PubMedCrossRefGoogle Scholar
  23. 23.
    Schumacker, R. E., & Lomax, R. G. (2004). A Beginner’s Guide to Structural Equation Modeling, Second Edition (2nd ed.). Mahwah, New Jersey: Lawrence Erlbaum Associates Inc.Google Scholar
  24. 24.
    Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.CrossRefGoogle Scholar
  25. 25.
    Sholom, G. (2012). PROMs: A Critical Step. But Only One of Many. HealthcarePapers, 11(4), 29–33.Google Scholar
  26. 26.
    Lemieux, V., Levesque, J. F., & Ehrmann-Feldman, D. (2011). Are primary healthcare organizational attributes associated with patient self-efficacy for managing chronic disease? Health Policy, 6(4), e89–e105.Google Scholar
  27. 27.
    Feldman, D. E., Bernatsky, S., Levesque, J. F., Van, M. T., Houde, M., & April, K. T. (2010). Access and perceived need for physical and occupational therapy in chronic arthritis. Disability and Rehabilitation, 32(22), 1827–1832.PubMedCrossRefGoogle Scholar
  28. 28.
    Ware, J. E, Jr, Kosinski, M., & Keller, S. D. (1994). SF-36 physical & mental scales: A user’s manual. Boston, Massachusetts: The Health Institute, New England Medical Center.Google Scholar
  29. 29.
    McHorney, C. A., Ware, J. E, Jr, & Raczek, A. E. (1993). The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31(3), 247–263.PubMedCrossRefGoogle Scholar
  30. 30.
    Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479–515.CrossRefGoogle Scholar
  31. 31.
    Byrne, B. M. (2012). Structural equation modeling with Mplus : basic concepts, applications, and programming. New York: Routledge Academic.Google Scholar
  32. 32.
    Inc, S. A. S. I. (2009). SAS/STAT 9.2 User’s Guide. Cary, NC: SAS Institute Inc.Google Scholar
  33. 33.
    Muthén, L. K., & Muthén, B. O. (1998–2010). Mplus user’s guide (Sixth Edition ed). Los Angeles, CA: Muthén & Muthén.Google Scholar
  34. 34.
    Muthén, B., & Muthén, L. (2011). MPlus (version 6.2). Los Angeles, CA: Statmodel.Google Scholar
  35. 35.
    Finney, S. J., & DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In I. G. R. H. R. O. Mueller (Ed.), Structural equation modeling: A second course (pp. 269-314). Greenwich, CT: Information Age Publishing.Google Scholar
  36. 36.
    Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186–203.CrossRefGoogle Scholar
  37. 37.
    Nussbeck, F. W., Eid, M., & Lischetzke, T. (2006). Analysing multitrait–multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: What sample size is needed for valid results? British Journal of Mathematical and Statistical Psychology, 59(1), 195–213.PubMedCrossRefGoogle Scholar
  38. 38.
    Asparouhov, T., & Muthén, B. (2006). Robust Chi Square difference testing with mean and variance adjusted test statistics. Mplus Web Notes: No. 10.
  39. 39.
    Wilson, I. B., & Cleary, P. D. (1995). Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA, 273(1), 59–65.PubMedCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Sara Ahmed
    • 1
    • 2
    • 3
    Email author
  • Richard Sawatzky
    • 4
    • 6
  • Jean-Frédéric Levesque
    • 5
    • 7
    • 8
  • Deborah Ehrmann-Feldman
    • 7
    • 9
  • Carolyn E. Schwartz
    • 10
    • 11
  1. 1.Faculty of Medicine, School of Physical and Occupational TherapyMcGill UniversityMontrealCanada
  2. 2.Clinical EpidemiologyMcGill University Health CenterMontrealCanada
  3. 3.Centre de recherche interdisciplinaire en réadaptation (CRIR)MontrealCanada
  4. 4.Trinity Western University School of NursingLangleyCanada
  5. 5.Centre de Recherche du Centre Hospitalier de l’Université de MontréalMontrealCanada
  6. 6.Centre for Health Evaluation and Outcome SciencesVancouverCanada
  7. 7.Université de MontréalMontrealCanada
  8. 8.Institut National de Santé Publique du QuébecMontrealCanada
  9. 9.Direction de Santé Publique de l’ASSS de MontréalMontrealCanada
  10. 10.DeltaQuest Foundation, Inc.ConcordUSA
  11. 11.Departments of Medicine and Orthopaedic SurgeryTufts University Medical SchoolBostonUSA

Personalised recommendations