Quality of Life Research

, Volume 27, Issue 5, pp 1133–1146 | Cite as

Scoping review of response shift methods: current reporting practices and recommendations

  • Tolulope T. Sajobi
  • Ronak Brahmbatt
  • Lisa M. Lix
  • Bruno D. Zumbo
  • Richard Sawatzky



Response shift (RS) has been defined as a change in the meaning of an individual’s self-evaluation of his/her health status and quality of life. Several statistical model- and design-based methods have been developed to test for RS in longitudinal data. We reviewed the uptake of these methods in patient-reported outcomes (PRO) literature.


CINHAHL, EMBASE, Medline, ProQuest, PsycINFO, and Web of Science were searched to identify English-language articles about RS published until 2016. Data on year and country of publication, PRO measure adopted, RS detection method, type of RS detected, and testing of underlying model assumptions were extracted from the included articles.


Of the 1032 articles identified, 101 (9.8%) articles were included in the study. While 54.5 of the articles reported on the Then-test, 30.7% of the articles reported on Oort’s or Schmitt’s structural equation modeling (SEM) procedure. Newer RS detection methods, such as relative importance analysis and random forest regression, have been used less frequently. Less than 25% reported on testing the assumptions underlying the adopted RS detection method(s).


Despite rapid methodological advancements in RS research, this review highlights the need for further research about RS detection methods for complex longitudinal data and standardized reporting guidelines.


Response shift Systematic review Patient-reported outcomes 



The Canadian Institutes of Health Research provided support (Grant # MOP-142404) to Drs Sajobi, Lix, Zumbo, and Sawatzky in this research. Dr. Sajobi is supported by the O’Brien Institute for Public Health; Dr. Lix is supported by the Manitoba Research Chair; and Dr. Sawatzky holds a Canada Research Chair in Patient-Reported Outcomes at Trinity Western University, Langley, British Columbia. We are grateful for the support in conducting literature searches provided by Duncan Dixon, health sciences librarian at Trinity Western University. This research was initiated during Dr. Sajobi’s visit to Trinity Western University in 2014.

Supplementary material

11136_2017_1751_MOESM1_ESM.docx (57 kb)
Supplementary material 1 (DOCX 57 KB)
11136_2017_1751_MOESM2_ESM.xlsx (20 kb)
Supplementary material 2 (XLSX 20 KB)


  1. 1.
    Revicki, D. A. (1989). Health-related quality of life in the evaluation of medical therapy for chronic illness. Journal of Family Practice, 29(4), 377–380.PubMedGoogle Scholar
  2. 2.
    Berzon, R., Hays, R. D., & Shumaker, S. A. (1993). International use, application and performance of health-related quality of life instruments. Quality of Life Research, 2(6), 367–368.CrossRefPubMedGoogle Scholar
  3. 3.
    Deshpande, P. R., Rajan, S., Sudeepthi, B. L., & Abdul Nazir, C. P. (2011). Patient-reported outcomes: A new era in clinical research. Perspectives in Clinical Research, 2(4), 137–144.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Golembiewski, R. T., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. The Journal of Applied Behavioral Science, 12, 133–157.CrossRefGoogle Scholar
  5. 5.
    Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, S. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and reevaluation of retrospective pretests. Applied Psychological Measurement, 3, 1–23.CrossRefGoogle Scholar
  6. 6.
    Breetvelt, I. S., & Van Dam, F. S. (1991). Underreporting by cancer patients: the case of response-shift. Social Science and Medicine, 32, 981–987.CrossRefPubMedGoogle Scholar
  7. 7.
    Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: a theoretical model. Social Science and Medicine, 48(11), 1507–1515.CrossRefPubMedGoogle Scholar
  8. 8.
    Taminiau-Bloem, E. F., van Zuuren, F. J., Visser, M. R. M., Tishelman, C., Schwartz, C. E., Koeneman, M. A., et al. (2011) Opening the black box of cancer patients’ quality-of-life change assessments: a think-aloud study examining the cognitive processes underlying responses to transition items. Psychol Health, 26(11), 1414–1428.CrossRefPubMedGoogle Scholar
  9. 9.
    Schwartz, C. E., Andresen, E. M., Nosek, M. A., & Krahn, G. L. & RRCT Expert Panel on Health Status Measurement. (2007). Response shift theory: Important implications for measuring quality of life in people with disability. Archives of Physical Medicine and Rehabilitation, 88(4), 529–536.CrossRefPubMedGoogle Scholar
  10. 10.
    Schwartz, C. E. (2010). Application of response shift theory and methods to participation measurement: A brief history of a young field. Archives of Physical Medicine and Rehabilitation, 91(9), S38–S43.CrossRefPubMedGoogle Scholar
  11. 11.
    Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health Qual Life Outcomes, 2, 14.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Schwartz, C. E., Bode, R., Repucci, N., Becker, J., Sprangers, M. A., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15(9), 1533–1550.CrossRefPubMedGoogle Scholar
  13. 13.
    Sawatzky, R., Sajobi, T. T., Brahmbhatt, R., Chan, E. K. H., Lix, L. M., & Zumbo, B. D. (2017). Longitudinal change in response processes: A response shift perspective. In B. D. Zumbo & A. M. Hubley (Eds.), Understanding and investigating response processes in validation research (pp. 251–276). New York: Springer.CrossRefGoogle Scholar
  14. 14.
    Balanin, B., Ennis, O., Kanes, G., Siinghal, R., Roberts, S. N., Rees, D., et al. (2009). Response shift in self-reported functional scores after knee microfracture for full thickness cartilage lesions. Osteoarthritis and Cartilage, 17(8), 1009–1013.CrossRefGoogle Scholar
  15. 15.
    Sajobi, T. T., Fiest, K. M., & Wiebe, S. (2014). Changes in quality of life after epilepsy surgery: the role of reprioritization response shift. Epilepsia, 55(9), 1331–1338.CrossRefPubMedGoogle Scholar
  16. 16.
    Vanier, A., Falissard, B., Sébille, V., & Hardouin, J. B. (2017). The complexity of interpreting changes observed over time in health-related quality of life: A short overview of 15 years of research on response shift theory. In F. Guillemin, A. Leplège, S. Briançon, E. Spitz & J. Coste (Eds.), Perceived health and adaptation in chronic disease. Stakes and future challenge (pp. 202–230). New York, NY: CRC Press.Google Scholar
  17. 17.
    Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984). Methods of analysis with response-shift bias. Educational and Psychological Measurement, 44(4), 781–804.CrossRefGoogle Scholar
  18. 18.
    Schmitt, N. (1982). The use of analysis of covariance structures to assess beta and gamma change. Multivariate Behavioral Research, 17, 343–358.CrossRefPubMedGoogle Scholar
  19. 19.
    Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14(3), 587–598.CrossRefPubMedGoogle Scholar
  20. 20.
    Oort, F. J., Visser, M. R. M., & Sprangers, M. A. G. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14(3), 599–609.CrossRefPubMedGoogle Scholar
  21. 21.
    Anota, A., Bascoul-Mollevi, C., Conroy, T., Guillemin, F., Velten, M., Jolly, D., et al. (2014). Item response theory and factor analysis as a mean to characterize occurrence of response shift in a longitudinal quality of life study in breast cancer patients. Health Qual Life Outcomes, 12, 32.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2015). RespOnse Shift ALgorithm in Item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies. Quality of Life Research, 24(3), 553–564.CrossRefPubMedGoogle Scholar
  23. 23.
    Lowy, A., & Bernhard, J. (2004). Quantitative assessment of changes in patients’ constructs of quality of life: an application of multilevel models. Quality of Life Research, 13(7), 1177–1185.CrossRefPubMedGoogle Scholar
  24. 24.
    Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17(4), 627–639.CrossRefPubMedGoogle Scholar
  25. 25.
    Lix, L. M., Sajobi, T. T., Sawatzky, R., Liu, J., Mayo, N. E., Huang, Y., et al. (2013). Relative importance measures for reprioritization response shift. Quality of Life Research, 22(4), 695–703.CrossRefPubMedGoogle Scholar
  26. 26.
    Li, Y., & Rapkin, B. (2009). Classification and regression tree uncovered hierarchy of psychosocial determinants underlying quality-of-life response shift in HIV/AIDS. Journal of Clinical Epidemiology, 62(11), 1138–1147.CrossRefPubMedPubMedCentralGoogle Scholar
  27. 27.
    Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., et al. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: a cohort study. BMC Medical Research Methodology, 13, 20.CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Verdam, M. G. E., Oort, F. J., & Sprangers, M. A. G. (2016). Using structural equation modeling to detect response shifts and true change in discrete variables: an application to the items of the SF-36. Quality of Life Research, 25(6), 1361–1383.CrossRefPubMedGoogle Scholar
  29. 29.
    Vanier, A., Sebille, V., Blanchin, M., Guilleaux, A., & Hardouin, J. (2015). Overall performance of Oort’s procedure for response shift detection at item-level. Quality of Life Research, 24(8), 1799–1807.CrossRefPubMedGoogle Scholar
  30. 30.
    Ahmed, S., Sawatzky, R., Levesque, J. F., Ehrmann-Feldman, D., & Schwartz, C. E. (2014). Minimal evidence of response shift in the absence of a catalyst. Quality of Life Research, 23(9), 2421–2430.CrossRefPubMedGoogle Scholar
  31. 31.
    Joyce, C. R. B., O’Boyle, C., & McGee, H. (1999). Individual quality of life: Approaches to conceptualization and assessment. Amsterdam: Harwood Academic Publishers.Google Scholar
  32. 32.
    Schwartz, C. E., & Sprangers, M. A. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48(11), 1531–1548.CrossRefPubMedGoogle Scholar
  33. 33.
    Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, R. S. (2005). Using the patient generated index to evaluate response shift post-stroke. Quality of Life Research, 14(10), 2247–2257.CrossRefPubMedGoogle Scholar
  34. 34.
    Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, S. R. (2005). The structural equation modeling technique did not show a response shift, contrary to the results of the then test and the individualized approaches. Journal of Clinical Epidemiology, 58(11), 1125–1133.CrossRefPubMedGoogle Scholar
  35. 35.
    Visser, M. R., Oort, F. J., & Sprangers, M. A. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14(3), 629–639.CrossRefPubMedGoogle Scholar
  36. 36.
    Mayo, N. E., Scott, S. C., Bernstein, C. N., & Lix, L. M. (2015). How are you? Do people with inflammatory bowel disease experience response shift on this question? Health Qual Life Outcomes, 13, 52.CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Lix, L. M., Chan, E. K. H., Sawatzky, R., Sajobi, T. T., Liu, J., Hopman, W., et al. (2016). Response shift and disease activity in inflammatory bowel disease. Quality of Life Research, 25(7), 1751–1760.CrossRefPubMedGoogle Scholar
  38. 38.
    Schwartz, C. E., Ahmed, S., Sawatzky, R., Sajobi, T., Mayo, N., Finkelstein, J., et al. (2013). Guidelines for secondary analysis in search of response shift. Quality of Life Research, 22(10), 2663–2673.CrossRefPubMedGoogle Scholar
  39. 39.
    Ware, J. E. Jr., Kosinski, M., & Keller, S. D. (1994). SF-36 physical and mental scales: A user’s manual. Boston: The Health Institute, New England Medical Center.Google Scholar
  40. 40.
    Herdman, M., Gudex, C., Lloyd, A., Jansen, M., Kind, P., Parkin, D., et al. (2011). Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L. Quality of Life Research, 20(10), 1727–1736.CrossRefPubMedPubMedCentralGoogle Scholar
  41. 41.
    Beaumont, J. L., Lix, L. M., Yost, K. J., & Hahn, E. A. (2006). Application of robust statistical methods for sensitivity analysis of health-related quality of life outcomes. Quality of Life Research, 15(3), 349–356.CrossRefPubMedGoogle Scholar
  42. 42.
    Grewal, R., Cote, J. A., & Baumgartner, H. (2004). Multicollinearity and measurement error in structural equation models: Implications for theory testing. Marketing Science, 23(4), 519–529.CrossRefGoogle Scholar
  43. 43.
    Muthén, B., & Kaplan, D. (1985). A comparison of methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38(1), 171–189.CrossRefGoogle Scholar
  44. 44.
    Schwartz, C. E., Sajobi, T. T., Verdam, M. G., Sebille, V., Lix, L. M., Guilleux, A., et al. (2015). Method variation in the impact of missing data on response shift detection. Quality of Life Research, 24(3), 521–528.CrossRefPubMedGoogle Scholar
  45. 45.
    Sajobi, T. T., Lix, L. M., Singh, G., Lowerison, M., Engbers, J., & Mayo, N. E. (2015). Identifying reprioritization response shift in a stroke caregiver population: A comparison of missing data methods. Quality of Life Research, 24(3), 529–540.CrossRefPubMedGoogle Scholar
  46. 46.
    Verdam, M. G., Oort, F. J., van der Linden, Y. M., & Sprangers, M. A. (2015). Taking into account the impact of attrition on the assessment of response shift and true change: A multigroup structural equation modeling approach. Quality of Life Research, 24(3), 541–551.CrossRefPubMedGoogle Scholar
  47. 47.
    Blanchin, M., Sebille, V., Guilleux, A., & Hardouin, J. (2016). The Guttman errors as a tool for response shift detection at subgroup and item levels. Quality of Life Research, 25(6), 1385–1393.CrossRefPubMedGoogle Scholar
  48. 48.
    Schwartz, C. E. (2016). Introduction to special section on response shift at the item level. Quality of Life Research, 25(6), 1323–1325.CrossRefPubMedGoogle Scholar
  49. 49.
    Verdam, M. G. E., Oort, F. J., & Sprangers, M. A. G. (2017). Structural equation modeling-based effect-size indices were used to evaluate and interpret the impact of response shift effects. Journal of Clinical Epidemiology, 85, 37–44.CrossRefPubMedGoogle Scholar
  50. 50.
    Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  51. 51.
    Jobe, J. B. (2003). Cognitive psychology and self-reports: Models and methods. Quality of Life Research, 12, 219–227.CrossRefPubMedGoogle Scholar
  52. 52.
    Sawatzky, R., Chan, E. K. H., Zumbo, B. D., Ahmed, S., Bartlett, S. J., Bingham, C. O., et al. (2016). Challenges and opportunities in patient-reported outcomes validation. Journal of Clinical Epidemiology. Scholar
  53. 53.
    Hubley, A. M., & Zumbo, B. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103(2), 219–230.CrossRefGoogle Scholar
  54. 54.
    Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.CrossRefGoogle Scholar
  55. 55.
    Zumbo, B. D., & Chan, E. K. H. (2014). Reflections on validation practices in the social, behavioral, and health sciences. In B. D. Zumbo & E. K. H. Chan (Eds.), Validity and validation in social, behavioral, and health sciences (Vol. 54, pp. 321–327). New York: Springer International Publishing.Google Scholar
  56. 56.
    von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P., STROBE Initiative. (2008). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Journal of Clinical Epidemiology, 61(4), 344–349.CrossRefGoogle Scholar
  57. 57.
    Schulz, K. F., Altman, D. G., & Moher, D., for the CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. Annals of Internal Medicine, 152(11), 726–732.CrossRefPubMedGoogle Scholar
  58. 58.
    StataCorp. (2015). Stata Statistical Software: Release 14. College Station, TX: StataCorp LP.Google Scholar
  59. 59.
    SAS Institute Inc. (2014). SAS/STAT version 9.4. Cary, NC: AS Institute Inc.Google Scholar
  60. 60.
    R Core Team. (2012). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
  61. 61.
    Muthén, L. K., & Muthén, B. O. (2011). Mplus user’s guide (6th Edn.). Los Angeles, CA: Muthén & Muthén.Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2017

Authors and Affiliations

  • Tolulope T. Sajobi
    • 1
  • Ronak Brahmbatt
    • 2
  • Lisa M. Lix
    • 3
  • Bruno D. Zumbo
    • 4
  • Richard Sawatzky
    • 2
    • 5
  1. 1.Department of Community Health Sciences & O’Brien Institute for Public Health, Cumming School of MedicineUniversity of CalgaryCalgaryCanada
  2. 2.School of NursingTrinity Western UniversityLangleyCanada
  3. 3.Department of Community Health SciencesUniversity of ManitobaWinnipegCanada
  4. 4.Department of Educational and Counselling Psychology, and Special EducationUniversity of British ColumbiaVancouverCanada
  5. 5.Centre for Health Evaluation and Outcome Sciences, Providence Health CareVancouverCanada

Personalised recommendations