Skip to main content

Advertisement

SpringerLink
Log in

Scoping review of response shift methods: current reporting practices and recommendations

  • Review
  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Background

Response shift (RS) has been defined as a change in the meaning of an individual’s self-evaluation of his/her health status and quality of life. Several statistical model- and design-based methods have been developed to test for RS in longitudinal data. We reviewed the uptake of these methods in patient-reported outcomes (PRO) literature.

Methods

CINHAHL, EMBASE, Medline, ProQuest, PsycINFO, and Web of Science were searched to identify English-language articles about RS published until 2016. Data on year and country of publication, PRO measure adopted, RS detection method, type of RS detected, and testing of underlying model assumptions were extracted from the included articles.

Results

Of the 1032 articles identified, 101 (9.8%) articles were included in the study. While 54.5 of the articles reported on the Then-test, 30.7% of the articles reported on Oort’s or Schmitt’s structural equation modeling (SEM) procedure. Newer RS detection methods, such as relative importance analysis and random forest regression, have been used less frequently. Less than 25% reported on testing the assumptions underlying the adopted RS detection method(s).

Conclusions

Despite rapid methodological advancements in RS research, this review highlights the need for further research about RS detection methods for complex longitudinal data and standardized reporting guidelines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Revicki, D. A. (1989). Health-related quality of life in the evaluation of medical therapy for chronic illness. Journal of Family Practice, 29(4), 377–380.

    CAS  PubMed  Google Scholar 

  2. Berzon, R., Hays, R. D., & Shumaker, S. A. (1993). International use, application and performance of health-related quality of life instruments. Quality of Life Research, 2(6), 367–368.

    Article  CAS  PubMed  Google Scholar 

  3. Deshpande, P. R., Rajan, S., Sudeepthi, B. L., & Abdul Nazir, C. P. (2011). Patient-reported outcomes: A new era in clinical research. Perspectives in Clinical Research, 2(4), 137–144.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Golembiewski, R. T., Billingsley, K., & Yeager, S. (1976). Measuring change and persistence in human affairs: Types of change generated by OD designs. The Journal of Applied Behavioral Science, 12, 133–157.

    Article  Google Scholar 

  5. Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, S. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and reevaluation of retrospective pretests. Applied Psychological Measurement, 3, 1–23.

    Article  Google Scholar 

  6. Breetvelt, I. S., & Van Dam, F. S. (1991). Underreporting by cancer patients: the case of response-shift. Social Science and Medicine, 32, 981–987.

    Article  CAS  PubMed  Google Scholar 

  7. Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: a theoretical model. Social Science and Medicine, 48(11), 1507–1515.

    Article  CAS  PubMed  Google Scholar 

  8. Taminiau-Bloem, E. F., van Zuuren, F. J., Visser, M. R. M., Tishelman, C., Schwartz, C. E., Koeneman, M. A., et al. (2011) Opening the black box of cancer patients’ quality-of-life change assessments: a think-aloud study examining the cognitive processes underlying responses to transition items. Psychol Health, 26(11), 1414–1428.

    Article  PubMed  Google Scholar 

  9. Schwartz, C. E., Andresen, E. M., Nosek, M. A., & Krahn, G. L. & RRCT Expert Panel on Health Status Measurement. (2007). Response shift theory: Important implications for measuring quality of life in people with disability. Archives of Physical Medicine and Rehabilitation, 88(4), 529–536.

    Article  PubMed  Google Scholar 

  10. Schwartz, C. E. (2010). Application of response shift theory and methods to participation measurement: A brief history of a young field. Archives of Physical Medicine and Rehabilitation, 91(9), S38–S43.

    Article  PubMed  Google Scholar 

  11. Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health Qual Life Outcomes, 2, 14.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Schwartz, C. E., Bode, R., Repucci, N., Becker, J., Sprangers, M. A., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15(9), 1533–1550.

    Article  PubMed  Google Scholar 

  13. Sawatzky, R., Sajobi, T. T., Brahmbhatt, R., Chan, E. K. H., Lix, L. M., & Zumbo, B. D. (2017). Longitudinal change in response processes: A response shift perspective. In B. D. Zumbo & A. M. Hubley (Eds.), Understanding and investigating response processes in validation research (pp. 251–276). New York: Springer.

    Chapter  Google Scholar 

  14. Balanin, B., Ennis, O., Kanes, G., Siinghal, R., Roberts, S. N., Rees, D., et al. (2009). Response shift in self-reported functional scores after knee microfracture for full thickness cartilage lesions. Osteoarthritis and Cartilage, 17(8), 1009–1013.

    Article  Google Scholar 

  15. Sajobi, T. T., Fiest, K. M., & Wiebe, S. (2014). Changes in quality of life after epilepsy surgery: the role of reprioritization response shift. Epilepsia, 55(9), 1331–1338.

    Article  PubMed  Google Scholar 

  16. Vanier, A., Falissard, B., Sébille, V., & Hardouin, J. B. (2017). The complexity of interpreting changes observed over time in health-related quality of life: A short overview of 15 years of research on response shift theory. In F. Guillemin, A. Leplège, S. Briançon, E. Spitz & J. Coste (Eds.), Perceived health and adaptation in chronic disease. Stakes and future challenge (pp. 202–230). New York, NY: CRC Press.

    Google Scholar 

  17. Bray, J. H., Maxwell, S. E., & Howard, G. S. (1984). Methods of analysis with response-shift bias. Educational and Psychological Measurement, 44(4), 781–804.

    Article  Google Scholar 

  18. Schmitt, N. (1982). The use of analysis of covariance structures to assess beta and gamma change. Multivariate Behavioral Research, 17, 343–358.

    Article  CAS  PubMed  Google Scholar 

  19. Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14(3), 587–598.

    Article  PubMed  Google Scholar 

  20. Oort, F. J., Visser, M. R. M., & Sprangers, M. A. G. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14(3), 599–609.

    Article  PubMed  Google Scholar 

  21. Anota, A., Bascoul-Mollevi, C., Conroy, T., Guillemin, F., Velten, M., Jolly, D., et al. (2014). Item response theory and factor analysis as a mean to characterize occurrence of response shift in a longitudinal quality of life study in breast cancer patients. Health Qual Life Outcomes, 12, 32.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2015). RespOnse Shift ALgorithm in Item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies. Quality of Life Research, 24(3), 553–564.

    Article  PubMed  Google Scholar 

  23. Lowy, A., & Bernhard, J. (2004). Quantitative assessment of changes in patients’ constructs of quality of life: an application of multilevel models. Quality of Life Research, 13(7), 1177–1185.

    Article  PubMed  Google Scholar 

  24. Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17(4), 627–639.

    Article  PubMed  Google Scholar 

  25. Lix, L. M., Sajobi, T. T., Sawatzky, R., Liu, J., Mayo, N. E., Huang, Y., et al. (2013). Relative importance measures for reprioritization response shift. Quality of Life Research, 22(4), 695–703.

    Article  PubMed  Google Scholar 

  26. Li, Y., & Rapkin, B. (2009). Classification and regression tree uncovered hierarchy of psychosocial determinants underlying quality-of-life response shift in HIV/AIDS. Journal of Clinical Epidemiology, 62(11), 1138–1147.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., et al. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: a cohort study. BMC Medical Research Methodology, 13, 20.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Verdam, M. G. E., Oort, F. J., & Sprangers, M. A. G. (2016). Using structural equation modeling to detect response shifts and true change in discrete variables: an application to the items of the SF-36. Quality of Life Research, 25(6), 1361–1383.

    Article  PubMed  Google Scholar 

  29. Vanier, A., Sebille, V., Blanchin, M., Guilleaux, A., & Hardouin, J. (2015). Overall performance of Oort’s procedure for response shift detection at item-level. Quality of Life Research, 24(8), 1799–1807.

    Article  PubMed  Google Scholar 

  30. Ahmed, S., Sawatzky, R., Levesque, J. F., Ehrmann-Feldman, D., & Schwartz, C. E. (2014). Minimal evidence of response shift in the absence of a catalyst. Quality of Life Research, 23(9), 2421–2430.

    Article  PubMed  Google Scholar 

  31. Joyce, C. R. B., O’Boyle, C., & McGee, H. (1999). Individual quality of life: Approaches to conceptualization and assessment. Amsterdam: Harwood Academic Publishers.

    Google Scholar 

  32. Schwartz, C. E., & Sprangers, M. A. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48(11), 1531–1548.

    Article  CAS  PubMed  Google Scholar 

  33. Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, R. S. (2005). Using the patient generated index to evaluate response shift post-stroke. Quality of Life Research, 14(10), 2247–2257.

    Article  PubMed  Google Scholar 

  34. Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, S. R. (2005). The structural equation modeling technique did not show a response shift, contrary to the results of the then test and the individualized approaches. Journal of Clinical Epidemiology, 58(11), 1125–1133.

    Article  PubMed  Google Scholar 

  35. Visser, M. R., Oort, F. J., & Sprangers, M. A. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14(3), 629–639.

    Article  PubMed  Google Scholar 

  36. Mayo, N. E., Scott, S. C., Bernstein, C. N., & Lix, L. M. (2015). How are you? Do people with inflammatory bowel disease experience response shift on this question? Health Qual Life Outcomes, 13, 52.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Lix, L. M., Chan, E. K. H., Sawatzky, R., Sajobi, T. T., Liu, J., Hopman, W., et al. (2016). Response shift and disease activity in inflammatory bowel disease. Quality of Life Research, 25(7), 1751–1760.

    Article  PubMed  Google Scholar 

  38. Schwartz, C. E., Ahmed, S., Sawatzky, R., Sajobi, T., Mayo, N., Finkelstein, J., et al. (2013). Guidelines for secondary analysis in search of response shift. Quality of Life Research, 22(10), 2663–2673.

    Article  PubMed  Google Scholar 

  39. Ware, J. E. Jr., Kosinski, M., & Keller, S. D. (1994). SF-36 physical and mental scales: A user’s manual. Boston: The Health Institute, New England Medical Center.

    Google Scholar 

  40. Herdman, M., Gudex, C., Lloyd, A., Jansen, M., Kind, P., Parkin, D., et al. (2011). Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L. Quality of Life Research, 20(10), 1727–1736.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Beaumont, J. L., Lix, L. M., Yost, K. J., & Hahn, E. A. (2006). Application of robust statistical methods for sensitivity analysis of health-related quality of life outcomes. Quality of Life Research, 15(3), 349–356.

    Article  PubMed  Google Scholar 

  42. Grewal, R., Cote, J. A., & Baumgartner, H. (2004). Multicollinearity and measurement error in structural equation models: Implications for theory testing. Marketing Science, 23(4), 519–529.

    Article  Google Scholar 

  43. Muthén, B., & Kaplan, D. (1985). A comparison of methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38(1), 171–189.

    Article  Google Scholar 

  44. Schwartz, C. E., Sajobi, T. T., Verdam, M. G., Sebille, V., Lix, L. M., Guilleux, A., et al. (2015). Method variation in the impact of missing data on response shift detection. Quality of Life Research, 24(3), 521–528.

    Article  PubMed  Google Scholar 

  45. Sajobi, T. T., Lix, L. M., Singh, G., Lowerison, M., Engbers, J., & Mayo, N. E. (2015). Identifying reprioritization response shift in a stroke caregiver population: A comparison of missing data methods. Quality of Life Research, 24(3), 529–540.

    Article  PubMed  Google Scholar 

  46. Verdam, M. G., Oort, F. J., van der Linden, Y. M., & Sprangers, M. A. (2015). Taking into account the impact of attrition on the assessment of response shift and true change: A multigroup structural equation modeling approach. Quality of Life Research, 24(3), 541–551.

    Article  PubMed  Google Scholar 

  47. Blanchin, M., Sebille, V., Guilleux, A., & Hardouin, J. (2016). The Guttman errors as a tool for response shift detection at subgroup and item levels. Quality of Life Research, 25(6), 1385–1393.

    Article  PubMed  Google Scholar 

  48. Schwartz, C. E. (2016). Introduction to special section on response shift at the item level. Quality of Life Research, 25(6), 1323–1325.

    Article  PubMed  Google Scholar 

  49. Verdam, M. G. E., Oort, F. J., & Sprangers, M. A. G. (2017). Structural equation modeling-based effect-size indices were used to evaluate and interpret the impact of response shift effects. Journal of Clinical Epidemiology, 85, 37–44.

    Article  PubMed  Google Scholar 

  50. Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  51. Jobe, J. B. (2003). Cognitive psychology and self-reports: Models and methods. Quality of Life Research, 12, 219–227.

    Article  PubMed  Google Scholar 

  52. Sawatzky, R., Chan, E. K. H., Zumbo, B. D., Ahmed, S., Bartlett, S. J., Bingham, C. O., et al. (2016). Challenges and opportunities in patient-reported outcomes validation. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2016.12.002.

    PubMed  Google Scholar 

  53. Hubley, A. M., & Zumbo, B. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103(2), 219–230.

    Article  Google Scholar 

  54. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.

    Article  Google Scholar 

  55. Zumbo, B. D., & Chan, E. K. H. (2014). Reflections on validation practices in the social, behavioral, and health sciences. In B. D. Zumbo & E. K. H. Chan (Eds.), Validity and validation in social, behavioral, and health sciences (Vol. 54, pp. 321–327). New York: Springer International Publishing.

    Google Scholar 

  56. von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P., STROBE Initiative. (2008). The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Journal of Clinical Epidemiology, 61(4), 344–349.

    Article  Google Scholar 

  57. Schulz, K. F., Altman, D. G., & Moher, D., for the CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. Annals of Internal Medicine, 152(11), 726–732.

    Article  PubMed  Google Scholar 

  58. StataCorp. (2015). Stata Statistical Software: Release 14. College Station, TX: StataCorp LP.

    Google Scholar 

  59. SAS Institute Inc. (2014). SAS/STAT version 9.4. Cary, NC: AS Institute Inc.

    Google Scholar 

  60. R Core Team. (2012). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

    Google Scholar 

  61. Muthén, L. K., & Muthén, B. O. (2011). Mplus user’s guide (6th Edn.). Los Angeles, CA: Muthén & Muthén.

    Google Scholar 

Download references

Acknowledgements

The Canadian Institutes of Health Research provided support (Grant # MOP-142404) to Drs Sajobi, Lix, Zumbo, and Sawatzky in this research. Dr. Sajobi is supported by the O’Brien Institute for Public Health; Dr. Lix is supported by the Manitoba Research Chair; and Dr. Sawatzky holds a Canada Research Chair in Patient-Reported Outcomes at Trinity Western University, Langley, British Columbia. We are grateful for the support in conducting literature searches provided by Duncan Dixon, health sciences librarian at Trinity Western University. This research was initiated during Dr. Sajobi’s visit to Trinity Western University in 2014.

Author information

Authors and Affiliations

  1. Department of Community Health Sciences & O’Brien Institute for Public Health, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada

    Tolulope T. Sajobi

  2. School of Nursing, Trinity Western University, Langley, Canada

    Ronak Brahmbatt & Richard Sawatzky

  3. Department of Community Health Sciences, University of Manitoba, Winnipeg, Canada

    Lisa M. Lix

  4. Department of Educational and Counselling Psychology, and Special Education, University of British Columbia, Vancouver, Canada

    Bruno D. Zumbo

  5. Centre for Health Evaluation and Outcome Sciences, Providence Health Care, Vancouver, Canada

    Richard Sawatzky

Authors
  1. Tolulope T. Sajobi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ronak Brahmbatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lisa M. Lix
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bruno D. Zumbo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Sawatzky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tolulope T. Sajobi.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 57 KB)

Supplementary material 2 (XLSX 20 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sajobi, T.T., Brahmbatt, R., Lix, L.M. et al. Scoping review of response shift methods: current reporting practices and recommendations. Qual Life Res 27, 1133–1146 (2018). https://doi.org/10.1007/s11136-017-1751-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-017-1751-x

Keywords

Search

Navigation