Skip to main content

Advertisement

SpringerLink
Log in

Guidelines for improving the stringency of response shift research using the thentest

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Objective

Health-state changes can lead to response shifts in internal standards. The most commonly used method for detecting such recalibration response shift is the retrospective pretest–posttest design, here referred to as thentest. Since this design faces significant problems, there is a pressing need to improve the stringency of studies using the thentest approach. Our objective is to provide guidelines for the optimal use of the thentest approach for detecting recalibration response shift.

Methods

Discussion of methods based on relevant literature.

Results

A checklist is provided that includes recommendations for studies using the thentest approach focusing on: (1) designing the study; (2) formulating hypotheses; (3) constructing the thentest; (4) identifying change; (6) taking alternative explanations into account; (7) using analytic standards; and (8) interpreting results.

Conclusions

The guidelines-checklist has the potential to stimulate rigorous and replicable research using the thentest. This checklist might also be of use for journal editors and reviewers as ‘gate keepers’ of stringent research. Many of these suggestions also apply to other methods of detecting response shift.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.

    Article  CAS  PubMed  Google Scholar 

  2. Schwartz, C. E., & Sprangers, M. A. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48(11), 1531–1548.

    Article  CAS  PubMed  Google Scholar 

  3. Schwartz, C. E., Repucci, N., Becker, J., Sprangers, M. A. G., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15, 1533–1550.

    Article  PubMed  Google Scholar 

  4. Barclay-Goddard, R., & Epstein, J. D. (2009). Response shift: A brief overview and proposed research priorities. Quality of Life Research, 18, 335–346.

    Article  PubMed  Google Scholar 

  5. Sprangers, M. A., Van Dam, F. S., Broersen, J., Lodder, L., Wever, L., Visser, M. R., et al. (1999). Revealing response shift in longitudinal research on fatigue–the use of the thentest approach. Acta Oncologica, 38(6), 709–718.

    Article  CAS  PubMed  Google Scholar 

  6. Andrykowski, M. A., Donovan, K. A., & Jacobsen, P. B. (2009). Magnitude and correlates of response shift in fatigue ratings in women undergoing adjuvant therapy for breast cancer. Journal of Pain and Symptom Management, 37(3), 341–351.

    Article  PubMed  Google Scholar 

  7. Bernhard, J., Hurny, C., Maibach, R., Herrmann, R., & Laffer, U. (1999). Quality of life as subjective experience: Reframing of perception in patients with colon cancer undergoing radical resection with or without adjuvant chemotherapy. Swiss Group for Clinical Cancer Research (SAKK). Annals of Oncology, 10(7), 775–782.

    Article  CAS  PubMed  Google Scholar 

  8. Bernhard, J., Lowy, A., Maibach, R., & Hurny, C. (2001). Response shift in the perception of health for utility evaluation. An explorative investigation. European Journal of Cancer, 37(14), 1729–1735.

    Article  CAS  PubMed  Google Scholar 

  9. Broberger, E., Sprangers, M. A. G., & Tishelman, C. (2006). Do internal standards of quality of life change in lung cancer patients? Nursing Research, 55(4), 274–282.

    Article  PubMed  Google Scholar 

  10. Jansen, S. J., Stiggelbout, A. M., Nooij, M. A., Noordijk, E. M., & Kievit, J. (2000). Response shift in quality of life measurement in early-stage breast cancer patients undergoing radiotherapy. Quality of Life Research, 9(6), 603–615.

    Article  CAS  PubMed  Google Scholar 

  11. Korfage, I. J., de Koning, H. J., & Essink-Bot, M. L. (2007). Response shift due to diagnosis and primary treatment of localized prostate cancer: A then-test and a vignette study. Quality of Life Research, 16, 1627–1634.

    Article  PubMed  Google Scholar 

  12. Rees, J., Waldron, D., O’Boyle, C., Ewings, P., & MacDonagh, R. (2003). Prospective vs retrospective assessment of lower urinary tract symptoms in patients with advanced prostate cancer: The effect of ‘response shift’. BJU International, 92(7), 703–706.

    Article  CAS  PubMed  Google Scholar 

  13. Rees, J., Waldron, D., O’Boyle, C., Ewings, P., & MacDonagh, R. (2005). The measurement of response shift in patients with advanced prostate cancer and their partners. Health and Quality of Life Outcomes, 3(21), 1–8.

    Google Scholar 

  14. Sprangers, M. A. G. (1996). Response-shift bias: A challenge to the assessment of patients’ quality of life in cancer clinical trials. Cancer Treatment Reviews, 22(Suppl. A), 55–62.

    Article  PubMed  Google Scholar 

  15. Visser, M. R., Smets, E. M., Sprangers, M. A., & de Haes, H. J. (2000). How response shift may affect the measurement of change in fatigue. Journal of Pain and Symptom Management, 20(1), 12–18.

    Article  CAS  PubMed  Google Scholar 

  16. Visser, M. R. M., Oort, F. J., & Sprangers, M. A. G. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14, 629–639.

    Article  PubMed  Google Scholar 

  17. Schwartz, C. E., Sprangers, M. A. G., Carey, A., & Reed, G. (2004). Exploring response shift in longitudinal data. Psychology and Health, 19(1), 51–69.

    Article  Google Scholar 

  18. Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, S. R. (2004). Response shift influenced estimates of change in health-related quality of life poststroke. Journal of Clinical Epidemiology, 57(6), 561–570.

    Article  PubMed  Google Scholar 

  19. Ahmed, S., Mayo, N. E., Wood-Dauphinee, S., Hanley, J. A., & Cohen, S. R. (2005). The structural equation modeling technique did not show a response shift, contrary to the results of the then test and the individualized approaches. Journal of Clinical Epidemiology, 58(11), 1125–1133.

    Article  PubMed  Google Scholar 

  20. Wagner, J. A. (2005). Response shift and glycemic control in children with diabetes. Health and Quality of Life Outcomes, 3(1), 38.

    Article  PubMed  Google Scholar 

  21. Nieuwkerk, P. T., Tollenaar, M. S., Oort, F. J., & Sprangers, M. A. G. (2007). Are retrospective measures of change in quality of life more valid than prospective measures? Medical Care, 45, 199–205.

    Article  PubMed  Google Scholar 

  22. Joore, M. A., Potjewijd, J., Timmerman, A. A., & Anteunis, L. J. (2002). Response shift in the measurement of quality of life in hearing impaired adults after hearing aid fitting. Quality of Life Research, 11(4), 299–307.

    Article  CAS  PubMed  Google Scholar 

  23. Chin, K., Fukuhara, S., Takahashi, K., Sumi, K., Nakamura, T., Matsumoto, H., et al. (2004). Response shift in perception of sleepiness in obstructive sleep apnea-hypopnea syndrome before and after treatment with nasal CPAP. Sleep, 27(3), 490–493.

    PubMed  Google Scholar 

  24. Kidd, P., Parshall, M. B., Wojcik, S., & Struttmann, T. (2004). Assessing recalibration as a response-shift phenomenon. Nursing Research, 53(2), 130–135.

    Article  PubMed  Google Scholar 

  25. Ring, L., Hofer, S., Heuston, F., Harris, D., & O’Boyle, C. A. (2005). Response shift masks the treatment impact on patient reported outcomes (PROs): The example of individual quality of life in edentulous patients. Health and Quality of Life Outcomes, 3, 55.

    Article  PubMed  Google Scholar 

  26. Kubota, Y., Yoneda, K., Nakai, K., Katsuura, J., Moriue, T., Matsuoka, Y., et al. (2009). Effect of sequential applications of topical tacrolimus and topical corticosteroids in the treatment of pediatric atopic dermatitis: An open-label pilot study. Journal of the American Academy of Dermatology, 60(2), 212–217.

    Article  PubMed  Google Scholar 

  27. Razmjou, H., Schwartz, C. E., Yee, A., & Finkelstein, J. A. (2009). Traditional assessment of health outcome following total knee arthroplasty was confounded by response shift phenomenon. Journal of Clinical Epidemiology, 62, 91–96.

    Article  PubMed  Google Scholar 

  28. Razmjou, H., Yee, A., Ford, M., & Finkelstein, J. A. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. The Journal of Bone and Joint Surgery, 88(12), 2590–2595.

    Article  PubMed  Google Scholar 

  29. Gillison, F., Skevington, S., & Standage, M. (2008). Exploring response shift in the quality of life of healthy adolescents over 1 year. Quality of Life Research, 17, 997–1008.

    Article  PubMed  Google Scholar 

  30. Timmerman, A. A., Anteunis, L. J., & Meesters, C. M. (2003). Response-shift bias and parent-reported quality of life in children with otitis media. Archives of Otolaryngology—Head & Neck Surgery, 129(9), 987–991.

    Article  Google Scholar 

  31. Schwartz, C. E., Wheeler, H. B., Hammes, B., Basque, N., Edmunds, J., Reed, G., et al. (2002). Early intervention in planning end-of-life care with ambulatory geriatric patients: Results of a pilot trial. Archives of Internal Medicine, 162(14), 1611–1618.

    Article  PubMed  Google Scholar 

  32. Sprangers, M., & Hoogstraten, J. (1988). Pretesting effects in retrospective pretest-posttest designs, in Response shift and the retrospective pretest: On the usefulness of retrospective pretest-posttest designs in detecting training related response shifts, het Instituut voor Onderzoek van het Onderwijs S.V.O.: Gravenhage. 27–44.

  33. Howard, G. S., Millham, J., Slaten, S., & O’Donnell, L. (1981). Influence of subject response style effects on retrospective measures. Applied Psychological Measurement, 5, 89–100.

    Article  Google Scholar 

  34. Howard, G. S., & Dailey, P. R. (1979). Response shift bias: A source of contamination of self-report measures. Journal of Applied Psychology, 64(2), 144–150.

    Article  Google Scholar 

  35. Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, D. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychology Measurement, 3(1), 1–23.

    Article  Google Scholar 

  36. Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2009). A test of measurement invariance fails to support the application of then-test questions as a remedy to response shift bias. Journal of Clinical Epidemiology, 62, 1173–1180.

    Article  PubMed  Google Scholar 

  37. Ahmed, S., Mayo, N. E., Corbiere, M., Wood-Dauphinee, S., Hanley, J., & Cohen, R. (2005). Change in quality of life in people with stroke over time: True change or response shift? Quality of Life Research, 14, 611–627.

    Article  PubMed  Google Scholar 

  38. Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2(1), 14.

    Article  PubMed  Google Scholar 

  39. Howard, G. S., & Dailey, P. R. (1979). Response-shift bias: A source of contamination of self-report measures. The Journal of Applied Psychology, 64, 144–150.

    Article  Google Scholar 

  40. Norman, G. (2003). Hi! How are you? Response shift, implicit theories and differing epistemologies. Quality of Life Research, 12(3), 239–249.

    Article  PubMed  Google Scholar 

  41. Schwartz, C. E., & Rapkin, B. D. (2004). Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health and Quality of Life Outcomes, 2, 16.

    Article  PubMed  Google Scholar 

  42. Sprangers, M. (1989). Subject bias and the retrospective pretest in retrospect. Bulletin of the Psychonomic Society, 27(1), 11–14.

    Google Scholar 

  43. Sprangers, M., & Hoogstraten, J. (1989). Pretesting effects in retrospective pretest-posttest designs. Journal of Applied Psychology, 74(2), 265–272.

    Article  Google Scholar 

  44. Finkelstein, J. A., Razmjou, H., & Schwartz, C. E. (2009). Response shift and outcome assessment in orthopedic surgery: Is there is a difference between complete vs. partial treatment? Journal of Clinical Epidemiology, 82, 1189–1190.

    Article  Google Scholar 

  45. Collins, L. M., Graham, J. W., Hansen, W. B., & Johnson, A. C. (1985). Agreement between retrospective accounts of substance use and ealier reported substance use. Applied Psychological Measurement, 9, 301–309.

    Article  Google Scholar 

  46. Ross, M. (1989). Relation of implicit theories to the construction of personal histories. Psychological Review, 96(2), 341–357.

    Article  Google Scholar 

  47. Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.

    Article  PubMed  Google Scholar 

  48. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences Hillsdale. NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  49. DeVet, H. C., Terwee, C. B., Ostelo, R. W., Beckerman, H., Knol, D. L., & Bouter, L. M. (2006). Minimal changes in health status questionnaires: Distinction between minimally detectable change and minimally important change. Health & Quality of Life Outcomes, 4, 54.

    Article  Google Scholar 

  50. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. New York: Cambridge University Press.

    Google Scholar 

  51. Bloem, E. F., van Zuuren, F. J., Koeneman, M. A., Rapkin, B. D., Visser, M. R. M., Koning, C. C. E., et al. (2008). Clarifying quality of life assessment: Do theoretical models capture the underlying cognitive processes? Quality of Life Research, 17, 1093–1102.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

We are indebted to Elsbeth Bloem for conducting a literature review to identify thentest papers; to Dr. Bruce Rapkin for helpful discussions on earlier drafts of this manuscript; and to anonymous reviewers for helpful comments on the manuscript.

Author information

Authors and Affiliations

  1. DeltaQuest Foundation, Inc., Concord, MA, USA

    Carolyn E. Schwartz

  2. Departments of Medicine and Orthopaedic Surgery, Tufts University School of Medicine, Boston, MA, USA

    Carolyn E. Schwartz

  3. Department of Medical Psychology, Academic Medical Center, University of Amsterdam, Meibergdreef 15, 1105 AZ, Amsterdam, The Netherlands

    Mirjam A. G. Sprangers

Authors
  1. Carolyn E. Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mirjam A. G. Sprangers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carolyn E. Schwartz.

Additional information

Carolyn E. Schwartz and Mirjam A. G. Sprangers contributed equally to this work.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Schwartz, C.E., Sprangers, M.A.G. Guidelines for improving the stringency of response shift research using the thentest. Qual Life Res 19, 455–464 (2010). https://doi.org/10.1007/s11136-010-9585-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-010-9585-9

Keywords

Search

Navigation