Methods to detect response shift in quality of life data: A convergent validity study Article DOI:
Cite this article as: Visser, M.R.M., Oort, F.J. & Sprangers, M.A.G. Qual Life Res (2005) 14: 629. doi:10.1007/s11136-004-2577-x Abstract
When measuring changes in quality of life (QL) with a pretest-posttest design, response shift can affect results. We investigated the convergent validity of three approaches to detect response shift. (1) In the thentest approach, response shift is measured using a retrospective judgment of pretest QL-levels (thentest). (2) In the anchor–recalibration approach response shift is measured, assessing shifts in patients’ individual definitions of the scale-anchors (worst and best imaginable QL) over time. (3) In the Structural Equation Modeling (SEM) approach response shift is indicated by mathematically defined changes in factor solutions and variance–covariance matrices over time. Prior to and three months after invasive surgery, 170 cancer patients completed the SF-36, the Multidimensional Fatigue Inventory (as pre-, post-, and thentest), and the anchor–recalibration task (as pre-, and posttest). Results showed agreement between the thentest and SEM approach on the absence (6 scales) and presence (2 scales) of response shift in 8 of the 9 scales. For the ninth scale both methods detected response shift, but in opposite directions. Possible explanations for this discrepancy are discussed. The anchor–recalibration task agreed with the other approaches on only the absence of response shift in 4 of the 7 scales. The convergent results of thentest and SEM support their validity, especially because they use statistically independent operationalizations of response shift. In this study, recall bias did not invalidate thentest results.
Keywords Quality of life Response shift Structural equation modeling Thentest Validity Abbreviations QL
quality of life
Structural Equation Modeling
Medical Outcomes Study 36 item Short Form health Survey
Multi-dimensional Fatigue Inventory
Fatigue (as measured with MFI)
References Sprangers, MAG, Schwartz, CE 1999 Integrating response shift into health-related quality of life research: A theoretical model. Soc Sci Med 48 1507 1515 Google Scholar Schwartz, CE, Sprangers, MAG 1999 Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Soc Sci Med 48 1507 1515 Google Scholar Howard, GS, Ralph, KM, Gulanick, NA, Maxwell, SE, Nance, SW, Gerber, SK 1979 Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Appl Psych Meas 3 1 23 Google Scholar Zmud, R, Armenakis, AA 1978 Understanding the measurement of change. Acad Manage Rev 3 661 669 Google Scholar Oort, FJ, Visser, MRM, Sprangers, MAG 2005 An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Qual Life Res 14 599 609 Google Scholar Oort, FJ 2005 Using structural equation modeling to detect response shifts and true change. Qual Life Res 14 587 598 Google Scholar Bernhard, J, Hurny, C, Maibach, R, Herrmann, R, Laffer, U 1999 Quality of life as subjective experience: Reframing of perception in patients with colon cancer undergoing radical resection with or without adjuvant chemotherapy. Ann Oncol 10 775 782 Google Scholar Jansen, JT, Stiggelbout, AM, Nooij, MA, Noordijk, EM, Kievit, J 2000 Response Shift in quality of life measurement in early-stage breast cancer patients undergoing radiotherapy. Qual Life Res 9 603 615 Google Scholar Lepore, S, Eton, D 2000 Response shifts in prostate cancer patients: An evaluation of suppressor and buffer Models. Schwartz, CE Sprangers, MAG eds. Adaptation to Changing Health: Response Shift in Quality of Life Research. American Psychological Association Washington 37 51 Google Scholar Sprangers, MAG, Dam, FSA, Broersen, J, Lodder, L, Wever, L, Visser, MRM, Oosterveld, P, Smets, EMA 1999 Revealing response shift in longitudinal research on fatigue. Acta Oncol 38 709 718 CrossRef PubMed Google Scholar Visser, MRM, Smets, EMA, Sprangers, MAG, Haes, JCJM 2000 How response shift may affect the measurement of change in fatigue. J Pain Symptom Manag 20 12 18 Google Scholar Norman, GR 2003 Hi! How are you? Response shift, implicit theories and differing epistemologies. Qual Life Res 12 371 383 Google Scholar Granier, MJ, Green, SB, Armenakis, AA 1991 An experimental approach to evaluate methods to detect scale recalibration. Educ Psychol Meas 51 597 607 Google Scholar Bernheim, JL, Buyse, M 1984 The Anamnestic Comparative Self-Assesment for Measuring the Subjective Quality of Life of Cancer Patients. J Psychosoc Oncol 1 25 38 Google Scholar Golembiewski, R, Billingsley, K, Yeager, S 1976 Measuring change and persistence in human affairs: Types of change generated by OD designs. J Appl Behav Sci 12 133 157 Google Scholar Armenakis, A, Feild, H, Wilmoth, J 1977 An algorithm for assessing factor structure congruence. Educ Psychol Meas 37 213 214 Google Scholar Schmitt, N 1982 The use of analysis of covariance structures too assess beta and gamma change. Multivar Behav Res 17 343 358 Google Scholar Schmitt, N, Pulakos, ED, Lieblein, A 1984 Comparison of three techniques to assess group-level beta and gamma change. Appl Psych Meas 8 249 260 Google Scholar
Armenakis AA, Randolph WA, Bedeian A. A comparison of two methods for evaluating the similarity of factor analytic solutions. Dallas: Proceedings of the Southwest Academy of Management Meeting, 1982: 138–142.
Sprangers, MAG Moinpour, CM Moynihan, TJ Patrick, DL Revicki, DA Clinical Significance Consensus Group 2002 Assessing meaningful change in quality of life over time: A users’ guide for clinicians. Mayo Clin Proc 77 561 571 Google Scholar Ware, JE, Snow, KK, Kosinski, M, Gandek, B 1993SF-36 Health Survey: Manual and Interpretation Guide. Nimrod Press Boston, Massachusetts Google Scholar Aaronson, N, Fayers, P 2002 Quality of life Souhami, RL Tannock, I Hohenberger, P Horiot, J eds. Oxford Textbook of Oncology Oxford University Press Oxford 1061 1078 Google Scholar Aaronson, NK, Muller, M, Cohen, PDA, Essink-Bot, M, Fekkes, M, Sanderman, R, Sprangers, MAG, te Velde, A, Verrips, E 1998 Translation, validation, and norming of the Dutch language version of the SF-36 health survey in community and chronic disease populations. J Clin Epidemiol 51 1055 1068 Google Scholar Smets, EMA, Garssen, B, Bonke, B, Haes, JCJM 1995 The Multidimensional Fatigue Inventory (MFI): psychometric qualities of an instrument to assess fatigue. J Psychosom Res 39 315 325 CrossRef PubMed Google Scholar Jöreskog, KG, Sörbom, D 1998Lisrel 8 user’s guide2 Scientific Software International, Inc Chigago, IL Google Scholar Cohen, J 1988Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates Hillsdale, New Jersey Google Scholar MacCallum, RC, Browne, MW, Sugawara, HM 1996 Power analysis and determination of sample size for covariance structure modeling. Psychol Methods 2 130 149 Google Scholar Schwartz, CE, Sprangers, MAG, Carey, A, Reed, G 2004 Exploring response shift in longitudinal data. Psychol Health 19 51 69 Google Scholar
Oort FJ, Visser MRM, Sprangers MAG. Incorporating the thentest into the structural equation modeling (SEM) approach to detect response shift. Poster presented at the conference of the International Society of Quality of Life Research (ISOQOL), Prague, Czech Republic, 2003.