Skip to main content

Advertisement

SpringerLink
Log in

Method variation in the impact of missing data on response shift detection

  • Response Shift methods
  • Published:
Quality of Life Research Aims and scope Submit manuscript

"The best solution to the missing data problem is not to have any."

Michael S. Lewis-Beck [1]

Abstract

Purpose

Missing data due to attrition or item non-response can result in biased estimates and loss of power in longitudinal quality-of-life (QOL) research. The impact of missing data on response shift (RS) detection is relatively unknown. This overview article synthesizes the findings of three methods tested in this special section regarding the impact of missing data patterns on RS detection in incomplete longitudinal data.

Methods

The RS detection methods investigated include: (1) Relative importance analysis to detect reprioritization RS in stroke caregivers; (2) Oort’s structural equation modeling (SEM) to detect recalibration, reprioritization, and reconceptualization RS in cancer patients; and (3) Rasch-based item-response theory-based (IRT) models as compared to SEM models to detect recalibration and reprioritization RS in hospitalized chronic disease patients. Each method dealt with missing data differently, either with imputation (1), attrition-based multi-group analysis (2), or probabilistic analysis that is robust to missingness due to the specific objectivity property (3).

Results

Relative importance analyses were sensitive to the type and amount of missing data and imputation method, with multiple imputation showing the largest RS effects. The attrition-based multi-group SEM revealed differential effects of both the changes in health-related QOL and the occurrence of response shift by attrition stratum, and enabled a more complete interpretation of findings. The IRT RS algorithm found evidence of small recalibration and reprioritization effects in General Health, whereas SEM mostly evidenced small recalibration effects. These differences may be due to differences between the two methods in handling of missing data.

Conclusions

Missing data imputation techniques result in different conclusions about the presence of reprioritization RS using the relative importance method, while the attrition-based SEM approach highlighted different recalibration and reprioritization RS effects by attrition group. The IRT analyses detected more recalibration and reprioritization RS effects than SEM, presumably due to IRT’s robustness to missing data. Future research should apply simulation techniques in order to make conclusive statements about the impacts of missing data according to the type and amount of RS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lewis-Beck, M. (2002). Series editor’s introduction. Missing data (pp. v–vi). Thousand Oaks, CA: Sage.

    Google Scholar 

  2. Becker, H., Stuifbergen, A., Rogers, S., & Timmerman, G. (2000). Goal attainment scaling to measure individual change in intervention studies. Nursing Research, 49(3), 176–180.

    Article  CAS  PubMed  Google Scholar 

  3. Ahmed, S., Mayo, N., Scott, S., Kuspinar, A., Hum, S., & Schwartz, C. E. (2011). Using latent trajectory analysis to detect response shift in general health among multiple sclerosis patients. Quality of Life Research, 20(10), 1555–1560.

    Article  PubMed  Google Scholar 

  4. King-Kallimanis, B. L., Oort, F. J., Nolte, S., Schwartz, C. E., & Sprangers, M. A. G. (2011). Using structural equation modeling to detect response shift in disability and QOL scores of multiple sclerosis patients. Quality of Life Research, 20(10), 1527–1540.

    Article  PubMed Central  PubMed  Google Scholar 

  5. Li, Y., & Schwartz, C. E. (2011). Using classification and regression tree to examine evidence of quality of life response shift in patients with multiple sclerosis. Quality of Life Research, 20(10), 1543–1553.

    Article  PubMed  Google Scholar 

  6. Schwartz, C. E. (1999). Teaching coping skills enhances quality of life more than peer support: results of a randomized trial with multiple sclerosis patients. Health Psychology, 18(3), 211–220.

    Article  CAS  PubMed  Google Scholar 

  7. Schwartz, C. E., Sprangers, M. A. G., Ahmed, S., Bode, R., Li, Y., Oort, F. J., et al. (2011). Response shift in patients with multiple sclerosis: An application of three statistical techniques. Quality of Life Research, 20(10), 1561–1572.

    Article  PubMed  Google Scholar 

  8. Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., et al. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: A cohort study. BMC Medical Research Methodology, 13, 20. [Comparative Study Randomized Controlled Trial Research Support, Non-U.S. Gov’t].

    Article  PubMed Central  PubMed  Google Scholar 

  9. Bernhard, J., Hurny, C., Maibach, R., Herrmann, R., & Laffer, U. (1999). Quality of life as subjective experience: reframing of perception in patients with colon cancer undergoing radical resection with or without adjuvant chemotherapy. Swiss Group for Clinical Cancer Research (SAKK). Annals of Oncology, 10(7), 775–782.

    Article  CAS  PubMed  Google Scholar 

  10. Bernhard, J., Lowy, A., Maibach, R., & Hurny, C. (2001). Response shift in the perception of health for utility evaluation. An explorative investigation. European Journal of Cancer, 37(14), 1729–1735.

    Article  CAS  PubMed  Google Scholar 

  11. Boyd, N. F., Sutherland, H. J., Heasman, K. Z., Tritchler, D. L., & Cummings, B. J. (1990). Whose utilities for decision analysis? Medical Decision Making, 10(1), 58–67.

    Article  CAS  PubMed  Google Scholar 

  12. Breetvelt, I. S., & Van Dam, F. S. (1991). Underreporting by cancer patients: The case of response-shift. Social Science and Medicine, 32(9), 981–987.

    Article  CAS  PubMed  Google Scholar 

  13. Cella, D., Hahn, E. A., & Dineen, K. (2002). Meaningful change in cancer-specific quality of life scores: Differences between improvement and worsening. Quality of Life Research, 11(3), 207–221.

    Article  PubMed  Google Scholar 

  14. Chapman, G. B., Elstein, A. S., Kuzel, T. M., Sharifi, R., Nadler, R. B., Andrews, A., et al. (1998). Prostate cancer patients’ utilities for health states: How it looks depends on where you stand. Medical Decision Making, 18(3), 278–286.

    Article  CAS  PubMed  Google Scholar 

  15. Hagedoorn, M., Sneeuw, K. C., & Aaronson, N. K. (2002). Changes in physical functioning and quality of life in patients with cancer: Response shift and relative evaluation of one’s condition. Journal of Clinical Epidemiology, 55(2), 176–183.

    Article  PubMed  Google Scholar 

  16. Jansen, S. J. T., Stiggelbout, A. M., Nooij, M. A., Noordijk, E. M., & Kievit, J. (2001). Response shift in quality of life measurement in early-stage breast cancer patients undergoing radiotherapy. Quality of Life Research, 9, 603–615.

    Article  Google Scholar 

  17. Kagawa-Singer, M. (1993). Redefining health: Living with cancer. Social Science and Medicine, 37(3), 295–304.

    Article  CAS  PubMed  Google Scholar 

  18. Oort, F. J., Visser, M. R. M., & Sprangers, M. A. G. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.

    Article  PubMed  Google Scholar 

  19. Rees, J., Waldron, D., O’Boyle, C., Ewings, P., & MacDonagh, R. (2005). The measurement of response shift in patients with advanced prostate cancer and their partners. Health and Quality of Life Outcomes, 3(21), 1–8.

    Google Scholar 

  20. Schwartz, C. E., Feinberg, R. G., Jilinskaia, E., & Applegate, J. C. (1999). An evaluation of a psychosocial intervention for survivors of childhood cancer: Paradoxical effects of response shift over time. Psychooncology, 8(4), 344–354.

    Article  CAS  PubMed  Google Scholar 

  21. Sprangers, M. A., Van Dam, F. S., Broersen, J., Lodder, L., Wever, L., Visser, M. R., et al. (1999). Revealing response shift in longitudinal research on fatigue—the use of the thentest approach. Acta Oncologica, 38(6), 709–718.

    Article  CAS  PubMed  Google Scholar 

  22. Tederous-Williams, M. (2003). Response shift in women who have been pregnant with cancer. Quality of Life Research, 12(7), 783.

    Google Scholar 

  23. Ahmed, S., Mayo, N., Wood-Dauphinee, S., Hanley, J., & Cohen, R. (2005). Using the patient generated index to evaluate response shift post-stroke. Quality of Life Research, 14, 2247–2257.

    Article  PubMed  Google Scholar 

  24. Ahmed, S., Mayo, N. E., Corbiere, M., Wood-Dauphinee, S., Hanley, J., & Cohen, R. (2005). Change in quality of life in people with stroke over time: True change or response shift? Quality of Life Research, 14, 611–627.

    Article  PubMed  Google Scholar 

  25. Postulart, D., & Adang, E. M. (2000). Response shift and adaptation in chronically ill patients. Medical Decision Making, 20(2), 186–193.

    Article  CAS  PubMed  Google Scholar 

  26. Wikby, A., Stenström, U., Hörnquist, J. O., & Andersson, P. O. (1993). Coping behavior and degree of discrepancy between retrospective and prospective self-ratings of change in quality of life in Type 1 diabetes mellitus. Diabetic Medicine, 10, 851–854.

    Article  CAS  PubMed  Google Scholar 

  27. Daltroy, L. H., Larson, M. G., Eaton, H. M., Phillips, C. B., & Liang, M. H. (1999). Discrepancies between self-reported and observed physical function in the elderly: The influence of response shift and other factors. Social Science and Medicine, 48(11), 1549–1561.

    Article  CAS  PubMed  Google Scholar 

  28. Heidrich, S. M., & Ryff, C. D. (1993). The role of social comparisons processes in the psychological adaptation of elderly adults. Journal of Gerontology, 48(3), 127–136.

    Article  Google Scholar 

  29. Rijken, M., Komproe, I. H., Ros, W. J. G., Winnubst, J. A. M., & van Heesch, N. C. A. (1995). Subjective well-being of elderly women: Conceptual differences between cancer patients, women suffering from chronic ailments and healthy women. British Journal of Clinical Psychology, 34, 289–300.

    Article  PubMed  Google Scholar 

  30. Rees, J., MacDonagh, R., Waldron, D., & O’Boyle, C. (2004). Measuring quality of life in patients with advanced cancer. European Journal of Palliative Care, 11(3), 104–106.

    Google Scholar 

  31. Schwartz, C. E., Merriman, M., Reed, G., & Hammes, B. (2004). Measuring patient treatment preferences in end-of-life care research: Applications for advance care planning interventions and response shift research. Journal of Palliative Medicine, 7(2), 233–245.

    Article  PubMed  Google Scholar 

  32. Schwartz, C. E., Wheeler, H. B., Hammes, B., Basque, N., Edmunds, J., Reed, G., et al. (2002). Early intervention in planning end-of-life care with ambulatory geriatric patients: Results of a pilot trial. Archives of Internal Medicine, 162(14), 1611–1618.

    Article  PubMed  Google Scholar 

  33. Schwartz, C. E., Merriman, M. P., Reed, G., & Byock, I. (2005). Evaluation of the Missoula-VITAS Quality of Life Index—Revised: Research tool or clinical tool? Journal of Palliative Medicine, 8(1), 121–135.

    Article  PubMed  Google Scholar 

  34. Ring, L., Höfer, S., Heuston, F., Harris, D., & O’Boyle, C. A. (2005). Response shift masks the treatment impact on patient reported outcomes (PROs): The example of individual quality of life in edentulous patients. Health & Quality of Life Outcomes, 3, 55.

    Article  Google Scholar 

  35. Li, Y., & Rapkin, B. (2009). Classification and regression tree analysis to identify complex cognitive paths underlying quality of life response shifts: A study of individuals living with HIV/AIDS. Journal of Clinical Epidemiology, 62, 1138–1147.

    Article  PubMed Central  PubMed  Google Scholar 

  36. Rapkin, B. D., Schwartz, C. E., & Sprangers, M. A. G. (2000). Personal goals and response shifts: Understanding the impact of illness and events on the quality of life of people living with AIDS 8400. Adaptation to changing health: Response shift in quality-of-life research. Washington, D.C.: American Psychological Association. pp. 53–71.

  37. Razmjou, H., Yee, A., Ford, M., & Finkelstein, J. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone & Joint Surgery, 88(12), 2590–2595.

    Article  Google Scholar 

  38. Schwartz, C. E., Bode, R., Repucci, N., Becker, J., Sprangers, M. A. G., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15, 1533–1550.

    Article  PubMed  Google Scholar 

  39. Sajobi, T. T., Lix, L., Schwartz, C. E., Quaranto, B. R., & Finkelstein, J. A. (Eds.). (2012). Estimating reprioritization response shift as a function of spine surgery outcome: The impact of missing data imputation method on response shift detection. International Society for Quality of Life Research. Budapest, Hungary: Springer.

  40. Little, R. J. A., & Rubin, B. D. (2002). Statistical analysis with missing data (2nd ed.). New Jersey: Wiley.

    Book  Google Scholar 

  41. Fairclough, D. L. (2010). Characterization of missing data. Design and analysis of quality of life studies in clinical trials (pp. 125–147). New York: Chapman & Hall/CRC.

    Google Scholar 

  42. Robins, J. M., Rotnitzky, A., & Scharfstein, D. O. (2000). Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In M. E. Halloran & D. Berry (Eds.), Statistical models in epidemiology, the environment, and clinical trials (pp. 1–94). New York: Springer.

    Chapter  Google Scholar 

  43. Allison, P. D. (2002). Missing data. In M. Lewis-Beck (Ed.). Thousand Oaks, CA: Sage Publications.

  44. Fairclough, D. L. (2010). Pattern mixture and other mixture models. Design and analysis of quality of life studies in clinical trials (2nd ed., pp. 209–238). Boca Raton, FL: Taylor & Francis Group.

    Google Scholar 

  45. Sajobi, T. T., Lix, L. M., & Mayo, N. E. (2014). Identifying reprioritization response shift in a stroke caregiver population: A comparison of missing data methods. Quality of Life Research. Revision under review.

  46. Lix, L. M., Sajobi, T. T., Sawatzky, R., Liu, J., Mayo, N. E., Huang, Y., et al. (2013). Relative importance measures for reprioritization response shift. Quality of Life Research, 22(4), 695–703.

    Article  PubMed  Google Scholar 

  47. Sajobi, T. T., Lix, L. M., Clara, I., Walker, J. R., Graff, L. A., Rawsthorne, P., et al. (2012). Measures of relative importance for health-related quality of life. Quality of Life Research, 21(1), 1–11.

    Article  PubMed  Google Scholar 

  48. Verdam, M. G. E., Oort, F. J., van der Linden, Y. M., & Sprangers, M. A. G. (2014). Taking into account the impact of attrition on the assessment of response shift and true change: A multigroup structural equation modeling approach. Quality of Life Research. Under review.

  49. Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.

    Article  PubMed  Google Scholar 

  50. Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2014). Response shift algorithm in item response theory (ROSALI) for response shift detection with missing data in patient-reported outcomes in longitudinal clinical trials. Quality of Life Research. Under review.

  51. De Bock, E., Hardouin, J. B., Blanchin, M., Le Neel, T., Kubis, G., Bonnaud-Antignac, A., et al. (2014). Rasch-family models are more valuable than score-based approaches for analyzing longitudinal PRO with missing data. Statistical Methods in Medical Research. In press.

  52. Inc. SI. (2012). SAS/STAT user’s guide, version 9.2. Cary, NC: SAS Institute Inc.

  53. Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., et al. (2011). Openmx: An open source extended structural equation modeling framework. Psychometrika, 76, 306–317.

    Article  PubMed Central  PubMed  Google Scholar 

  54. Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.

    Google Scholar 

Download references

Acknowledgments

This collaboration was inspired and facilitated by the RS Special Interest Group (SIG) of the International Society for Quality of Life Research. Without such a unique organization for bringing together methodologists from around the globe, such work would not be possible. Drs. Sajobi and Lix are supported by an operating grant from the Canadian Institutes of Health Research (#122110). Dr. Sajobi is also supported by the University of Calgary Seed Grant. The work by Verdam is supported by a Grant from the Dutch Cancer Society (Project Number UVA 2011-4985). The work of Guilleux is supported by a grant from the Institut National du Cancer, under reference “INCA_6931”. We are grateful for helpful comments on an earlier draft of the manuscript from Jean-Benoit Hardouin, Ph.D., Sc.D.; Myriam Blanchin, Ph.D., and Antoine Vanier, M.D., M.Sc.

Author information

Authors and Affiliations

  1. DeltaQuest Foundation, Inc, Concord, MA, USA

    Carolyn E. Schwartz

  2. Departments of Medicine and Orthopaedic Surgery, Tufts University Medical School, Boston, MA, USA

    Carolyn E. Schwartz

  3. Department of Community Health Sciences and Institute for Public Health, University of Calgary, Calgary, Canada

    Tolulope T. Sajobi

  4. Department of Clinical Neurosciences, University of Calgary, Calgary, Canada

    Tolulope T. Sajobi

  5. Department of Medical Psychology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

    Mathilde G. E. Verdam & Mirjam A.G. Sprangers

  6. Department of Child Development and Education, University of Amsterdam, Amsterdam, The Netherlands

    Mathilde G. E. Verdam

  7. Department of Biostatistics, Pharmacoepidemiology and Subjective Measures in Health Sciences, EA 4275, University of Nantes, Nantes, France

    Veronique Sebille & Alice Guilleux

  8. Department of Community Health Sciences, University of Manitoba, Winnipeg, Canada

    Lisa M. Lix

Authors
  1. Carolyn E. Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tolulope T. Sajobi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mathilde G. E. Verdam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Veronique Sebille
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lisa M. Lix
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alice Guilleux
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mirjam A.G. Sprangers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carolyn E. Schwartz.

Additional information

On behalf of the Response Shift Special Interest Group of ISOQOL.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schwartz, C.E., Sajobi, T.T., Verdam, M.G.E. et al. Method variation in the impact of missing data on response shift detection. Qual Life Res 24, 521–528 (2015). https://doi.org/10.1007/s11136-014-0746-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-014-0746-0

Keywords

Search

Navigation