Skip to main content

Relative importance measures for reprioritization response shift

Quality of Life Research volume 22, pages 695–703 (2013)Cite this article

Abstract

Purpose

Response shift (RS), a change in the meaning of an individual’s self-evaluation of a target construct, such as health-related quality of life (HRQOL), can affect the interpretation of change in measures of the construct collected over time. This study proposes new statistical methods to test for reprioritization RS, in which the relative importance of HRQOL domains changes over time.

Methods

The methods use descriptive discriminant analysis or logistic regression models and bootstrap inference to test for change in relative importance weights (Method 1) or ranks (Method 2) for discriminating between patient groups at two occasions. The methods are demonstrated using data from the Manitoba Inflammatory Bowel Disease (IBD) Cohort Study (n = 388). Reprioritization of domains from the IBD Questionnaire (IBDQ) and SF-36 was investigated for groups with active and inactive disease symptoms.

Results

The IBDQ bowel symptoms and SF-36 bodily pain domains had the highest ranks for group discrimination. Using Method 1, there was evidence of reprioritization RS in the IBDQ social functioning domain and the SF-36 bodily pain and social functioning domains. Method 2 did not detect change for any of the domains.

Conclusions

Compared to IBD patients without active disease symptoms, those with active symptoms were likely to change the meaning of their self-evaluations of pain and social interactions. Further research is needed to compare these new RS detection methods under a variety of data analytic conditions before recommendations about the optimal method can be made.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

Abbreviations

DDA:

Descriptive discriminant analysis

DRC:

Discriminant ratio coefficient

HRQOL:

Health-related quality of life

IBD:

Inflammatory bowel disease

IBDQ:

Inflammatory Bowel Disease Questionnaire

LR:

Logistic regression

LPI:

Logistic Pratt’s index

RS:

Response shift

SDFC:

Standardized discriminant function coefficient

SEM:

Structural equation modeling

SF-36:

36-item Short-Form Questionnaire

SLRC:

Standardized logistic regression coefficient

References

  1. Schwartz, C. E., & Sprangers, M. A. G. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48, 1531–1548.

    PubMed  Article  CAS  Google Scholar 

  2. Dempster, M., Carney, R., & McClements, R. (2010). Response shift in the assessment of quality of life among people attending cardiac rehabilitation. British Journal of Health Psychology, 15, 307–319.

    PubMed  Article  Google Scholar 

  3. Ring, L., Hofer, S., Heuston, F., Harris, D., & O’Boyle, C. A. (2005). Response shift masks the treatment impact on patient reported outcomes (PROs): The example of individual quality of life in edentulous patients. Health and Quality of Life Outcomes, 3, 55.

    PubMed  Article  Google Scholar 

  4. Razmjou, H., Yee, A., Ford, M., & Finkelsten, J. A. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery, 88, 2590–2595.

    PubMed  Article  Google Scholar 

  5. McPhail, S., Cormans, T., & Haines, T. (2010). Evidence of disagreement between patient-perceived change and conventional longitudinal evaluation of change in health-related quality of life among older adults. Journal of Clinical Rehabilitation, 24, 1036–1044.

    Article  Google Scholar 

  6. Visser, M. R., Oort, J. F., & Sprangers, M. A. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14, 629–639.

    PubMed  Article  Google Scholar 

  7. McPhail, S., & Haines, T. (2010). Response shift, recall bias and their effect on measuring change in health-related quality of life amongst older hospital patients. Health and Quality of Life Outcomes, 8, 65.

    PubMed  Article  Google Scholar 

  8. Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2009). Tests of measurement invariance failed to support the application of the “then-test”. Journal of Clinical Epidemiology, 62, 1173–1180.

    PubMed  Article  Google Scholar 

  9. Brossart, D. F., Clay, D. L., & Willson, V. L. (2002). Methodological and statistical considerations for threats to internal validity in pediatric outcome data: Response shift in self-report outcomes. Journal of Pediatric Psychology, 27, 97–107.

    PubMed  Article  Google Scholar 

  10. Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.

    PubMed  Article  Google Scholar 

  11. Oort, F. J., Visser, M. R., & Sprangers, M. A. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.

    PubMed  Article  Google Scholar 

  12. Barclay-Goddard, R., Lix, L. M., Tate, R., Weinberg, L., & Mayo, N. E. (2009). Response shift was identified over multiple occasions with a structural equation modeling framework. Journal of Clinical Epidemiology, 62, 1181–1188.

    PubMed  Article  Google Scholar 

  13. Lowy, A., & Bernhard, J. (2004). Quantitative assessment of changes in patients’ constructs of quality of life: An application of multilevel models. Quality of Life Research, 13, 1177–1185.

    PubMed  Article  Google Scholar 

  14. Draper, N. H., & Smith, H. (1998). Applied regression analysis (3rd ed.). New York: Wiley.

    Google Scholar 

  15. Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17, 627–639.

    PubMed  Article  Google Scholar 

  16. Huberty, C. J., & Olejnik, S. (2006). Applied MANOVA and discriminant analysis (2nd ed.). New Jersey: Wiley.

    Book  Google Scholar 

  17. Agresti, A. (1996). An introduction to categorical data analysis. New York: Wiley.

    Google Scholar 

  18. Press, S. J., & Wilson, S. (1978). Choosing between logistic regression and discriminant analysis. Journal of the American Statistical Association, 73, 699–705.

    Article  Google Scholar 

  19. Kruskal, W., & Majors, R. (1989). Concepts of relative importance in recent scientific literature. The American Statistician, 43, 2–6.

    Google Scholar 

  20. Johnson, J. W., & Lebreton, J. M. (2004). History and use of relative importance indices in organizational research. Organizational Research Methods, 7, 238–257.

    Article  Google Scholar 

  21. Baek, S., Moon, H., Ahn, H., Kodell, R. L., Lin, C.-J., & Chen, J. J. (2008). Identifying high-dimensional biomarkers for personalized medicine via variable importance ranking. Journal of Biopharmaceutical Statistics, 18, 853–868.

    PubMed  Article  Google Scholar 

  22. Sajobi, T. T., Lix, L. M., Clara, I., Walker, J., Graff, L. L., Rawsthorne, P., et al. (2011). Measures of relative importance for health-related quality of life. Quality of Life Research, 21, 1–11.

    PubMed  Article  Google Scholar 

  23. Huberty, C. J., & Wisenbaker, J. M. (1992). Variable importance in multivariate group comparisons. Journal of Educational Statistics, 17, 75–91.

    Article  Google Scholar 

  24. Thomas, D. R., Zumbo, B. D., Zhu, P., & Dutta, S. (2008). On measuring the relative importance of explanatory variables in a logistic regression. Journal of Modern Applied Statistical Methods, 7, 21–38.

    Google Scholar 

  25. Thomas, D. R. (1992). Interpreting discriminant functions: A data analytic approach. Multivariate Behavioral Research, 27, 323–333.

    Article  Google Scholar 

  26. Menard, S. (2004). Six approaches to calculating standardized logistic regression coefficients. The American Statistician, 58, 218–223.

    Article  Google Scholar 

  27. Rencher, A. C. (1993). The contribution of individual variables to Hotelling’s T 2, Wilks’ Λ, and R 2. Biometrics, 49, 479–489.

    PubMed  Article  CAS  Google Scholar 

  28. Bull, S. B., & Donner, A. (1987). The efficiency of multinomial logistic regression compared with multiple group discriminant analysis. Journal of the American Statistical Association, 82, 1118–1122.

    Article  Google Scholar 

  29. Thomas, D. R., Hughes, E., & Zumbo, B. D. (1998). On variable importance in linear regression. Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, 45, 253–275.

    Article  Google Scholar 

  30. Williams, B. K., & Titus, K. (1998). Assessment of sampling stability in ecological applications of discriminant analysis. Ecology, 69, 1275–1285.

    Article  Google Scholar 

  31. Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49, 1373–1379.

    PubMed  Article  CAS  Google Scholar 

  32. Chernick, M. R. (2008). Bootstrap methods: A guide for practitioners and researchers. New Jersey: Wiley.

    Google Scholar 

  33. Dalgleish, L. I. (1994). Discriminant analysis: Statistical inference using the jackknife and bootstrap procedures. Psychological Bulletin, 116, 498–508.

    Article  Google Scholar 

  34. Walters, S. J., & Campbell, M. J. (2004). The use of bootstrap methods for analysing health-related quality of life outcomes (particularly the SF-36). Health and Quality of Life Outcomes, 2, 70.

    PubMed  Article  Google Scholar 

  35. Dunn, O. J. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56, 52–64.

    Article  Google Scholar 

  36. Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800–802.

    Article  Google Scholar 

  37. Graff, L. A., Walker, J., Lix, L. M., Clara, I., Rawsthorne, P., Rogala, L., et al. (2006). The relationship of disease type and activity to psychological functioning and quality of life. Clinical Gastroenterology and Hepatology, 4, 1491–1501.

    PubMed  Article  Google Scholar 

  38. Lix, L. M., Graff, L. A., Walker, J. R., Clara, I., Rawsthorne, P., Rogala, L., et al. (2008). Longitudinal study of quality of life and psychological functioning for active, fluctuating, and inactive disease patterns in inflammatory bowel disease. Inflammatory Bowel Disease, 14, 1575–1584.

    Article  Google Scholar 

  39. Clara, I., Lix, L. M., Walker, J. R., Graff, L. A., Miller, N., Rogala, L., et al. (2009). The Manitoba IBD index: Evidence for a new and simple indicator of IBD activity over time. Gastroenterology, 104, 1754–1763.

    Google Scholar 

  40. Guyatt, G. H., Mitchell, A., Irvine, E. J., Singer, J., Williams, N., Goodacre, R., et al. (1989). A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology, 96, 804–810.

    PubMed  CAS  Google Scholar 

  41. Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483.

    PubMed  Article  Google Scholar 

  42. Han, S. W., McColl, E., Steen, N., Barton, J. R., & Welfare, M. R. (1998). The inflammatory bowel disease questionnaire: A valid and reliable measure in ulcerative colitis patients in the North East of England. Scandinavian Journal of Gastroenterology, 33, 961–966.

    PubMed  Article  CAS  Google Scholar 

  43. Ware, J. E., Snow, K. K., Kosinski, M., & Gandek, B. (1993). SF-36 health survey: Manual and interpretation guide. Boston, MA: New England Medical Center, The Health Institute.

    Google Scholar 

  44. Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New Jersey: Wiley.

    Google Scholar 

  45. SAS Institute Inc. (2008). SAS/STAT user’s guide, version 9.2. Cary, NC: SAS Institute Inc.

    Google Scholar 

  46. Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment. New York: Wiley.

    Google Scholar 

  47. Urbakh, V. Y. (1971). Linear discriminant analysis: Loss of discriminating power when a variate is omitted. Biometrics, 27, 531–534.

    Article  Google Scholar 

  48. Ludbrook, J. (1998). Multiple comparison procedures updated. Clinical and Experimental Pharmacology and Physiology, 25, 1032–1037.

    PubMed  Article  CAS  Google Scholar 

  49. Jennrich, R. I. (1977). Stepwise discriminant analysis. In K. Enslein, A. Ralston, & H. S. Wilf (Eds.), Mathematical methods for digital computers (Vol. 3). New York: Wiley.

    Google Scholar 

  50. Thomas, D. R., & Zumbo, B. D. (1996). Using a measure of variable importance to investigate the standardization of discriminant coefficients. Journal of Educational and Behavioral Statistics, 21, 110–130.

    Google Scholar 

  51. Razmjou, H., Yee, A., Ford, M., & Finkelstein, J. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery, 88-A, 2590–2595.

    Article  Google Scholar 

  52. Ahmed, S., Mayo, N., Wood-Dauphinee, S., Hanley, J., & Cohen, S. (2004). Response shift influenced estimates of change in health-related quality of life poststroke. Journal of Clinical Epidemiology, 57, 561–570.

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

This research was supported by a Canadian Institutes of Health Research (CIHR) New Investigator award and University of Saskatchewan Centennial Chair to the first author, a CIHR Vanier Graduate Scholarship to the second author, and a CIHR Operating Grant to the research team.

Author information

Authors and Affiliations

  1. School of Public Health, University of Saskatchewan, 107 Wiggins Road, Saskatoon, SK, S7N 5E5, Canada

    Lisa M. Lix & Tolulope T. Sajobi

  2. Trinity Western University, Langley, Canada

    Richard Sawatzky

  3. McGill University Health Centre, Montreal, Canada

    Nancy E. Mayo

  4. University of Regina, Regina, Canada

    Yuhui Huang

  5. University of Manitoba, Winnipeg, Canada

    Lesley A. Graff, John R. Walker, Jason Ediger, Ian Clara, Kathryn Sexton, Rachel Carr & Charles N. Bernstein

  6. University of Saskatchewan, Saskatoon, SK, Canada

    Juxin Liu

Authors
  1. Lisa M. Lix
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tolulope T. Sajobi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard Sawatzky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Juxin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nancy E. Mayo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuhui Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lesley A. Graff
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. John R. Walker
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jason Ediger
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Ian Clara
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Kathryn Sexton
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Rachel Carr
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Charles N. Bernstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lisa M. Lix.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 63 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lix, L.M., Sajobi, T.T., Sawatzky, R. et al. Relative importance measures for reprioritization response shift. Qual Life Res 22, 695–703 (2013). https://doi.org/10.1007/s11136-012-0198-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-012-0198-3

Keywords

  • Discriminant analysis
  • Inflammatory bowel disease
  • Logistic regression
  • Longitudinal
  • Relative importance
  • Response shift