Quality of Life Research

, Volume 22, Issue 4, pp 695–703 | Cite as

Relative importance measures for reprioritization response shift

  • Lisa M. Lix
  • Tolulope T. Sajobi
  • Richard Sawatzky
  • Juxin Liu
  • Nancy E. Mayo
  • Yuhui Huang
  • Lesley A. Graff
  • John R. Walker
  • Jason Ediger
  • Ian Clara
  • Kathryn Sexton
  • Rachel Carr
  • Charles N. Bernstein



Response shift (RS), a change in the meaning of an individual’s self-evaluation of a target construct, such as health-related quality of life (HRQOL), can affect the interpretation of change in measures of the construct collected over time. This study proposes new statistical methods to test for reprioritization RS, in which the relative importance of HRQOL domains changes over time.


The methods use descriptive discriminant analysis or logistic regression models and bootstrap inference to test for change in relative importance weights (Method 1) or ranks (Method 2) for discriminating between patient groups at two occasions. The methods are demonstrated using data from the Manitoba Inflammatory Bowel Disease (IBD) Cohort Study (n = 388). Reprioritization of domains from the IBD Questionnaire (IBDQ) and SF-36 was investigated for groups with active and inactive disease symptoms.


The IBDQ bowel symptoms and SF-36 bodily pain domains had the highest ranks for group discrimination. Using Method 1, there was evidence of reprioritization RS in the IBDQ social functioning domain and the SF-36 bodily pain and social functioning domains. Method 2 did not detect change for any of the domains.


Compared to IBD patients without active disease symptoms, those with active symptoms were likely to change the meaning of their self-evaluations of pain and social interactions. Further research is needed to compare these new RS detection methods under a variety of data analytic conditions before recommendations about the optimal method can be made.


Discriminant analysis Inflammatory bowel disease Logistic regression Longitudinal Relative importance Response shift 



Descriptive discriminant analysis


Discriminant ratio coefficient


Health-related quality of life


Inflammatory bowel disease


Inflammatory Bowel Disease Questionnaire


Logistic regression


Logistic Pratt’s index


Response shift


Standardized discriminant function coefficient


Structural equation modeling


36-item Short-Form Questionnaire


Standardized logistic regression coefficient



This research was supported by a Canadian Institutes of Health Research (CIHR) New Investigator award and University of Saskatchewan Centennial Chair to the first author, a CIHR Vanier Graduate Scholarship to the second author, and a CIHR Operating Grant to the research team.

Supplementary material

11136_2012_198_MOESM1_ESM.doc (64 kb)
Supplementary material 1 (DOC 63 kb)


  1. 1.
    Schwartz, C. E., & Sprangers, M. A. G. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48, 1531–1548.PubMedCrossRefGoogle Scholar
  2. 2.
    Dempster, M., Carney, R., & McClements, R. (2010). Response shift in the assessment of quality of life among people attending cardiac rehabilitation. British Journal of Health Psychology, 15, 307–319.PubMedCrossRefGoogle Scholar
  3. 3.
    Ring, L., Hofer, S., Heuston, F., Harris, D., & O’Boyle, C. A. (2005). Response shift masks the treatment impact on patient reported outcomes (PROs): The example of individual quality of life in edentulous patients. Health and Quality of Life Outcomes, 3, 55.PubMedCrossRefGoogle Scholar
  4. 4.
    Razmjou, H., Yee, A., Ford, M., & Finkelsten, J. A. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery, 88, 2590–2595.PubMedCrossRefGoogle Scholar
  5. 5.
    McPhail, S., Cormans, T., & Haines, T. (2010). Evidence of disagreement between patient-perceived change and conventional longitudinal evaluation of change in health-related quality of life among older adults. Journal of Clinical Rehabilitation, 24, 1036–1044.CrossRefGoogle Scholar
  6. 6.
    Visser, M. R., Oort, J. F., & Sprangers, M. A. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14, 629–639.PubMedCrossRefGoogle Scholar
  7. 7.
    McPhail, S., & Haines, T. (2010). Response shift, recall bias and their effect on measuring change in health-related quality of life amongst older hospital patients. Health and Quality of Life Outcomes, 8, 65.PubMedCrossRefGoogle Scholar
  8. 8.
    Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2009). Tests of measurement invariance failed to support the application of the “then-test”. Journal of Clinical Epidemiology, 62, 1173–1180.PubMedCrossRefGoogle Scholar
  9. 9.
    Brossart, D. F., Clay, D. L., & Willson, V. L. (2002). Methodological and statistical considerations for threats to internal validity in pediatric outcome data: Response shift in self-report outcomes. Journal of Pediatric Psychology, 27, 97–107.PubMedCrossRefGoogle Scholar
  10. 10.
    Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.PubMedCrossRefGoogle Scholar
  11. 11.
    Oort, F. J., Visser, M. R., & Sprangers, M. A. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.PubMedCrossRefGoogle Scholar
  12. 12.
    Barclay-Goddard, R., Lix, L. M., Tate, R., Weinberg, L., & Mayo, N. E. (2009). Response shift was identified over multiple occasions with a structural equation modeling framework. Journal of Clinical Epidemiology, 62, 1181–1188.PubMedCrossRefGoogle Scholar
  13. 13.
    Lowy, A., & Bernhard, J. (2004). Quantitative assessment of changes in patients’ constructs of quality of life: An application of multilevel models. Quality of Life Research, 13, 1177–1185.PubMedCrossRefGoogle Scholar
  14. 14.
    Draper, N. H., & Smith, H. (1998). Applied regression analysis (3rd ed.). New York: Wiley.Google Scholar
  15. 15.
    Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17, 627–639.PubMedCrossRefGoogle Scholar
  16. 16.
    Huberty, C. J., & Olejnik, S. (2006). Applied MANOVA and discriminant analysis (2nd ed.). New Jersey: Wiley.CrossRefGoogle Scholar
  17. 17.
    Agresti, A. (1996). An introduction to categorical data analysis. New York: Wiley.Google Scholar
  18. 18.
    Press, S. J., & Wilson, S. (1978). Choosing between logistic regression and discriminant analysis. Journal of the American Statistical Association, 73, 699–705.CrossRefGoogle Scholar
  19. 19.
    Kruskal, W., & Majors, R. (1989). Concepts of relative importance in recent scientific literature. The American Statistician, 43, 2–6.Google Scholar
  20. 20.
    Johnson, J. W., & Lebreton, J. M. (2004). History and use of relative importance indices in organizational research. Organizational Research Methods, 7, 238–257.CrossRefGoogle Scholar
  21. 21.
    Baek, S., Moon, H., Ahn, H., Kodell, R. L., Lin, C.-J., & Chen, J. J. (2008). Identifying high-dimensional biomarkers for personalized medicine via variable importance ranking. Journal of Biopharmaceutical Statistics, 18, 853–868.PubMedCrossRefGoogle Scholar
  22. 22.
    Sajobi, T. T., Lix, L. M., Clara, I., Walker, J., Graff, L. L., Rawsthorne, P., et al. (2011). Measures of relative importance for health-related quality of life. Quality of Life Research, 21, 1–11.PubMedCrossRefGoogle Scholar
  23. 23.
    Huberty, C. J., & Wisenbaker, J. M. (1992). Variable importance in multivariate group comparisons. Journal of Educational Statistics, 17, 75–91.CrossRefGoogle Scholar
  24. 24.
    Thomas, D. R., Zumbo, B. D., Zhu, P., & Dutta, S. (2008). On measuring the relative importance of explanatory variables in a logistic regression. Journal of Modern Applied Statistical Methods, 7, 21–38.Google Scholar
  25. 25.
    Thomas, D. R. (1992). Interpreting discriminant functions: A data analytic approach. Multivariate Behavioral Research, 27, 323–333.CrossRefGoogle Scholar
  26. 26.
    Menard, S. (2004). Six approaches to calculating standardized logistic regression coefficients. The American Statistician, 58, 218–223.CrossRefGoogle Scholar
  27. 27.
    Rencher, A. C. (1993). The contribution of individual variables to Hotelling’s T 2, Wilks’ Λ, and R 2. Biometrics, 49, 479–489.PubMedCrossRefGoogle Scholar
  28. 28.
    Bull, S. B., & Donner, A. (1987). The efficiency of multinomial logistic regression compared with multiple group discriminant analysis. Journal of the American Statistical Association, 82, 1118–1122.CrossRefGoogle Scholar
  29. 29.
    Thomas, D. R., Hughes, E., & Zumbo, B. D. (1998). On variable importance in linear regression. Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, 45, 253–275.CrossRefGoogle Scholar
  30. 30.
    Williams, B. K., & Titus, K. (1998). Assessment of sampling stability in ecological applications of discriminant analysis. Ecology, 69, 1275–1285.CrossRefGoogle Scholar
  31. 31.
    Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49, 1373–1379.PubMedCrossRefGoogle Scholar
  32. 32.
    Chernick, M. R. (2008). Bootstrap methods: A guide for practitioners and researchers. New Jersey: Wiley.Google Scholar
  33. 33.
    Dalgleish, L. I. (1994). Discriminant analysis: Statistical inference using the jackknife and bootstrap procedures. Psychological Bulletin, 116, 498–508.CrossRefGoogle Scholar
  34. 34.
    Walters, S. J., & Campbell, M. J. (2004). The use of bootstrap methods for analysing health-related quality of life outcomes (particularly the SF-36). Health and Quality of Life Outcomes, 2, 70.PubMedCrossRefGoogle Scholar
  35. 35.
    Dunn, O. J. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56, 52–64.CrossRefGoogle Scholar
  36. 36.
    Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800–802.CrossRefGoogle Scholar
  37. 37.
    Graff, L. A., Walker, J., Lix, L. M., Clara, I., Rawsthorne, P., Rogala, L., et al. (2006). The relationship of disease type and activity to psychological functioning and quality of life. Clinical Gastroenterology and Hepatology, 4, 1491–1501.PubMedCrossRefGoogle Scholar
  38. 38.
    Lix, L. M., Graff, L. A., Walker, J. R., Clara, I., Rawsthorne, P., Rogala, L., et al. (2008). Longitudinal study of quality of life and psychological functioning for active, fluctuating, and inactive disease patterns in inflammatory bowel disease. Inflammatory Bowel Disease, 14, 1575–1584.CrossRefGoogle Scholar
  39. 39.
    Clara, I., Lix, L. M., Walker, J. R., Graff, L. A., Miller, N., Rogala, L., et al. (2009). The Manitoba IBD index: Evidence for a new and simple indicator of IBD activity over time. Gastroenterology, 104, 1754–1763.Google Scholar
  40. 40.
    Guyatt, G. H., Mitchell, A., Irvine, E. J., Singer, J., Williams, N., Goodacre, R., et al. (1989). A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology, 96, 804–810.PubMedGoogle Scholar
  41. 41.
    Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483.PubMedCrossRefGoogle Scholar
  42. 42.
    Han, S. W., McColl, E., Steen, N., Barton, J. R., & Welfare, M. R. (1998). The inflammatory bowel disease questionnaire: A valid and reliable measure in ulcerative colitis patients in the North East of England. Scandinavian Journal of Gastroenterology, 33, 961–966.PubMedCrossRefGoogle Scholar
  43. 43.
    Ware, J. E., Snow, K. K., Kosinski, M., & Gandek, B. (1993). SF-36 health survey: Manual and interpretation guide. Boston, MA: New England Medical Center, The Health Institute.Google Scholar
  44. 44.
    Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New Jersey: Wiley.Google Scholar
  45. 45.
    SAS Institute Inc. (2008). SAS/STAT user’s guide, version 9.2. Cary, NC: SAS Institute Inc.Google Scholar
  46. 46.
    Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment. New York: Wiley.Google Scholar
  47. 47.
    Urbakh, V. Y. (1971). Linear discriminant analysis: Loss of discriminating power when a variate is omitted. Biometrics, 27, 531–534.CrossRefGoogle Scholar
  48. 48.
    Ludbrook, J. (1998). Multiple comparison procedures updated. Clinical and Experimental Pharmacology and Physiology, 25, 1032–1037.PubMedCrossRefGoogle Scholar
  49. 49.
    Jennrich, R. I. (1977). Stepwise discriminant analysis. In K. Enslein, A. Ralston, & H. S. Wilf (Eds.), Mathematical methods for digital computers (Vol. 3). New York: Wiley.Google Scholar
  50. 50.
    Thomas, D. R., & Zumbo, B. D. (1996). Using a measure of variable importance to investigate the standardization of discriminant coefficients. Journal of Educational and Behavioral Statistics, 21, 110–130.Google Scholar
  51. 51.
    Razmjou, H., Yee, A., Ford, M., & Finkelstein, J. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery, 88-A, 2590–2595.CrossRefGoogle Scholar
  52. 52.
    Ahmed, S., Mayo, N., Wood-Dauphinee, S., Hanley, J., & Cohen, S. (2004). Response shift influenced estimates of change in health-related quality of life poststroke. Journal of Clinical Epidemiology, 57, 561–570.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  • Lisa M. Lix
    • 1
  • Tolulope T. Sajobi
    • 1
  • Richard Sawatzky
    • 2
  • Juxin Liu
    • 6
  • Nancy E. Mayo
    • 3
  • Yuhui Huang
    • 4
  • Lesley A. Graff
    • 5
  • John R. Walker
    • 5
  • Jason Ediger
    • 5
  • Ian Clara
    • 5
  • Kathryn Sexton
    • 5
  • Rachel Carr
    • 5
  • Charles N. Bernstein
    • 5
  1. 1.School of Public HealthUniversity of SaskatchewanSaskatoonCanada
  2. 2.Trinity Western UniversityLangleyCanada
  3. 3.McGill University Health CentreMontrealCanada
  4. 4.University of ReginaReginaCanada
  5. 5.University of ManitobaWinnipegCanada
  6. 6.University of SaskatchewanSaskatoonCanada

Personalised recommendations