Response shift (RS), a change in the meaning of an individual’s self-evaluation of a target construct, such as health-related quality of life (HRQOL), can affect the interpretation of change in measures of the construct collected over time. This study proposes new statistical methods to test for reprioritization RS, in which the relative importance of HRQOL domains changes over time.
The methods use descriptive discriminant analysis or logistic regression models and bootstrap inference to test for change in relative importance weights (Method 1) or ranks (Method 2) for discriminating between patient groups at two occasions. The methods are demonstrated using data from the Manitoba Inflammatory Bowel Disease (IBD) Cohort Study (n = 388). Reprioritization of domains from the IBD Questionnaire (IBDQ) and SF-36 was investigated for groups with active and inactive disease symptoms.
The IBDQ bowel symptoms and SF-36 bodily pain domains had the highest ranks for group discrimination. Using Method 1, there was evidence of reprioritization RS in the IBDQ social functioning domain and the SF-36 bodily pain and social functioning domains. Method 2 did not detect change for any of the domains.
Compared to IBD patients without active disease symptoms, those with active symptoms were likely to change the meaning of their self-evaluations of pain and social interactions. Further research is needed to compare these new RS detection methods under a variety of data analytic conditions before recommendations about the optimal method can be made.
This is a preview of subscription content,to check access.
Access this article
Similar content being viewed by others
Descriptive discriminant analysis
Discriminant ratio coefficient
Health-related quality of life
Inflammatory bowel disease
Inflammatory Bowel Disease Questionnaire
Logistic Pratt’s index
Standardized discriminant function coefficient
Structural equation modeling
36-item Short-Form Questionnaire
Standardized logistic regression coefficient
Schwartz, C. E., & Sprangers, M. A. G. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48, 1531–1548.
Dempster, M., Carney, R., & McClements, R. (2010). Response shift in the assessment of quality of life among people attending cardiac rehabilitation. British Journal of Health Psychology, 15, 307–319.
Ring, L., Hofer, S., Heuston, F., Harris, D., & O’Boyle, C. A. (2005). Response shift masks the treatment impact on patient reported outcomes (PROs): The example of individual quality of life in edentulous patients. Health and Quality of Life Outcomes, 3, 55.
Razmjou, H., Yee, A., Ford, M., & Finkelsten, J. A. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery, 88, 2590–2595.
McPhail, S., Cormans, T., & Haines, T. (2010). Evidence of disagreement between patient-perceived change and conventional longitudinal evaluation of change in health-related quality of life among older adults. Journal of Clinical Rehabilitation, 24, 1036–1044.
Visser, M. R., Oort, J. F., & Sprangers, M. A. (2005). Methods to detect response shift in quality of life data: A convergent validity study. Quality of Life Research, 14, 629–639.
McPhail, S., & Haines, T. (2010). Response shift, recall bias and their effect on measuring change in health-related quality of life amongst older hospital patients. Health and Quality of Life Outcomes, 8, 65.
Nolte, S., Elsworth, G. R., Sinclair, A. J., & Osborne, R. H. (2009). Tests of measurement invariance failed to support the application of the “then-test”. Journal of Clinical Epidemiology, 62, 1173–1180.
Brossart, D. F., Clay, D. L., & Willson, V. L. (2002). Methodological and statistical considerations for threats to internal validity in pediatric outcome data: Response shift in self-report outcomes. Journal of Pediatric Psychology, 27, 97–107.
Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.
Oort, F. J., Visser, M. R., & Sprangers, M. A. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.
Barclay-Goddard, R., Lix, L. M., Tate, R., Weinberg, L., & Mayo, N. E. (2009). Response shift was identified over multiple occasions with a structural equation modeling framework. Journal of Clinical Epidemiology, 62, 1181–1188.
Lowy, A., & Bernhard, J. (2004). Quantitative assessment of changes in patients’ constructs of quality of life: An application of multilevel models. Quality of Life Research, 13, 1177–1185.
Draper, N. H., & Smith, H. (1998). Applied regression analysis (3rd ed.). New York: Wiley.
Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17, 627–639.
Huberty, C. J., & Olejnik, S. (2006). Applied MANOVA and discriminant analysis (2nd ed.). New Jersey: Wiley.
Agresti, A. (1996). An introduction to categorical data analysis. New York: Wiley.
Press, S. J., & Wilson, S. (1978). Choosing between logistic regression and discriminant analysis. Journal of the American Statistical Association, 73, 699–705.
Kruskal, W., & Majors, R. (1989). Concepts of relative importance in recent scientific literature. The American Statistician, 43, 2–6.
Johnson, J. W., & Lebreton, J. M. (2004). History and use of relative importance indices in organizational research. Organizational Research Methods, 7, 238–257.
Baek, S., Moon, H., Ahn, H., Kodell, R. L., Lin, C.-J., & Chen, J. J. (2008). Identifying high-dimensional biomarkers for personalized medicine via variable importance ranking. Journal of Biopharmaceutical Statistics, 18, 853–868.
Sajobi, T. T., Lix, L. M., Clara, I., Walker, J., Graff, L. L., Rawsthorne, P., et al. (2011). Measures of relative importance for health-related quality of life. Quality of Life Research, 21, 1–11.
Huberty, C. J., & Wisenbaker, J. M. (1992). Variable importance in multivariate group comparisons. Journal of Educational Statistics, 17, 75–91.
Thomas, D. R., Zumbo, B. D., Zhu, P., & Dutta, S. (2008). On measuring the relative importance of explanatory variables in a logistic regression. Journal of Modern Applied Statistical Methods, 7, 21–38.
Thomas, D. R. (1992). Interpreting discriminant functions: A data analytic approach. Multivariate Behavioral Research, 27, 323–333.
Menard, S. (2004). Six approaches to calculating standardized logistic regression coefficients. The American Statistician, 58, 218–223.
Rencher, A. C. (1993). The contribution of individual variables to Hotelling’s T 2, Wilks’ Λ, and R 2. Biometrics, 49, 479–489.
Bull, S. B., & Donner, A. (1987). The efficiency of multinomial logistic regression compared with multiple group discriminant analysis. Journal of the American Statistical Association, 82, 1118–1122.
Thomas, D. R., Hughes, E., & Zumbo, B. D. (1998). On variable importance in linear regression. Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, 45, 253–275.
Williams, B. K., & Titus, K. (1998). Assessment of sampling stability in ecological applications of discriminant analysis. Ecology, 69, 1275–1285.
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49, 1373–1379.
Chernick, M. R. (2008). Bootstrap methods: A guide for practitioners and researchers. New Jersey: Wiley.
Dalgleish, L. I. (1994). Discriminant analysis: Statistical inference using the jackknife and bootstrap procedures. Psychological Bulletin, 116, 498–508.
Walters, S. J., & Campbell, M. J. (2004). The use of bootstrap methods for analysing health-related quality of life outcomes (particularly the SF-36). Health and Quality of Life Outcomes, 2, 70.
Dunn, O. J. (1961). Multiple comparisons among means. Journal of the American Statistical Association, 56, 52–64.
Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance. Biometrika, 75, 800–802.
Graff, L. A., Walker, J., Lix, L. M., Clara, I., Rawsthorne, P., Rogala, L., et al. (2006). The relationship of disease type and activity to psychological functioning and quality of life. Clinical Gastroenterology and Hepatology, 4, 1491–1501.
Lix, L. M., Graff, L. A., Walker, J. R., Clara, I., Rawsthorne, P., Rogala, L., et al. (2008). Longitudinal study of quality of life and psychological functioning for active, fluctuating, and inactive disease patterns in inflammatory bowel disease. Inflammatory Bowel Disease, 14, 1575–1584.
Clara, I., Lix, L. M., Walker, J. R., Graff, L. A., Miller, N., Rogala, L., et al. (2009). The Manitoba IBD index: Evidence for a new and simple indicator of IBD activity over time. Gastroenterology, 104, 1754–1763.
Guyatt, G. H., Mitchell, A., Irvine, E. J., Singer, J., Williams, N., Goodacre, R., et al. (1989). A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology, 96, 804–810.
Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483.
Han, S. W., McColl, E., Steen, N., Barton, J. R., & Welfare, M. R. (1998). The inflammatory bowel disease questionnaire: A valid and reliable measure in ulcerative colitis patients in the North East of England. Scandinavian Journal of Gastroenterology, 33, 961–966.
Ware, J. E., Snow, K. K., Kosinski, M., & Gandek, B. (1993). SF-36 health survey: Manual and interpretation guide. Boston, MA: New England Medical Center, The Health Institute.
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New Jersey: Wiley.
SAS Institute Inc. (2008). SAS/STAT user’s guide, version 9.2. Cary, NC: SAS Institute Inc.
Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment. New York: Wiley.
Urbakh, V. Y. (1971). Linear discriminant analysis: Loss of discriminating power when a variate is omitted. Biometrics, 27, 531–534.
Ludbrook, J. (1998). Multiple comparison procedures updated. Clinical and Experimental Pharmacology and Physiology, 25, 1032–1037.
Jennrich, R. I. (1977). Stepwise discriminant analysis. In K. Enslein, A. Ralston, & H. S. Wilf (Eds.), Mathematical methods for digital computers (Vol. 3). New York: Wiley.
Thomas, D. R., & Zumbo, B. D. (1996). Using a measure of variable importance to investigate the standardization of discriminant coefficients. Journal of Educational and Behavioral Statistics, 21, 110–130.
Razmjou, H., Yee, A., Ford, M., & Finkelstein, J. (2006). Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery, 88-A, 2590–2595.
Ahmed, S., Mayo, N., Wood-Dauphinee, S., Hanley, J., & Cohen, S. (2004). Response shift influenced estimates of change in health-related quality of life poststroke. Journal of Clinical Epidemiology, 57, 561–570.
This research was supported by a Canadian Institutes of Health Research (CIHR) New Investigator award and University of Saskatchewan Centennial Chair to the first author, a CIHR Vanier Graduate Scholarship to the second author, and a CIHR Operating Grant to the research team.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Lix, L.M., Sajobi, T.T., Sawatzky, R. et al. Relative importance measures for reprioritization response shift. Qual Life Res 22, 695–703 (2013). https://doi.org/10.1007/s11136-012-0198-3