Abstract
Purpose
In health-related quality of life (HRQOL) studies, data are often collected on multiple domains for two or more groups of study participants. Quantitative measures of relative importance, which are used to rank order the domains based on their ability to discriminate between groups, are an alternative to multiple tests of significance on the group differences. This study describes relative importance measures based on logistic regression (LR) and multivariate analysis of variance (MANOVA) models.
Methods
Relative importance measures are illustrated using data from the Manitoba Inflammatory Bowel Disease (IBD) Cohort Study. Study participants with self-reported active (n = 244) and inactive (n = 105) disease were compared on 12 HRQOL domains from the Inflammatory Bowel Disease Questionnaire (IBDQ) and Medical Outcomes Study 36-item Short-Form (SF-36) Questionnaire.
Results
All but two relative importance measures ranked the IBDQ bowel symptoms and emotional health domains as most important.
Conclusions
MANOVA-based importance measures are recommended for multivariate normal data and when group covariances are equal, while LR measures are recommended for non-normal data and when the correlations among the domains are small. Relative importance measures can be used in exploratory studies to identify a small set of domains for further research.
Similar content being viewed by others
Abbreviations
- ADRC:
-
Adjusted discriminant ratio coefficient
- API:
-
Adjusted Pratt’s Index
- BP:
-
Bodily pain
- BS:
-
Bowel symptoms
- DDA:
-
Descriptive discriminant analysis
- DRC:
-
Discriminant ratio coefficient
- EH:
-
Emotional health
- GH:
-
General health
- HRQOL:
-
Health-related quality of life
- IBD:
-
Inflammatory bowel disease
- IBDQ:
-
Inflammatory Bowel Disease Questionnaire
- LR:
-
Logistic regression
- MANOVA:
-
Multivariate analysis of variance
- MH:
-
Mental health
- OLS:
-
Ordinary least squares
- PF:
-
Physical functioning
- PI:
-
Pratt’s index
- RE:
-
Role emotional
- RP:
-
Role physical
- RW:
-
Relative weight
- RRW:
-
Rescaled relative weight
- SDFC:
-
Standardized discriminant function coefficient
- SF:
-
Social functioning
- SF-36:
-
36-Item Short Form Questionnaire
- SLRC:
-
Standardized logistic regression coefficient
- SS:
-
Systemic symptoms
- VT:
-
Vitality
References
Fairclough, D. L. (2002). Design and analysis of quality of life studies in clinical trials. New York: Chapman & Hall.
Sheehan-Holt, J. K. (1998). MANOVA simultaneous test procedures: The power and robustness of restricted multivariate contrasts. Journal of Educational and Psychological Measurement, 58, 861–881.
Wu, Y., Hu, W., Xia, Y., Ma, J., Liu, M., & Cui, N. (2006). Quality of life of nasopharyngeal carcinoma survivors in Mainland China. Quality of Life Research, 16, 65–74.
Laaksonen, C., Aromaa, M., Heinonen, O. J., Koivusilta, L., Koski, P., Suominen, S., et al. (2008). Health related quality of life in 10-year-old school children. Quality of Life Research, 17, 1049–1054.
Dunn, W. S., Mount, M. K., Barrick, M. R., & Ones, D. S. (1995). Relative importance of personality and general mentality ability in managers’ judgments of applicant qualifications. Journal of Applied Psychology, 80, 500–509.
Hobson, C. J., & Gibson, F. W. (1983). Policy capturing as an approach to understanding and improving performance appraisal: A review of the literature. Academy of Management Review, 8, 640–649.
Healy, M. J. R. (1990). Measuring importance. Statistics in Medicine, 9, 633–637.
Curran, D., Molenberghs, G., Fayers, P. M., & Machin, D. (1998). Incomplete quality of life data in randomized trials: Missing forms. Statistics in Medicine, 17, 697–709.
Rose, M. S., & Koshman, M. L. (1999). Statistical issues encountered in the comparison of health-related quality of life in diseased patients to published general population norms: Problems and solutions. Journal of Clinical Epidemiology, 52, 405–412.
Beaumont, J. L., Lix, L. M., Yost, K. J., & Hahn, E. A. (2006). Application of robust statistical methods for sensitivity analysis of health-related quality of life outcomes. Quality of Life Research, 15, 349–356.
Menard, S. (1995). Applied logistic regression analysis. Thousand Oaks, CA: Sage.
Thomas, D. R., Zumbo, B. D., Zhu, P., & Dutta, S. (2008). On measuring the relative importance of explanatory variables in a logistic regression. Journal of Modern Applied Statistical Methods, 7, 21–38.
Traxel, N. M., & Azen, R. (2006). Predictor importance in logistic regression: An extension of dominance analysis. Poster presented at the 114th annual convention of the American psychological association, New Orleans, LA.
Tonidandale, S., & LeBreton, J. M. (2010). Determining the relative importance of predictors in logistic regression: An extension of relative weight analysis. Organizational Research Methods, 13, 767–781.
Huberty, C. J., & Wisenbaker, J. M. (1992). Variable importance in multivariate group comparisons. Journal of Educational Statistics, 17, 75–91.
Thomas, D. R. (1992). Interpreting discriminant functions: A data analytic approach. Multivariate Behavioral Research, 27, 335–362.
Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145–153.
Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Educational Psychology, 25, 241–286.
Agresti, A. (1996). An introduction to categorical data analysis. New York: Wiley.
Menard, S. (2004). Six approaches to calculating standardized logistic regression coefficients. The American Statistician, 58, 218–223.
Agresti, A., & Finlay, B. (1986). Statistical methods for the social sciences (2nd ed.). San Francisco: Dellen/Macmillan.
Kaufman, R. L. (1996). Comparing effects in dichotomous logistic regression: A variety of standardized coefficients. Social Science Quarterly, 77, 90–110.
Nagelkerke, N. J. D. (1991). A note on a general definition of coefficient of determination. Biometrika, 78, 691–692.
Cox, D. R., & Snell, E. J. (1989). The analysis of binary data (2nd ed.). London: Chapman and Hall.
Estrella, A. (1998). A new measure of fit for equations with dichotomous dependent variables. Journal of Business and Economic Statistics, 16, 198–205.
Pratt, J. W. (1987). Dividing the indivisible: Using simple symmetry to partition variance explained. In T. Pukkila & S. Puntanen (Eds.), Proceedings of the second international conference in statistics (pp. 245–260). Tampere, Finland: University of Tampere.
Thomas, D. R., Hughes, E., & Zumbo, B. D. (1998). On variable importance in linear regression. Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, 45, 253–275.
Ochieng, C. O., & Zumbo, B. D. (2001). Examination of a variable ordering index in linear regression models: An assessment of the relative Pratt index in Likert data. Paper presented at the Bob Conry conference on measurement evaluation and research and methodology, Vancouver, Canada.
Witherill, G. G. (1986). Regression analysis with applications. London: Chapman & Hall.
Azen, R., & Budescu, D. V. (2003). Comparing predictors in multivariate regression models: An extension of dominance analysis. Journal of Educational and Behavioral Statistics, 31, 157–180.
Johnson, J. W. (2000). A heuristic method for estimating the relative weight of predictor variables in multiple regression. Multivariate Behavioral Research, 35, 1–19.
Thomas, D. R., & Zumbo, B. D. (1996). Using a measure of variable importance to investigate the standardization of discriminant coefficients. Journal of Educational and Behavioral Statistics, 21, 110–130.
Huberty, C. J., & Olejnik, S. (2006). Applied MANOVA and discriminant analysis. Hoboken, New Jersey: Wiley.
Pohar, M., Blas, M., & Turk, S. (2004). Comparison of logistic regression and linear discriminant analysis: A simulation study. Metodoloski zveski, 1, 143–161.
Jennrich, R. I. (1977). Stepwise discriminant analysis. In K. Enslein, A. Ralston, & H. S. Wilf (Eds.), Mathematical methods for digital computers (Vol. 3). New York: Wiley.
Rencher, A. C. (1992). Interpretation of canonical discriminant functions, canonical variates, and principal components. The American Statistician, 46, 217–225.
Finch, W. H., & Laking, T. (2008). Evaluation of the use of standardized weights for interpreting results from a descriptive discriminant analysis. Multiple Linear Regression Viewpoints, 34, 19–34.
Gibbons, L. B., & Hosmer, D. W. (1991). Conditional logistic regression with missing data. Communications in Statistics, Part B, 20, 109–120.
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New Jersey: Wiley.
SAS Institute Inc. (2008). SAS/STAT user’s guide, version 9.2. Cary, NC: SAS Institute Inc.
Lix, L. M., Graff, L. A., Walker, J. R. L., et al. (2008). Longitudinal study of quality of life and psychological functioning for active, fluctuating, and inactive disease patterns. Inflammatory Bowel Diseases, 14, 1575–1584.
Graff, L. A., Walker, J. R., Lix, L. M., et al. (2006). The relationship of inflammatory bowel disease type and activity to psychological functioning and quality of life. Clinical Gastoroenterology and Hepatology, 4, 1491–1501.
Clara, I., Lix, L. M., Walker, J. R., et al. (2009). The Manitoba IBD index: Evidence for a new and simple indicator of IBD activity. American Journal of Gastroenterology, 104, 1754–1763.
Guyatt, G. H., Mitchell, A., Irvine, E. J., et al. (1989). A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology, 96, 804–810.
Ware, J. E., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30, 473–483.
Irvine, E. J. (1996). Effects of budesonide therapy on quality of life in active Crohn′s disease. Respiratory Clinical Forums, 18, 81–89.
Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52, 591–611.
LeBreton, J. M., & Tonidandel, S. (2008). Multivariate relative importance: Extending relative weight analysis to ultivariate criterion spaces. Journal of Applied Psychology, 93, 329–345.
Lievens, F., Van Hoye, G., & Schreurs, B. (2005). Examining the relationship between employer knowledge dimensions and organizational attractiveness: An application in a military context. Journal of Occupational and Organizational Psychology, 78, 553–572.
Glomb, T. M., Kammeyer-Mueller, J. D., & Rotundo, M. (2004). Emotional labor demands and compensating wage differentials. Journal of Applied Psychology, 84, 700–714.
Hawkins, D. M. (1981). A new test for multivariate normality and homoscedasticity. Technometrics, 23, 105–110.
Box, G. E. P. (1949). A general distribution theory for a class of likelihood criteria. Biometrika, 36, 317–346.
Mardia, K. V. (1980). Tests of univariate and multivariate normality. In P. R. Krishnaiah (Ed.), Handbook of statistics (Vol. 1, pp. 279–320). Amsterdam: North-Holland.
Shapiro, S. S., Wilk, M. B., & Chen, H. J. (1968). A comparative study of various tests for normality. Journal of the American Statistical Association, 63, 1343–1372.
Huberty, C. J. (1975). The stability of three indices of relative variable contribution in discriminant analysis. Journal of Experimental Education, 44, 59–64.
Dalgleish, L. I. (1994). Discriminant analysis: Statistical inference using the jackknife and bootstrap procedures. Psychological Bulletin, 116, 498–508.
Guyatt, G. H., Osoba, D., Wu, A. W., Wyrwich, K. W., Norman, G. R., & the Clinical Significance Consensus Meeting Group. (2002). Methods to explain the clinical significance of health status measures. Mayo Clinic Proceedings, 77, 371–383.
Wyrwich, K. W., Nienaber, N. A., Tierney, W. M., & Wolinsky, F. D. (1999). Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Medical Care, 37, 469–478.
Kim, K., & Timm, N. (2007). Univariate and multivariate general linear models: Theory and applications with SAS (2nd ed.). New York: Chapman & Hall/CRC.
Benjamini, Y., & Hochberg, Y. (1997). Multiple hypotheses testing with weights. Scandinavian Journal of Statistics, 24, 407–418.
Maurer, W., Hothorn, L. A., & Lechmacher, W. (1995). Multiple comparisons in drug clinical trials and preclinical assays: A priori ordered hypotheses. In J. Vollman (Ed.), Biometrie in der chemische-pharmazeutichen Industrie (Vol. 6). Stuttgart: Fischer Verlag.
Acknowledgments
This research was supported by a Canadian Institutes of Health Research (CIHR) Vanier Graduate Scholarship to the first author, funding from the Manitoba Health Research Council and a CIHR New Investigator Award to the second author, funding from a Crohn’s and Colitis Foundation of Canada Research Investigator Award and the Bingham Chair in Gastroenterology to the last author and funding from a CIHR Operating Grant to the research team.
Conflict of interest
Dr. Lix has received funding from Amgen in the form of an unrestricted research grant. In the past year, Dr. Bernstein has received consulting fees from Abbott Canada and an unrestricted educational grant from Axcan Pharma.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Additional formulae used in calculating measures of relative importance
Measures based on the LR model
The estimated LR coefficient for the kth domain (k = 1,…, m) can be written as
where \( r_{{\log {\text{it}}(\hat{p})k}} \) is the correlation between the kth domain and the logit of the predicted probabilities, \( R_{( - k)}^{2} \) is the R 2 value for a LR model in which the kth domain is excluded, and \( R_{k|( - k)}^{2} \) is the R 2 value for a model in which the kth domain is regressed on the other (m − 1) domains.
Pratt’s index can also be expressed as,
where X is the N × m data matrix, X [k] is the N × 1 vector of measurements on the kth domain, \( {\hat{\mathbf{\varvec{\upbeta}}}} \) is a m × 1 vector of estimated unstandardized LR coefficients, \( {\mathbf{Q}} = {\mathbf{I}}_{N} - \left( {{\mathbf{1}}_{N}{\mathbf{1}}_{N}^{\text{T}} /N} \right) \), I N is a N × N identity matrix, 1 N is a N × 1 matrix of ones, and T is the transpose operator.
Measures based on the MANOVA model
The vectors of discriminant function coefficients corresponds to the eigenvectors associated with E −1 H, where
is the error sum of squares and cross product matrix, and
is the hypothesis sum of squares and cross product matrix. The number of statistically significant discriminant functions is c = min (m, g − 1). The discriminant function score, z ij , for the ith study participant in the jth (i = 1,…, n j ; j = 1, 2) group is,
The discriminant function coefficient for the kth variable can also be expressed as
where \( u_{k} \) is the kth element of \( \text{\bf{S}}^{ - 1} \left( {{\bar{\mathbf{X}}}_{1} -{\bar{\mathbf{X}}}_{2} } \right) \), S is the pooled sample covariance matrix, and \( \overline{X}_{j} \) is the vector of means for the jth group.An equivalent formula for computing the F-to-remove statistic for the kth domain is
where \( k_{2} = (n_{1} + n_{2} - 2 - m),\,k_{3} = (n_{1} + n_{2} )(n_{1} + n_{2} )/n_{1} n_{2} ,\,\hat{a}_{k} \) is the discriminant function coefficient for the kth domain, \( \bar{z}_{1} \,{\text{and}}\,\bar{z}_{2} \) are the group means for the discriminant function score corresponding to \( {\hat{\mathbf{a}}} \), and \( s_{(kk)} \) is the positive square root of the kth diagonal element of the inverse of E, the error sums of square and cross product matrix.
Appendix 2
See Table 5.
Rights and permissions
About this article
Cite this article
Sajobi, T.T., Lix, L.M., Clara, I. et al. Measures of relative importance for health-related quality of life. Qual Life Res 21, 1–11 (2012). https://doi.org/10.1007/s11136-011-9914-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-011-9914-7