Abstract
Purpose
To evaluate the reliability and validity of six PROMIS measures (anxiety, depression, fatigue, pain interference, physical function, and sleep disturbance) telephone-administered to a diverse, population-based cohort of localized prostate cancer patients.
Methods
Newly diagnosed men were enrolled in the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study. PROMIS measures were telephone-administered pre-treatment (baseline), and at 3-months and 12-months post-treatment initiation (N = 778). Reliability was evaluated using Cronbach’s alpha. Dimensionality was examined with bifactor models and explained common variance (ECV). Ordinal logistic regression models were used to detect potential differential item functioning (DIF) for key demographic groups. Convergent and discriminant validity were assessed by correlations with the legacy instruments Memorial Anxiety Scale for Prostate Cancer and SF-12v2. Known-groups validity was examined by age, race/ethnicity, comorbidity, and treatment.
Results
Each PROMIS measure had high Cronbach’s alpha values (0.86–0.96) and was sufficiently unidimensional. Floor effects were observed for anxiety, depression, and pain interference measures; ceiling effects were observed for physical function. No DIF was detected. Convergent validity was established with moderate to strong correlations between PROMIS and legacy measures (0.41–0.77) of similar constructs. Discriminant validity was demonstrated with weak correlations between measures of dissimilar domains (−0.20–−0.31). PROMIS measures detected differences across age, race/ethnicity, and comorbidity groups; no differences were found by treatment.
Conclusions
This study provides support for the reliability and construct validity of six PROMIS measures in prostate cancer, as well as the utility of telephone administration for assessing HRQoL in low literacy and hard-to-reach populations.
Similar content being viewed by others
References
National Cancer Institute (2015). SEER Cancer Statistics Factsheets: Prostate Cancer. http://seer.cancer.gov/statfacts/html/prost.html.
American Cancer Society (2015). Prostate Cancer Overview. http://www.cancer.org/cancer/prostatecancer/detailedguide/prostate-cancer-what-is-prostate-cancer.
Wilt, T. J., MacDonald, R., Rutks, I., Shamliyan, T. A., Taylor, B. C., & Kane, R. L. (2008). Systematic review: Comparative effectiveness and harms of treatments for clinically localized prostate cancer. Annals of Internal Medicine, 148(6), 435–448. doi:10.7326/0003-4819-148-6-200803180-00209.
Xiong, T., Turner, R. M., Wei, Y., Neal, D. E., Lyratzopoulos, G., & Higgins, J. P. (2014). Comparative efficacy and safety of treatments for localised prostate cancer: An application of network meta-analysis. BMJ Open, 4(5), e004285. doi:10.1136/bmjopen-2013-004285.
Sun F, Oyesanmi O, Fontanarosa J, Reston J, Guzzo T, & Schoelles K (December 2014). Therapies for Clinically Localized Prostate Cancer: Update of a 2008 Systematic Review. Comparative Effectiveness Review No. 146. (Prepared by the ECRI Institute–Penn Medicine Evidence-based Practice Center under Contract No. 290-2007-10063.) AHRQ Publication No. 15-EHC004-EF. Rockville, MD: Agency for Healthcare Research and Quality.
Hoffman, R. M., Penson, D. F., Zietman, A. L., & Barry, M. J. (2013). Comparative effectiveness research in localized prostate cancer treatment. Journal of Comparative Effectiveness Research, 2(6), 583–593. doi:10.2217/cer.13.66.
Aaronson, N., Alonso, J., Burnam, A., Lohr, K. N., Patrick, D. L., Perrin, E., et al. (2002). Assessing health status and quality-of-life instruments: attributes and review criteria. Quality of Life Research, 11(3), 193–205.
Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–s11. doi:10.1097/01.mlr.0000258615.42478.55.
Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. doi:10.1016/j.jclinepi.2010.04.011.
Jensen, R. E., Potosky, A. L., Reeve, B. B., Hahn, E., Cella, D., Fries, J., et al. (2015). Validation of the PROMIS physical function measures in a diverse US population-based cohort of cancer patients. Quality of Life Research,. doi:10.1007/s11136-015-0992-9.
Hahn, E. A., Cella, D., Dobrez, D., Shiomoto, G., Marcus, E., Taylor, S. G., et al. (2004). The talking touchscreen: a new approach to outcomes assessment in low literacy. Psycho-Oncology, 13(2), 86–95.
Chen, R. C., Carpenter, W. R., Kim, M., Hendrix, L. H., Agans, R. P., Meyer, A. M., et al. (2015). Design of the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study (NC ProCESS). Journal of Comparative Effectiveness Research, 4(1), 3–9. doi:10.2217/cer.14.67.
Greenberg, C. C., Wind, J. K., Chang, G. J., Chen, R. C., & Schrag, D. (2013). Stakeholder engagement for comparative effectiveness research in cancer care: Experience of the DEcIDE cancer consortium. Journal of Comparative Effectiveness Research, 2(2), 117–125. doi:10.2217/cer.12.80.
Gershon, R. C., Rothrock, N., Hanrahan, R., Bass, M., & Cella, D. (2010). The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. Journal of Applied Measurement, 11(3), 304–314.
Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–31. doi:10.1097/01.mlr.0000250483.85507.04.
Riley, W. T., Rothrock, N., Bruce, B., Christodolou, C., Cook, K., Hahn, E. A., et al. (2010). Patient-reported outcomes measurement information system (PROMIS) domain names and definitions revisions: further evaluation of content validity in IRT-derived item banks. Quality of Life Research, 19(9), 1311–1321. doi:10.1007/s11136-010-9694-5.
Ware, J. E., Kosinski, M., Turner-Bowker, D., & Gandek, B. (2002). User’s manual for the SF-12v2 ® health survey (With a supplement documenting SF-12 ® health survey). Lincoln: QualityMetric Incorporated.
Ware, J. E., Kosinski, M., Turner-Bowker, D. M., & Gandek, B. (2002). How to score version 2 of the SF-12 health survey (with a supplement documenting version 1). Lincoln: QualityMetric Incorporated.
Farivar, S. S., Cunningham, W. E., & Hays, R. D. (2007). Correlated physical and mental health summary scores for the SF-36 and SF-12 Health Survey, V.I. Health and Quality of Life Outcomes, 5, 54. doi:10.1186/1477-7525-5-54.
Roth, A. J., Rosenfeld, B., Kornblith, A. B., Gibson, C., Scher, H. I., Curley-Smart, T., et al. (2003). The memorial anxiety scale for prostate cancer: Validation of a new scale to measure anxiety in men with with prostate cancer. Cancer, 97(11), 2910–2918. doi:10.1002/cncr.11386.
Roth, A., Nelson, C. J., Rosenfeld, B., Warshowski, A., O’Shea, N., Scher, H., et al. (2006). Assessing anxiety in men with prostate cancer: further data on the reliability and validity of the Memorial Anxiety Scale for Prostate Cancer (MAX-PC). Psychosomatics, 47(4), 340–347. doi:10.1176/appi.psy.47.4.340.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.
Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use. Oxford: Oxford University Press.
Nunnally, J. (1978). Psychometric methods. New York: McGraw-Hill.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.
Samejima F (1997). Graded response model. In Handbook of modern item response theory (pp. 85–100) Springer.
Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City, IA
Browne, M. W., Cudeck, R., Bollen, K. A., & Long, J. S. (1993). Alternative ways of assessing model fit (Vol. 154). Newbury Park: Sage Publications.
Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64.
Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298.
Kang, T., & Chen, T. T. (2011). Performance of the generalized S-X2 item fit index for the graded response model. Asia Pacific Education Review, 12(1), 89–96. doi:10.1007/s12564-010-9082-4.
Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.
Cai, L., Yang, J. S., & Hansen, M. (2011). Generalized full-information item bifactor analysis. Psychological Methods, 16(3), 221–248.
Ten Berge, J. M., & Sočan, G. (2004). The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69(4), 613–625.
Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137–143.
Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B, (Methodological), 57(1), 289–300.
Williams, V. S., Jones, L. V., & Tukey, J. W. (1999). Controlling error in multiple comparisons, with examples from state-to-state differences in educational achievement. Journal of Educational and Behavioral Statistics, 24(1), 42–69.
Watt, T., Groenvold, M., Hegedüs, L., Bonnema, S. J., Rasmussen, Å. K., Feldt-Rasmussen, U., et al. (2014). Few items in the thyroid-related quality of life instrument ThyPRO exhibited differential item functioning. Quality of Life Research, 23(1), 327–338.
Dancey, C., & Reidy, J. (2004). Statistics without maths for psychology: Using SPSS for windows (3rd ed.). Harlow: Prentice Hall.
Yost, K. J., Eton, D. T., Garcia, S. F., & Cella, D. (2011). Minimally important differences were estimated for six patient-reported outcomes measurement information system-cancer scales in advanced-stage cancer patients. Journal of Clinical Epidemiology, 64(5), 507–516. doi:10.1016/j.jclinepi.2010.11.018.
Emons, W. H., Sijtsma, K., & Meijer, R. R. (2007). On the consistency of individual classification using short scales. Psychological Methods, 12(1), 105–120. doi:10.1037/1082-989x.12.1.105.
Kruyen, P. M., Emons, W. H., & Sijtsma, K. (2013). Assessing individual change using short tests and questionnaires. Applied Psychological Measurement,. doi:10.1177/0146621613510061.
Heene, M., Bollmann, S., & Bühner, M. (2014). Much ado about nothing, or much to do about something? effects of scale shortening on criterion validity and mean differences. Journal of Individual Differences, 35(4), 245–249. doi:10.1027/1614-0001/a000146.
Hampson, L. A., Cowan, J. E., Zhao, S., Carroll, P. R., & Cooperberg, M. R. (2015). Impact of age on quality-of-life outcomes after treatment for localized prostate cancer. European Urology,. doi:10.1016/j.eururo.2015.01.008.
Penson, D. F., Stoddard, M. L., Pasta, D. J., Lubeck, D. P., Flanders, S. C., & Litwin, M. S. (2001). The association between socioeconomic status, health insurance coverage, and quality of life in men with prostate cancer. Journal of Clinical Epidemiology, 54(4), 350–358.
Eton, D. T., Lepore, S. J., & Helgeson, V. S. (2001). Early quality of life in patients with localized prostate carcinoma: an examination of treatment-related, demographic, and psychosocial factors. Cancer, 92(6), 1451–1459.
Given, B., Given, C., Azzouz, F., & Stommel, M. (2001). Physical functioning of elderly cancer patients prior to diagnosis and following initial treatment. Nursing Research, 50(4), 222–232.
Jayadevappa, R., Johnson, J. C., Chhatre, S., Wein, A. J., & Malkowicz, S. B. (2007). Ethnic variation in return to baseline values of patient-reported outcomes in older prostate cancer patients. Cancer, 109(11), 2229–2238. doi:10.1002/cncr.22675.
Jayadevappa, R., Chhatre, S., Wein, A. J., & Malkowicz, S. B. (2009). Predictors of patient reported outcomes and cost of care in younger men with newly diagnosed prostate cancer. Prostate, 69(10), 1067–1076. doi:10.1002/pros.20955.
Brassell, S. A., Elsamanoudi, S. I., Cullen, J., Williams, M. E., & McLeod, D. G. (2013). Health-related quality of life for men with prostate cancer–an evaluation of outcomes 12–24 months after treatment. Urologic Oncology, 31(8), 1504–1510. doi:10.1016/j.urolonc.2012.04.008.
Rothrock, N. E., Hays, R. D., Spritzer, K., Yount, S. E., Riley, W., & Cella, D. (2010). Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology, 63(11), 1195–1204. doi:10.1016/j.jclinepi.2010.04.012.
Litwin, M. S., Hays, R. D., Fink, A., Ganz, P. A., Leake, B., Leach, G. E., et al. (1995). Quality-of-life outcomes in men treated for localized prostate cancer. JAMA, 273(2), 129–135.
Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E, Jr. (2014). Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. Journal of Clinical Epidemiology, 67(1), 108–113. doi:10.1016/j.jclinepi.2013.07.016.
PROMIS Health Organization and PROMIS Cooperative Group (2011). PROMIS instrument-level statistics including gender, education level, age bracket, clinical, and levels of self-rated general health subgroups. http://www.nihpromis.org/science/validitystudies.
Lubeck, D. P., Kim, H., Grossfeld, G., Ray, P., Penson, D. F., Flanders, S. C., et al. (2001). Health related quality of life differences between black and white men with prostate cancer: data from the cancer of the prostate strategic urologic research endeavor. Journal of Urology, 166(6), 2281–2285.
Grandner, M. A., Martin, J. L., Patel, N. P., Jackson, N. P., Gehrman, P. R., Pien, G., et al. (2012). Age and sleep disturbances among American men and women: data from the U.S. behavioral risk factor surveillance system. Sleep, 35(3), 395–406. doi:10.5665/sleep.1704.
Piccolo, R. S., Yang, M., Bliwise, D. L., Yaggi, H. K., & Araujo, A. B. (2013). Racial and socioeconomic disparities in sleep and chronic disease: results of a longitudinal investigation. Ethnicity and Disease, 23(4), 499–507.
Funding
This research was supported by grants from the Agency for Healthcare Research and Quality (HHSA29020050040ITO6) and the National Cancer Institute (R01CA174453).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest related to this research.
Research involving human participants and/or animals
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Rights and permissions
About this article
Cite this article
Quach, C.W., Langer, M.M., Chen, R.C. et al. Reliability and validity of PROMIS measures administered by telephone interview in a longitudinal localized prostate cancer study. Qual Life Res 25, 2811–2823 (2016). https://doi.org/10.1007/s11136-016-1325-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-016-1325-3