Reliability and validity of PROMIS measures administered by telephone interview in a longitudinal localized prostate cancer study
- 599 Downloads
To evaluate the reliability and validity of six PROMIS measures (anxiety, depression, fatigue, pain interference, physical function, and sleep disturbance) telephone-administered to a diverse, population-based cohort of localized prostate cancer patients.
Newly diagnosed men were enrolled in the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study. PROMIS measures were telephone-administered pre-treatment (baseline), and at 3-months and 12-months post-treatment initiation (N = 778). Reliability was evaluated using Cronbach’s alpha. Dimensionality was examined with bifactor models and explained common variance (ECV). Ordinal logistic regression models were used to detect potential differential item functioning (DIF) for key demographic groups. Convergent and discriminant validity were assessed by correlations with the legacy instruments Memorial Anxiety Scale for Prostate Cancer and SF-12v2. Known-groups validity was examined by age, race/ethnicity, comorbidity, and treatment.
Each PROMIS measure had high Cronbach’s alpha values (0.86–0.96) and was sufficiently unidimensional. Floor effects were observed for anxiety, depression, and pain interference measures; ceiling effects were observed for physical function. No DIF was detected. Convergent validity was established with moderate to strong correlations between PROMIS and legacy measures (0.41–0.77) of similar constructs. Discriminant validity was demonstrated with weak correlations between measures of dissimilar domains (−0.20–−0.31). PROMIS measures detected differences across age, race/ethnicity, and comorbidity groups; no differences were found by treatment.
This study provides support for the reliability and construct validity of six PROMIS measures in prostate cancer, as well as the utility of telephone administration for assessing HRQoL in low literacy and hard-to-reach populations.
KeywordsProstate cancer Reliability Validity Psychometric validation Comparative effectiveness research
This research was supported by grants from the Agency for Healthcare Research and Quality (HHSA29020050040ITO6) and the National Cancer Institute (R01CA174453).
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflicts of interest related to this research.
Research involving human participants and/or animals
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed consent was obtained from all individual participants included in the study.
- 1.National Cancer Institute (2015). SEER Cancer Statistics Factsheets: Prostate Cancer. http://seer.cancer.gov/statfacts/html/prost.html.
- 2.American Cancer Society (2015). Prostate Cancer Overview. http://www.cancer.org/cancer/prostatecancer/detailedguide/prostate-cancer-what-is-prostate-cancer.
- 3.Wilt, T. J., MacDonald, R., Rutks, I., Shamliyan, T. A., Taylor, B. C., & Kane, R. L. (2008). Systematic review: Comparative effectiveness and harms of treatments for clinically localized prostate cancer. Annals of Internal Medicine, 148(6), 435–448. doi: 10.7326/0003-4819-148-6-200803180-00209.PubMedCrossRefGoogle Scholar
- 4.Xiong, T., Turner, R. M., Wei, Y., Neal, D. E., Lyratzopoulos, G., & Higgins, J. P. (2014). Comparative efficacy and safety of treatments for localised prostate cancer: An application of network meta-analysis. BMJ Open, 4(5), e004285. doi: 10.1136/bmjopen-2013-004285.PubMedPubMedCentralCrossRefGoogle Scholar
- 5.Sun F, Oyesanmi O, Fontanarosa J, Reston J, Guzzo T, & Schoelles K (December 2014). Therapies for Clinically Localized Prostate Cancer: Update of a 2008 Systematic Review. Comparative Effectiveness Review No. 146. (Prepared by the ECRI Institute–Penn Medicine Evidence-based Practice Center under Contract No. 290-2007-10063.) AHRQ Publication No. 15-EHC004-EF. Rockville, MD: Agency for Healthcare Research and Quality.Google Scholar
- 8.Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–s11. doi: 10.1097/01.mlr.0000258615.42478.55.PubMedPubMedCentralCrossRefGoogle Scholar
- 9.Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. doi: 10.1016/j.jclinepi.2010.04.011.PubMedPubMedCentralCrossRefGoogle Scholar
- 12.Chen, R. C., Carpenter, W. R., Kim, M., Hendrix, L. H., Agans, R. P., Meyer, A. M., et al. (2015). Design of the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study (NC ProCESS). Journal of Comparative Effectiveness Research, 4(1), 3–9. doi: 10.2217/cer.14.67.PubMedCrossRefGoogle Scholar
- 13.Greenberg, C. C., Wind, J. K., Chang, G. J., Chen, R. C., & Schrag, D. (2013). Stakeholder engagement for comparative effectiveness research in cancer care: Experience of the DEcIDE cancer consortium. Journal of Comparative Effectiveness Research, 2(2), 117–125. doi: 10.2217/cer.12.80.PubMedCrossRefGoogle Scholar
- 15.Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–31. doi: 10.1097/01.mlr.0000250483.85507.04.PubMedCrossRefGoogle Scholar
- 16.Riley, W. T., Rothrock, N., Bruce, B., Christodolou, C., Cook, K., Hahn, E. A., et al. (2010). Patient-reported outcomes measurement information system (PROMIS) domain names and definitions revisions: further evaluation of content validity in IRT-derived item banks. Quality of Life Research, 19(9), 1311–1321. doi: 10.1007/s11136-010-9694-5.PubMedPubMedCentralCrossRefGoogle Scholar
- 17.Ware, J. E., Kosinski, M., Turner-Bowker, D., & Gandek, B. (2002). User’s manual for the SF-12v2 ® health survey (With a supplement documenting SF-12 ® health survey). Lincoln: QualityMetric Incorporated.Google Scholar
- 18.Ware, J. E., Kosinski, M., Turner-Bowker, D. M., & Gandek, B. (2002). How to score version 2 of the SF-12 health survey (with a supplement documenting version 1). Lincoln: QualityMetric Incorporated.Google Scholar
- 20.Roth, A. J., Rosenfeld, B., Kornblith, A. B., Gibson, C., Scher, H. I., Curley-Smart, T., et al. (2003). The memorial anxiety scale for prostate cancer: Validation of a new scale to measure anxiety in men with with prostate cancer. Cancer, 97(11), 2910–2918. doi: 10.1002/cncr.11386.PubMedCrossRefGoogle Scholar
- 21.Roth, A., Nelson, C. J., Rosenfeld, B., Warshowski, A., O’Shea, N., Scher, H., et al. (2006). Assessing anxiety in men with prostate cancer: further data on the reliability and validity of the Memorial Anxiety Scale for Prostate Cancer (MAX-PC). Psychosomatics, 47(4), 340–347. doi: 10.1176/appi.psy.47.4.340.PubMedCrossRefGoogle Scholar
- 23.Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use. Oxford: Oxford University Press.Google Scholar
- 24.Nunnally, J. (1978). Psychometric methods. New York: McGraw-Hill.Google Scholar
- 25.Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.Google Scholar
- 26.Samejima F (1997). Graded response model. In Handbook of modern item response theory (pp. 85–100) Springer.Google Scholar
- 27.Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City, IAGoogle Scholar
- 28.Browne, M. W., Cudeck, R., Bollen, K. A., & Long, J. S. (1993). Alternative ways of assessing model fit (Vol. 154). Newbury Park: Sage Publications.Google Scholar
- 37.Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B, (Methodological), 57(1), 289–300.Google Scholar
- 40.Dancey, C., & Reidy, J. (2004). Statistics without maths for psychology: Using SPSS for windows (3rd ed.). Harlow: Prentice Hall.Google Scholar
- 41.Yost, K. J., Eton, D. T., Garcia, S. F., & Cella, D. (2011). Minimally important differences were estimated for six patient-reported outcomes measurement information system-cancer scales in advanced-stage cancer patients. Journal of Clinical Epidemiology, 64(5), 507–516. doi: 10.1016/j.jclinepi.2010.11.018.PubMedPubMedCentralCrossRefGoogle Scholar
- 46.Penson, D. F., Stoddard, M. L., Pasta, D. J., Lubeck, D. P., Flanders, S. C., & Litwin, M. S. (2001). The association between socioeconomic status, health insurance coverage, and quality of life in men with prostate cancer. Journal of Clinical Epidemiology, 54(4), 350–358.PubMedCrossRefGoogle Scholar
- 51.Brassell, S. A., Elsamanoudi, S. I., Cullen, J., Williams, M. E., & McLeod, D. G. (2013). Health-related quality of life for men with prostate cancer–an evaluation of outcomes 12–24 months after treatment. Urologic Oncology, 31(8), 1504–1510. doi: 10.1016/j.urolonc.2012.04.008.PubMedCrossRefGoogle Scholar
- 52.Rothrock, N. E., Hays, R. D., Spritzer, K., Yount, S. E., Riley, W., & Cella, D. (2010). Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology, 63(11), 1195–1204. doi: 10.1016/j.jclinepi.2010.04.012.PubMedPubMedCentralCrossRefGoogle Scholar
- 54.Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E, Jr. (2014). Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. Journal of Clinical Epidemiology, 67(1), 108–113. doi: 10.1016/j.jclinepi.2013.07.016.PubMedPubMedCentralCrossRefGoogle Scholar
- 55.PROMIS Health Organization and PROMIS Cooperative Group (2011). PROMIS instrument-level statistics including gender, education level, age bracket, clinical, and levels of self-rated general health subgroups. http://www.nihpromis.org/science/validitystudies.
- 56.Lubeck, D. P., Kim, H., Grossfeld, G., Ray, P., Penson, D. F., Flanders, S. C., et al. (2001). Health related quality of life differences between black and white men with prostate cancer: data from the cancer of the prostate strategic urologic research endeavor. Journal of Urology, 166(6), 2281–2285.PubMedCrossRefGoogle Scholar
- 57.Grandner, M. A., Martin, J. L., Patel, N. P., Jackson, N. P., Gehrman, P. R., Pien, G., et al. (2012). Age and sleep disturbances among American men and women: data from the U.S. behavioral risk factor surveillance system. Sleep, 35(3), 395–406. doi: 10.5665/sleep.1704.PubMedPubMedCentralGoogle Scholar