Quality of Life Research

, Volume 25, Issue 11, pp 2811–2823 | Cite as

Reliability and validity of PROMIS measures administered by telephone interview in a longitudinal localized prostate cancer study

  • Caroleen W. Quach
  • Michelle M. Langer
  • Ronald C. Chen
  • David Thissen
  • Deborah S. Usinger
  • Marc A. Emerson
  • Bryce B. Reeve



To evaluate the reliability and validity of six PROMIS measures (anxiety, depression, fatigue, pain interference, physical function, and sleep disturbance) telephone-administered to a diverse, population-based cohort of localized prostate cancer patients.


Newly diagnosed men were enrolled in the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study. PROMIS measures were telephone-administered pre-treatment (baseline), and at 3-months and 12-months post-treatment initiation (N = 778). Reliability was evaluated using Cronbach’s alpha. Dimensionality was examined with bifactor models and explained common variance (ECV). Ordinal logistic regression models were used to detect potential differential item functioning (DIF) for key demographic groups. Convergent and discriminant validity were assessed by correlations with the legacy instruments Memorial Anxiety Scale for Prostate Cancer and SF-12v2. Known-groups validity was examined by age, race/ethnicity, comorbidity, and treatment.


Each PROMIS measure had high Cronbach’s alpha values (0.86–0.96) and was sufficiently unidimensional. Floor effects were observed for anxiety, depression, and pain interference measures; ceiling effects were observed for physical function. No DIF was detected. Convergent validity was established with moderate to strong correlations between PROMIS and legacy measures (0.41–0.77) of similar constructs. Discriminant validity was demonstrated with weak correlations between measures of dissimilar domains (−0.20–−0.31). PROMIS measures detected differences across age, race/ethnicity, and comorbidity groups; no differences were found by treatment.


This study provides support for the reliability and construct validity of six PROMIS measures in prostate cancer, as well as the utility of telephone administration for assessing HRQoL in low literacy and hard-to-reach populations.


Prostate cancer Reliability Validity Psychometric validation Comparative effectiveness research 



This research was supported by grants from the Agency for Healthcare Research and Quality (HHSA29020050040ITO6) and the National Cancer Institute (R01CA174453).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflicts of interest related to this research.

Research involving human participants and/or animals

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    National Cancer Institute (2015). SEER Cancer Statistics Factsheets: Prostate Cancer.
  2. 2.
  3. 3.
    Wilt, T. J., MacDonald, R., Rutks, I., Shamliyan, T. A., Taylor, B. C., & Kane, R. L. (2008). Systematic review: Comparative effectiveness and harms of treatments for clinically localized prostate cancer. Annals of Internal Medicine, 148(6), 435–448. doi: 10.7326/0003-4819-148-6-200803180-00209.PubMedCrossRefGoogle Scholar
  4. 4.
    Xiong, T., Turner, R. M., Wei, Y., Neal, D. E., Lyratzopoulos, G., & Higgins, J. P. (2014). Comparative efficacy and safety of treatments for localised prostate cancer: An application of network meta-analysis. BMJ Open, 4(5), e004285. doi: 10.1136/bmjopen-2013-004285.PubMedPubMedCentralCrossRefGoogle Scholar
  5. 5.
    Sun F, Oyesanmi O, Fontanarosa J, Reston J, Guzzo T, & Schoelles K (December 2014). Therapies for Clinically Localized Prostate Cancer: Update of a 2008 Systematic Review. Comparative Effectiveness Review No. 146. (Prepared by the ECRI Institute–Penn Medicine Evidence-based Practice Center under Contract No. 290-2007-10063.) AHRQ Publication No. 15-EHC004-EF. Rockville, MD: Agency for Healthcare Research and Quality.Google Scholar
  6. 6.
    Hoffman, R. M., Penson, D. F., Zietman, A. L., & Barry, M. J. (2013). Comparative effectiveness research in localized prostate cancer treatment. Journal of Comparative Effectiveness Research, 2(6), 583–593. doi: 10.2217/cer.13.66.PubMedCrossRefGoogle Scholar
  7. 7.
    Aaronson, N., Alonso, J., Burnam, A., Lohr, K. N., Patrick, D. L., Perrin, E., et al. (2002). Assessing health status and quality-of-life instruments: attributes and review criteria. Quality of Life Research, 11(3), 193–205.PubMedCrossRefGoogle Scholar
  8. 8.
    Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–s11. doi: 10.1097/01.mlr.0000258615.42478.55.PubMedPubMedCentralCrossRefGoogle Scholar
  9. 9.
    Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. doi: 10.1016/j.jclinepi.2010.04.011.PubMedPubMedCentralCrossRefGoogle Scholar
  10. 10.
    Jensen, R. E., Potosky, A. L., Reeve, B. B., Hahn, E., Cella, D., Fries, J., et al. (2015). Validation of the PROMIS physical function measures in a diverse US population-based cohort of cancer patients. Quality of Life Research,. doi: 10.1007/s11136-015-0992-9.Google Scholar
  11. 11.
    Hahn, E. A., Cella, D., Dobrez, D., Shiomoto, G., Marcus, E., Taylor, S. G., et al. (2004). The talking touchscreen: a new approach to outcomes assessment in low literacy. Psycho-Oncology, 13(2), 86–95.PubMedCrossRefGoogle Scholar
  12. 12.
    Chen, R. C., Carpenter, W. R., Kim, M., Hendrix, L. H., Agans, R. P., Meyer, A. M., et al. (2015). Design of the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study (NC ProCESS). Journal of Comparative Effectiveness Research, 4(1), 3–9. doi: 10.2217/cer.14.67.PubMedCrossRefGoogle Scholar
  13. 13.
    Greenberg, C. C., Wind, J. K., Chang, G. J., Chen, R. C., & Schrag, D. (2013). Stakeholder engagement for comparative effectiveness research in cancer care: Experience of the DEcIDE cancer consortium. Journal of Comparative Effectiveness Research, 2(2), 117–125. doi: 10.2217/cer.12.80.PubMedCrossRefGoogle Scholar
  14. 14.
    Gershon, R. C., Rothrock, N., Hanrahan, R., Bass, M., & Cella, D. (2010). The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. Journal of Applied Measurement, 11(3), 304–314.PubMedPubMedCentralGoogle Scholar
  15. 15.
    Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–31. doi: 10.1097/01.mlr.0000250483.85507.04.PubMedCrossRefGoogle Scholar
  16. 16.
    Riley, W. T., Rothrock, N., Bruce, B., Christodolou, C., Cook, K., Hahn, E. A., et al. (2010). Patient-reported outcomes measurement information system (PROMIS) domain names and definitions revisions: further evaluation of content validity in IRT-derived item banks. Quality of Life Research, 19(9), 1311–1321. doi: 10.1007/s11136-010-9694-5.PubMedPubMedCentralCrossRefGoogle Scholar
  17. 17.
    Ware, J. E., Kosinski, M., Turner-Bowker, D., & Gandek, B. (2002). User’s manual for the SF-12v2 ® health survey (With a supplement documenting SF-12 ® health survey). Lincoln: QualityMetric Incorporated.Google Scholar
  18. 18.
    Ware, J. E., Kosinski, M., Turner-Bowker, D. M., & Gandek, B. (2002). How to score version 2 of the SF-12 health survey (with a supplement documenting version 1). Lincoln: QualityMetric Incorporated.Google Scholar
  19. 19.
    Farivar, S. S., Cunningham, W. E., & Hays, R. D. (2007). Correlated physical and mental health summary scores for the SF-36 and SF-12 Health Survey, V.I. Health and Quality of Life Outcomes, 5, 54. doi: 10.1186/1477-7525-5-54.PubMedPubMedCentralCrossRefGoogle Scholar
  20. 20.
    Roth, A. J., Rosenfeld, B., Kornblith, A. B., Gibson, C., Scher, H. I., Curley-Smart, T., et al. (2003). The memorial anxiety scale for prostate cancer: Validation of a new scale to measure anxiety in men with with prostate cancer. Cancer, 97(11), 2910–2918. doi: 10.1002/cncr.11386.PubMedCrossRefGoogle Scholar
  21. 21.
    Roth, A., Nelson, C. J., Rosenfeld, B., Warshowski, A., O’Shea, N., Scher, H., et al. (2006). Assessing anxiety in men with prostate cancer: further data on the reliability and validity of the Memorial Anxiety Scale for Prostate Cancer (MAX-PC). Psychosomatics, 47(4), 340–347. doi: 10.1176/appi.psy.47.4.340.PubMedCrossRefGoogle Scholar
  22. 22.
    Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.CrossRefGoogle Scholar
  23. 23.
    Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use. Oxford: Oxford University Press.Google Scholar
  24. 24.
    Nunnally, J. (1978). Psychometric methods. New York: McGraw-Hill.Google Scholar
  25. 25.
    Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.Google Scholar
  26. 26.
    Samejima F (1997). Graded response model. In Handbook of modern item response theory (pp. 85–100) Springer.Google Scholar
  27. 27.
    Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City, IAGoogle Scholar
  28. 28.
    Browne, M. W., Cudeck, R., Bollen, K. A., & Long, J. S. (1993). Alternative ways of assessing model fit (Vol. 154). Newbury Park: Sage Publications.Google Scholar
  29. 29.
    Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64.CrossRefGoogle Scholar
  30. 30.
    Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298.CrossRefGoogle Scholar
  31. 31.
    Kang, T., & Chen, T. T. (2011). Performance of the generalized S-X2 item fit index for the graded response model. Asia Pacific Education Review, 12(1), 89–96. doi: 10.1007/s12564-010-9082-4.CrossRefGoogle Scholar
  32. 32.
    Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.CrossRefGoogle Scholar
  33. 33.
    Cai, L., Yang, J. S., & Hansen, M. (2011). Generalized full-information item bifactor analysis. Psychological Methods, 16(3), 221–248.PubMedPubMedCentralCrossRefGoogle Scholar
  34. 34.
    Ten Berge, J. M., & Sočan, G. (2004). The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69(4), 613–625.CrossRefGoogle Scholar
  35. 35.
    Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137–143.PubMedPubMedCentralCrossRefGoogle Scholar
  36. 36.
    Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559.PubMedPubMedCentralCrossRefGoogle Scholar
  37. 37.
    Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B, (Methodological), 57(1), 289–300.Google Scholar
  38. 38.
    Williams, V. S., Jones, L. V., & Tukey, J. W. (1999). Controlling error in multiple comparisons, with examples from state-to-state differences in educational achievement. Journal of Educational and Behavioral Statistics, 24(1), 42–69.CrossRefGoogle Scholar
  39. 39.
    Watt, T., Groenvold, M., Hegedüs, L., Bonnema, S. J., Rasmussen, Å. K., Feldt-Rasmussen, U., et al. (2014). Few items in the thyroid-related quality of life instrument ThyPRO exhibited differential item functioning. Quality of Life Research, 23(1), 327–338.PubMedCrossRefGoogle Scholar
  40. 40.
    Dancey, C., & Reidy, J. (2004). Statistics without maths for psychology: Using SPSS for windows (3rd ed.). Harlow: Prentice Hall.Google Scholar
  41. 41.
    Yost, K. J., Eton, D. T., Garcia, S. F., & Cella, D. (2011). Minimally important differences were estimated for six patient-reported outcomes measurement information system-cancer scales in advanced-stage cancer patients. Journal of Clinical Epidemiology, 64(5), 507–516. doi: 10.1016/j.jclinepi.2010.11.018.PubMedPubMedCentralCrossRefGoogle Scholar
  42. 42.
    Emons, W. H., Sijtsma, K., & Meijer, R. R. (2007). On the consistency of individual classification using short scales. Psychological Methods, 12(1), 105–120. doi: 10.1037/1082-989x.12.1.105.PubMedCrossRefGoogle Scholar
  43. 43.
    Kruyen, P. M., Emons, W. H., & Sijtsma, K. (2013). Assessing individual change using short tests and questionnaires. Applied Psychological Measurement,. doi: 10.1177/0146621613510061.Google Scholar
  44. 44.
    Heene, M., Bollmann, S., & Bühner, M. (2014). Much ado about nothing, or much to do about something? effects of scale shortening on criterion validity and mean differences. Journal of Individual Differences, 35(4), 245–249. doi: 10.1027/1614-0001/a000146.CrossRefGoogle Scholar
  45. 45.
    Hampson, L. A., Cowan, J. E., Zhao, S., Carroll, P. R., & Cooperberg, M. R. (2015). Impact of age on quality-of-life outcomes after treatment for localized prostate cancer. European Urology,. doi: 10.1016/j.eururo.2015.01.008.PubMedGoogle Scholar
  46. 46.
    Penson, D. F., Stoddard, M. L., Pasta, D. J., Lubeck, D. P., Flanders, S. C., & Litwin, M. S. (2001). The association between socioeconomic status, health insurance coverage, and quality of life in men with prostate cancer. Journal of Clinical Epidemiology, 54(4), 350–358.PubMedCrossRefGoogle Scholar
  47. 47.
    Eton, D. T., Lepore, S. J., & Helgeson, V. S. (2001). Early quality of life in patients with localized prostate carcinoma: an examination of treatment-related, demographic, and psychosocial factors. Cancer, 92(6), 1451–1459.PubMedPubMedCentralCrossRefGoogle Scholar
  48. 48.
    Given, B., Given, C., Azzouz, F., & Stommel, M. (2001). Physical functioning of elderly cancer patients prior to diagnosis and following initial treatment. Nursing Research, 50(4), 222–232.PubMedCrossRefGoogle Scholar
  49. 49.
    Jayadevappa, R., Johnson, J. C., Chhatre, S., Wein, A. J., & Malkowicz, S. B. (2007). Ethnic variation in return to baseline values of patient-reported outcomes in older prostate cancer patients. Cancer, 109(11), 2229–2238. doi: 10.1002/cncr.22675.PubMedCrossRefGoogle Scholar
  50. 50.
    Jayadevappa, R., Chhatre, S., Wein, A. J., & Malkowicz, S. B. (2009). Predictors of patient reported outcomes and cost of care in younger men with newly diagnosed prostate cancer. Prostate, 69(10), 1067–1076. doi: 10.1002/pros.20955.PubMedCrossRefGoogle Scholar
  51. 51.
    Brassell, S. A., Elsamanoudi, S. I., Cullen, J., Williams, M. E., & McLeod, D. G. (2013). Health-related quality of life for men with prostate cancer–an evaluation of outcomes 12–24 months after treatment. Urologic Oncology, 31(8), 1504–1510. doi: 10.1016/j.urolonc.2012.04.008.PubMedCrossRefGoogle Scholar
  52. 52.
    Rothrock, N. E., Hays, R. D., Spritzer, K., Yount, S. E., Riley, W., & Cella, D. (2010). Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology, 63(11), 1195–1204. doi: 10.1016/j.jclinepi.2010.04.012.PubMedPubMedCentralCrossRefGoogle Scholar
  53. 53.
    Litwin, M. S., Hays, R. D., Fink, A., Ganz, P. A., Leake, B., Leach, G. E., et al. (1995). Quality-of-life outcomes in men treated for localized prostate cancer. JAMA, 273(2), 129–135.PubMedCrossRefGoogle Scholar
  54. 54.
    Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E, Jr. (2014). Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. Journal of Clinical Epidemiology, 67(1), 108–113. doi: 10.1016/j.jclinepi.2013.07.016.PubMedPubMedCentralCrossRefGoogle Scholar
  55. 55.
    PROMIS Health Organization and PROMIS Cooperative Group (2011). PROMIS instrument-level statistics including gender, education level, age bracket, clinical, and levels of self-rated general health subgroups.
  56. 56.
    Lubeck, D. P., Kim, H., Grossfeld, G., Ray, P., Penson, D. F., Flanders, S. C., et al. (2001). Health related quality of life differences between black and white men with prostate cancer: data from the cancer of the prostate strategic urologic research endeavor. Journal of Urology, 166(6), 2281–2285.PubMedCrossRefGoogle Scholar
  57. 57.
    Grandner, M. A., Martin, J. L., Patel, N. P., Jackson, N. P., Gehrman, P. R., Pien, G., et al. (2012). Age and sleep disturbances among American men and women: data from the U.S. behavioral risk factor surveillance system. Sleep, 35(3), 395–406. doi: 10.5665/sleep.1704.PubMedPubMedCentralGoogle Scholar
  58. 58.
    Piccolo, R. S., Yang, M., Bliwise, D. L., Yaggi, H. K., & Araujo, A. B. (2013). Racial and socioeconomic disparities in sleep and chronic disease: results of a longitudinal investigation. Ethnicity and Disease, 23(4), 499–507.PubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Caroleen W. Quach
    • 1
  • Michelle M. Langer
    • 2
    • 7
  • Ronald C. Chen
    • 2
    • 3
    • 4
  • David Thissen
    • 5
  • Deborah S. Usinger
    • 2
    • 3
  • Marc A. Emerson
    • 6
  • Bryce B. Reeve
    • 1
    • 2
  1. 1.Department of Health Policy and Management, Gillings School of Global Public HealthUniversity of North Carolina at Chapel HillChapel HillUSA
  2. 2.Lineberger Comprehensive Cancer CenterUniversity of North Carolina at Chapel HillChapel HillUSA
  3. 3.Cecil G. Sheps Center for Health Services ResearchUniversity of North Carolina at Chapel HillChapel HillUSA
  4. 4.Department of Radiation Oncology, School of MedicineUniversity of North Carolina at Chapel HillChapel HillUSA
  5. 5.Department of Psychology and NeuroscienceUniversity of North Carolina at Chapel HillChapel HillUSA
  6. 6.Department of Epidemiology, Gillings School of Global Public HealthUniversity of North Carolina at Chapel HillChapel HillUSA
  7. 7.American Institutes of ResearchChapel HillUSA

Personalised recommendations