Skip to main content

Advertisement

SpringerLink
Log in

Reliability and validity of PROMIS measures administered by telephone interview in a longitudinal localized prostate cancer study

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

To evaluate the reliability and validity of six PROMIS measures (anxiety, depression, fatigue, pain interference, physical function, and sleep disturbance) telephone-administered to a diverse, population-based cohort of localized prostate cancer patients.

Methods

Newly diagnosed men were enrolled in the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study. PROMIS measures were telephone-administered pre-treatment (baseline), and at 3-months and 12-months post-treatment initiation (N = 778). Reliability was evaluated using Cronbach’s alpha. Dimensionality was examined with bifactor models and explained common variance (ECV). Ordinal logistic regression models were used to detect potential differential item functioning (DIF) for key demographic groups. Convergent and discriminant validity were assessed by correlations with the legacy instruments Memorial Anxiety Scale for Prostate Cancer and SF-12v2. Known-groups validity was examined by age, race/ethnicity, comorbidity, and treatment.

Results

Each PROMIS measure had high Cronbach’s alpha values (0.86–0.96) and was sufficiently unidimensional. Floor effects were observed for anxiety, depression, and pain interference measures; ceiling effects were observed for physical function. No DIF was detected. Convergent validity was established with moderate to strong correlations between PROMIS and legacy measures (0.41–0.77) of similar constructs. Discriminant validity was demonstrated with weak correlations between measures of dissimilar domains (−0.20–−0.31). PROMIS measures detected differences across age, race/ethnicity, and comorbidity groups; no differences were found by treatment.

Conclusions

This study provides support for the reliability and construct validity of six PROMIS measures in prostate cancer, as well as the utility of telephone administration for assessing HRQoL in low literacy and hard-to-reach populations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. National Cancer Institute (2015). SEER Cancer Statistics Factsheets: Prostate Cancer. http://seer.cancer.gov/statfacts/html/prost.html.

  2. American Cancer Society (2015). Prostate Cancer Overview. http://www.cancer.org/cancer/prostatecancer/detailedguide/prostate-cancer-what-is-prostate-cancer.

  3. Wilt, T. J., MacDonald, R., Rutks, I., Shamliyan, T. A., Taylor, B. C., & Kane, R. L. (2008). Systematic review: Comparative effectiveness and harms of treatments for clinically localized prostate cancer. Annals of Internal Medicine, 148(6), 435–448. doi:10.7326/0003-4819-148-6-200803180-00209.

    Article  PubMed  Google Scholar 

  4. Xiong, T., Turner, R. M., Wei, Y., Neal, D. E., Lyratzopoulos, G., & Higgins, J. P. (2014). Comparative efficacy and safety of treatments for localised prostate cancer: An application of network meta-analysis. BMJ Open, 4(5), e004285. doi:10.1136/bmjopen-2013-004285.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Sun F, Oyesanmi O, Fontanarosa J, Reston J, Guzzo T, & Schoelles K (December 2014). Therapies for Clinically Localized Prostate Cancer: Update of a 2008 Systematic Review. Comparative Effectiveness Review No. 146. (Prepared by the ECRI Institute–Penn Medicine Evidence-based Practice Center under Contract No. 290-2007-10063.) AHRQ Publication No. 15-EHC004-EF. Rockville, MD: Agency for Healthcare Research and Quality.

  6. Hoffman, R. M., Penson, D. F., Zietman, A. L., & Barry, M. J. (2013). Comparative effectiveness research in localized prostate cancer treatment. Journal of Comparative Effectiveness Research, 2(6), 583–593. doi:10.2217/cer.13.66.

    Article  PubMed  Google Scholar 

  7. Aaronson, N., Alonso, J., Burnam, A., Lohr, K. N., Patrick, D. L., Perrin, E., et al. (2002). Assessing health status and quality-of-life instruments: attributes and review criteria. Quality of Life Research, 11(3), 193–205.

    Article  PubMed  Google Scholar 

  8. Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., et al. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–s11. doi:10.1097/01.mlr.0000258615.42478.55.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., et al. (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. doi:10.1016/j.jclinepi.2010.04.011.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Jensen, R. E., Potosky, A. L., Reeve, B. B., Hahn, E., Cella, D., Fries, J., et al. (2015). Validation of the PROMIS physical function measures in a diverse US population-based cohort of cancer patients. Quality of Life Research,. doi:10.1007/s11136-015-0992-9.

    Google Scholar 

  11. Hahn, E. A., Cella, D., Dobrez, D., Shiomoto, G., Marcus, E., Taylor, S. G., et al. (2004). The talking touchscreen: a new approach to outcomes assessment in low literacy. Psycho-Oncology, 13(2), 86–95.

    Article  PubMed  Google Scholar 

  12. Chen, R. C., Carpenter, W. R., Kim, M., Hendrix, L. H., Agans, R. P., Meyer, A. M., et al. (2015). Design of the North Carolina Prostate Cancer Comparative Effectiveness and Survivorship Study (NC ProCESS). Journal of Comparative Effectiveness Research, 4(1), 3–9. doi:10.2217/cer.14.67.

    Article  PubMed  Google Scholar 

  13. Greenberg, C. C., Wind, J. K., Chang, G. J., Chen, R. C., & Schrag, D. (2013). Stakeholder engagement for comparative effectiveness research in cancer care: Experience of the DEcIDE cancer consortium. Journal of Comparative Effectiveness Research, 2(2), 117–125. doi:10.2217/cer.12.80.

    Article  PubMed  Google Scholar 

  14. Gershon, R. C., Rothrock, N., Hanrahan, R., Bass, M., & Cella, D. (2010). The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. Journal of Applied Measurement, 11(3), 304–314.

    PubMed  PubMed Central  Google Scholar 

  15. Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–31. doi:10.1097/01.mlr.0000250483.85507.04.

    Article  PubMed  Google Scholar 

  16. Riley, W. T., Rothrock, N., Bruce, B., Christodolou, C., Cook, K., Hahn, E. A., et al. (2010). Patient-reported outcomes measurement information system (PROMIS) domain names and definitions revisions: further evaluation of content validity in IRT-derived item banks. Quality of Life Research, 19(9), 1311–1321. doi:10.1007/s11136-010-9694-5.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Ware, J. E., Kosinski, M., Turner-Bowker, D., & Gandek, B. (2002). User’s manual for the SF-12v2 ® health survey (With a supplement documenting SF-12 ® health survey). Lincoln: QualityMetric Incorporated.

    Google Scholar 

  18. Ware, J. E., Kosinski, M., Turner-Bowker, D. M., & Gandek, B. (2002). How to score version 2 of the SF-12 health survey (with a supplement documenting version 1). Lincoln: QualityMetric Incorporated.

    Google Scholar 

  19. Farivar, S. S., Cunningham, W. E., & Hays, R. D. (2007). Correlated physical and mental health summary scores for the SF-36 and SF-12 Health Survey, V.I. Health and Quality of Life Outcomes, 5, 54. doi:10.1186/1477-7525-5-54.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Roth, A. J., Rosenfeld, B., Kornblith, A. B., Gibson, C., Scher, H. I., Curley-Smart, T., et al. (2003). The memorial anxiety scale for prostate cancer: Validation of a new scale to measure anxiety in men with with prostate cancer. Cancer, 97(11), 2910–2918. doi:10.1002/cncr.11386.

    Article  PubMed  Google Scholar 

  21. Roth, A., Nelson, C. J., Rosenfeld, B., Warshowski, A., O’Shea, N., Scher, H., et al. (2006). Assessing anxiety in men with prostate cancer: further data on the reliability and validity of the Memorial Anxiety Scale for Prostate Cancer (MAX-PC). Psychosomatics, 47(4), 340–347. doi:10.1176/appi.psy.47.4.340.

    Article  PubMed  Google Scholar 

  22. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.

    Article  Google Scholar 

  23. Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use. Oxford: Oxford University Press.

    Google Scholar 

  24. Nunnally, J. (1978). Psychometric methods. New York: McGraw-Hill.

    Google Scholar 

  25. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.

  26. Samejima F (1997). Graded response model. In Handbook of modern item response theory (pp. 85–100) Springer.

  27. Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society, Iowa City, IA

  28. Browne, M. W., Cudeck, R., Bollen, K. A., & Long, J. S. (1993). Alternative ways of assessing model fit (Vol. 154). Newbury Park: Sage Publications.

    Google Scholar 

  29. Orlando, M., & Thissen, D. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64.

    Article  Google Scholar 

  30. Orlando, M., & Thissen, D. (2003). Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Applied Psychological Measurement, 27(4), 289–298.

    Article  Google Scholar 

  31. Kang, T., & Chen, T. T. (2011). Performance of the generalized S-X2 item fit index for the graded response model. Asia Pacific Education Review, 12(1), 89–96. doi:10.1007/s12564-010-9082-4.

    Article  Google Scholar 

  32. Chen, W.-H., & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22(3), 265–289.

    Article  Google Scholar 

  33. Cai, L., Yang, J. S., & Hansen, M. (2011). Generalized full-information item bifactor analysis. Psychological Methods, 16(3), 221–248.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Ten Berge, J. M., & Sočan, G. (2004). The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69(4), 613–625.

    Article  Google Scholar 

  35. Bentler, P. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137–143.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B, (Methodological), 57(1), 289–300.

    Google Scholar 

  38. Williams, V. S., Jones, L. V., & Tukey, J. W. (1999). Controlling error in multiple comparisons, with examples from state-to-state differences in educational achievement. Journal of Educational and Behavioral Statistics, 24(1), 42–69.

    Article  Google Scholar 

  39. Watt, T., Groenvold, M., Hegedüs, L., Bonnema, S. J., Rasmussen, Å. K., Feldt-Rasmussen, U., et al. (2014). Few items in the thyroid-related quality of life instrument ThyPRO exhibited differential item functioning. Quality of Life Research, 23(1), 327–338.

    Article  PubMed  Google Scholar 

  40. Dancey, C., & Reidy, J. (2004). Statistics without maths for psychology: Using SPSS for windows (3rd ed.). Harlow: Prentice Hall.

    Google Scholar 

  41. Yost, K. J., Eton, D. T., Garcia, S. F., & Cella, D. (2011). Minimally important differences were estimated for six patient-reported outcomes measurement information system-cancer scales in advanced-stage cancer patients. Journal of Clinical Epidemiology, 64(5), 507–516. doi:10.1016/j.jclinepi.2010.11.018.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Emons, W. H., Sijtsma, K., & Meijer, R. R. (2007). On the consistency of individual classification using short scales. Psychological Methods, 12(1), 105–120. doi:10.1037/1082-989x.12.1.105.

    Article  PubMed  Google Scholar 

  43. Kruyen, P. M., Emons, W. H., & Sijtsma, K. (2013). Assessing individual change using short tests and questionnaires. Applied Psychological Measurement,. doi:10.1177/0146621613510061.

    Google Scholar 

  44. Heene, M., Bollmann, S., & Bühner, M. (2014). Much ado about nothing, or much to do about something? effects of scale shortening on criterion validity and mean differences. Journal of Individual Differences, 35(4), 245–249. doi:10.1027/1614-0001/a000146.

    Article  Google Scholar 

  45. Hampson, L. A., Cowan, J. E., Zhao, S., Carroll, P. R., & Cooperberg, M. R. (2015). Impact of age on quality-of-life outcomes after treatment for localized prostate cancer. European Urology,. doi:10.1016/j.eururo.2015.01.008.

    PubMed  Google Scholar 

  46. Penson, D. F., Stoddard, M. L., Pasta, D. J., Lubeck, D. P., Flanders, S. C., & Litwin, M. S. (2001). The association between socioeconomic status, health insurance coverage, and quality of life in men with prostate cancer. Journal of Clinical Epidemiology, 54(4), 350–358.

    Article  CAS  PubMed  Google Scholar 

  47. Eton, D. T., Lepore, S. J., & Helgeson, V. S. (2001). Early quality of life in patients with localized prostate carcinoma: an examination of treatment-related, demographic, and psychosocial factors. Cancer, 92(6), 1451–1459.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Given, B., Given, C., Azzouz, F., & Stommel, M. (2001). Physical functioning of elderly cancer patients prior to diagnosis and following initial treatment. Nursing Research, 50(4), 222–232.

    Article  CAS  PubMed  Google Scholar 

  49. Jayadevappa, R., Johnson, J. C., Chhatre, S., Wein, A. J., & Malkowicz, S. B. (2007). Ethnic variation in return to baseline values of patient-reported outcomes in older prostate cancer patients. Cancer, 109(11), 2229–2238. doi:10.1002/cncr.22675.

    Article  PubMed  Google Scholar 

  50. Jayadevappa, R., Chhatre, S., Wein, A. J., & Malkowicz, S. B. (2009). Predictors of patient reported outcomes and cost of care in younger men with newly diagnosed prostate cancer. Prostate, 69(10), 1067–1076. doi:10.1002/pros.20955.

    Article  PubMed  Google Scholar 

  51. Brassell, S. A., Elsamanoudi, S. I., Cullen, J., Williams, M. E., & McLeod, D. G. (2013). Health-related quality of life for men with prostate cancer–an evaluation of outcomes 12–24 months after treatment. Urologic Oncology, 31(8), 1504–1510. doi:10.1016/j.urolonc.2012.04.008.

    Article  PubMed  Google Scholar 

  52. Rothrock, N. E., Hays, R. D., Spritzer, K., Yount, S. E., Riley, W., & Cella, D. (2010). Relative to the general US population, chronic diseases are associated with poorer health-related quality of life as measured by the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology, 63(11), 1195–1204. doi:10.1016/j.jclinepi.2010.04.012.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Litwin, M. S., Hays, R. D., Fink, A., Ganz, P. A., Leake, B., Leach, G. E., et al. (1995). Quality-of-life outcomes in men treated for localized prostate cancer. JAMA, 273(2), 129–135.

    Article  CAS  PubMed  Google Scholar 

  54. Bjorner, J. B., Rose, M., Gandek, B., Stone, A. A., Junghaenel, D. U., & Ware, J. E, Jr. (2014). Method of administration of PROMIS scales did not significantly impact score level, reliability, or validity. Journal of Clinical Epidemiology, 67(1), 108–113. doi:10.1016/j.jclinepi.2013.07.016.

    Article  PubMed  PubMed Central  Google Scholar 

  55. PROMIS Health Organization and PROMIS Cooperative Group (2011). PROMIS instrument-level statistics including gender, education level, age bracket, clinical, and levels of self-rated general health subgroups. http://www.nihpromis.org/science/validitystudies.

  56. Lubeck, D. P., Kim, H., Grossfeld, G., Ray, P., Penson, D. F., Flanders, S. C., et al. (2001). Health related quality of life differences between black and white men with prostate cancer: data from the cancer of the prostate strategic urologic research endeavor. Journal of Urology, 166(6), 2281–2285.

    Article  CAS  PubMed  Google Scholar 

  57. Grandner, M. A., Martin, J. L., Patel, N. P., Jackson, N. P., Gehrman, P. R., Pien, G., et al. (2012). Age and sleep disturbances among American men and women: data from the U.S. behavioral risk factor surveillance system. Sleep, 35(3), 395–406. doi:10.5665/sleep.1704.

    PubMed  PubMed Central  Google Scholar 

  58. Piccolo, R. S., Yang, M., Bliwise, D. L., Yaggi, H. K., & Araujo, A. B. (2013). Racial and socioeconomic disparities in sleep and chronic disease: results of a longitudinal investigation. Ethnicity and Disease, 23(4), 499–507.

    PubMed  PubMed Central  Google Scholar 

Download references

Funding

This research was supported by grants from the Agency for Healthcare Research and Quality (HHSA29020050040ITO6) and the National Cancer Institute (R01CA174453).

Author information

Author notes

  1. Michelle M. Langer

    Present address: American Institutes of Research, Chapel Hill, NC, USA

Authors and Affiliations

  1. Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Caroleen W. Quach & Bryce B. Reeve

  2. Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Michelle M. Langer, Ronald C. Chen, Deborah S. Usinger & Bryce B. Reeve

  3. Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Ronald C. Chen & Deborah S. Usinger

  4. Department of Radiation Oncology, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Ronald C. Chen

  5. Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    David Thissen

  6. Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Marc A. Emerson

Authors
  1. Caroleen W. Quach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michelle M. Langer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ronald C. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Thissen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Deborah S. Usinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marc A. Emerson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bryce B. Reeve
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Caroleen W. Quach.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest related to this research.

Research involving human participants and/or animals

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Quach, C.W., Langer, M.M., Chen, R.C. et al. Reliability and validity of PROMIS measures administered by telephone interview in a longitudinal localized prostate cancer study. Qual Life Res 25, 2811–2823 (2016). https://doi.org/10.1007/s11136-016-1325-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-016-1325-3

Keywords

Search

Navigation