Quality of Life Research

, Volume 25, Issue 3, pp 615–624 | Cite as

Evaluating PROMIS® instruments and methods for patient-centered outcomes research: Patient and provider voices in a substance use treatment setting

  • Kelly L. Johnston
  • Suzanne M. Lawrence
  • Nathan E. Dodds
  • Lan Yu
  • Dennis C. Daley
  • Paul A. Pilkonis
Special Section: PROs in Non-Standard Settings (by invitation only)



Our work as a primary research site of the Patient-Reported Outcomes Measurement Information System (PROMIS®), combined with support from the Patient-Centered Outcomes Research Institute, allowed us to evaluate the real-world applicability and acceptability of PROMIS measures in an addiction medicine setting.


As part of a 3-month prospective observational study, 225 outpatients at a substance abuse treatment clinic completed PROMIS item banks for alcohol use (as well as 15 additional item banks from 8 other PROMIS domains, including emotional distress, sleep, and pain), with assessments at intake, 1-month follow-up, and 3-month follow-up. A subsample of therapists and their patients completed health domain importance ratings and qualitative interviews to elicit feedback regarding the content and format of the patients’ assessment results.


The importance ratings revealed that depression, anxiety, and lack of emotional support were rated highest of the non-alcohol-related domains among both patients and clinicians. General alcohol use was considered most important by both patients and clinicians. Based on their suggestions, changes were made to item response feedback to facilitate comprehension and communication.


Both therapists and patients agreed that their review of the graphical display of scores, as well as individual item responses, helped them to identify areas of greatest concern and was useful for treatment planning. The results of our pilot work demonstrated the value and practicality of incorporating a comprehensive health assessment within a substance abuse treatment setting.


Patient-reported outcomes Substance abuse Qualitative research Interviews 



This project was funded by the Patient-Centered Outcomes Research Institute (1IP2PI000189; PI: Paul A. Pilkonis, PhD). We acknowledge the work of our colleagues within the NIDA Clinical Trials Network that collaborated in this work. Dennis Daley, PhD, is the principal investigator of the Tri-State Appalachian node, and clinical data from this node were collected in Pittsburgh, PA, where we are grateful for the help of Trey Ghee, LSW, Dorothy Sandstrom, MS, and Janis McDonald.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standard

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This study was reviewed and approved by the University of Pittsburgh Institutional Review Board.


  1. 1.
    American Psychiatric Association. (2013). Assessment Measures. In Diagnostic and statistical manual of mental disorders (5th ed., pp. 733–748). Arlington, VA: American Psychiatric Association.Google Scholar
  2. 2.
    U.S. Government Accountability Office (GAO). (2015). Comparative effectiveness research: HHS needs to strengthen dissemination and data-capacity-building efforts: GAO-15-280.
  3. 3.
    Snyder, C. F., Jensen, R. E., Geller, G., Carducci, M. A., & Wu, A. W. (2010). Relevant content for a patient-reported outcomes questionnaire for use in oncology clinical practic: Putting doctors and patients on the same page. Quality of Life Research, 19, 1045–1055.CrossRefPubMedGoogle Scholar
  4. 4.
    Wu, A. W., Kharrazi, H., Boulwar, L. E., & Snyder, C. F. (2013). Measure once, cut twice–adding patient-reported outcome measures to the electronic health record for comparative effectiveness research. Journal of Clinical Epidemiology, 66(8 Suppl), S12–S20.PubMedCentralCrossRefPubMedGoogle Scholar
  5. 5.
    Marshall, S., Haywood, K., & Fitzpatrick, R. (2006). Impact of patient-reported outcome measures on routine practice: A structured review. Journal of Evaluation in Clinical Practice, 12(5), 559–568.CrossRefPubMedGoogle Scholar
  6. 6.
    Rotgers, F. (2002). Clinically useful, research validated assessment of persons with alcohol problems. Behaviour Research and Therapy, 40(12), 1425–1441.CrossRefPubMedGoogle Scholar
  7. 7.
    Cella, D., Gershon, R., Lai, J. S., & Choi, S. (2007). The future of outcomes measurement: Item banking, tailored short forms, and computerized adaptive assessment. Quality of Life Research, 16(Suppl 1), 133–144.CrossRefPubMedGoogle Scholar
  8. 8.
    Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., & Cella, D. (2011). Item banks for measuring emotional distress from the patient-reported outcomes measurement information system: Depression, anxiety, anger. Assessment., 18(3), 263–283.PubMedCentralCrossRefPubMedGoogle Scholar
  9. 9.
    Broderick, J. E., DeWitt, E. M., Rothrock, N., Crane, P. K., & Forrest, C. B. (2013). Advances in patient reported outcomes: The NIH PROMIS measures. eGEMs (Generating Evidence & Methods to improve patient outcomes), 1(1), Article 12, 1–7.Google Scholar
  10. 10.
    Buysse, D., Yu, L., Moul, D. E., Germain, A., Stover, A., Dodds, N. E., et al. (2010). Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep, 33(6), 781–792.Google Scholar
  11. 11.
    Revicki, D., Chen, W., Harnam, N., Cook, K., Amtmann, D., Callahan, L. F., et al. (2009). Development and psychometric analysis of the PROMIS pain behavior item bank. Pain, 146(1–2), 158–169.PubMedCentralCrossRefPubMedGoogle Scholar
  12. 12.
    Fries, J. F., Cella, D., Rose, M., Krishnan, E., & Bruce, B. (2009). Progress in assessing physical function in arthritis: PROMIS short forms and computerized adaptive testing. Journal of Rheumatology, 36(9), 2061–2066.CrossRefPubMedGoogle Scholar
  13. 13.
    Pilkonis, P. A., Yu, L., Dodds, N. E., Johnston, K. L., Maihoefer, C., & Lawrence, S. M. (2014). Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS®) in a three-month observational study. Journal of Psychiatric Research, 56, 112–119.PubMedCentralCrossRefPubMedGoogle Scholar
  14. 14.
    Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcome Measurement Information System (PROMIS). Medical Care, 45, S22–S31.CrossRefPubMedGoogle Scholar
  15. 15.
    DeWitt, E. M., Stucky, B. D., Thissen, D., Irwin, D. E., Langer, M., Varni, J. W., et al. (2011). Construction of the eight item PROMIS pediatric physical function scales: Built using item response theory. Journal of Clinical Epidemiology, 64(7), 794–804.PubMedCentralCrossRefPubMedGoogle Scholar
  16. 16.
    Varni, J. W., Stucky, B. D., Thissen, D., DeWitt, E. M., Irwin, D., Lai, J. S., et al. (2010). PROMIS Pediatric Pain Interference Scale: An item response theory analysis of the Pediatric Pain Item Bank. Journal of Pain, 11(11), 1109–1119.PubMedCentralCrossRefPubMedGoogle Scholar
  17. 17.
    Amtmann, D., Cook, K., Jensen, M. P., Chen, W.-H., Choi, S. W., Revicki, D., et al. (2010). Development of a PROMIS item bank to measure pain interference. Pain, 150(1), 173–182.PubMedCentralCrossRefPubMedGoogle Scholar
  18. 18.
    Flynn, K. E., Jeffrey, D. D., Keefe, F. J., Porter, L. S., Shelby, R. A., Fawzy, M. R., et al. (2011). Sexual functioning along the cancer continuum: focus group results from the Patient-Reported Outcomes Measurement Information System (PROMIS). Psycho-Oncology, 20(4), 378–386.PubMedCentralCrossRefPubMedGoogle Scholar
  19. 19.
    Forrest, C. B., Bevans, K. B., Tucker, C., Riley, A. W., Ravens-Sieberer, U., Gardner, W., et al. (2012). Commentary: the Patient-Reported Outcome Measurement Information System (PROMIS) for Children and Youth: Application to pediatric psychology. Journal of Pediatric Psychology37(6), 614–621.Google Scholar
  20. 20.
    Hahn, E. A., DeWalt, D. A., Bode, R. K., Garcia, S. F., DeVellis, R. F., Correia, H., et al. (2014). New English and Spanish social health measures will facilitate evaluating health determinants. Health Psychology, 33(5), 490–499.Google Scholar
  21. 21.
    Hays, R. D., Bjorner, J. B., Revicki, R. A., Spritzer, K. L., & Cella, D. (2009). Development of physical and mental health summary scores from the Patient Reported Outcomes Measurement Information System (PROMIS) global items. Quality of Life Research, 18(7), 873–880.PubMedCentralCrossRefPubMedGoogle Scholar
  22. 22.
    Patel, K. (2010). Health reform’s tortuous route to the Patient-Centered Outcomes Research Institute. Health Affairs, 29(10), 1777–1782.CrossRefPubMedGoogle Scholar
  23. 23.
    Patient-Centered Outcomes Research Institute. (2011–2015). from
  24. 24.
    Pilkonis, P. A., Yu, L., Colditz, J., Dodds, N. E., Johnston, K. L., Maihoefer, C., et al. (2013). Item banks for alcohol use from the Patient-Reported Outcomes Measurement Information System (PROMIS®): use, consequences, and expectancies. Drug and Alcohol Dependence, 130, 167–177.PubMedCentralCrossRefPubMedGoogle Scholar
  25. 25.
    DeWalt, D. A., Rothrock, N., Yount, S., Stone, A. A., & on behalf of the PROMIS Cooperative Group. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45, S12–S21.Google Scholar
  26. 26.
    Saunders, J. B., Aasland, O. G., Babor, T. F., Delafuente, J. R., & Grant, M. (1993). Development of the Alcohol-Use Disorders Identification Test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption. Addiction, 88, 791–804.CrossRefPubMedGoogle Scholar
  27. 27.
    Stinson, F. S., Grant, B. F., Dawson, D. A., Ruan, W. J., Huang, B., & Saha, T. (2005). Comorbidity between DSM-IV alcohol and specific drug use disorders in the United States: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Drug and Alcohol Dependence, 80(1), 105–116.CrossRefPubMedGoogle Scholar
  28. 28.
    Jensen, R. E., Rothrock, N. E., DeWitt, E. M., Spiegel, B., Tucker, C. A., Crane, H., et al. (2015). The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Medical Care, 53(2), 153–159.CrossRefPubMedGoogle Scholar
  29. 29.
    Lohr, K. N., & Zebrack, B. J. (2009). Using patient-reported outcomes in clinical practice: Challenges and opportunities. Quality of Life Research, 18, 99–107.CrossRefPubMedGoogle Scholar
  30. 30.
    Rose, M., & Bezjak, A. (2009). Logistics of collecting patient-reported outcomes (PROs) in clinical practice: An overview and practical examples. Quality of Life Research, 18, 125–136.CrossRefPubMedGoogle Scholar
  31. 31.
    Graff, F. S., Morgan, T. J., Epstein, E. E., McCrady, B. S., Cook, S. M., Jensen, N. K., & Kelly, S. (2009). Engagement and retention in outpatient alcoholism treatment for women. American Journal of Addictions, 18(4), 277–288.CrossRefGoogle Scholar
  32. 32.
    Loveland, D., & Driscoll, H. (2014). Examining attrition rates at one specialty addiction treatment provider in the United States: a case study using a retrospective chart review. Substance Abuse Treatment, Prevention, and Policy, 9(1), 41.PubMedCentralCrossRefPubMedGoogle Scholar
  33. 33.
    Substance Abuse and Mental Health Services Administration Treatment Episode Data Set. (2010). Discharges from substance abuse treatment services. BHSIS Series S-67, HHS Publication No. (SMA) 14-4817. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2013.Google Scholar
  34. 34.
    Stark, M. J. (1992). Dropping out of substance abuse treatment: A clinically oriented review. Clinical Psychology Review, 12(1), 93–116.CrossRefGoogle Scholar
  35. 35.
    Babor, T. F., Higgins-Biddle, J. C., Saunders, J. B., & Monteiro, M. G. (2001). The alcohol use disorders identification test: Guidelines for use in primary care (No. WHO/MSD/MSB/01.6a). World Health Organization.Google Scholar
  36. 36.
    Tracy, E. M. (2010). Social support: A mixed blessing for women in substance abuse treatment. Journal of Social Work Practice in the Addictions, 10(3), 257–282.PubMedCentralCrossRefPubMedGoogle Scholar
  37. 37.
    Chou, K. L., Liang, K., & Sareen, J. (2011). The association between social isolation and DSM-IV mood, anxiety, and substance use disorders: Wave 2 of the National Epidemiologic Survey on Alcohol and Related Conditions. Journal of Clinical Psychiatry, 72(11), 1468–1476.CrossRefPubMedGoogle Scholar
  38. 38.
    Warren, J. I., Stein, J. A., & Grella, C. E. (2007). Role of social support and self-efficacy in treatment outcomes among clients with co-occurring disorders. Drug and Alcohol Dependence, 89(2–3), 267–274.PubMedCentralCrossRefPubMedGoogle Scholar
  39. 39.
    Richter, S. S., Brown, S. A., & Mott, M. A. (1991). The impact of social support and self-esteem on adolescent substance abuse treatment outcome. Journal of Substance Abuse, 3(4), 371–385.CrossRefPubMedGoogle Scholar
  40. 40.
    Ciraulo, D. A., Piechniczek-Buczek, J., & Iscan, E. N. (2003). Outcome predictors in substance use disorders. Psychiatric Clinics of North America, 26(2), 381–409.CrossRefPubMedGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Kelly L. Johnston
    • 1
  • Suzanne M. Lawrence
    • 1
  • Nathan E. Dodds
    • 1
  • Lan Yu
    • 1
    • 2
  • Dennis C. Daley
    • 1
  • Paul A. Pilkonis
    • 1
  1. 1.Department of Psychiatry, Western Psychiatric Institute and ClinicUniversity of Pittsburgh School of MedicinePittsburghUSA
  2. 2.Department of MedicineUniversity of Pittsburgh School of MedicinePittsburghUSA

Personalised recommendations