Skip to main content

Measuring Use of Evidence Based Psychotherapy for Posttraumatic Stress Disorder

Administration and Policy in Mental Health and Mental Health Services Research volume 40, pages 311–318 (2013)Cite this article

Abstract

To improve methods of estimating use of evidence-based psychotherapy for posttraumatic stress disorder in the Veteran’s health administration, we evaluated administrative data and note text for patients newly enrolling in six VHA outpatient PTSD clinics in New England during the 2010 fiscal year (n = 1,924). Using natural language processing, we developed machine learning algorithms that mimic human raters in classifying note text. We met our targets for algorithm performance as measured by precision, recall, and F-measure. We found that 6.3 % of our study population received at least one session of evidence-based psychotherapy during the initial 6 months of treatment. Evidence-based psychotherapies appear to be infrequently utilized in VHA outpatient PTSD clinics in New England. Our method could support efforts to improve use of these treatments.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

References

  • Benneyan, J. C., Lloyd, R. C., & Plsek, P. E. (2003). Statistical process control as a tool for research and healthcare improvement. Quality and Safety in Health Care, 12(6), 458–464.

    PubMed  Article  CAS  Google Scholar 

  • Bisson, J. & Ehlers, A. (2005). Clinical guideline 26: Post-traumatic stress disorder (PTSD): The management of PTSD in adults and children in primary and secondary care. National Institute for Clinical Excellence. Retrieved October 22, 2011 from http://www.nice.org.uk/nicemedia/pdf/CG026NICEguideline.pdf.

  • Chapman, W. W., Nadkarni, P. M., Hirschman, L., D’Avolio, L. W., Savova, G. K., & Uzuner, O. (2011). Overcoming barriers to NLP for clinical text: The role of shared tasks and the need for additional creative solutions. Journal of the American Medical Informatics Association, 18(5), 540–543.

    PubMed  Article  Google Scholar 

  • Chard, K. M., Schumm, J. A., Owens, G. P., & Cottingham, S. M. (2010). A comparison of OEF and OIF veterans and Vietnam veterans receiving cognitive processing therapy. Journal of Traumatic Stress, 23(1), 25–32.

    PubMed  Google Scholar 

  • Cully, J. A., Tolpin, L., Henderson, L., Jimenez, D., Kunik, M. E., & Peterson, L. A. (2008). Psychotherapy in the Veterans health administration: Missed opportunities? Psychological Services, 5(4), 320–331.

    Article  Google Scholar 

  • Curran, E., Harper, P., Loveday, H., Gilmour, H., Jones, S., Benneyan, J., et al. (2008). Results of a multicentre randomised controlled trial of statistical process control charts and structured diagnostic tools to reduce ward-acquired meticillin-resistant Staphylococcus aureus: The CHART Project. Journal of Hospital Infection, 70(2), 127–135.

    PubMed  Article  CAS  Google Scholar 

  • Davidson, P. R., & Parker, K. C. (2001). Eye movement desensitization and reprocessing (EMDR): A meta-analysis. Journal of Consulting and Clinical Psychology, 69(2), 305–316.

    PubMed  Article  CAS  Google Scholar 

  • D’Avolio, L. W., Nguyen, T. M., Farwell, W. R., Chen, Y., Fitzmeyer, F., Harris, O. M., et al. (2010). Evaluation of a generalizable approach to clinical information retrieval using the automated retrieval console (ARC). Journal of the American Medical Informatics Association, 17(4), 375–382.

    PubMed  Article  Google Scholar 

  • D’Avolio, L. W., Nguyen, T. M., Goryachev, S., & Fiore, L. D. (2011). Automated concept-level information extraction to reduce the need for custom software and rules development. Journal of the American Medical Informatics Association, 18(5), 607–613.

    PubMed  Article  Google Scholar 

  • Desai, R., Spencer, H., Gray, S., & Pilver, C. (2010). The long journey home XVIII: Treatment of posttraumatic stress disorder in the Department of Veterans affairs: Fiscal year 2009 Service delivery and performance. Northeast Program Evaluation Center. Retrieved October 22, 2011 from http://vaww.nepec.mentalhealth.va.gov/PTSD/ptsLJHAI.pdf.

  • Dieperink, M., Erbes, C., Leskela, J., Kaloupek, D., Farrer, M. K., Fisher, L., et al. (2005). Comparison of treatment for post-traumatic stress disorder among three Department of Veterans Affairs medical centers. Military Medicine, 170(4), 305–308.

    PubMed  Google Scholar 

  • Foa, E. B., Dancu, C. V., Hembree, E. A., Jaycox, L. H., Meadows, E. A., & Street, G. P. (1999). A comparison of exposure therapy, stress inoculation training, and their combination for reducing posttraumatic stress disorder in female assault victims. Journal of Consulting and Clinical Psychology, 67(2), 194–200.

    PubMed  Article  CAS  Google Scholar 

  • Friedman, M. J., (2009). PTSD 101: Pharmacologic treatment of PTSD and comorbid disorders. National Center for PTSD. Retrieved October 22, 2011 from http://www.ptsd.va.gov/professional/ptsd101/course-modules/pharmacotherapy.asp.

  • Friedman, M. J., Lowry, P., & Ruzek, J. (2010). VA/DoD clinical practice guidelines for the management of post-traumatic stress. The Management of Post-Traumatic Stress Working Group. Retrieved October 22, 2011 from http://www.healthquality.va.gov/PTSD-FULL-2010c.pdf.

  • Harpaz-Rotem, I., & Rosenheck, R. A. (2011). Serving those who served: Retention of newly returning veterans from Iraq and Afghanistan in mental health treatment. Psychiatric Services (Washington, D. C.), 62(1), 22–27.

    Article  Google Scholar 

  • Harris, A. H. S., Reeder, R. N., Ellerbe, L., & Bowe, T. (2010). Are VHA administrative location codes valide indicators of specialty substance use disorder treatment? Journal of Rehabilitation Research and Development, 47(8), 699–708.

    PubMed  Article  Google Scholar 

  • Hripcsak, G., & Wilcox, A. (2002). Reference standards, judges, and comparison subjects: Roles for experts in evaluating system performance. Journal of the American Medical Informatics Association, 9(1), 1–15.

    PubMed  Article  Google Scholar 

  • Karlin, B. E., Ruzek, J. I., Chard, K. M., Eftekhari, A., Monson, C. M., Hembree, E. A., et al. (2010). Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the veterans health administration. Journal of Traumatic Stress, 23, 663–673.

    PubMed  Article  Google Scholar 

  • Krystal, J. H., Rosenheck, R. A., Cramer, J. A., Vessicchio, J. C., Jones, K. M., Vertrees, J. E., et al. (2011). Adjunctive risperidone treatment for antidepressant-resistant symptoms of chronic military service-related PTSD: A randomized trial. Journal of the American Medical Association, 306(5), 493–502.

    PubMed  Article  CAS  Google Scholar 

  • Kussman, M. J. (2008). VHA Handbook 1160.01: Uniform mental helath services in VA medical centers and clinics. Retrieved March 3, 2010 from http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1762.

  • Meystre, S. M., Savova, G. K., Kipper-Schuler, K. C., & Hurdle, J. F. (2008). Extracting information from textual documents in the electronic health record: A review of recent research. Yearbook of Medical Informatics, 128–44.

  • Mohamed, S., & Rosenheck, R. A. (2008). Pharmacotherapy of PTSD in the U.S. Department of Veterans Affairs: Diagnostic- and symptom-guided drug selection. Journal of Clinical Psychiatry, 69(6), 959–965.

    PubMed  Article  Google Scholar 

  • Monson, C. M., Schnurr, P. P., Resick, P. A., Friedman, M. J., Young-Xu, Y., & Stevens, S. P. (2006). Cognitive processing therapy for veterans with military-related posttraumatic stress disorder. Journal of Consulting and Clinical Psychology, 74(5), 898–907.

    PubMed  Article  Google Scholar 

  • Murff, H. J., FitzHenry, F., Matheny, M. E., Gentry, N., Kotter, K. L., Crimin, K., et al. (2011). Automated identification of postoperative complications within an electronic medical record using natural language processing. Journal of American Medical Association, 306(8), 848–855.

    Article  CAS  Google Scholar 

  • Rosenheck, R. A., & Fontana, A. F. (2007). Recent trends In VA treatment of post-traumatic stress disorder and other mental disorders. Health Affairs (Millwood), 26(6), 1720–1727.

    Article  Google Scholar 

  • Schnurr, P. P., Friedman, M. J., Engel, C. C., Foa, E. B., Shea, M. T., Chow, B. K., et al. (2007). Cognitive behavioral therapy for posttraumatic stress disorder in women: A randomized controlled trial. Journal of American Medical Association, 297(8), 820–830.

    Article  CAS  Google Scholar 

  • Schnurr, P. P., Lunney, C. A., Bovin, M. J., & Marx, B. P. (2009). Posttraumatic stress disorder and quality of life: Extension of findings to veterans of the wars in Iraq and Afghanistan. Clinical Psychology Review, 29(8), 727–735.

    PubMed  Article  Google Scholar 

  • Seal, K. H., Bertenthal, D., Miner, C. R., Sen, S., & Marmar, C. (2007). Bringing the war back home: Mental health disorders among 103,788 US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs facilities. Archives of Internal Medicine, 167(5), 476–482.

    PubMed  Article  Google Scholar 

  • Seal, K. H., Maguen, S., Cohen, B., Gima, K. S., Metzler, T. J., Ren, L., et al. (2010). VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. Journal of Traumatic Stress, 23(1), 5–16.

    PubMed  Google Scholar 

  • Shiner, B., D’Avolio, L. W., Nguyen, T. M., Zayed M. H., Watts, B. V., & Fiore, L. (2011). Automated classification of psychotherapy note text: Implications for quality assessment in PTSD care. Journal of Evaluation in Clinical Practice. doi:10.1111/j.1365-2753.2011.01634.x.

  • Spoont, M. R., Murdoch, M., Hodges, J., & Nugent, S. (2010). Treatment receipt by veterans after a PTSD diagnosis in PTSD, mental health, or general medical clinics. Psychiatric Services (Washington, DC), 61(1), 58–63.

    Article  Google Scholar 

  • Tuerk, P. W., Yoder, M., Grubaugh, A., Myrick, H., Hamner, M., & Acierno, R. (2011). Prolonged exposure therapy for combat-related posttraumatic stress disorder: An examination of treatment effectiveness for veterans of the wars in Afghanistan and Iraq. Journal of Anxiety Disorders, 25(3), 397–403.

    PubMed  Article  Google Scholar 

  • Whitley, R., & Crawford, M. (2005). Qualitative research in psychiatry. Canadian Journal of Psychiatry, 50(2), 108–114.

    Google Scholar 

  • Yehuda, R., Golier, J. A., Bierer, L. M., Mikhno, A., Pratchett, L. C., Burton, C. L., et al. (2010). Hydrocortisone responsiveness in Gulf War veterans with PTSD: Effects on ACTH, declarative memory hippocampal [(18)F] FDG uptake on PET. Psychiatry Research, 184(2), 117–127.

    PubMed  Article  CAS  Google Scholar 

  • Yoder, M., Tuerk, P. W., Price, M., Grubaugh, A. L., Strachan, M., Myrick, H., et al. (2012). Prolonged exposure therapy for combat-related posttraumatic stress disorder: Comparing outcomes for Veterans of different wars. Psychological Services, 9(1), 16–25.

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

Dr. Shiner’s time is supported by a VA-New England Early Career Development Award (V1CDA2010-03). The ARC was developed with support from the VA Consortium for Health Informatics Research (HIR09-007) and VA Cooperative Studies Program to Dr. D’Avolio and applied to this project with support from the VA Quality Enhancement Research Initiative (RRP11-432) to Dr. Watts. The VA-New England Veterans Engineering Resource Center provided support for administrative data acquisition and the VA National Center for PTSD provided research assistant support.

Ethical Standard

This work was approved by the White River Junction VA Medical Center Research and Development Committee and was granted a waiver of informed consent by the Dartmouth Committee for the Protection of Human Subjects (CPHS 22440).

Disclaimer

The views expressed in this manuscript do not necessarily represent the views of the Department of Veterans Affairs or the United States Government. Dr. Shiner had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the results.

Author information

Authors and Affiliations

  1. VA Medical Center, White River Junction, VT, USA

    Brian Shiner, Maha H. Zayed & Bradley V. Watts

  2. VA New England Veterans Engineering Resource Center, White River Junction, VT, USA

    Brian Shiner & Bradley V. Watts

  3. VA National Center for PTSD, White River Junction, VT, USA

    Brian Shiner, Rani A. Desai & Paula P. Schnurr

  4. Geisel School of Medicine at Dartmouth, Hanover, NH, USA

    Brian Shiner, Yinong Young-Xu, Paula P. Schnurr & Bradley V. Watts

  5. VA Boston Health Care System, Jamaica Plain, MA, USA

    Leonard W. D’Avolio, Thien M. Nguyen & Louis D. Fiore

  6. Massachusetts Veterans Epidemiology and Information Center, Jamaica Plain, MA, USA

    Leonard W. D’Avolio, Thien M. Nguyen & Louis D. Fiore

  7. Harvard Medical School, Boston, MA, USA

    Leonard W. D’Avolio

  8. VA National Center for Patient Safety, White River Junction, VT, USA

    Yinong Young-Xu

  9. VA Northeast Program Evaluation Center, West Haven, CT, USA

    Rani A. Desai

  10. Yale University Schools of Medicine and Public Health, New Haven, CT, USA

    Rani A. Desai

  11. Boston University Schools of Medicine and Public Health, Boston, MA, USA

    Louis D. Fiore

Authors
  1. Brian Shiner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leonard W. D’Avolio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thien M. Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maha H. Zayed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yinong Young-Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rani A. Desai
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Paula P. Schnurr
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Louis D. Fiore
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Bradley V. Watts
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian Shiner.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Shiner, B., D’Avolio, L.W., Nguyen, T.M. et al. Measuring Use of Evidence Based Psychotherapy for Posttraumatic Stress Disorder. Adm Policy Ment Health 40, 311–318 (2013). https://doi.org/10.1007/s10488-012-0421-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-012-0421-0

Keywords

  • Psychotherapy
  • Evidence-based medicine
  • Posttraumatic stress disorder
  • Natural language processing
  • Health services utilization