Abstract
To improve methods of estimating use of evidence-based psychotherapy for posttraumatic stress disorder in the Veteran’s health administration, we evaluated administrative data and note text for patients newly enrolling in six VHA outpatient PTSD clinics in New England during the 2010 fiscal year (n = 1,924). Using natural language processing, we developed machine learning algorithms that mimic human raters in classifying note text. We met our targets for algorithm performance as measured by precision, recall, and F-measure. We found that 6.3 % of our study population received at least one session of evidence-based psychotherapy during the initial 6 months of treatment. Evidence-based psychotherapies appear to be infrequently utilized in VHA outpatient PTSD clinics in New England. Our method could support efforts to improve use of these treatments.
Similar content being viewed by others
References
Benneyan, J. C., Lloyd, R. C., & Plsek, P. E. (2003). Statistical process control as a tool for research and healthcare improvement. Quality and Safety in Health Care, 12(6), 458–464.
Bisson, J. & Ehlers, A. (2005). Clinical guideline 26: Post-traumatic stress disorder (PTSD): The management of PTSD in adults and children in primary and secondary care. National Institute for Clinical Excellence. Retrieved October 22, 2011 from http://www.nice.org.uk/nicemedia/pdf/CG026NICEguideline.pdf.
Chapman, W. W., Nadkarni, P. M., Hirschman, L., D’Avolio, L. W., Savova, G. K., & Uzuner, O. (2011). Overcoming barriers to NLP for clinical text: The role of shared tasks and the need for additional creative solutions. Journal of the American Medical Informatics Association, 18(5), 540–543.
Chard, K. M., Schumm, J. A., Owens, G. P., & Cottingham, S. M. (2010). A comparison of OEF and OIF veterans and Vietnam veterans receiving cognitive processing therapy. Journal of Traumatic Stress, 23(1), 25–32.
Cully, J. A., Tolpin, L., Henderson, L., Jimenez, D., Kunik, M. E., & Peterson, L. A. (2008). Psychotherapy in the Veterans health administration: Missed opportunities? Psychological Services, 5(4), 320–331.
Curran, E., Harper, P., Loveday, H., Gilmour, H., Jones, S., Benneyan, J., et al. (2008). Results of a multicentre randomised controlled trial of statistical process control charts and structured diagnostic tools to reduce ward-acquired meticillin-resistant Staphylococcus aureus: The CHART Project. Journal of Hospital Infection, 70(2), 127–135.
Davidson, P. R., & Parker, K. C. (2001). Eye movement desensitization and reprocessing (EMDR): A meta-analysis. Journal of Consulting and Clinical Psychology, 69(2), 305–316.
D’Avolio, L. W., Nguyen, T. M., Farwell, W. R., Chen, Y., Fitzmeyer, F., Harris, O. M., et al. (2010). Evaluation of a generalizable approach to clinical information retrieval using the automated retrieval console (ARC). Journal of the American Medical Informatics Association, 17(4), 375–382.
D’Avolio, L. W., Nguyen, T. M., Goryachev, S., & Fiore, L. D. (2011). Automated concept-level information extraction to reduce the need for custom software and rules development. Journal of the American Medical Informatics Association, 18(5), 607–613.
Desai, R., Spencer, H., Gray, S., & Pilver, C. (2010). The long journey home XVIII: Treatment of posttraumatic stress disorder in the Department of Veterans affairs: Fiscal year 2009 Service delivery and performance. Northeast Program Evaluation Center. Retrieved October 22, 2011 from http://vaww.nepec.mentalhealth.va.gov/PTSD/ptsLJHAI.pdf.
Dieperink, M., Erbes, C., Leskela, J., Kaloupek, D., Farrer, M. K., Fisher, L., et al. (2005). Comparison of treatment for post-traumatic stress disorder among three Department of Veterans Affairs medical centers. Military Medicine, 170(4), 305–308.
Foa, E. B., Dancu, C. V., Hembree, E. A., Jaycox, L. H., Meadows, E. A., & Street, G. P. (1999). A comparison of exposure therapy, stress inoculation training, and their combination for reducing posttraumatic stress disorder in female assault victims. Journal of Consulting and Clinical Psychology, 67(2), 194–200.
Friedman, M. J., (2009). PTSD 101: Pharmacologic treatment of PTSD and comorbid disorders. National Center for PTSD. Retrieved October 22, 2011 from http://www.ptsd.va.gov/professional/ptsd101/course-modules/pharmacotherapy.asp.
Friedman, M. J., Lowry, P., & Ruzek, J. (2010). VA/DoD clinical practice guidelines for the management of post-traumatic stress. The Management of Post-Traumatic Stress Working Group. Retrieved October 22, 2011 from http://www.healthquality.va.gov/PTSD-FULL-2010c.pdf.
Harpaz-Rotem, I., & Rosenheck, R. A. (2011). Serving those who served: Retention of newly returning veterans from Iraq and Afghanistan in mental health treatment. Psychiatric Services (Washington, D. C.), 62(1), 22–27.
Harris, A. H. S., Reeder, R. N., Ellerbe, L., & Bowe, T. (2010). Are VHA administrative location codes valide indicators of specialty substance use disorder treatment? Journal of Rehabilitation Research and Development, 47(8), 699–708.
Hripcsak, G., & Wilcox, A. (2002). Reference standards, judges, and comparison subjects: Roles for experts in evaluating system performance. Journal of the American Medical Informatics Association, 9(1), 1–15.
Karlin, B. E., Ruzek, J. I., Chard, K. M., Eftekhari, A., Monson, C. M., Hembree, E. A., et al. (2010). Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the veterans health administration. Journal of Traumatic Stress, 23, 663–673.
Krystal, J. H., Rosenheck, R. A., Cramer, J. A., Vessicchio, J. C., Jones, K. M., Vertrees, J. E., et al. (2011). Adjunctive risperidone treatment for antidepressant-resistant symptoms of chronic military service-related PTSD: A randomized trial. Journal of the American Medical Association, 306(5), 493–502.
Kussman, M. J. (2008). VHA Handbook 1160.01: Uniform mental helath services in VA medical centers and clinics. Retrieved March 3, 2010 from http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1762.
Meystre, S. M., Savova, G. K., Kipper-Schuler, K. C., & Hurdle, J. F. (2008). Extracting information from textual documents in the electronic health record: A review of recent research. Yearbook of Medical Informatics, 128–44.
Mohamed, S., & Rosenheck, R. A. (2008). Pharmacotherapy of PTSD in the U.S. Department of Veterans Affairs: Diagnostic- and symptom-guided drug selection. Journal of Clinical Psychiatry, 69(6), 959–965.
Monson, C. M., Schnurr, P. P., Resick, P. A., Friedman, M. J., Young-Xu, Y., & Stevens, S. P. (2006). Cognitive processing therapy for veterans with military-related posttraumatic stress disorder. Journal of Consulting and Clinical Psychology, 74(5), 898–907.
Murff, H. J., FitzHenry, F., Matheny, M. E., Gentry, N., Kotter, K. L., Crimin, K., et al. (2011). Automated identification of postoperative complications within an electronic medical record using natural language processing. Journal of American Medical Association, 306(8), 848–855.
Rosenheck, R. A., & Fontana, A. F. (2007). Recent trends In VA treatment of post-traumatic stress disorder and other mental disorders. Health Affairs (Millwood), 26(6), 1720–1727.
Schnurr, P. P., Friedman, M. J., Engel, C. C., Foa, E. B., Shea, M. T., Chow, B. K., et al. (2007). Cognitive behavioral therapy for posttraumatic stress disorder in women: A randomized controlled trial. Journal of American Medical Association, 297(8), 820–830.
Schnurr, P. P., Lunney, C. A., Bovin, M. J., & Marx, B. P. (2009). Posttraumatic stress disorder and quality of life: Extension of findings to veterans of the wars in Iraq and Afghanistan. Clinical Psychology Review, 29(8), 727–735.
Seal, K. H., Bertenthal, D., Miner, C. R., Sen, S., & Marmar, C. (2007). Bringing the war back home: Mental health disorders among 103,788 US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs facilities. Archives of Internal Medicine, 167(5), 476–482.
Seal, K. H., Maguen, S., Cohen, B., Gima, K. S., Metzler, T. J., Ren, L., et al. (2010). VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. Journal of Traumatic Stress, 23(1), 5–16.
Shiner, B., D’Avolio, L. W., Nguyen, T. M., Zayed M. H., Watts, B. V., & Fiore, L. (2011). Automated classification of psychotherapy note text: Implications for quality assessment in PTSD care. Journal of Evaluation in Clinical Practice. doi:10.1111/j.1365-2753.2011.01634.x.
Spoont, M. R., Murdoch, M., Hodges, J., & Nugent, S. (2010). Treatment receipt by veterans after a PTSD diagnosis in PTSD, mental health, or general medical clinics. Psychiatric Services (Washington, DC), 61(1), 58–63.
Tuerk, P. W., Yoder, M., Grubaugh, A., Myrick, H., Hamner, M., & Acierno, R. (2011). Prolonged exposure therapy for combat-related posttraumatic stress disorder: An examination of treatment effectiveness for veterans of the wars in Afghanistan and Iraq. Journal of Anxiety Disorders, 25(3), 397–403.
Whitley, R., & Crawford, M. (2005). Qualitative research in psychiatry. Canadian Journal of Psychiatry, 50(2), 108–114.
Yehuda, R., Golier, J. A., Bierer, L. M., Mikhno, A., Pratchett, L. C., Burton, C. L., et al. (2010). Hydrocortisone responsiveness in Gulf War veterans with PTSD: Effects on ACTH, declarative memory hippocampal [(18)F] FDG uptake on PET. Psychiatry Research, 184(2), 117–127.
Yoder, M., Tuerk, P. W., Price, M., Grubaugh, A. L., Strachan, M., Myrick, H., et al. (2012). Prolonged exposure therapy for combat-related posttraumatic stress disorder: Comparing outcomes for Veterans of different wars. Psychological Services, 9(1), 16–25.
Acknowledgments
Dr. Shiner’s time is supported by a VA-New England Early Career Development Award (V1CDA2010-03). The ARC was developed with support from the VA Consortium for Health Informatics Research (HIR09-007) and VA Cooperative Studies Program to Dr. D’Avolio and applied to this project with support from the VA Quality Enhancement Research Initiative (RRP11-432) to Dr. Watts. The VA-New England Veterans Engineering Resource Center provided support for administrative data acquisition and the VA National Center for PTSD provided research assistant support.
Ethical Standard
This work was approved by the White River Junction VA Medical Center Research and Development Committee and was granted a waiver of informed consent by the Dartmouth Committee for the Protection of Human Subjects (CPHS 22440).
Disclaimer
The views expressed in this manuscript do not necessarily represent the views of the Department of Veterans Affairs or the United States Government. Dr. Shiner had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the results.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Shiner, B., D’Avolio, L.W., Nguyen, T.M. et al. Measuring Use of Evidence Based Psychotherapy for Posttraumatic Stress Disorder. Adm Policy Ment Health 40, 311–318 (2013). https://doi.org/10.1007/s10488-012-0421-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-012-0421-0