Measuring Use of Evidence Based Psychotherapy for Posttraumatic Stress Disorder
To improve methods of estimating use of evidence-based psychotherapy for posttraumatic stress disorder in the Veteran’s health administration, we evaluated administrative data and note text for patients newly enrolling in six VHA outpatient PTSD clinics in New England during the 2010 fiscal year (n = 1,924). Using natural language processing, we developed machine learning algorithms that mimic human raters in classifying note text. We met our targets for algorithm performance as measured by precision, recall, and F-measure. We found that 6.3 % of our study population received at least one session of evidence-based psychotherapy during the initial 6 months of treatment. Evidence-based psychotherapies appear to be infrequently utilized in VHA outpatient PTSD clinics in New England. Our method could support efforts to improve use of these treatments.
KeywordsPsychotherapy Evidence-based medicine Posttraumatic stress disorder Natural language processing Health services utilization
Dr. Shiner’s time is supported by a VA-New England Early Career Development Award (V1CDA2010-03). The ARC was developed with support from the VA Consortium for Health Informatics Research (HIR09-007) and VA Cooperative Studies Program to Dr. D’Avolio and applied to this project with support from the VA Quality Enhancement Research Initiative (RRP11-432) to Dr. Watts. The VA-New England Veterans Engineering Resource Center provided support for administrative data acquisition and the VA National Center for PTSD provided research assistant support.
This work was approved by the White River Junction VA Medical Center Research and Development Committee and was granted a waiver of informed consent by the Dartmouth Committee for the Protection of Human Subjects (CPHS 22440).
The views expressed in this manuscript do not necessarily represent the views of the Department of Veterans Affairs or the United States Government. Dr. Shiner had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the results.
- Bisson, J. & Ehlers, A. (2005). Clinical guideline 26: Post-traumatic stress disorder (PTSD): The management of PTSD in adults and children in primary and secondary care. National Institute for Clinical Excellence. Retrieved October 22, 2011 from http://www.nice.org.uk/nicemedia/pdf/CG026NICEguideline.pdf.
- Chapman, W. W., Nadkarni, P. M., Hirschman, L., D’Avolio, L. W., Savova, G. K., & Uzuner, O. (2011). Overcoming barriers to NLP for clinical text: The role of shared tasks and the need for additional creative solutions. Journal of the American Medical Informatics Association, 18(5), 540–543.PubMedCrossRefGoogle Scholar
- Curran, E., Harper, P., Loveday, H., Gilmour, H., Jones, S., Benneyan, J., et al. (2008). Results of a multicentre randomised controlled trial of statistical process control charts and structured diagnostic tools to reduce ward-acquired meticillin-resistant Staphylococcus aureus: The CHART Project. Journal of Hospital Infection, 70(2), 127–135.PubMedCrossRefGoogle Scholar
- D’Avolio, L. W., Nguyen, T. M., Farwell, W. R., Chen, Y., Fitzmeyer, F., Harris, O. M., et al. (2010). Evaluation of a generalizable approach to clinical information retrieval using the automated retrieval console (ARC). Journal of the American Medical Informatics Association, 17(4), 375–382.PubMedCrossRefGoogle Scholar
- Desai, R., Spencer, H., Gray, S., & Pilver, C. (2010). The long journey home XVIII: Treatment of posttraumatic stress disorder in the Department of Veterans affairs: Fiscal year 2009 Service delivery and performance. Northeast Program Evaluation Center. Retrieved October 22, 2011 from http://vaww.nepec.mentalhealth.va.gov/PTSD/ptsLJHAI.pdf.
- Foa, E. B., Dancu, C. V., Hembree, E. A., Jaycox, L. H., Meadows, E. A., & Street, G. P. (1999). A comparison of exposure therapy, stress inoculation training, and their combination for reducing posttraumatic stress disorder in female assault victims. Journal of Consulting and Clinical Psychology, 67(2), 194–200.PubMedCrossRefGoogle Scholar
- Friedman, M. J., (2009). PTSD 101: Pharmacologic treatment of PTSD and comorbid disorders. National Center for PTSD. Retrieved October 22, 2011 from http://www.ptsd.va.gov/professional/ptsd101/course-modules/pharmacotherapy.asp.
- Friedman, M. J., Lowry, P., & Ruzek, J. (2010). VA/DoD clinical practice guidelines for the management of post-traumatic stress. The Management of Post-Traumatic Stress Working Group. Retrieved October 22, 2011 from http://www.healthquality.va.gov/PTSD-FULL-2010c.pdf.
- Karlin, B. E., Ruzek, J. I., Chard, K. M., Eftekhari, A., Monson, C. M., Hembree, E. A., et al. (2010). Dissemination of evidence-based psychological treatments for posttraumatic stress disorder in the veterans health administration. Journal of Traumatic Stress, 23, 663–673.PubMedCrossRefGoogle Scholar
- Krystal, J. H., Rosenheck, R. A., Cramer, J. A., Vessicchio, J. C., Jones, K. M., Vertrees, J. E., et al. (2011). Adjunctive risperidone treatment for antidepressant-resistant symptoms of chronic military service-related PTSD: A randomized trial. Journal of the American Medical Association, 306(5), 493–502.PubMedCrossRefGoogle Scholar
- Kussman, M. J. (2008). VHA Handbook 1160.01: Uniform mental helath services in VA medical centers and clinics. Retrieved March 3, 2010 from http://www1.va.gov/vhapublications/ViewPublication.asp?pub_ID=1762.
- Meystre, S. M., Savova, G. K., Kipper-Schuler, K. C., & Hurdle, J. F. (2008). Extracting information from textual documents in the electronic health record: A review of recent research. Yearbook of Medical Informatics, 128–44.Google Scholar
- Murff, H. J., FitzHenry, F., Matheny, M. E., Gentry, N., Kotter, K. L., Crimin, K., et al. (2011). Automated identification of postoperative complications within an electronic medical record using natural language processing. Journal of American Medical Association, 306(8), 848–855.CrossRefGoogle Scholar
- Seal, K. H., Bertenthal, D., Miner, C. R., Sen, S., & Marmar, C. (2007). Bringing the war back home: Mental health disorders among 103,788 US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs facilities. Archives of Internal Medicine, 167(5), 476–482.PubMedCrossRefGoogle Scholar
- Shiner, B., D’Avolio, L. W., Nguyen, T. M., Zayed M. H., Watts, B. V., & Fiore, L. (2011). Automated classification of psychotherapy note text: Implications for quality assessment in PTSD care. Journal of Evaluation in Clinical Practice. doi: 10.1111/j.1365-2753.2011.01634.x.
- Tuerk, P. W., Yoder, M., Grubaugh, A., Myrick, H., Hamner, M., & Acierno, R. (2011). Prolonged exposure therapy for combat-related posttraumatic stress disorder: An examination of treatment effectiveness for veterans of the wars in Afghanistan and Iraq. Journal of Anxiety Disorders, 25(3), 397–403.PubMedCrossRefGoogle Scholar
- Whitley, R., & Crawford, M. (2005). Qualitative research in psychiatry. Canadian Journal of Psychiatry, 50(2), 108–114.Google Scholar
- Yehuda, R., Golier, J. A., Bierer, L. M., Mikhno, A., Pratchett, L. C., Burton, C. L., et al. (2010). Hydrocortisone responsiveness in Gulf War veterans with PTSD: Effects on ACTH, declarative memory hippocampal [(18)F] FDG uptake on PET. Psychiatry Research, 184(2), 117–127.PubMedCrossRefGoogle Scholar