The authors aimed to develop an easily administered and scored written test of clinical reasoning for psychiatry residents and to explore its internal reliability and correlation with parameters of training.
The authors developed a case-based, multiple-choice test comprising 83 questions related to data gathering and interpretation, diagnosis, hypothesis generation and testing, and treatment planning. Postgraduate years 1–4 residents at 18 diverse residency programs, along with their Program Directors and/or Associate Program Directors, took the test. Outcome measures included internal reliability statistics, performance across levels of training, performance in different test categories, performance in programs with high vs. low emphasis on integrative case formulation, and performance in native English speakers vs. others.
A total of 359 residents and 23 faculty members participated. The KR-20 statistic of 0.78 indicated that the test was internally reliable. Faculty performed better than residents, who began to approach faculty level only in their fourth year. Residents in programs with high emphasis on formulation and treatment planning tended to score better than those from low emphasis programs on hypothesis generation and testing, but not other categories of question. There was no evidence that non-native English speakers were at a disadvantage on the test.
A novel test of formulation and treatment planning has met criteria for internal reliability and provided preliminary data about development of reasoning skills in residents. The authors hope that taking and discussing it will also be useful as a training exercise in integrative case formulation.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
McClain T, O’Sullivan PS, Clardy J. Biopsychosocial formulation: educational shortcomings. Acad Psychiatry. 2004;28(2):88–94.
Miresco MJ, Kirmayer LJ. The persistence of mind-brain dualism in psychiatric reasoning about clinical scenarios. Am J Psychiatry. 2006;163(5):913–8.
Mirowsky J, Ross C. Psychiatric diagnosis as reified measurement. J Health & Social Behavior. 1989;30:11–25.
Larsen DP, Butler AC, Lawson AL, Roediger HL. The importance of seeing the patient: test-enhanced learning with standardized patients and written tests improves clinical application of knowledge. Adv in Health Sci Educ. 2012;18:409–25. https://doi.org/10.1007/s10459-012-9379-7.
Fitzhenry-Coor I, Nurcombe B. Assessing clinical reasoning: the development of a new test in psychiatric education. J Psychiatric Educ. 1983;7:183–96.
Linstone HA, Turoff M. The Delphi method: techniques and applications. Reading: Addison-Wesley; 1975.
Cook D, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015;49:560–75.
Mullen LS, Rieder RO, Glick RA, Luber B, Rosen PJ. Testing psychodynamic psychotherapy skills among psychiatric residents: the psychodynamic psychotherapy competency test. Am J Psychiatry. 2004;161(9):1658–64.
Dreyfus S, Dreyfus H. A five-stage model of the mental activity involved in directed skill acquisition. Washington, DC: Storming Media; 1980.
Gladwell M. Outliers: the story of success. Boston: Little Brown; 2008.
Kirwin P, Conroy M, Lyketsos C, Greenwald B, Forester B, deVries C, et al. A call to restructure psychiatry general and subspecialty training. Acad Psychiatry. 2016;40(1):145–8.
Moltjabi R, Olfson M. National patterns in antidepressant treatment by psychiatrists and general medical providers: results from the National Comorbidity Survey replication. J Clin Psychiatry. 2008;69:1064–74.
Dewan M, Walia K, Meszaros Z, Manring J, Satish U. Using meaningful outcomes to differentiate change from innovation in medical education. Acad Psychiatry. 2017;41(1):100–5.
This study was judged “Exempt” by the Tufts Health Sciences IRB.
Dr. Adler reports grants from Janssen Pharmaceuticals, outside the submitted work, and activity as co-owner of Health and Productivity Sciences, which currently has no assets or patents. Dr. Silberman has filed a patent for copyright of the test (pending). The other authors have no disclosures.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Silberman, E.K., Ramesh, S., Adler, D. et al. Assessing Residents’ Skills in Psychiatric Reasoning: the Tufts Test of Formulation and Treatment Planning. Acad Psychiatry 44, 701–708 (2020). https://doi.org/10.1007/s40596-020-01234-9
- Clinical reasoning