Assessing Residents’ Skills in Psychiatric Reasoning: the Tufts Test of Formulation and Treatment Planning

Abstract

Objective

The authors aimed to develop an easily administered and scored written test of clinical reasoning for psychiatry residents and to explore its internal reliability and correlation with parameters of training.

Methods

The authors developed a case-based, multiple-choice test comprising 83 questions related to data gathering and interpretation, diagnosis, hypothesis generation and testing, and treatment planning. Postgraduate years 1–4 residents at 18 diverse residency programs, along with their Program Directors and/or Associate Program Directors, took the test. Outcome measures included internal reliability statistics, performance across levels of training, performance in different test categories, performance in programs with high vs. low emphasis on integrative case formulation, and performance in native English speakers vs. others.

Results

A total of 359 residents and 23 faculty members participated. The KR-20 statistic of 0.78 indicated that the test was internally reliable. Faculty performed better than residents, who began to approach faculty level only in their fourth year. Residents in programs with high emphasis on formulation and treatment planning tended to score better than those from low emphasis programs on hypothesis generation and testing, but not other categories of question. There was no evidence that non-native English speakers were at a disadvantage on the test.

Conclusions

A novel test of formulation and treatment planning has met criteria for internal reliability and provided preliminary data about development of reasoning skills in residents. The authors hope that taking and discussing it will also be useful as a training exercise in integrative case formulation.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. 1.

    McClain T, O’Sullivan PS, Clardy J. Biopsychosocial formulation: educational shortcomings. Acad Psychiatry. 2004;28(2):88–94.

    Article  Google Scholar 

  2. 2.

    Miresco MJ, Kirmayer LJ. The persistence of mind-brain dualism in psychiatric reasoning about clinical scenarios. Am J Psychiatry. 2006;163(5):913–8.

    Article  Google Scholar 

  3. 3.

    Mirowsky J, Ross C. Psychiatric diagnosis as reified measurement. J Health & Social Behavior. 1989;30:11–25.

    CAS  Article  Google Scholar 

  4. 4.

    Larsen DP, Butler AC, Lawson AL, Roediger HL. The importance of seeing the patient: test-enhanced learning with standardized patients and written tests improves clinical application of knowledge. Adv in Health Sci Educ. 2012;18:409–25. https://doi.org/10.1007/s10459-012-9379-7.

    Article  Google Scholar 

  5. 5.

    Fitzhenry-Coor I, Nurcombe B. Assessing clinical reasoning: the development of a new test in psychiatric education. J Psychiatric Educ. 1983;7:183–96.

    Google Scholar 

  6. 6.

    Linstone HA, Turoff M. The Delphi method: techniques and applications. Reading: Addison-Wesley; 1975.

    Google Scholar 

  7. 7.

    Cook D, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015;49:560–75.

    Article  Google Scholar 

  8. 8.

    Mullen LS, Rieder RO, Glick RA, Luber B, Rosen PJ. Testing psychodynamic psychotherapy skills among psychiatric residents: the psychodynamic psychotherapy competency test. Am J Psychiatry. 2004;161(9):1658–64.

    Article  Google Scholar 

  9. 9.

    Dreyfus S, Dreyfus H. A five-stage model of the mental activity involved in directed skill acquisition. Washington, DC: Storming Media; 1980.

    Google Scholar 

  10. 10.

    Gladwell M. Outliers: the story of success. Boston: Little Brown; 2008.

    Google Scholar 

  11. 11.

    Kirwin P, Conroy M, Lyketsos C, Greenwald B, Forester B, deVries C, et al. A call to restructure psychiatry general and subspecialty training. Acad Psychiatry. 2016;40(1):145–8.

    Article  Google Scholar 

  12. 12.

    Moltjabi R, Olfson M. National patterns in antidepressant treatment by psychiatrists and general medical providers: results from the National Comorbidity Survey replication. J Clin Psychiatry. 2008;69:1064–74.

    Article  Google Scholar 

  13. 13.

    Dewan M, Walia K, Meszaros Z, Manring J, Satish U. Using meaningful outcomes to differentiate change from innovation in medical education. Acad Psychiatry. 2017;41(1):100–5.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Edward K. Silberman.

Ethics declarations

Ethical Considerations

This study was judged “Exempt” by the Tufts Health Sciences IRB.

Disclosures

Dr. Adler reports grants from Janssen Pharmaceuticals, outside the submitted work, and activity as co-owner of Health and Productivity Sciences, which currently has no assets or patents. Dr. Silberman has filed a patent for copyright of the test (pending). The other authors have no disclosures.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Silberman, E.K., Ramesh, S., Adler, D. et al. Assessing Residents’ Skills in Psychiatric Reasoning: the Tufts Test of Formulation and Treatment Planning. Acad Psychiatry 44, 701–708 (2020). https://doi.org/10.1007/s40596-020-01234-9

Download citation

Keywords

  • Residents
  • Clinical reasoning
  • Assessment
  • Training