Assessing Residents’ Skills in Psychiatric Reasoning: the Tufts Test of Formulation and Treatment Planning

Abstract

Objective

The authors aimed to develop an easily administered and scored written test of clinical reasoning for psychiatry residents and to explore its internal reliability and correlation with parameters of training.

Methods

The authors developed a case-based, multiple-choice test comprising 83 questions related to data gathering and interpretation, diagnosis, hypothesis generation and testing, and treatment planning. Postgraduate years 1–4 residents at 18 diverse residency programs, along with their Program Directors and/or Associate Program Directors, took the test. Outcome measures included internal reliability statistics, performance across levels of training, performance in different test categories, performance in programs with high vs. low emphasis on integrative case formulation, and performance in native English speakers vs. others.

Results

A total of 359 residents and 23 faculty members participated. The KR-20 statistic of 0.78 indicated that the test was internally reliable. Faculty performed better than residents, who began to approach faculty level only in their fourth year. Residents in programs with high emphasis on formulation and treatment planning tended to score better than those from low emphasis programs on hypothesis generation and testing, but not other categories of question. There was no evidence that non-native English speakers were at a disadvantage on the test.

Conclusions

A novel test of formulation and treatment planning has met criteria for internal reliability and provided preliminary data about development of reasoning skills in residents. The authors hope that taking and discussing it will also be useful as a training exercise in integrative case formulation.