Abstract
Objective
While standardized patients (SPs) remain the gold standard for assessing clinical competence in a standardized setting, clinical case vignettes that allow free-text, open-ended written responses are more resource- and time-efficient assessment tools. It remains unknown, however, whether this is a valid method for assessing competence in the management of agitation.
Methods
Twenty-six psychiatry residents partook in a randomized controlled study evaluating a simulation-based teaching intervention on the management of agitated patients. Competence in the management of agitation was assessed using three separate modalities: simulation with SPs, open-ended clinical vignettes, and self-report questionnaires.
Results
Performance on clinical vignettes correlated significantly with SP-based assessments (r = 0.59, p = 0.002); self-report questionnaires that assessed one’s own ability to manage agitation did not correlate with SP-based assessments (r = −0.06, p = 0.77).
Conclusions
Standardized clinical vignettes may be a simple, time-efficient, and valid tool for assessing residents’ competence in the management of agitation.
This is a preview of subscription content, log in via an institution to check access.
References
Webb LC, Juul D, Reynolds CF, Ruiz B, Ruiz P, Scheiber SC, et al. How well does the psychiatry residency in-training examination predict performance on the American Board of Psychiatry and Neurology. Part I. Examination? Am J Psychiatry. 1996;153:831–2.
Colliver JA, Swartz MH. Assessing clinical performance with standardized patients. JAMA. 1997;278:790–1.
Peabody JW, Luck J, Glassman P, Jain S, Hansen J, Spell M, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med. 2004;141:771–80.
Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283:1715–22.
Morgan PJ, Cleave-Hogg DM, Guest CB, Herold J. Validity and reliability of undergraduate performance assessments in an anesthesia simulator. Can J Anaesth. 2001;48:225–33.
Vestal HS, Sowden G, Nejad S, Stoklosa J, Valcourt SC, Keary C, et al. Simulation-based training for residents in the management of acute agitation: a cluster randomized controlled trial. Acad Psychiatry. 2016. doi:10.1007/s40596-016-0559-2.
Nordstrom K, Zun LS, Wilson MP, Md VS, Ng AT, Bregman B, et al. Medical evaluation and triage of the agitated patient: consensus statement of the American association for emergency psychiatry project beta medical evaluation workgroup. West J Emerg Med. 2012;13:3–10.
Wilson MP, Pepper D, Currier GW, Holloman GH, Feifel D. The psychopharmacology of agitation: consensus statement of the American association for emergency psychiatry project beta psychopharmacology workgroup. West J Emerg Med. 2012;13:26–34.
Hasser C, Hung E, Steward C, Weiss A, Williams N, Young J. Proposed EPAs for psychiatry residency programs: what are they and how can they be useful? 2016.
Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77:1121–34.
Acknowledgments
The authors wish to thank Jeanne McHale, RN and Gail Alexander, RN of the Knight Simulation Program within The Institute for Patient Care at Massachusetts General Hospital, as well as Qure Healthcare, for their help and contributions to this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The study was approved by the Partners Healthcare Institutional Review Board, and all participants gave informed consent to be part of the study.
Disclosure of Funding
Grant support was provided by the Partners Centers of Expertise in Medical Education.
Conflicts of Interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Rights and permissions
About this article
Cite this article
Sowden, G.L., Vestal, H.S., Stoklosa, J.B. et al. Clinical Case Vignettes: A Promising Tool to Assess Competence in the Management of Agitation. Acad Psychiatry 41, 364–368 (2017). https://doi.org/10.1007/s40596-016-0604-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40596-016-0604-1