Abstract
Direct observation of clinical skills is central to assessment in a competency-based medical education model, yet little is known about how direct observation is experienced by trainees and observers. The objective of the study is to explore how direct observation was experienced by residents and faculty in the context of the I-PASS Handoff Study. In this multi-center qualitative study, we conducted focus groups and semi-structured interviews of residents and faculty members at eight tertiary pediatric centers in North America that implemented the I-PASS Handoff Bundle. We employed qualitative thematic analysis to interpret the data. Barriers to and strategies for direct observation were described relating to the observer, trainee, and clinical environment. Residents and faculty described a mutual awareness that residents change their performance of handoffs when observed, in contrast to their usual behavior in a clinical setting. Changes in handoff performance may depend on the nature of the observer or ‘audience’. Direct observation also highlighted the importance of handoffs to participants, recognized as a clinical activity that warrants feedback and assessment. Dramaturgical theory can be used to understand our finding of ‘front-stage’ (observed) versus ‘backstage’ (unobserved) handoffs as distinct performances, tailored to an “audience”. Educators must be cognizant of changes in performance of routine clinical activities when using direct observation to assess clinical competence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albardiaz, R. (2011). Dramaturgy in medical education. Medical Education, 45(7), 756. https://doi.org/10.1111/j.1365-2923.2011.03998.x.
Archer, J. C. (2010). State of the science in health professional education: Effective feedback. Medical Education, 44, 101–108. https://doi.org/10.1111/j.1365-2923.2009.03546.x.
Armstrong, J., Pitt, L., & Berthon, P. (2006). From production to performance: Solving the positioning dilemma in dental practice. Journal of the American Dental Association (1939), 137(9), 1283–1288.
Arpin, J. (2014). Masters of their Conditions III: Clinical applications of theater anthropology in cultural psychiatry. Transcultural Psychiatry, 51(4), 461–478. https://doi.org/10.1177/1363461512443527.
Beard, J. D., Marriott, J., Purdie, H., & Crossley, J. (2011). Assessing the surgical skills of trainees in the operating theatre: A prospective observational study of the methodology. Health Technology Assessment (Winchester, England). https://doi.org/10.3310/hta15010.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa.
Clynes, M. P., & Raftery, S. E. C. (2008). Feedback: An essential element of student learning in clinical practice. Nurse Education in Practice, 8, 405–411. https://doi.org/10.1016/j.nepr.2008.02.003.
Coffey, M., Li, S., Bismilla, Z., Starmer, A. J., Toole, J. K. O., Blankenberg, R. L., et al. (2017). Resident experiences with implementation of the I-PASS handoff bundle. Journal of Graduate Medical Education. https://doi.org/10.4300/JGME-D-16-00616.1.
Donato, A. A. (2014). Direct observation of residents: A model for an assessment system. The American Journal of Medicine, 127(5), 455–460. https://doi.org/10.1016/j.amjmed.2014.01.016.
Epstein, R. M., & Hundert, E. M. (2002). Defining and assessing professional competence. JAMA, 287(2), 226–235.
Frank, J. R., & Langer, B. (2003). Collaboration, communication, management, and advocacy: Teaching surgeons new skills through the CanMEDS Project. World Journal of Surgery, 27(8), 972–978. https://doi.org/10.1007/s00268-003-7102-9. (discussion 978).
Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., et al. (2010). Competency-based medical education: Theory to practice. Medical Teacher, 32(8), 638–645. https://doi.org/10.3109/0142159X.2010.501190.
Fromme, H. B., Karani, R., & Downing, S. M. (2009). Direct observation in medical education: Review of the literature and evidence for validity. The Mount Sinai Journal of Medicine, 76, 365–371. https://doi.org/10.1002/MSJ.
Goffman, E. (1959). The presentation of self in everyday life. Teacher, 21(5), 259. https://doi.org/10.2307/2089106.
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82(4), 581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x.
Haas, J., & Shaffir, W. (1982). Taking on the role of the doctor: A dramaturgical analysis of professionalization. Symbolic Interaction, 5(2), 187–203. https://doi.org/10.1525/si.1982.5.2.187.
Hauer, K. E., Holmboe, E. S., Kogan, J. R., Hauer, K. E., Holmboe, E. S., Kogan, J. R., et al. (2011). Twelve tips for implementing tools for direct observation of medical trainees’ clinical skills during patient encounters. Medical Teacher, 33(1), 27–33. https://doi.org/10.3109/0142159X.2010.507710.
Henderson, A. (2005). The value of integrating interpretive research approaches in the exposition of health care context. Journal of Advanced Nursing, 52(5), 554–560. https://doi.org/10.1111/j.1365-2648.2005.03622.x.
Holmes, C. A. (1992). The drama of nursing. Journal of Advanced Nursing, 17(8), 941–950.
Issenberg, S. B., Mcgaghie, W. C., Petrusa, E. R., Gordon, D. L., & Scalese, R. J. (2009). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10–28. https://doi.org/10.1080/01421590500046924.
Kogan, J. R., Hatala, R., Hauer, K. E., & Holmboe, E. (2017). Guidelines: The do’s, don’ts and don’t knows of direct observation of clinical skills in medical education. Perspectives on Medical Education, 6(5), 286–305. https://doi.org/10.1007/s40037-017-0376-7.
Kromann, C. B., Jensen, M. L., & Ringsted, C. (2009). The effect of testing on skills learning. Medical Education, 43, 21–27. https://doi.org/10.1111/j.1365-2923.2008.03245.x.
LaDonna, K. A., Hatala, R., Lingard, L., Voyer, S., & Watling, C. (2017). Staging a performance: Learners’ perceptions about direct observation during residency. Medical Education, 2111, 1–13. https://doi.org/10.1111/medu.13232.
Macionis, J., & Gerber, L. (2010). Sociology (7th. ed.). London: Pearson Education.
Mann, K., & MacLeod, A. (2015). Constructivism: Learning theories and approaches to research. In J. Cleland & S. Durning (Eds.), Researching medical education (pp. 51–66). Chichester: Wiley.
Mccambridge, J., Witton, J., & Elbourne, D. R. (2014). Systematic review of the Hawthorne effect: New concepts are needed to study research participation effects. Journal of Clinical Epidemiology, 67(3), 267–277. https://doi.org/10.1016/j.jclinepi.2013.08.015.
Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine. https://doi.org/10.1097/00001888-199009000-00045.
Monrouxe, L. V., Rees, C. E., & Bradley, P. (2009). The construction of patients’ involvement in hospital bedside teaching encounters. Qualitative Health Research, 19(7), 918–930. https://doi.org/10.1177/1049732309338583.
Morse, J. M. (1995). The significance of saturation. Qualitative Health Research, 5(2), 147–149. https://doi.org/10.1177/104973239500500201.
Nasca, T. J., Philibert, I., Brigham, T., & Flynn, T. C. (2012). The next GME accreditation system—rationale and benefits. The New England Journal of Medicine, 366(11), 1051–1056. https://doi.org/10.1056/nejmsr1200117.
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research. Academic Medicine, 89(9), 1245–1251. https://doi.org/10.1097/ACM.0000000000000388.
Paradis, E., & Sutkin, G. (2017). Beyond a good story: From Hawthorne effect to reactivity in health professions education research. Medical Education, 51, 31–39. https://doi.org/10.1111/medu.13122.
Parshuram, C. S., Dhanani, S., Kirsh, J. A., & Cox, P. N. (2004). Fellowship training, workload, fatigue and physical stress: A prospective observational study. CMAJ: Canadian Medical Association Journal = Journal de l’Association Medicale Canadienne, 170(6), 965–970.
Pearce, C., Arnold, M., Phillips, C., Trumble, S., & Dwan, K. (2011). The patient and the computer in the primary care consultation. Journal of the American Medical Informatics Association: JAMIA, 18(2), 138–142. https://doi.org/10.1136/jamia.2010.006486.
Peel, J. L., & Nolan, R. J. (2015). You can’t start a central line? Residents at different stages of the learning cycle. Journal of Graduate Medical Education, 7(4), 536–538. https://doi.org/10.4300/JGME-D-15-00025.1.
Ritzer, G., & Yagatich, W. (2012). Contemporary sociological theory. In The Wiley-Blackwell companion to sociology (pp. 98–118). https://doi.org/10.1002/9781444347388.ch6.
Rossiter, K., Gray, J., Kontos, P., Keightley, M., Colantonio, A., & Gilbert, J. (2008). From page to stage: Dramaturgy and the art of interdisciplinary translation. Journal of Health Psychology, 13(2), 277–286. https://doi.org/10.1177/1359105307086707.
Sectish, T. C., Starmer, A. J., Landrigan, C. P., & Spector, N. D. (2010). Establishing a multisite education and research project requires leadership, expertise, collaboration, and an important aim. Pediatrics, 126, 619–622. https://doi.org/10.1542/peds.2010-1793.
Shea, J. A., & Norcini, J. J. (2017). All the [training] world’s a stage…. Medical Education, 51(5), 458–460. https://doi.org/10.1111/medu.13269.
Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22(2), 63–75. https://doi.org/10.3233/EFI-2004-22201.
Spector, N., Starmer, A., Allen, A., Bale, J., Bismilla, Z., Calaman, S., et al. (2013). I-PASS handoff curriculum: Core resident workshop. MedEdPORTAL. https://doi.org/10.15766/mep_2374-8265.9311.
Starmer, A., Spector, N., Srivastava, R., Allen, A., Landrigan, C., & Sectish, T. (2012). I-PASS, a mnemonic to standardize verbal handoffs. Pediatrics, 129(2), 201–204. https://doi.org/10.1542/peds.2011-2966.
Starmer, A. J., Spector, N. D., Srivastava, R., West, D. C., Rosenbluth, G., Allen, A. D., et al. (2014). Changes in medical errors after implementation of a handoff program. New England Journal of Medicine, 371(19), 1803–1812. https://doi.org/10.1056/NEJMsa1405556.
ten Cate, O., & Scheele, F. (2007). Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Academic Medicine: Journal of the Association of American Medical Colleges, 82(6), 542–547. https://doi.org/10.1097/ACM.0b013e31805559c7.
Watling, C., LaDonna, K. A., Lingard, L., Voyer, S., & Hatala, R. (2016). “Sometimes the work just needs to be done”: Socio-cultural influences on direct observation in medical training. Medical Education, 50(10), 1054–1064. https://doi.org/10.1111/medu.13062.
Acknowledgements
We thank all resident and faculty participants, the I-PASS site leads, and the I-PASS Executive Council for their contributions to this research. We acknowledge Physician Services Incorporated (#11-28) and the US Department of Health and Human Services (R18 AE000029) for financial support of this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
The study was supported by funding from: (1) improving resident handoff in teaching hospitals: understanding implementation and effectiveness of a handoff bundle. Coffey M, Starmer A, Mahant S, Matlow A, Bismilla Z, Landrigan C. Physician Services Incorporated Health Research Grant. ($154,820 2011–2013). (2) Building Effective Resident Hand Off Practices To Improve Patient Safety. Landrigan C et al. U.S. Dept. of Health and Human Services (ARRA funding) ($3,000,000—2010–2013)
Conflict of interest
Drs. Landrigan, West, Spector and Starmer have consulted with and hold equity in the I-PASS Patient Safety Institute, which seeks to train institutions in best handoff practices and aid in their implementation. Drs. Starmer, Spector, and West reported receiving honoraria and travel reimbursement from multiple academic and professional organizations for delivering lectures on handoffs and patient safety. Dr. Landrigan has been supported in part by the Children’s Hospital Association for his work as an Executive Council member of the Pediatric Research in Inpatient Settings (PRIS) network. In addition, Dr. Landrigan has received monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for teaching and consulting on sleep deprivation, physician performance, handoffs, and safety, and has served as an expert witness in cases regarding patient safety and sleep deprivation.
Ethical approval
The study was approved by the Hospital for Sick Children Research Ethics Board.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The data was presented in abstract form at the Pediatric Academic Societies Annual Meeting, San Francisco, California, May 6, 2017.
Rights and permissions
About this article
Cite this article
Huth, K., Schwartz, S., Li, SA. et al. “All the ward’s a stage”: a qualitative study of the experience of direct observation of handoffs. Adv in Health Sci Educ 24, 301–315 (2019). https://doi.org/10.1007/s10459-018-9867-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-018-9867-5