Skip to main content
SpringerLink
Log in

Using Peers to Assess Handoffs: A Pilot Study

  • Original Research
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

ABSTRACT

Background

Handoffs among post-graduate year 1 (PGY1) trainees occur with high frequency. Peer assessment of handoff competence would add a new perspective on how well the handoff information helped them to provide optimal patient care.

Objective

The goals of this study were to test the feasibility of the approach of an instrument for peer assessment of handoffs by meeting criteria of being able to use technology to capture evaluations in real time, exhibiting strong psychometric properties, and having high PGY1 satisfaction scores.

Design

An iPad® application was built for a seven-item handoff instrument. Over a two-month period, post-call PGY1s completed assessments of three co-PGY1s from whom they received handoffs the prior evening.

Participants

Internal Medicine PGY1s at the University of Pennsylvania.

Main Measures

ANOVA was used to explore interperson score differences (validity). Generalizability analyses provided estimates of score precision (reproducibility). PGY1s completed satisfaction surveys about the process.

Key Results

Sixty-two PGY1s (100 %) participated in the study. 59 % of the targeted evaluations were completed. The major limitations were network connectivity and inability to find the post-call trainee. PGY1 scores on the single item of “overall competency” ranged from 4 to 9 with a mean of 7.31 (SD 1.09). Generalizability coefficients approached 0.60 for 10 evaluations per PGY1 for a single rotation and 12 evaluations per PGY1 across multiple rotations. The majority of PGY1s believed that they could adequately assess handoff competence and that the peer assessment process was valuable (70 and 77 %, respectively).

Conclusion

Psychometric properties of an instrument for peer assessment of handoffs are encouraging. Obtaining 10 or 12 evaluations per PGY1 allowed for reliable assessment of handoff skills. Peer evaluations of handoffs using mobile technology were feasible, and were well received by PGY1s.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  1. Joint Commission on Accreditation of Healthcare Organizations. Sentinel Event Statistics. http://www.jointcommission.org/topics/default.aspx?k=795. Accessed January 9, 2013.

  2. Accreditation Standards. Oakbrook, IL: The Joint Commission; 2010.

  3. Horwitz LI, Krumholz HM, Green ML, Huot SJ. Transfers of patient care between house staff on internal medicine wards: a national survey. Arch Intern Med. 2006;166(11):1173–1177.

    Article  PubMed  Google Scholar 

  4. Institute of Medicine Report on Resident Duty Hours: Enhancing Sleep, Supervision, and Safety http://iom.edu/Reports/2008/Resident-Duty-Hours-Enhancing-Sleep-Supervision-and-Safety.aspx. Accessed January 9, 2013.

  5. American College of Graduate Medical Education Standards. Common Program Requirements. http://www.acgme.org/acgmeweb/Portals/0/dh_dutyhoursCommonPR07012007.pdf Accessed January 9, 2013.

  6. Kemp CD, Bath JM, Berger J, et al. The top 10 list for a safe and effective sign-out. Arch Surg. 2008;143(10):1008–1010.

    Article  PubMed  Google Scholar 

  7. Vidyarthi AR, Arora V, Schnipper JL, Wall SD, Wachter RM. Managing discontinuity in academic medical centers: strategies for a safe and effective resident sign-out. J Hosp Med. 2006;1(4):257–266.

    Article  PubMed  Google Scholar 

  8. Arora VM, Johnson JK, Meltzer DO, Humphrey HJ. A theoretical framework and competency-based approach to improving handoffs. Qual Saf Health Care. 2008;17(1):11–14.

    Article  PubMed  CAS  Google Scholar 

  9. Berkenstadt H, Haviv Y, Tuval A, et al. Improving handoff communications in critical care: utilizing simulation-based training toward process improvement in managing patient risk. Chest. 2008;134(1):158–162.

    Article  PubMed  Google Scholar 

  10. Chu ES, Reid M, Schulz T, et al. A structured handoff program for interns. Acad Med. 2009;84(3):347–352.

    Article  PubMed  Google Scholar 

  11. Helms AS, Perez TE, Baltz J, et al. Use of an appreciative inquiry approach to improve resident sign-out in an era of multiple shift changes. J Gen Intern Med. 2012;27(3):287–291.

    Article  PubMed  Google Scholar 

  12. Horwitz LI, Moin T, Green ML. Development and implementation of an oral sign-out skills curriculum. J Gen Intern Med. 2007;22(10):1470–1474.

    Article  PubMed  Google Scholar 

  13. Riesenberg LA, Leitzsch J, Massucci JL, et al. Residents’ and attending physicians’ handoffs: a systematic review of the literature. Acad Med. 2009;84(12):1775–1787.

    Article  PubMed  Google Scholar 

  14. Gakhar B, Spencer AL. Using direct observation, formal evaluation, and an interactive curriculum to improve the sign-out practices of internal medicine interns. Acad Med. 2010;85(7):1182–1188.

    Article  PubMed  Google Scholar 

  15. Evans SM, Berry JG, Smith BJ, et al. Attitudes and barriers to incident reporting: a collaborative hospital study. Qual Saf Health Care. 2006;15(1):39–43.

    Article  PubMed  CAS  Google Scholar 

  16. Schectman JM, Plews-Ogan ML. Physician perception of hospital safety and barriers to incident reporting. Jt Comm J Qual Patient Saf. 2006;32(6):337–343.

    PubMed  Google Scholar 

  17. Norcini JJ. Peer assessment of competence. Med Educ. 2003;37(6):539–543.

    Article  PubMed  Google Scholar 

  18. Maker VK, Donnelly MB. Surgical resident peer evaluations-what have we learned. J Surg Educ. 2008;65(1):8–16.

    Article  PubMed  Google Scholar 

  19. Co JP, Mohamed H, Kelleher ML, Edgman-Levitan S, Perrin JM. Feasibility of using a tablet computer survey for parental assessment of resident communication skills. Ambul Pediatr. 2008;8(6):375–378.

    Article  PubMed  Google Scholar 

  20. Holzinger A, Kosec P, Schwantzer G, Debevc M, Hofmann-Wellenhof R, Fruhauf J. Design and development of a mobile computer application to reengineer workflows in the hospital and the methodology to evaluate its effectiveness. J Biomed Inform. 2011;44(6):968–977.

    Article  PubMed  Google Scholar 

  21. Arora VM, Manjarrez E, Dressler DD, Basaviah P, Halasyamani L, Kripalani S. Hospitalist handoffs: a systematic review and task force recommendations. J Hosp Med. 2009;4(7):433–440.

    Article  PubMed  Google Scholar 

  22. Airan-Javia SL, Kogan JR, Smith M, et al. Effect of handoff education on the quality of interns’ verbal and written handoff skills. J Grad Med Educ. 2012;4(2):209–214.

    Article  PubMed  Google Scholar 

  23. Farnan JM, Paro JA, Rodriguez RM, et al. Hand-off education and evaluation: piloting the observed simulated hand-off experience (OSHE). J Gen Intern Med. 2010;25(2):129–134.

    Article  PubMed  Google Scholar 

  24. Ende J. Feedback in clinical medical education. JAMA. 1983;250(6):777–781.

    Article  PubMed  CAS  Google Scholar 

  25. Bump GM, Jacob J, Abisse SS, Bost JE, Elnicki DM. Implementing faculty evaluation of written sign-out. Teach Learn Med. 2012;24(3):231–237.

    Article  PubMed  Google Scholar 

  26. Archer JC. State of the science in health professional education: effective feedback. Med Educ. 2010;44(1):101–108.

    Article  PubMed  Google Scholar 

  27. Hewson MG, Little ML. Giving feedback in medical education: verification of recommended techniques. J Gen Intern Med. 1998;13(2):111–116.

    Article  PubMed  CAS  Google Scholar 

  28. Arora V, Johnson J, Lovinger D, Humphrey HJ, Meltzer DO. Communication failures in patient sign-out and suggestions for improvement: a critical incident analysis. Qual Saf Health Care. 2005;14(6):401–407.

    Article  PubMed  CAS  Google Scholar 

  29. Sterkenburg A, Barach P, Kalkman C, Gielen M, ten Cate O. When do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med. 2010;85(9):1408–1417.

    Article  PubMed  Google Scholar 

  30. ten Cate O, Snell L, Carraccio C. Medical competence: the interplay between individual ability and the health care environment. Med Teach. 2010;32(8):669–675.

    Article  PubMed  Google Scholar 

  31. Wohlauer MV, Arora VM, Horwitz LI, Bass EJ, Mahar SE, Philibert I. The patient handoff: a comprehensive curricular blueprint for resident education to improve continuity of care. Acad Med. 2012;87(4):411–418.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

Contributors

The authors would like to thank all the housestaff of the internal medicine residency program at the University of Pennsylvania who participated in this study as well as their Program Leadership. In addition, significant informational technology support was provided by the Hospital of University IT Division. The authors would also like to thank the National Board of Medical Examiners

Funders

This study was conducted in part with funds provided by the National Board of Medical Examiners. Dr. Myers was supported in part by the Josiah Macy Foundation.

Prior presentations

The abstract of an earlier version of this article was presented at the AAMC Annual Meeting in 2011.

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Author information

Authors and Affiliations

  1. Perelman School of Medicine at the University of Pennsylvania, Pulmonary Penn Lung Cancer, PCAM, 1 West Pavilion 3400 Civic Center Boulevard, Philadelphia, PA, 19104, USA

    C. Jessica Dine MD, MSHP, Ilene M. Rosen MD, MSCE, Jennifer S. Myers MD, Jennifer Lapin PhD, Jennifer R. Kogan MD & Judy A. Shea PhD

  2. Thomas Jefferson School of Medicine, Philadelphia, PA, USA

    Nicholas Wingate BS

Authors
  1. C. Jessica Dine MD, MSHP
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicholas Wingate BS
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilene M. Rosen MD, MSCE
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jennifer S. Myers MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jennifer Lapin PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jennifer R. Kogan MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Judy A. Shea PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Jessica Dine MD, MSHP.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dine, C.J., Wingate, N., Rosen, I.M. et al. Using Peers to Assess Handoffs: A Pilot Study. J GEN INTERN MED 28, 1008–1013 (2013). https://doi.org/10.1007/s11606-013-2355-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-013-2355-y

KEY WORDS

Search

Navigation