World Journal of Surgery

, Volume 34, Issue 2, pp 353–361 | Cite as

Development and Evaluation of an Observational Tool for Assessing Surgical Flow Disruptions and Their Impact on Surgical Performance

  • Sarah E. Henrickson ParkerEmail author
  • Aaron A. Laviana
  • Rishi K. Wadhera
  • Douglas A. Wiegmann
  • Thoralf M. SundtIII



Many researchers have previously explored the correlation between surgical flow disruptions and adverse events in cardiac surgery; however, there is no reliable tool to prospectively categorize surgical flow disruptions and the conditions that predispose a surgical team to adverse events.


Two independent raters of different medical and human factors expertise observed 12 cardiovascular operations and iteratively designed a surgical flow disruption tool (SFDT) to characterize surgical flow disruptions and the latent factors that contribute to adverse events. Categories to characterize surgical flow disruptions were created based on human factors models of human error. After the design period, both raters observed ten surgical cases using the tool to assess validity and inter-rater reliability.


Rating agreement (weighted kappa) for each category across the ten surgeries was moderate to very high, resulting in strong inter-rater reliability for each category on the surgical flow disruption tool. Use of the SFDT was simple and clear for observers of diverse backgrounds, including human factors experts and medical personnel.


This research depicts the development and utility of a tool to analyze surgical flow disruptions in the cardiovascular operating room with satisfactory inter-rater reliability. This tool is an important first step in systematically categorizing and measuring surgical flow disruptions and their impact on patient safety in the operating room.


Operating Room Cardiovascular Surgery Surgical Team Kappa Score Flow Disruption 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Chantler C (1999) The role and education of doctors in the delivery of health care. Lancet 353:1178–1181CrossRefPubMedGoogle Scholar
  2. 2.
    Committee on Quality of Health Care in America, Institute of Medicine (1999) Kohn LT, Corrigan JM, Donaldson MS (eds) To err is human: building a safer health system. Washington, DC: National Academy PressGoogle Scholar
  3. 3.
    Guru V, Tu JV, Etchells E et al (2008) Relationship between preventability of death after coronary artery bypass graft surgery and all-cause risk-adjusted mortality rates. Circulation 117:2969–2976CrossRefPubMedGoogle Scholar
  4. 4.
    Vincent C, Taylor-Adams S, Stanhope N (1998) Framework for analyzing risk and safety in clinical medicine. BMJ 316:1154–1157PubMedGoogle Scholar
  5. 5.
    Wiegmann DA, Shappell SA (2003) A human error approach to aviation accident analysis: the human factors analysis and classification system. Ashgate, Burlington, VTGoogle Scholar
  6. 6.
    Reason JT (1990) Human error. Cambridge University Press, Cambridge, EnglandGoogle Scholar
  7. 7.
    de Leval MR, Carthey J, Wright DJ et al (2000) Human factors and cardiac surgery: a multicenter study. J Thorac Cardiovasc Surg 119:661–672CrossRefPubMedGoogle Scholar
  8. 8.
    Lingard L, Espin S, Whyte S et al (2004) Communication failures in the operating room: an observational classification of recurrent types and effects. Qual Safe Health Care 13:330–334CrossRefGoogle Scholar
  9. 9.
    de Leval MR, Rancois K, Bull C et al (1994) Analysis of a cluster of surgical failures: application to a series of neonatal arterial switch operations. J Thorac Cardiovasc Surg 107:914–924PubMedGoogle Scholar
  10. 10.
    Wiegmann DA, Elbardissi AW, Dearani JA et al (2007) Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery 142:658–665CrossRefPubMedGoogle Scholar
  11. 11.
    Elbardissi AW, Wiegmann DA, Dearani JA et al (2007) Application of the human factors analysis and classification system methodology to the cardiovascular surgery operating room. Ann Thorac Surg 83:1412–1419CrossRefPubMedGoogle Scholar
  12. 12.
    Cohen J (1960) A coefficient of agreement for nominal scales. Edu Psych Measure 20:37–46CrossRefGoogle Scholar
  13. 13.
    Altman DG (1991) Practical statistics for medical research. Chapman and Hall, LondonGoogle Scholar
  14. 14.
    MedCalc for Windows version (2009) MedCalc Software, Mariakerke, BelgiumGoogle Scholar
  15. 15.
    Lingard L, Regher G, Espin S et al (2006) A theory-based instrument to evaluate team communication in the operating room: balancing measurement authenticity and reliability. Qual Saf Health Care 15:422–426CrossRefPubMedGoogle Scholar
  16. 16.
    Flin R, Martin L, Goeters K et al (2003) Development of the NOTECHS (non-technical skills) system for rating pilots’ CRM skills. Hum Fac Aero Safety 3:95–117Google Scholar
  17. 17.
    Helmreich RL, Roushee HC (1993) Why crew resource management? Empirical and theoretical bases of human factors training in aviation. In: Wiener E, Kanaki B, Helmreich R (eds) Cockpit resource management. Academic Press, San Diego, pp 3–45Google Scholar
  18. 18.
    Yule S, Flin R, Paterson-Brown S et al (2006) Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 139:140–149CrossRefPubMedGoogle Scholar
  19. 19.
    Yule S, Flin R, Maran N et al (2008) Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg 32:548–556CrossRefPubMedGoogle Scholar
  20. 20.
    Sevdalis N, Davis R, Koutantji M et al (2008) Reliability of a revised NOTECHS scale for use in surgical teams. Am J Surg 196:184–190CrossRefPubMedGoogle Scholar
  21. 21.
    Healey AN, Undre S, Vincent CA (2004) Developing observational measures of performance in surgical teams. Qual Saf Health Care 13(Suppl 1):i33–i40CrossRefPubMedGoogle Scholar
  22. 22.
    Wong DR, Torchiana DF, Vadner Salm TJ et al (2007) Impact of cardiac intraoperative precursor events on adverse outcomes. Surgery 141:715–722CrossRefPubMedGoogle Scholar
  23. 23.
    Solis-Trapala IL, Carthey J, Farwell VT et al (2007) Dynamic modeling in a study of surgical error management. Stat Med 26:5189–5202CrossRefPubMedGoogle Scholar
  24. 24.
    Nowicki ER, Birkmeyer NJ, Weintraub RW et al (2004) Multivariable prediction of in-hospital mortality associated with aortic and mitral valve surgery in Northern New England. Ann Thorac Surg 77:1966–1977CrossRefPubMedGoogle Scholar
  25. 25.
    O’Connor GT, Malenka DJ, Quinton H et al (1999) Multivariate prediction of in-hospital mortality after percutaneous coronary interventions in 1994–1996. Northern New England Cardiovascular Disease Study Group. J Am Coll Cardiol 34:681–691CrossRefPubMedGoogle Scholar
  26. 26.
    O’Connor GT, Plume SK, Olmstead EM, Coffin LH, Morton JR, Maloney CT, Nowicki ER, Levy DG, Tryzelaar JF, Hernandez F (1992) Multivariate prediction of in-hospital mortality associated with coronary artery bypass graft surgery. Northern New England Cardiovascular Disease Study Group. Circulation 85:2110–2118PubMedGoogle Scholar

Copyright information

© Société Internationale de Chirurgie 2009

Authors and Affiliations

  • Sarah E. Henrickson Parker
    • 1
    Email author
  • Aaron A. Laviana
    • 2
  • Rishi K. Wadhera
    • 3
    • 4
  • Douglas A. Wiegmann
    • 5
  • Thoralf M. SundtIII
    • 6
  1. 1.School of Psychology, College of Life Sciences and Medicine, Kings CollegeUniversity of AberdeenAberdeenScotland, UK
  2. 2.Georgetown University School of MedicineWashingtonUSA
  3. 3.Institute of Public Health, School of Clinical MedicineUniversity of CambridgeCambridgeUK
  4. 4.Mayo Medical School, Mayo ClinicRochesterUSA
  5. 5.Department of Industrial and Systems EngineeringUniversity of Wisconsin-MadisonMadisonUSA
  6. 6.Division of Cardiovascular SurgeryMayo ClinicRochesterUSA

Personalised recommendations