Skip to main content

Measuring errors and adverse events in health care

Abstract

In this paper, we identify 8 methods used to measure errors and adverse events in health care and discuss their strengths and weaknesses. We focus on the reliability and validity of each, as well as the ability to detect latent errors (or system errors) versus active errors and adverse events. We propose a general framework to help health care providers, researchers, and administrators choose the most appropriate methods to meet their patient safety measurement goals.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human. Washington DC: National Academy Press, 1999.

    Google Scholar 

  2. 2.

    Expert Group on Learning from Adverse Events in the NHS. An Organisation With a Memory. London: Stationery Office; 2000.

    Google Scholar 

  3. 3.

    Bates DW, Gawande AA. Error in medicine: what have we learned. Ann Intern Med. 2000;132:763–7.

    PubMed  CAS  Google Scholar 

  4. 4.

    Weingart SN, Wilson RM, Gibberd RW, Harrison B. Epidemiology of medical error. BMJ. 2000;320:774–7.

    PubMed  Article  CAS  Google Scholar 

  5. 5.

    Brennan TA. The Institute of Medicine report on medical errors—could it do harm? N Engl J Med. 2000;342:1123–5.

    PubMed  Article  CAS  Google Scholar 

  6. 6.

    McDonald CJ, Weiner M, Hui SL. Deaths due to medical errors are exaggerated in Institute of Medicine report. JAMA. 2000;284:93–5.

    PubMed  Article  CAS  Google Scholar 

  7. 7.

    Sox HC, Woloshin S. How many deaths are due to medical error? Getting the number right. Eff Clin Pract. 2000;3:277–83.

    Google Scholar 

  8. 8.

    Hofer TP, Kerr EA. What is an error? Eff Clin Pract. 2000;3:261–9.

    PubMed  CAS  Google Scholar 

  9. 9.

    Wears RL, Janiak B, Moorehead JC, et al. Human error in medicine: promise and pitfalls, part 2. Ann Emerg Med. 2000;36:142–4.

    PubMed  Article  CAS  Google Scholar 

  10. 10.

    Leape LL, Berwick DM, Bates DW. What practices will most improve patient safety? Evidence-based medicine meets patient safety. JAMA. 2002;288:501–7.

    PubMed  Article  Google Scholar 

  11. 11.

    Shojania KG, Duncan BW, McDonald KM, Wachter RM. Safe but sound. Patient safety meets evidence-based medicine. JAMA. 2002;288:508–13.

    PubMed  Article  Google Scholar 

  12. 12.

    Reason J. Human Error. Cambridge: Cambridge University Press; 1990.

    Google Scholar 

  13. 13.

    Hulley SB, Martin JN, Cummings SR. Planning the measurements: precision and accuracy. In: Designing Clinical Research: An Epidemiologic Approach. Hulley SB, Cummings SR, eds. Philadelphia: Lippincott; 2001:37.

    Google Scholar 

  14. 14.

    Leape LL, Bates DW, Cullen DJ. Systems analysis of adverse drug events. ADE prevention study. JAMA. 1995;274:35–43.

    PubMed  Article  CAS  Google Scholar 

  15. 15.

    Perrow C. Normal Accidents: Living With High Risk Technologies. New York: Basic Books; 1984.

    Google Scholar 

  16. 16.

    Gordon L. Gordon’s Guide to the Surgical Morbidity and Mortality Conference. Philadelphia: Hanley and Belfus; 1994.

    Google Scholar 

  17. 17.

    Accreditation Council for Graduate Medical Education. Essentials and Information Items. Graduate Medical Education Directory. Chicago Ill: American Medical Association; 1995.

    Google Scholar 

  18. 18.

    Harbison S, Regehr G. Faculty and resident opinions regarding the role of Morbidity and Mortality Conference. Am J Surg 1999;177:136–9.

    PubMed  Article  CAS  Google Scholar 

  19. 19.

    Goldman L, Sayson R, Robbins S, Cohn LH, Bettmann M, Weisberg M. The value of the autopsy in three medical eras. N Engl J Med. 1983;308:1000–5.

    PubMed  CAS  Article  Google Scholar 

  20. 20.

    Cameron HM, McGoogan E. A prospective study of 1152 hospital autopsies. I: inaccuracies in death certification. J Pathol. 1981;133:273–83.

    PubMed  Article  CAS  Google Scholar 

  21. 21.

    Anderson RE, Hill RB, Key CR. The sensitivity and specificity of clinical diagnostics during five decades: toward an understanding of necessary fallibility. JAMA. 1989;261:1610–17.

    PubMed  Article  CAS  Google Scholar 

  22. 22.

    Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. Boston: Little Brown, and Company; 1991.

    Google Scholar 

  23. 23.

    Fischoff B. Hindsight does not equal foresight: the effect of outcome knowledge on judgment under uncertainty. J Exp Psychol Hum Percept Perform. 1975;1:288–99.

    Article  Google Scholar 

  24. 24.

    Caplan RA, Posner KL, Cheney FW. Effect of outcome on physician judgments of appropriateness of care. JAMA. 1991;265:1957–60.

    PubMed  Article  CAS  Google Scholar 

  25. 25.

    Cheney FW, Posner K, Caplan RA, Ward RJ. Standard of care and anesthesia liability. JAMA. 1989;261:1599–1603.

    PubMed  Article  CAS  Google Scholar 

  26. 26.

    Rolph JE, Kravitz RL, McGuigan K. Malpractice claims data as a quality improvement tool. II. Is targeting effective? JAMA. 1991;266:2093–7.

    PubMed  Article  CAS  Google Scholar 

  27. 27.

    Weiler PC. Medical Malpractice on Trial. Cambridge, Mass: Harvard University Press; 1991.

    Google Scholar 

  28. 28.

    Eichhorn JH, Cooper JB, Cullen DJ, Maier WR, Philip JH, Seeman RG. Standards for patient monitoring during anesthesia at Harvard Medical School. JAMA. 1986;256:1017–20.

    PubMed  Article  CAS  Google Scholar 

  29. 29.

    Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ. 2000;320:759–63.

    PubMed  Article  CAS  Google Scholar 

  30. 30.

    Sexton JB, Thomas EJ, Helmreich RL. Error, stress, and teamwork in medicine and aviation: cross-sectional surveys. BMJ. 2000;320:745–9.

    PubMed  Article  CAS  Google Scholar 

  31. 31.

    Brennan TA, Lee TH, O’Neil AC, Petersen LA. Integrating providers into quality improvement: a pilot project at one hospital. Qual Manag Health Care. 1992;1:29–35.

    PubMed  CAS  Article  Google Scholar 

  32. 32.

    Edmonson AC. Learning from mistakes is easier said than done: group and organizational influences on the detection and correction of human error. J Appl Behav Sci. 1996;32:5–28.

    Article  Google Scholar 

  33. 33.

    O’Neil AC, Petersen LA, Cook EF, Bates DW, Lee TH, Brennan TA. A comparison of physicians self-reporting to medical record review to identify medical adverse events. Ann Intern Med. 1993;119:370–6.

    PubMed  CAS  Google Scholar 

  34. 34.

    Petersen LA, Brennan TA, O’Neil AC, Cook EF, Lee TH. Does house staff discontinuity of care increase the risk for preventable adverse events? Ann Intern Med. 1994;121:866–72.

    PubMed  CAS  Google Scholar 

  35. 35.

    Petersen LA, Orav EJ, Teich JM, O’Neil AC, Brennan TA. Using a computerized sign-out to improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improv. 1998;24:77–87.

    PubMed  CAS  Google Scholar 

  36. 36.

    Iezzoni LI. Assessing quality using administrative data. Ann Intern Med. 1997;127:666–74.

    PubMed  CAS  Google Scholar 

  37. 37.

    Iezzoni LI. Identifying complications of care using administrative data. Med Care. 1994;32:700–15.

    PubMed  Article  CAS  Google Scholar 

  38. 38.

    Iezzoni LI, Davis RB, Palmer RH, et al. Does the Complications Screening Program flag cases with process of care problems? Using explicit criteria to judge processes. Int J Qual Health Care. 1999;11:107–18.

    PubMed  Article  CAS  Google Scholar 

  39. 39.

    Weingart SN, Iezzoni LI, Davis RB, et al. Use of administrative data to find substandard care: validation of the complications screening program. Med Care. 2000;38:796–806.

    PubMed  Article  CAS  Google Scholar 

  40. 40.

    Bates DW, O’Neil AC, Petersen LA, Lee TH, Brennan TA. Evaluation of screening criteria for adverse events in medical patients. Med Care. 1995;33:452–62.

    PubMed  Article  CAS  Google Scholar 

  41. 41.

    Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence care in hospitalized patients. N Engl J Med. 1991;324:370–6.

    PubMed  CAS  Article  Google Scholar 

  42. 42.

    Ashton CM, Kuykendall DH, Johnson ML, Wray NP. An empirical assessment of the validity of explicit and implicit process of care criteria for quality assessment. Med Care. 1999;37:798–808.

    PubMed  Article  CAS  Google Scholar 

  43. 43.

    Localio AR, Lawthers A, Brennan TA. Identifying adverse events caused by medical care: degree of physician agreement in retrospective chart review. Ann Intern Med. 1996;125:457–64.

    PubMed  CAS  Google Scholar 

  44. 44.

    Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med. 2000;108:642–9.

    PubMed  Article  CAS  Google Scholar 

  45. 45.

    Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. JAMA. 1991;266:2847–51.

    PubMed  Article  CAS  Google Scholar 

  46. 46.

    Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: development of a computer-based monitor and comparison to chart-review and stimulated voluntary report. J Am Med Inform Assoc. 1998;5:305–14.

    PubMed  CAS  Google Scholar 

  47. 47.

    Helmreich RL, Schaefer HG. Team performance in the operating room. In: Bogner MS, ed. Human Error in Medicine. Hillsdale NJ: Lawrence Erlbaum Associates; 1994:225–53.

    Google Scholar 

  48. 48.

    Donchin Y, Gopher D, Olin M, et al. A look into the nature and causes of human errors in the intensive care unit. Crit Care Med. 1995;23:294–300.

    PubMed  Article  CAS  Google Scholar 

  49. 49.

    Andrews LB, Stocking C, Krizek T, et al. An alternative strategy for studying adverse events in medical care. Lancet. 1997;349:309–13.

    PubMed  Article  CAS  Google Scholar 

  50. 50.

    Barker KN. Data collection techniques: observation. Am J Hosp Pharm. 1980;37:1235–43.

    PubMed  CAS  Google Scholar 

  51. 51.

    Cook RI, Woods DD. Operating at the sharp end: the complexity of human error. In: Bogner MS, ed. Human Error in Medicine. Hillsdale NJ: Lawrence Erlbaum Associates; 1994.

    Google Scholar 

  52. 52.

    Last JM. A Dictionary of Epidemiology. New York: Oxford University Press; 1995.

    Google Scholar 

  53. 53.

    Mangano DT, Goldman L. Preoperative assessment of patients with known or suspected coronary disease. N Engl J Med. 1995;333:1750–6.

    PubMed  Article  CAS  Google Scholar 

  54. 54.

    Lee TH, Marcantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100:1043–9.

    PubMed  CAS  Google Scholar 

  55. 55.

    Gaynes RP, Horan TC. Surveillance of nosocomial infections. In: Mayhall GC, ed, Hospital Epidemiology and Infection Control. Philidelphia: Lippincott Williams and Wilkins; 1999.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Eric J. Thomas MD, MPH.

Additional information

Dr. Thomas is a Robert Wood Johnson Foundation Generalist Physician Faculty Scholar. Dr. Petersen was an awardee in the Research Career Development Award Program of the VA HSR&D Service (grant RCD 95-306) and is a Robert Wood Johnson Foundation Generalist Physician Faculty Scholar.

The authors have no potential conflicts of interest. The authors’ funding agencies had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Thomas, E.J., Petersen, L.A. Measuring errors and adverse events in health care. J GEN INTERN MED 18, 61–67 (2003). https://doi.org/10.1046/j.1525-1497.2003.20147.x

Download citation

Key words

  • medical error
  • adverse events
  • patient safety
  • measurement