Journal of General Internal Medicine

, Volume 5, Issue 3, pp 218–224

Use of an outpatient medical record audit to achieve educational objectives

Changes in residents’ performances over six years
  • David E. Kern
  • William L. Harris
  • Bradley O. Boekeloo
  • Lee Randol Barker
  • Patricia Hogeland
Original Articles


Objective:To evaluate the effectiveness of a process whereby a faculty-resident committee annually audits outpatient record keeping and preventive care practices and provides feedback to resident physicians.

Design:Pre- and postfeedback audits with interventions and observations repeated over six consecutive academic years.

Setting:The adult primary care practice of housestaff in a university-affiliated hospital.

Subjects:All 139 physicians in an internal medicine residency program from 1981–82 through 1986–87, of whom 37 were present for three consecutive years.

Intervention:Each year, residents were given individualized, detailed, typewritten feedback based on audits of their outpatient records.

Measurements and main results:Each resident physician had a minimum of four (mean 5.2) outpatient records per year audited against standards for record-keeping practices and the provision of preventive care. Overall performance scores for each resident audit improved from a mean of 39.7±12.3 (SD) in 1981–82 to a mean of 58.5±14.1 (SD) in 1986–87 (possible range 0 to 100, observed range 9.4 to 86.6). The overall performance scores of individual residents, who received two cycles of feedback, improved an average of 11.5 (95% confidence limits 7.6, 15.3), from a mean of 48.4±11.4 (SD) during their first year of residency to 59.8±13.9 (SD) during their third year. General (primary care) and traditional-track residents improved at similar rates, although mean performance scores were consistently higher for general than for traditional-track residents. Analysis of variance revealed that all changes and differences were statistically significant.

Conclusions:An ongoing chart audit and feedback system can be associated with improvements both in the performance of individual residents and in the long-term performance of a residency program.

Key words

medical audit feedback primary health care internship and residency medical records 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Frame PC, Kowulich BA, Llewellyn AM. Improving physician compliance with a health maintenance protocol. J Fam Pract. 1984;19:341–4.PubMedGoogle Scholar
  2. 2.
    Lau RR, Williams HS, Williams LC, Ware JE, Brook RH. Psychosocial problems in chronically ill children: physician concern, patient satisfaction, and the validity of medical records. J Comm Health. 1982;7:250–61.CrossRefGoogle Scholar
  3. 3.
    Moran MT, Wiser TH, Nada J, Gross H. Measuring medical residents’ chart documentation practices. J Med Educ. 1988;63:860–5.Google Scholar
  4. 4.
    Zuckerman AE, Starfield B, Hochreiter C, Kovasznay B. Validating the content of pediatric medical records by means of tape-recording doctor-patient encounters. Pediatrics. 1975;56:407–11.PubMedGoogle Scholar
  5. 5.
    Brook RH, Fink A, Kosecoff J, et al. Educating physicians and treating patients in the ambulatory setting: where are we going and how will we know when we arrive? Ann Intern Med. 1987;107:392–8.PubMedGoogle Scholar
  6. 6.
    Cheney C, Ramsdell JW. Effect of medical record checklists on implementation of periodic health measures. Am J Med. 1987;83:129–36.PubMedCrossRefGoogle Scholar
  7. 7.
    Davidson RA, Fletcher SW, Retchins DVHS. A nurse-initiated reminder system for the periodic health examination. Arch Intern Med. 1984;144:2167–70.PubMedCrossRefGoogle Scholar
  8. 8.
    Dietrich AJ, Goldberg H. Preventive content of adult primary care: do generalists and subspecialists differ? Am J Public Health. 1984;74:223–7.PubMedGoogle Scholar
  9. 9.
    Fleming DM, Lawrence MSTA. Impact of audit on preventive measures. Br Med J. 1983;287:1852–4.Google Scholar
  10. 10.
    Korn JE, Schlossberg LA, Rich EC. Improved preventive care following an intervention during an ambulatory care rotation: carryover to a second setting. J Gen Intern Med. 1988;3:156–60.PubMedGoogle Scholar
  11. 11.
    Lurie N, Manning WG, Peterson C, Goldberg GA, Phelps CA, Lillard L. Preventive care: do we practice what we preach? Am J Public Health. 1987;77:801–4.PubMedGoogle Scholar
  12. 12.
    McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record: a two-year randomized trial. Ann Intern Med. 1984;100:130–8.PubMedGoogle Scholar
  13. 13.
    Davis JE, Meyer DL, Love RR. Cancer prevention activities in primary care group practice: physician estimates, physician performance, and the structure of practice. Submitted Med. Care.Google Scholar
  14. 14.
    Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Med Care. 1986;24:659–66.PubMedCrossRefGoogle Scholar
  15. 15.
    Winickoff RN, Coltin K, Morgan MM, Buxbaum RC, Barnett GO. Improving physician performance through peer comparison feedback. Med Care. 1984;22:527–34.PubMedCrossRefGoogle Scholar
  16. 16.
    Duggan AK. Analysis of the concordance between provider performance and recording [dissertation]. Baltimore, MD: The Johns Hopkins University, 1985.Google Scholar
  17. 17.
    Schulman J, Wood C. Flow sheets for charts of ambulatory patients. JAMA. 1971;217:933–7.PubMedCrossRefGoogle Scholar
  18. 18.
    Starfield B, Simborg D, Johns C, Horn S. Coordination of care and its relationship to continuity and medical records. Med Care. 1977;15:929–38.PubMedCrossRefGoogle Scholar
  19. 19.
    Williamson J. Continuing education and patient care research: physician response to screening test results. JAMA. 1967;201:118–22.CrossRefGoogle Scholar
  20. 20.
    Wooliscroft JO, Calhoon JG, Billiu GA, Beauchamp C. Medical student attention to preventive medicine: change with time and reinforcement. Am J Prev Med. 1988;4:166–71.Google Scholar
  21. 21.
    Bouchard RE, Tufo HM, Beaty HN. The impact of a quality assurance program on postgraduate training in internal medicine. JAMA. 1985;253:1146–50.PubMedCrossRefGoogle Scholar
  22. 22.
    Nelson GE, Graves SM, Holland RR, Nelson JM, Ratner J, Weed LL. A performance-based method of student evaluation. J Med Educ. 1976;10:33–42.Google Scholar
  23. 23.
    Boekeloo B. Evaluation of strategies for increasing cholesterol management practices in inpatients [dissertation]. Baltimore, MD: The Johns Hopkins University, 1988.Google Scholar
  24. 24.
    Kern DE, Grayson M, Barker LR, et al. Residency training in interviewing skills and the psychosocial domain of medical practice. J Gen Intern Med. 1989;4:421–31.PubMedCrossRefGoogle Scholar
  25. 25.
    Whiting-O’Keefe QE, Henke C, Simborg DW. Choosing the correct unit of analysis in medical care experiments. Med Care. 1984;22:1101–13.PubMedCrossRefGoogle Scholar
  26. 26.
    Linn BS. Continuing medical education: impact on emergency room burn care. JAMA. 1980;244:565–70.PubMedCrossRefGoogle Scholar
  27. 27.
    Martin AR, Wolfe MA, Thibodeau LA, Dzau V, Braunwald E. A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med. 1980;303:1330–6.PubMedCrossRefGoogle Scholar
  28. 28.
    Sanazaro PJ, Worth RM. Concurrent quality assurance in hospital care: report of a study by private initiative in PSRO. N Engl J Med. 1978;298:1171–7.PubMedCrossRefGoogle Scholar
  29. 29.
    Bennett SE, Goodson JD, Izem E, et al. Comparing ambulatory care practices of primary care and traditional medicine residents. Med Care. 1985;23:816–22.PubMedCrossRefGoogle Scholar
  30. 30.
    Ramsdell JW, Berry CC. Evaluation of general and traditional internal medicine residents utilizing a medical records audit based on educational objectives. Med Care. 1983;21:1144–53.PubMedCrossRefGoogle Scholar
  31. 31.
    Makuc DM, Fried IM, Kleinman JC. National trends in the use of preventive health care by women. Am J Public Health. 1989;79:21–6.PubMedCrossRefGoogle Scholar
  32. 32.
    Dickie GL, Bass MJ. Improving problem oriented medical records through self-audit. J Fam Pract. 1980;10:487–90.PubMedGoogle Scholar
  33. 33.
    Barker LR, Starfield B, Gross RJ, Kern DE, Levine D, Fishelman P. Recognition of information and coordination of ambulatory care by medical residents. Med Care. 1989;27(5):558–62.PubMedCrossRefGoogle Scholar
  34. 34.
    Pinkerton RE, Tinanoff N, Williams JL, Tapp JT. Resident physician performance in a continuing education format: does newly acquired knowledge improve patient care? JAMA. 1980;244:2183–5.PubMedCrossRefGoogle Scholar
  35. 35.
    Goldenberg K. Periodic health examination: comparison of residency programs and national recommendations. J Gen Intern Med. 1986;1:282–6.PubMedCrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 1990

Authors and Affiliations

  • David E. Kern
    • 1
  • William L. Harris
    • 2
  • Bradley O. Boekeloo
    • 3
  • Lee Randol Barker
    • 1
  • Patricia Hogeland
    • 1
  1. 1.the Department of MedicineThe Johns Hopkins University, School of Medicine, Francis Scott Key Medical CenterUSA
  2. 2.the Department of Information Systems and Decision SciencesLoyola College in MarylandUSA
  3. 3.the Department of Health Policy and ManagementThe Johns Hopkins University, School of Hygiene and Public Health (BOB)USA
  4. 4.Division of General Internal MedicineFrancis Scott Key Medical CenterBaltimore

Personalised recommendations