Journal of General Internal Medicine

, Volume 15, Issue 11, pp 782–788 | Cite as

Measuring compliance with preventive care guidelines

Standardized patients, clinical vignettes, and the medical record
  • Timothy R. Dresselhaus
  • John W. Peabody
  • Martin Lee
  • Ming Ming Wang
  • Jeff Luck
Original Articles


OBJECTIVE: To determine how accurately preventive care reported in the medical record reflects actual physician practice or competence.

DESIGN: Scoring criteria based on national guidelines were developed for 7 separate items of preventive care. The preventive care provided by randomly selected physicians was measured prospectively for each of the 7 items. Three measurement methods were used for comparison: (1) the abstracted medical record from a standardized patient (SP) visit; (2) explicit reports of physician practice during those visits from the SPs, who were actors trained to present undetected as patients; and (3) physician responses to written case scenarios (vignettes) identical to the SP presentations.

SETTING: The general medicine primary care clinics of two university-affiliated VA medical centers.

PARTICIPANTS: Twenty randomly selected physicians (10 at each site) from among eligible second- and third-year general internal medicine residents and attending physicians.

MEASUREMENTS AND MAIN RESULTS: Physicians saw 160 SPs (8 cases × 20 physicians). We calculated the percentage of visits in which each prevention item was recorded in the chart, determined the marginal percentage improvement of SP checklists and vignettes over chart abstraction alone, and compared the three methods using an analysis-of-variance model. We found that chart abstraction underestimated overall prevention compliance by 16% (P < .01) compared with SP checklists. Chart abstraction scores were lower than SP checklists for all seven items and lower than vignettes for four items. The marginal percentage improvement of SP checklists and vignettes to performance as measured by chart abstraction was significant for all seven prevention items and raised the overall prevention scores from 46% to 72% (P < .0001).

CONCLUSIONS: These data indicate that physicians perform more preventive care than they report in the medical record. Thus, benchmarks of preventive care by individual physicians and institutions that rely solely on the medical record may be misleading, at best.

Key words

compliance preventive care guidelines physician practice 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    US Preventive Services Task Force. Guide to Clinical Preventive Services. 2nd ed. Baltimore: Williams & Wilkins; 1996.Google Scholar
  2. 2.
    McGinnis JM, Lee PR. Healthy people 2000 at mid decade. JAMA. 1995;273:1123–9.PubMedCrossRefGoogle Scholar
  3. 3.
    Benson V, Marano MA. Current estimates from the National Health Interview Survey. Hyattsville, Md: National Center for Health Statistics; 1994. Vital Health Statistics 10 (189).Google Scholar
  4. 4.
    Audet AM, Scott HD. The uniform clinical data set: an evaluation of the proposed national database for Medicare’s quality review program. Ann Intern Med. 1993;119:1209–13.PubMedGoogle Scholar
  5. 5.
    Lawthers AG, Palmer RH, Edwards JE, et al. Developing and evaluating performance measures for ambulatory care quality: a preliminary report of the DEMPAQ project. Jt Comm J Qual Improv. 1993;19:552–65.PubMedGoogle Scholar
  6. 6.
    McGlynn EA. Six challenges in measuring the quality of health care. Health Aff. 1997;16:7–21.CrossRefGoogle Scholar
  7. 7.
    Leshan LA, Fitzsimmons M, Marbella A, Gottlieb M. Increasing clinical prevention efforts in a family practice residency program through CQI methods. Jt Comm Qual Improv. 1997;23:391–400.Google Scholar
  8. 8.
    Sanazaro PJ, Worth RM. Measuring clinical performance of individual internists in office and hospital practice. Med Care. 1985;23:1097–114.PubMedCrossRefGoogle Scholar
  9. 9.
    Dietrich AJ, Goldberg H. Preventive content of adult primary care: do generalists and subspecialists differ? Am J Public Health. 1984;74:223–7.PubMedGoogle Scholar
  10. 10.
    Carey TS, Levis D, Pickard CG, Bernstein J. Development of a model quality-of-care assessment program for adult preventive care in rural medical practices. QRB Qual Rev Bull. 1991;17:54–9.PubMedGoogle Scholar
  11. 11.
    Garnick DW, Fowles J, Lawthers AG, et al. Focus on quality: profiling physicians’ practice patterns. J Ambulatory Care Manage. 1994;17:44–75.PubMedGoogle Scholar
  12. 12.
    Davies AR, Ware JE. Involving consumers in quality of care assessment. Health Aff. 1988;7(1):33–48.CrossRefGoogle Scholar
  13. 13.
    Safran DG, Kosinski M, Tarlov AR, et al. The Primary Care Assessment Survey: tests of data quality and measurement performance. Med Care. 1998;36:728–39.PubMedCrossRefGoogle Scholar
  14. 14.
    Woo B, Woo B, Cook EF, Weisberg M, Goldman L. Screening procedures in the asymptomatic adult: comparison of physicians’ recommendations, patients’ desires, published guidelines, and actual practice. JAMA. 1985;254:1480–4.PubMedCrossRefGoogle Scholar
  15. 15.
    Romm FJ, Fletcher SW, Hulks BS. The periodic health examination: comparison of recommendations and internists’ performance. South Med J. 1981;74:265–71.PubMedGoogle Scholar
  16. 16.
    McPhee SJ, Richard RJ, Solkowitz SN. Performance of cancer screening in a university general internal medicine practice: comparison with the 1980 American Cancer Society Guidelines. J Gen Intern Med. 1986;1:275–81.PubMedCrossRefGoogle Scholar
  17. 17.
    Hershey CO, Karuza J. Assessment of preventive health care. Design consideration. Prev Med. 1997;26:59–67.PubMedCrossRefGoogle Scholar
  18. 18.
    Lawthers AG, Palmer RH, Banks N, Garnick DW, Fowles J, Weiner JP. Designing and using measures of quality based on physician office records. J Ambulatory Care Manage. 1995;18:56–72.PubMedGoogle Scholar
  19. 19.
    Beullens J, Rethans JJ, Goedhuys J, Buntinx F. The use of standardized patients in research in general practice. Fam Pract. 1997;14:58–62.PubMedCrossRefGoogle Scholar
  20. 20.
    Vu NV, Marcy MM, Colliver JA, Verhulst SJ, Travis TA, Barrows HS. Standardized (simulated) patients’ accuracy in recording clinical performance check-list items. Med Educ. 1992;26:99–104.PubMedCrossRefGoogle Scholar
  21. 21.
    Schwartz MH, Colliver JA. Using standardized patients for assessing clinical performance: an overview. Mt Sinai J Med. 1996;63:241–9.Google Scholar
  22. 22.
    Ferrell BG. Clinical performance assessment using standardized patients. A primer. Special series: core concepts in family medicine education. Fam Med. 1995;27:14–9.PubMedGoogle Scholar
  23. 23.
    Kinnersley P, Pill R. Potential of using simulated patients to study the performance of general practitioners. Br J Gen Pract. 1993;43:297–300.PubMedGoogle Scholar
  24. 24.
    Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients and chart abstraction: a prospective validation study of three methods for measuring quality. JAMA. 2000;283:1715–22.PubMedCrossRefGoogle Scholar
  25. 25.
    Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of quality between standardized patients and the medical record. Am J Med. 2000;108:642–9.PubMedCrossRefGoogle Scholar
  26. 26.
    Gerbert B, Hargreaves WA. Measuring physician behavior. Med Care. 1986;24:838.PubMedCrossRefGoogle Scholar
  27. 27.
    Bates DW, Pappius E, Kuperman GJ, et al. Using information systems to measure and improve quality. Int J Med Inf. 1999;53:115–24.CrossRefGoogle Scholar
  28. 28.
    Bates DW, Kuperman G, Teich JM. Computerized physician order entry and quality of care. Quality Manag Health Care. 1994;2:18–27.Google Scholar

Copyright information

© Society of General Internal Medicine 2000

Authors and Affiliations

  • Timothy R. Dresselhaus
    • 1
    • 2
  • John W. Peabody
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
  • Martin Lee
    • 12
    • 8
    • 9
    • 10
  • Ming Ming Wang
    • 12
  • Jeff Luck
    • 12
    • 10
    • 11
  1. 1.the Veterans Affairs San Diego Healthcare SystemSan Diego
  2. 2.University of California, San Diego, School of MedicineSan Diego
  3. 3.the San Francisco Veterans Affairs Medical CenterSan Francisco
  4. 4.the University of CaliforniaSan Francisco
  5. 5.School of MedicineSan Francisco
  6. 6.the Veterans Affairs Center for the Study of Healthcare Provider BehaviorSepulveda
  7. 7.the University of CaliforniaLos Angeles
  8. 8.Schools of MedicineLos Angeles
  9. 9.Public HealthLos Angeles
  10. 10.RANDSanta Monica
  11. 11.the Veterans Affairs Greater Los Angeles Healthcare SystemLos Angeles
  12. 12.Institute for Global HealthSan Francisco

Personalised recommendations