Journal of General Internal Medicine

, Volume 29, Issue 8, pp 1183–1187 | Cite as

Directly Observed Care: Can Unannounced Standardized Patients Address a Gap in Performance Measurement?

  • Saul J. WeinerEmail author
  • Alan Schwartz


There are three potential sources of information for evaluating a clinician’s performance: documentation, patient report, and directly observed care. Current measures draw on just two of these: data recorded in the medical record and surveys of patients. Neither captures an array of performance characteristics, including clinician attention to symptoms and signs while taking a history or conducting a physical exam, accurate recording in the medical record of information obtained during the encounter, evidence based communication strategies for preventive care counseling, and effective communication behavior. Unannounced Standardized Patients (USPs) have been widely deployed as a research strategy for systematically uncovering significant performance deficits in each of these areas, but have not been adopted for quality improvement. Likely obstacles include concerns about the ethics of sending health professionals sham patients, the technical challenges of the subterfuge, and concerns about the relatively small sample sizes and substantial costs involved. However, the high frequency of significant and remediable performance deficits unmasked by USPs, and the potential to adapt registration and record keeping systems to accommodate their visits, suggest that their selective and purposeful deployment could be a cost effective and powerful strategy for addressing a gap in performance measurement.


Psoriatic Arthritis Continuous Quality Improvement Performance Deficit Preventive Care Service Clinician Performance Measurement 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Drs. Weiner and Schwartz’s research on unannounced standardized patients has been supported by the Veterans Affairs, Health Services Research and Development, and the National Board of Medical Examiners.


The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs or the United States government.


The authors are co-principals of the Institute for Practice and Provider Performance Improvement, founded to employ unannounced standardized patient assessments as a quality improvement service. They have not at this point received any fees or contracts. There are no other relationships or activities that could appear to have influenced the submitted work.


  1. 1.
    National Center for Quality Assurance. 2013, at Accessed March 2014.
  2. 2.
    Luck J, Peabody JW. Using standardised patients to measure physicians’ practice: validation study using audio recordings. BMJ. 2002;325:679.PubMedCentralPubMedCrossRefGoogle Scholar
  3. 3.
    Glassman PA, Luck J, O’Gara EM, Peabody JW. Using standardized patients to measure quality: evidence from the literature and a prospective study. Jt Comm J Qual Improv. 2000;26:644–53.PubMedGoogle Scholar
  4. 4.
    Dresselhaus TR, Luck J, Peabody JW. The ethical problem of false positives: a prospective evaluation of physician reporting in the medical record. J Med Ethics. 2002;28:291–4.PubMedCentralPubMedCrossRefGoogle Scholar
  5. 5.
    Dresselhaus TR, Peabody JW, Lee M, Wang MM, Luck J. Measuring compliance with preventive care guidelines: standardized patients, clinical vignettes, and the medical record. J Gen Intern Med. 2000;15:782–8.PubMedCentralPubMedCrossRefGoogle Scholar
  6. 6.
    Krane NK, Anderson D, Lazarus CJ, et al. Physician practice behavior and practice guidelines: using unannounced standardized patients to gather data. J Gen Intern Med. 2009;24:53–6.PubMedCentralPubMedCrossRefGoogle Scholar
  7. 7.
    Hornung RL, Hansen LA, Sharp LK, Poorsattar SP, Lipsky MS. Skin cancer prevention in the primary care setting: assessment using a standardized patient. Pediatr Dermatol. 2007;24:108–12.PubMedCrossRefGoogle Scholar
  8. 8.
    Gorter S, van der Heijde DM, van der Linden S, et al. Psoriatic arthritis: performance of rheumatologists in daily practice. Ann Rheum Dis. 2002;61:219–24.PubMedCentralPubMedCrossRefGoogle Scholar
  9. 9.
    Weiner SJ, Schwartz A, Weaver F, et al. Contextual errors and failures in individualizing patient care: a multicenter study. Ann Intern Med. 2010;153:69–75.PubMedCrossRefGoogle Scholar
  10. 10.
    Finn A. Shopper benchmarking of durable-goods chains and stores. J Serv Res. 2001;3:310–20.CrossRefGoogle Scholar
  11. 11.
    Levine M. CEJA Report 3-A-08 Secret Shopper “Patients.”. Chicago, IL: American Medical Association; 2008.Google Scholar
  12. 12.
    Franz CE, Epstein R, Miller KN, et al. Caught in the act? Prevalence, predictors, and consequences of physician detection of unannounced standardized patients. Health Serv Res. 2006;41:2290–302.PubMedCentralPubMedCrossRefGoogle Scholar
  13. 13.
    Speedie SM, Niewoehner C. The Minnesota Virtual Clinic: using a simulated EMR to teach medical students basic science and clinical concepts. AMIA Annual Symposium proceedings / AMIA Symposium AMIA Symposium 2003:1013.Google Scholar
  14. 14.
    van Zanten M, Boulet JR, McKinley D. Using standardized patients to assess the interpersonal skills of physicians: six years’ experience with a high-stakes certification examination. Health Commun. 2007;22:195–205.PubMedCrossRefGoogle Scholar
  15. 15.
    Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA. 2007;298:993–1001.PubMedCrossRefGoogle Scholar
  16. 16.
    Whelan GP, Boulet JR, McKinley DW, et al. Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG Clinical Skills Assessment (CSA). Med Teach. 2005;27:200–6.PubMedCrossRefGoogle Scholar
  17. 17.
    Schwartz A, Weiner SJ, Weaver F, et al. Uncharted territory: measuring costs of diagnostic errors outside the medical record. BMJ Qual Saf 2012.Google Scholar
  18. 18.
    Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Measure, learn, and improve: physicians’ involvement in quality improvement. Health Aff (Millwood). 2005;24:843–53.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2014

Authors and Affiliations

  1. 1.VA Center of Innovation for Complex Chronic HealthcareJesse Brown VA Medical CenterChicagoUSA
  2. 2.Departments of Medicine and PediatricsUniversity of Illinois at ChicagoChicagoUSA
  3. 3.Department of Medical EducationUniversity of Illinois at ChicagoChicagoUSA

Personalised recommendations