Skip to main content

Training and Validation of Standardized Patients for Unannounced Assessment of Physicians’ Management of Depression

Academic Psychiatry volume 35, pages 382–387 (2011)Cite this article

Abstract

Objective

Standardized patients (SPs) have been developed to measure practitioner performance in actual practice settings, but results have not been fully validated for psychiatric disorders. This study describes the process of creating reliable and valid SPs for unannounced assessment of general-practitioners’ management of depression disorders in Iran.

Method

Ten psychology and nursing students (potential SPs) took part in a five-session course involving training in dialogue and body language. Five scenarios, along with corresponding checklists representing common presentations of mood disorders in primary-care settings, were developed by an expert group. The SPs’ role-play performance of their respective scenario was videotaped and scored independently by three psychiatrists according to an observational rating scale to assess validity. The role-play was repeated after 1 week with the same scenario and the same doctor, to assess test-retest reliability. The reliability of each checklist to be used by the SPs was assessed by testing interrater reliability between groups of SPs.

Results

The cutoff score for the SPs’ portrayal validity was 90% or above for all SPs. Mean interrater reliability for the checklists was acceptable for the SPs watching the same videos and filling in the checklists, while the mean kappa for assessing concurrent validity in filling in the checklists was lower. The test-retest performance for assessing reliability resulted in a mean kappa of 0.72. All SPs except one, who was not recruited, performed acceptably well.

Conclusion

The authors have demonstrated a thorough validation of the technique of using standardized patients in the portrayal of depressive disorders in primary-care settings in Iran, which creates confidence in employing this technique to evaluate doctors’ performance, for example, after an educational intervention. Similar methods of validation can be used for SPs’ portrayal of other psychiatric disorders.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Boulet JR, Murray D, Kras J, et al: Reliability and validity of a simulation-based acute-care skills assessment for medical students and residents. Anesthesiology 2003; 99:1270–1280

    PubMed  Article  Google Scholar 

  2. Barrows HS, Abrahamson S: The programmed patient: a technique for appraising student performance in clinical neurology. J Med Educ 1964; 39:802–805

    CAS  PubMed  Google Scholar 

  3. Rethans JJ, Gorter S, Bokken L, et al: Unannounced standardized patients in real practice: a systematic literature review. Med Educ 2007; 41:537–549

    PubMed  Article  Google Scholar 

  4. Carney PA, Dietrich AJ, Freeman DH Jr, et al: A standardized-patient assessment of a continuing medical education program to improve physicians’ cancer-control clinical skills. Acad Med 1995; 70:52–58

    CAS  PubMed  Article  Google Scholar 

  5. Maiburg BH, Rethans JJ, Schuwirth LW, et al: Controlled trial of effect of computer-based nutrition course on knowledge and practice of general-practitioner trainees. Am J Clin Nutr2003; 77(suppl):1019S–1024S

    CAS  PubMed  Google Scholar 

  6. Mann K: Continuing medical education, in International Handbook of Research in Medical Education. Edited by Norman G, van der Vluten CPM, Newble DI. Dordrecht, The Netherlands, Kluwer Academic, 2002, pp 415–457

  7. McNaughton N, Ravitz P, Wadell A, et al: Psychiatric education and simulation: a review of the literature. Can J Psychiatry 2008; 53:85–93

    PubMed  Google Scholar 

  8. Shirazi M, Zeinaloo A, Parish SV, et al: Effects on readiness to change of an educational intervention on depressive disorders for general physicians in primary care based on a modified Prochaska model: a randomized, controlled study. Fam Pract 2008; 25:98–104

    CAS  PubMed  Article  Google Scholar 

  9. Shirazi M, Parikh SV, Alaeddini F, et al: Effects on knowledge and attitudes of using stages of change to train general practitioners on management of depression: a randomized, controlled study. Can J Psychiatry 2009; 54:43–50

    Google Scholar 

  10. Shirazi M, Assadi SM, Sadeghi M, et al: Applying a modified Prochaska’s model of readiness to change for general practitioners on depressive disorders in CME programmes: validation of tool. J Eval Clin Pract 2007; 13:298–302

    PubMed  Article  Google Scholar 

  11. Sadeghi M, Shirazi M, Arbabi M, et al:. Guidebook for Management and Treatment of Depression Disorders by General Physicians. Tehran, Iran, Tehran University of Medical Sciences, 2007

    Google Scholar 

  12. Ormel J, von Korff M, Ustun TB, et al: Common mental disorders and disability across cultures: results from the World Health Organization Collaborative Study on Psychological Problems in General Health Care. JAMA 1994; 272: 1741–1748

    CAS  PubMed  Article  Google Scholar 

  13. Moussaoui M, Lopez-Ibore J, Okasha A, et al: Depression in general practice. WPA Bull on Depression 2003 (10-Year Special)

  14. American Psychiatrie Association: Diagnosis and statistical Manual of Mental Disorders, 4th Edition (DSM-IV-TR). Washington, DC, American Psychiatric Association, 2000

    Google Scholar 

  15. McLeod PJ, Tamblyn RM, Gayton D, et al: Use of standardized patients to assess between-physician variations in resource utilization. JAMA 1997; 278:1164–1168

    CAS  PubMed  Article  Google Scholar 

  16. Ramsey PG, Curtis JR, Paauw DS, et al: History-taking and preventive medicine skills among primary care physicians: an assessment using standardized patients. Am J Med 1998; 104:152–158

    CAS  PubMed  Article  Google Scholar 

  17. Glassman PA, Luck J, O’Gara EM, et al: Using standardized patients to measure quality: evidence from the literature and a prospective study. Jt Comm J Qual Improv 2000; 26:644–653

    CAS  PubMed  Google Scholar 

  18. Kravitz RL, Epstein RM, Feldman MD, et al: Influence of patients’ requests for direct-to-consumer advertised antidepressants: a randomized controlled trial. JAMA 2005; 293:1995–2002

    CAS  PubMed Central  PubMed  Article  Google Scholar 

  19. Mohl PC: Of standardized patients and OSCEs. Acad Psychiatry 2002; 26:133

    PubMed  Article  Google Scholar 

  20. Hodges B, Regehr G, McNaughton N, et al: OSCE checklists do not capture increasing levels of expertise. Acad Med 1999; 74:1129–1134

    CAS  PubMed  Article  Google Scholar 

  21. Regehr G, Freeman R, Hodges B, et al: Assessing the generalizability of OSCE measures across content domains. Acad Med 1999; 74:1320–1322

    CAS  PubMed  Article  Google Scholar 

  22. Lafave MR, Katz L, Donnon T, et al: Initial reliability of the standardized orthopedic assessment tool (SOAT). J Athl Train 2008; 43:483–488

    PubMed Central  PubMed  Article  Google Scholar 

  23. Tamblyn R, Grad R, Gayton D, et al: Impact of inaccuracies in standardized patient portrayal and recording on physician performance during blinded clinic visits. Teach Learn Med 1997; 9:25–38

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Psychiatry, Tehran University of Medical Sciences, Tehran, Iran

    Mandana Shirazi Ph.D., Majid Sadeghi M.D., A. Emami M.D., A. Sabouri Kashani M.D., Sagar Parikh M.D., F. Alaeddini M.D., Mohammad Arbabi M.D. & Rolf Wahlstrom M.D.

Authors
  1. Mandana Shirazi Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Majid Sadeghi M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Emami M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. Sabouri Kashani M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sagar Parikh M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. F. Alaeddini M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mohammad Arbabi M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rolf Wahlstrom M.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mandana Shirazi Ph.D..

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Shirazi, M., Sadeghi, M., Emami, A. et al. Training and Validation of Standardized Patients for Unannounced Assessment of Physicians’ Management of Depression. Acad Psychiatry 35, 382–387 (2011). https://doi.org/10.1176/appi.ap.35.6.382

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1176/appi.ap.35.6.382

Keywords

  • Academic Psychiatry
  • Standard Patient
  • Continue Medical Education
  • Global Rating Scale
  • Continue Medical Education Programme