Advances in Health Sciences Education

, Volume 9, Issue 4, pp 309–323 | Cite as

Demonstration of Portfolios to Assess Competency of Residents

  • Patricia S. O’sullivan
  • Mark D. Reckase
  • Tina McClain
  • Mildred A. Savidge
  • James A. Clardy
Article

Abstract

Residency educators are identifying approaches to measure resident competence. Portfolios are well suited since they require work already completed as part of patient care where competency must be demonstrated. This paper describes assessment of the reliability and validity of portfolios in a psychiatry residency program. This was a cross-sectional study across 4years of residency education. Using guidelines, 18 residents assembled portfolios containing five entries chosen from 13 skills. Trained raters scored the portfolios. Residents and faculty were interviewed about their perceptions. Generalizability results indicated five entries and two raters were sufficient for relative decisions. Six entries or a third rater would be sufficient for absolute decisions. Portfolio scores tended to improve with years of training and correlated with psychiatric knowledge but not clinical performance. Residents and faculty identified benefits to assembling a portfolio. Portfolios incorporate tasks embedded in the residency to provide evidence of resident competency. The results support that the score is reliable and valid.

Keywords

medical education portfolios reliability residency education validity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Cogbill, K.K., O’Sullivan, P.S. & Clardy, J. (in press). Residents’ perceptions of effectiveness of twelve evaluation methods for measuring competency. Academic Psychiatry.Google Scholar
  2. Davis, M.H., Friedman Ben-David, M., Harden, R.M., Howie, P., Ker, J., McGhee, C., Pippard, M.J., Snadden, D. 2001Portfolio assessment in medical students’ final examinationsMedical Teacher23357366Google Scholar
  3. Feldt, L.S., Brennan, R.L. 1989

    Reliability

    Linn, R L. eds. Educational Measurement3American Council on Education/MacmillanNew York105146
    Google Scholar
  4. Friedman Ben David, M., Davis, M.H., Harden, R.M., Howie, P.W., Ker, J., Pippard, M.J. 2001AMEE Medical Education Guide No. 24. Portfolios as a method student assessmentMedical Teacher23535551Google Scholar
  5. Gallagher, P. 2001An evaluation of a standards based portfolioNurse Education Today21409416Google Scholar
  6. Gannon, F.T., Draper, P.R., Watson, R., Proctor, S., Norman, I.J. 2001Putting portfolios in their placeNurse Education Today21534540Google Scholar
  7. Holmboe, E.S., Hawkins, R.E. 1998Methods for evaluating the clinical competence of residents in internal medicineAnnals of Internal Medicine1294248Google Scholar
  8. Lettus, M.K , Moessner, P.H., Dooley, L. 2001The clinical portfolio as an assessment toolNursing Administration Quarterly257479Google Scholar
  9. Norcini, J.J., Blank, L.L., Arnold, G.K., Kimball, H.R. 1995The mini-CEX (clinical evaluation exercise): A preliminary investigationAnnals of Internal Medicine123795799Google Scholar
  10. O ‘Sullivan, P.S., Cogbill, K., McClain, C., Clardy, J., Reckase, M. 2002Portfolios as a novel approach for residency evaluationsAcademic Psychiatry26173179Google Scholar
  11. Pitts, J., Coles, C., Thomas, P., Smith, F. 2002Enhancing reliability in portfolio assessment: Discussions between assessorsMedical Teacher24197201Google Scholar
  12. Reckase, M.D. 1995Portfolio assessment: A theoretical estimate of score reliabilityEducational Measurement: Issues and Practice141231Google Scholar
  13. Snadden, D., Thomas, M. 1998The use of portfolio learning in medical educationMedical Teacher20192208Google Scholar
  14. Swing, S.R. 2002Assessing the ACGME general competencies: General considerations and assessment methodsAcademic Emergency Medicine912781288Google Scholar
  15. Wenzel, L.S., Briggs, K.L., Puryear, B.L. 1998Portfolio: Authentic assessment in the age of the curriculum revolutionJournal of Nursing Education37208212Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Patricia S. O’sullivan
    • 1
  • Mark D. Reckase
    • 2
  • Tina McClain
    • 3
  • Mildred A. Savidge
    • 1
  • James A. Clardy
    • 3
  1. 1.Office of Educational DevelopmentUniversity of Arkansas for Medical SciencesLittle RockUSA
  2. 2.College of EducationMichigan State UniversityUSA
  3. 3.Department of Psychiatry, College of MedicineUniversity of Arkansas for Medical SciencesLittle RockUSA

Personalised recommendations