Advertisement

Advances in Health Sciences Education

, Volume 7, Issue 1, pp 63–80 | Cite as

Measuring Self-assessment: Current State of the Art

  • Mylène Ward
  • Larry Gruppen
  • Glenn RegehrEmail author
Article

Abstract

The competent physician pursues lifelong learning through the recognition of deficiencies and the formulation of appropriate learning goals. Despite the accepted theoretical value of self-assessment, studies have consistently shown that the accuracy of self-assessment is poor. This paper examines the methodological issues that plague the measurement of self-assessment ability and presents several strategies that address these methodological problems within the current paradigm. In addition, the article proposes an alternative conceptualization of self-assessment and describes its associated methods. The conclusions of prior research in this domain must be re-examined in light of the common pitfalls encountered in the design of the studies and the analyses of the data. Future efforts to elucidate self-assessment phenomena need to consider the implications of this review.

education educational measurement medical self-assessment self-evaluation programs 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abrams, R.G. & Kelley, M.L. (1974). Student self-evaluation in a pediatric-operative technique course. Journal of Dental Education 38: 385–391.Google Scholar
  2. Antonelli, M.A.S. (1997). Accuracy of second-year medical students' self-assessment of clinical skills. Academic Medicine 72(10 Supplement 1): S63–S65.Google Scholar
  3. Arnold, L., Willoughby, T.L. & Calkins, E.V. (1985). Self-evaluation in undergraduate medical education: A longitudinal perspective. Journal of Medical Education 60: 21–28.Google Scholar
  4. Bem, D.J. & Funder, D.C. (1974). Predicting more of the people more of the time: assessing the personality of situations. Psychological Review 8: 506–520.CrossRefGoogle Scholar
  5. Bergee, M.J. (1997). Relationships among faculty, peer, and self-evaluations of applied performances. Journal of Research in Music Education 45: 601–612.CrossRefGoogle Scholar
  6. Bishop, J.B. (1971). Another look at counselor, client, and supervisor ratings of counselor effectiveness. Counselor Education and Supervision 10: 319–323.Google Scholar
  7. Block, J. (1978). The Q Sort Method in Personality Assessment and Psychiatric Research. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
  8. Boud, D.J., Churches, A.E. & Smith, E.M. (1986). Student self assessment in an engineering design course: An evaluation. International Journal of Applied Engineering Education 2(2): 83–90.Google Scholar
  9. Boud, D.J. & Tyree, A.L. (1979). Self and peer assessment in professional education: A preliminary study in law. Journal of the Society of Public Teachers of Law 15(1): 65–74.Google Scholar
  10. Burke, R.J. (1969). Some preliminary data on the use of self-evaluations and peer-ratings in assigning university course grades. Journal of Educational Research 62(10): 444–4448.Google Scholar
  11. Calhoun, J.G., Ten Haken, J.D. & Wooliscroft, J.O. (1990). Medical students' development of self and peer assessment skills: a longitudinal study. Teaching and Learning in Medicine 2: 25–29.CrossRefGoogle Scholar
  12. Calhoun, J.G., Wooliscroft, J.O., Ten Haken J.D., Wolf F.M. & Davis, W.K. (1988). Evaluating medical student clinical skill performance: Relationships among self, peer, and expert ratings. Evaluation & the Health Professions 11: 201–212.Google Scholar
  13. Chiu, L.H. (1975). Influence of student teaching on perceived teaching competence. Perceptual and Motor Skills 40: 872–874.Google Scholar
  14. Cochran, S.B. & Spears, M.C. (1980). Student self-assessment and instructors' ratings: A comparison. Journal of the American Dietetic Association 76: 253–257.Google Scholar
  15. Coutts, L. & Rogers, J. (1999). Predictors of student self-assessment accuracy during a clinical performance exam: comparisons between over-estimators and under-estimators of SP-evaluated performance. Academic Medicine 74(10 Supplement): S128–S130.Google Scholar
  16. Daniel, S.J., Scruggs, R.R. & Grady, J.J. (1990). Accuracy of student self-evaluations of dental sealants. Journal of Dental Hygiene 64: 339–342.Google Scholar
  17. Das, M., Mpofu, D., Dunn, E. & Lanphear, J.H. (1998). Self and tutor evaluations in problem-based learning tutorials: Is there a relationship? Medical Education 32: 411–418.CrossRefGoogle Scholar
  18. D'Augelli, A.R. (1973). The assessment of interpersonal skills: A comparison of observer, peer and self ratings. Journal of Community Psychology 1: 177–179.Google Scholar
  19. Davis, J.K. & Rand, D.C. (1980). Self-grading versus instructor grading. Journal of Educational Research 73(4): 207–211.Google Scholar
  20. Doleys, E.J. & Renzaglia, G.A. (1963). Accuracy of student prediction of college grades. Personnel and Guidance Journal 41(6): 528–530.Google Scholar
  21. Everett, M.S. (1983). Influence of trait anxiety on self-grading. Educational Directions 8(1): 4–9.Google Scholar
  22. Falchikov, N. (1986). Product comparisons and process benefits of collaborative peer group and self assessments. Assessment and Evaluation in Higher Education 11(2): 146–166.Google Scholar
  23. Falchikov, N. & Boud, D. (1989). Student self-assessment in higher education: A meta-analysis. Review of Educational Research 59: 395–430.CrossRefGoogle Scholar
  24. Falchikov, N. & Goldfinch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of Educational Research 70: 287–322.CrossRefGoogle Scholar
  25. Farnill, D., Hayes, S.C. & Todisco, J. (1997) Interviewing skills: Self-evaluation by medical students. Medical Education 31: 122–127.Google Scholar
  26. Filene, P.O. (1969). Self-grading: An experiment in learning. Journal of Higher Education 40: 451–458.CrossRefGoogle Scholar
  27. Fitzgerald, J.T., Gruppen, L.D. & White, C.B. (2000). The influence of task formats on the accuracy of medical students' self-assessments. Academic Medicine 75: 737–741.CrossRefGoogle Scholar
  28. Forehand, L.S., Vann, W.F. & Shugars, D.A. (1982). Student self-evaluation in preclinical restorative dentistry. Journal of Dental Education 46(4): 221–226.Google Scholar
  29. Fuqua, D.R., Johnson, A.W., Newman, J.L., Anderson, M.W. & Gade, E.M. (1984). Variability across sources of performance ratings. Journal of Counselling Psychology 31(2): 249–252.CrossRefGoogle Scholar
  30. Gaier, E.L. (1961). Student self estimates of final course grades. Journal of Genetic Psychology 98: 63–67.Google Scholar
  31. Geissler, P.R. (1973). Student self-assessment in dental technology. Journal of Dental Education 37: 19–21.Google Scholar
  32. Gordon, M.J. (1991). A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine 66: 762–769.CrossRefGoogle Scholar
  33. Gordon, M.J. (1992). Self-assessment programs and their implications for health professions training. Academic Medicine 67: 672–679.CrossRefGoogle Scholar
  34. Gray, J.D. (1996). Global rating scales in residency education. Academic Medicine 71(1 Supplement): S55–S63.Google Scholar
  35. Gray, T.G.F. (1987). An exercise in improving the potential of exams for learning. European Journal of Engineering Education 12(4): 311–323.Google Scholar
  36. Greenfield, D.G. (1978). Evaluation of music therapy practicum competencies: Comparisons of self and instructor ratings of videotapes. Journal of Music Therapy 15(1): 15–20.Google Scholar
  37. Gruppen, L.D., White, C., Fitzgerald, J.T., Grum, C.M. & Wooliscroft, J.O. (2000). Medical students' self-assessments and their allocations of learning time. Academic Medicine 75: 374–379.CrossRefGoogle Scholar
  38. Gruppen, L.D., Garcia, J., Grum, C.M., Fitzgerald, J.T., White, C.A., Dicken, L. et al. (1997). Medical students' self-assessment accuracy in communication skills. Academic Medicine 72(10 Supplement 1): S57–S59.CrossRefGoogle Scholar
  39. Gruppen, L.D., Baliga, S., Fitzgerald, J.T. et al. (1998). Do personal characteristics influence self-assessment accuracy? In, Proceedings of the 8th International Ottawa Conference on Medical Education, pp. 304–309. Philadelphia, PA.Google Scholar
  40. Harrington, J.P., Murnaghan, J.J. & Regehr, G. (1997). Applying a relative ranking model to the self-assessment of extended performances. Advances in Health Sciences Education 2: 17–25.CrossRefGoogle Scholar
  41. Hay, J.A. (1995). Investigating the development of self-evaluation skills in problem-based tutorial course. Academic Medicine 70: 733–735.CrossRefGoogle Scholar
  42. Henbest, R.J. & Fehrsen, G.S. (1985). Preliminary study at the medical university of Southern Africa on student self-assessment as a means of evaluation. Journal of Medical Education 60: 66–68.Google Scholar
  43. Herbert, W.N.P., McGaghie, W.C., Droegemueller, W., Riddle, M.H. & Maxwell, K.L. (1990). Student evaluation in obstetrics and gynecology: self versus departmental assessment. Obstetrics & Gynecology 76: 458–461.Google Scholar
  44. Hodges, B., Regehr, G. & Martin D. (2001) Difficulties in recognizing one's own incompetence: novice physicians who are unskilled and unaware of it. Academic Medicine; 76(10 Supplement): S87–89.Google Scholar
  45. Irvine, J.J. (1983). The accuracy of pre-service teachers' assessments of their classroom behaviors. Journal of Research and Development in Education 17(1): 25–31.Google Scholar
  46. Israelite, L. (1983). Adult student self-evaluation. Performance and Instruction Journal 22(5): 15–16.Google Scholar
  47. Johnson, D. & Cujec, B. (1998). Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit. Critical Care Medicine 26: 1811–1816.Google Scholar
  48. Kaiser, S. & Bauer, J.J. (1995). Checklist self-evaluation in a standardized patient exercise. American Journal of Surgery 169: 418–420.CrossRefGoogle Scholar
  49. Keefer, K.E. (1971). Characteristics of students who make accurate and inaccurate self-predictions of college achievement. Journal of Educational Research 64(9): 401–404.Google Scholar
  50. Kegel-Flom, P. (1975). Predicting supervisor, peer, and self ratings of intern performance. Journal of Medical Educcation 50: 812–815.Google Scholar
  51. Keynan, A., Friedman, M. & Benbassat, J. (1987). Reliability of global rating scales in the assessment of clinical competence of medical students. Medical Education 21: 477–481.Google Scholar
  52. Kolm, P. & Verhulst, S.J. (1987). Comparing self and supervisor evaluations - a different view. Evaluation & the Health Professions 10: 80–89.Google Scholar
  53. LeBlanc, R. & Painchaud, G. (1985). Self assessment as a second language placement instrument. TESOL Quarterly 19(4): 673–687.Google Scholar
  54. Leichner, P. & Kalin, R. (1980). Results of the first Canadian psychiatric knowledge self assessment for residents. Canadian Journal of Psychiatry 25(4): 281–289.Google Scholar
  55. Linn, B.S., Arostegui, M. & Zeppa, R. (1975). Performance rating scale for peer and self assessment. British Journal of Medical Education 9: 98–101.Google Scholar
  56. Linn, B.S., Arostegui, M. & Zeppa, R. (1976). Peer and self assessment in undergraduate surgery. Journal of Surgical Research 21: 453–456.CrossRefGoogle Scholar
  57. MacFadyen, J. & Turnbull, J. (1985). Effect of curriculum on student self-assessment of communication skills. In Proceedings of the 6th Ottawa Conference of Medical Education and Assessment, pp. 153–154, Toronto, ON.Google Scholar
  58. Mast, T.A. & Bethart, H. (1978). Evaluation of clinical dental procedures by senior dental students. Journal of Dental Education 42(4): 196–197.Google Scholar
  59. Martin, D., Regehr, G., Hodges, B. & McNaughton, N. (1998). Using videotaped benchmarks to improve the self-assessment ability of family practice residents. Academic Medicine 73: 1201–1206.CrossRefGoogle Scholar
  60. McGeever, P.J. (1978). Student self-grading in the introductory American politics course. Teaching Political Science 5(3): 319–330.Google Scholar
  61. Mihal, W.L. & Graumenz, J.L. (1984). An assessment of the accuracy of self-assessment of career decision making. Journal of Vocational Behavior 25: 245–253.CrossRefGoogle Scholar
  62. Moore, D.E. & Cordes, D.L. (1992). Needs assessment. In A.B. Rosof & W.C. Felch (eds.), Continuing Medical Education: A Primer, 2nd ed., pp. 42–51. Westport, CT: Praegar.Google Scholar
  63. Morton, J.B. & MacBeth, W.A. (1977). Correlations between staff, peer and self assessments of fourth-year students in surgery. Medical Education 11: 167–170.Google Scholar
  64. Mowrer, O.H. (1953). 'Q-technique' - description, history, and critique. In O.H. Mowrer (ed.), Psychotherapy Theory and Research, pp. 316–375. New York: Ronald.Google Scholar
  65. Mueller, R.H. (1970). Is self-grading the answer? Journal of Higher Education 41(3): 221–224.CrossRefGoogle Scholar
  66. Murstein, B.I. (1965). The relationship of grade expectations and grades believed to be deserved to actual grades received. Journal of Experimental Education 33(4): 357–362.Google Scholar
  67. O'Neill, G.P. (1985). Self, teacher, and faculty assessments of student teaching performance: A second scenario. Alberta Journal of Educational Research 31(2): 88–98.Google Scholar
  68. Palmer, P.B., Henry, J.N. & Rohe, D.A. (1985). Effect of videotape replay on the quality and accuracy of student self-evaluation. Physical Therapy 65: 497–501.Google Scholar
  69. Pease, D. (1975). Comparing faculty and school supervisor ratings for education students. College Student Journal 9(1): 91–94.Google Scholar
  70. Pitishkin-Potanich, V. (1983). On evaluating students' knowledge. Higher Education in Europe 8(2): 18–22.Google Scholar
  71. Plorde, D.S., Wollitzer, A.O. & Blossom, H. J. (1985). A self-assessment questionnaire for patient educators. Journal of Medical Education 60: 728–730.Google Scholar
  72. Pohlmann, J.L. & Beggs, D.L. (1974). A study of the validity of self-reported measures of academic growth. Journal of Educational Measurement 12: 115–118.CrossRefGoogle Scholar
  73. Regehr, G., Hodges, B., Tiberius, R. & Lofchy, J. (1996). Measuring self-assessment skills: An innovative relative ranking model. Academic Medicine 71(10 Supplement): S52–S4.CrossRefGoogle Scholar
  74. Rezler, A.G. (1989). Self-assessment in problem-based groups. Medical Teacher 11: 151–156.Google Scholar
  75. Risucci, D.A., Tortolani, A.J. & Ward, R.J. (1989). Ratings of surgical residents by self, supervisors, and peers. Surgery, Gynecology & Obstetrics 169: 519–526.Google Scholar
  76. Sclabassi, S.E. & Woelfel, S.K. (1984). Development of self-assessment skills in medical students. Medical Education 84: 226–231.CrossRefGoogle Scholar
  77. Spencer, J.A. & Jordan, R.K. (1999). Learner centred approaches in medical education. BMJ 318: 1280–1283.Google Scholar
  78. Stanton, H.E. (1978). Self-grading as an assessment method. Improving College and University Teaching 26: 236–238.Google Scholar
  79. Stover, R.V. (1976). The impact of self-grading on performance and evaluation in a constitutional law course. Teaching Political Science 3(3): 303–310.Google Scholar
  80. Stuart, M.R., Goldstein, H.S. & Snope, F.C. (1980). Self-evaluation by residents in family medicine. Journal of Family Practrice 10: 639–642.Google Scholar
  81. Turnbull, J., Gray, J. & MacFadyen, J. (1998). Improving in-training evaluation programs. JGIM 13: 317–323.CrossRefGoogle Scholar
  82. Wheeler, A.E. & Knoop, H.R. (1981). Self, teacher and faculty assessments of student teaching performance. Journal of Educational Research 75(3): 171–181.Google Scholar
  83. Wooliscroft, J.O., TenHaken, J., Smith, J. & Calhoun, J.G.. (1993) Medical students' clinical self-assessments: comparisons with external measures of performance and the students' selfassessments of overall performance and effort. Academic Medicine 68: 285–294.Google Scholar
  84. Zonia, S.C. & Stommel, M. (2000). Interns' self-evaluations compared with their faculty's evaluations. Academic Medicine 75: 742.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  1. 1.Department of SurgeryUniversity of TorontoTorontoCanada
  2. 2.Department of Medical EducationUniversity of MichiganAnn ArborUSA
  3. 3.Department of PsychiatryUniversity of TorontoTorontoCanada

Personalised recommendations