Advances in Health Sciences Education

, Volume 7, Issue 1, pp 63–80 | Cite as

Measuring Self-assessment: Current State of the Art

Article

Abstract

The competent physician pursues lifelong learning through the recognition of deficiencies and the formulation of appropriate learning goals. Despite the accepted theoretical value of self-assessment, studies have consistently shown that the accuracy of self-assessment is poor. This paper examines the methodological issues that plague the measurement of self-assessment ability and presents several strategies that address these methodological problems within the current paradigm. In addition, the article proposes an alternative conceptualization of self-assessment and describes its associated methods. The conclusions of prior research in this domain must be re-examined in light of the common pitfalls encountered in the design of the studies and the analyses of the data. Future efforts to elucidate self-assessment phenomena need to consider the implications of this review.

education educational measurement medical self-assessment self-evaluation programs 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abrams, R.G. & Kelley, M.L. (1974). Student self-evaluation in a pediatric-operative technique course. Journal of Dental Education 38: 385–391.Google Scholar
  2. Antonelli, M.A.S. (1997). Accuracy of second-year medical students' self-assessment of clinical skills. Academic Medicine 72(10 Supplement 1): S63–S65.Google Scholar
  3. Arnold, L., Willoughby, T.L. & Calkins, E.V. (1985). Self-evaluation in undergraduate medical education: A longitudinal perspective. Journal of Medical Education 60: 21–28.Google Scholar
  4. Bem, D.J. & Funder, D.C. (1974). Predicting more of the people more of the time: assessing the personality of situations. Psychological Review 8: 506–520.CrossRefGoogle Scholar
  5. Bergee, M.J. (1997). Relationships among faculty, peer, and self-evaluations of applied performances. Journal of Research in Music Education 45: 601–612.CrossRefGoogle Scholar
  6. Bishop, J.B. (1971). Another look at counselor, client, and supervisor ratings of counselor effectiveness. Counselor Education and Supervision 10: 319–323.Google Scholar
  7. Block, J. (1978). The Q Sort Method in Personality Assessment and Psychiatric Research. Palo Alto, CA: Consulting Psychologists Press.Google Scholar
  8. Boud, D.J., Churches, A.E. & Smith, E.M. (1986). Student self assessment in an engineering design course: An evaluation. International Journal of Applied Engineering Education 2(2): 83–90.Google Scholar
  9. Boud, D.J. & Tyree, A.L. (1979). Self and peer assessment in professional education: A preliminary study in law. Journal of the Society of Public Teachers of Law 15(1): 65–74.Google Scholar
  10. Burke, R.J. (1969). Some preliminary data on the use of self-evaluations and peer-ratings in assigning university course grades. Journal of Educational Research 62(10): 444–4448.Google Scholar
  11. Calhoun, J.G., Ten Haken, J.D. & Wooliscroft, J.O. (1990). Medical students' development of self and peer assessment skills: a longitudinal study. Teaching and Learning in Medicine 2: 25–29.CrossRefGoogle Scholar
  12. Calhoun, J.G., Wooliscroft, J.O., Ten Haken J.D., Wolf F.M. & Davis, W.K. (1988). Evaluating medical student clinical skill performance: Relationships among self, peer, and expert ratings. Evaluation & the Health Professions 11: 201–212.Google Scholar
  13. Chiu, L.H. (1975). Influence of student teaching on perceived teaching competence. Perceptual and Motor Skills 40: 872–874.Google Scholar
  14. Cochran, S.B. & Spears, M.C. (1980). Student self-assessment and instructors' ratings: A comparison. Journal of the American Dietetic Association 76: 253–257.Google Scholar
  15. Coutts, L. & Rogers, J. (1999). Predictors of student self-assessment accuracy during a clinical performance exam: comparisons between over-estimators and under-estimators of SP-evaluated performance. Academic Medicine 74(10 Supplement): S128–S130.Google Scholar
  16. Daniel, S.J., Scruggs, R.R. & Grady, J.J. (1990). Accuracy of student self-evaluations of dental sealants. Journal of Dental Hygiene 64: 339–342.Google Scholar
  17. Das, M., Mpofu, D., Dunn, E. & Lanphear, J.H. (1998). Self and tutor evaluations in problem-based learning tutorials: Is there a relationship? Medical Education 32: 411–418.CrossRefGoogle Scholar
  18. D'Augelli, A.R. (1973). The assessment of interpersonal skills: A comparison of observer, peer and self ratings. Journal of Community Psychology 1: 177–179.Google Scholar
  19. Davis, J.K. & Rand, D.C. (1980). Self-grading versus instructor grading. Journal of Educational Research 73(4): 207–211.Google Scholar
  20. Doleys, E.J. & Renzaglia, G.A. (1963). Accuracy of student prediction of college grades. Personnel and Guidance Journal 41(6): 528–530.Google Scholar
  21. Everett, M.S. (1983). Influence of trait anxiety on self-grading. Educational Directions 8(1): 4–9.Google Scholar
  22. Falchikov, N. (1986). Product comparisons and process benefits of collaborative peer group and self assessments. Assessment and Evaluation in Higher Education 11(2): 146–166.Google Scholar
  23. Falchikov, N. & Boud, D. (1989). Student self-assessment in higher education: A meta-analysis. Review of Educational Research 59: 395–430.CrossRefGoogle Scholar
  24. Falchikov, N. & Goldfinch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of Educational Research 70: 287–322.CrossRefGoogle Scholar
  25. Farnill, D., Hayes, S.C. & Todisco, J. (1997) Interviewing skills: Self-evaluation by medical students. Medical Education 31: 122–127.Google Scholar
  26. Filene, P.O. (1969). Self-grading: An experiment in learning. Journal of Higher Education 40: 451–458.CrossRefGoogle Scholar
  27. Fitzgerald, J.T., Gruppen, L.D. & White, C.B. (2000). The influence of task formats on the accuracy of medical students' self-assessments. Academic Medicine 75: 737–741.CrossRefGoogle Scholar
  28. Forehand, L.S., Vann, W.F. & Shugars, D.A. (1982). Student self-evaluation in preclinical restorative dentistry. Journal of Dental Education 46(4): 221–226.Google Scholar
  29. Fuqua, D.R., Johnson, A.W., Newman, J.L., Anderson, M.W. & Gade, E.M. (1984). Variability across sources of performance ratings. Journal of Counselling Psychology 31(2): 249–252.CrossRefGoogle Scholar
  30. Gaier, E.L. (1961). Student self estimates of final course grades. Journal of Genetic Psychology 98: 63–67.Google Scholar
  31. Geissler, P.R. (1973). Student self-assessment in dental technology. Journal of Dental Education 37: 19–21.Google Scholar
  32. Gordon, M.J. (1991). A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine 66: 762–769.CrossRefGoogle Scholar
  33. Gordon, M.J. (1992). Self-assessment programs and their implications for health professions training. Academic Medicine 67: 672–679.CrossRefGoogle Scholar
  34. Gray, J.D. (1996). Global rating scales in residency education. Academic Medicine 71(1 Supplement): S55–S63.Google Scholar
  35. Gray, T.G.F. (1987). An exercise in improving the potential of exams for learning. European Journal of Engineering Education 12(4): 311–323.Google Scholar
  36. Greenfield, D.G. (1978). Evaluation of music therapy practicum competencies: Comparisons of self and instructor ratings of videotapes. Journal of Music Therapy 15(1): 15–20.Google Scholar
  37. Gruppen, L.D., White, C., Fitzgerald, J.T., Grum, C.M. & Wooliscroft, J.O. (2000). Medical students' self-assessments and their allocations of learning time. Academic Medicine 75: 374–379.CrossRefGoogle Scholar
  38. Gruppen, L.D., Garcia, J., Grum, C.M., Fitzgerald, J.T., White, C.A., Dicken, L. et al. (1997). Medical students' self-assessment accuracy in communication skills. Academic Medicine 72(10 Supplement 1): S57–S59.CrossRefGoogle Scholar
  39. Gruppen, L.D., Baliga, S., Fitzgerald, J.T. et al. (1998). Do personal characteristics influence self-assessment accuracy? In, Proceedings of the 8th International Ottawa Conference on Medical Education, pp. 304–309. Philadelphia, PA.Google Scholar
  40. Harrington, J.P., Murnaghan, J.J. & Regehr, G. (1997). Applying a relative ranking model to the self-assessment of extended performances. Advances in Health Sciences Education 2: 17–25.CrossRefGoogle Scholar
  41. Hay, J.A. (1995). Investigating the development of self-evaluation skills in problem-based tutorial course. Academic Medicine 70: 733–735.CrossRefGoogle Scholar
  42. Henbest, R.J. & Fehrsen, G.S. (1985). Preliminary study at the medical university of Southern Africa on student self-assessment as a means of evaluation. Journal of Medical Education 60: 66–68.Google Scholar
  43. Herbert, W.N.P., McGaghie, W.C., Droegemueller, W., Riddle, M.H. & Maxwell, K.L. (1990). Student evaluation in obstetrics and gynecology: self versus departmental assessment. Obstetrics & Gynecology 76: 458–461.Google Scholar
  44. Hodges, B., Regehr, G. & Martin D. (2001) Difficulties in recognizing one's own incompetence: novice physicians who are unskilled and unaware of it. Academic Medicine; 76(10 Supplement): S87–89.Google Scholar
  45. Irvine, J.J. (1983). The accuracy of pre-service teachers' assessments of their classroom behaviors. Journal of Research and Development in Education 17(1): 25–31.Google Scholar
  46. Israelite, L. (1983). Adult student self-evaluation. Performance and Instruction Journal 22(5): 15–16.Google Scholar
  47. Johnson, D. & Cujec, B. (1998). Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit. Critical Care Medicine 26: 1811–1816.Google Scholar
  48. Kaiser, S. & Bauer, J.J. (1995). Checklist self-evaluation in a standardized patient exercise. American Journal of Surgery 169: 418–420.CrossRefGoogle Scholar
  49. Keefer, K.E. (1971). Characteristics of students who make accurate and inaccurate self-predictions of college achievement. Journal of Educational Research 64(9): 401–404.Google Scholar
  50. Kegel-Flom, P. (1975). Predicting supervisor, peer, and self ratings of intern performance. Journal of Medical Educcation 50: 812–815.Google Scholar
  51. Keynan, A., Friedman, M. & Benbassat, J. (1987). Reliability of global rating scales in the assessment of clinical competence of medical students. Medical Education 21: 477–481.Google Scholar
  52. Kolm, P. & Verhulst, S.J. (1987). Comparing self and supervisor evaluations - a different view. Evaluation & the Health Professions 10: 80–89.Google Scholar
  53. LeBlanc, R. & Painchaud, G. (1985). Self assessment as a second language placement instrument. TESOL Quarterly 19(4): 673–687.Google Scholar
  54. Leichner, P. & Kalin, R. (1980). Results of the first Canadian psychiatric knowledge self assessment for residents. Canadian Journal of Psychiatry 25(4): 281–289.Google Scholar
  55. Linn, B.S., Arostegui, M. & Zeppa, R. (1975). Performance rating scale for peer and self assessment. British Journal of Medical Education 9: 98–101.Google Scholar
  56. Linn, B.S., Arostegui, M. & Zeppa, R. (1976). Peer and self assessment in undergraduate surgery. Journal of Surgical Research 21: 453–456.CrossRefGoogle Scholar
  57. MacFadyen, J. & Turnbull, J. (1985). Effect of curriculum on student self-assessment of communication skills. In Proceedings of the 6th Ottawa Conference of Medical Education and Assessment, pp. 153–154, Toronto, ON.Google Scholar
  58. Mast, T.A. & Bethart, H. (1978). Evaluation of clinical dental procedures by senior dental students. Journal of Dental Education 42(4): 196–197.Google Scholar
  59. Martin, D., Regehr, G., Hodges, B. & McNaughton, N. (1998). Using videotaped benchmarks to improve the self-assessment ability of family practice residents. Academic Medicine 73: 1201–1206.CrossRefGoogle Scholar
  60. McGeever, P.J. (1978). Student self-grading in the introductory American politics course. Teaching Political Science 5(3): 319–330.Google Scholar
  61. Mihal, W.L. & Graumenz, J.L. (1984). An assessment of the accuracy of self-assessment of career decision making. Journal of Vocational Behavior 25: 245–253.CrossRefGoogle Scholar
  62. Moore, D.E. & Cordes, D.L. (1992). Needs assessment. In A.B. Rosof & W.C. Felch (eds.), Continuing Medical Education: A Primer, 2nd ed., pp. 42–51. Westport, CT: Praegar.Google Scholar
  63. Morton, J.B. & MacBeth, W.A. (1977). Correlations between staff, peer and self assessments of fourth-year students in surgery. Medical Education 11: 167–170.Google Scholar
  64. Mowrer, O.H. (1953). 'Q-technique' - description, history, and critique. In O.H. Mowrer (ed.), Psychotherapy Theory and Research, pp. 316–375. New York: Ronald.Google Scholar
  65. Mueller, R.H. (1970). Is self-grading the answer? Journal of Higher Education 41(3): 221–224.CrossRefGoogle Scholar
  66. Murstein, B.I. (1965). The relationship of grade expectations and grades believed to be deserved to actual grades received. Journal of Experimental Education 33(4): 357–362.Google Scholar
  67. O'Neill, G.P. (1985). Self, teacher, and faculty assessments of student teaching performance: A second scenario. Alberta Journal of Educational Research 31(2): 88–98.Google Scholar
  68. Palmer, P.B., Henry, J.N. & Rohe, D.A. (1985). Effect of videotape replay on the quality and accuracy of student self-evaluation. Physical Therapy 65: 497–501.Google Scholar
  69. Pease, D. (1975). Comparing faculty and school supervisor ratings for education students. College Student Journal 9(1): 91–94.Google Scholar
  70. Pitishkin-Potanich, V. (1983). On evaluating students' knowledge. Higher Education in Europe 8(2): 18–22.Google Scholar
  71. Plorde, D.S., Wollitzer, A.O. & Blossom, H. J. (1985). A self-assessment questionnaire for patient educators. Journal of Medical Education 60: 728–730.Google Scholar
  72. Pohlmann, J.L. & Beggs, D.L. (1974). A study of the validity of self-reported measures of academic growth. Journal of Educational Measurement 12: 115–118.CrossRefGoogle Scholar
  73. Regehr, G., Hodges, B., Tiberius, R. & Lofchy, J. (1996). Measuring self-assessment skills: An innovative relative ranking model. Academic Medicine 71(10 Supplement): S52–S4.CrossRefGoogle Scholar
  74. Rezler, A.G. (1989). Self-assessment in problem-based groups. Medical Teacher 11: 151–156.Google Scholar
  75. Risucci, D.A., Tortolani, A.J. & Ward, R.J. (1989). Ratings of surgical residents by self, supervisors, and peers. Surgery, Gynecology & Obstetrics 169: 519–526.Google Scholar
  76. Sclabassi, S.E. & Woelfel, S.K. (1984). Development of self-assessment skills in medical students. Medical Education 84: 226–231.CrossRefGoogle Scholar
  77. Spencer, J.A. & Jordan, R.K. (1999). Learner centred approaches in medical education. BMJ 318: 1280–1283.Google Scholar
  78. Stanton, H.E. (1978). Self-grading as an assessment method. Improving College and University Teaching 26: 236–238.Google Scholar
  79. Stover, R.V. (1976). The impact of self-grading on performance and evaluation in a constitutional law course. Teaching Political Science 3(3): 303–310.Google Scholar
  80. Stuart, M.R., Goldstein, H.S. & Snope, F.C. (1980). Self-evaluation by residents in family medicine. Journal of Family Practrice 10: 639–642.Google Scholar
  81. Turnbull, J., Gray, J. & MacFadyen, J. (1998). Improving in-training evaluation programs. JGIM 13: 317–323.CrossRefGoogle Scholar
  82. Wheeler, A.E. & Knoop, H.R. (1981). Self, teacher and faculty assessments of student teaching performance. Journal of Educational Research 75(3): 171–181.Google Scholar
  83. Wooliscroft, J.O., TenHaken, J., Smith, J. & Calhoun, J.G.. (1993) Medical students' clinical self-assessments: comparisons with external measures of performance and the students' selfassessments of overall performance and effort. Academic Medicine 68: 285–294.Google Scholar
  84. Zonia, S.C. & Stommel, M. (2000). Interns' self-evaluations compared with their faculty's evaluations. Academic Medicine 75: 742.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  1. 1.Department of SurgeryUniversity of TorontoTorontoCanada
  2. 2.Department of Medical EducationUniversity of MichiganAnn ArborUSA
  3. 3.Department of PsychiatryUniversity of TorontoTorontoCanada

Personalised recommendations