Advertisement

Advances in Health Sciences Education

, Volume 9, Issue 3, pp 211–224 | Cite as

How Can I Know What I Don't Know? Poor Self Assessment in a Well-Defined Domain

  • Kevin W. Eva
  • John P.W. Cunnington
  • Harold I. Reiter
  • David R. Keane
  • Geoffrey R. Norman
Article

Abstract

As the rapidity with which medical knowledge isgenerated and disseminated becomes amplified,an increasing emphasis has been placed on theneed for physicians to develop the skillsnecessary for life-long learning. One suchskill is the ability to evaluate one's owndeficiencies. A ubiquitous finding in thestudy of self-assessment, however, is thatself-ratings are poorly correlated with otherperformance measures. Still, many educatorsview the ability to recognize and communicateone's deficiencies as an important component ofadult learning. As a result, two studies havebeen performed in an attempt to improve uponthis status quo.First, we tried to re-define the limits withinwhich self-assessments should be used, usingRosenblit and Keil's argument that calibrationbetween perceived and actual performance willbe better within taxonomies that are regularlytested (e.g., factual knowledge) compared tothose that are not (e.g., conceptualknowledge). Second, we tried to norm referenceindividuals based on both the performance oftheir colleagues and their own historicalperformance on McMaster's Personal ProgressInventory (a multiple choice question test ofmedical knowledge).While it appears that students are able to (a)make macro-level self-assessments (i.e., torecognize that third year students typicallyoutperform first year students), and (b) judgetheir performance relatively accurately afterthe fact, students were unable to predict thepercentage of questions they would answercorrectly with a testing procedure in whichthey have had a substantial amount of feedback. Previous test score was a much better predictorof current test performance than wereindividuals' expectations.

evaluation life-long learning professionalism self-assessment 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arnold, L., Willoughby, T.L. & Calkins, E.V. (1985). Self-evaluation in undergraduate medical education: A longitudinal perspective. Journal of Medical Education 60: 21–28.Google Scholar
  2. Blake, J.M., Norman, G.R. & Smith, E.K.M. (1995). Report card from McMaster: student evaluation at a problem-based medical school. Lancet 345: 899–902.Google Scholar
  3. Blake, J.M., Norman, G.R., Keane, D.R., Mueller, C.B., Cunnington, J. & Didyk, N. (1996). Introducing progress testing inMcMaster University's problem-based medical curriculum: psychometric properties and effect on learning. Academic Medicine 71: 1002–1007.Google Scholar
  4. e.pdf} (last accessed Dec. 12, 2002).Google Scholar
  5. Cunnington, J.P.W., Norman, G.R., Blake, J.M., Dauphinee, W.D. & Blackmore, D.E. (1996).Applying Learning Taxonomies to Test Items: is a fact an artifact? Academic Medicine 71(suppl): S31–S33.Google Scholar
  6. Day, S.C., Norcini, J.J., Webster, G.D., Viner, E.D. & Chirico, A.M. (1988). The effect of changes in medical knowledge on examination performance at the time of recertification. Proceedings of the Annual Conference of Research in Medical Education 27: 139–144.Google Scholar
  7. Fitzgerald, J.T., White, C.B. & Gruppen, L.D. (2002). A longitudinal study of self-assessment accuracy. Abstract presented at the annual meeting of the Association of American Medical Colleges.Google Scholar
  8. Gordon, M.J. (1991). A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine 66: 762–769.Google Scholar
  9. Gordon, M.J. (1992). Self-assessment programs and their implications for health professions training. Academic Medicine 67: 672–679.Google Scholar
  10. Keil, F. (2001). The ends, and beginnings, or science in the head: coming to grasp the causal structure of the world. Presented at Department of Psychology Colloquium, McMaster University.Google Scholar
  11. Kruger, J. & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology 77: 1121–1134.Google Scholar
  12. Muller, S. (Chair) (1984). Physicians for the twenty-first century: report of the project panel on the general professional education of the physician and college preparation for medicine. JMed Educ 59, Part 2.Google Scholar
  13. Rezler, A. (1989). Self-assessment in problem-based groups. Medical Teacher 11: 151–156.Google Scholar
  14. Rozenblit, L.R. & Keil, F.C. (2002). The misunderstood limits of folk science: an illusion of explanatory depth. Cognitive Science 26: 521–562.Google Scholar
  15. Sackett, D.L. & Rosenberg, W.M. (1995). The need for evidence-based medicine. Journal of the Royal Society of Medicine 88: 620–624.Google Scholar
  16. Shin, J.H., Haynes, R.B. & Johnston, M.E. (1993). Effect of problem-based, self-directed undergraduate education on life-long learning. CMAJ 148: 969–976.Google Scholar
  17. Sibley, J.C., Sackett, D.L., Neufeld, V., Gerrard, B., Rudnick, K.V. & Fraser, W. (1982). A randomized trial of continuing medical education. New England Journal of Medicine 306: 511–515.Google Scholar
  18. Walton, H.J. & Matthews, M.B. (1989). Essentials of problem-based learning. Med Educ 23: 542–558.Google Scholar
  19. Ward, M., Gruppen, L. & Regehr, G. (2002). Measuring self-assessment: current state of the art.Advances in Health Sciences Education 7: 63–80.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Kevin W. Eva
    • 1
  • John P.W. Cunnington
    • 1
  • Harold I. Reiter
    • 1
  • David R. Keane
    • 1
  • Geoffrey R. Norman
    • 1
  1. 1.Program for Educational Research and Development, Room 101, Building T-13McMaster UniversityHamiltonCanada (Corresponding author

Personalised recommendations