Self and Peer Assessment of Pediatricians, Psychiatrists and Medicine Specialists: Implications for Self-Directed Learning
Received: 03 February 2005 Accepted: 02 December 2005 DOI:
Cite this article as: Violato, C. & Lockyer, J. Adv Health Sci Educ Theory Pract (2006) 11: 235. doi:10.1007/s10459-005-5639-0 Abstract
Self-regulation in medicine depends on accurate self-assessment. The purpose of the present study was to examine the discrepancy between self and peer assessments for a group of specialist physicians from internal medicine (IM), pediatrics, and psychiatry clinical domains (i.e., patient management, clinical assessment, professional development, and communication). Data from 304 psychiatrists, pediatricians and internal medicine specialists were used. Each physician had data from an identical self and 8 peer (38 item/4 clinical domains assessment). A total of 2306 peer assessments were available. Physicians were classified into quartiles based on mean assessment peer data and compared with self-assessment data. The analyses showed that self and peer assessment profiles were consistent across specialties and domains. Physicians assessed in the lowest and highest quartiles (i.e., <25th and >75th) by colleagues tended to rate themselves 30–40 percentile ranks higher and lower than peers, respectively. This study suggests that practicing physicians are inaccurate in assessing their own performance. These data suggest that systems to provide practicing physicians with regular and routine feedback may be appropriate if we are to ensure physicians are able to accurately assess themselves in a profession in which self-regulation is predicated upon the assumption that physicians know their capabilities and limitations.
Keywords continuing medical education continuing professional development internal medicine specialists pediatricians psychiatrists peer assessment self assessment self directed learning References
Accreditation Council for Graduate Medical Education (2006). ACGME Outcome Project. General competencies, http://www.acgme.org/outcome/comp/compFull.asp#3. Accessed January 5
CanMEDs 2000 Project (1996). Skills for the new millennium: report of the societal needs working group. Royal College of Physicians and Surgeons of Canada: http://rcpsc.medical.org/canmeds/index.php. Accessed November 24, 2004
College of Physicians and Surgeons of Alberta (2004). Physician Achievement Review Program http://www.par-program.org/ Accessed November 23
Cronbach, L.J. 1990Essentials of Psychological Measurement2 Macmillan New York Google Scholar Eva, K.W., Cunnington, J.P.W., Reiter, H.I., Keane, D.R., Norman, G.R. 2004 How can I know what I don’t know? Poor self assessment in a well-defined domain Advances in Health Sciences Education 9 211 224 CrossRef Google Scholar Evans, R., Elwyn, G., Edwards, A. 2004 Review of instruments for peer assessment of physicians British Medical Journal 328 1243 Google Scholar Festinger, L. 1954 A theory of social comparison processes Human Relations 7 117 140 CrossRef Google Scholar Fitzgerald, J.T., Gruppen, L.D., White, C.B. 2000 The influence of task formats on the accuracy of medical students’ self-assessments Academic Medicine 75 737 741 CrossRef Google Scholar Friedman, C.P., Gatti, G.G., Franz, T.M., Murphy, G.C., Wolf, F.M., Heckerling, P.S, Fine, P.L., Miller, T.M., Elstein, A.S. 2005 Do physicians know when their diagnoses are correct? Implications for decision support and error reduction Journal of General Internal Medicine 20 334 339 CrossRef Google Scholar Fox, R.D., Mazmanian, P.E., Putnam, R.W. 1989Changing and Learning in the Lives of Physicians Prager Publishers New York, NY Google Scholar Hall, W., Violato, C., Lewkonia, R., Lockyer, J., Fidler, H., Toews, J., Jennett, P., Donoff, M., Moores, D. 1999 Assessment of physician performance in Alberta: The physician achievement review Canadian Medical Association Journal 161 52 57 Google Scholar Harvey, B.J., Rothman, A.I., Frecker, R.C. 2003 Effect of an undergraduate medical curriculum on students’ self-directed learning Academic Medicine 78 1259 1265 CrossRef Google Scholar Hodges, B., Regehr, G., Martin, D. 2001 Difficulties in recognizing one’s own incompetence: Novice physicians who are unskilled and unaware of it Academic Medicine 76 S87 S89 CrossRef Google Scholar Kruger, J., Dunning, D. 1999 Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments Journal of Personality and Social Psychology 77 1121 1134 CrossRef Google Scholar Leach, D. 2000 Evaluation of Competence: An ACGME perspective American Journal of Physical Medicine and Rehabilitation 79 487 489 CrossRef Google Scholar Lockyer, J.M., Violato, C. 2004 An examination of the appropriateness of using a common peer assessment instrument to assess physician skills across specialties Academic Medicine 10 S5 S8 CrossRef Google Scholar Mann, K.V., Gelula, M.H. 2003 How to facilitate self-directed learning Davis, D. Barnes, B.E. Fox, R. eds. The Continuing Development of Physicians AMA Press Chicago 121 143 Google Scholar Parboosingh, J.T., Badet, M.C., Wooster, D.L. 2003 Individualized learning strategies Davis, D. Barnes, B.E. Fox, R. eds. The Continuing Development of Physicians AMA Press Chicago 145 167 Google Scholar Reiter, H.I., Eva, K.W., Hatala, R.M., Norman, G.R. 2002 Self and peer assessment in tutorials: Application of a relative-ranking model Academic Medicine 77 1134 1139 CrossRef Google Scholar Ross, L., Greene, D., House, P. 1977 The false consensus effect: An egocentric bias in social perception and attributional processes Journal of Experimental Social Psychology 13 279 301 CrossRef Google Scholar Sargeant, J., Mann, K., Ferrier, S. 2005 Understanding family physicians reactions to MSF performance assessment: Perceptions of credibility and usefulness Medical Education 39 497 504 CrossRef Google Scholar Sargeant, J.M., Mann, K.V., Ferrier, S.N., Langille, D.B., Muirhead, P.D., Sinclair, D.E. 2003 Responses of rural family physicians and their colleagues and coworker raters to a multi-source feedback process: A pilot study Academic Medicine 78 S42 S44 CrossRef Google Scholar Sibley, J.C., Sackett, D.L., Neufeld, V., Gerrard, B., Rudnick, R.V., Fraser, W. 1982 A randomized trial of continuing medical education New England Journal of Medicine 306 511 515 CrossRef Google Scholar Violato, C., Lockyer, J., Fidler, H. 2003 Multi source feedback: A method of assessing surgical practice British Medical Journal 326 546 548 Google Scholar
Violato, C., Lockyer, J., Fidler, H., Toews, J., (2006).
Testing a theoretical model of multi source feedback physician performance using structural equation modeling. Paper presented at the 5th Hawaii International Conference on Statistics, Mathematics and Related Fields, January 16–18 Ward, M., Gruppen, L., Regehr, G. 2002 Measuring self-assessment: Current state of the art Advances in Health Sciences Education 7 63 80 CrossRef Google Scholar Ward, M., MacRae, H., Schlachta, C., Mamazza, J., Puolin, E., Reznick, R., Regher, G. 2003 Resident self-assessment of operative performance American Journal of Surgery 185 521 524 CrossRef Google Scholar